ELK环境搭建

作者: 尘埃落定_kinslau | 来源:发表于2018-12-20 19:06 被阅读5次

Elasticsearch 基础概念
ELK 搭建及实战
ELK基础环境搭建
Elasticsearch 的基本CRUD 操作
ELK日志管理系统搭建
基于 ELK + Filebeat 搭建日志中心
docker-compose搭建ELK环境
ELK环境搭建
ELK环境搭建
ELK环境搭建

服务器环境

服务器地址：192.168.1.143
filebeat目录：/opt/filebeat/filebeat-6.4.1-linux-x86_64

通过Docker安装

$ docker search elk
NAME                              DESCRIPTION                                     STARS               OFFICIAL            AUTOMATED
sebp/elk                          Collect, search and visualise log data with …   781                                     [OK]
qnib/elk                          Dockerfile providing ELK services (Elasticse…   108                                     [OK]
willdurand/elk                    Creating an ELK stack could not be easier.      95                                      [OK]

选择 sebp/elk 镜像

$ docker pull sebp/elk

开启容器，并且挂载一个磁盘

$ docker run  --name elk  -v /mnt/disk2/es:/es  -p 5601:5601 -p 9200:9200 -p 5044:5044 -e ES_MIN_MEM=128m  -e ES_MAX_MEM=1024m -it -d  2fbf0a30426d

注意事项

vm.max_map_count至少需要262144 .附：修改方式

启动filebeat：nohup ./filebeat -e -c filebeat.yml -d kinslau >/dev/null 2>&1 &

基于filebeat作日志代理时，请到容器内修改认证配置

logstash配置

output{
    if [fields][service] == "nginx" {
        elasticsearch{
            hosts => ["localhost:9200"]
            manage_template => false
            index => "nginx-logstash"
            document_type => "%{[@metadata][type]}"
        }
    } else if [fields][service] == "zuul" {
        elasticsearch{
            hosts => ["localhost:9200"]
            manage_template => false
            index => "zuul-logstash"
            document_type => "%{[@metadata][type]}"
        }

    } else if [fields][service] == "ms-user" {
        elasticsearch{
            hosts => ["localhost:9200"]
            manage_template => false
            index => "ms-user-logstash"
            document_type => "%{[@metadata][type]}"
        }
    } else if [fields][service] == "admin-java" {
        elasticsearch{
            hosts => ["localhost:9200"]
            manage_template => false
            index => "admin-java-logstash"
            document_type => "%{[@metadata][type]}"
        }
    } else if [fields][service] == "micro.dynamic" {
        elasticsearch{
            hosts => ["localhost:9200"]
            manage_template => false
            index => "micro.dynamicuul-logstash"
            document_type => "%{[@metadata][type]}"
        }
    } else if [fields][service] == "micro.admin.dynamic" {
      elasticsearch{
            hosts => ["localhost:9200"]
            manage_template => false
            index => "micro.admin.dynamic-logstash"
            document_type => "%{[@metadata][type]}"
        }
    }
}

filebeat配置

#=========================== Filebeat inputs =============================

filebeat.inputs:

- type: log
  enabled: true
  paths:
    - /var/log/nginx/*.log
  fields:
      service: nginx
- type: log
  enabled: true
  paths:
    - /opt/zuul/logs/*.log
  fields:
      service: zuul
- type: log
  enabled: true
  paths:
    - /opt/user/logs/*.log
  fields:
      service: ms-user

- type: log
  enabled: true
  paths:
    - /opt/manager/logs/*.log
  fields:
      service: admin-java

- type: log
  enabled: true
  paths:
    - /opt/micro.dynamic/*.log
  fields:
      service: micro.dynamic

- type: log
  enabled: true
  paths:
    - /opt/micro.admin.dynamic/*.log
  fields:
      service: micro.admin.dynamic
#============================= Filebeat modules ===============================
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.kibana:
#----------------------------- Logstash output --------------------------------
output.logstash:
  hosts: ["localhost:5044"]