Docker 系列 - ELK 部署

1、前言
ELK由Elastaicsearch 、 Logstash 和 Kibana 组合，如下图所示，Logstash 从各种不同的数据源收集数据，通过内置的管道对输入的数据进行加工。最终，这些数据会被存储到 Elastaicsearch 中供 Kibana 完成数据可视化。

2、创建Docker自定义网络

docker network create esnet

3、Elasticsearch 安装

镜像拉取

docker pull elasticsearch:7.6.2

Docker运行

docker run -d --name elasticsearch --net esnet -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:7.6.2

配置 elasticsearch.yml

//添加
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
//配置完成后，重启elasticsearch容器
docker exec  -it elasticsearch bash
执行
bin/elasticsearch-setup-passwords interactive

[root@node01 elasticsearch-7.7.0]# bin/elasticsearch-setup-passwords interactive
future versions of Elasticsearch will require Java 11; your Java version from [/opt/app/jdk1.8.0_181/jre] does not meet this requirement
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y
Enter password for [elastic]: 
Reenter password for [elastic]: 
Enter password for [apm_system]: 
Reenter password for [apm_system]: 
Enter password for [kibana]: 
Reenter password for [kibana]: 
Enter password for [logstash_system]: 
Reenter password for [logstash_system]: 
Enter password for [beats_system]: 
Reenter password for [beats_system]: 
Enter password for [remote_monitoring_user]: 
Reenter password for [remote_monitoring_user]: 
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

4、Kibana 安装

镜像拉取

docker pull kibana:7.6.2

docker run -d --name kibana --net esnet -p 5601:5601 kibana：7.6.2

配置修改kibana yml

server.name: kibana
server.host: "0"
elasticsearch.hosts: [ "http://192.168.50.16:9200" ]
xpack.monitoring.ui.container.elasticsearch.enabled: true
i18n.locale: "zh-CN"
elasticsearch.username: "elastic"
elasticsearch.password: "123456"

5、Logstash 安装

镜像拉取

docker pull logstash:7.6.2

docker  run -it -d -p 5044:5044 --name logstash --net esnet logstash:7.6.2

配置修改logstash yml

//屏蔽掉
#http.host: "0.0.0.0"
#xpack.monitoring.elasticsearch.hosts: [ "http://elasticsearch:9200" ]

//宿主拷贝到logstash容器
docker cp mysql-connector-java-8.0.23.jar logstash:/usr/share/logstash/config
docker cp logstash-user.conf logstash:/usr/share/logstash/pipeline

logstash-user.conf修改配置

input {
 stdin { }
    jdbc {
	    type => "usertrack"
        #注意mysql连接地址一定要用ip，不能使用localhost等
        jdbc_connection_string => "jdbc:mysql://192.168.19.113:3306/test_shop?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&useSSL=false"
        jdbc_user => "root"
        jdbc_password => "123456"
        #这个jar包的地址是容器内的地址
        jdbc_driver_library => "/usr/share/logstash/config/mysql-connector-java-8.0.23.jar"
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        jdbc_paging_enabled => "true"
        statement => "SELECT * FROM t_user_login"
        schedule => "* * * * *"
		jdbc_default_timezone => "Asia/Shanghai"
	    jdbc_page_size => "500"
	    record_last_run => true
		#use_column_value => true
		clean_run => false
    }
 }
  
 output {
     stdout {
        codec => json_lines
    }
    elasticsearch {
        #注意mysql连接地址一定要用ip，不能使用localhost等
        hosts => "192.168.50.16:9200"
		index => "user-%{type}-%{+YYYY.MM.dd}"
		document_id => "%{id}"
		user => "elastic"
		password => "123456"
        #document_type => "_doc"
        
    }
}

//重启logstash (docker restart logstash)