1、单节点elasticsearch的安装配置

作者: LH_0811 | 来源:发表于2019-04-17 21:43 被阅读0次

ElasticSearch 安装 (单机单节点/单机多节点)
1、单节点elasticsearch的安装配置
ElasticSearch 构建ES集群
MQ RabbitMQ 高可用集群（二）：集群搭建
ELK之ElasticSearch 6.X安全认证Search
elasticsearch 单节点配置
elk大数据搜索与数据挖掘环境搭建
多服务器 docker elasticsearch集群
redis docker安装就是这么简单（起码节约 10分钟）
ElasticSearch配置

百度百科：
ElasticSearch是一个基于Lucene的搜索服务器。它提供了一个分布式多用户能力的全文搜索引擎，基于RESTful web接口。Elasticsearch是用Java开发的，并作为Apache许可条款下的开放源码发布，是当前流行的企业级搜索引擎。设计用于云计算中，能够达到实时搜索，稳定，可靠，快速，安装使用方便。

要一步一个坎的学习搞ElasticSearch的搭建了。。

1、首先搞了个虚拟机我用的是Parallels Desktop

2、然后我弄了个CentOS6.8的linux镜像

3、然后配置下虚拟机

  1.第一件事是保证虚拟机和主机可以相互ping通。因为是在局域网我就直接关闭了虚拟机的防火墙。
  2.因为elasticsearch好像不能用root权限启动 所以要创建一个用户。
 添加用户的命令
  adduser els  
设置密码的命令
  passwd els

4、ElasticSearch6.2.4需要 jdk1.8 所以下载并按照jdk

这里最好是切到root权限下
1.检查原本系统中的jdk版本 如果有就卸载掉
   检查：   rpm -qa|grep jdk
   卸载：  sudo yum remove xxx(检查指令中找的的结果)
2.修改jdk-8u211-linux-x64.rpm 的权限
    chmod 777 jdk-8u211-linux-x64.rpm
3.安装jdk
  rpm -ivh jdk-8u211-linux-x64.rpm

5、上传elasticsearch-6.2.4.tar.gz

  虚拟机切换到els用户。
  然后把elasticsearch-6.2.4.tar.gz上传到用户的根路径下了。
  解压: tar -zxvf elasticsearch-6.2.4.tar.gz 
  然后为了方便 我把解压后的文件夹重命名为elasticsearch
  mv elasticsearch-6.2.4/  elasticsearch

6、修改下elasticsearch的配置文件

首先是jvm.options这里只改了内存的占用一个配置项

## JVM configuration

################################################################
## IMPORTANT: JVM heap size
################################################################
##
## You should always set the min and max JVM heap
## size to the same value. For example, to set
## the heap to 4 GB, set:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

#这里默认都是1g 可以视情况而定虚拟机资源有限 搞小一点
-Xms256m
-Xmx256m

################################################################
## Expert settings
################################################################
##
## All settings below this section are considered
## expert settings. Don't tamper with them unless
## you understand what you are doing
##
################################################################

## GC configuration
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly

## optimizations

# pre-touch memory pages used by the JVM during initialization
-XX:+AlwaysPreTouch

## basic

# explicitly set the stack size
-Xss1m

# set to headless, just in case
-Djava.awt.headless=true

# ensure UTF-8 encoding by default (e.g. filenames)
-Dfile.encoding=UTF-8

# use our provided JNA always versus the system one
-Djna.nosys=true

# turn off a JDK optimization that throws away stack traces for common
# exceptions because stack traces are important for debugging
-XX:-OmitStackTraceInFastThrow

# flags to configure Netty
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=0

# log4j 2
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true

-Djava.io.tmpdir=${ES_TMPDIR}

## heap dumps

# generate a heap dump when an allocation from the Java heap fails
# heap dumps are created in the working directory of the JVM
-XX:+HeapDumpOnOutOfMemoryError

# specify an alternative path for heap dumps
# ensure the directory exists and has sufficient space
#-XX:HeapDumpPath=/heap/dump/path

## JDK 8 GC logging

8:-XX:+PrintGCDetails
8:-XX:+PrintGCDateStamps
8:-XX:+PrintTenuringDistribution
8:-XX:+PrintGCApplicationStoppedTime
8:-Xloggc:logs/gc.log
8:-XX:+UseGCLogFileRotation
8:-XX:NumberOfGCLogFiles=32
8:-XX:GCLogFileSize=64m

# JDK 9+ GC logging
9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m
# due to internationalization enhancements in JDK 9 Elasticsearch need to set the provider to COMPAT otherwise
# time/date parsing will break in an incompatible way for some date patterns and locals
9-:-Djava.locale.providers=COMPAT

然后是配置elasticsearch.yml 单机环境下配置data 目录和logs目录和ip绑定其余的就有默认的

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application #如果是集群的话 要配置下名称
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1     #如果是集群要配置下节点名称 保证不重复
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /home/els/elasticsearch/data  #这里是指定下数据路径 没有的话要创建这个路径
#
# Path to log files:
#
path.logs: /home/els/elasticsearch/logs #这是指定日志路径 没有的话要创建这个路径
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0 #这里改成0.0.0.0 不限制ip访问。谁都可以访问
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"] #集群环境下 其他节点的ip地址
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes: 
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true

7、测试启动

到elasticsearch/bin下执行elasticsearch脚本。。

这里没有意外的话会报错如下

[2019-04-17T21:18:02,614][WARN ][o.e.b.JNANatives         ] unable to install syscall filter: 
java.lang.UnsupportedOperationException: seccomp unavailable: CONFIG_SECCOMP not compiled into kernel, CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER are needed
        at org.elasticsearch.bootstrap.SystemCallFilter.linuxImpl(SystemCallFilter.java:341) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.SystemCallFilter.init(SystemCallFilter.java:616) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.JNANatives.tryInstallSystemCallFilter(JNANatives.java:258) [elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Natives.tryInstallSystemCallFilter(Natives.java:113) [elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:110) [elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:172) [elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) [elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) [elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) [elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.2.4.jar:6.2.4]
        at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) [elasticsearch-6.2.4.jar:6.2.4]
[2019-04-17T21:18:02,948][INFO ][o.e.n.Node               ] [] initializing ...
[2019-04-17T21:18:03,120][INFO ][o.e.e.NodeEnvironment    ] [ym33Dbv] using [1] data paths, mounts [[/home (/dev/mapper/VolGroup-lv_home)]], net usable_space [28.7gb], net total_space [30.4gb], types [ext4]
[2019-04-17T21:18:03,121][INFO ][o.e.e.NodeEnvironment    ] [ym33Dbv] heap size [247.5mb], compressed ordinary object pointers [true]
[2019-04-17T21:18:03,128][INFO ][o.e.n.Node               ] node name [ym33Dbv] derived from node ID [ym33DbvfTNGz8_FduAAzEA]; set [node.name] to override
[2019-04-17T21:18:03,129][INFO ][o.e.n.Node               ] version[6.2.4], pid[8686], build[ccec39f/2018-04-12T20:37:28.497551Z], OS[Linux/2.6.32-642.el6.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_211/25.211-b12]
[2019-04-17T21:18:03,131][INFO ][o.e.n.Node               ] JVM arguments [-Xms256m, -Xmx256m, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.xZsOoEHi, -XX:+HeapDumpOnOutOfMemoryError, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/home/els/elasticsearch, -Des.path.conf=/home/els/elasticsearch/config]
[2019-04-17T21:18:04,339][INFO ][o.e.p.PluginsService     ] [ym33Dbv] loaded module [aggs-matrix-stats]
[2019-04-17T21:18:04,340][INFO ][o.e.p.PluginsService     ] [ym33Dbv] loaded module [analysis-common]
[2019-04-17T21:18:04,341][INFO ][o.e.p.PluginsService     ] [ym33Dbv] loaded module [ingest-common]
[2019-04-17T21:18:04,342][INFO ][o.e.p.PluginsService     ] [ym33Dbv] loaded module [lang-expression]
[2019-04-17T21:18:04,342][INFO ][o.e.p.PluginsService     ] [ym33Dbv] loaded module [lang-mustache]
[2019-04-17T21:18:04,346][INFO ][o.e.p.PluginsService     ] [ym33Dbv] loaded module [lang-painless]
[2019-04-17T21:18:04,347][INFO ][o.e.p.PluginsService     ] [ym33Dbv] loaded module [mapper-extras]
[2019-04-17T21:18:04,347][INFO ][o.e.p.PluginsService     ] [ym33Dbv] loaded module [parent-join]
[2019-04-17T21:18:04,348][INFO ][o.e.p.PluginsService     ] [ym33Dbv] loaded module [percolator]
[2019-04-17T21:18:04,348][INFO ][o.e.p.PluginsService     ] [ym33Dbv] loaded module [rank-eval]
[2019-04-17T21:18:04,349][INFO ][o.e.p.PluginsService     ] [ym33Dbv] loaded module [reindex]
[2019-04-17T21:18:04,350][INFO ][o.e.p.PluginsService     ] [ym33Dbv] loaded module [repository-url]
[2019-04-17T21:18:04,350][INFO ][o.e.p.PluginsService     ] [ym33Dbv] loaded module [transport-netty4]
[2019-04-17T21:18:04,350][INFO ][o.e.p.PluginsService     ] [ym33Dbv] loaded module [tribe]
[2019-04-17T21:18:04,352][INFO ][o.e.p.PluginsService     ] [ym33Dbv] no plugins loaded
[2019-04-17T21:18:09,509][INFO ][o.e.d.DiscoveryModule    ] [ym33Dbv] using discovery type [zen]
[2019-04-17T21:18:10,880][INFO ][o.e.n.Node               ] initialized
[2019-04-17T21:18:10,880][INFO ][o.e.n.Node               ] [ym33Dbv] starting ...
[2019-04-17T21:18:11,282][INFO ][o.e.t.TransportService   ] [ym33Dbv] publish_address {192.168.122.1:9300}, bound_addresses {[::]:9300}
[2019-04-17T21:18:11,345][INFO ][o.e.b.BootstrapChecks    ] [ym33Dbv] bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [4] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[2]: max number of threads [1024] for user [els] is too low, increase to at least [4096]
[3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[4]: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
[2019-04-17T21:18:11,418][INFO ][o.e.n.Node               ] [ym33Dbv] stopping ...
[2019-04-17T21:18:11,565][INFO ][o.e.n.Node               ] [ym33Dbv] stopped
[2019-04-17T21:18:11,565][INFO ][o.e.n.Node               ] [ym33Dbv] closing ...
[2019-04-17T21:18:11,610][INFO ][o.e.n.Node               ] [ym33Dbv] closed

总结下问题就是
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[2]: max number of threads [1024] for user [els] is too low, increase to at least [4096]
[3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[4]: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk

8、然后一个个的解决上面的问题

【1】第一个问题

每个进程最大同时打开文件数太小
在root权限下修改limits.conf
vim /etc/security/limits.conf
添加

* soft nofile 65536
* hard nofile 131072
* soft nproc 4096
* hard nproc 4096

【2】: 线程数不够

在root权限下修改文件
vim /etc/security/limits.d/90-nproc.conf

* soft nproc 1024
改成
* soft nproc 4096

【3】:虚拟内存修改

root权限下修改文件
vim /etc/sysctl.conf
添加一个配置

vm.max_map_count=655360

添加完成后执行命令

sysctl -p

让配置生效

【4】禁用到system call

修改elasticsearch.yml 在文件的最后添加一个配置
bootstrap.system_call_filter: false

9、修改完成后最后是重新创建一次ssh链接。然后在重启elasticsearch

10、最终访问虚拟机ip：9200 验证是否启动成功

出现这个就是成功了

{
  "name" : "ym33Dbv",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "86quFGDnSsaIOZu6hAzXyQ",
  "version" : {
    "number" : "6.2.4",
    "build_hash" : "ccec39f",
    "build_date" : "2018-04-12T20:37:28.497551Z",
    "build_snapshot" : false,
    "lucene_version" : "7.2.1",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

ElasticSearch 安装 (单机单节点/单机多节点)
ElasticSearch 安装 (单机单节点/单机多节点) ElasticSearch 简介 ElasticSe...
1、单节点elasticsearch的安装配置
百度百科：ElasticSearch是一个基于Lucene的搜索服务器。它提供了一个分布式多用户能力的全文搜索引擎...
ElasticSearch 构建ES集群
引言操作手册在集群的每个节点上，将ElasticSearch的单节点安装好修改配置文件elasticsear...
MQ RabbitMQ 高可用集群（二）：集群搭建
1 集群节点安装单节点安装RabbitMQ，可参考博文： MQ RocketMQ安装部署和配置： https:...
ELK之ElasticSearch 6.X安全认证Search
ElasticSearch单节点安装Search Guard插件 Search Guard（安全认证）插件安装切...
elasticsearch 单节点配置
Discovery and cluster formation settings | Elasticsearch ...
elk大数据搜索与数据挖掘环境搭建
1、elasticsearch （1）下载并安装elasticsearch （2）配置elasticsearch目...
多服务器 docker elasticsearch集群
单服务器 elasticsearch集群手动配置多服务器 elasticsearch集群两台服务器，主节点：...
redis docker安装就是这么简单（起码节约 10分钟）
docker安装 1、单节点安装 redis.conf 配置文件可以通过 http://download.redi...
ElasticSearch配置
ElasticSearch配置概念节点 (node)，集群 (cluster)单台服务器，存储数据并参与集群...