首页 > 其他 > 详细

Elasticsearch集群搭建

时间:2019-11-04 01:59:34      阅读:108      评论:0      收藏:0      [点我收藏+]

一、前言

  Elasticsearch是一个开源的高扩展的分布式全文检索引擎,它可以近乎实时的存储、检索数据;本身扩展性很好,可以扩展到上百台服务器,处理PB级别的数据。Elasticsearch也使用Java开发并使用Lucene作为其核心来实现所有索引和搜索的功能,但是它的目的是通过简单的RESTful API来隐藏Lucene的复杂性,从而让全文搜索变得简单。

二、准备

2.1、如果安装的ES7.x,则安装OpenJDK11

否则会报错:

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2019-11-04T00:12:07,213][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-1] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.4.2.jar:7.4.2]
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.4.2.jar:7.4.2]
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.4.2.jar:7.4.2]
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.4.2.jar:7.4.2]
    at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.4.2.jar:7.4.2]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.4.2.jar:7.4.2]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.4.2.jar:7.4.2]
Caused by: java.lang.RuntimeException: can not run elasticsearch as root
    at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:105) ~[elasticsearch-7.4.2.jar:7.4.2]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:172) ~[elasticsearch-7.4.2.jar:7.4.2]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.4.2.jar:7.4.2]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.4.2.jar:7.4.2]
    ... 6 more

下载安装OpenJDK11

wget https://download.java.net/java/GA/jdk11/13/GPL/openjdk-11.0.1_linux-x64_bin.tar.gz
tar -xzvf jdk-11.0.4_linux-x64_bin.tar.gz /opt/

修改ES启动脚本加上jdk11的配置:

vi /home/elsearch/elasticsearch-7.4.2/bin/elasticsearch

为了方便大家参考,这里贴上完整的配置文件技术分享图片

#!/bin/bash

# CONTROLLING STARTUP:
#
# This script relies on a few environment variables to determine startup
# behavior, those variables are:
#
#   ES_PATH_CONF -- Path to config directory
#   ES_JAVA_OPTS -- External Java Opts on top of the defaults set
#
# Optionally, exact memory values can be set using the `ES_JAVA_OPTS`. Note that
# the Xms and Xmx lines in the JVM options file must be commented out. Example
# values are "512m", and "10g".
#
#   ES_JAVA_OPTS="-Xms8g -Xmx8g" ./bin/elasticsearch
# 配置自己的jdk11 export JAVA_HOME
=/home/elsearch/jdk-11.0.1 export PATH=$JAVA_HOME/bin:$PATH source "`dirname "$0"`"/elasticsearch-env if [ -z "$ES_TMPDIR" ]; then ES_TMPDIR=`"$JAVA" -cp "$ES_CLASSPATH" org.elasticsearch.tools.launchers.TempDirectory` fi ES_JVM_OPTIONS="$ES_PATH_CONF"/jvm.options JVM_OPTIONS=`"$JAVA" -cp "$ES_CLASSPATH" org.elasticsearch.tools.launchers.JvmOptionsParser "$ES_JVM_OPTIONS"` ES_JAVA_OPTS="${JVM_OPTIONS//\$\{ES_TMPDIR\}/$ES_TMPDIR}" # 添加jdk判断 if [ -x "$JAVA_HOME/bin/java" ]; then JAVA="/home/elsearch/jdk-11.0.1/bin/java" else JAVA=`which java` fi # manual parsing to find out, if process should be detached if ! echo $* | grep -E (^-d |-d$| -d |--daemonize$|--daemonize ) > /dev/null; then exec "$JAVA" $ES_JAVA_OPTS -Des.path.home="$ES_HOME" -Des.path.conf="$ES_PATH_CONF" -Des.distribution.flavor="$ES_DISTRIBUTION_FLAVOR" -Des.distribution.type="$ES_DISTRIBUTION_TYPE" -Des.bundled_jdk="$ES_BUNDLED_JDK" -cp "$ES_CLASSPATH" org.elasticsearch.bootstrap.Elasticsearch "$@" else exec "$JAVA" $ES_JAVA_OPTS -Des.path.home="$ES_HOME" -Des.path.conf="$ES_PATH_CONF" -Des.distribution.flavor="$ES_DISTRIBUTION_FLAVOR" -Des.distribution.type="$ES_DISTRIBUTION_TYPE" -Des.bundled_jdk="$ES_BUNDLED_JDK" -cp "$ES_CLASSPATH" org.elasticsearch.bootstrap.Elasticsearch "$@" <&- & retval=$? pid=$! [ $retval -eq 0 ] || exit $retval if [ ! -z "$ES_STARTUP_SLEEP_TIME" ]; then sleep $ES_STARTUP_SLEEP_TIME fi if ! ps -p $pid > /dev/null ; then exit 1 fi exit 0 fi exit $?

2.1、机器环境准备
  我们准备3台机器,并都装好JDK且设置好hostname。

机器IP hostname
192.168.182.132 node-1
192.168.182.133 node-2
192.168.182.134 node-3

2.2、系统优化设置

  • 文件描述符配置
    • 临时生效
      • 使用 root 用户
      • 运行 ulimit -n 65536
    • 永久生效
      • 使用 root 用户
      • 进入 /etc/security/limits.conf
      • 添加一行 user - nofile 65536
  • 虚拟内存配置
    • 临时生效
      • 使用 root 用户
      • 运行 sysctl -w vm.max_map_count=262144
    • 永久生效
      • 使用 root 用户
      • 进入 /etc/sysctl.conf
      • 添加或更新一行 vm.max_map_count=262144
  • 自动发现配置
    • 单节点
    • 进入 elasticsearch 安装目录,打开 config/elasticsearch.yml
    • 添加或更新一行 discovery.type: single-node

3.2、创建elsearch用户(Linux 下不要使用 root 用户运行 Elasticsearch, 否则会报异常 can not run elasticsearch as root

 

# 创建用户组
groupadd elsearch

# 创建用户,-p : 登录密码
useradd elsearch -g elsearch -p elsearch

 

 

三、搭建

3.1、下载并安装

官网下载:https://www.elastic.co/cn/downloads/elasticsearch

# 下载
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.2-linux-x86_64.tar.gz

# 解压
tar zxvf
elasticsearch-7.4.2-linux-x86_64.tar.gz && cd elasticsearch-7.4.2/

# 切换用户组:所有者,跟上面创建的用户相对应
chown -R elsearch:elsearch .

 

 

3.2、修改配置文件以支持集群

1.注意修改network.host为各节点IP地址
2.注意修改discovery.zen.ping.unicast.hosts列表

备份:

cp -a config/elasticsearch.yml config/elasticsearch.yml.bak

配置:

cat <<EOF >>/etc/elasticsearch/elasticsearch.yml
## 集群名称
cluster.name: my-cs-cluster

## 节点名称(每个节点名称不能相同)
node.name: ‘mode-1‘

## 允许 JVM 锁住内存,禁止操作系统交换出去
# bootstrap.memory_lock: true

## 是否有资格成为主节点
## 通过 node.master 可以配置该节点是否有资格成为主节点,如果配置为 true,则主机有资格成为主节点
## 注意这里是有资格成为主节点,不是一定会成为主节点
node.master: true

## 是否是数据节点
## 当 node.master 和 node.data 均为 false,则该主机会作为负载均衡节点
node.data: true

## 设置访问的地址和端口
network.host: 192.168.182.132
http.port: 9200

## 集群地址设置
## 配置之后集群的主机之间可以自动发现
discovery.zen.ping.unicast.hosts: ["192.168.30.41", "192.168.30.42", "192.168.30.43"]

## 配置大多数节点(通常为主节点的节点总数/ 2 + 1)来防止“裂脑”:
discovery.zen.minimum_master_nodes: 2

## 在完全集群重启后阻止初始恢复,直到启动N个节点
gateway.recover_after_nodes: 2
EOF
vim /etc/elasticsearch/elasticsearch.yml

3.3、JVM内存大小指定太大,但本机内存不够用

[root@file elasticsearch-7.1.1]# ./bin/elasticsearch
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c5330000, 986513408, 0) failed; error=Cannot allocate memory (errno
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 986513408 bytes for committing reserved memory.
# An error report file with more information is saved as:
# logs/hs_err_pid27766.log

解决:

# 修改jvm.options文件配置即可,从1g改成了100m

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms100m
-Xmx100m

 

 

Elasticsearch集群搭建

原文:https://www.cnblogs.com/liugp/p/11789933.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!