👈🏻 Select language
安装和配置
自制带插件的ES镜像
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
FROM elasticsearch:6.5.0
#或者手动下载后然后安装也行
# COPY elasticsearch-analysis-ik-6.5.0.zip /
# elasticsearch-plugin install --batch file:///elasticsearch-analysis-ik-6.5.0.zip
#IK Analyzer是一个开源的，基于java语言开发的中文分词工具包。是开源社区中处理中分分词非常热门的插件。
RUN elasticsearch-plugin install --batch https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.5.0/elasticsearch-analysis-ik-6.5.0.zip && \
# 拼音分词器
elasticsearch-plugin install --batch https://github.com/medcl/elasticsearch-analysis-pinyin/releases/download/v6.5.0/elasticsearch-analysis-pinyin-6.5.0.zip && \
# Smart Chinese Analysis Plugin
elasticsearch-plugin install analysis-icu && \
# 日文分词器
elasticsearch-plugin install analysis-kuromoji && \
# 语音分析
elasticsearch-plugin install analysis-phonetic && \
# 计算字符哈希
elasticsearch-plugin install mapper-murmur3 && \
# 在_source中提供size字段
elasticsearch-plugin install mapper-size
编排文件
- RBAC 相关内容
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
apiVersion: v1
kind: ServiceAccount
metadata:
name: elasticsearch-admin
namespace: default
labels:
app: elasticsearch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: default
name: elasticsearch
labels:
app: elasticsearch
subjects:
- kind: ServiceAccount
name: elasticsearch-admin
namespace: default
apiGroup: ""
roleRef:
kind: ClusterRole
name: elasticsearch
apiGroup: ""
- 主体程序
存储使用了hostpath,需要先在宿主机闯将目录,并赋予适当的权限,不然会出错
1
2
3
4
cd /root/kubernetes/$(namespace)/elasticsearch/data
mkdir -p $(pwd)
sudo chmod 775 $(pwd) -R
chown 1000:0 $(pwd) -R
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
# kubectl get po -l app=myelasticsearch -o wide -n default
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: myelasticsearch
namespace: default
labels:
app: myelasticsearch
elasticsearch-role: all
spec:
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
serviceName: elasticsearch-master
replicas: 2
selector:
matchLabels:
app: myelasticsearch
elasticsearch-role: all
template:
metadata:
labels:
app: myelasticsearch
elasticsearch-role: all
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: elasticsearch-test-ready
operator: Exists
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: 'app'
operator: In
values:
- myelasticsearch
- key: 'elasticsearch-role'
operator: In
values:
- all
topologyKey: "kubernetes.io/hostname"
namespaces:
- default
serviceAccountName: elasticsearch-admin
terminationGracePeriodSeconds: 180
# Elasticsearch requires vm.max_map_count to be at least 262144.
# If your OS already sets up this number to a higher value, feel free
# to remove this init container.
initContainers:
- image: alpine:3.6
command: ["/sbin/sysctl", "-w", "vm.max_map_count=262144"]
name: elasticsearch-init
securityContext:
privileged: true
imagePullSecrets:
- name: vpc-shenzhen
containers:
- image: elasticsearch:6.5.0-plugin-in-remote-ik
name: elasticsearch
resources:
# need more cpu upon initialization, therefore burstable class
limits:
# cpu: 2
memory: 4Gi
requests:
# cpu: 1
memory: 1Gi
ports:
- name: restful
containerPort: 9200
protocol: TCP
- name: discovery
containerPort: 9300
protocol: TCP
readinessProbe:
failureThreshold: 3
initialDelaySeconds: 2
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: 9200
timeoutSeconds: 1
# livenessProbe:
# failureThreshold: 3
# initialDelaySeconds: 7
# periodSeconds: 10
# successThreshold: 1
# tcpSocket:
# port: 9200
# timeoutSeconds: 1
volumeMounts:
- name: host
mountPath: /usr/share/elasticsearch/data
env:
- name: "NAMESPACE"
valueFrom:
fieldRef:
fieldPath: metadata.namespace
# - name: "ES_JAVA_OPTS"
# value: "-Xms256m -Xmx256m"
- name: "cluster.name"
value: "myelasticsearch"
- name: "bootstrap.memory_lock"
value: "true"
- name: "discovery.zen.ping.unicast.hosts"
value: "myelasticsearch"
- name: "discovery.zen.minimum_master_nodes"
value: "2"
- name: "discovery.zen.ping_timeout"
value: "5s"
#因为是测试,所以master,data,ingest都混用
- name: "node.master"
value: "true"
- name: "node.data"
value: "true"
- name: "node.ingest"
value: "true"
- name: xpack.monitoring.collection.enabled
value: "true"
- name: node.name
valueFrom:
fieldRef:
fieldPath: metadata.name
# fieldPath: spec.nodeName
securityContext:
privileged: true
volumes:
- name: host
hostPath:
path: /root/kubernetes/default/myelasticsearch/data
type: DirectoryOrCreate
---
kind: Service
apiVersion: v1
metadata:
labels:
app: myelasticsearch
elasticsearch-role: all
name: myelasticsearch
namespace: default
spec:
ports:
- name: discovery
port: 9300
targetPort: discovery
- name: restful
port: 9200
protocol: TCP
targetPort: restful
selector:
app: myelasticsearch
elasticsearch-role: all
type: NodePort
这个编排精髓的一点在于用了节点affinity使每一个节点最多会运行一个容器,确保了高可用.
如果要把节点的角色再抽取出来,那么其实抽取一个service作为相互发现的,即可.
1
2
3
4
5
6
7
8
9
10
11
12
13
kind: Service
apiVersion: v1
metadata:
labels:
app: elasticsearch
name: elasticsearch-discovery
namespace: default
spec:
ports:
- port: 9300
targetPort: discovery
selector:
app: myelasticsearch
- 在Kubernetes上部署Elasticsearch集群
- 重要配置的修改
- Install Elasticsearch with Docker
- 学习Elasticsearch之4：配置一个3节点Elasticsearch集群(不区分主节点和数据节点)
- 谈一谈Elasticsearch的集群部署
- 官方docker镜像构建项目
- Elasticsearch模块功能之-自动发现（Discovery）
- 订阅费用
- 故障转移
- 节点配置
- Elasticsearch 5.X集群多节点角色配置深入详解
- elasticsearch-cloud-kubernetes
- 吃透Elasticsearch 堆内存
安装插件
elasticsearch-docker-plugin-management
GET /_cat/plugins?v&s=component&h=name,component,version,description
压力测试
1
2
3
esrally configure
esrally list tracks
esrally --pipeline=benchmark-only --target-hosts=127.0.0.1:9200 --track=geonames
1
2
3
4
datastore.type = elasticsearch
datastore.host = 127.0.0.1
datastore.port = 9200
datastore.secure = False
通过 Elasticsearch 官方提供的 benchmark 脚本 rally
ElasticSearch集群的维护
更新的时候务必使用灰度更新,从序号最大的镜像开始更新,不然分片丢失了,相信我,你会死的很惨
1
2
3
4
5
6
7
8
9
10
11
kubectl patch statefulset elasticsearch -p \
'{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":0}}}}' \
-n test
kubectl patch statefulset elasticsearch -p \
'{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":1}}}}' \
-n test
kubectl patch statefulset elasticsearch -p \
'{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":2}}}}' \
-n test
Installation and Configuration
Custom ES Image with Plugins
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
FROM elasticsearch:6.5.0
# Or manually download and install
# COPY elasticsearch-analysis-ik-6.5.0.zip /
# elasticsearch-plugin install --batch file:///elasticsearch-analysis-ik-6.5.0.zip
# IK Analyzer is an open-source Chinese word segmentation toolkit developed in Java. It's a very popular plugin in the open-source community for handling Chinese word segmentation.
RUN elasticsearch-plugin install --batch https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.5.0/elasticsearch-analysis-ik-6.5.0.zip && \
# Pinyin analyzer
elasticsearch-plugin install --batch https://github.com/medcl/elasticsearch-analysis-pinyin/releases/download/v6.5.0/elasticsearch-analysis-pinyin-6.5.0.zip && \
# Smart Chinese Analysis Plugin
elasticsearch-plugin install analysis-icu && \
# Japanese analyzer
elasticsearch-plugin install analysis-kuromoji && \
# Phonetic analysis
elasticsearch-plugin install analysis-phonetic && \
# Calculate character hash
elasticsearch-plugin install mapper-murmur3 && \
# Provide size field in _source
elasticsearch-plugin install mapper-size
Orchestration Files
- RBAC Related Content
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
apiVersion: v1
kind: ServiceAccount
metadata:
name: elasticsearch-admin
namespace: default
labels:
app: elasticsearch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: default
name: elasticsearch
labels:
app: elasticsearch
subjects:
- kind: ServiceAccount
name: elasticsearch-admin
namespace: default
apiGroup: ""
roleRef:
kind: ClusterRole
name: elasticsearch
apiGroup: ""
- Main Program
Storage uses hostpath. You need to create the directory on the host first and assign appropriate permissions, otherwise it will error.
1
2
3
4
cd /root/kubernetes/$(namespace)/elasticsearch/data
mkdir -p $(pwd)
sudo chmod 775 $(pwd) -R
chown 1000:0 $(pwd) -R
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
# kubectl get po -l app=myelasticsearch -o wide -n default
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: myelasticsearch
namespace: default
labels:
app: myelasticsearch
elasticsearch-role: all
spec:
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
serviceName: elasticsearch-master
replicas: 2
selector:
matchLabels:
app: myelasticsearch
elasticsearch-role: all
template:
metadata:
labels:
app: myelasticsearch
elasticsearch-role: all
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: elasticsearch-test-ready
operator: Exists
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: 'app'
operator: In
values:
- myelasticsearch
- key: 'elasticsearch-role'
operator: In
values:
- all
topologyKey: "kubernetes.io/hostname"
namespaces:
- default
serviceAccountName: elasticsearch-admin
terminationGracePeriodSeconds: 180
# Elasticsearch requires vm.max_map_count to be at least 262144.
# If your OS already sets up this number to a higher value, feel free
# to remove this init container.
initContainers:
- image: alpine:3.6
command: ["/sbin/sysctl", "-w", "vm.max_map_count=262144"]
name: elasticsearch-init
securityContext:
privileged: true
imagePullSecrets:
- name: vpc-shenzhen
containers:
- image: elasticsearch:6.5.0-plugin-in-remote-ik
name: elasticsearch
resources:
# need more cpu upon initialization, therefore burstable class
limits:
# cpu: 2
memory: 4Gi
requests:
# cpu: 1
memory: 1Gi
ports:
- name: restful
containerPort: 9200
protocol: TCP
- name: discovery
containerPort: 9300
protocol: TCP
readinessProbe:
failureThreshold: 3
initialDelaySeconds: 2
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: 9200
timeoutSeconds: 1
# livenessProbe:
# failureThreshold: 3
# initialDelaySeconds: 7
# periodSeconds: 10
# successThreshold: 1
# tcpSocket:
# port: 9200
# timeoutSeconds: 1
volumeMounts:
- name: host
mountPath: /usr/share/elasticsearch/data
env:
- name: "NAMESPACE"
valueFrom:
fieldRef:
fieldPath: metadata.namespace
# - name: "ES_JAVA_OPTS"
# value: "-Xms256m -Xmx256m"
- name: "cluster.name"
value: "myelasticsearch"
- name: "bootstrap.memory_lock"
value: "true"
- name: "discovery.zen.ping.unicast.hosts"
value: "myelasticsearch"
- name: "discovery.zen.minimum_master_nodes"
value: "2"
- name: "discovery.zen.ping_timeout"
value: "5s"
# Because it's for testing, master, data, and ingest are all mixed
- name: "node.master"
value: "true"
- name: "node.data"
value: "true"
- name: "node.ingest"
value: "true"
- name: xpack.monitoring.collection.enabled
value: "true"
- name: node.name
valueFrom:
fieldRef:
fieldPath: metadata.name
# fieldPath: spec.nodeName
securityContext:
privileged: true
volumes:
- name: host
hostPath:
path: /root/kubernetes/default/myelasticsearch/data
type: DirectoryOrCreate
---
kind: Service
apiVersion: v1
metadata:
labels:
app: myelasticsearch
elasticsearch-role: all
name: myelasticsearch
namespace: default
spec:
ports:
- name: discovery
port: 9300
targetPort: discovery
- name: restful
port: 9200
protocol: TCP
targetPort: restful
selector:
app: myelasticsearch
elasticsearch-role: all
type: NodePort
The essence of this orchestration is using node affinity so that each node runs at most one container, ensuring high availability.
If you want to extract the node roles further, you can extract a service for mutual discovery.
1
2
3
4
5
6
7
8
9
10
11
12
13
kind: Service
apiVersion: v1
metadata:
labels:
app: elasticsearch
name: elasticsearch-discovery
namespace: default
spec:
ports:
- port: 9300
targetPort: discovery
selector:
app: myelasticsearch
- Deploying Elasticsearch Cluster on Kubernetes
- Important Configuration Changes
- Install Elasticsearch with Docker
- Learning Elasticsearch 4: Configuring a 3-Node Elasticsearch Cluster (No Distinction Between Master and Data Nodes)
- Talking About Elasticsearch Cluster Deployment
- Official Docker Image Build Project
- Elasticsearch Module Function - Auto Discovery (Discovery)
- Subscription Fees
- Failover
- Node Configuration
- Elasticsearch 5.X Cluster Multi-Node Role Configuration In-Depth Explanation
- elasticsearch-cloud-kubernetes
- Understanding Elasticsearch Heap Memory
Installing Plugins
elasticsearch-docker-plugin-management
GET /_cat/plugins?v&s=component&h=name,component,version,description
Stress Testing
1
2
3
esrally configure
esrally list tracks
esrally --pipeline=benchmark-only --target-hosts=127.0.0.1:9200 --track=geonames
1
2
3
4
datastore.type = elasticsearch
datastore.host = 127.0.0.1
datastore.port = 9200
datastore.secure = False
Through the official Elasticsearch benchmark script rally
ElasticSearch Cluster Maintenance
When updating, be sure to use grayscale updates, starting from the image with the largest sequence number. Otherwise, if shards are lost, believe me, you’ll be in big trouble.
1
2
3
4
5
6
7
8
9
10
11
kubectl patch statefulset elasticsearch -p \
'{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":0}}}}' \
-n test
kubectl patch statefulset elasticsearch -p \
'{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":1}}}}' \
-n test
kubectl patch statefulset elasticsearch -p \
'{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":2}}}}' \
-n test
インストールと設定
プラグイン付きのカスタムESイメージ
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
FROM elasticsearch:6.5.0
#または手動でダウンロードしてインストールすることもできます
# COPY elasticsearch-analysis-ik-6.5.0.zip /
# elasticsearch-plugin install --batch file:///elasticsearch-analysis-ik-6.5.0.zip
# IK Analyzerは、Java言語で開発されたオープンソースの中国語分かち書きツールキットです。オープンソースコミュニティで中国語分かち書きを処理する非常に人気のあるプラグインです。
RUN elasticsearch-plugin install --batch https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.5.0/elasticsearch-analysis-ik-6.5.0.zip && \
# ピンイン分かち書き器
elasticsearch-plugin install --batch https://github.com/medcl/elasticsearch-analysis-pinyin/releases/download/v6.5.0/elasticsearch-analysis-pinyin-6.5.0.zip && \
# Smart Chinese Analysis Plugin
elasticsearch-plugin install analysis-icu && \
# 日本語分かち書き器
elasticsearch-plugin install analysis-kuromoji && \
# 音声分析
elasticsearch-plugin install analysis-phonetic && \
# 文字ハッシュの計算
elasticsearch-plugin install mapper-murmur3 && \
# _sourceでsizeフィールドを提供
elasticsearch-plugin install mapper-size
オーケストレーションファイル
- RBAC関連コンテンツ
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
apiVersion: v1
kind: ServiceAccount
metadata:
name: elasticsearch-admin
namespace: default
labels:
app: elasticsearch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: default
name: elasticsearch
labels:
app: elasticsearch
subjects:
- kind: ServiceAccount
name: elasticsearch-admin
namespace: default
apiGroup: ""
roleRef:
kind: ClusterRole
name: elasticsearch
apiGroup: ""
- メインプログラム
ストレージはhostpathを使用しています。ホスト上でディレクトリを先に作成し、適切な権限を付与する必要があります。そうしないとエラーになります。
1
2
3
4
cd /root/kubernetes/$(namespace)/elasticsearch/data
mkdir -p $(pwd)
sudo chmod 775 $(pwd) -R
chown 1000:0 $(pwd) -R
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
# kubectl get po -l app=myelasticsearch -o wide -n default
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: myelasticsearch
namespace: default
labels:
app: myelasticsearch
elasticsearch-role: all
spec:
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
serviceName: elasticsearch-master
replicas: 2
selector:
matchLabels:
app: myelasticsearch
elasticsearch-role: all
template:
metadata:
labels:
app: myelasticsearch
elasticsearch-role: all
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: elasticsearch-test-ready
operator: Exists
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: 'app'
operator: In
values:
- myelasticsearch
- key: 'elasticsearch-role'
operator: In
values:
- all
topologyKey: "kubernetes.io/hostname"
namespaces:
- default
serviceAccountName: elasticsearch-admin
terminationGracePeriodSeconds: 180
# Elasticsearchにはvm.max_map_countが少なくとも262144である必要があります。
# OSがすでにこの数値をより高い値に設定している場合は、このinitコンテナを削除しても問題ありません。
initContainers:
- image: alpine:3.6
command: ["/sbin/sysctl", "-w", "vm.max_map_count=262144"]
name: elasticsearch-init
securityContext:
privileged: true
imagePullSecrets:
- name: vpc-shenzhen
containers:
- image: elasticsearch:6.5.0-plugin-in-remote-ik
name: elasticsearch
resources:
# 初期化時にCPUがより必要なので、バースト可能クラス
limits:
# cpu: 2
memory: 4Gi
requests:
# cpu: 1
memory: 1Gi
ports:
- name: restful
containerPort: 9200
protocol: TCP
- name: discovery
containerPort: 9300
protocol: TCP
readinessProbe:
failureThreshold: 3
initialDelaySeconds: 2
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: 9200
timeoutSeconds: 1
# livenessProbe:
# failureThreshold: 3
# initialDelaySeconds: 7
# periodSeconds: 10
# successThreshold: 1
# tcpSocket:
# port: 9200
# timeoutSeconds: 1
volumeMounts:
- name: host
mountPath: /usr/share/elasticsearch/data
env:
- name: "NAMESPACE"
valueFrom:
fieldRef:
fieldPath: metadata.namespace
# - name: "ES_JAVA_OPTS"
# value: "-Xms256m -Xmx256m"
- name: "cluster.name"
value: "myelasticsearch"
- name: "bootstrap.memory_lock"
value: "true"
- name: "discovery.zen.ping.unicast.hosts"
value: "myelasticsearch"
- name: "discovery.zen.minimum_master_nodes"
value: "2"
- name: "discovery.zen.ping_timeout"
value: "5s"
# テスト用なので、master、data、ingestがすべて混在しています
- name: "node.master"
value: "true"
- name: "node.data"
value: "true"
- name: "node.ingest"
value: "true"
- name: xpack.monitoring.collection.enabled
value: "true"
- name: node.name
valueFrom:
fieldRef:
fieldPath: metadata.name
# fieldPath: spec.nodeName
securityContext:
privileged: true
volumes:
- name: host
hostPath:
path: /root/kubernetes/default/myelasticsearch/data
type: DirectoryOrCreate
---
kind: Service
apiVersion: v1
metadata:
labels:
app: myelasticsearch
elasticsearch-role: all
name: myelasticsearch
namespace: default
spec:
ports:
- name: discovery
port: 9300
targetPort: discovery
- name: restful
port: 9200
protocol: TCP
targetPort: restful
selector:
app: myelasticsearch
elasticsearch-role: all
type: NodePort
このオーケストレーションの要点は、ノードaffinityを使用して、各ノードが最大1つのコンテナを実行するようにし、高可用性を確保することです。
ノードの役割をさらに抽出する場合は、相互検出用のサービスを抽出できます。
1
2
3
4
5
6
7
8
9
10
11
12
13
kind: Service
apiVersion: v1
metadata:
labels:
app: elasticsearch
name: elasticsearch-discovery
namespace: default
spec:
ports:
- port: 9300
targetPort: discovery
selector:
app: myelasticsearch
- KubernetesでElasticsearchクラスターをデプロイ
- 重要な設定の変更
- DockerでElasticsearchをインストール
- Elasticsearchを学ぶ4：3ノードElasticsearchクラスターの設定（マスターノードとデータノードを区別しない）
- Elasticsearchクラスターのデプロイメントについて
- 公式Dockerイメージビルドプロジェクト
- Elasticsearchモジュール機能-自動検出（Discovery）
- サブスクリプション料金
- フェイルオーバー
- ノード設定
- Elasticsearch 5.Xクラスターのマルチノード役割設定の詳細説明
- elasticsearch-cloud-kubernetes
- Elasticsearchヒープメモリを理解する
プラグインのインストール
elasticsearch-docker-plugin-management
GET /_cat/plugins?v&s=component&h=name,component,version,description
ストレステスト
1
2
3
esrally configure
esrally list tracks
esrally --pipeline=benchmark-only --target-hosts=127.0.0.1:9200 --track=geonames
1
2
3
4
datastore.type = elasticsearch
datastore.host = 127.0.0.1
datastore.port = 9200
datastore.secure = False
Elasticsearch公式が提供するベンチマークスクリプトrallyを使用
ElasticSearchクラスターのメンテナンス
更新する際は、必ずグレースケール更新を使用し、最大のシーケンス番号のイメージから開始してください。そうしないと、シャードが失われた場合、信じてください、大変なことになります。
1
2
3
4
5
6
7
8
9
10
11
kubectl patch statefulset elasticsearch -p \
'{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":0}}}}' \
-n test
kubectl patch statefulset elasticsearch -p \
'{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":1}}}}' \
-n test
kubectl patch statefulset elasticsearch -p \
'{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":2}}}}' \
-n test
Установка и конфигурация
Пользовательский образ ES с плагинами
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
FROM elasticsearch:6.5.0
# Или вручную загрузить и установить
# COPY elasticsearch-analysis-ik-6.5.0.zip /
# elasticsearch-plugin install --batch file:///elasticsearch-analysis-ik-6.5.0.zip
# IK Analyzer — это набор инструментов для сегментации китайского текста с открытым исходным кодом, разработанный на Java. Это очень популярный плагин в сообществе с открытым исходным кодом для обработки сегментации китайского текста.
RUN elasticsearch-plugin install --batch https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.5.0/elasticsearch-analysis-ik-6.5.0.zip && \
# Анализатор пиньинь
elasticsearch-plugin install --batch https://github.com/medcl/elasticsearch-analysis-pinyin/releases/download/v6.5.0/elasticsearch-analysis-pinyin-6.5.0.zip && \
# Smart Chinese Analysis Plugin
elasticsearch-plugin install analysis-icu && \
# Японский анализатор
elasticsearch-plugin install analysis-kuromoji && \
# Фонетический анализ
elasticsearch-plugin install analysis-phonetic && \
# Вычисление хеша символов
elasticsearch-plugin install mapper-murmur3 && \
# Предоставление поля size в _source
elasticsearch-plugin install mapper-size
Файлы оркестрации
- RBAC Связанный контент
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
apiVersion: v1
kind: ServiceAccount
metadata:
name: elasticsearch-admin
namespace: default
labels:
app: elasticsearch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: default
name: elasticsearch
labels:
app: elasticsearch
subjects:
- kind: ServiceAccount
name: elasticsearch-admin
namespace: default
apiGroup: ""
roleRef:
kind: ClusterRole
name: elasticsearch
apiGroup: ""
- Основная программа
Хранилище использует hostpath. Сначала нужно создать директорию на хосте и назначить соответствующие разрешения, иначе будет ошибка.
1
2
3
4
cd /root/kubernetes/$(namespace)/elasticsearch/data
mkdir -p $(pwd)
sudo chmod 775 $(pwd) -R
chown 1000:0 $(pwd) -R
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
# kubectl get po -l app=myelasticsearch -o wide -n default
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: myelasticsearch
namespace: default
labels:
app: myelasticsearch
elasticsearch-role: all
spec:
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
serviceName: elasticsearch-master
replicas: 2
selector:
matchLabels:
app: myelasticsearch
elasticsearch-role: all
template:
metadata:
labels:
app: myelasticsearch
elasticsearch-role: all
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: elasticsearch-test-ready
operator: Exists
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: 'app'
operator: In
values:
- myelasticsearch
- key: 'elasticsearch-role'
operator: In
values:
- all
topologyKey: "kubernetes.io/hostname"
namespaces:
- default
serviceAccountName: elasticsearch-admin
terminationGracePeriodSeconds: 180
# Elasticsearch требует, чтобы vm.max_map_count было не менее 262144.
# Если ваша ОС уже устанавливает это число на более высокое значение, можете
# удалить этот init контейнер.
initContainers:
- image: alpine:3.6
command: ["/sbin/sysctl", "-w", "vm.max_map_count=262144"]
name: elasticsearch-init
securityContext:
privileged: true
imagePullSecrets:
- name: vpc-shenzhen
containers:
- image: elasticsearch:6.5.0-plugin-in-remote-ik
name: elasticsearch
resources:
# нужно больше CPU при инициализации, поэтому burstable класс
limits:
# cpu: 2
memory: 4Gi
requests:
# cpu: 1
memory: 1Gi
ports:
- name: restful
containerPort: 9200
protocol: TCP
- name: discovery
containerPort: 9300
protocol: TCP
readinessProbe:
failureThreshold: 3
initialDelaySeconds: 2
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: 9200
timeoutSeconds: 1
# livenessProbe:
# failureThreshold: 3
# initialDelaySeconds: 7
# periodSeconds: 10
# successThreshold: 1
# tcpSocket:
# port: 9200
# timeoutSeconds: 1
volumeMounts:
- name: host
mountPath: /usr/share/elasticsearch/data
env:
- name: "NAMESPACE"
valueFrom:
fieldRef:
fieldPath: metadata.namespace
# - name: "ES_JAVA_OPTS"
# value: "-Xms256m -Xmx256m"
- name: "cluster.name"
value: "myelasticsearch"
- name: "bootstrap.memory_lock"
value: "true"
- name: "discovery.zen.ping.unicast.hosts"
value: "myelasticsearch"
- name: "discovery.zen.minimum_master_nodes"
value: "2"
- name: "discovery.zen.ping_timeout"
value: "5s"
# Поскольку это для тестирования, master, data и ingest все смешаны
- name: "node.master"
value: "true"
- name: "node.data"
value: "true"
- name: "node.ingest"
value: "true"
- name: xpack.monitoring.collection.enabled
value: "true"
- name: node.name
valueFrom:
fieldRef:
fieldPath: metadata.name
# fieldPath: spec.nodeName
securityContext:
privileged: true
volumes:
- name: host
hostPath:
path: /root/kubernetes/default/myelasticsearch/data
type: DirectoryOrCreate
---
kind: Service
apiVersion: v1
metadata:
labels:
app: myelasticsearch
elasticsearch-role: all
name: myelasticsearch
namespace: default
spec:
ports:
- name: discovery
port: 9300
targetPort: discovery
- name: restful
port: 9200
protocol: TCP
targetPort: restful
selector:
app: myelasticsearch
elasticsearch-role: all
type: NodePort
Суть этой оркестрации заключается в использовании affinity узла, чтобы каждый узел запускал максимум один контейнер, обеспечивая высокую доступность.
Если вы хотите извлечь роли узлов дальше, вы можете извлечь сервис для взаимного обнаружения.
1
2
3
4
5
6
7
8
9
10
11
12
13
kind: Service
apiVersion: v1
metadata:
labels:
app: elasticsearch
name: elasticsearch-discovery
namespace: default
spec:
ports:
- port: 9300
targetPort: discovery
selector:
app: myelasticsearch
- Развертывание кластера Elasticsearch на Kubernetes
- Важные изменения конфигурации
- Установка Elasticsearch с Docker
- Изучение Elasticsearch 4: Настройка кластера Elasticsearch с 3 узлами (без различия между мастером и узлами данных)
- Разговор о развертывании кластера Elasticsearch
- Официальный проект сборки образа Docker
- Функция модуля Elasticsearch - Автоматическое обнаружение (Discovery)
- Стоимость подписки
- Отказоустойчивость
- Конфигурация узла
- Подробное объяснение конфигурации ролей нескольких узлов кластера Elasticsearch 5.X
- elasticsearch-cloud-kubernetes
- Понимание кучи памяти Elasticsearch
Установка плагинов
elasticsearch-docker-plugin-management
GET /_cat/plugins?v&s=component&h=name,component,version,description
Стресс-тестирование
1
2
3
esrally configure
esrally list tracks
esrally --pipeline=benchmark-only --target-hosts=127.0.0.1:9200 --track=geonames
1
2
3
4
datastore.type = elasticsearch
datastore.host = 127.0.0.1
datastore.port = 9200
datastore.secure = False
Через официальный скрипт бенчмарка Elasticsearch rally
Обслуживание кластера ElasticSearch
При обновлении обязательно используйте серое обновление, начиная с образа с наибольшим порядковым номером. Иначе, если шарды потеряны, поверьте мне, вам будет очень плохо.
1
2
3
4
5
6
7
8
9
10
11
kubectl patch statefulset elasticsearch -p \
'{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":0}}}}' \
-n test
kubectl patch statefulset elasticsearch -p \
'{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":1}}}}' \
-n test
kubectl patch statefulset elasticsearch -p \
'{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":2}}}}' \
-n test