ETCD 集群部署

本文内容纲要:

- 部署ETCD集群

- Etcd 解析

- 方法一 修改hosts文件

- 方法二 增加bind解析

- 下载和分发etcd二进制文件

- 创建etcd证书和私钥

- 创建etcd的启动文件

- 测试 ETCD 集群状态

目录

  • 部署ETCD集群

  • Etcd 解析

    • 方法一 修改hosts文件
    • 方法二 增加bind解析

  • 下载和分发etcd二进制文件

  • 创建etcd证书和私钥

  • 创建etcd的启动文件

  • 测试 ETCD 集群状态

下面的操作依托于上一篇文章

部署前期准备工作

部署ETCD集群

这里使用的ETCD为三节点高可用集群,步骤如下

  • 下载和分发etcd二进制文件
  • 创建etcd集群各节点的x509证书,用于加密客户端(如kubectl)与etcd集群、etcd集群之间的数据流
  • 创建etcd的system unit文件,配置服务参数
  • 检查集群工作状态

注意: 没有特殊说明都在node01节点操作

Etcd 解析

本次使用etcd单独的域名解析

方法一 修改hosts文件

在所有机器上操作, /etc/hosts 文件最后增加下面内容,也可以替换成自己的IP

10.0.20.11 etcd01 etcd01.k8s.com

10.0.20.12 etcd02 etcd02.k8s.com

10.0.20.13 etcd03 etcd03.k8s.com

方法二 增加bind解析

如果使用的内网 DNS bind 做内网解析增加下面解析

etcd01  IN  A   10.0.20.11

etcd02 IN A 10.0.20.12

etcd03 IN A 10.0.20.13

下载和分发etcd二进制文件

二进制文件在 部署前期准备工作 文章中已经下载好,直接使用;

分发二进制文件到ETCD集群节点

cd /opt/k8s/work

source /opt/k8s/bin/environment.sh

for node_ip in ${ETCD_IPS[@]}

do

echo ">>> ${node_ip}"

scp etcd-v3.3.13-linux-amd64/etcd* root@${node_ip}:/opt/k8s/bin

ssh root@${node_ip} "chmod +x /opt/k8s/bin/*"

done

创建etcd证书和私钥

cd /opt/k8s/work

cat > etcd-csr.json <<EOF

{

"CN": "etcd",

"hosts": [

"127.0.0.1",

"10.0.20.11",

"10.0.20.12",

"10.0.20.13",

"etcd01.k8s.com",

"etcd02.k8s.com",

"etcd03.k8s.com"

],

"key": {

"algo": "rsa",

"size": 2048

},

"names": [

{

"C": "CN",

"ST": "BeiJing",

"L": "BeiJing",

"O": "k8s",

"OU": "4Paradigm"

}

]

}

EOF

#host字段指定授权使用该证书的etcd节点IP或域名列表,需要将etcd集群的3个节点都添加其中

生成证书和私钥

cd /opt/k8s/work

cfssl gencert -ca=/opt/k8s/work/ca.pem \

-ca-key=/opt/k8s/work/ca-key.pem \

-config=/opt/k8s/work/ca-config.json \

-profile=kubernetes etcd-csr.json | cfssljson -bare etcd

ls etcd*pem

分发证书和私钥到etcd各个节点

cd /opt/k8s/work

source /opt/k8s/bin/environment.sh

for node_ip in ${ETCD_IPS[@]}

do

echo ">>> ${node_ip}"

ssh root@${node_ip} "mkdir -p /etc/etcd/cert"

scp etcd*.pem root@${node_ip}:/etc/etcd/cert/

done

创建etcd的启动文件

这里相对应的etcd 的配置,就保存在启动文件中

cd /opt/k8s/work

source /opt/k8s/bin/environment.sh

cat > etcd.service.template <<EOF

[Unit]

Description=Etcd Server

After=network.target

After=network-online.target

Wants=network-online.target

Documentation=https://github.com/coreos

[Service]

Type=notify

WorkingDirectory=${ETCD_DATA_DIR}

ExecStart=/opt/k8s/bin/etcd \\

--data-dir=${ETCD_DATA_DIR} \\

--wal-dir=${ETCD_WAL_DIR} \\

--name=##NODE_NAME## \\

--cert-file=/etc/etcd/cert/etcd.pem \\

--key-file=/etc/etcd/cert/etcd-key.pem \\

--trusted-ca-file=/etc/kubernetes/cert/ca.pem \\

--peer-cert-file=/etc/etcd/cert/etcd.pem \\

--peer-key-file=/etc/etcd/cert/etcd-key.pem \\

--peer-trusted-ca-file=/etc/kubernetes/cert/ca.pem \\

--peer-client-cert-auth \\

--client-cert-auth \\

--listen-peer-urls=https://##NODE_IP##:2380 \\

--initial-advertise-peer-urls=https://##NODE_IP##:2380 \\

--listen-client-urls=https://##NODE_IP##:2379,http://127.0.0.1:2379 \\

--advertise-client-urls=https://##NODE_IP##:2379 \\

--initial-cluster-token=etcd-cluster-0 \\

--initial-cluster=${ETCD_NODES} \\

--initial-cluster-state=new \\

--auto-compaction-mode=periodic \\

--auto-compaction-retention=1 \\

--max-request-bytes=33554432 \\

--quota-backend-bytes=6442450944 \\

--heartbeat-interval=250 \\

--election-timeout=2000

Restart=on-failure

RestartSec=5

LimitNOFILE=65536

[Install]

WantedBy=multi-user.target

EOF

配置说明 (此处不需要修改任何配置)

  • WorkDirectory、–data-dir 指定etcd工作目录和数据存储为${ETCD_DATA_DIR},需要在启动前创建这个目录 (后面跟着我操作就可以,会有创建步骤)
  • –wal-dir 指定wal目录,为了提高性能,一般使用SSD和–data-dir不同的盘
  • –name 指定节点名称,当–initial-cluster-state值为new时,–name的参数值必须位于–initial-cluster列表中
  • –cert-file、–key-file ETCD server与client通信时使用的证书和私钥
  • –trusted-ca-file 签名client证书的CA证书,用于验证client证书
  • –peer-cert-file、–peer-key-file ETCD与peer通信使用的证书和私钥
  • –peer-trusted-ca-file 签名peer证书的CA证书,用于验证peer证书

拆分三个配置文件,并修改信息

cd /opt/k8s/work

source /opt/k8s/bin/environment.sh

for (( i=0; i < 3; i++ ))

do

sed -e "s/##NODE_NAME##/${ETCD_NAMES[i]}/" -e "s/##NODE_IP##/${ETCD_IPS[i]}/" etcd.service.template > etcd-${ETCD_IPS[i]}.service

done

etcd*.service

分发生成的etcd启动文件到对应的服务器

cd /opt/k8s/work

source /opt/k8s/bin/environment.sh

for node_ip in ${ETCD_IPS[@]}

do

echo ">>> ${node_ip}"

scp etcd-${node_ip}.service root@${node_ip}:/etc/systemd/system/etcd.service

done

启动etcd服务

etcd首次进程启动会等待其他节点加入etcd集群,执行启动命令会卡顿一会,为正常现象

远程创建对应 ETCD 的数据目录等

source /opt/k8s/bin/environment.sh

for node_ip in ${ETCD_IPS[@]}

do

echo ">>> ${node_ip}"

ssh root@${node_ip} "mkdir -p ${ETCD_DATA_DIR} ${ETCD_WAL_DIR}"

ssh root@${node_ip} "systemctl daemon-reload && systemctl enable etcd && systemctl restart etcd " &

done

测试 ETCD 集群状态

检查启动结果

cd /opt/k8s/work

source /opt/k8s/bin/environment.sh

for node_ip in ${MASTER_IPS[@]}

do

echo ">>> ${node_ip}"

ssh root@${node_ip} "systemctl status etcd|grep Active"

done

输出结果:

[root@node01 work]# for node_ip in ${MASTER_IPS[@]}

> do

> echo ">>> ${node_ip}"

> ETCDCTL_API=3 /opt/k8s/bin/etcdctl \

> --endpoints=https://${node_ip}:2379 \

> --cacert=/etc/kubernetes/cert/ca.pem \

> --cert=/etc/etcd/cert/etcd.pem \

> --key=/etc/etcd/cert/etcd-key.pem endpoint health

> done

>>> 10.0.20.11

https://10.0.20.11:2379 is healthy: successfully committed proposal: took = 1.609991ms

>>> 10.0.20.12

https://10.0.20.12:2379 is healthy: successfully committed proposal: took = 1.117871ms

>>> 10.0.20.13

https://10.0.20.13:2379 is healthy: successfully committed proposal: took = 1.49139ms

通过下面命令查看当前etcd集群leader

source /opt/k8s/bin/environment.sh

ETCDCTL_API=3 /opt/k8s/bin/etcdctl \

-w table --cacert=/etc/kubernetes/cert/ca.pem \

--cert=/etc/etcd/cert/etcd.pem \

--key=/etc/etcd/cert/etcd-key.pem \

--endpoints=${ETCD_ENDPOINTS} endpoint status

输出结果如下:

[root@node01 work]# source /opt/k8s/bin/environment.sh

[root@node01 work]# ETCDCTL_API=3 /opt/k8s/bin/etcdctl \

> -w table --cacert=/etc/kubernetes/cert/ca.pem \

> --cert=/etc/etcd/cert/etcd.pem \

> --key=/etc/etcd/cert/etcd-key.pem \

> --endpoints=${ETCD_ENDPOINTS} endpoint status

+-----------------------------+------------------+---------+---------+-----------+-----------+------------+

| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | RAFT TERM | RAFT INDEX |

+-----------------------------+------------------+---------+---------+-----------+-----------+------------+

| https://etcd01.k8s.com:2379 | 6330dc0a28f62066 | 3.3.13 | 16 kB | false | 35 | 14 |

| https://etcd02.k8s.com:2379 | 77bc4da10f4c40bb | 3.3.13 | 16 kB | true | 35 | 14 |

| https://etcd03.k8s.com:2379 | d2573d5cc998d0f0 | 3.3.13 | 16 kB | false | 35 | 14 |

+-----------------------------+------------------+---------+---------+-----------+-----------+------------+



如果对ETCD集群安装不熟悉的,可以参考文章 CentOS 7 ETCD集群配置大全

本文内容总结:部署ETCD集群,Etcd 解析,方法一 修改hosts文件,方法二 增加bind解析,下载和分发etcd二进制文件,创建etcd证书和私钥,创建etcd的启动文件,测试 ETCD 集群状态,

原文链接:https://www.cnblogs.com/winstom/p/11992124.html

以上是 ETCD 集群部署 的全部内容, 来源链接: utcz.com/z/296994.html

回到顶部