Kubernetes m6S之存储点Secret精解

 

K8S之存储Secret概述与类型说明,并详解常用Secret示例

 

主机配置规划

服务器名称(hostname)系统版本配置内网IP外网IP(模拟)
k8s-masterCentOS7.72C/4G/20G172.16.1.11010.0.0.110
k8s-node01CentOS7.72C/4G/20G172.16.1.11110.0.0.111
k8s-node02CentOS7.72C/4G/20G172.16.1.11210.0.0.112

 

Secret概述

Secret解决了密码、token、秘钥等敏感数据的配置问题,而不需要把这些敏感数据暴露到镜像或者Pod Spec中。Secret可以以Volume或者环境变量的方式使用

用户可以创建 secret,同时系统也创建了一些 secret。

要使用 secret,pod 需要引用 secret。Pod 可以用两种方式使用 secret:作为 volume 中的文件被挂载到 pod 中的一个或者多个容器里,或者当 kubelet 为 pod 拉取镜像时使用。

 

Secret类型

  • Service Account:用来访问Kubernetes API,由Kubernetes自动创建,并且会自动挂载到Pod的 /run/secrets/kubernetes.io/serviceaccount 目录中。
  • Opaque:base64编码格式的Secret,用来存储密码、秘钥等。
  • kubernetes.io/dockerconfigjson:用来存储私有docker registry的认证信息。

 

Service Account

通过kube-proxy查看

 1 [root@k8s-master ~]# kubectl get pod -A | grep'kube-proxy'

2 kube-system kube-proxy-6bfh7 1/1 Running 12 7d3h

3 kube-system kube-proxy-6vfkf 1/1 Running 11 7d3h

4 kube-system kube-proxy-bvl9n 1/1 Running 11 7d3h

5 [root@k8s-master ~]#

6 [root@k8s-master ~]# kubectl exec -it -n kube-system kube-proxy-6bfh7 -- /bin/sh

7 # ls -l /run/secrets/kubernetes.io/serviceaccount

8 total 0

9 lrwxrwxrwx 1 root root 13 Jun 813:39 ca.crt -> ..data/ca.crt

10 lrwxrwxrwx 1 root root 16 Jun 813:39 namespace -> ..data/namespace

11 lrwxrwxrwx 1 root root 12 Jun 813:39 token -> ..data/token

 

Opaque Secret

创建secret

手动加密,基于base64加密

1 [root@k8s-master ~]# echo -n 'admin' | base64

2 YWRtaW4=

3 [root@k8s-master ~]# echo -n '1f2d1e2e67df' | base64

4 MWYyZDFlMmU2N2Rm

 

yaml文件

 1 [root@k8s-master secret]# pwd

2 /root/k8s_practice/secret

3 [root@k8s-master secret]# cat secret.yaml

4apiVersion: v1

5kind: Secret

6metadata:

7 name: mysecret

8type: Opaque

9data:

10 username: YWRtaW4=

11 password: MWYyZDFlMmU2N2Rm

 

或者通过如下命令行创建【secret名称故意设置不一样,以方便查看对比】,生成secret后会自动加密,而非明文存储。

kubectl create secret generic db-user-pass --from-literal=username=admin --from-literal=password=1f2d1e2e67df

 

生成secret,并查看状态

 1 [root@k8s-master secret]# kubectl apply -f secret.yaml

2 secret/mysecret created

3 [root@k8s-master secret]#

4 [root@k8s-master secret]# kubectl get secret ### 查看默认名称空间的secret简要信息

5NAME TYPE DATA AGE

6 basic-auth Opaque 1 2d12h

7 default-token-v48g4 kubernetes.io/service-account-token 3 27d

8 mysecret Opaque 2 23s ### 可见已创建

9 tls-secret kubernetes.io/tls 2 3d2h

10 [root@k8s-master secret]#

11 [root@k8s-master secret]# kubectl get secret mysecret -o yaml ### 查看mysecret详细信息

12apiVersion: v1

13data:

14 password: MWYyZDFlMmU2N2Rm

15 username: YWRtaW4=

16kind: Secret

17metadata:

18 annotations:

19 kubectl.kubernetes.io/last-applied-configuration: |

20 {"apiVersion":"v1","data":{"password":"MWYyZDFlMmU2N2Rm","username":"YWRtaW4="},"kind":"Secret","metadata":{"annotations":{},"name":"mysecret","namespace":"default"},"type":"Opaque"}

21 creationTimestamp: "2020-06-08T14:08:59Z"

22 name: mysecret

23 namespace: default

24 resourceVersion: "987419"

25 selfLink: /api/v1/namespaces/default/secrets/mysecret

26 uid: 27b58929-71c4-495b-99a5-0d411910a529

27type: Opaque

28 [root@k8s-master secret]#

29 [root@k8s-master secret]# kubectl describe secret mysecret ### 查看描述信息

30Name: mysecret

31Namespace: default

32 Labels: <none>

33Annotations:

34Type: Opaque

35

36Data

37 ====

38 password: 12 bytes

39 username: 5 bytes

 

将Secret挂载到Volume中

yaml文件

 1 [root@k8s-master secret]# pwd

2 /root/k8s_practice/secret

3 [root@k8s-master secret]# cat pod_secret_volume.yaml

4apiVersion: v1

5kind: Pod

6metadata:

7 name: pod-secret-volume

8spec:

9 containers:

10 - name: myapp

11 image: registry.cn-beijing.aliyuncs.com/google_registry/myapp:v1

12 volumeMounts:

13 - name: secret-volume

14 mountPath: /etc/secret

15 readOnly: true

16 volumes:

17 - name: secret-volume

18 secret:

19 secretName: mysecret

 

启动pod并查看状态

1 [root@k8s-master secret]# kubectl apply -f pod_secret_volume.yaml

2 pod/pod-secret-volume created

3 [root@k8s-master secret]#

4 [root@k8s-master secret]# kubectl get pod -o wide

5NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

6 pod-secret-volume 1/1 Running 0 16s 10.244.2.159 k8s-node02 <none> <none>

 

查看secret信息

1 [root@k8s-master secret]# kubectl exec -it pod-secret-volume -- /bin/sh

2 / # ls /etc/secret

3password username

4 / #

5 / # cat /etc/secret/username

6 admin/ #

7 / #

8 / # cat /etc/secret/password

9 1f2d1e2e67df/ #

由上可见,在pod中的secret信息实际已经被解密。

 

将Secret导入到环境变量中

yaml文件

 1 [root@k8s-master secret]# pwd

2 /root/k8s_practice/secret

3 [root@k8s-master secret]# cat pod_secret_env.yaml

4apiVersion: v1

5kind: Pod

6metadata:

7 name: pod-secret-env

8spec:

9 containers:

10 - name: myapp

11 image: registry.cn-beijing.aliyuncs.com/google_registry/myapp:v1

12env:

13 - name: SECRET_USERNAME

14 valueFrom:

15 secretKeyRef:

16 name: mysecret

17 key: username

18 - name: SECRET_PASSWORD

19 valueFrom:

20 secretKeyRef:

21 name: mysecret

22 key: password

23 restartPolicy: Never

 

启动pod并查看状态

1 [root@k8s-master secret]# kubectl apply -f pod_secret_env.yaml

2 pod/pod-secret-env created

3 [root@k8s-master secret]#

4 [root@k8s-master secret]# kubectl get pod -o wide

5NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

6 pod-secret-env1/1 Running 0 6s 10.244.2.160 k8s-node02 <none> <none>

 

查看secret信息

 1 [root@k8s-master secret]# kubectl exec -it pod-secret-env -- /bin/sh

2 / # env

3………………

4 HOME=/root

5 SECRET_PASSWORD=1f2d1e2e67df ### secret信息

6 MYAPP_SVC_PORT_80_TCP=tcp://10.98.57.156:80

7 TERM=xterm

8 NGINX_VERSION=1.12.2

9 KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1

10 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

11 KUBERNETES_PORT_443_TCP_PORT=443

12 KUBERNETES_PORT_443_TCP_PROTO=tcp

13 MYAPP_SVC_SERVICE_HOST=10.98.57.156

14 SECRET_USERNAME=admin ### secret信息

15 KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443

16 ………………

由上可见,在pod中的secret信息实际已经被解密。

 

docker-registry Secret

harbor镜像仓库

首先使用harbor搭建镜像仓库,搭建部署过程参考:「Harbor企业级私有Docker镜像仓库部署」

harbor部分配置文件信息

 1 [root@k8s-master harbor]# pwd

2 /root/App/harbor

3 [root@k8s-master harbor]# vim harbor.yml

4 # Configuration file of Harbor

5hostname: 172.16.1.110

6

7# http related config

8http:

9 # port for http, default is 80. If https enabled, this port will redirect to https port

10 port: 5000

11

12# https related config

13https:

14 # https port for harbor, default is 443

15 port: 443

16 # The path of cert and key files for nginx

17 certificate: /etc/harbor/cert/httpd.crt

18 private_key: /etc/harbor/cert/httpd.key

19 harbor_admin_password: Harbor12345

 

启动harbor后客户端http设置

集群所有机器都要操作

 1 [root@k8s-master ~]# vim /etc/docker/daemon.json

2{

3"exec-opts": ["native.cgroupdriver=systemd"],

4"log-driver": "json-file",

5"log-opts": {

6"max-size": "100m"

7 },

8"insecure-registries": ["172.16.1.110:5000"]

9}

10 [root@k8s-master ~]#

11 [root@k8s-master ~]# systemctl restart docker # 重启docker服务

 添加了 “insecure-registries”: [“172.16.1.110:5000”] 这行,其中172.16.1.110为内网IP地址。该文件必须符合 json 规范,否则 Docker 将不能启动。

如果在Harbor所在的机器重启了docker服务,记得要重新启动Harbor。

 

创建「私有」仓库

 

 

镜像上传

1 docker pull registry.cn-beijing.aliyuncs.com/google_registry/myapp:v1

2 docker tag registry.cn-beijing.aliyuncs.com/google_registry/myapp:v1 172.16.1.110:5000/k8s-secret/myapp:v1

3# 登录

4 docker login172.16.1.110:5000 -u admin -p Harbor12345

5# 上传

6 docker push 172.16.1.110:5000/k8s-secret/myapp:v1

 

 

退出登录

之后在操作机上退出harbor登录,便于后面演示

1### 退出harbor登录

2 [root@k8s-node02 ~]# docker logout 172.16.1.110:5000

3 Removing login credentials for172.16.1.110:5000

4### 拉取失败,需要先登录。表明完成准备工作

5 [root@k8s-master secret]# docker pull 172.16.1.110:5000/k8s-secret/myapp:v1

6 Error response from daemon: pull access denied for172.16.1.110:5000/k8s-secret/myapp, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

 

pod直接下载镜像

在yaml文件中指定image后,直接启动pod

 1 [root@k8s-master secret]# pwd

2 /root/k8s_practice/secret

3 [root@k8s-master secret]# cat pod_secret_registry.yaml

4apiVersion: v1

5kind: Pod

6metadata:

7 name: pod-secret-registry

8spec:

9 containers:

10 - name: myapp

11 image: 172.16.1.110:5000/k8s-secret/myapp:v1

 

启动pod并查看状态

 1 [root@k8s-master secret]# kubectl apply -f pod_secret_registry.yaml

2 pod/pod-secret-registry created

3 [root@k8s-master secret]#

4 [root@k8s-master secret]# kubectl get pod -o wide ### 可见镜像下载失败

5NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

6 pod-secret-registry 0/1 ImagePullBackOff 0 7s 10.244.2.161 k8s-node02 <none> <none>

7 [root@k8s-master secret]#

8 [root@k8s-master secret]# kubectl describe pod pod-secret-registry ### 查看pod详情

9 Name: pod-secret-registry

10Namespace: default

11 Priority: 0

12 Node: k8s-node02/172.16.1.112

13 Start Time: Mon, 08 Jun 202023:59:07 +0800

14 Labels: <none>

15 Annotations: kubectl.kubernetes.io/last-applied-configuration:

16 {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"name":"pod-secret-registry","namespace":"default"},"spec":{"containers":[{"i...

17Status: Pending

18 IP: 10.244.2.161

19IPs:

20 IP: 10.244.2.161

21Containers:

22 myapp:

23 Container ID:

24 Image: 172.16.1.110:5000/k8s-secret/myapp:v1

25 Image ID:

26………………

27Events:

28 Type Reason Age From Message

29 ---- ------ ---- ---- -------

30 Normal Scheduled 23s default-scheduler Successfully assigned default/pod-secret-registry to k8s-node02

31 Normal BackOff 19s (x2 over 20s) kubelet, k8s-node02 Back-off pulling image "172.16.1.110:5000/k8s-secret/myapp:v1"

32 Warning Failed 19s (x2 over 20s) kubelet, k8s-node02 Error: ImagePullBackOff

33 Normal Pulling 9s (x2 over 21s) kubelet, k8s-node02 Pulling image "172.16.1.110:5000/k8s-secret/myapp:v1"

34 Warning Failed 9s (x2 over 21s) kubelet, k8s-node02 Failed to pull image "172.16.1.110:5000/k8s-secret/myapp:v1": rpc error: code = Unknown desc = Error response from daemon: pull access denied for172.16.1.110:5000/k8s-secret/myapp, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

35 Warning Failed 9s (x2 over 21s) kubelet, k8s-node02 Error: ErrImagePull

36 [root@k8s-master secret]#

37 [root@k8s-master secret]# kubectl delete -f pod_secret_registry.yaml

可见拉取私有镜像失败。

 

pod通过Secret下载镜像

通过命令行创建Secret,并查看其描述信息

 1 [root@k8s-master secret]# kubectl create secret docker-registry myregistrysecret --docker-server='172.16.1.110:5000' --docker-username='admin' --docker-password='Harbor12345'

2 secret/myregistrysecret created

3 [root@k8s-master secret]#

4 [root@k8s-master secret]# kubectl get secret

5NAME TYPE DATA AGE

6 basic-auth Opaque 1 2d14h

7 default-token-v48g4 kubernetes.io/service-account-token 3 27d

8 myregistrysecret kubernetes.io/dockerconfigjson 1 8s # 刚刚创建的

9 mysecret Opaque 2 118m

10 tls-secret kubernetes.io/tls 2 3d4h

11 [root@k8s-master secret]#

12 [root@k8s-master secret]# kubectl get secret myregistrysecret -o yaml ### 查看详细信息

13apiVersion: v1

14data:

15 .dockerconfigjson: eyJhdXRocyI6eyIxMC4wLjAuMTEwOjUwMDAiOnsidXNlcm5hbWUiOiJhZG1pbiIsInBhc3N3b3JkIjoiSGFyYm9yMTIzNDUiLCJhdXRoIjoiWVdSdGFXNDZTR0Z5WW05eU1USXpORFU9In19fQ==

16kind: Secret

17metadata:

18 creationTimestamp: "2020-06-08T16:07:32Z"

19 name: myregistrysecret

20 namespace: default

21 resourceVersion: "1004582"

22 selfLink: /api/v1/namespaces/default/secrets/myregistrysecret

23 uid: b95f4386-64bc-4ba3-b43a-08afb1c1eb9d

24 type: kubernetes.io/dockerconfigjson

25 [root@k8s-master secret]#

26 [root@k8s-master secret]# kubectl describe secret myregistrysecret ### 查看描述信息

27Name: myregistrysecret

28Namespace: default

29 Labels: <none>

30 Annotations: <none>

31

32 Type: kubernetes.io/dockerconfigjson

33

34Data

35 ====

36 .dockerconfigjson: 109 bytes

 

修改之前的yaml文件

 1 [root@k8s-master secret]# cat pod_secret_registry.yaml

2apiVersion: v1

3kind: Pod

4metadata:

5 name: pod-secret-registry

6spec:

7 containers:

8 - name: myapp

9 image: 172.16.1.110:5000/k8s-secret/myapp:v1

10 imagePullSecrets:

11 - name: myregistrysecret

 

启动pod并查看状态

 1 [root@k8s-master secret]# kubectl apply -f pod_secret_registry.yaml

2 pod/pod-secret-registry created

3 [root@k8s-master secret]#

4 [root@k8s-master secret]# kubectl get pod -o wide

5NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

6 pod-secret-registry 1/1 Running 0 8s 10.244.2.162 k8s-node02 <none> <none>

7 [root@k8s-master secret]#

8 [root@k8s-master secret]# kubectl describe pod pod-secret-registry

9 Name: pod-secret-registry

10Namespace: default

11 Priority: 0

12 Node: k8s-node02/172.16.1.112

13 Start Time: Tue, 09 Jun 202000:22:40 +0800

14 Labels: <none>

15 Annotations: kubectl.kubernetes.io/last-applied-configuration:

16 {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"name":"pod-secret-registry","namespace":"default"},"spec":{"containers":[{"i...

17Status: Running

18 IP: 10.244.2.162

19IPs:

20 IP: 10.244.2.162

21Containers:

22 myapp:

23 Container ID: docker://ef4d42f1f1616a44c2a6c0a5a71333b27f46dfe76eb392962813a28d69150c00

24 Image: 172.16.1.110:5000/k8s-secret/myapp:v1

25 Image ID: docker-pullable://172.16.1.110:5000/k8s-secret/myapp@sha256:9eeca44ba2d410e54fccc54cbe9c021802aa8b9836a0bcf3d3229354e4c8870e

26 Port: <none>

27 Host Port: <none>

28 State: Running

29 Started: Tue, 09 Jun 202000:22:41 +0800

30 Ready: True

31 Restart Count: 0

32 Environment: <none>

33 Mounts:

34 /var/run/secrets/kubernetes.io/serviceaccount from default-token-v48g4 (ro)

35Conditions:

36 Type Status

37 Initialized True

38 Ready True

39 ContainersReady True

40 PodScheduled True

41Volumes:

42 default-token-v48g4:

43 Type: Secret (a volume populated by a Secret)

44 SecretName: default-token-v48g4

45 Optional: false

46QoS Class: BestEffort

47 Node-Selectors: <none>

48 Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s

49 node.kubernetes.io/unreachable:NoExecute for 300s

50Events:

51 Type Reason Age From Message

52 ---- ------ ---- ---- -------

53 Normal Scheduled 22s default-scheduler Successfully assigned default/pod-secret-registry to k8s-node02

54 Normal Pulling 22s kubelet, k8s-node02 Pulling image "172.16.1.110:5000/k8s-secret/myapp:v1"

55 Normal Pulled 22s kubelet, k8s-node02 Successfully pulled image "172.16.1.110:5000/k8s-secret/myapp:v1"

56 Normal Created 22s kubelet, k8s-node02 Created container myapp

57 Normal Started 21s kubelet, k8s-node02 Started container myapp

由上可见,通过secret认证后pod拉取私有镜像是可以的。

 

相关阅读

1、Harbor企业级私有Docker镜像仓库部署

 


 

 

———END———
如果觉得不错就关注下呗 (-^O^-) !

 

以上是 Kubernetes m6S之存储点Secret精解 的全部内容, 来源链接: utcz.com/a/48348.html

回到顶部