002.OpenShift安装与部署
一 前置条件说明1.1 安装准备概述
Red Hat OpenShift容器平台是由Red Hat作为RPM包和容器映像两种类型存在。RPM包使用订阅管理器从标准Red Hat存储库(即Yum存储库)下载，容器映像来自Red Hat私有仓库。OpenShift容器平台安装需要多个服务器，支持服务器或虚拟机的多种形式。同时为了简化OpenShift集群的部署，Red Hat提供了一个基于Ansible的安装程序，它可以通过交互运行，也可以使用包含环境配置细节的应答文件以自动的非交互方式运行。在运行安装程序之前，需要执行一些预安装任务，以及安装后的安装任务，以获得功能齐全的OpenShift容器平台集群。RedHat为安装OpenShift容器平台提供了两种不同的方法。第一种方法使用快速安装程序，可用于简单的集群设置。
- 第二种方法是较为精细的安装方式，并使用Ansible playbook来自动化该过程。
本实验使用Ansible来自动配置OpenShift集群。同时，Ansible可以为OpenShift安装准备主机，例如包安装、禁用服务和客户化配置。提示：更多Ansible内容参考https://www.cnblogs.com/itzgr/category/1333622.html。1.2 节点准备
需要相应的master和node节点互通，并且配置master至所有节点的免秘钥登录。同时能解析所有FQDN，及注册相应repo库。提示：以上准备工作也可通过Ansible直接跑相应的yml完成。二 实验一：前置条件操作2.1 环境准备
[student@workstation ~]$ lab install-prepare setup #运行准备脚本提示：本环境基于RedHat RH280环境，所有lab命令为环境自动化准备命令，后续不再赘述。2.2 安装Ansible
[student@workstation ~]$ rpm -qa | grep ansible[student@workstation ~]$ sudo yum -y install ansible2.3 验证Ansible
[student@workstation ~]$ cd /home/student/DO280/labs/install-prepare/[student@workstation ~]$ ansible --version[student@workstation install-prepare]$ cat ansible.cfg[student@workstation install-prepare]$ cat inventoryInventory文件解释：Inventory定义了六个主机组:- workstations：为developer节点，即运行playbook的节点；
- nfs：为集群存储提供nfs服务的环境中的vm；
- masters：OpenShift集群中用作master角色的节点；
- etcd：用于OpenShift集群的etcd服务的节点，本环境中使用master节点；
- node：OpenShift集群中的node节点；
- OSEv3：组成OpenShift集群的所有接待，包括master、etcd、node或nfs组中的节点。
注意：默认情况下，docker使用在线仓库下载容器映像。本环境内部无网络，因此将docker仓库配置为内部私有仓库。在yml中使用变量引入仓库配置。此外，安装会在每个主机上配置docker守护进程，以使用overlay2 image驱动程序存储容器映像。Docker支持许多不同的image驱动。如AUFS、Btrfs、Device mapper、OverlayFS。2.4 检查节点连通性
[student@workstation install-prepare]$ cat ping.yml 1 --- 2 - name: Verify Connectivity
3 hosts: all
4 gather_facts: no
5 tasks:
6 - name: "Test connectivity to machines."
7 shell: "whoami"
8 changed_when: false
[student@workstation install-prepare]$ ansible-playbook -v ping.yml2.5 确认yml
[student@workstation install-prepare]$ cat prepare_install.yml 解释：如上yml引入了三个role。docker-storage内容如下，该role定义相关docker的后端存储驱动以及创建docker所需的image存储路径，并最终启动docker。[student@workstation install-prepare]$ cat roles/docker-storage/tasks/main.yml 1 --- 2 - block:
3 - name: Customize default /etc/sysconfig/docker-storage-setup
4 template:
5 src: docker-storage-setup
6 dest: /etc/sysconfig/docker-storage-setup
7 owner: root
8group: root
9 mode: 0644
10 when: not use_overlay2_driver
11 - name: Customize /etc/sysconfig/docker-storage-setup using overlay2 storage driver
12 template:
13 src: docker-storage-setup-overlay2
14 dest: /etc/sysconfig/docker-storage-setup
15 owner: root
16group: root
17 mode: 0644
18 when: use_overlay2_driver
19 - name: Verify existence of /dev/docker-vg/docker-pool
20 stat:
21path: /dev/docker-vg/docker-pool
22register: p
23 - name: Stop docker
24 service:
25name: docker
26 state: stopped
27 when: p.stat.exists == False
28 - name: Remove loopback docker files
29 file:
30 dest: /var/lib/docker
31 state: absent
32 when: p.stat.exists == False
33 - name: Run docker-storage-setup
34 command: /usr/bin/docker-storage-setup
35 when: p.stat.exists == False
36 - name: Start and enable docker
37 service:
38name: docker
39 state: started
40 when: p.stat.exists == False
41 when: docker_storage_device is defined
42
[student@workstation install-prepare]$ cat roles/docker-storage/templates/docker-storage-setup 1 DEVS={{ docker_storage_device }} 2 VG=docker-vg
3 SETUP_LVM_THIN_POOL=yes
docker-registry-cert内容如下，该role定义相关docker的使用私有仓库，并且导入了相关crt证书。
[student@workstation install-prepare]$ cat roles/docker-registry-cert/tasks/main.yml 1 --- 2 - name: Enable the Trust
3 shell: update-ca-trust enable
4 - name: Retrieve the certificate
5 fetch:
6 src: "{{ cacert }}"
7 dest: "{{ local_destination }}"
8 delegate_to: "{{ registry_host }}"
9 - name: Copy the certificate
10copy:
11 src: "{{ source }}"
12 dest: "{{ destination }}"
13 owner: root
14group: root
15 mode: 0755
16 - name: Update the Trust
17 shell: update-ca-trust extract
18 - name: Restart Docker
19 service:
20name: docker
21 state: restarted
22
[student@workstation install-prepare]$ cat roles/docker-registry-cert/vars/main.yml 1 registry_host: services.lab.example.com 2 cacert: /etc/pki/tls/certs/example.com.crt
3 local_destination: /tmp/
4 source: "/tmp/{{ ansible_fqdn }}/etc/pki/tls/certs/example.com.crt"
5 destination: /etc/pki/ca-trust/source/anchors/example.com.crt
openshift-node内容如下，该role定义相关安装OpenShift所需的所有依赖包任务。
[student@workstation install-prepare]$ ll roles/openshift-node/files/ total 4-rw-r--r--. 1 student student 389 Jul 19 2018 id_rsa.pub[student@workstation install-prepare]$ cat roles/openshift-node/meta/main.yml 1 --- 2 dependencies:
3 - { role: docker }
[student@workstation install-prepare]$ cat roles/openshift-node/tasks/main.yml 1 --- 2 - name: Deploy ssh key to root at all nodes
3 authorized_key:
4user: root
5 key: "{{ item }}"
6 with_file:
7 - id_rsa.pub
8 - name: Install required packages
9 yum:
10name: "{{ item }}"
11 state: latest
12 with_items:
13 - wget
14 - git
15 - net-tools
16 - bind-utils
17 - iptables-services
18 - bridge-utils
19 - bash-completion
20 - kexec-tools
21 - sos
22 - psacct
23 - atomic-openshift-clients
24 - atomic-openshift-utils
25 - atomic-openshift
26
2.6 运行playbook
[student@workstation ~]$ cd /home/student/DO280/labs/install-prepare/[student@workstation install-prepare]$ ansible-playbook prepare_install.yml提示：该准备工作将完成如下操作：- 在每个节点上安装并运行Docker；
- 在每个节点上Docker使用一个逻辑卷存储；
- 每个节点使用自签名证书信任私有Docker仓库；
- 在每个节点上都会安装基本包。
2.7 确认验证
[student@workstation install-prepare]$ for vm in master node1 node2;do echo -e "$vm"
ssh $vm sudo systemctl status docker | head -n3done #验证docker服务[student@workstation install-prepare]$ for vm in master node1 node2;do echo -e "$vm : lvs"
ssh $vm sudo lvsecho -e "$vm : df -h"
ssh $vm sudo df -h | grep vg-dockerdone #查看docker使用的lvm[student@workstation install-prepare]$ for vm in master node1 node2;do echo -e "$vm"
ssh $vm docker pull rhel7:latestdone #测试pull image[student@workstation install-prepare]$ for vm in master node1 node2; do echo -e "$vm"
ssh $vm rpm -qa wget git net-tools bind-utils yum-utils iptables-services bridge-utils bash-completion kexec-tools sos psacct atomic-openshift-utilsdone #检查相关依赖包是否安装成功三 正式安装说明3.1 安装步骤
安装准备完成后正式安装包括四个步骤：- 编写一个目录文件来描述所需的集群特性和体系结构；
- 执行prerequisites.yml的playbook；
- 执行deploy_cluster,yml的playbook；
- 验证安装。
3.2 安装和配置节点
OpenShift Inventory定义了以下主机组。master：对于OpenShift，这是必须的组，定义了OpenShift集群中哪些主机充当master节点；node：对于OpenShift，这是必须的组，它定义了OpenShift集群中哪些主机充当node节点；etcd：[master]部分中列出的所有主机也应属于etcd；nfs：这个组是可选的，应该只包含一个主机。如果Inventory文件中存在特定的变量，OpenShift playbook将在这台机器上安装并配置NFS；OSEv3：这个组包含任何属于OpenShift集群的机器。安装剧本引用这个组来运行在集群全范围内的任务。[student@workstation install-prepare]$ cat inventory说明：- 安装所需版本的OpenShift容器平台；
- 用户使用htpasswd身份验证对集群进行身份验证；
- DNS条目apps.lab.example.com用作OpenShift应用程序的子域；
- NFS存储用于OpenShift etcd服务和OpenShift 内部仓库；
- classroom container registry用作仓库。
变量说明：OpenShift安装变量记录在Inventory的[OSEv3:vars]部分。安装变量用于配置多个OpenShift组件，例如：- 一个内部容器仓库；
- Gluster、Ceph等以便于提供持久性存储；
- 集群日志；
- 自定义集群证书。
3.3 配置OpenShift版本
可通过在[OSEv3:vars]中指定如下配置确定OpenShift所安装的版本：openshift_deployment_type=openshift-enterpriseopenshift_release=v3.9指定OpenShift部署类型，可选值为openshift-enterprise和origin。openshift_image_tag=v3.9.14openshift_disable_check=disk_availability,docker_storage,memory_availability容器化的OpenShift服务使用带有“v3.9.14”标记的图像。这将阻止集群自动升级到更新的容器映像；对于非生产集群，可以禁用对系统需求的检查。3.4 配置验证
OpenShift容器平台身份验证基于OAuth, OAuth提供了一个基于HTTP的APl，用于对交互式和非交互式客户端进行身份验证。OpenShift master运行一个OAuth服务器，OpenShift可以支持多种Provider，这些Provider可以与特定于组织的身份管理产品集成。支持的OpenShift身份验证的Provider:- HTTP Basic，外部单点登录(SSO)系统；
- 使用GitHub和GitLab帐号；
- OpenID连接，使用OpenID-compatible SSO和谷歌帐户；
- OpenStack Keystone v3；
- LDAP v3服务器。
OpenShift安装程序使用默认的安全方法，DenyAllPasswordIdentityProvider是缺省提供程序。使用此Provider，表示只有master主机上的root用户才能使用OpenShift客户端命令和API。3.5 配置htpasswd验证
OpenShift HTPasswdPasswordIdentityProvider根据Apache HTTPD htpasswd程序生成的文件验证用户和密码。htpasswd程序将用户名和密码保存在纯文本文件中，每行一条记录，字段用冒号分隔。密码使用MD5散列。如果将此文件添加或删除用户，或更改用户密码，OpenShift OAuth服务器将自动重新读取该文件。要将OpenShift master配置使用HTPasswdPasswordIdentityProvider，需要配置openshift_master_identity_providers。 1 openshift_master_identity_providers。 2 openshift_master_identity_providers=[{"name": "htpasswd_auth", "login": "true",
3 "challenge": "true", "kind": "HTPasswdPasswordIdentityProvider", #配置后端驱动
4 "filename": "/etc/origin/master/htpasswd"}] #制定master主机上
也支持在配置文件中直接指定初始的用户名和密码。openshift_master_htpasswd_users="{"user1":"$apr1$.NHMsZYc$MdmfWN5DM3q280/W7c51c/","user2":"$apr1$.NHMsZYc$MdmfWN5DM3q280/W7c51c/"}"生产hash密码可参考如下： 1 [student@workstation ~]$ htpasswd -nb admin redhat 2 [student@workstation ~]$ openssl passwd -apr1 redhat
3.6 网络要求
集群节点的通配符DNS条目允许任何新创建的路由自动路由到subdomain的集群。通配符DNS条目必须存在于唯一的子域中，例如apps.mycluster.com，并解析为主机名或集群节点的IP地址。inventory文件中通配符DNS条目是通过变量openshift_master_default_subdomain进行设置 。openshift_master_default_subdomain=apps.mycluster.com3.7 master服务端口
主服务端口openshift_master_api_port变量定义主API的监听端口。缺省端口8443，当master使用SSL时，也可以使用443端口。从而在连接的时候省略端口号。master console端口由openshift_master_console_port变量的值设置，默认端口是8443。master console端口也可以设置为443，从而在连接的时候省略端口号。3.8 防火墙OpenShift节点上的默认防火墙服务是iptables。若要在所有节点上使用firewalld作为防火墙服务，需要将操作系统防火墙使用firewalld变量设置为true，即os_firewall_use_firewalld=true。四 配置持久化存储4.1 持久存储配置
默认情况下，容器数据是临时的，并且在容器被销毁时丢失。Kubernetes持久卷框架为容器请求和使用持久存储提供了一种机制。为了避免数据丢失，这些服务被配置为使用持久卷。OpenShift支持多个插件，使用各种存储技术创建持久卷。可以使用NFS、iSCSI、GlusterFS、Ceph或其他商业云存储。本环境中，OpenShift容器registry和OpenShift Ansible Broker服务被配置为使用NFS持久性存储。提示：生产环境默认OpenShift不支持NFS持久存储集群，要允许NFS在非生产集群上持久存储，需要配置openshift_enable_unsupported_configurations=true。4.2 container仓库
要为OpenShift容器registry配置NFS持久性存储，请将以下内容添加到Inventory文件中： 1 openshift_hosted_registry_storage_kind=nfs 2 openshift_hosted_registry_storage_nfs_directory=/exports
3 openshift_hosted_registry_storage_volume_name=registry
4 openshift_hosted_registry_storage_nfs_options="*(rw,root_squash)"
5 openshift_hosted_registry_storage_volume_size=40G
6 openshift_hosted_registry_storage_access_modes=["ReadWriteMany"]
4.3 OpenShift Ansible Broker
OpenShift Ansible Broker(OAB)是一个容器化的OpenShift服务，部署自己的etcd服务。持久Etcd存储所需的配置与registry所需的配置类似。 1 openshift_hosted_etcd_storage_kind=nfs 2 openshift_hosted_etcd_storage_nfs_directory=/exports
3 openshift_hosted_etcd_storage_volume_name=etcd-vol2
4 openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)"
5 openshift_hosted_etcd_storage_volume_size=1G
6 openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"]
7 openshift_hosted_etcd_storage_labels={"storage": "etcd"}
五 OpenShift其他配置5.1 配置离线本地registry
本环境OpenShift使用容器仓库为registry.lab.example.com，要将集群配置为从内部仓库pull image，需要在Inventory中进行如下配置：
1#Modifications Needed for a Disconnected Install 2 oreg_url=registry.lab.example.com/openshift3/ose-${component}:${version}
3#可访问image仓库的位置，必须以ose-${component}:${version}结尾。
4 openshift_examples_modify_imagestreams=true
5#OpenShift安装了用于部署示例应用程序的模板。这个变量指示playbook修改所有示例的IS，使其指向私有仓库，而不是registry.access.redhat.com。
6 openshift_docker_additional_registries=registry.lab.example.com
7#此变量用于将本地可访问仓库添加到每个节点上的docker配置中。
8 openshift_docker_blocked_registries=registry.access.redhat.com,docker.io
9#此变量用于在OpenShift节点上配置docker的blocked_registries。
1#Image Prefix Modifications 2 openshift_web_console_prefix=registry.lab.example.com/openshift3/oseopenshift_cockpit_deployer_prefix="registry.lab.example.com/openshift3/"
3 openshift_service_catalog_image_prefix=registry.lab.example.com/openshift3/osetemplate_service_broker_prefix=registry.lab.example.com/openshift3/oseansible_service_broker_image_prefix=registry.lab.example.com/openshift3/oseansible_service_broker_etcd_image_prefix=registry.lab.example.com/rhel7/
#通过在容器image名称前面加上registry.lab.example.com以确保OpenShift服务的容器image可以从私有内部仓库下载。5.2 配置NODE labels
节点label是分配给每个节点的任意key/value描述。node label通常用于区分地理数据中心或标识节点上的可用资源的有意义的描述。应用程序可以在其deployment中根据node lables配置一个选择器。如果匹配到，应用程序的pod必须部署在其符合node labels的节点上。使用主机变量openshift_node_tags在Inventory文件中设置节点标签。 1 [nodes] 2 ...output omitted...
3 nodeX.example.com openshift_node_labels="{"zone":"west", "gpu":"true"}"
4 ...output omitted...
如上所示配置给nodeX.example.com配置两个labels，zone=west和gpu=true。OpenShift集群的一个常见架构是区分master、infrastructure node和compute node。在此架构中，infrastructure node承载OpenShift Pod的registry和路由器，而compute node承载来自用户项目的应用程序pod。master节点不承载应用程序或infrastructure pod。可使用 node label 来标识特定节点的角色，通常master node label 为 node-role.kubernetes.io/master=true，infrastructure node label 为 region=infra，compute node label 为 noderole.kubernetes.io/compute=true。 1 [nodes] 2 master.lab.example.com
3 node1.lab.example.com openshift_node_labels="{"region":"infra"}"
4 node2.lab.example.com
提示：如果一个节点设计为同时承载infrastructure 和 application pods，则必须显式定义两个节点标签。[nodes]...nodeX.example.com openshift_node_labels="{"region":"infra", "noderole.kubernetes.io/compute":"true"}"...六 执行剧本6.1 剧本说明
安装OpenShift需要执行prerequisites.yml 和deploy_cluster.yml，由 atomic-openshift-utils 软件包安装。首先执行 prequisites.yml playbook 检查所有主机能够满足OpenShift 的部署，同时尝试修改主机以满足部署需求。然后执行 doploy_cluster.yml playbook 开始正式集群部署6.2 验证OpenShift
部署完成后，可访问：https://master.lab.example.com 进行验证。七 正式安装OpenShift7.1 前置准备
[student@workstation ~]$ lab install-prepare setup[student@workstation ~]$ sudo yum -y install ansible[student@workstation ~]$ cd /home/student/do280-ansible/[student@workstation do280-ansible]$ ansible-playbook playbooks/prepare_install.yml #设置相关环境[student@workstation do280-ansible]$ lab install-run setup[student@workstation do280-ansible]$ cd /home/student/DO280/labs/install-run/7.2 安装atomic
[student@workstation install-run]$ sudo yum -y install atomic-openshift-utils提示：atomic-openshift-utils提供了安装OpenShift所需的Ansible playbook和role。7.3 创建Inventory
[student@workstation install-run]$ cp inventory.initial inventory[student@workstation install-run]$ cat inventory[student@workstation install-run]$ echo -e "[OSEv3:vars]" >> inventory
7.4 配置相关安装版本
1 [student@workstation install-run]$ vi general_vars.txt 2#General Cluster Variables
3 openshift_deployment_type=openshift-enterprise #配置为openshift-enterprise版本
4 openshift_release=v3.9 #配置版本为v3.9
5 openshift_image_tag=v3.9.14
6 openshift_disable_check=disk_availability,docker_storage,memory_availability #禁用check
7.5 设置htpasswd认证
1 [student@workstation install-run]$ openssl passwd -apr1 redhat 2 $apr1$/d1L7fdX$duViLRE.JG012VkZDq8bs0
3 [student@workstation install-run]$ openssl passwd -apr1 redhat
4 $apr1$rUMMfQfD$J8CEqQK.YenyNwYwKN1lA1 #创建两个用户密码都为redhat
5 [student@workstation install-run]$ vi authentication_vars.txt
6#Cluster Authentication Variables
7 openshift_master_identity_providers=[{"name": "htpasswd_auth", "login": "true", "challenge": "true", "kind": "HTPasswdPasswordIdentityProvider", "filename": "/etc/origin/master/htpasswd"}]
8 openshift_master_htpasswd_users={"admin":"$apr1$/d1L7fdX$duViLRE.JG012VkZDq8bs0", "developer":"$apr1$rUMMfQfD$J8CEqQK.YenyNwYwKN1lA1"}
7.6 配置集群网络
1 [student@workstation install-run]$ vi networking_vars.txt 2#OpenShift Networking Variables
3 os_firewall_use_firewalld=true #开启firewall防火墙
4 openshift_master_api_port=443 #启用端口
5 openshift_master_console_port=443 #启用控制端口
6 openshift_master_default_subdomain=apps.lab.example.com #指定subdomain
7.7 配置NFS
1 [student@workstation install-run]$ vi persistence_vars.txt 2#NFS is an unsupported configuration
3 openshift_enable_unsupported_configurations=true
4
5#OCR configuration variables
6 openshift_hosted_registry_storage_kind=nfs
7 openshift_hosted_registry_storage_access_modes=["ReadWriteMany"]
8 openshift_hosted_registry_storage_nfs_directory=/exports
9 openshift_hosted_registry_storage_nfs_options="*(rw,root_squash)"
10 openshift_hosted_registry_storage_volume_name=registry
11 openshift_hosted_registry_storage_volume_size=40Gi
12
13#OAB"s etcd configuration variables
14 openshift_hosted_etcd_storage_kind=nfs
15 openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)"
16 openshift_hosted_etcd_storage_nfs_directory=/exports
17 openshift_hosted_etcd_storage_volume_name=etcd-vol2
18 openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"]
19 openshift_hosted_etcd_storage_volume_size=1G
20 openshift_hosted_etcd_storage_labels={"storage": "etcd"}
7.8 配置离线仓库
1#Modifications Needed for a Disconnected Install 2 oreg_url=registry.lab.example.com/openshift3/ose-${component}:${version} #添加内部仓库
3 openshift_examples_modify_imagestreams=true #修改IS
4 openshift_docker_additional_registries=registry.lab.example.com #内部仓库至docker配置
5 openshift_docker_blocked_registries=registry.access.redhat.com,docker.io #禁止外部官方仓库
6#Image Prefixes
7 openshift_web_console_prefix=registry.lab.example.com/openshift3/ose-
8 openshift_cockpit_deployer_prefix="registry.lab.example.com/openshift3/"
9 openshift_service_catalog_image_prefix=registry.lab.example.com/openshift3/ose-
10 template_service_broker_prefix=registry.lab.example.com/openshift3/ose-
11 ansible_service_broker_image_prefix=registry.lab.example.com/openshift3/ose-
12 ansible_service_broker_etcd_image_prefix=registry.lab.example.com/rhel7/
7.9 设置label
[student@workstation install-run]$ vi inventory 1 …… 2 [nodes]
3 master.lab.example.com
4 node1.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
5 node2.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
7.10 合并并校对Inventory
[student@workstation install-run]$ cat general_vars.txt networking_vars.txt authentication_vars.txt persistence_vars.txt disconnected_vars.txt >> inventory[student@workstation install-run]$ lab install-run grade #本环境提供检查Inventory的脚本[student@workstation install-run]$ cat inventory 1 [student@workstation install-run]$ cat general_vars.txt networking_vars.txt authentication_vars.txt persistence_vars.txt disconnected_vars.txt >> inventory 2 [student@workstation install-run]$ lab install-run grade #本环境提供检查Inventory的脚本
3 [student@workstation install-run]$ cat inventory
4 [workstations]
5 workstation.lab.example.com
6
7 [nfs]
8 services.lab.example.com
9
10 [masters]
11 master.lab.example.com
12
13 [etcd]
14 master.lab.example.com
15
16 [nodes]
17 master.lab.example.com
18 node1.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
19 node2.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
20
21 [OSEv3:children]
22 masters
23 etcd
24 nodes
25 nfs
26
27#Variables needed by classroom host preparation playbooks.
28 [nodes:vars]
29 registry_local=registry.lab.example.com
30 use_overlay2_driver=true
31 insecure_registry=false
32 run_docker_offline=true
33 docker_storage_device=/dev/vdb
34
35
36 [OSEv3:vars]
37#General Cluster Variables
38 openshift_deployment_type=openshift-enterprise
39 openshift_release=v3.9
40 openshift_image_tag=v3.9.14
41 openshift_disable_check=disk_availability,docker_storage,memory_availability
42#OpenShift Networking Variables
43 os_firewall_use_firewalld=true
44 openshift_master_api_port=443
45 openshift_master_console_port=443
46 openshift_master_default_subdomain=apps.lab.example.com
47#Cluster Authentication Variables
48 openshift_master_identity_providers=[{"name": "htpasswd_auth", "login": "true", "challenge": "true", "kind": "HTPasswdPasswordIdentityProvider", "filename": "/etc/origin/master/htpasswd"}]
49 openshift_master_htpasswd_users={"admin":"$apr1$/d1L7fdX$duViLRE.JG012VkZDq8bs0", "developer":"$apr1$rUMMfQfD$J8CEqQK.YenyNwYwKN1lA1"}
50
51#NFS is an unsupported configuration
52 openshift_enable_unsupported_configurations=true
53
54#OCR configuration variables
55 openshift_hosted_registry_storage_kind=nfs
56 openshift_hosted_registry_storage_access_modes=["ReadWriteMany"]
57 openshift_hosted_registry_storage_nfs_directory=/exports
58 openshift_hosted_registry_storage_nfs_options="*(rw,root_squash)"
59 openshift_hosted_registry_storage_volume_name=registry
60 openshift_hosted_registry_storage_volume_size=40Gi
61
62#OAB"s etcd configuration variables
63 openshift_hosted_etcd_storage_kind=nfs
64 openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)"
65 openshift_hosted_etcd_storage_nfs_directory=/exports
66 openshift_hosted_etcd_storage_volume_name=etcd-vol2
67 openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"]
68 openshift_hosted_etcd_storage_volume_size=1G
69 openshift_hosted_etcd_storage_labels={"storage": "etcd"}
70
71#Modifications Needed for a Disconnected Install
72 oreg_url=registry.lab.example.com/openshift3/ose-${component}:${version}
73 openshift_examples_modify_imagestreams=true
74 openshift_docker_additional_registries=registry.lab.example.com
75 openshift_docker_blocked_registries=registry.access.redhat.com,docker.io
76
77#Image Prefixes
78 openshift_web_console_prefix=registry.lab.example.com/openshift3/ose-
79 openshift_cockpit_deployer_prefix="registry.lab.example.com/openshift3/"
80 openshift_service_catalog_image_prefix=registry.lab.example.com/openshift3/ose-
81 template_service_broker_prefix=registry.lab.example.com/openshift3/ose-
82 ansible_service_broker_image_prefix=registry.lab.example.com/openshift3/ose-
83 ansible_service_broker_etcd_image_prefix=registry.lab.example.com/rhel7/
7.11 执行安装剧本
[student@workstation install-run]$ ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/prerequisites.yml#执行准备工作playbook[student@workstation install-run]$ ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/deploy_cluster.yml提示：整个部署log保存至本地目录的ansible.log中。八 验证测试8.1 确认验证说明
要验证OpenShift安装，必须测试和验证所有OpenShift组件。仅仅从示例容器映像启动pod是不够的，因为这并不使用OpenShift builders、deployer、router或内部registry。- 建议通过以下方式完整验证OpenShift:
- 检查所有OpenShift节点状态；
- 检查相应的OpenShift registry和router的pod；
- 使用OpenShift从源代码构建一个应用程序，OpenShift从构建结果生成容器image，并从该映像启动pod；
- 创建一个service，以便可以从内部容器网络和OpenShift节点访问应用程序；
- 创建一个route，以便可以从OpenShift集群外部的计算机访问应用程序。
安装完成后，OpenShift客户端可以使用oc，master节点可以使用oadm命令。master节点的root用户将被配置为云管理员的身份运行OpenShift客户机和管理员命令。一些OpenShift内部服务，如内部仓库和router，默认情况下由安装程序配置。运行oc get nodes和oc get pods命令，以验证安装成功。8.2 登录测试
浏览器访问：https://master.lab.example.com
8.3 验证OpenShift功能
[student@workstation ~]$ oc login -uadmin -predhat https://master.lab.example.com提示：账号权限需要单独授予，安装过程中创建的adminn并没有集群的administration特权。8.4 授予权限
system:admin是唯一一个拥有集群administration权限的账户。master节点的root用户被都为集群的system:admin用户。[root@master ~]# oc whoamisystem:admin[root@master ~]# oc adm policy add-cluster-role-to-user cluster-admin admin #添加admin为集群管理员提示：cluster-admin角色权限非常高，允许管理用户销毁和修改集群资源，必须谨慎使用。8.5 查看节点状态
再次使用命令登录。[student@workstation ~]$ oc login -uadmin -predhat https://master.lab.example.com[student@workstation ~]$ oc get nodesNAME STATUS ROLES AGE VERSIONmaster.lab.example.com Ready master 14h v1.9.1+a0ce1bc657node1.lab.example.com Ready compute 14h v1.9.1+a0ce1bc657node2.lab.example.com Ready compute 14h v1.9.1+a0ce1bc657[student@workstation ~]$ oc get podsNAME READY STATUS RESTARTS AGEdocker-registry-1-4w5tb 1/1 Running 1 14hdocker-registry-1-j7k59 1/1 Running 1 14hregistry-console-1-mtkxc 1/1 Running 1 14hrouter-4-9dfxc 1/1 Running 0 4hrouter-4-kh7th 1/1 Running 0 5h8.6 创建项目
[student@workstation ~]$ oc new-project smoke-test8.7 创建应用
[student@workstation ~]$ oc new-app --name=hello -i php:7.0 http://registry.lab.example.com/php-helloworld[student@workstation ~]$ oc get pods -w #监视pod创建8.8 查看route
[student@workstation ~]$ oc get routesNAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARDhello hello-smoke-test.apps.lab.example.com hello 8080-tcp None8.9 公开服务
[student@workstation ~]$ oc expose service hello #向外部网络公开服务8.10 测试服务
[student@workstation ~]$ curl http://hello-smoke-test.apps.lab.example.comHello, World! php version is 7.0.10[student@workstation ~]$ oc delete project install-post #删除项目8.11 测试developer
[student@workstation ~]$ oc login -u developer #使用redhat密码登录[student@workstation ~]$ oc new-project smoke-test[student@workstation ~]$ oc new-app php:5.6~http://services.lab.example.com/php-helloworld --name hello[student@workstation ~]$ oc logs -f bc/hello #监视构建过程提示：输出表明OpenShift能够从仓库clone代码、并且构建image，同时将新image推入内部仓库。[student@workstation ~]$ oc expose svc hello route "hello" exposed[student@workstation ~]$ oc get routesNAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARDhello hello-smoke-test.apps.lab.example.com hello 8080-tcp None[student@workstation ~]$ curl hello-smoke-test.apps.lab.example.comHello, World! php version is 5.6.25
第一种方法使用快速安装程序，可用于简单的集群设置。
2.1 环境准备
[student@workstation ~]$ lab install-prepare setup #运行准备脚本提示：本环境基于RedHat RH280环境，所有lab命令为环境自动化准备命令，后续不再赘述。2.2 安装Ansible
[student@workstation ~]$ rpm -qa | grep ansible[student@workstation ~]$ sudo yum -y install ansible2.3 验证Ansible
[student@workstation ~]$ cd /home/student/DO280/labs/install-prepare/[student@workstation ~]$ ansible --version[student@workstation install-prepare]$ cat ansible.cfg[student@workstation install-prepare]$ cat inventoryInventory文件解释：Inventory定义了六个主机组:- workstations：为developer节点，即运行playbook的节点；
- nfs：为集群存储提供nfs服务的环境中的vm；
- masters：OpenShift集群中用作master角色的节点；
- etcd：用于OpenShift集群的etcd服务的节点，本环境中使用master节点；
- node：OpenShift集群中的node节点；
- OSEv3：组成OpenShift集群的所有接待，包括master、etcd、node或nfs组中的节点。
2.4 检查节点连通性
[student@workstation install-prepare]$ cat ping.yml1 ---[student@workstation install-prepare]$ ansible-playbook -v ping.yml2 - name: Verify Connectivity
3 hosts: all
4 gather_facts: no
5 tasks:
6 - name: "Test connectivity to machines."
7 shell: "whoami"
8 changed_when: false
2.5 确认yml
[student@workstation install-prepare]$ cat prepare_install.yml 解释：如上yml引入了三个role。docker-storage内容如下，该role定义相关docker的后端存储驱动以及创建docker所需的image存储路径，并最终启动docker。[student@workstation install-prepare]$ cat roles/docker-storage/tasks/main.yml1 ---[student@workstation install-prepare]$ cat roles/docker-storage/templates/docker-storage-setup2 - block:
3 - name: Customize default /etc/sysconfig/docker-storage-setup
4 template:
5 src: docker-storage-setup
6 dest: /etc/sysconfig/docker-storage-setup
7 owner: root
8group: root
9 mode: 0644
10 when: not use_overlay2_driver
11 - name: Customize /etc/sysconfig/docker-storage-setup using overlay2 storage driver
12 template:
13 src: docker-storage-setup-overlay2
14 dest: /etc/sysconfig/docker-storage-setup
15 owner: root
16group: root
17 mode: 0644
18 when: use_overlay2_driver
19 - name: Verify existence of /dev/docker-vg/docker-pool
20 stat:
21path: /dev/docker-vg/docker-pool
22register: p
23 - name: Stop docker
24 service:
25name: docker
26 state: stopped
27 when: p.stat.exists == False
28 - name: Remove loopback docker files
29 file:
30 dest: /var/lib/docker
31 state: absent
32 when: p.stat.exists == False
33 - name: Run docker-storage-setup
34 command: /usr/bin/docker-storage-setup
35 when: p.stat.exists == False
36 - name: Start and enable docker
37 service:
38name: docker
39 state: started
40 when: p.stat.exists == False
41 when: docker_storage_device is defined
42
1 DEVS={{ docker_storage_device }}docker-registry-cert内容如下，该role定义相关docker的使用私有仓库，并且导入了相关crt证书。2 VG=docker-vg
3 SETUP_LVM_THIN_POOL=yes
[student@workstation install-prepare]$ cat roles/docker-registry-cert/tasks/main.yml
1 ---[student@workstation install-prepare]$ cat roles/docker-registry-cert/vars/main.yml2 - name: Enable the Trust
3 shell: update-ca-trust enable
4 - name: Retrieve the certificate
5 fetch:
6 src: "{{ cacert }}"
7 dest: "{{ local_destination }}"
8 delegate_to: "{{ registry_host }}"
9 - name: Copy the certificate
10copy:
11 src: "{{ source }}"
12 dest: "{{ destination }}"
13 owner: root
14group: root
15 mode: 0755
16 - name: Update the Trust
17 shell: update-ca-trust extract
18 - name: Restart Docker
19 service:
20name: docker
21 state: restarted
22
1 registry_host: services.lab.example.comopenshift-node内容如下，该role定义相关安装OpenShift所需的所有依赖包任务。2 cacert: /etc/pki/tls/certs/example.com.crt
3 local_destination: /tmp/
4 source: "/tmp/{{ ansible_fqdn }}/etc/pki/tls/certs/example.com.crt"
5 destination: /etc/pki/ca-trust/source/anchors/example.com.crt
[student@workstation install-prepare]$ ll roles/openshift-node/files/ total 4-rw-r--r--. 1 student student 389 Jul 19 2018 id_rsa.pub[student@workstation install-prepare]$ cat roles/openshift-node/meta/main.yml
1 ---[student@workstation install-prepare]$ cat roles/openshift-node/tasks/main.yml2 dependencies:
3 - { role: docker }
1 ---2 - name: Deploy ssh key to root at all nodes
3 authorized_key:
4user: root
5 key: "{{ item }}"
6 with_file:
7 - id_rsa.pub
8 - name: Install required packages
9 yum:
10name: "{{ item }}"
11 state: latest
12 with_items:
13 - wget
14 - git
15 - net-tools
16 - bind-utils
17 - iptables-services
18 - bridge-utils
19 - bash-completion
20 - kexec-tools
21 - sos
22 - psacct
23 - atomic-openshift-clients
24 - atomic-openshift-utils
25 - atomic-openshift
26
2.6 运行playbook
[student@workstation ~]$ cd /home/student/DO280/labs/install-prepare/[student@workstation install-prepare]$ ansible-playbook prepare_install.yml提示：该准备工作将完成如下操作：- 在每个节点上安装并运行Docker；
- 在每个节点上Docker使用一个逻辑卷存储；
- 每个节点使用自签名证书信任私有Docker仓库；
- 在每个节点上都会安装基本包。
2.7 确认验证
[student@workstation install-prepare]$ for vm in master node1 node2;do echo -e "$vm"
ssh $vm sudo systemctl status docker | head -n3done #验证docker服务[student@workstation install-prepare]$ for vm in master node1 node2;do echo -e "$vm : lvs"
ssh $vm sudo lvsecho -e "$vm : df -h"
ssh $vm sudo df -h | grep vg-dockerdone #查看docker使用的lvm[student@workstation install-prepare]$ for vm in master node1 node2;do echo -e "$vm"
ssh $vm docker pull rhel7:latestdone #测试pull image[student@workstation install-prepare]$ for vm in master node1 node2; do echo -e "$vm"
ssh $vm rpm -qa wget git net-tools bind-utils yum-utils iptables-services bridge-utils bash-completion kexec-tools sos psacct atomic-openshift-utilsdone #检查相关依赖包是否安装成功三 正式安装说明3.1 安装步骤
安装准备完成后正式安装包括四个步骤：- 编写一个目录文件来描述所需的集群特性和体系结构；
- 执行prerequisites.yml的playbook；
- 执行deploy_cluster,yml的playbook；
- 验证安装。
3.2 安装和配置节点
OpenShift Inventory定义了以下主机组。master：对于OpenShift，这是必须的组，定义了OpenShift集群中哪些主机充当master节点；node：对于OpenShift，这是必须的组，它定义了OpenShift集群中哪些主机充当node节点；etcd：[master]部分中列出的所有主机也应属于etcd；nfs：这个组是可选的，应该只包含一个主机。如果Inventory文件中存在特定的变量，OpenShift playbook将在这台机器上安装并配置NFS；OSEv3：这个组包含任何属于OpenShift集群的机器。安装剧本引用这个组来运行在集群全范围内的任务。[student@workstation install-prepare]$ cat inventory说明：- 安装所需版本的OpenShift容器平台；
- 用户使用htpasswd身份验证对集群进行身份验证；
- DNS条目apps.lab.example.com用作OpenShift应用程序的子域；
- NFS存储用于OpenShift etcd服务和OpenShift 内部仓库；
- classroom container registry用作仓库。
变量说明：OpenShift安装变量记录在Inventory的[OSEv3:vars]部分。安装变量用于配置多个OpenShift组件，例如：- 一个内部容器仓库；
- Gluster、Ceph等以便于提供持久性存储；
- 集群日志；
- 自定义集群证书。
3.3 配置OpenShift版本
可通过在[OSEv3:vars]中指定如下配置确定OpenShift所安装的版本：openshift_deployment_type=openshift-enterpriseopenshift_release=v3.9指定OpenShift部署类型，可选值为openshift-enterprise和origin。openshift_image_tag=v3.9.14openshift_disable_check=disk_availability,docker_storage,memory_availability容器化的OpenShift服务使用带有“v3.9.14”标记的图像。这将阻止集群自动升级到更新的容器映像；对于非生产集群，可以禁用对系统需求的检查。3.4 配置验证
OpenShift容器平台身份验证基于OAuth, OAuth提供了一个基于HTTP的APl，用于对交互式和非交互式客户端进行身份验证。OpenShift master运行一个OAuth服务器，OpenShift可以支持多种Provider，这些Provider可以与特定于组织的身份管理产品集成。支持的OpenShift身份验证的Provider:- HTTP Basic，外部单点登录(SSO)系统；
- 使用GitHub和GitLab帐号；
- OpenID连接，使用OpenID-compatible SSO和谷歌帐户；
- OpenStack Keystone v3；
- LDAP v3服务器。
OpenShift安装程序使用默认的安全方法，DenyAllPasswordIdentityProvider是缺省提供程序。使用此Provider，表示只有master主机上的root用户才能使用OpenShift客户端命令和API。3.5 配置htpasswd验证
OpenShift HTPasswdPasswordIdentityProvider根据Apache HTTPD htpasswd程序生成的文件验证用户和密码。htpasswd程序将用户名和密码保存在纯文本文件中，每行一条记录，字段用冒号分隔。密码使用MD5散列。如果将此文件添加或删除用户，或更改用户密码，OpenShift OAuth服务器将自动重新读取该文件。要将OpenShift master配置使用HTPasswdPasswordIdentityProvider，需要配置openshift_master_identity_providers。 1 openshift_master_identity_providers。 2 openshift_master_identity_providers=[{"name": "htpasswd_auth", "login": "true",
3 "challenge": "true", "kind": "HTPasswdPasswordIdentityProvider", #配置后端驱动
4 "filename": "/etc/origin/master/htpasswd"}] #制定master主机上
也支持在配置文件中直接指定初始的用户名和密码。openshift_master_htpasswd_users="{"user1":"$apr1$.NHMsZYc$MdmfWN5DM3q280/W7c51c/","user2":"$apr1$.NHMsZYc$MdmfWN5DM3q280/W7c51c/"}"生产hash密码可参考如下： 1 [student@workstation ~]$ htpasswd -nb admin redhat 2 [student@workstation ~]$ openssl passwd -apr1 redhat
3.6 网络要求
集群节点的通配符DNS条目允许任何新创建的路由自动路由到subdomain的集群。通配符DNS条目必须存在于唯一的子域中，例如apps.mycluster.com，并解析为主机名或集群节点的IP地址。inventory文件中通配符DNS条目是通过变量openshift_master_default_subdomain进行设置 。openshift_master_default_subdomain=apps.mycluster.com3.7 master服务端口
主服务端口openshift_master_api_port变量定义主API的监听端口。缺省端口8443，当master使用SSL时，也可以使用443端口。从而在连接的时候省略端口号。master console端口由openshift_master_console_port变量的值设置，默认端口是8443。master console端口也可以设置为443，从而在连接的时候省略端口号。3.8 防火墙OpenShift节点上的默认防火墙服务是iptables。若要在所有节点上使用firewalld作为防火墙服务，需要将操作系统防火墙使用firewalld变量设置为true，即os_firewall_use_firewalld=true。四 配置持久化存储4.1 持久存储配置
默认情况下，容器数据是临时的，并且在容器被销毁时丢失。Kubernetes持久卷框架为容器请求和使用持久存储提供了一种机制。为了避免数据丢失，这些服务被配置为使用持久卷。OpenShift支持多个插件，使用各种存储技术创建持久卷。可以使用NFS、iSCSI、GlusterFS、Ceph或其他商业云存储。本环境中，OpenShift容器registry和OpenShift Ansible Broker服务被配置为使用NFS持久性存储。提示：生产环境默认OpenShift不支持NFS持久存储集群，要允许NFS在非生产集群上持久存储，需要配置openshift_enable_unsupported_configurations=true。4.2 container仓库
要为OpenShift容器registry配置NFS持久性存储，请将以下内容添加到Inventory文件中： 1 openshift_hosted_registry_storage_kind=nfs 2 openshift_hosted_registry_storage_nfs_directory=/exports
3 openshift_hosted_registry_storage_volume_name=registry
4 openshift_hosted_registry_storage_nfs_options="*(rw,root_squash)"
5 openshift_hosted_registry_storage_volume_size=40G
6 openshift_hosted_registry_storage_access_modes=["ReadWriteMany"]
4.3 OpenShift Ansible Broker
OpenShift Ansible Broker(OAB)是一个容器化的OpenShift服务，部署自己的etcd服务。持久Etcd存储所需的配置与registry所需的配置类似。 1 openshift_hosted_etcd_storage_kind=nfs 2 openshift_hosted_etcd_storage_nfs_directory=/exports
3 openshift_hosted_etcd_storage_volume_name=etcd-vol2
4 openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)"
5 openshift_hosted_etcd_storage_volume_size=1G
6 openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"]
7 openshift_hosted_etcd_storage_labels={"storage": "etcd"}
五 OpenShift其他配置5.1 配置离线本地registry
本环境OpenShift使用容器仓库为registry.lab.example.com，要将集群配置为从内部仓库pull image，需要在Inventory中进行如下配置：
1#Modifications Needed for a Disconnected Install 2 oreg_url=registry.lab.example.com/openshift3/ose-${component}:${version}
3#可访问image仓库的位置，必须以ose-${component}:${version}结尾。
4 openshift_examples_modify_imagestreams=true
5#OpenShift安装了用于部署示例应用程序的模板。这个变量指示playbook修改所有示例的IS，使其指向私有仓库，而不是registry.access.redhat.com。
6 openshift_docker_additional_registries=registry.lab.example.com
7#此变量用于将本地可访问仓库添加到每个节点上的docker配置中。
8 openshift_docker_blocked_registries=registry.access.redhat.com,docker.io
9#此变量用于在OpenShift节点上配置docker的blocked_registries。
1#Image Prefix Modifications 2 openshift_web_console_prefix=registry.lab.example.com/openshift3/oseopenshift_cockpit_deployer_prefix="registry.lab.example.com/openshift3/"
3 openshift_service_catalog_image_prefix=registry.lab.example.com/openshift3/osetemplate_service_broker_prefix=registry.lab.example.com/openshift3/oseansible_service_broker_image_prefix=registry.lab.example.com/openshift3/oseansible_service_broker_etcd_image_prefix=registry.lab.example.com/rhel7/
#通过在容器image名称前面加上registry.lab.example.com以确保OpenShift服务的容器image可以从私有内部仓库下载。5.2 配置NODE labels
节点label是分配给每个节点的任意key/value描述。node label通常用于区分地理数据中心或标识节点上的可用资源的有意义的描述。应用程序可以在其deployment中根据node lables配置一个选择器。如果匹配到，应用程序的pod必须部署在其符合node labels的节点上。使用主机变量openshift_node_tags在Inventory文件中设置节点标签。 1 [nodes] 2 ...output omitted...
3 nodeX.example.com openshift_node_labels="{"zone":"west", "gpu":"true"}"
4 ...output omitted...
如上所示配置给nodeX.example.com配置两个labels，zone=west和gpu=true。OpenShift集群的一个常见架构是区分master、infrastructure node和compute node。在此架构中，infrastructure node承载OpenShift Pod的registry和路由器，而compute node承载来自用户项目的应用程序pod。master节点不承载应用程序或infrastructure pod。可使用 node label 来标识特定节点的角色，通常master node label 为 node-role.kubernetes.io/master=true，infrastructure node label 为 region=infra，compute node label 为 noderole.kubernetes.io/compute=true。 1 [nodes] 2 master.lab.example.com
3 node1.lab.example.com openshift_node_labels="{"region":"infra"}"
4 node2.lab.example.com
提示：如果一个节点设计为同时承载infrastructure 和 application pods，则必须显式定义两个节点标签。[nodes]...nodeX.example.com openshift_node_labels="{"region":"infra", "noderole.kubernetes.io/compute":"true"}"...六 执行剧本6.1 剧本说明
安装OpenShift需要执行prerequisites.yml 和deploy_cluster.yml，由 atomic-openshift-utils 软件包安装。首先执行 prequisites.yml playbook 检查所有主机能够满足OpenShift 的部署，同时尝试修改主机以满足部署需求。然后执行 doploy_cluster.yml playbook 开始正式集群部署6.2 验证OpenShift
部署完成后，可访问：https://master.lab.example.com 进行验证。七 正式安装OpenShift7.1 前置准备
[student@workstation ~]$ lab install-prepare setup[student@workstation ~]$ sudo yum -y install ansible[student@workstation ~]$ cd /home/student/do280-ansible/[student@workstation do280-ansible]$ ansible-playbook playbooks/prepare_install.yml #设置相关环境[student@workstation do280-ansible]$ lab install-run setup[student@workstation do280-ansible]$ cd /home/student/DO280/labs/install-run/7.2 安装atomic
[student@workstation install-run]$ sudo yum -y install atomic-openshift-utils提示：atomic-openshift-utils提供了安装OpenShift所需的Ansible playbook和role。7.3 创建Inventory
[student@workstation install-run]$ cp inventory.initial inventory[student@workstation install-run]$ cat inventory[student@workstation install-run]$ echo -e "[OSEv3:vars]" >> inventory
7.4 配置相关安装版本
1 [student@workstation install-run]$ vi general_vars.txt 2#General Cluster Variables
3 openshift_deployment_type=openshift-enterprise #配置为openshift-enterprise版本
4 openshift_release=v3.9 #配置版本为v3.9
5 openshift_image_tag=v3.9.14
6 openshift_disable_check=disk_availability,docker_storage,memory_availability #禁用check
7.5 设置htpasswd认证
1 [student@workstation install-run]$ openssl passwd -apr1 redhat 2 $apr1$/d1L7fdX$duViLRE.JG012VkZDq8bs0
3 [student@workstation install-run]$ openssl passwd -apr1 redhat
4 $apr1$rUMMfQfD$J8CEqQK.YenyNwYwKN1lA1 #创建两个用户密码都为redhat
5 [student@workstation install-run]$ vi authentication_vars.txt
6#Cluster Authentication Variables
7 openshift_master_identity_providers=[{"name": "htpasswd_auth", "login": "true", "challenge": "true", "kind": "HTPasswdPasswordIdentityProvider", "filename": "/etc/origin/master/htpasswd"}]
8 openshift_master_htpasswd_users={"admin":"$apr1$/d1L7fdX$duViLRE.JG012VkZDq8bs0", "developer":"$apr1$rUMMfQfD$J8CEqQK.YenyNwYwKN1lA1"}
7.6 配置集群网络
1 [student@workstation install-run]$ vi networking_vars.txt 2#OpenShift Networking Variables
3 os_firewall_use_firewalld=true #开启firewall防火墙
4 openshift_master_api_port=443 #启用端口
5 openshift_master_console_port=443 #启用控制端口
6 openshift_master_default_subdomain=apps.lab.example.com #指定subdomain
7.7 配置NFS
1 [student@workstation install-run]$ vi persistence_vars.txt 2#NFS is an unsupported configuration
3 openshift_enable_unsupported_configurations=true
4
5#OCR configuration variables
6 openshift_hosted_registry_storage_kind=nfs
7 openshift_hosted_registry_storage_access_modes=["ReadWriteMany"]
8 openshift_hosted_registry_storage_nfs_directory=/exports
9 openshift_hosted_registry_storage_nfs_options="*(rw,root_squash)"
10 openshift_hosted_registry_storage_volume_name=registry
11 openshift_hosted_registry_storage_volume_size=40Gi
12
13#OAB"s etcd configuration variables
14 openshift_hosted_etcd_storage_kind=nfs
15 openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)"
16 openshift_hosted_etcd_storage_nfs_directory=/exports
17 openshift_hosted_etcd_storage_volume_name=etcd-vol2
18 openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"]
19 openshift_hosted_etcd_storage_volume_size=1G
20 openshift_hosted_etcd_storage_labels={"storage": "etcd"}
7.8 配置离线仓库
1#Modifications Needed for a Disconnected Install 2 oreg_url=registry.lab.example.com/openshift3/ose-${component}:${version} #添加内部仓库
3 openshift_examples_modify_imagestreams=true #修改IS
4 openshift_docker_additional_registries=registry.lab.example.com #内部仓库至docker配置
5 openshift_docker_blocked_registries=registry.access.redhat.com,docker.io #禁止外部官方仓库
6#Image Prefixes
7 openshift_web_console_prefix=registry.lab.example.com/openshift3/ose-
8 openshift_cockpit_deployer_prefix="registry.lab.example.com/openshift3/"
9 openshift_service_catalog_image_prefix=registry.lab.example.com/openshift3/ose-
10 template_service_broker_prefix=registry.lab.example.com/openshift3/ose-
11 ansible_service_broker_image_prefix=registry.lab.example.com/openshift3/ose-
12 ansible_service_broker_etcd_image_prefix=registry.lab.example.com/rhel7/
7.9 设置label
[student@workstation install-run]$ vi inventory 1 …… 2 [nodes]
3 master.lab.example.com
4 node1.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
5 node2.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
7.10 合并并校对Inventory
[student@workstation install-run]$ cat general_vars.txt networking_vars.txt authentication_vars.txt persistence_vars.txt disconnected_vars.txt >> inventory[student@workstation install-run]$ lab install-run grade #本环境提供检查Inventory的脚本[student@workstation install-run]$ cat inventory 1 [student@workstation install-run]$ cat general_vars.txt networking_vars.txt authentication_vars.txt persistence_vars.txt disconnected_vars.txt >> inventory 2 [student@workstation install-run]$ lab install-run grade #本环境提供检查Inventory的脚本
3 [student@workstation install-run]$ cat inventory
4 [workstations]
5 workstation.lab.example.com
6
7 [nfs]
8 services.lab.example.com
9
10 [masters]
11 master.lab.example.com
12
13 [etcd]
14 master.lab.example.com
15
16 [nodes]
17 master.lab.example.com
18 node1.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
19 node2.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
20
21 [OSEv3:children]
22 masters
23 etcd
24 nodes
25 nfs
26
27#Variables needed by classroom host preparation playbooks.
28 [nodes:vars]
29 registry_local=registry.lab.example.com
30 use_overlay2_driver=true
31 insecure_registry=false
32 run_docker_offline=true
33 docker_storage_device=/dev/vdb
34
35
36 [OSEv3:vars]
37#General Cluster Variables
38 openshift_deployment_type=openshift-enterprise
39 openshift_release=v3.9
40 openshift_image_tag=v3.9.14
41 openshift_disable_check=disk_availability,docker_storage,memory_availability
42#OpenShift Networking Variables
43 os_firewall_use_firewalld=true
44 openshift_master_api_port=443
45 openshift_master_console_port=443
46 openshift_master_default_subdomain=apps.lab.example.com
47#Cluster Authentication Variables
48 openshift_master_identity_providers=[{"name": "htpasswd_auth", "login": "true", "challenge": "true", "kind": "HTPasswdPasswordIdentityProvider", "filename": "/etc/origin/master/htpasswd"}]
49 openshift_master_htpasswd_users={"admin":"$apr1$/d1L7fdX$duViLRE.JG012VkZDq8bs0", "developer":"$apr1$rUMMfQfD$J8CEqQK.YenyNwYwKN1lA1"}
50
51#NFS is an unsupported configuration
52 openshift_enable_unsupported_configurations=true
53
54#OCR configuration variables
55 openshift_hosted_registry_storage_kind=nfs
56 openshift_hosted_registry_storage_access_modes=["ReadWriteMany"]
57 openshift_hosted_registry_storage_nfs_directory=/exports
58 openshift_hosted_registry_storage_nfs_options="*(rw,root_squash)"
59 openshift_hosted_registry_storage_volume_name=registry
60 openshift_hosted_registry_storage_volume_size=40Gi
61
62#OAB"s etcd configuration variables
63 openshift_hosted_etcd_storage_kind=nfs
64 openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)"
65 openshift_hosted_etcd_storage_nfs_directory=/exports
66 openshift_hosted_etcd_storage_volume_name=etcd-vol2
67 openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"]
68 openshift_hosted_etcd_storage_volume_size=1G
69 openshift_hosted_etcd_storage_labels={"storage": "etcd"}
70
71#Modifications Needed for a Disconnected Install
72 oreg_url=registry.lab.example.com/openshift3/ose-${component}:${version}
73 openshift_examples_modify_imagestreams=true
74 openshift_docker_additional_registries=registry.lab.example.com
75 openshift_docker_blocked_registries=registry.access.redhat.com,docker.io
76
77#Image Prefixes
78 openshift_web_console_prefix=registry.lab.example.com/openshift3/ose-
79 openshift_cockpit_deployer_prefix="registry.lab.example.com/openshift3/"
80 openshift_service_catalog_image_prefix=registry.lab.example.com/openshift3/ose-
81 template_service_broker_prefix=registry.lab.example.com/openshift3/ose-
82 ansible_service_broker_image_prefix=registry.lab.example.com/openshift3/ose-
83 ansible_service_broker_etcd_image_prefix=registry.lab.example.com/rhel7/
7.11 执行安装剧本
[student@workstation install-run]$ ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/prerequisites.yml#执行准备工作playbook[student@workstation install-run]$ ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/deploy_cluster.yml提示：整个部署log保存至本地目录的ansible.log中。八 验证测试8.1 确认验证说明
要验证OpenShift安装，必须测试和验证所有OpenShift组件。仅仅从示例容器映像启动pod是不够的，因为这并不使用OpenShift builders、deployer、router或内部registry。- 建议通过以下方式完整验证OpenShift:
- 检查所有OpenShift节点状态；
- 检查相应的OpenShift registry和router的pod；
- 使用OpenShift从源代码构建一个应用程序，OpenShift从构建结果生成容器image，并从该映像启动pod；
- 创建一个service，以便可以从内部容器网络和OpenShift节点访问应用程序；
- 创建一个route，以便可以从OpenShift集群外部的计算机访问应用程序。
安装完成后，OpenShift客户端可以使用oc，master节点可以使用oadm命令。master节点的root用户将被配置为云管理员的身份运行OpenShift客户机和管理员命令。一些OpenShift内部服务，如内部仓库和router，默认情况下由安装程序配置。运行oc get nodes和oc get pods命令，以验证安装成功。8.2 登录测试
浏览器访问：https://master.lab.example.com
8.3 验证OpenShift功能
[student@workstation ~]$ oc login -uadmin -predhat https://master.lab.example.com提示：账号权限需要单独授予，安装过程中创建的adminn并没有集群的administration特权。8.4 授予权限
system:admin是唯一一个拥有集群administration权限的账户。master节点的root用户被都为集群的system:admin用户。[root@master ~]# oc whoamisystem:admin[root@master ~]# oc adm policy add-cluster-role-to-user cluster-admin admin #添加admin为集群管理员提示：cluster-admin角色权限非常高，允许管理用户销毁和修改集群资源，必须谨慎使用。8.5 查看节点状态
再次使用命令登录。[student@workstation ~]$ oc login -uadmin -predhat https://master.lab.example.com[student@workstation ~]$ oc get nodesNAME STATUS ROLES AGE VERSIONmaster.lab.example.com Ready master 14h v1.9.1+a0ce1bc657node1.lab.example.com Ready compute 14h v1.9.1+a0ce1bc657node2.lab.example.com Ready compute 14h v1.9.1+a0ce1bc657[student@workstation ~]$ oc get podsNAME READY STATUS RESTARTS AGEdocker-registry-1-4w5tb 1/1 Running 1 14hdocker-registry-1-j7k59 1/1 Running 1 14hregistry-console-1-mtkxc 1/1 Running 1 14hrouter-4-9dfxc 1/1 Running 0 4hrouter-4-kh7th 1/1 Running 0 5h8.6 创建项目
[student@workstation ~]$ oc new-project smoke-test8.7 创建应用
[student@workstation ~]$ oc new-app --name=hello -i php:7.0 http://registry.lab.example.com/php-helloworld[student@workstation ~]$ oc get pods -w #监视pod创建8.8 查看route
[student@workstation ~]$ oc get routesNAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARDhello hello-smoke-test.apps.lab.example.com hello 8080-tcp None8.9 公开服务
[student@workstation ~]$ oc expose service hello #向外部网络公开服务8.10 测试服务
[student@workstation ~]$ curl http://hello-smoke-test.apps.lab.example.comHello, World! php version is 7.0.10[student@workstation ~]$ oc delete project install-post #删除项目8.11 测试developer
[student@workstation ~]$ oc login -u developer #使用redhat密码登录[student@workstation ~]$ oc new-project smoke-test[student@workstation ~]$ oc new-app php:5.6~http://services.lab.example.com/php-helloworld --name hello[student@workstation ~]$ oc logs -f bc/hello #监视构建过程提示：输出表明OpenShift能够从仓库clone代码、并且构建image，同时将新image推入内部仓库。[student@workstation ~]$ oc expose svc hello route "hello" exposed[student@workstation ~]$ oc get routesNAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARDhello hello-smoke-test.apps.lab.example.com hello 8080-tcp None[student@workstation ~]$ curl hello-smoke-test.apps.lab.example.comHello, World! php version is 5.6.25
2 openshift_master_identity_providers=[{"name": "htpasswd_auth", "login": "true",
3 "challenge": "true", "kind": "HTPasswdPasswordIdentityProvider", #配置后端驱动
4 "filename": "/etc/origin/master/htpasswd"}] #制定master主机上
2 [student@workstation ~]$ openssl passwd -apr1 redhat
4.1 持久存储配置
默认情况下，容器数据是临时的，并且在容器被销毁时丢失。Kubernetes持久卷框架为容器请求和使用持久存储提供了一种机制。为了避免数据丢失，这些服务被配置为使用持久卷。OpenShift支持多个插件，使用各种存储技术创建持久卷。可以使用NFS、iSCSI、GlusterFS、Ceph或其他商业云存储。本环境中，OpenShift容器registry和OpenShift Ansible Broker服务被配置为使用NFS持久性存储。提示：生产环境默认OpenShift不支持NFS持久存储集群，要允许NFS在非生产集群上持久存储，需要配置openshift_enable_unsupported_configurations=true。4.2 container仓库
要为OpenShift容器registry配置NFS持久性存储，请将以下内容添加到Inventory文件中：1 openshift_hosted_registry_storage_kind=nfs2 openshift_hosted_registry_storage_nfs_directory=/exports
3 openshift_hosted_registry_storage_volume_name=registry
4 openshift_hosted_registry_storage_nfs_options="*(rw,root_squash)"
5 openshift_hosted_registry_storage_volume_size=40G
6 openshift_hosted_registry_storage_access_modes=["ReadWriteMany"]
4.3 OpenShift Ansible Broker
OpenShift Ansible Broker(OAB)是一个容器化的OpenShift服务，部署自己的etcd服务。持久Etcd存储所需的配置与registry所需的配置类似。1 openshift_hosted_etcd_storage_kind=nfs2 openshift_hosted_etcd_storage_nfs_directory=/exports
3 openshift_hosted_etcd_storage_volume_name=etcd-vol2
4 openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)"
5 openshift_hosted_etcd_storage_volume_size=1G
6 openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"]
7 openshift_hosted_etcd_storage_labels={"storage": "etcd"}
五 OpenShift其他配置5.1 配置离线本地registry
本环境OpenShift使用容器仓库为registry.lab.example.com，要将集群配置为从内部仓库pull image，需要在Inventory中进行如下配置：
1#Modifications Needed for a Disconnected Install 2 oreg_url=registry.lab.example.com/openshift3/ose-${component}:${version}
3#可访问image仓库的位置，必须以ose-${component}:${version}结尾。
4 openshift_examples_modify_imagestreams=true
5#OpenShift安装了用于部署示例应用程序的模板。这个变量指示playbook修改所有示例的IS，使其指向私有仓库，而不是registry.access.redhat.com。
6 openshift_docker_additional_registries=registry.lab.example.com
7#此变量用于将本地可访问仓库添加到每个节点上的docker配置中。
8 openshift_docker_blocked_registries=registry.access.redhat.com,docker.io
9#此变量用于在OpenShift节点上配置docker的blocked_registries。
1#Image Prefix Modifications 2 openshift_web_console_prefix=registry.lab.example.com/openshift3/oseopenshift_cockpit_deployer_prefix="registry.lab.example.com/openshift3/"
3 openshift_service_catalog_image_prefix=registry.lab.example.com/openshift3/osetemplate_service_broker_prefix=registry.lab.example.com/openshift3/oseansible_service_broker_image_prefix=registry.lab.example.com/openshift3/oseansible_service_broker_etcd_image_prefix=registry.lab.example.com/rhel7/
#通过在容器image名称前面加上registry.lab.example.com以确保OpenShift服务的容器image可以从私有内部仓库下载。5.2 配置NODE labels
节点label是分配给每个节点的任意key/value描述。node label通常用于区分地理数据中心或标识节点上的可用资源的有意义的描述。应用程序可以在其deployment中根据node lables配置一个选择器。如果匹配到，应用程序的pod必须部署在其符合node labels的节点上。使用主机变量openshift_node_tags在Inventory文件中设置节点标签。 1 [nodes] 2 ...output omitted...
3 nodeX.example.com openshift_node_labels="{"zone":"west", "gpu":"true"}"
4 ...output omitted...
如上所示配置给nodeX.example.com配置两个labels，zone=west和gpu=true。OpenShift集群的一个常见架构是区分master、infrastructure node和compute node。在此架构中，infrastructure node承载OpenShift Pod的registry和路由器，而compute node承载来自用户项目的应用程序pod。master节点不承载应用程序或infrastructure pod。可使用 node label 来标识特定节点的角色，通常master node label 为 node-role.kubernetes.io/master=true，infrastructure node label 为 region=infra，compute node label 为 noderole.kubernetes.io/compute=true。 1 [nodes] 2 master.lab.example.com
3 node1.lab.example.com openshift_node_labels="{"region":"infra"}"
4 node2.lab.example.com
提示：如果一个节点设计为同时承载infrastructure 和 application pods，则必须显式定义两个节点标签。[nodes]...nodeX.example.com openshift_node_labels="{"region":"infra", "noderole.kubernetes.io/compute":"true"}"...六 执行剧本6.1 剧本说明
安装OpenShift需要执行prerequisites.yml 和deploy_cluster.yml，由 atomic-openshift-utils 软件包安装。首先执行 prequisites.yml playbook 检查所有主机能够满足OpenShift 的部署，同时尝试修改主机以满足部署需求。然后执行 doploy_cluster.yml playbook 开始正式集群部署6.2 验证OpenShift
部署完成后，可访问：https://master.lab.example.com 进行验证。七 正式安装OpenShift7.1 前置准备
[student@workstation ~]$ lab install-prepare setup[student@workstation ~]$ sudo yum -y install ansible[student@workstation ~]$ cd /home/student/do280-ansible/[student@workstation do280-ansible]$ ansible-playbook playbooks/prepare_install.yml #设置相关环境[student@workstation do280-ansible]$ lab install-run setup[student@workstation do280-ansible]$ cd /home/student/DO280/labs/install-run/7.2 安装atomic
[student@workstation install-run]$ sudo yum -y install atomic-openshift-utils提示：atomic-openshift-utils提供了安装OpenShift所需的Ansible playbook和role。7.3 创建Inventory
[student@workstation install-run]$ cp inventory.initial inventory[student@workstation install-run]$ cat inventory[student@workstation install-run]$ echo -e "[OSEv3:vars]" >> inventory
7.4 配置相关安装版本
1 [student@workstation install-run]$ vi general_vars.txt 2#General Cluster Variables
3 openshift_deployment_type=openshift-enterprise #配置为openshift-enterprise版本
4 openshift_release=v3.9 #配置版本为v3.9
5 openshift_image_tag=v3.9.14
6 openshift_disable_check=disk_availability,docker_storage,memory_availability #禁用check
7.5 设置htpasswd认证
1 [student@workstation install-run]$ openssl passwd -apr1 redhat 2 $apr1$/d1L7fdX$duViLRE.JG012VkZDq8bs0
3 [student@workstation install-run]$ openssl passwd -apr1 redhat
4 $apr1$rUMMfQfD$J8CEqQK.YenyNwYwKN1lA1 #创建两个用户密码都为redhat
5 [student@workstation install-run]$ vi authentication_vars.txt
6#Cluster Authentication Variables
7 openshift_master_identity_providers=[{"name": "htpasswd_auth", "login": "true", "challenge": "true", "kind": "HTPasswdPasswordIdentityProvider", "filename": "/etc/origin/master/htpasswd"}]
8 openshift_master_htpasswd_users={"admin":"$apr1$/d1L7fdX$duViLRE.JG012VkZDq8bs0", "developer":"$apr1$rUMMfQfD$J8CEqQK.YenyNwYwKN1lA1"}
7.6 配置集群网络
1 [student@workstation install-run]$ vi networking_vars.txt 2#OpenShift Networking Variables
3 os_firewall_use_firewalld=true #开启firewall防火墙
4 openshift_master_api_port=443 #启用端口
5 openshift_master_console_port=443 #启用控制端口
6 openshift_master_default_subdomain=apps.lab.example.com #指定subdomain
7.7 配置NFS
1 [student@workstation install-run]$ vi persistence_vars.txt 2#NFS is an unsupported configuration
3 openshift_enable_unsupported_configurations=true
4
5#OCR configuration variables
6 openshift_hosted_registry_storage_kind=nfs
7 openshift_hosted_registry_storage_access_modes=["ReadWriteMany"]
8 openshift_hosted_registry_storage_nfs_directory=/exports
9 openshift_hosted_registry_storage_nfs_options="*(rw,root_squash)"
10 openshift_hosted_registry_storage_volume_name=registry
11 openshift_hosted_registry_storage_volume_size=40Gi
12
13#OAB"s etcd configuration variables
14 openshift_hosted_etcd_storage_kind=nfs
15 openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)"
16 openshift_hosted_etcd_storage_nfs_directory=/exports
17 openshift_hosted_etcd_storage_volume_name=etcd-vol2
18 openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"]
19 openshift_hosted_etcd_storage_volume_size=1G
20 openshift_hosted_etcd_storage_labels={"storage": "etcd"}
7.8 配置离线仓库
1#Modifications Needed for a Disconnected Install 2 oreg_url=registry.lab.example.com/openshift3/ose-${component}:${version} #添加内部仓库
3 openshift_examples_modify_imagestreams=true #修改IS
4 openshift_docker_additional_registries=registry.lab.example.com #内部仓库至docker配置
5 openshift_docker_blocked_registries=registry.access.redhat.com,docker.io #禁止外部官方仓库
6#Image Prefixes
7 openshift_web_console_prefix=registry.lab.example.com/openshift3/ose-
8 openshift_cockpit_deployer_prefix="registry.lab.example.com/openshift3/"
9 openshift_service_catalog_image_prefix=registry.lab.example.com/openshift3/ose-
10 template_service_broker_prefix=registry.lab.example.com/openshift3/ose-
11 ansible_service_broker_image_prefix=registry.lab.example.com/openshift3/ose-
12 ansible_service_broker_etcd_image_prefix=registry.lab.example.com/rhel7/
7.9 设置label
[student@workstation install-run]$ vi inventory 1 …… 2 [nodes]
3 master.lab.example.com
4 node1.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
5 node2.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
7.10 合并并校对Inventory
[student@workstation install-run]$ cat general_vars.txt networking_vars.txt authentication_vars.txt persistence_vars.txt disconnected_vars.txt >> inventory[student@workstation install-run]$ lab install-run grade #本环境提供检查Inventory的脚本[student@workstation install-run]$ cat inventory 1 [student@workstation install-run]$ cat general_vars.txt networking_vars.txt authentication_vars.txt persistence_vars.txt disconnected_vars.txt >> inventory 2 [student@workstation install-run]$ lab install-run grade #本环境提供检查Inventory的脚本
3 [student@workstation install-run]$ cat inventory
4 [workstations]
5 workstation.lab.example.com
6
7 [nfs]
8 services.lab.example.com
9
10 [masters]
11 master.lab.example.com
12
13 [etcd]
14 master.lab.example.com
15
16 [nodes]
17 master.lab.example.com
18 node1.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
19 node2.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
20
21 [OSEv3:children]
22 masters
23 etcd
24 nodes
25 nfs
26
27#Variables needed by classroom host preparation playbooks.
28 [nodes:vars]
29 registry_local=registry.lab.example.com
30 use_overlay2_driver=true
31 insecure_registry=false
32 run_docker_offline=true
33 docker_storage_device=/dev/vdb
34
35
36 [OSEv3:vars]
37#General Cluster Variables
38 openshift_deployment_type=openshift-enterprise
39 openshift_release=v3.9
40 openshift_image_tag=v3.9.14
41 openshift_disable_check=disk_availability,docker_storage,memory_availability
42#OpenShift Networking Variables
43 os_firewall_use_firewalld=true
44 openshift_master_api_port=443
45 openshift_master_console_port=443
46 openshift_master_default_subdomain=apps.lab.example.com
47#Cluster Authentication Variables
48 openshift_master_identity_providers=[{"name": "htpasswd_auth", "login": "true", "challenge": "true", "kind": "HTPasswdPasswordIdentityProvider", "filename": "/etc/origin/master/htpasswd"}]
49 openshift_master_htpasswd_users={"admin":"$apr1$/d1L7fdX$duViLRE.JG012VkZDq8bs0", "developer":"$apr1$rUMMfQfD$J8CEqQK.YenyNwYwKN1lA1"}
50
51#NFS is an unsupported configuration
52 openshift_enable_unsupported_configurations=true
53
54#OCR configuration variables
55 openshift_hosted_registry_storage_kind=nfs
56 openshift_hosted_registry_storage_access_modes=["ReadWriteMany"]
57 openshift_hosted_registry_storage_nfs_directory=/exports
58 openshift_hosted_registry_storage_nfs_options="*(rw,root_squash)"
59 openshift_hosted_registry_storage_volume_name=registry
60 openshift_hosted_registry_storage_volume_size=40Gi
61
62#OAB"s etcd configuration variables
63 openshift_hosted_etcd_storage_kind=nfs
64 openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)"
65 openshift_hosted_etcd_storage_nfs_directory=/exports
66 openshift_hosted_etcd_storage_volume_name=etcd-vol2
67 openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"]
68 openshift_hosted_etcd_storage_volume_size=1G
69 openshift_hosted_etcd_storage_labels={"storage": "etcd"}
70
71#Modifications Needed for a Disconnected Install
72 oreg_url=registry.lab.example.com/openshift3/ose-${component}:${version}
73 openshift_examples_modify_imagestreams=true
74 openshift_docker_additional_registries=registry.lab.example.com
75 openshift_docker_blocked_registries=registry.access.redhat.com,docker.io
76
77#Image Prefixes
78 openshift_web_console_prefix=registry.lab.example.com/openshift3/ose-
79 openshift_cockpit_deployer_prefix="registry.lab.example.com/openshift3/"
80 openshift_service_catalog_image_prefix=registry.lab.example.com/openshift3/ose-
81 template_service_broker_prefix=registry.lab.example.com/openshift3/ose-
82 ansible_service_broker_image_prefix=registry.lab.example.com/openshift3/ose-
83 ansible_service_broker_etcd_image_prefix=registry.lab.example.com/rhel7/
7.11 执行安装剧本
[student@workstation install-run]$ ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/prerequisites.yml#执行准备工作playbook[student@workstation install-run]$ ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/deploy_cluster.yml提示：整个部署log保存至本地目录的ansible.log中。八 验证测试8.1 确认验证说明
要验证OpenShift安装，必须测试和验证所有OpenShift组件。仅仅从示例容器映像启动pod是不够的，因为这并不使用OpenShift builders、deployer、router或内部registry。- 建议通过以下方式完整验证OpenShift:
- 检查所有OpenShift节点状态；
- 检查相应的OpenShift registry和router的pod；
- 使用OpenShift从源代码构建一个应用程序，OpenShift从构建结果生成容器image，并从该映像启动pod；
- 创建一个service，以便可以从内部容器网络和OpenShift节点访问应用程序；
- 创建一个route，以便可以从OpenShift集群外部的计算机访问应用程序。
安装完成后，OpenShift客户端可以使用oc，master节点可以使用oadm命令。master节点的root用户将被配置为云管理员的身份运行OpenShift客户机和管理员命令。一些OpenShift内部服务，如内部仓库和router，默认情况下由安装程序配置。运行oc get nodes和oc get pods命令，以验证安装成功。8.2 登录测试
浏览器访问：https://master.lab.example.com
8.3 验证OpenShift功能
[student@workstation ~]$ oc login -uadmin -predhat https://master.lab.example.com提示：账号权限需要单独授予，安装过程中创建的adminn并没有集群的administration特权。8.4 授予权限
system:admin是唯一一个拥有集群administration权限的账户。master节点的root用户被都为集群的system:admin用户。[root@master ~]# oc whoamisystem:admin[root@master ~]# oc adm policy add-cluster-role-to-user cluster-admin admin #添加admin为集群管理员提示：cluster-admin角色权限非常高，允许管理用户销毁和修改集群资源，必须谨慎使用。8.5 查看节点状态
再次使用命令登录。[student@workstation ~]$ oc login -uadmin -predhat https://master.lab.example.com[student@workstation ~]$ oc get nodesNAME STATUS ROLES AGE VERSIONmaster.lab.example.com Ready master 14h v1.9.1+a0ce1bc657node1.lab.example.com Ready compute 14h v1.9.1+a0ce1bc657node2.lab.example.com Ready compute 14h v1.9.1+a0ce1bc657[student@workstation ~]$ oc get podsNAME READY STATUS RESTARTS AGEdocker-registry-1-4w5tb 1/1 Running 1 14hdocker-registry-1-j7k59 1/1 Running 1 14hregistry-console-1-mtkxc 1/1 Running 1 14hrouter-4-9dfxc 1/1 Running 0 4hrouter-4-kh7th 1/1 Running 0 5h8.6 创建项目
[student@workstation ~]$ oc new-project smoke-test8.7 创建应用
[student@workstation ~]$ oc new-app --name=hello -i php:7.0 http://registry.lab.example.com/php-helloworld[student@workstation ~]$ oc get pods -w #监视pod创建8.8 查看route
[student@workstation ~]$ oc get routesNAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARDhello hello-smoke-test.apps.lab.example.com hello 8080-tcp None8.9 公开服务
[student@workstation ~]$ oc expose service hello #向外部网络公开服务8.10 测试服务
[student@workstation ~]$ curl http://hello-smoke-test.apps.lab.example.comHello, World! php version is 7.0.10[student@workstation ~]$ oc delete project install-post #删除项目8.11 测试developer
[student@workstation ~]$ oc login -u developer #使用redhat密码登录[student@workstation ~]$ oc new-project smoke-test[student@workstation ~]$ oc new-app php:5.6~http://services.lab.example.com/php-helloworld --name hello[student@workstation ~]$ oc logs -f bc/hello #监视构建过程提示：输出表明OpenShift能够从仓库clone代码、并且构建image，同时将新image推入内部仓库。[student@workstation ~]$ oc expose svc hello route "hello" exposed[student@workstation ~]$ oc get routesNAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARDhello hello-smoke-test.apps.lab.example.com hello 8080-tcp None[student@workstation ~]$ curl hello-smoke-test.apps.lab.example.comHello, World! php version is 5.6.25
2 oreg_url=registry.lab.example.com/openshift3/ose-${component}:${version}
3#可访问image仓库的位置，必须以ose-${component}:${version}结尾。
4 openshift_examples_modify_imagestreams=true
5#OpenShift安装了用于部署示例应用程序的模板。这个变量指示playbook修改所有示例的IS，使其指向私有仓库，而不是registry.access.redhat.com。
6 openshift_docker_additional_registries=registry.lab.example.com
7#此变量用于将本地可访问仓库添加到每个节点上的docker配置中。
8 openshift_docker_blocked_registries=registry.access.redhat.com,docker.io
9#此变量用于在OpenShift节点上配置docker的blocked_registries。
2 openshift_web_console_prefix=registry.lab.example.com/openshift3/oseopenshift_cockpit_deployer_prefix="registry.lab.example.com/openshift3/"
3 openshift_service_catalog_image_prefix=registry.lab.example.com/openshift3/osetemplate_service_broker_prefix=registry.lab.example.com/openshift3/oseansible_service_broker_image_prefix=registry.lab.example.com/openshift3/oseansible_service_broker_etcd_image_prefix=registry.lab.example.com/rhel7/
2 ...output omitted...
3 nodeX.example.com openshift_node_labels="{"zone":"west", "gpu":"true"}"
4 ...output omitted...
2 master.lab.example.com
3 node1.lab.example.com openshift_node_labels="{"region":"infra"}"
4 node2.lab.example.com
6.1 剧本说明
安装OpenShift需要执行prerequisites.yml 和deploy_cluster.yml，由 atomic-openshift-utils 软件包安装。首先执行 prequisites.yml playbook 检查所有主机能够满足OpenShift 的部署，同时尝试修改主机以满足部署需求。然后执行 doploy_cluster.yml playbook 开始正式集群部署6.2 验证OpenShift
部署完成后，可访问：https://master.lab.example.com 进行验证。七 正式安装OpenShift7.1 前置准备
[student@workstation ~]$ lab install-prepare setup[student@workstation ~]$ sudo yum -y install ansible[student@workstation ~]$ cd /home/student/do280-ansible/[student@workstation do280-ansible]$ ansible-playbook playbooks/prepare_install.yml #设置相关环境[student@workstation do280-ansible]$ lab install-run setup[student@workstation do280-ansible]$ cd /home/student/DO280/labs/install-run/7.2 安装atomic
[student@workstation install-run]$ sudo yum -y install atomic-openshift-utils提示：atomic-openshift-utils提供了安装OpenShift所需的Ansible playbook和role。7.3 创建Inventory
[student@workstation install-run]$ cp inventory.initial inventory[student@workstation install-run]$ cat inventory[student@workstation install-run]$ echo -e "[OSEv3:vars]" >> inventory
7.4 配置相关安装版本
1 [student@workstation install-run]$ vi general_vars.txt 2#General Cluster Variables
3 openshift_deployment_type=openshift-enterprise #配置为openshift-enterprise版本
4 openshift_release=v3.9 #配置版本为v3.9
5 openshift_image_tag=v3.9.14
6 openshift_disable_check=disk_availability,docker_storage,memory_availability #禁用check
7.5 设置htpasswd认证
1 [student@workstation install-run]$ openssl passwd -apr1 redhat 2 $apr1$/d1L7fdX$duViLRE.JG012VkZDq8bs0
3 [student@workstation install-run]$ openssl passwd -apr1 redhat
4 $apr1$rUMMfQfD$J8CEqQK.YenyNwYwKN1lA1 #创建两个用户密码都为redhat
5 [student@workstation install-run]$ vi authentication_vars.txt
6#Cluster Authentication Variables
7 openshift_master_identity_providers=[{"name": "htpasswd_auth", "login": "true", "challenge": "true", "kind": "HTPasswdPasswordIdentityProvider", "filename": "/etc/origin/master/htpasswd"}]
8 openshift_master_htpasswd_users={"admin":"$apr1$/d1L7fdX$duViLRE.JG012VkZDq8bs0", "developer":"$apr1$rUMMfQfD$J8CEqQK.YenyNwYwKN1lA1"}
7.6 配置集群网络
1 [student@workstation install-run]$ vi networking_vars.txt 2#OpenShift Networking Variables
3 os_firewall_use_firewalld=true #开启firewall防火墙
4 openshift_master_api_port=443 #启用端口
5 openshift_master_console_port=443 #启用控制端口
6 openshift_master_default_subdomain=apps.lab.example.com #指定subdomain
7.7 配置NFS
1 [student@workstation install-run]$ vi persistence_vars.txt 2#NFS is an unsupported configuration
3 openshift_enable_unsupported_configurations=true
4
5#OCR configuration variables
6 openshift_hosted_registry_storage_kind=nfs
7 openshift_hosted_registry_storage_access_modes=["ReadWriteMany"]
8 openshift_hosted_registry_storage_nfs_directory=/exports
9 openshift_hosted_registry_storage_nfs_options="*(rw,root_squash)"
10 openshift_hosted_registry_storage_volume_name=registry
11 openshift_hosted_registry_storage_volume_size=40Gi
12
13#OAB"s etcd configuration variables
14 openshift_hosted_etcd_storage_kind=nfs
15 openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)"
16 openshift_hosted_etcd_storage_nfs_directory=/exports
17 openshift_hosted_etcd_storage_volume_name=etcd-vol2
18 openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"]
19 openshift_hosted_etcd_storage_volume_size=1G
20 openshift_hosted_etcd_storage_labels={"storage": "etcd"}
7.8 配置离线仓库
1#Modifications Needed for a Disconnected Install 2 oreg_url=registry.lab.example.com/openshift3/ose-${component}:${version} #添加内部仓库
3 openshift_examples_modify_imagestreams=true #修改IS
4 openshift_docker_additional_registries=registry.lab.example.com #内部仓库至docker配置
5 openshift_docker_blocked_registries=registry.access.redhat.com,docker.io #禁止外部官方仓库
6#Image Prefixes
7 openshift_web_console_prefix=registry.lab.example.com/openshift3/ose-
8 openshift_cockpit_deployer_prefix="registry.lab.example.com/openshift3/"
9 openshift_service_catalog_image_prefix=registry.lab.example.com/openshift3/ose-
10 template_service_broker_prefix=registry.lab.example.com/openshift3/ose-
11 ansible_service_broker_image_prefix=registry.lab.example.com/openshift3/ose-
12 ansible_service_broker_etcd_image_prefix=registry.lab.example.com/rhel7/
7.9 设置label
[student@workstation install-run]$ vi inventory 1 …… 2 [nodes]
3 master.lab.example.com
4 node1.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
5 node2.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
7.10 合并并校对Inventory
[student@workstation install-run]$ cat general_vars.txt networking_vars.txt authentication_vars.txt persistence_vars.txt disconnected_vars.txt >> inventory[student@workstation install-run]$ lab install-run grade #本环境提供检查Inventory的脚本[student@workstation install-run]$ cat inventory 1 [student@workstation install-run]$ cat general_vars.txt networking_vars.txt authentication_vars.txt persistence_vars.txt disconnected_vars.txt >> inventory 2 [student@workstation install-run]$ lab install-run grade #本环境提供检查Inventory的脚本
3 [student@workstation install-run]$ cat inventory
4 [workstations]
5 workstation.lab.example.com
6
7 [nfs]
8 services.lab.example.com
9
10 [masters]
11 master.lab.example.com
12
13 [etcd]
14 master.lab.example.com
15
16 [nodes]
17 master.lab.example.com
18 node1.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
19 node2.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
20
21 [OSEv3:children]
22 masters
23 etcd
24 nodes
25 nfs
26
27#Variables needed by classroom host preparation playbooks.
28 [nodes:vars]
29 registry_local=registry.lab.example.com
30 use_overlay2_driver=true
31 insecure_registry=false
32 run_docker_offline=true
33 docker_storage_device=/dev/vdb
34
35
36 [OSEv3:vars]
37#General Cluster Variables
38 openshift_deployment_type=openshift-enterprise
39 openshift_release=v3.9
40 openshift_image_tag=v3.9.14
41 openshift_disable_check=disk_availability,docker_storage,memory_availability
42#OpenShift Networking Variables
43 os_firewall_use_firewalld=true
44 openshift_master_api_port=443
45 openshift_master_console_port=443
46 openshift_master_default_subdomain=apps.lab.example.com
47#Cluster Authentication Variables
48 openshift_master_identity_providers=[{"name": "htpasswd_auth", "login": "true", "challenge": "true", "kind": "HTPasswdPasswordIdentityProvider", "filename": "/etc/origin/master/htpasswd"}]
49 openshift_master_htpasswd_users={"admin":"$apr1$/d1L7fdX$duViLRE.JG012VkZDq8bs0", "developer":"$apr1$rUMMfQfD$J8CEqQK.YenyNwYwKN1lA1"}
50
51#NFS is an unsupported configuration
52 openshift_enable_unsupported_configurations=true
53
54#OCR configuration variables
55 openshift_hosted_registry_storage_kind=nfs
56 openshift_hosted_registry_storage_access_modes=["ReadWriteMany"]
57 openshift_hosted_registry_storage_nfs_directory=/exports
58 openshift_hosted_registry_storage_nfs_options="*(rw,root_squash)"
59 openshift_hosted_registry_storage_volume_name=registry
60 openshift_hosted_registry_storage_volume_size=40Gi
61
62#OAB"s etcd configuration variables
63 openshift_hosted_etcd_storage_kind=nfs
64 openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)"
65 openshift_hosted_etcd_storage_nfs_directory=/exports
66 openshift_hosted_etcd_storage_volume_name=etcd-vol2
67 openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"]
68 openshift_hosted_etcd_storage_volume_size=1G
69 openshift_hosted_etcd_storage_labels={"storage": "etcd"}
70
71#Modifications Needed for a Disconnected Install
72 oreg_url=registry.lab.example.com/openshift3/ose-${component}:${version}
73 openshift_examples_modify_imagestreams=true
74 openshift_docker_additional_registries=registry.lab.example.com
75 openshift_docker_blocked_registries=registry.access.redhat.com,docker.io
76
77#Image Prefixes
78 openshift_web_console_prefix=registry.lab.example.com/openshift3/ose-
79 openshift_cockpit_deployer_prefix="registry.lab.example.com/openshift3/"
80 openshift_service_catalog_image_prefix=registry.lab.example.com/openshift3/ose-
81 template_service_broker_prefix=registry.lab.example.com/openshift3/ose-
82 ansible_service_broker_image_prefix=registry.lab.example.com/openshift3/ose-
83 ansible_service_broker_etcd_image_prefix=registry.lab.example.com/rhel7/
7.11 执行安装剧本
[student@workstation install-run]$ ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/prerequisites.yml#执行准备工作playbook[student@workstation install-run]$ ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/deploy_cluster.yml提示：整个部署log保存至本地目录的ansible.log中。八 验证测试8.1 确认验证说明
要验证OpenShift安装，必须测试和验证所有OpenShift组件。仅仅从示例容器映像启动pod是不够的，因为这并不使用OpenShift builders、deployer、router或内部registry。- 建议通过以下方式完整验证OpenShift:
- 检查所有OpenShift节点状态；
- 检查相应的OpenShift registry和router的pod；
- 使用OpenShift从源代码构建一个应用程序，OpenShift从构建结果生成容器image，并从该映像启动pod；
- 创建一个service，以便可以从内部容器网络和OpenShift节点访问应用程序；
- 创建一个route，以便可以从OpenShift集群外部的计算机访问应用程序。
安装完成后，OpenShift客户端可以使用oc，master节点可以使用oadm命令。master节点的root用户将被配置为云管理员的身份运行OpenShift客户机和管理员命令。一些OpenShift内部服务，如内部仓库和router，默认情况下由安装程序配置。运行oc get nodes和oc get pods命令，以验证安装成功。8.2 登录测试
浏览器访问：https://master.lab.example.com
8.3 验证OpenShift功能
[student@workstation ~]$ oc login -uadmin -predhat https://master.lab.example.com提示：账号权限需要单独授予，安装过程中创建的adminn并没有集群的administration特权。8.4 授予权限
system:admin是唯一一个拥有集群administration权限的账户。master节点的root用户被都为集群的system:admin用户。[root@master ~]# oc whoamisystem:admin[root@master ~]# oc adm policy add-cluster-role-to-user cluster-admin admin #添加admin为集群管理员提示：cluster-admin角色权限非常高，允许管理用户销毁和修改集群资源，必须谨慎使用。8.5 查看节点状态
再次使用命令登录。[student@workstation ~]$ oc login -uadmin -predhat https://master.lab.example.com[student@workstation ~]$ oc get nodesNAME STATUS ROLES AGE VERSIONmaster.lab.example.com Ready master 14h v1.9.1+a0ce1bc657node1.lab.example.com Ready compute 14h v1.9.1+a0ce1bc657node2.lab.example.com Ready compute 14h v1.9.1+a0ce1bc657[student@workstation ~]$ oc get podsNAME READY STATUS RESTARTS AGEdocker-registry-1-4w5tb 1/1 Running 1 14hdocker-registry-1-j7k59 1/1 Running 1 14hregistry-console-1-mtkxc 1/1 Running 1 14hrouter-4-9dfxc 1/1 Running 0 4hrouter-4-kh7th 1/1 Running 0 5h8.6 创建项目
[student@workstation ~]$ oc new-project smoke-test8.7 创建应用
[student@workstation ~]$ oc new-app --name=hello -i php:7.0 http://registry.lab.example.com/php-helloworld[student@workstation ~]$ oc get pods -w #监视pod创建8.8 查看route
[student@workstation ~]$ oc get routesNAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARDhello hello-smoke-test.apps.lab.example.com hello 8080-tcp None8.9 公开服务
[student@workstation ~]$ oc expose service hello #向外部网络公开服务8.10 测试服务
[student@workstation ~]$ curl http://hello-smoke-test.apps.lab.example.comHello, World! php version is 7.0.10[student@workstation ~]$ oc delete project install-post #删除项目8.11 测试developer
[student@workstation ~]$ oc login -u developer #使用redhat密码登录[student@workstation ~]$ oc new-project smoke-test[student@workstation ~]$ oc new-app php:5.6~http://services.lab.example.com/php-helloworld --name hello[student@workstation ~]$ oc logs -f bc/hello #监视构建过程提示：输出表明OpenShift能够从仓库clone代码、并且构建image，同时将新image推入内部仓库。[student@workstation ~]$ oc expose svc hello route "hello" exposed[student@workstation ~]$ oc get routesNAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARDhello hello-smoke-test.apps.lab.example.com hello 8080-tcp None[student@workstation ~]$ curl hello-smoke-test.apps.lab.example.comHello, World! php version is 5.6.25
2#General Cluster Variables
3 openshift_deployment_type=openshift-enterprise #配置为openshift-enterprise版本
4 openshift_release=v3.9 #配置版本为v3.9
5 openshift_image_tag=v3.9.14
6 openshift_disable_check=disk_availability,docker_storage,memory_availability #禁用check
2 $apr1$/d1L7fdX$duViLRE.JG012VkZDq8bs0
3 [student@workstation install-run]$ openssl passwd -apr1 redhat
4 $apr1$rUMMfQfD$J8CEqQK.YenyNwYwKN1lA1 #创建两个用户密码都为redhat
5 [student@workstation install-run]$ vi authentication_vars.txt
6#Cluster Authentication Variables
7 openshift_master_identity_providers=[{"name": "htpasswd_auth", "login": "true", "challenge": "true", "kind": "HTPasswdPasswordIdentityProvider", "filename": "/etc/origin/master/htpasswd"}]
8 openshift_master_htpasswd_users={"admin":"$apr1$/d1L7fdX$duViLRE.JG012VkZDq8bs0", "developer":"$apr1$rUMMfQfD$J8CEqQK.YenyNwYwKN1lA1"}
2#OpenShift Networking Variables
3 os_firewall_use_firewalld=true #开启firewall防火墙
4 openshift_master_api_port=443 #启用端口
5 openshift_master_console_port=443 #启用控制端口
6 openshift_master_default_subdomain=apps.lab.example.com #指定subdomain
2#NFS is an unsupported configuration
3 openshift_enable_unsupported_configurations=true
4
5#OCR configuration variables
6 openshift_hosted_registry_storage_kind=nfs
7 openshift_hosted_registry_storage_access_modes=["ReadWriteMany"]
8 openshift_hosted_registry_storage_nfs_directory=/exports
9 openshift_hosted_registry_storage_nfs_options="*(rw,root_squash)"
10 openshift_hosted_registry_storage_volume_name=registry
11 openshift_hosted_registry_storage_volume_size=40Gi
12
13#OAB"s etcd configuration variables
14 openshift_hosted_etcd_storage_kind=nfs
15 openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)"
16 openshift_hosted_etcd_storage_nfs_directory=/exports
17 openshift_hosted_etcd_storage_volume_name=etcd-vol2
18 openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"]
19 openshift_hosted_etcd_storage_volume_size=1G
20 openshift_hosted_etcd_storage_labels={"storage": "etcd"}
2 oreg_url=registry.lab.example.com/openshift3/ose-${component}:${version} #添加内部仓库
3 openshift_examples_modify_imagestreams=true #修改IS
4 openshift_docker_additional_registries=registry.lab.example.com #内部仓库至docker配置
5 openshift_docker_blocked_registries=registry.access.redhat.com,docker.io #禁止外部官方仓库
6#Image Prefixes
7 openshift_web_console_prefix=registry.lab.example.com/openshift3/ose-
8 openshift_cockpit_deployer_prefix="registry.lab.example.com/openshift3/"
9 openshift_service_catalog_image_prefix=registry.lab.example.com/openshift3/ose-
10 template_service_broker_prefix=registry.lab.example.com/openshift3/ose-
11 ansible_service_broker_image_prefix=registry.lab.example.com/openshift3/ose-
12 ansible_service_broker_etcd_image_prefix=registry.lab.example.com/rhel7/
2 [nodes]
3 master.lab.example.com
4 node1.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
5 node2.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
2 [student@workstation install-run]$ lab install-run grade #本环境提供检查Inventory的脚本
3 [student@workstation install-run]$ cat inventory
4 [workstations]
5 workstation.lab.example.com
6
7 [nfs]
8 services.lab.example.com
9
10 [masters]
11 master.lab.example.com
12
13 [etcd]
14 master.lab.example.com
15
16 [nodes]
17 master.lab.example.com
18 node1.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
19 node2.lab.example.com openshift_node_labels="{"region":"infra", "node-role.kubernetes.io/compute":"true"}"
20
21 [OSEv3:children]
22 masters
23 etcd
24 nodes
25 nfs
26
27#Variables needed by classroom host preparation playbooks.
28 [nodes:vars]
29 registry_local=registry.lab.example.com
30 use_overlay2_driver=true
31 insecure_registry=false
32 run_docker_offline=true
33 docker_storage_device=/dev/vdb
34
35
36 [OSEv3:vars]
37#General Cluster Variables
38 openshift_deployment_type=openshift-enterprise
39 openshift_release=v3.9
40 openshift_image_tag=v3.9.14
41 openshift_disable_check=disk_availability,docker_storage,memory_availability
42#OpenShift Networking Variables
43 os_firewall_use_firewalld=true
44 openshift_master_api_port=443
45 openshift_master_console_port=443
46 openshift_master_default_subdomain=apps.lab.example.com
47#Cluster Authentication Variables
48 openshift_master_identity_providers=[{"name": "htpasswd_auth", "login": "true", "challenge": "true", "kind": "HTPasswdPasswordIdentityProvider", "filename": "/etc/origin/master/htpasswd"}]
49 openshift_master_htpasswd_users={"admin":"$apr1$/d1L7fdX$duViLRE.JG012VkZDq8bs0", "developer":"$apr1$rUMMfQfD$J8CEqQK.YenyNwYwKN1lA1"}
50
51#NFS is an unsupported configuration
52 openshift_enable_unsupported_configurations=true
53
54#OCR configuration variables
55 openshift_hosted_registry_storage_kind=nfs
56 openshift_hosted_registry_storage_access_modes=["ReadWriteMany"]
57 openshift_hosted_registry_storage_nfs_directory=/exports
58 openshift_hosted_registry_storage_nfs_options="*(rw,root_squash)"
59 openshift_hosted_registry_storage_volume_name=registry
60 openshift_hosted_registry_storage_volume_size=40Gi
61
62#OAB"s etcd configuration variables
63 openshift_hosted_etcd_storage_kind=nfs
64 openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)"
65 openshift_hosted_etcd_storage_nfs_directory=/exports
66 openshift_hosted_etcd_storage_volume_name=etcd-vol2
67 openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"]
68 openshift_hosted_etcd_storage_volume_size=1G
69 openshift_hosted_etcd_storage_labels={"storage": "etcd"}
70
71#Modifications Needed for a Disconnected Install
72 oreg_url=registry.lab.example.com/openshift3/ose-${component}:${version}
73 openshift_examples_modify_imagestreams=true
74 openshift_docker_additional_registries=registry.lab.example.com
75 openshift_docker_blocked_registries=registry.access.redhat.com,docker.io
76
77#Image Prefixes
78 openshift_web_console_prefix=registry.lab.example.com/openshift3/ose-
79 openshift_cockpit_deployer_prefix="registry.lab.example.com/openshift3/"
80 openshift_service_catalog_image_prefix=registry.lab.example.com/openshift3/ose-
81 template_service_broker_prefix=registry.lab.example.com/openshift3/ose-
82 ansible_service_broker_image_prefix=registry.lab.example.com/openshift3/ose-
83 ansible_service_broker_etcd_image_prefix=registry.lab.example.com/rhel7/
8.1 确认验证说明
要验证OpenShift安装，必须测试和验证所有OpenShift组件。仅仅从示例容器映像启动pod是不够的，因为这并不使用OpenShift builders、deployer、router或内部registry。- 建议通过以下方式完整验证OpenShift:
- 检查所有OpenShift节点状态；
- 检查相应的OpenShift registry和router的pod；
- 使用OpenShift从源代码构建一个应用程序，OpenShift从构建结果生成容器image，并从该映像启动pod；
- 创建一个service，以便可以从内部容器网络和OpenShift节点访问应用程序；
- 创建一个route，以便可以从OpenShift集群外部的计算机访问应用程序。
8.2 登录测试
浏览器访问：https://master.lab.example.com8.3 验证OpenShift功能
[student@workstation ~]$ oc login -uadmin -predhat https://master.lab.example.com提示：账号权限需要单独授予，安装过程中创建的adminn并没有集群的administration特权。8.4 授予权限
system:admin是唯一一个拥有集群administration权限的账户。master节点的root用户被都为集群的system:admin用户。[root@master ~]# oc whoamisystem:admin[root@master ~]# oc adm policy add-cluster-role-to-user cluster-admin admin #添加admin为集群管理员提示：cluster-admin角色权限非常高，允许管理用户销毁和修改集群资源，必须谨慎使用。8.5 查看节点状态
再次使用命令登录。[student@workstation ~]$ oc login -uadmin -predhat https://master.lab.example.com[student@workstation ~]$ oc get nodesNAME STATUS ROLES AGE VERSIONmaster.lab.example.com Ready master 14h v1.9.1+a0ce1bc657node1.lab.example.com Ready compute 14h v1.9.1+a0ce1bc657node2.lab.example.com Ready compute 14h v1.9.1+a0ce1bc657[student@workstation ~]$ oc get podsNAME READY STATUS RESTARTS AGEdocker-registry-1-4w5tb 1/1 Running 1 14hdocker-registry-1-j7k59 1/1 Running 1 14hregistry-console-1-mtkxc 1/1 Running 1 14hrouter-4-9dfxc 1/1 Running 0 4hrouter-4-kh7th 1/1 Running 0 5h8.6 创建项目
[student@workstation ~]$ oc new-project smoke-test8.7 创建应用
[student@workstation ~]$ oc new-app --name=hello -i php:7.0 http://registry.lab.example.com/php-helloworld[student@workstation ~]$ oc get pods -w #监视pod创建8.8 查看route
[student@workstation ~]$ oc get routesNAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARDhello hello-smoke-test.apps.lab.example.com hello 8080-tcp None8.9 公开服务
[student@workstation ~]$ oc expose service hello #向外部网络公开服务8.10 测试服务
[student@workstation ~]$ curl http://hello-smoke-test.apps.lab.example.comHello, World! php version is 7.0.10[student@workstation ~]$ oc delete project install-post #删除项目8.11 测试developer
[student@workstation ~]$ oc login -u developer #使用redhat密码登录[student@workstation ~]$ oc new-project smoke-test[student@workstation ~]$ oc new-app php:5.6~http://services.lab.example.com/php-helloworld --name hello[student@workstation ~]$ oc logs -f bc/hello #监视构建过程提示：输出表明OpenShift能够从仓库clone代码、并且构建image，同时将新image推入内部仓库。[student@workstation ~]$ oc expose svc hello route "hello" exposed[student@workstation ~]$ oc get routesNAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARDhello hello-smoke-test.apps.lab.example.com hello 8080-tcp None[student@workstation ~]$ curl hello-smoke-test.apps.lab.example.comHello, World! php version is 5.6.25原文链接：https://www.cnblogs.com/itzgr/archive/2020/06/19/13161796.html
以上是 002.OpenShift安装与部署 的全部内容， 来源链接： utcz.com/z/517629.html
