RBAC
基于角色（Role）的访问控制（RBAC）是一种基于组织中用户的角色来调节控制对计算机或网络资源的访问的方法。
RBAC 鉴权机制使用 rbac.authorization.k8s.io API 组来驱动鉴权决定， 允许你通过 Kubernetes API 动态配置策略
要启用 RBAC，在启动 API 服务器时将 --authorization-mode 参数设置为一个逗号分隔的列表并确保其中包含 RBAC。
Role 和 ClusterRole
RBAC 的 Role 或 ClusterRole 中包含一组代表相关权限的规则。 这些权限是纯粹累加的（不存在拒绝某操作的规则）, Role 总是用来在某个命名空间内设置访问权限； 在你创建 Role 时，你必须指定该 Role 所属的命名空间。与之相对，ClusterRole 则是一个集群作用域的资源。这两种资源的名字不同（Role 和 ClusterRole） 是因为 Kubernetes 对象要么是命名空间作用域的，要么是集群作用域的，不可两者兼具。
role示例
下面是一个位于 "default" 名字空间的 Role 的示例，可用来授予对 pods 的读访问权限：
$ cat > role-simple.yaml << EOF apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: namespace: default name: pod-reader rules: - apiGroups: [""] # "" 标明 core API 组 resources: ["pods"] verbs: ["get", "watch", "list"] EOF ClusterRole示例
ClusterRole 可以和 Role 相同完成授权。 因为 ClusterRole 属于集群范围，所以它也可以为以下资源授予访问权限：
- 集群范围资源（比如节点（Node））
- 跨命名空间访问的命名空间作用域的资源（如 Pod）
比如，你可以使用 ClusterRole 来允许某特定用户执行 kubectl get pods --all-namespaces
下面是一个 ClusterRole 的示例，可用来为任一特定命名空间中的 Secret 授予读访问权限， 或者跨命名空间的访问权限（取决于该角色是如何绑定的）：
$ cat > cluster-role-simple.yaml << EOF apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: # "namespace" 被忽略，因为 ClusterRoles 不受命名空间限制 name: secret-reader rules: - apiGroups: [""] # 在 HTTP 层面，用来访问 Secret 资源的名称为 "secrets" resources: ["secrets"] verbs: ["get", "watch", "list"] EOF RoleBinding 和 ClusterRoleBinding
角色绑定（Role Binding）是将角色中定义的权限赋予一个或者一组用户。 它包含若干 主体（用户、组或服务账户）的列表和对这些主体所获得的角色的引用。 RoleBinding 在指定的命名空间中执行授权，而 ClusterRoleBinding 在集群范围执行授权。
一个 RoleBinding 可以引用同一的命名空间中的任何 Role。 或者，一个 RoleBinding 可以引用某 ClusterRole 并将该 ClusterRole 绑定到 RoleBinding 所在的命名空间。 如果你希望将某 ClusterRole 绑定到集群中所有命名空间，你要使用 ClusterRoleBinding。
RoleBinding 示例
下面的例子中的 RoleBinding 将 "pod-reader" Role 授予在 "default" 命名空间中的用户 "jane"。 这样，用户 "jane" 就具有了读取 "default" 命名空间中 pods 的权限。
$ cat > rolebinding-simple.yaml << EOF apiVersion: rbac.authorization.k8s.io/v1。 # 此角色绑定允许 "jane" 读取 "default" 命名空间中的 Pod # 你需要在该命名空间中有一个名为 “pod-reader” 的 Role kind: RoleBinding metadata: name: read-pods namespace: default subjects: # 你可以指定不止一个“subject（主体）” - kind: User name: jane # "name" 是区分大小写的 apiGroup: rbac.authorization.k8s.io roleRef: # "roleRef" 指定与某 Role 或 ClusterRole 的绑定关系 kind: Role # 此字段必须是 Role 或 ClusterRole name: pod-reader # 此字段必须与你要绑定的 Role 或 ClusterRole 的名称匹配 apiGroup: rbac.authorization.k8s.io EOF RoleBinding 也可以引用 ClusterRole，以将对应 ClusterRole 中定义的访问权限授予 RoleBinding 所在命名空间的资源。这种引用使得你可以跨整个集群定义一组通用的角色，之后在多个命名空间中复用。
尽管下面的 RoleBinding 引用的是一个 ClusterRole，"dave"（这里的主体， 区分大小写）只能访问 "development" 命名空间中的 Secrets 对象，因为 RoleBinding 所在的命名空间（由其 metadata 决定）是 "development"。
$ cat > olebinding-clusterrole-simple.yaml << EOF apiVersion: rbac.authorization.k8s.io/v1 # 此角色绑定使得用户 "dave" 能够读取 "development" 命名空间中的 Secrets # 你需要一个名为 "secret-reader" 的 ClusterRole kind: RoleBinding metadata: name: read-secrets # RoleBinding 的命名空间决定了访问权限的授予范围。 # 这里隐含授权仅在 "development" 命名空间内的访问权限。 namespace: development subjects: - kind: User name: dave # 'name' 是区分大小写的 apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: secret-reader apiGroup: rbac.authorization.k8s.io EOF ClusterRoleBinding示例
要跨整个集群完成访问权限的授予，你可以使用一个 ClusterRoleBinding。 下面的 ClusterRoleBinding 允许 "manager" 组内的所有用户访问任何命名空间中的 Secrets。
cat > clusterrolebinding.yaml << EOF apiVersion: rbac.authorization.k8s.io/v1 # 此集群角色绑定允许 “manager” 组中的任何人访问任何名字空间中的 Secret 资源 kind: ClusterRoleBinding metadata: name: read-secrets-global subjects: - kind: Group name: manager # 'name' 是区分大小写的 apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: secret-reader apiGroup: rbac.authorization.k8s.io EOF kubernetes RBAC鉴权实战
一、创建普通用户，并使用kubectl工具
目标：Linux下建立一个新的普通用户(Normal User)，k8s集群创建 2 个新的 namespace，然后把新的用户设置到其中一个 namespace 当中，让用户只能在一个namespace 中操作。
# 在k8s集群下创建user1和user2 namespce $ kubectl create ns user1 $ kubectl create ns user2 # 创建user1用户并切换到用户家目录下(当然也可以不需要创建用户，可以通过k8s来切换use-contexts来实现) $ useradd -m -d /home/user1 -s /bin/bash user1 $ passwd user1 $ cd /home/user1/ # 创建用户私钥 $ openssl genrsa -out user1.key 2048 # 生成一个待签名文件(user1.csr)，注意 O=user代表的是它的组，而不是 namespace。 $ openssl req -new -key user1.key -out user1.csr -subj "/CN=user1/O=user" # 用 k8s 的 ca 文件来签名这个 user1.csr, 最终产生一个有效期为 3600 天的证书文件。 $ openssl x509 -req -in user1.csr -CA /opt/kubelw/cfssl/ca.pem -CAkey /opt/kubelw/cfssl/ca-key.pem -CAcreateserial -out user1.crt -days 3600 注意：/opt/kubelw/cfssl/ca.pem 、/opt/kubelw/cfssl/ca-key.pem 根据自身集群实际路径填写（多数情况下是在 /etc/kubernetes/pki/路径下）。 $ cat /opt/kubelw/cfssl/ca.pem -----BEGIN CERTIFICATE----- MIIDxjCCAq6gAwIBAgIUFJFR4wlmMYvYMzfVcmfL+K2VLDowDQYJKoZIhvcNAQEL BQAwaTELMAkGA1UEBhMCQ04xEjAQBgNVBAgTCUdVQU5HRE9ORzESMBAGA1UEBxMJ R1VBTkdaSE9VMQwwCgYDVQQKEwNrOHMxDzANBgNVBAsTBnN5c3RlbTETMBEGA1UE AxMKa3ViZXJuZXRlczAeFw0yMjA2MjgwNjUyMDBaFw0zMjA2MjUwNjUyMDBaMGkx CzAJBgNVBAYTAkNOMRIwEAYDVQQIEwlHVUFOR0RPTkcxEjAQBgNVBAcTCUdVQU5H WkhPVTEMMAoGA1UEChMDazhzMQ8wDQYDVQQLEwZzeXN0ZW0xEzARBgNVBAMTCmt1 YmVybmV0ZXMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCwbY5jGqc0 uupFuPB3lWtoCXr3UciqfISbGcR+ZzoqRqGOvwp0OLzqsW6zEMvXvJZpTJjrcc+x SJtHC3eccTrGGM24RsLxgzz++MJFM5Rzr2gDDAspt/5wox6wRK0iLBa07EXztMJR Tzq5VBqCiLQ8Spte5s9Ghy+MwOWajJojQFHgD7y93QFQMmmyz5Az9GQtL5sIJoIg vGNUfO++stniceBuhRrpVYWOwNd/ZidHlus7rqtpFKEedOAHFzAUPlZrHf7wQ9QL 9KMYGnaea3uuhEzx/EuuHhozSdR7/lDRNKaQ+8u2e+WeDvXjABhZLgORIqfSOnQq ncuaBp9svl03AgMBAAGjZjBkMA4GA1UdDwEB/wQEAwIBBjASBgNVHRMBAf8ECDAG AQH/AgECMB0GA1UdDgQWBBQcgKUPl6v2oiZcLVJWKaIZGn1I1jAfBgNVHSMEGDAW gBQcgKUPl6v2oiZcLVJWKaIZGn1I1jANBgkqhkiG9w0BAQsFAAOCAQEAhc8gjihU 8RUyxHwnY0UWReJ7BtBIk+UZqbphKKIbtCeIHMgl/I2aSNIpmZsxFhlccnoDRu4S K8lVI1ck+agUwdDw1BOih5e35bN+ooNWNO6rpywEU8r4kqc9ghUb69QZF5eK6UZU fxYUf5c78QsZGcvL7iYRGQ/LiNMfdkdzvADBJXy50Li+d+RIZZk7DMCSki8KoRqg fsC/IzKCV5mvXqEOuXCUK+EjDjdHxeyRA1wyTmQyxfu2kF4M9BKY/Y/mRR54ARvU 6J/xQtkRXw2xn6/e5H2fAFKjPEDRyrXWsfV6Okgl55UAx3Pa+E89lcmMRnVLLFkq n7hXQuAz4zImXA== -----END CERTIFICATE----- $ cat /opt/kubelw/cfssl/ca-key.pem -----BEGIN RSA PRIVATE KEY----- MIIEpQIBAAKCAQEAsG2OYxqnNLrqRbjwd5VraAl691HIqnyEmxnEfmc6Kkahjr8K dDi86rFusxDL17yWaUyY63HPsUibRwt3nHE6xhjNuEbC8YM8/vjCRTOUc69oAwwL Kbf+cKMesEStIiwWtOxF87TCUU86uVQagoi0PEqbXubPRocvjMDlmoyaI0BR4A+8 vd0BUDJpss+QM/RkLS+bCCaCILxjVHzvvrLZ4nHgboUa6VWFjsDXf2YnR5brO66r aRShHnTgBxcwFD5Wax3+8EPUC/SjGBp2nmt7roRM8fxLrh4aM0nUe/5Q0TSmkPvL tnvlng714wAYWS4DkSKn0jp0Kp3LmgafbL5dNwIDAQABAoIBAQCHt18m4WPqbja0 97UTaH+9Aj3zbpg8fZjMbx/2VJYr2zWAR3lVOigpKeCMIsmL5WiXC/M+eshYChBY sHuMfpXFuWLW9KgVfO04/kcDUNBLxYzvex5DM2SpZPHAirPca6nz9yVAebZZMeds lUPnUh3Dm2i1sjuUd32eeuyk3K/dmN862G+wigqlkKtjihp0sNbdB1ucFsEVUkAt eQ3dq0YwL1f7nCg9ZY7PMQ2eh/2JkG0jwTsZA41odyA1OkZBAPBevdgH8hdxV2/y YNnDWAoxDhOhD4q7tCg862z3F1/55oj9w24Rsm38hJYLl1D0/zHzkmeX1lgkpMQD LGFK7gsxAoGBAMcsF2yRl7GZfmmiILJPvjua3PQv+YCUIaqr/xFIRMhOBD0dUw67 pAPOxTgEH81hqyY9/CkTH/ZCKLgGz35OPYaJYsTZVM5aSoWkWNJfOqeYNCzaZJwT O9vVa7KHr0QcfymN8yRLlT4J9f2oPknQt2QqmvwoPDvlsHJhcBzCTWOFAoGBAOLE LjU21ZHupTVaJ0uposc3uySENw3wZlRcHOyOKy69exf7wMy/N+tTs4g/NvmKhzO9 gpQ9SKJj8n8YrA7I8TxuA4sQT7hCESFKw1tyw3EH8OZrzz1lPvJyHsu/k9xTqqhY eSwRfvl7KXCOSus+Anb3R8N/rHb/TcPwjNG+QkSLAoGBAI4IDEA45vMYYYRUwHpH 0YHR4sUjvQoLGKML+l3Jqnso327xjXxRJRouBof2sPMWNiWUSFDGOaGz9jOdb7RD eS6KpGt6DDcHPmNlGo4SqNJBANwHdX2zXZlb7WwnxD2PEMOCXaRBXhEaq1gS9TBQ bac5lsJAswuHtTcr8vYfPW69AoGADS37xYn/VbD6FyS7PfGJDW0WymOI052SRPrp j3If3mKS4ez24q+Gb3345EVQS6aafw5XpYf+TbnjYTGs5lsVcj6upAl5qKrmVfoD arA73bjpbmr7q4TT6MFrOspSrK6ML6acvEv0Bkn7OZh7kDqVaBatLBaijnP+MBIu DQ6yyUsCgYEAgzBdL2bu6/lHNykHIYlsAECVePAoDRwSNkyDBblhyJgdE/qw1+8c 6qKKm5KhHAOOsfhxCKLUgDxnNJyk17Hu9NKMDHS9HBK8DZSZLR97D1aNETBZzs3R Aj1/DdFQKYkvG7Tj4S4kTv7OwZ/Z+yMOQO7usgAGUpMJgBs1Wo7l9F4= -----END RSA PRIVATE KEY----- # 赋予所有权限 $ chmod 777 -R /home/user1/* # 修改/root/.kube/config 文件,会自动添加一个user1用户的配置项在/root/.kube/config 文件中。 $ kubectl config set-credentials user1 --client-certificate=/home/user1/user1.crt --client-key=/home/user1/user1.key $ cat /root/.kube/config ............ - name: user1 user: client-certificate: /home/user1/user1.crt client-key: /home/user1/user1.key # 复制/root.kube/config 文件到/home/user1/.kube $ mkdir /home/user1/.kube && cp -r /root/.kube/config /home/user1/.kube/ $ chown user1:user1 /home/user1/.kube/* # 切换到user1用户。 $ su user1 # 修改/home/user1/.kube/config文件。 # 修改前/home/user1/.kube/config 文件。 $ cat /home/user1/.kube/config apiVersion: v1 clusters: - cluster: certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUR4akNDQXE2Z0F3SUJBZ0lVRkpGUjR3bG1NWXZZTXpmVmNtZkwrSzJWTERvd0RRWUpLb1pJaHZjTkFRRUwKQlFBd2FURUxNQWtHQTFVRUJoTUNRMDR4RWpBUUJnTlZCQWdUQ1VkVlFVNUhSRTlPUnpFU01CQUdBMVVFQnhNSgpSMVZCVGtkYVNFOVZNUXd3Q2dZRFZRUUtFd05yT0hNeER6QU5CZ05WQkFzVEJuTjVjM1JsYlRFVE1CRUdBMVVFCkF4TUthM1ZpWlhKdVpYUmxjekFlRncweU1qQTJNamd3TmpVeU1EQmFGdzB6TWpBMk1qVXdOalV5TURCYU1Ha3gKQ3pBSkJnTlZCQVlUQWtOT01SSXdFQVlEVlFRSUV3bEhWVUZPUjBSUFRrY3hFakFRQmdOVkJBY1RDVWRWUVU1SApXa2hQVlRFTU1Bb0dBMVVFQ2hNRGF6aHpNUTh3RFFZRFZRUUxFd1p6ZVhOMFpXMHhFekFSQmdOVkJBTVRDbXQxClltVnlibVYwWlhNd2dnRWlNQTBHQ1NxR1NJYjNEUUVCQVFVQUE0SUJEd0F3Z2dFS0FvSUJBUUN3Ylk1akdxYzAKdXVwRnVQQjNsV3RvQ1hyM1VjaXFmSVNiR2NSK1p6b3FScUdPdndwME9MenFzVzZ6RU12WHZKWnBUSmpyY2MreApTSnRIQzNlY2NUckdHTTI0UnNMeGd6eisrTUpGTTVSenIyZ0REQXNwdC81d294NndSSzBpTEJhMDdFWHp0TUpSClR6cTVWQnFDaUxROFNwdGU1czlHaHkrTXdPV2FqSm9qUUZIZ0Q3eTkzUUZRTW1teXo1QXo5R1F0TDVzSUpvSWcKdkdOVWZPKytzdG5pY2VCdWhScnBWWVdPd05kL1ppZEhsdXM3cnF0cEZLRWVkT0FIRnpBVVBsWnJIZjd3UTlRTAo5S01ZR25hZWEzdXVoRXp4L0V1dUhob3pTZFI3L2xEUk5LYVErOHUyZStXZUR2WGpBQmhaTGdPUklxZlNPblFxCm5jdWFCcDlzdmwwM0FnTUJBQUdqWmpCa01BNEdBMVVkRHdFQi93UUVBd0lCQmpBU0JnTlZIUk1CQWY4RUNEQUcKQVFIL0FnRUNNQjBHQTFVZERnUVdCQlFjZ0tVUGw2djJvaVpjTFZKV0thSVpHbjFJMWpBZkJnTlZIU01FR0RBVwpnQlFjZ0tVUGw2djJvaVpjTFZKV0thSVpHbjFJMWpBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWhjOGdqaWhVCjhSVXl4SHduWTBVV1JlSjdCdEJJaytVWnFicGhLS0lidENlSUhNZ2wvSTJhU05JcG1ac3hGaGxjY25vRFJ1NFMKSzhsVkkxY2srYWdVd2REdzFCT2loNWUzNWJOK29vTldOTzZycHl3RVU4cjRrcWM5Z2hVYjY5UVpGNWVLNlVaVQpmeFlVZjVjNzhRc1pHY3ZMN2lZUkdRL0xpTk1mZGtkenZBREJKWHk1MExpK2QrUklaWms3RE1DU2tpOEtvUnFnCmZzQy9JektDVjVtdlhxRU91WENVSytFakRqZEh4ZXlSQTF3eVRtUXl4ZnUya0Y0TTlCS1kvWS9tUlI1NEFSdlUKNkoveFF0a1JYdzJ4bjYvZTVIMmZBRktqUEVEUnlyWFdzZlY2T2tnbDU1VUF4M1BhK0U4OWxjbU1SblZMTEZrcQpuN2hYUXVBejR6SW1YQT09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K server: https://10.20.43.147:6443 name: kubernetes contexts: - context: cluster: kubernetes user: admin name: kubernetes current-context: kubernetes kind: Config preferences: {} users: - name: admin user: client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUQ1VENDQXMyZ0F3SUJBZ0lVREozRTY0NDRKVXlEdWh5dEg4czVwWXR5eTZRd0RRWUpLb1pJaHZjTkFRRUwKQlFBd2FURUxNQWtHQTFVRUJoTUNRMDR4RWpBUUJnTlZCQWdUQ1VkVlFVNUhSRTlPUnpFU01CQUdBMVVFQnhNSgpSMVZCVGtkYVNFOVZNUXd3Q2dZRFZRUUtFd05yT0hNeER6QU5CZ05WQkFzVEJuTjVjM1JsYlRFVE1CRUdBMVVFCkF4TUthM1ZpWlhKdVpYUmxjekFlRncweU1qQTJNamd3TmpVek1EQmFGdzB6TWpBMk1qVXdOalV6TURCYU1HOHgKQ3pBSkJnTlZCQVlUQWtOT01SSXdFQVlEVlFRSUV3bEhWVUZPUjBSUFRrY3hFakFRQmdOVkJBY1RDVWRWUVU1SApXa2hQVlRFWE1CVUdBMVVFQ2hNT2MzbHpkR1Z0T20xaGMzUmxjbk14RHpBTkJnTlZCQXNUQm5ONWMzUmxiVEVPCk1Bd0dBMVVFQXhNRllXUnRhVzR3Z2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLQW9JQkFRRHgKT0c3a3psd1hTc243dS9GV2JNWGMyRlEyTS9SWDliSjhDNW5ER3ZpMHRRc096RndhcUZwSkx4cCtZakk4K2Z0cwpZU3c1STNpNk5vcEdzNk5yNkhhV0VpVkZHdkFjREdORUhZYTVFNkRXNkNjN1NiSDVvOS8zdmVPZTNZQTF6V3h5CnAxejNGQjl4Tll0dmkyUkpJenZPdnpWRDI4MUdJQ1lSN1ZnZXM3VDJGQUp2V2t5SzRDT0NMbWppRjlRTkNKKzcKVGtYVFZVZjFuRThjZ0lCWHMyVllxNjhFQzlCTDlzUjBJaGFNa05MTHRZcFVVV040UVNGN3hscTZLT0tTUlZXKwpqRXozR3hjK0NsaVNIL2kwSDZ4ZnZYbVBXMWhDQXNqWVhvT3dKWWxpRlYreVFLSktYd241dHVFRVRaTEFPbmJZCjQyNC9MQi9JL2hjK0JXSXBza1piQWdNQkFBR2pmekI5TUE0R0ExVWREd0VCL3dRRUF3SUZvREFkQmdOVkhTVUUKRmpBVUJnZ3JCZ0VGQlFjREFRWUlLd1lCQlFVSEF3SXdEQVlEVlIwVEFRSC9CQUl3QURBZEJnTlZIUTRFRmdRVQpDMjVOZU4zQmFVV3ZYRWlFallqSHI5WkJybVV3SHdZRFZSMGpCQmd3Rm9BVUhJQ2xENWVyOXFJbVhDMVNWaW1pCkdScDlTTll3RFFZSktvWklodmNOQVFFTEJRQURnZ0VCQUZsclFaZW9XVUU5VjJMUUM5Tnl5Ynp1NkdsN3dsdVYKd3YwTW9HcWZnNVIwbnJFRDdrbUl5cy9zd0FXQ3FCYkRETjhldU5KUDFWZXp1QnpNUWVQWm9OdXU2ZzhhbEtYVwp0eW92M25EOU1TTTdIc3hvbXM4cjVvOVN2eTNoRHNCTVplUDlMWUlIbzJhUHp4enpzMEhjNGR0Zy9KS1JraCswCm8xbm9lSVBldTNpSXpLeXZDWnZJUk5hd1Y4TzhFVThTL3VRWmhrcXR3d1dCTFVZanF2MTF0WkxYVEdicmMybkoKRTc1TW0zb3B1NTNkR1RjeVFRT0NGOXhuZ09GS0VHNnVjSDNSMHpibnNXNmlySzlDWnFLV0dZZnk3c0NlRGlzeApQbGI1YWZlTkErTGc0dnF1ZENlZi8ycjdBZjNaQXZVNUFzTzFMY3NhMW1URUxnb1ZkaDdEMm44PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg== client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBOFRodTVNNWNGMHJKKzd2eFZtekYzTmhVTmpQMFYvV3lmQXVad3hyNHRMVUxEc3hjCkdxaGFTUzhhZm1JeVBQbjdiR0VzT1NONHVqYUtSck9qYStoMmxoSWxSUnJ3SEF4alJCMkd1Uk9nMXVnbk8wbXgKK2FQZjk3M2pudDJBTmMxc2NxZGM5eFFmY1RXTGI0dGtTU003enI4MVE5dk5SaUFtRWUxWUhyTzA5aFFDYjFwTQppdUFqZ2k1bzRoZlVEUWlmdTA1RjAxVkg5WnhQSElDQVY3TmxXS3V2QkF2UVMvYkVkQ0lXakpEU3k3V0tWRkZqCmVFRWhlOFphdWlqaWtrVlZ2b3hNOXhzWFBncFlraC80dEIrc1g3MTVqMXRZUWdMSTJGNkRzQ1dKWWhWZnNrQ2kKU2w4SitiYmhCRTJTd0RwMjJPTnVQeXdmeVA0WFBnVmlLYkpHV3dJREFRQUJBb0lCQVFEYWxLS1lIdzUvNUx0bQpxd1dqcE9rZWw4Q25aU2pTMDhjcWRIQ2V4VC84cjlrWFRjTmdQSkdqbzFWRWxNS2xVbmlyMnRueDVOeXZFR0Q0CmRDdHZ5clE2aExMVkRmWHAxS2ZXdjFLblBzd09NVXZyZzNvTWxweUtwNzBzNWRZWjZzMk1qMi9FVEsyNUxpWHQKcThmeUprVTVzVFlaQ1lIWE82YUR0Q2lYbVl3dk5td21ra3Y1TjBSQ3YyZmdlQVZ2aCtKV1R2eDE3ZDkwNnMwdgo5aUpXYkhhbXk1QUhYK3JLUDc1K3FlR25ja1IvaDN4R3hWZkFIYXFYaTVZWXBWMFo3OE43eGtwRlZEMWg2OVlPCm00emNCUUVWZi80RFhXWnpHYmQrdmpqOWtmUVYrMi9zd1MwRWJOOVRha2I3VVZKcDVmYy9vcFNKNlpKWW9ZcFMKa280M2ZGUnhBb0dCQVBwbUtiRXoyWXowT2dIdjRWaU15TUlkVWQ0L1V4bldRVnM3RURaMTQ5bStwdXhmaFRxNApSSStMOGpZNzhBNWVZYTh5L1I4T1VnR0EvdFpjV015SVdnV1VtQkVWNW9BWXBDVjZRVU0xTjRiYmxGNzNzc0pDClpTZWRQZWN2YVlVK1lYamxZVW9HZzhyeE1iYmdlV3gzOGhNRFdFdGhTcTg2V0NSSjVVdk5pL1l2QW9HQkFQYWQKdGlrNklTbDVkbkp5cGczd3F6OXZ3RjZPV2ZFUjNydVdVREtuTnhPMm9mUWt4WXY5TWlPSVpDZnY2V01PV0FuTQpMeXpQY0MvSEY2T0dGaU1wN1dTNFBKUEZBSW42OHpTT2x5MmMrSVhUVmhvV0plSkZjeDFFY3RHdWVmTmN1QnY2CkxJc1FwUlA1MlJYTnJrYXo2YWE5T3orQlkxYnhFb3o4WmZQOFVaT1ZBb0dBRU9ja21WbXVyZDB1clVMTUszWVYKZDBVVGFiVk1uc25mejVERkgyZ0Y0WVVGTDUrakZydXBwU2NGU3JQeEdJYzJnT1VvUVJodVlMdWNlRXZ6a3BzQgp1SzYzTlRBTlIwaGlqRVVjY3JUODhwV1FCbmtpRUFyN1dSazhQSWJ1MEpmTmJLUFBWWGZySFovNmd5WFVESVpzClNPeEk0WTNIeE54alpzKzJNcy9GU3ZFQ2dZRUF0Z05HTzMxRWxtaW42K1lEK292aXgrb0JqNGYraDdhSnJlZE4KZjJUOGVGYzFob3hSSkhXVnVMWGtQYm1LaGVwRFBjL3VEV040U0RybmptL2JETTdYLzVzZVVtMTJiVi9DZWNxMwpkeG5BTG4wQXNqWWdkYlNPTms2YmMzZ1RWM0xhQ3dRRU5ncnQyeVZ1ZS9JV3F1WUVEMlRnUW9tTE9OS3B2MVpWCkpOTFhubFVDZ1lCY0pTRTgyeEZ0Z1VmanpDdmxXNlVaK0laeE1QNXRFWEpuVDZ0eG42anRQb3VESUU2bTNqdVgKYW1wbEJwZmEyYnA5TExncTRIMWMrSUhRL0E2dGptU1pJaDN1NEFIT3FDcjMwaHptR0NOaTZSU2RGeVdJMWZDSQp0R244QWViUUdxVzVSMjg0amlpWFBROER6d2ZGUWF3UHpyZE5PRUo0Vzc2S3hWTVp0cEpvWGc9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo= - name: user1 user: client-certificate: /home/user1/user1.crt client-key: /home/user1/user1.key # 修改内容一：修改用户 - context: cluster: kubernetes user: admin # 此处修改为：user1 name: kubernetes # 此处修改为：user1 current-context: kubernetes # 此处修改为：user1 # 修改内容二：admin用户数据删除掉。 - name: admin user: client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUQ1VENDQXMyZ0F3SUJBZ0lVREozRTY0NDRKVXlEdWh5dEg4czVwWXR5eTZRd0RRWUpLb1pJaHZjTkFRRUwKQlFBd2FURUxNQWtHQTFVRUJoTUNRMDR4RWpBUUJnTlZCQWdUQ1VkVlFVNUhSRTlPUnpFU01CQUdBMVVFQnhNSgpSMVZCVGtkYVNFOVZNUXd3Q2dZRFZRUUtFd05yT0hNeER6QU5CZ05WQkFzVEJuTjVjM1JsYlRFVE1CRUdBMVVFCkF4TUthM1ZpWlhKdVpYUmxjekFlRncweU1qQTJNamd3TmpVek1EQmFGdzB6TWpBMk1qVXdOalV6TURCYU1HOHgKQ3pBSkJnTlZCQVlUQWtOT01SSXdFQVlEVlFRSUV3bEhWVUZPUjBSUFRrY3hFakFRQmdOVkJBY1RDVWRWUVU1SApXa2hQVlRFWE1CVUdBMVVFQ2hNT2MzbHpkR1Z0T20xaGMzUmxjbk14RHpBTkJnTlZCQXNUQm5ONWMzUmxiVEVPCk1Bd0dBMVVFQXhNRllXUnRhVzR3Z2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLQW9JQkFRRHgKT0c3a3psd1hTc243dS9GV2JNWGMyRlEyTS9SWDliSjhDNW5ER3ZpMHRRc096RndhcUZwSkx4cCtZakk4K2Z0cwpZU3c1STNpNk5vcEdzNk5yNkhhV0VpVkZHdkFjREdORUhZYTVFNkRXNkNjN1NiSDVvOS8zdmVPZTNZQTF6V3h5CnAxejNGQjl4Tll0dmkyUkpJenZPdnpWRDI4MUdJQ1lSN1ZnZXM3VDJGQUp2V2t5SzRDT0NMbWppRjlRTkNKKzcKVGtYVFZVZjFuRThjZ0lCWHMyVllxNjhFQzlCTDlzUjBJaGFNa05MTHRZcFVVV040UVNGN3hscTZLT0tTUlZXKwpqRXozR3hjK0NsaVNIL2kwSDZ4ZnZYbVBXMWhDQXNqWVhvT3dKWWxpRlYreVFLSktYd241dHVFRVRaTEFPbmJZCjQyNC9MQi9JL2hjK0JXSXBza1piQWdNQkFBR2pmekI5TUE0R0ExVWREd0VCL3dRRUF3SUZvREFkQmdOVkhTVUUKRmpBVUJnZ3JCZ0VGQlFjREFRWUlLd1lCQlFVSEF3SXdEQVlEVlIwVEFRSC9CQUl3QURBZEJnTlZIUTRFRmdRVQpDMjVOZU4zQmFVV3ZYRWlFallqSHI5WkJybVV3SHdZRFZSMGpCQmd3Rm9BVUhJQ2xENWVyOXFJbVhDMVNWaW1pCkdScDlTTll3RFFZSktvWklodmNOQVFFTEJRQURnZ0VCQUZsclFaZW9XVUU5VjJMUUM5Tnl5Ynp1NkdsN3dsdVYKd3YwTW9HcWZnNVIwbnJFRDdrbUl5cy9zd0FXQ3FCYkRETjhldU5KUDFWZXp1QnpNUWVQWm9OdXU2ZzhhbEtYVwp0eW92M25EOU1TTTdIc3hvbXM4cjVvOVN2eTNoRHNCTVplUDlMWUlIbzJhUHp4enpzMEhjNGR0Zy9KS1JraCswCm8xbm9lSVBldTNpSXpLeXZDWnZJUk5hd1Y4TzhFVThTL3VRWmhrcXR3d1dCTFVZanF2MTF0WkxYVEdicmMybkoKRTc1TW0zb3B1NTNkR1RjeVFRT0NGOXhuZ09GS0VHNnVjSDNSMHpibnNXNmlySzlDWnFLV0dZZnk3c0NlRGlzeApQbGI1YWZlTkErTGc0dnF1ZENlZi8ycjdBZjNaQXZVNUFzTzFMY3NhMW1URUxnb1ZkaDdEMm44PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg== client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBOFRodTVNNWNGMHJKKzd2eFZtekYzTmhVTmpQMFYvV3lmQXVad3hyNHRMVUxEc3hjCkdxaGFTUzhhZm1JeVBQbjdiR0VzT1NONHVqYUtSck9qYStoMmxoSWxSUnJ3SEF4alJCMkd1Uk9nMXVnbk8wbXgKK2FQZjk3M2pudDJBTmMxc2NxZGM5eFFmY1RXTGI0dGtTU003enI4MVE5dk5SaUFtRWUxWUhyTzA5aFFDYjFwTQppdUFqZ2k1bzRoZlVEUWlmdTA1RjAxVkg5WnhQSElDQVY3TmxXS3V2QkF2UVMvYkVkQ0lXakpEU3k3V0tWRkZqCmVFRWhlOFphdWlqaWtrVlZ2b3hNOXhzWFBncFlraC80dEIrc1g3MTVqMXRZUWdMSTJGNkRzQ1dKWWhWZnNrQ2kKU2w4SitiYmhCRTJTd0RwMjJPTnVQeXdmeVA0WFBnVmlLYkpHV3dJREFRQUJBb0lCQVFEYWxLS1lIdzUvNUx0bQpxd1dqcE9rZWw4Q25aU2pTMDhjcWRIQ2V4VC84cjlrWFRjTmdQSkdqbzFWRWxNS2xVbmlyMnRueDVOeXZFR0Q0CmRDdHZ5clE2aExMVkRmWHAxS2ZXdjFLblBzd09NVXZyZzNvTWxweUtwNzBzNWRZWjZzMk1qMi9FVEsyNUxpWHQKcThmeUprVTVzVFlaQ1lIWE82YUR0Q2lYbVl3dk5td21ra3Y1TjBSQ3YyZmdlQVZ2aCtKV1R2eDE3ZDkwNnMwdgo5aUpXYkhhbXk1QUhYK3JLUDc1K3FlR25ja1IvaDN4R3hWZkFIYXFYaTVZWXBWMFo3OE43eGtwRlZEMWg2OVlPCm00emNCUUVWZi80RFhXWnpHYmQrdmpqOWtmUVYrMi9zd1MwRWJOOVRha2I3VVZKcDVmYy9vcFNKNlpKWW9ZcFMKa280M2ZGUnhBb0dCQVBwbUtiRXoyWXowT2dIdjRWaU15TUlkVWQ0L1V4bldRVnM3RURaMTQ5bStwdXhmaFRxNApSSStMOGpZNzhBNWVZYTh5L1I4T1VnR0EvdFpjV015SVdnV1VtQkVWNW9BWXBDVjZRVU0xTjRiYmxGNzNzc0pDClpTZWRQZWN2YVlVK1lYamxZVW9HZzhyeE1iYmdlV3gzOGhNRFdFdGhTcTg2V0NSSjVVdk5pL1l2QW9HQkFQYWQKdGlrNklTbDVkbkp5cGczd3F6OXZ3RjZPV2ZFUjNydVdVREtuTnhPMm9mUWt4WXY5TWlPSVpDZnY2V01PV0FuTQpMeXpQY0MvSEY2T0dGaU1wN1dTNFBKUEZBSW42OHpTT2x5MmMrSVhUVmhvV0plSkZjeDFFY3RHdWVmTmN1QnY2CkxJc1FwUlA1MlJYTnJrYXo2YWE5T3orQlkxYnhFb3o4WmZQOFVaT1ZBb0dBRU9ja21WbXVyZDB1clVMTUszWVYKZDBVVGFiVk1uc25mejVERkgyZ0Y0WVVGTDUrakZydXBwU2NGU3JQeEdJYzJnT1VvUVJodVlMdWNlRXZ6a3BzQgp1SzYzTlRBTlIwaGlqRVVjY3JUODhwV1FCbmtpRUFyN1dSazhQSWJ1MEpmTmJLUFBWWGZySFovNmd5WFVESVpzClNPeEk0WTNIeE54alpzKzJNcy9GU3ZFQ2dZRUF0Z05HTzMxRWxtaW42K1lEK292aXgrb0JqNGYraDdhSnJlZE4KZjJUOGVGYzFob3hSSkhXVnVMWGtQYm1LaGVwRFBjL3VEV040U0RybmptL2JETTdYLzVzZVVtMTJiVi9DZWNxMwpkeG5BTG4wQXNqWWdkYlNPTms2YmMzZ1RWM0xhQ3dRRU5ncnQyeVZ1ZS9JV3F1WUVEMlRnUW9tTE9OS3B2MVpWCkpOTFhubFVDZ1lCY0pTRTgyeEZ0Z1VmanpDdmxXNlVaK0laeE1QNXRFWEpuVDZ0eG42anRQb3VESUU2bTNqdVgKYW1wbEJwZmEyYnA5TExncTRIMWMrSUhRL0E2dGptU1pJaDN1NEFIT3FDcjMwaHptR0NOaTZSU2RGeVdJMWZDSQp0R244QWViUUdxVzVSMjg0amlpWFBROER6d2ZGUWF3UHpyZE5PRUo0Vzc2S3hWTVp0cEpvWGc9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo= # 修改后/home/user1/.kube/config文件。 $ cat /home/user1/.kube/config apiVersion: v1 clusters: - cluster: certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUR4akNDQXE2Z0F3SUJBZ0lVRkpGUjR3bG1NWXZZTXpmVmNtZkwrSzJWTERvd0RRWUpLb1pJaHZjTkFRRUwKQlFBd2FURUxNQWtHQTFVRUJoTUNRMDR4RWpBUUJnTlZCQWdUQ1VkVlFVNUhSRTlPUnpFU01CQUdBMVVFQnhNSgpSMVZCVGtkYVNFOVZNUXd3Q2dZRFZRUUtFd05yT0hNeER6QU5CZ05WQkFzVEJuTjVjM1JsYlRFVE1CRUdBMVVFCkF4TUthM1ZpWlhKdVpYUmxjekFlRncweU1qQTJNamd3TmpVeU1EQmFGdzB6TWpBMk1qVXdOalV5TURCYU1Ha3gKQ3pBSkJnTlZCQVlUQWtOT01SSXdFQVlEVlFRSUV3bEhWVUZPUjBSUFRrY3hFakFRQmdOVkJBY1RDVWRWUVU1SApXa2hQVlRFTU1Bb0dBMVVFQ2hNRGF6aHpNUTh3RFFZRFZRUUxFd1p6ZVhOMFpXMHhFekFSQmdOVkJBTVRDbXQxClltVnlibVYwWlhNd2dnRWlNQTBHQ1NxR1NJYjNEUUVCQVFVQUE0SUJEd0F3Z2dFS0FvSUJBUUN3Ylk1akdxYzAKdXVwRnVQQjNsV3RvQ1hyM1VjaXFmSVNiR2NSK1p6b3FScUdPdndwME9MenFzVzZ6RU12WHZKWnBUSmpyY2MreApTSnRIQzNlY2NUckdHTTI0UnNMeGd6eisrTUpGTTVSenIyZ0REQXNwdC81d294NndSSzBpTEJhMDdFWHp0TUpSClR6cTVWQnFDaUxROFNwdGU1czlHaHkrTXdPV2FqSm9qUUZIZ0Q3eTkzUUZRTW1teXo1QXo5R1F0TDVzSUpvSWcKdkdOVWZPKytzdG5pY2VCdWhScnBWWVdPd05kL1ppZEhsdXM3cnF0cEZLRWVkT0FIRnpBVVBsWnJIZjd3UTlRTAo5S01ZR25hZWEzdXVoRXp4L0V1dUhob3pTZFI3L2xEUk5LYVErOHUyZStXZUR2WGpBQmhaTGdPUklxZlNPblFxCm5jdWFCcDlzdmwwM0FnTUJBQUdqWmpCa01BNEdBMVVkRHdFQi93UUVBd0lCQmpBU0JnTlZIUk1CQWY4RUNEQUcKQVFIL0FnRUNNQjBHQTFVZERnUVdCQlFjZ0tVUGw2djJvaVpjTFZKV0thSVpHbjFJMWpBZkJnTlZIU01FR0RBVwpnQlFjZ0tVUGw2djJvaVpjTFZKV0thSVpHbjFJMWpBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWhjOGdqaWhVCjhSVXl4SHduWTBVV1JlSjdCdEJJaytVWnFicGhLS0lidENlSUhNZ2wvSTJhU05JcG1ac3hGaGxjY25vRFJ1NFMKSzhsVkkxY2srYWdVd2REdzFCT2loNWUzNWJOK29vTldOTzZycHl3RVU4cjRrcWM5Z2hVYjY5UVpGNWVLNlVaVQpmeFlVZjVjNzhRc1pHY3ZMN2lZUkdRL0xpTk1mZGtkenZBREJKWHk1MExpK2QrUklaWms3RE1DU2tpOEtvUnFnCmZzQy9JektDVjVtdlhxRU91WENVSytFakRqZEh4ZXlSQTF3eVRtUXl4ZnUya0Y0TTlCS1kvWS9tUlI1NEFSdlUKNkoveFF0a1JYdzJ4bjYvZTVIMmZBRktqUEVEUnlyWFdzZlY2T2tnbDU1VUF4M1BhK0U4OWxjbU1SblZMTEZrcQpuN2hYUXVBejR6SW1YQT09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K server: https://10.20.43.147:6443 name: kubernetes contexts: - context: cluster: kubernetes user: user1 name: user1 current-context: user1 kind: Config preferences: {} users: - name: user1 user: client-certificate: /home/user1/user1.crt client-key: /home/user1/user1.key # 切换到root用户，创建role，并通过rolebinding与用户绑定，赋予user1用户操作权限 $ cat > user1_role.yaml<< EOF --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: namespace: user1 # 通过此角色绑定，"user1"可以读取"user1"名称空间中的Pod、replicasets、deployments。 name: user1Role rules: - apiGroups: ["apps"] #目标api群组 resources: ["replicasets", "deployments"] #目标资源的操作权限 verbs: ["get", "watch", "list", "create", "delete", "edit", "exec"] - apiGroups: [""] resources: ["pods"] #目标资源的操作权限 verbs: ["get", "watch", "list", "create", "delete", "edit", "exec"] EOF $ kubectl create -f user1_role.yaml $ cat > user1Rolebinding.yaml << EOF --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: rolebind-user1 namespace: user1 # 修改user需要访问的namespace subjects: - kind: User name: user1 # 修改自己定义的user1，user1根role进行绑定。 apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: user1Role # 修改自己定义的role名字 apiGroup: rbac.authorization.k8s.io EOF $ kubectl create -f user1Rolebinding.yaml # 进行测试 $ su user1 $ kubectl get deployments -n user1 NAME READY UP-TO-DATE AVAILABLE AGE deploy-web-user1 3/3 3 3 4h2m $ kubectl get rs -n user1 NAME DESIRED CURRENT READY AGE deploy-web-user1-7797f778bd 3 3 3 4h2m $ kubectl get pod -n user1 NAME READY STATUS RESTARTS AGE deploy-web-user1-7797f778bd-7sf6g 1/1 Running 0 4h2m deploy-web-user1-7797f778bd-bhrjj 1/1 Running 0 4h2m deploy-web-user1-7797f778bd-hrsw4 1/1 Running 0 4h2m nginx 1/1 Running 0 4h31m # 如果你试图访问其它namespace你会发现报了权限错误。因为user1没有访问名为user2 namespace资源的权限。 $ kubectl get pod -n user2 Error from server (Forbidden): pods is forbidden: User "user1" cannot list resource "pods" in API group "" in the namespace "user2" 二、创建普通用户赋予超级权限
目的：创建kubenetes集群内一个普通用户，然后直接通过Group(组)来绑定它的权限，我们的目标是要绑定默认的超级角色 cluster-admin
# 这次我们为了省时间，就不会在linux创建一个用户了，直接创建一个k8s集群普通用户。 $ mkdir /root/superuser # 生成私钥 $ openssl genrsa -out superuser.key 2048 # 生成一个待签名文件(superuser.csr)，注意 O= system:masters 代表它的超管权限组, 直接代表了superuser用户加入后拥有超管权限。 $ openssl req -new -key superuser.key -out superuser.csr -subj "/CN= superuser /O=system:masters" # 用 k8s 的 ca 文件来签名这个superuser.csr,最终产生一个有效期为 3600 天的证书文件。 $ openssl x509 -req -in superuser.csr -CA /opt/kubelw/cfssl/ca.pem -CAkey /opt/kubelw/cfssl/ca-key.pem -CAcreateserial -out superuser.crt -days 3600 注意：/opt/kubelw/cfssl/ca.pem 、/opt/kubelw/cfssl/ca-key.pem 根据自身集群实际路径填写（多数情况下是在 /etc/kubernetes/pki/路径下）。 # 修改/root/.kube/config 文件,会自动添加一个用的配置项在/root/.kube/config 文件中。 $ kubectl config set-credentials superuser --client-certificate=/root/superuser/superuser.crt --client-key=/root/superuser/superuser.key $ cat /root/.kube/config .......... - name: superuser user: client-certificate: /root/superuser/superuser.crt client-key: /root/superuser/superuser.key # 创建一个superuser context $ kubectl config set-context superuser-context --cluster=kubernetes --user=superuser # 该命令行在/root/.kube/config 文件中添加了以下文本。 - context: cluster: kubernetes user: superuser name: superuser-context # 获取当前contexts 环境信息。 $ kubectl config get-contexts CURRENT NAME CLUSTER AUTHINFO NAMESPACE * kubernetes kubernetes admin superuser-context kubernetes superuser user1 kubernetes user1 # 设置 superuser 为当前使用的 context(环境身份文件), 切换后superuser可以操作任意资源。 $ kubectl config use-context superuser-context $ kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE default daemon-web-4z9xk 1/1 Running 0 13h default daemon-web-bh5pc 1/1 Running 0 13h default daemon-web-t5zb9 1/1 Running 0 13h default deploy-web-7797f778bd-lzzkg 1/1 Running 0 14h default deploy-web-7797f778bd-qwh9r 1/1 Running 0 14h default deploy-web-7797f778bd-xgpmg 1/1 Running 0 14h ingress-nginx ingress-nginx-admission-create-2fknd 0/1 Completed 0 14d ingress-nginx ingress-nginx-admission-patch-l5lbm 0/1 Completed 1 14d ingress-nginx ingress-nginx-controller-559fb9c8bd-7vzxs 1/1 Running 2 14d ingress-nginx ingress-nginx-controller-559fb9c8bd-plz4c 1/1 Running 2 14d ingress-nginx ingress-nginx-controller-559fb9c8bd-wkcnt 1/1 Running 4 14d kube-system coredns-75674bbdf4-h4p24 1/1 Running 2 14d kube-system kube-flannel-ds-gkxfr 1/1 Running 2 14d kube-system kube-flannel-ds-scpxh 1/1 Running 4 14d kube-system kube-flannel-ds-vgnp4 1/1 Running 3 14d quota-mem-cpu-example quota-mem-cpu-demo 1/1 Running 0 37h quota-mem-cpu-example quota-mem-cpu-demo2 1/1 Running 0 37h user1 deploy-web-user1-7797f778bd-7sf6g 1/1 Running 0 5h1m user1 deploy-web-user1-7797f778bd-bhrjj 1/1 Running 0 5h1m user1 deploy-web-user1-7797f778bd-hrsw4 1/1 Running 0 5h1m user1 nginx 1/1 Running 0 5h29m 
