kubernetes GCP GKE:服务帐户““正在尝试授予当前不拥有得RBAC权限”

f0brbegy  于 2022-11-02  发布在  Kubernetes
关注(0)|答案(1)|浏览(236)

我正在设置一个CI/CD管道,以便以自动化的方式部署基于Kubernetes的应用程序。
当我的管道运行时,部署失败,并显示以下错误消息:

Error: roles.rbac.authorization.k8s.io "mongodb-kubernetes-operator" is forbidden: user "cicd-bot@my-project.iam.gserviceaccount.com" (groups=["system:authenticated"]) is attempting to grant RBAC permissions not currently held:
│ {APIGroups:[""], Resources:["configmaps"], Verbs:["list" "watch" "create" "update" "patch" "get" "delete"]}
│ {APIGroups:[""], Resources:["pods"], Verbs:["list" "watch" "create" "update" "patch" "get" "delete"]}
│ {APIGroups:[""], Resources:["secrets"], Verbs:["list" "watch" "create" "update" "patch" "get" "delete"]}
│ {APIGroups:[""], Resources:["services"], Verbs:["list" "watch" "create" "update" "patch" "get" "delete"]}
│ {APIGroups:["apps"], Resources:["statefulsets"], Verbs:["list" "watch" "create" "update" "patch" "get" "delete"]}
│ {APIGroups:["mongodbcommunity.mongodb.com"], Resources:["mongodbcommunity"], Verbs:["list" "watch" "update" "patch" "get"]}
│ {APIGroups:["mongodbcommunity.mongodb.com"], Resources:["mongodbcommunity/finalizers"], Verbs:["list" "watch" "update" "patch" "get"]}
│ {APIGroups:["mongodbcommunity.mongodb.com"], Resources:["mongodbcommunity/spec"], Verbs:["list" "watch" "update" "patch" "get"]}
│ {APIGroups:["mongodbcommunity.mongodb.com"], Resources:["mongodbcommunity/status"], Verbs:["list" "watch" "update" "patch" "get"]}
│ 
│   with module.db_document.kubernetes_role.operator_mongodb,
│   on modules/db_document/main.tf line 17, in resource "kubernetes_role" "operator_mongodb":
│   17: resource "kubernetes_role" "operator_mongodb" {
│

这个错误看起来很简单:我的服务帐户无法授予它不具有的权限。由于错误消息提到了我的GCP服务帐户cicd-bot@my-project.iam.gserviceaccount.com,因此我将我认为匹配的权限添加到了角色定义中。
下面是我得到的角色,它对configMaps、pod、secrets、services、statefulsets和thirdPartyObjects具有创建、删除、获取、列出和更新权限,我认为这些权限应该可以满足需求。

resource "google_project_iam_custom_role" "cicd_bot_role" {
  project = var.project
  role_id = "cicd_bot"
  title   = "CICD Bot"
  permissions = [
    "artifactregistry.repositories.downloadArtifacts",
    "artifactregistry.repositories.uploadArtifacts",
    "compute.instanceGroupManagers.get",
    "container.clusters.get",
    "container.configMaps.create",
    "container.configMaps.delete",
    "container.configMaps.get",
    "container.configMaps.list",
    "container.configMaps.update",
    "container.cronJobs.create",
    "container.cronJobs.delete",
    "container.cronJobs.get",
    "container.cronJobs.update",
    "container.customResourceDefinitions.create",
    "container.customResourceDefinitions.delete",
    "container.customResourceDefinitions.get",
    "container.customResourceDefinitions.list",
    "container.customResourceDefinitions.update",
    "container.deployments.create",
    "container.deployments.delete",
    "container.deployments.get",
    "container.deployments.update",
    "container.ingresses.create",
    "container.ingresses.delete",
    "container.ingresses.get",
    "container.ingresses.update",
    "container.jobs.create",
    "container.jobs.delete",
    "container.jobs.get",
    "container.jobs.update",
    "container.namespaces.get",
    "container.persistentVolumeClaims.create",
    "container.persistentVolumeClaims.delete",
    "container.persistentVolumeClaims.get",
    "container.persistentVolumeClaims.update",
    "container.pods.create",
    "container.pods.delete",
    "container.pods.get",
    "container.pods.list",
    "container.pods.update",
    "container.roleBindings.create",
    "container.roleBindings.delete",
    "container.roleBindings.get",
    "container.roleBindings.update",
    "container.roles.create",
    "container.roles.delete",
    "container.roles.get",
    "container.roles.update",
    "container.secrets.create",
    "container.secrets.delete",
    "container.secrets.get",
    "container.secrets.list",
    "container.secrets.update",
    "container.serviceAccounts.create",
    "container.serviceAccounts.delete",
    "container.serviceAccounts.get",
    "container.serviceAccounts.update",
    "container.services.create",
    "container.services.delete",
    "container.services.get",
    "container.services.list",
    "container.services.update",
    "container.statefulSets.create",
    "container.statefulSets.delete",
    "container.statefulSets.get",
    "container.statefulSets.list",
    "container.statefulSets.update",
    "container.thirdPartyObjects.create",
    "container.thirdPartyObjects.delete",
    "container.thirdPartyObjects.get",
    "container.thirdPartyObjects.list",
    "container.thirdPartyObjects.update",
    "dns.changes.create",
    "dns.changes.get",
    "dns.resourceRecordSets.get",
    "dns.resourceRecordSets.list",
    "dns.resourceRecordSets.update",
    "storage.buckets.get",
    "storage.objects.create",
    "storage.objects.delete",
    "storage.objects.get",
    "storage.objects.list",
  ]
}

但是,在部署这个之后,错误仍然是一样的。我想知道是否有必要在kubernetes端添加等效的权限,所以我也创建了下面的ClusterRole和ClusterRoleBinding。

resource "kubernetes_cluster_role" "cicd_bot" {
  metadata {
    name = kubernetes_service_account.cicd_bot.metadata[0].name
  }
  rule {
    api_groups = [""]
    resources  = ["namespaces"]
    verbs      = ["create", "delete", "get"]
  }
  rule {
    api_groups = [""]
    resources  = ["configmaps"]
    verbs      = ["list", "watch", "create", "update", "patch", "get", "delete"]
  }
  rule {
    api_groups = [""]
    resources  = ["pods"]
    verbs      = ["list", "watch", "create", "update", "patch", "get", "delete"]
  }
  rule {
    api_groups = [""]
    resources  = ["secrets"]
    verbs      = ["list", "watch", "create", "update", "patch", "get", "delete"]
  }
  rule {
    api_groups = [""]
    resources  = ["services"]
    verbs      = ["list", "watch", "create", "update", "patch", "get", "delete"]
  }
  rule {
    api_groups = ["apps"]
    resources  = ["statefulsets"]
    verbs      = ["list", "watch", "create", "update", "patch", "get", "delete"]
  }
  rule {
    api_groups = ["mongodbcommunity.mongodb.com"]
    resources  = ["mongodbcommunity"]
    verbs      = ["list", "watch", "update", "patch", "get"]
  }
  rule {
    api_groups = ["mongodbcommunity.mongodb.com"]
    resources  = ["mongodbcommunity/finalizers"]
    verbs      = ["list", "watch", "update", "patch", "get"]
  }
  rule {
    api_groups = ["mongodbcommunity.mongodb.com"]
    resources  = ["mongodbcommunity/spec"]
    verbs      = ["list", "watch", "update", "patch", "get"]
  }
  rule {
    api_groups = ["mongodbcommunity.mongodb.com"]
    resources  = ["mongodbcommunity/status"]
    verbs      = ["list", "watch", "update", "patch", "get"]
  }
}

resource "kubernetes_cluster_role_binding" "cicd_bot" {
  metadata {
    name = kubernetes_service_account.cicd_bot.metadata[0].name
  }
  subject {
    kind      = "ServiceAccount"
    namespace = kubernetes_service_account.cicd_bot.metadata[0].namespace
    name      = kubernetes_service_account.cicd_bot.metadata[0].name
  }
  role_ref {
    api_group = "rbac.authorization.k8s.io"
    kind      = "ClusterRole"
    name      = kubernetes_cluster_role.cicd_bot.metadata[0].name
  }
}

不幸的是,管道仍然因为同样的错误而失败。我过去曾经克服过类似的错误,但这次不行了。我错过了什么?
更新:通过将角色roles/container.admin附加到我的服务帐户,我能够成功地部署。因此,现在我需要确定roles/container.admin具有哪些权限,而我的自定义角色没有。

yjghlzjz

yjghlzjz1#

遗憾的是,缺少的一个权限是

container.roles.escalate

即使包括其他container.*权限也是不够的;仍然需要container.roles.escalate
这是不幸的,因为它使集群更容易受到权限提升攻击。如果有更安全的方法来实现这一点,我很乐意听到它。我不会标记我自己的答案为“正确”,因为我不满意它。但嘿,至少它是工作...

相关问题