kubernetes 条件“尝试批准具有”certificates.k8s.io/v1“的CSR失败”

72qzrwbm  于 2022-12-17  发布在  Kubernetes
关注(0)|答案(3)|浏览(133)

我有一个正在运行的ValidatingWebhookConfiguration,并且一直在创建|批准具有certificates.k8s.io/v1beta的CSR。
我将(MicroK 8 s)从1.18升级到1.20,并收到certificates.k8s.io/v1beta在1.19+中已过时的警告,我想我应该尝试(没有成功)升级到certificates.k8s.io/v1
现有(工作)CSR:

apiVersion: certificates.k8s.io/v1beta1
kind: CertificateSigningRequest
metadata:
  name: ${SERVICE}.${NAMESPACE}
spec:
  groups:
  - system:authenticated
  request: $(cat ${FILENAME}.csr | base64 | tr -d '\n')
  usages:
  - digital signature
  - key encipherment
  - server auth

升级API时生成错误:

missing required field "signerName" in io.k8s.api.certificates.v1.CertificateSigningRequestSpec;

我阅读了CSR文档,特别是关于Kubernetes签名者的部分,因为我现有的规范使用server auth,所以假设我可以使用kubernetes.io/kubelet-serving,因为这是唯一允许server auth的规范。

apiVersion: certificates.k8s.io/v1                      <<--- UPGRADED
kind: CertificateSigningRequest
metadata:
  name: ${SERVICE}.${NAMESPACE}
spec:
  groups:
  - system:authenticated
  request: $(cat ${FILENAME}.csr | base64 | tr -d '\n')
  signerName: kubernetes.io/kubelet-serving              <<--- ADDED
  usages:
  - digital signature
  - key encipherment
  - server auth

但是,我在尝试批准CSR(作为群集管理员)时遇到错误:

kubectl certificate approve ${SERVICE}.${NAMESPACE}
certificatesigningrequest.certificates.k8s.io/${SERVICE}.${NAMESPACE} approved

kubectl get csr ${SERVICE}.${NAMESPACE}
NAME                      SIGNERNAME                      REQUESTOR   CONDITION
${SERVICE}.${NAMESPACE}   kubernetes.io/kubelet-serving   admin       Approved,Failed

x为1米6英寸,但x为1米7英寸

我无法获得证书(可能是因为它是Failed):

kubectl get csr ${SERVICE}.${NAMESPACE} \
--output=jsonpath='{.status.certificate}'

我应该如何使用certificates.k8s.io/v1 API?

更新:2021-01-06

好吧,所以我意识到我有更多关于“失败”的信息这给了我一些调查的东西...

kubectl get csr/${SERVICE}.${NAMESPACE} \
--output=jsonpath="{.status}" \
| jq .

产量:

{
  "conditions": [
    {
      "lastTransitionTime": "2021-01-06T18:52:15Z",
      "lastUpdateTime": "2021-01-06T18:52:15Z",
      "message": "This CSR was approved by kubectl certificate approve.",
      "reason": "KubectlApprove",
      "status": "True",
      "type": "Approved"
    },
    {
      "lastTransitionTime": "2021-01-06T18:52:15Z",
      "lastUpdateTime": "2021-01-06T18:52:15Z",
      "message": "subject organization is not system:nodes",
      "reason": "SignerValidationFailure",
      "status": "True",
      "type": "Failed"
    }
  ]
}

更新:2021-01-07

谢谢@Pjoters

ubectl describe csr/${SERVICE}.${NAMESPACE}
Name:         eldlund.utopial
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"certificates.k8s.io/v1","kind":"CertificateSigningRequest","metadata":{"annotations":{},"name":"eldlund.utopial"},"spec":{"groups":["system:authenticated"],"request":"LS0tLS1C...LS0tLS0K","signerName":"kubernetes.io/kubelet-serving","usages":["digital signature","key encipherment","server auth"]}}

CreationTimestamp:  Thu, 07 Jan 2021 17:03:23 +0000
Requesting User:    admin
Signer:             kubernetes.io/kubelet-serving
Status:             Pending
Subject:
  Common Name:    eldlund.utopial.svc
  Serial Number:  
Subject Alternative Names:
         DNS Names:  eldlund.utopial.svc
                     eldlund.utopial.svc.cluster.local
Events:  <none>

使用OpenSSL(而非Kubernetes)签名

我尝试创建CA crt|密钥,然后是服务密钥|CSR并与CA签署服务CSR,但Kubernetes抱怨:

x509: certificate is not valid for any names, but wanted to match ainsley.utopial.svc

然而,证书似乎同时包含CN和SAN条目:

不起作用

openssl x509 -in ${FILENAME}.crt --noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            6f:14:25:8c:...
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = Validating Webhook CA
        Validity
            Not Before: Jan  7 18:10:50 2021 GMT
            Not After : Feb  6 18:10:50 2021 GMT
        Subject: CN = ainsley.utopial.svc
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:ca:56:15:...
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Alternative Name: 
                DNS:ainsley.utopial.svc, DNS:ainsley.utopial.svc.cluster.local
    Signature Algorithm: sha256WithRSAEncryption
         b2:ec:22:b6:...

注意CN是上面的DNS名称,但下面的IP是???

返回到我的工作解决方案v1beta1,并更改服务名以确保完整性(loi),Webhook成功,证书看起来与上面显示的证书没有什么不同(除了服务名不同):

工程

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            ff:b3:cb:11:...
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = 10.152.183.1
        Validity
            Not Before: Jan  7 18:18:45 2021 GMT
            Not After : Jan  7 18:18:45 2022 GMT
        Subject: CN = loi.utopial.svc
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:d2:cc:c2:...
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Authority Key Identifier: 
                keyid:E7:AE:3A:25:95:D2:F7:5B:C6:EA:50:56:07:E8:25:83:60:88:68:7A

            X509v3 Subject Alternative Name: 
                DNS:loi.utopial.svc, DNS:loi.utopial.svc.cluster.local
    Signature Algorithm: sha256WithRSAEncryption
         48:a1:b2:e2:...
nzk0hqpo

nzk0hqpo1#

更新已切换到cert-manager,一切正常

我让它工作,但我不知道为什么我现在做的是正确的。
而且openssl感觉很笨拙(建议很感谢)。
环境

DIR=${PWD}/secrets
SERVICE="..."
NAMESPACE="..."

FILENAME="${DIR}/${SERVICE}.${NAMESPACE}"

加利福尼亚州

openssl req \
-nodes \
-new \
-x509 \
-keyout ${FILENAME}.ca.key \
-out ${FILENAME}.ca.crt \
-subj "/CN=Validating Webhook CA"

创建(Webhook)服务

需要将服务证书的CN设置为IP

cat ./kubernetes/service.yaml \
| sed "s|SERVICE|${SERVICE}|g" \
| sed "s|NAMESPACE|${NAMESPACE}|g" \
| kubectl apply --filename=- --namespace=${NAMESPACE}

ENDPOINT=$(\
  kubectl get service/${SERVICE} \
  --namespace=${NAMESPACE} \
  --output=jsonpath="{.spec.clusterIP}") && echo ${ENDPOINT}

创建CSR

即使我在这里包括了CN和alt_names,我也必须复制SAN内容(下一步)

echo "[ req ]
default_bits = 2048
prompt = no
default_md = sha256
distinguished_name = dn
req_extensions = req_ext

[ dn ]
commonName = ${ENDPOINT}

[ req_ext ]
subjectAltName = @alt_names

[alt_names]
DNS.1 = ${SERVICE}.${NAMESPACE}.svc
DNS.2 = ${SERVICE}.${NAMESPACE}.svc.cluster.local
" > ${FILENAME}.cfg

openssl req \
-nodes \
-new \
-sha256 \
-newkey rsa:2048 \
-keyout ${FILENAME}.key \
-out ${FILENAME}.csr \
-config ${FILENAME}.cfg

创建CSR扩展

不确定为什么必须复制(或分离)此内容。如果在openssl x509 -extfile中省略此内容,则证书不包含SAN扩展。

printf "subjectAltName=DNS:${SERVICE}.${NAMESPACE}.svc,DNS:${SERVICE}.${NAMESPACE}.svc.cluster.local" > ${FILENAME}.ext

创建服务证书

如何使用单个CSR而不是CSR+EXT来处理所有问题?

openssl x509 \
-req \
-in ${FILENAME}.csr \
-extfile ${FILENAME}.ext \
-CA ${FILENAME}.ca.crt \
-CAkey ${FILENAME}.ca.key \
-CAcreateserial \
-out ${FILENAME}.crt

创建(Webhook)部署

webhook的底层实现需要服务的crt|键

kubectl create secret tls ${SERVICE} \
--namespace=${NAMESPACE} \
--cert=${FILENAME}.crt \
--key=${FILENAME}.key

cat ./kubernetes/deployment.yaml \
| sed "s|SERVICE|${SERVICE}|g" \
| sed "s|NAMESPACE|${NAMESPACE}|g" \
| kubectl apply --filename=- --namespace=${NAMESPACE}

创建网钩

获取CA证书

CABUNDLE=$(openssl base64 -A <"${FILENAME}.ca.crt")

cat ./kubernetes/webhook.yaml \
| sed "s|SERVICE|${SERVICE}|g" \
| sed "s|NAMESPACE|${NAMESPACE}|g" \
| sed "s|CABUNDLE|${CABUNDLE}|g" \
| kubectl apply --filename=- --namespace=${NAMESPACE}
ndasle7k

ndasle7k2#

顺便说一句,通过用类似kubelet的请求来模拟你的应用程序CSR,你可以继续创建k8s证书。我在这个问题https://github.com/kubernetes/website/issues/26111中有详细信息,试图让k8s文档在这个主题上更有用。

ohfgkhjo

ohfgkhjo3#

在从OpenSSL生成.csr时,您可以按如下方式在CN(即通用名称)中添加“system:node”和“O=system:nodes”,然后使用升级后的certificates.k8s.io/v1。

  • 更改前的OpenSSL命令 * openssl请求-新建-密钥${tmpdir}/服务器密钥.pem -主题“/CN=${title}.${namespace}.svc”-输出${tmpdir}/服务器.csr -配置${tmpdir}/csr.conf
  • 更改OpenSSL命令后 * openssl请求-新-密钥${tmpdir}/服务器-密钥.pem -subj“/CN=系统:节点:${title}.${namespace}.svc;/O=系统:节点”-输出${tmpdir}/服务器.csr -配置${tmpdir}/csr.conf

相关问题