Credential Provider Package with IAM Roles Anywhere

This tutorial demonstrates how to configure the credential provider package to authenticate using IAM Roles Anywhere to pull from a private AWS Elastic Container Registry (ECR).
IAM Roles Anywhere enables workloads outside of AWS to access AWS resources by using X.509 digital certificates to obtain temporary AWS credentials. A trust anchor is used to reference a certificate authority with IAM Roles Anywhere. For this use case, the Kubernetes Cluster CA can be registered and each kubelet client’s x509 cert can be used to authenticate to get temporary AWS credentials.

Prerequisites

  1. For setting up the certificate authority later, you will need to obtain your cluster’s CA. This can be obtain by:

    # Assuming CLUSTER_NAME and KUBECONFIG are set:
    kubectl get secret -n eksa-system ${CLUSTER_NAME}-ca -o yaml | yq '.data."tls.crt"' | base64 -d
    
  2. A role should be created to allow read access for curated packages. This role can be extended to include private registries that you would also like to pull from into your cluster. A sample policy for curated packages would be.

    {
     "Version": "2012-10-17",
     "Statement": [
         {
             "Sid": "ECRRead",
             "Effect": "Allow",
             "Action": [
                 "ecr:DescribeImageScanFindings",
                 "ecr:GetDownloadUrlForLayer",
                 "ecr:DescribeRegistry",
                 "ecr:DescribePullThroughCacheRules",
                 "ecr:DescribeImageReplicationStatus",
                 "ecr:ListTagsForResource",
                 "ecr:ListImages",
                 "ecr:BatchGetImage",
                 "ecr:DescribeImages",
                 "ecr:DescribeRepositories",
                 "ecr:BatchCheckLayerAvailability"
             ],
             "Resource": "arn:aws:ecr:*:783794618700:repository/*"
         },
         {
             "Sid": "ECRLogin",
             "Effect": "Allow",
             "Action": [
                 "ecr:GetAuthorizationToken"
             ],
             "Resource": "*"
         }
      ]
     }
    
  3. Next create a trust anchor and profile. The trust anchor will be a reference to the CA certificate from step 1 and the profile should point to the role created in step 2. See here for instructions on creating the trust anchor and profile.

  4. Create a secret that will be referenced by the credential-provider-package to authenticate the kubelet with ECR.

    # Set PROFILE_ARN, ROLE_ARN, and TRUST_ANCHOR_ARN obtained in previous step
    # Set AWS_REGION to region to pull images from
    # This will create a file credfile which will then be turned into a secret
    cat << EOF >> credfile
    [default]
    region = $AWS_REGION
    credential_process = aws_signing_helper credential-process --certificate /var/lib/kubelet/pki/kubelet-client-current.pem --private-key /var/lib/kubelet/pki/kubelet-client-current.pem --profile-arn $PROFILE_ARN --role-arn $ROLE_ARN --trust-anchor-arn $TRUST_ANCHOR_ARN
    EOF
    
    # Create secret, for this example the secret name aws-config is used and the package will be installed in eksa-packages
    kubectl create secret generic aws-config --from-file=config=credfile -n eksa-packages
    
  5. Either edit the existing package or delete and create a new credential-provider-package that points towards the new secret. For more information on specific configuration option refer to installation guide for details]
    The example below changes the default secret name from aws-secret to newly created aws-config. It also changes the match images to pull from multiple regions as well as across multiple accounts. Make sure to change cluster-name to match your CLUSTER_NAME

    apiVersion: packages.eks.amazonaws.com/v1alpha1
    kind: Package
    metadata:
      name: my-credential-provider-package
      namespace: eksa-packages-<clusterName>
      annotations:
        "helm.sh/resource-policy": keep
        "anywhere.eks.aws.com/internal": "true"
    spec:
      packageName: credential-provider-package
      targetNamespace: eksa-packages
      config: |-
        tolerations:
          - key: "node-role.kubernetes.io/master"
            operator: "Exists"
            effect: "NoSchedule"
          - key: "node-role.kubernetes.io/control-plane"
            operator: "Exists"
            effect: "NoSchedule"
        sourceRegistry: public.ecr.aws/eks-anywhere
        credential:
          - matchImages:
            - *.dkr.ecr.*.amazonaws.com
            profile: "default"
            secretName: aws-config
            defaultCacheDuration: "5h"