Skip to content

Conversation

@iftachk
Copy link
Contributor

@iftachk iftachk commented Nov 18, 2025

No description provided.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. area/infra Infrastructure management, infrastructure design, code in infra/ area/infra/aws Issues or PRs related to Kubernetes AWS infrastructure sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. labels Nov 18, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: iftachk
Once this PR has been reviewed and has the lgtm label, please assign xmudrii for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Nov 18, 2025
@upodroid
Copy link
Member

atlantis plan

@k8s-infra-ci-robot
Copy link
Contributor

Ran Plan for dir: infra/aws/terraform/kops-infra-ci workspace: default

Plan Error

Show Output
running 'sh -c' '/usr/local/bin/terraform1.9.8 plan -input=false -refresh -out "/atlantis/repos/kubernetes/k8s.io/8763/default/infra/aws/terraform/kops-infra-ci/default.tfplan"' in '/atlantis/repos/kubernetes/k8s.io/8763/default/infra/aws/terraform/kops-infra-ci': exit status 1
data.aws_iam_roles.sso_admins: Reading...
data.aws_organizations_organization.current: Reading...
data.aws_iam_roles.sso_admins: Read complete after 0s [id=us-east-2]
module.ebs_csi_irsa.data.aws_partition.current[0]: Reading...
data.aws_region.current: Reading...
module.ci_iam_group.data.aws_caller_identity.current[0]: Reading...
module.eks.module.kms.data.aws_partition.current[0]: Reading...
module.eks.data.aws_iam_policy_document.assume_role_policy[0]: Reading...
module.vpc.data.aws_region.current[0]: Reading...
module.eks.data.aws_caller_identity.current[0]: Reading...
module.vpc_cni_irsa.data.aws_partition.current[0]: Reading...
module.ci_iam_group.aws_iam_group.this[0]: Refreshing state... [id=ci-admins]
module.vpc.data.aws_region.current[0]: Read complete after 0s [id=us-east-2]
module.ebs_csi_irsa.data.aws_partition.current[0]: Read complete after 0s [id=aws]
data.aws_region.current: Read complete after 0s [id=us-east-2]
module.vpc_cni_irsa.data.aws_partition.current[0]: Read complete after 0s [id=aws]
module.eks.module.kms.data.aws_partition.current[0]: Read complete after 0s [id=aws]
module.eks.data.aws_iam_policy_document.assume_role_policy[0]: Read complete after 0s [id=2830595799]
module.ci_iam_group.data.aws_partition.current: Reading...
data.aws_iam_policy_document.eks_pod_identity_policy: Reading...
module.eks.data.aws_partition.current[0]: Reading...
data.aws_availability_zones.available: Reading...
module.vpc.data.aws_caller_identity.current[0]: Reading...
module.vpc.data.aws_partition.current[0]: Reading...
module.eks.data.aws_partition.current[0]: Read complete after 0s [id=aws]
module.ci_iam_group.data.aws_partition.current: Read complete after 0s [id=aws]
module.vpc.data.aws_partition.current[0]: Read complete after 0s [id=aws]
data.aws_iam_policy_document.eks_pod_identity_policy: Read complete after 0s [id=819195744]
module.eks.module.kms.data.aws_caller_identity.current[0]: Reading...
module.vpc.data.aws_iam_policy_document.flow_log_cloudwatch_assume_role[0]: Reading...
module.cluster_autoscaler_irsa_role.data.aws_partition.current[0]: Reading...
data.aws_caller_identity.current: Reading...
module.cluster_autoscaler_irsa_role.data.aws_partition.current[0]: Read complete after 0s [id=aws]
module.vpc.data.aws_iam_policy_document.flow_log_cloudwatch_assume_role[0]: Read complete after 0s [id=1021377347]
aws_vpc_ipam.main: Refreshing state... [id=ipam-0977c2d0978192252]
module.ci_iam_group.data.aws_caller_identity.current[0]: Read complete after 0s [id=348685125169]
aws_iam_openid_connect_provider.google_prow_idp: Refreshing state... [id=arn:aws:iam::808842816990:oidc-provider/container.googleapis.com/v1/projects/k8s-prow/locations/us-central1-f/clusters/prow]
aws_s3_bucket.kops_state_store: Refreshing state... [id=k8s-kops-ci-prow-state-store]
module.eks.data.aws_caller_identity.current[0]: Read complete after 0s [id=348685125169]
aws_s3_bucket.kops_oidc_store: Refreshing state... [id=k8s-kops-ci-prow]
module.vpc.data.aws_caller_identity.current[0]: Read complete after 0s [id=348685125169]
aws_iam_role.eks_pod_identity_role: Refreshing state... [id=EKSPodIdentityRole]
module.eks.module.kms.data.aws_caller_identity.current[0]: Read complete after 0s [id=348685125169]
module.ebs_csi_irsa.data.aws_iam_policy_document.ebs_csi[0]: Reading...
data.aws_caller_identity.current: Read complete after 0s [id=348685125169]
module.ebs_csi_irsa.data.aws_iam_policy_document.ebs_csi[0]: Read complete after 0s [id=4189668531]
module.vpc.aws_iam_role.vpc_flow_log_cloudwatch[0]: Refreshing state... [id=vpc-flow-log-role-20230808165034642700000002]
module.eks.aws_iam_role.this[0]: Refreshing state... [id=k8s-infra-kops-prow-build-cluster-20230808174304353200000004]
module.vpc_cni_irsa.data.aws_iam_policy_document.vpc_cni[0]: Reading...
data.aws_availability_zones.available: Read complete after 0s [id=us-east-2]
module.vpc_cni_irsa.data.aws_iam_policy_document.vpc_cni[0]: Read complete after 0s [id=1079910255]
module.eks.aws_cloudwatch_log_group.this[0]: Refreshing state... [id=/aws/eks/k8s-infra-kops-prow-build/cluster]
module.eks.data.aws_iam_session_context.current[0]: Reading...
module.kops_ci_user.aws_iam_user.this[0]: Refreshing state... [id=kops-ci-user]
module.ci_iam_group.data.aws_iam_policy_document.iam_self_management: Reading...
module.ci_iam_group.data.aws_iam_policy_document.iam_self_management: Read complete after 0s [id=888356565]
module.ci_iam_group.aws_iam_group_policy_attachment.custom_arns[0]: Refreshing state... [id=ci-admins-20250922120814428600000002]
module.ci_iam_group.aws_iam_policy.iam_self_management[0]: Refreshing state... [id=arn:aws:iam::808842816990:policy/IAMSelfManagement-20250922120813674200000001]
module.ebs_csi_irsa.data.aws_iam_policy_document.this[0]: Reading...
module.vpc_cni_irsa.data.aws_iam_policy_document.this[0]: Reading...
module.vpc_cni_irsa.data.aws_iam_policy_document.this[0]: Read complete after 0s [id=1079910255]
module.ebs_csi_irsa.data.aws_iam_policy_document.this[0]: Read complete after 0s [id=4189668531]
aws_iam_role_policy_attachment.eks_pod_identity_policy: Refreshing state... [id=EKSPodIdentityRole-20231201210421345600000001]
module.vpc_cni_irsa.aws_iam_policy.this[0]: Refreshing state... [id=arn:aws:iam::808842816990:policy/VPC_CNI_IPv4-20251016085047699500000001]
module.ebs_csi_irsa.aws_iam_policy.this[0]: Refreshing state... [id=arn:aws:iam::808842816990:policy/EBS_CSI-20251016084828254900000001]
module.eks.aws_iam_role_policy_attachment.this["AmazonEKSClusterPolicy"]: Refreshing state... [id=k8s-infra-kops-prow-build-cluster-20230808174304353200000004-2023080817430568360000000e]
module.eks.data.aws_iam_session_context.current[0]: Read complete after 0s [id=arn:aws:sts::348685125169:assumed-role/atlantis/1763470458794565738]
module.kops_ci_user.aws_iam_access_key.this_no_pgp[0]: Refreshing state... [id=AKIA3YUWJ5XPHTELQU62]
module.eks.module.kms.aws_kms_key.this[0]: Refreshing state... [id=6abd02ac-74e1-4268-af15-46290f75b193]
aws_vpc_ipam_pool.main: Refreshing state... [id=ipam-pool-0cefae006ed9905cc]
aws_vpc_ipam_scope.main: Refreshing state... [id=ipam-scope-0acd86d69916beff7]
module.ci_iam_group.aws_iam_group_membership.this[0]: Refreshing state... [id=ci-admins]
module.ci_iam_group.aws_iam_group_policy_attachment.iam_self_management[0]: Refreshing state... [id=ci-admins-20250922120814511600000003]
aws_vpc_ipam_preview_next_cidr.main: Refreshing state... [id=10.128.0.0/20_ipam-pool-0cefae006ed9905cc]
aws_vpc_ipam_pool_cidr.main: Refreshing state... [id=10.128.0.0/16_ipam-pool-0cefae006ed9905cc]
module.vpc.aws_vpc.this[0]: Refreshing state... [id=vpc-0245c64d94e931733]
module.eks.module.kms.aws_kms_alias.this["cluster"]: Refreshing state... [id=alias/eks/k8s-infra-kops-prow-build]
module.eks.aws_iam_policy.cluster_encryption[0]: Refreshing state... [id=arn:aws:iam::808842816990:policy/k8s-infra-kops-prow-build-cluster-ClusterEncryption2023080817432884320000000f]
module.eks.aws_iam_role_policy_attachment.cluster_encryption[0]: Refreshing state... [id=k8s-infra-kops-prow-build-cluster-20230808174304353200000004-20230808174329378200000010]
module.vpc.aws_default_security_group.this[0]: Refreshing state... [id=sg-0371829023d3c0434]
module.eks.aws_security_group.cluster[0]: Refreshing state... [id=sg-0f18cdd53efb544cb]
module.vpc.aws_default_route_table.default[0]: Refreshing state... [id=rtb-0187095ac125b632e]
module.vpc_endpoints.aws_security_group.this[0]: Refreshing state... [id=sg-0abee7c9c53d92aa4]
module.vpc.aws_default_network_acl.this[0]: Refreshing state... [id=acl-020ee7ceef539578c]
module.eks.aws_security_group.node[0]: Refreshing state... [id=sg-0b12625763689d8c0]
module.vpc.aws_cloudwatch_log_group.flow_log[0]: Refreshing state... [id=/aws/vpc-flow-log/vpc-0245c64d94e931733]
data.aws_organizations_organization.current: Read complete after 1s [id=o-kz4vlkihvy]
module.vpc.aws_flow_log.this[0]: Refreshing state... [id=fl-02f31515eecb7b61b]
module.vpc.aws_iam_policy.vpc_flow_log_cloudwatch[0]: Refreshing state... [id=arn:aws:iam::808842816990:policy/vpc-flow-log-to-cloudwatch-20230808165034642300000001]
module.vpc.aws_iam_role_policy_attachment.vpc_flow_log_cloudwatch[0]: Refreshing state... [id=vpc-flow-log-role-20230808165034642700000002-20230808165035586800000003]
module.vpc_endpoints.aws_security_group_rule.this["ingress_https"]: Refreshing state... [id=sgrule-1387374841]
module.eks.aws_security_group_rule.node["ingress_cluster_9443_webhook"]: Refreshing state... [id=sgrule-3331351739]
module.eks.aws_security_group_rule.node["ingress_cluster_4443_webhook"]: Refreshing state... [id=sgrule-2016349383]
module.eks.aws_security_group_rule.node["ingress_cluster_8443_webhook"]: Refreshing state... [id=sgrule-2721142640]
module.eks.aws_security_group_rule.node["ingress_self_coredns_tcp"]: Refreshing state... [id=sgrule-1236246780]
module.eks.aws_security_group_rule.node["ingress_cluster_443"]: Refreshing state... [id=sgrule-3012627233]
module.eks.aws_security_group_rule.node["ingress_self_coredns_udp"]: Refreshing state... [id=sgrule-3991929619]
module.eks.aws_security_group_rule.node["ingress_cluster_kubelet"]: Refreshing state... [id=sgrule-1426706750]
module.eks.aws_security_group_rule.node["ingress_cluster_6443_webhook"]: Refreshing state... [id=sgrule-2976738129]
module.eks.aws_security_group_rule.node["egress_all"]: Refreshing state... [id=sgrule-2839492080]
module.eks.aws_security_group_rule.node["ingress_nodes_ephemeral"]: Refreshing state... [id=sgrule-2689944412]
module.eks.aws_security_group_rule.cluster["ingress_nodes_443"]: Refreshing state... [id=sgrule-4148544704]

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the
last "terraform apply" which may have affected this plan:

  # aws_iam_role.eks_pod_identity_role has been deleted
  - resource "aws_iam_role" "eks_pod_identity_role" {
        id                    = "EKSPodIdentityRole"
      - name                  = "EKSPodIdentityRole" -> null
        tags                  = {
            "Shared"     = "Ignore"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
        # (12 unchanged attributes hidden)
    }

  # aws_vpc_ipam.main has been deleted
  - resource "aws_vpc_ipam" "main" {
      - id                                        = "ipam-0977c2d0978192252" -> null
      - private_default_scope_id                  = "ipam-scope-0d83bef2d8f6ebe0b" -> null
        tags                                      = {
            "Shared"     = "Ignore"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
        # (11 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # aws_vpc_ipam_pool.main has been deleted
  - resource "aws_vpc_ipam_pool" "main" {
      - id                       = "ipam-pool-0cefae006ed9905cc" -> null
        tags                     = {
            "Shared"     = "Ignore"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
        # (16 unchanged attributes hidden)
    }

  # module.ci_iam_group.aws_iam_group.this[0] has been deleted
  - resource "aws_iam_group" "this" {
      - arn       = "arn:aws:iam::808842816990:group/ci-admins" -> null
      - id        = "ci-admins" -> null
      - name      = "ci-admins" -> null
        # (2 unchanged attributes hidden)
    }

  # module.ci_iam_group.aws_iam_policy.iam_self_management[0] has been deleted
  - resource "aws_iam_policy" "iam_self_management" {
      - arn              = "arn:aws:iam::808842816990:policy/IAMSelfManagement-20250922120813674200000001" -> null
        id               = "arn:aws:iam::808842816990:policy/IAMSelfManagement-20250922120813674200000001"
        name             = "IAMSelfManagement-20250922120813674200000001"
        tags             = {
            "Shared"     = "Ignore"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
        # (7 unchanged attributes hidden)
    }

  # module.ebs_csi_irsa.aws_iam_policy.this[0] has been deleted
  - resource "aws_iam_policy" "this" {
      - arn              = "arn:aws:iam::808842816990:policy/EBS_CSI-20251016084828254900000001" -> null
        id               = "arn:aws:iam::808842816990:policy/EBS_CSI-20251016084828254900000001"
        name             = "EBS_CSI-20251016084828254900000001"
      - policy           = jsonencode(
            {
              - Statement = [
                  - {
                      - Action   = [
                          - "ec2:DescribeVolumesModifications",
                          - "ec2:DescribeVolumes",
                          - "ec2:DescribeTags",
                          - "ec2:DescribeSnapshots",
                          - "ec2:DescribeInstances",
                          - "ec2:DescribeAvailabilityZones",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                  - {
                      - Action   = [
                          - "ec2:ModifyVolume",
                          - "ec2:CreateSnapshot",
                        ]
                      - Effect   = "Allow"
                      - Resource = "arn:aws:ec2:*:*:volume/*"
                    },
                  - {
                      - Action   = [
                          - "ec2:DetachVolume",
                          - "ec2:AttachVolume",
                        ]
                      - Effect   = "Allow"
                      - Resource = [
                          - "arn:aws:ec2:*:*:volume/*",
                          - "arn:aws:ec2:*:*:instance/*",
                        ]
                    },
                  - {
                      - Action   = [
                          - "ec2:EnableFastSnapshotRestores",
                          - "ec2:CreateVolume",
                        ]
                      - Effect   = "Allow"
                      - Resource = "arn:aws:ec2:*:*:snapshot/*"
                    },
                  - {
                      - Action    = "ec2:CreateTags"
                      - Condition = {
                          - StringEquals = {
                              - "ec2:CreateAction" = [
                                  - "CreateVolume",
                                  - "CreateSnapshot",
                                ]
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = [
                          - "arn:aws:ec2:*:*:volume/*",
                          - "arn:aws:ec2:*:*:snapshot/*",
                        ]
                    },
                  - {
                      - Action   = "ec2:DeleteTags"
                      - Effect   = "Allow"
                      - Resource = [
                          - "arn:aws:ec2:*:*:volume/*",
                          - "arn:aws:ec2:*:*:snapshot/*",
                        ]
                    },
                  - {
                      - Action    = "ec2:CreateVolume"
                      - Condition = {
                          - StringLike = {
                              - "aws:RequestTag/ebs.csi.aws.com/cluster" = "true"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "arn:aws:ec2:*:*:volume/*"
                    },
                  - {
                      - Action    = "ec2:CreateVolume"
                      - Condition = {
                          - StringLike = {
                              - "aws:RequestTag/CSIVolumeName" = "*"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "arn:aws:ec2:*:*:volume/*"
                    },
                  - {
                      - Action    = "ec2:DeleteVolume"
                      - Condition = {
                          - StringLike = {
                              - "aws:ResourceTag/ebs.csi.aws.com/cluster" = "true"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "arn:aws:ec2:*:*:volume/*"
                    },
                  - {
                      - Action    = "ec2:DeleteVolume"
                      - Condition = {
                          - StringLike = {
                              - "aws:ResourceTag/CSIVolumeName" = "*"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "arn:aws:ec2:*:*:volume/*"
                    },
                  - {
                      - Action    = "ec2:DeleteVolume"
                      - Condition = {
                          - StringLike = {
                              - "ec2:ResourceTag/kubernetes.io/created-for/pvc/name" = "*"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "arn:aws:ec2:*:*:volume/*"
                    },
                  - {
                      - Action    = "ec2:CreateSnapshot"
                      - Condition = {
                          - StringLike = {
                              - "aws:RequestTag/CSIVolumeSnapshotName" = "*"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "arn:aws:ec2:*:*:snapshot/*"
                    },
                  - {
                      - Action    = "ec2:CreateSnapshot"
                      - Condition = {
                          - StringLike = {
                              - "aws:RequestTag/ebs.csi.aws.com/cluster" = "true"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "arn:aws:ec2:*:*:snapshot/*"
                    },
                  - {
                      - Action    = "ec2:DeleteSnapshot"
                      - Condition = {
                          - StringLike = {
                              - "aws:ResourceTag/CSIVolumeSnapshotName" = "*"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "arn:aws:ec2:*:*:snapshot/*"
                    },
                  - {
                      - Action    = "ec2:DeleteSnapshot"
                      - Condition = {
                          - StringLike = {
                              - "aws:ResourceTag/ebs.csi.aws.com/cluster" = "true"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "arn:aws:ec2:*:*:snapshot/*"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> null
        tags             = {
            "Shared"     = "Ignore"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
        # (6 unchanged attributes hidden)
    }

  # module.eks.aws_cloudwatch_log_group.this[0] has been deleted
  - resource "aws_cloudwatch_log_group" "this" {
      - arn               = "arn:aws:logs:us-east-2:808842816990:log-group:/aws/eks/k8s-infra-kops-prow-build/cluster" -> null
        id                = "/aws/eks/k8s-infra-kops-prow-build/cluster"
      - name              = "/aws/eks/k8s-infra-kops-prow-build/cluster" -> null
        tags              = {
            "Name"       = "/aws/eks/k8s-infra-kops-prow-build/cluster"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
        # (7 unchanged attributes hidden)
    }

  # module.eks.aws_iam_policy.cluster_encryption[0] has been deleted
  - resource "aws_iam_policy" "cluster_encryption" {
      - arn              = "arn:aws:iam::808842816990:policy/k8s-infra-kops-prow-build-cluster-ClusterEncryption2023080817432884320000000f" -> null
        id               = "arn:aws:iam::808842816990:policy/k8s-infra-kops-prow-build-cluster-ClusterEncryption2023080817432884320000000f"
        name             = "k8s-infra-kops-prow-build-cluster-ClusterEncryption2023080817432884320000000f"
        tags             = {
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
        # (7 unchanged attributes hidden)
    }

  # module.eks.aws_iam_role.this[0] has been deleted
  - resource "aws_iam_role" "this" {
      - arn                   = "arn:aws:iam::808842816990:role/k8s-infra-kops-prow-build-cluster-20230808174304353200000004" -> null
        id                    = "k8s-infra-kops-prow-build-cluster-20230808174304353200000004"
      - name                  = "k8s-infra-kops-prow-build-cluster-20230808174304353200000004" -> null
        tags                  = {
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
      - unique_id             = "AROA3YUWJ5XPFDEQTZ24I" -> null
        # (10 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.eks.aws_security_group.cluster[0] has been deleted
  - resource "aws_security_group" "cluster" {
      - arn                    = "arn:aws:ec2:us-east-2:808842816990:security-group/sg-0f18cdd53efb544cb" -> null
      - id                     = "sg-0f18cdd53efb544cb" -> null
        name                   = "k8s-infra-kops-prow-build-cluster-20230808174304354400000006"
        tags                   = {
            "Name"       = "k8s-infra-kops-prow-build-cluster"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
        # (9 unchanged attributes hidden)
    }

  # module.eks.aws_security_group.node[0] has been deleted
  - resource "aws_security_group" "node" {
      - arn                    = "arn:aws:ec2:us-east-2:808842816990:security-group/sg-0b12625763689d8c0" -> null
      - id                     = "sg-0b12625763689d8c0" -> null
        name                   = "k8s-infra-kops-prow-build-node-20230808174304353400000005"
        tags                   = {
            "Name"                                            = "k8s-infra-kops-prow-build-node"
            "githubRepo"                                      = "git.k8s.io/k8s.io"
            "group"                                           = "sig-cluster-lifecycle"
            "kubernetes.io/cluster/k8s-infra-kops-prow-build" = "owned"
            "managed-by"                                      = "Terraform"
            "region"                                          = "us-east-2"
            "subproject"                                      = "kops"
        }
        # (9 unchanged attributes hidden)
    }

  # module.kops_ci_user.aws_iam_access_key.this_no_pgp[0] has been deleted
  - resource "aws_iam_access_key" "this_no_pgp" {
      - id                   = "AKIA3YUWJ5XPHTELQU62" -> null
      - secret               = (sensitive value) -> null
      - ses_smtp_password_v4 = (sensitive value) -> null
      - status               = "Active" -> null
        # (2 unchanged attributes hidden)
    }

  # module.kops_ci_user.aws_iam_user.this[0] has changed
  ~ resource "aws_iam_user" "this" {
      ~ arn                  = "arn:aws:iam::808842816990:user/kops-ci-user" -> "arn:aws:iam::348685125169:user/kops-ci-user"
        id                   = "kops-ci-user"
        name                 = "kops-ci-user"
        tags                 = {
            "Shared"     = "Ignore"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
      ~ unique_id            = "AIDA3YUWJ5XPLI357O2XB" -> "AIDAVCL2AYYY6UXGZCWL5"
        # (4 unchanged attributes hidden)
    }

  # module.vpc.aws_cloudwatch_log_group.flow_log[0] has been deleted
  - resource "aws_cloudwatch_log_group" "flow_log" {
      - arn               = "arn:aws:logs:us-east-2:808842816990:log-group:/aws/vpc-flow-log/vpc-0245c64d94e931733" -> null
        id                = "/aws/vpc-flow-log/vpc-0245c64d94e931733"
        name              = "/aws/vpc-flow-log/vpc-0245c64d94e931733"
        tags              = {
            "Shared"     = "Ignore"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
        # (7 unchanged attributes hidden)
    }

  # module.vpc.aws_flow_log.this[0] has been deleted
  - resource "aws_flow_log" "this" {
      - id                         = "fl-02f31515eecb7b61b" -> null
        tags                       = {
            "Shared"     = "Ignore"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
        # (11 unchanged attributes hidden)
    }

  # module.vpc.aws_iam_policy.vpc_flow_log_cloudwatch[0] has been deleted
  - resource "aws_iam_policy" "vpc_flow_log_cloudwatch" {
      - arn              = "arn:aws:iam::808842816990:policy/vpc-flow-log-to-cloudwatch-20230808165034642300000001" -> null
        id               = "arn:aws:iam::808842816990:policy/vpc-flow-log-to-cloudwatch-20230808165034642300000001"
        name             = "vpc-flow-log-to-cloudwatch-20230808165034642300000001"
        tags             = {
            "Shared"     = "Ignore"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
        # (7 unchanged attributes hidden)
    }

  # module.vpc.aws_iam_role.vpc_flow_log_cloudwatch[0] has been deleted
  - resource "aws_iam_role" "vpc_flow_log_cloudwatch" {
      - arn                   = "arn:aws:iam::808842816990:role/vpc-flow-log-role-20230808165034642700000002" -> null
        id                    = "vpc-flow-log-role-20230808165034642700000002"
      - name                  = "vpc-flow-log-role-20230808165034642700000002" -> null
        tags                  = {
            "Shared"     = "Ignore"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
        # (11 unchanged attributes hidden)
    }

  # module.vpc.aws_vpc.this[0] has been deleted
  - resource "aws_vpc" "this" {
      - arn                                  = "arn:aws:ec2:us-east-2:808842816990:vpc/vpc-0245c64d94e931733" -> null
      - cidr_block                           = "10.128.0.0/20" -> null
      - default_network_acl_id               = "acl-020ee7ceef539578c" -> null
      - default_route_table_id               = "rtb-0187095ac125b632e" -> null
      - default_security_group_id            = "sg-0371829023d3c0434" -> null
      - enable_dns_hostnames                 = true -> null
      - enable_dns_support                   = true -> null
      - id                                   = "vpc-0245c64d94e931733" -> null
      - instance_tenancy                     = "default" -> null
      - main_route_table_id                  = "rtb-0187095ac125b632e" -> null
      - owner_id                             = "808842816990" -> null
        tags                                 = {
            "Name"       = "k8s-infra-kops-vpc"
            "Shared"     = "Ignore"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
        # (11 unchanged attributes hidden)
    }

  # module.vpc_cni_irsa.aws_iam_policy.this[0] has been deleted
  - resource "aws_iam_policy" "this" {
      - arn              = "arn:aws:iam::808842816990:policy/VPC_CNI_IPv4-20251016085047699500000001" -> null
        id               = "arn:aws:iam::808842816990:policy/VPC_CNI_IPv4-20251016085047699500000001"
        name             = "VPC_CNI_IPv4-20251016085047699500000001"
      - policy           = jsonencode(
            {
              - Statement = [
                  - {
                      - Action   = [
                          - "ec2:UnassignPrivateIpAddresses",
                          - "ec2:ModifyNetworkInterfaceAttribute",
                          - "ec2:DetachNetworkInterface",
                          - "ec2:DescribeTags",
                          - "ec2:DescribeSubnets",
                          - "ec2:DescribeNetworkInterfaces",
                          - "ec2:DescribeInstances",
                          - "ec2:DescribeInstanceTypes",
                          - "ec2:DeleteNetworkInterface",
                          - "ec2:CreateNetworkInterface",
                          - "ec2:AttachNetworkInterface",
                          - "ec2:AssignPrivateIpAddresses",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                      - Sid      = "IPV4"
                    },
                  - {
                      - Action   = "ec2:CreateTags"
                      - Effect   = "Allow"
                      - Resource = "arn:aws:ec2:*:*:network-interface/*"
                      - Sid      = "CreateTags"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> null
        tags             = {
            "Shared"     = "Ignore"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
        # (6 unchanged attributes hidden)
    }

  # module.vpc_endpoints.aws_security_group.this[0] has been deleted
  - resource "aws_security_group" "this" {
      - arn                    = "arn:aws:ec2:us-east-2:808842816990:security-group/sg-0abee7c9c53d92aa4" -> null
      - id                     = "sg-0abee7c9c53d92aa4" -> null
        name                   = "k8s-infra-kops-vpc-endpoints-20231204231519059500000001"
        tags                   = {
            "Name"       = "k8s-infra-kops-vpc-endpoints-"
            "Shared"     = "Ignore"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
            "region"     = "us-east-2"
            "subproject" = "kops"
        }
        # (9 unchanged attributes hidden)
    }

  # module.eks.module.kms.aws_kms_key.this[0] has been deleted
  - resource "aws_kms_key" "this" {
      - arn                                = "arn:aws:kms:us-east-2:808842816990:key/6abd02ac-74e1-4268-af15-46290f75b193" -> null
        id                                 = "6abd02ac-74e1-4268-af15-46290f75b193"
      - key_id                             = "6abd02ac-74e1-4268-af15-46290f75b193" -> null
      - policy                             = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = "kms:*"
                      - Effect    = "Allow"
                      - Principal = {
                          - AWS = "arn:aws:iam::808842816990:root"
                        }
                      - Resource  = "*"
                      - Sid       = "Default"
                    },
                  - {
                      - Action    = [
                          - "kms:Update*",
                          - "kms:UntagResource",
                          - "kms:TagResource",
                          - "kms:ScheduleKeyDeletion",
                          - "kms:Revoke*",
                          - "kms:ReplicateKey",
                          - "kms:Put*",
                          - "kms:List*",
                          - "kms:ImportKeyMaterial",
                          - "kms:Get*",
                          - "kms:Enable*",
                          - "kms:Disable*",
                          - "kms:Describe*",
                          - "kms:Delete*",
                          - "kms:Create*",
                          - "kms:CancelKeyDeletion",
                        ]
                      - Effect    = "Allow"
                      - Principal = {
                          - AWS = "arn:aws:iam::808842816990:role/OrganizationAccountAccessRole"
                        }
                      - Resource  = "*"
                      - Sid       = "KeyAdministration"
                    },
                  - {
                      - Action    = [
                          - "kms:ReEncrypt*",
                          - "kms:GenerateDataKey*",
                          - "kms:Encrypt",
                          - "kms:DescribeKey",
                          - "kms:Decrypt",
                        ]
                      - Effect    = "Allow"
                      - Principal = {
                          - AWS = "arn:aws:iam::808842816990:role/k8s-infra-kops-prow-build-cluster-20230808174304353200000004"
                        }
                      - Resource  = "*"
                      - Sid       = "KeyUsage"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> null
      - region                             = "us-east-2" -> null
        tags                               = {
            "githubRepo"            = "git.k8s.io/k8s.io"
            "group"                 = "sig-cluster-lifecycle"
            "managed-by"            = "Terraform"
            "region"                = "us-east-2"
            "subproject"            = "kops"
            "terraform-aws-modules" = "eks"
        }
        # (11 unchanged attributes hidden)
    }


Unless you have made equivalent changes to your configuration, or ignored the
relevant attributes using ignore_changes, the following plan may include
actions to undo or respond to these changes.

─────────────────────────────────────────────────────────────────────────────

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place
+/- create replacement and then destroy
 <= read (data resources)

Terraform planned the following actions, but then encountered a problem:

  # aws_iam_role.eks_pod_identity_role will be created
  + resource "aws_iam_role" "eks_pod_identity_role" {
      + arn                   = (known after apply)
      + assume_role_policy    = jsonencode(
            {
              + Statement = [
                  + {
                      + Action    = [
                          + "sts:TagSession",
                          + "sts:AssumeRole",
                        ]
                      + Effect    = "Allow"
                      + Principal = {
                          + Service = "pods.eks.amazonaws.com"
                        }
                    },
                ]
              + Version   = "2012-10-17"
            }
        )
      + create_date           = (known after apply)
      + force_detach_policies = false
      + id                    = (known after apply)
      + managed_policy_arns   = (known after apply)
      + max_session_duration  = 3600
      + name                  = "EKSPodIdentityRo

Warning: Output length greater than max comment size. Continued in next comment.

@k8s-infra-ci-robot
Copy link
Contributor

Continued plan output from previous comment.

Show Output
le"
      + name_prefix           = (known after apply)
      + path                  = "/"
      + tags                  = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all              = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + unique_id             = (known after apply)

      + inline_policy (known after apply)
    }

  # aws_iam_role_policy_attachment.eks_pod_identity_policy will be created
  + resource "aws_iam_role_policy_attachment" "eks_pod_identity_policy" {
      + id         = (known after apply)
      + policy_arn = "arn:aws:iam::aws:policy/AdministratorAccess"
      + role       = "EKSPodIdentityRole"
    }

  # aws_vpc_ipam.main will be created
  + resource "aws_vpc_ipam" "main" {
      + arn                                       = (known after apply)
      + default_resource_discovery_association_id = (known after apply)
      + default_resource_discovery_id             = (known after apply)
      + description                               = "k8s-infra-kops-us-east-2-ipam"
      + enable_private_gua                        = false
      + id                                        = (known after apply)
      + metered_account                           = (known after apply)
      + private_default_scope_id                  = (known after apply)
      + public_default_scope_id                   = (known after apply)
      + region                                    = "us-east-2"
      + scope_count                               = (known after apply)
      + tags                                      = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all                                  = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tier                                      = "advanced"

      + operating_regions {
          + region_name = "us-east-2"
        }
    }

  # aws_vpc_ipam_pool.main will be created
  + resource "aws_vpc_ipam_pool" "main" {
      + address_family  = "ipv4"
      + arn             = (known after apply)
      + auto_import     = false
      + id              = (known after apply)
      + ipam_scope_id   = (known after apply)
      + ipam_scope_type = (known after apply)
      + locale          = "us-east-2"
      + pool_depth      = (known after apply)
      + region          = "us-east-2"
      + state           = (known after apply)
      + tags            = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all        = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
    }

  # aws_vpc_ipam_pool_cidr.main will be created
  + resource "aws_vpc_ipam_pool_cidr" "main" {
      + cidr              = "10.128.0.0/16"
      + id                = (known after apply)
      + ipam_pool_cidr_id = (known after apply)
      + ipam_pool_id      = (known after apply)
      + netmask_length    = (known after apply)
      + region            = "us-east-2"
    }

  # aws_vpc_ipam_preview_next_cidr.main must be replaced
+/- resource "aws_vpc_ipam_preview_next_cidr" "main" {
      ~ cidr           = "10.128.0.0/20" -> (known after apply)
      ~ id             = "10.128.0.0/20_ipam-pool-0cefae006ed9905cc" -> (known after apply)
      ~ ipam_pool_id   = "ipam-pool-0cefae006ed9905cc" -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)
    }

  # aws_vpc_ipam_scope.main will be created
  + resource "aws_vpc_ipam_scope" "main" {
      + arn             = (known after apply)
      + description     = "k8s-infra-kops-us-east-2-ipam-scope"
      + id              = (known after apply)
      + ipam_arn        = (known after apply)
      + ipam_id         = (known after apply)
      + ipam_scope_type = (known after apply)
      + is_default      = (known after apply)
      + pool_count      = (known after apply)
      + region          = "us-east-2"
      + tags            = {
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all        = {
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
    }

  # module.ci_iam_group.aws_iam_group.this[0] will be created
  + resource "aws_iam_group" "this" {
      + arn       = (known after apply)
      + id        = (known after apply)
      + name      = "ci-admins"
      + path      = "/"
      + unique_id = (known after apply)
    }

  # module.ci_iam_group.aws_iam_group_membership.this[0] will be created
  + resource "aws_iam_group_membership" "this" {
      + group = (known after apply)
      + id    = (known after apply)
      + name  = "ci-admins"
      + users = [
          + "kops-ci-user",
        ]
    }

  # module.ci_iam_group.aws_iam_group_policy_attachment.custom_arns[0] will be created
  + resource "aws_iam_group_policy_attachment" "custom_arns" {
      + group      = (known after apply)
      + id         = (known after apply)
      + policy_arn = "arn:aws:iam::aws:policy/AdministratorAccess"
    }

  # module.ci_iam_group.aws_iam_group_policy_attachment.iam_self_management[0] will be created
  + resource "aws_iam_group_policy_attachment" "iam_self_management" {
      + group      = (known after apply)
      + id         = (known after apply)
      + policy_arn = (known after apply)
    }

  # module.ci_iam_group.aws_iam_policy.iam_self_management[0] will be created
  + resource "aws_iam_policy" "iam_self_management" {
      + arn              = (known after apply)
      + attachment_count = (known after apply)
      + id               = (known after apply)
      + name             = (known after apply)
      + name_prefix      = "IAMSelfManagement-"
      + path             = "/"
      + policy           = jsonencode(
            {
              + Statement = [
                  + {
                      + Action   = [
                          + "iam:ListVirtualMFADevices",
                          + "iam:ListAccountAliases",
                          + "iam:GetAccountSummary",
                          + "iam:GetAccountPasswordPolicy",
                        ]
                      + Effect   = "Allow"
                      + Resource = "*"
                      + Sid      = "AllowViewAccountInfo"
                    },
                  + {
                      + Action   = [
                          + "iam:UpdateLoginProfile",
                          + "iam:GetUser",
                          + "iam:GetLoginProfile",
                          + "iam:ChangePassword",
                        ]
                      + Effect   = "Allow"
                      + Resource = [
                          + "arn:aws:iam::348685125169:user/*/${aws:username}",
                          + "arn:aws:iam::348685125169:user/${aws:username}",
                        ]
                      + Sid      = "AllowManageOwnPasswords"
                    },
                  + {
                      + Action   = [
                          + "iam:UpdateAccessKey",
                          + "iam:UntagUser",
                          + "iam:TagUser",
                          + "iam:ListUserTags",
                          + "iam:ListAccessKeys",
                          + "iam:GetAccessKeyLastUsed",
                          + "iam:DeleteAccessKey",
                          + "iam:CreateAccessKey",
                        ]
                      + Effect   = "Allow"
                      + Resource = [
                          + "arn:aws:iam::348685125169:user/*/${aws:username}",
                          + "arn:aws:iam::348685125169:user/${aws:username}",
                        ]
                      + Sid      = "AllowManageOwnAccessKeys"
                    },
                  + {
                      + Action   = [
                          + "iam:UploadSigningCertificate",
                          + "iam:UpdateSigningCertificate",
                          + "iam:ListSigningCertificates",
                          + "iam:DeleteSigningCertificate",
                        ]
                      + Effect   = "Allow"
                      + Resource = [
                          + "arn:aws:iam::348685125169:user/*/${aws:username}",
                          + "arn:aws:iam::348685125169:user/${aws:username}",
                        ]
                      + Sid      = "AllowManageOwnSigningCertificates"
                    },
                  + {
                      + Action   = [
                          + "iam:UploadSSHPublicKey",
                          + "iam:UpdateSSHPublicKey",
                          + "iam:ListSSHPublicKeys",
                          + "iam:GetSSHPublicKey",
                          + "iam:DeleteSSHPublicKey",
                        ]
                      + Effect   = "Allow"
                      + Resource = [
                          + "arn:aws:iam::348685125169:user/*/${aws:username}",
                          + "arn:aws:iam::348685125169:user/${aws:username}",
                        ]
                      + Sid      = "AllowManageOwnSSHPublicKeys"
                    },
                  + {
                      + Action   = [
                          + "iam:UpdateServiceSpecificCredential",
                          + "iam:ResetServiceSpecificCredential",
                          + "iam:ListServiceSpecificCredentials",
                          + "iam:DeleteServiceSpecificCredential",
                          + "iam:CreateServiceSpecificCredential",
                        ]
                      + Effect   = "Allow"
                      + Resource = [
                          + "arn:aws:iam::348685125169:user/*/${aws:username}",
                          + "arn:aws:iam::348685125169:user/${aws:username}",
                        ]
                      + Sid      = "AllowManageOwnGitCredentials"
                    },
                  + {
                      + Action   = "iam:CreateVirtualMFADevice"
                      + Effect   = "Allow"
                      + Resource = "arn:aws:iam::348685125169:mfa/*"
                      + Sid      = "AllowManageOwnVirtualMFADevice"
                    },
                  + {
                      + Action   = [
                          + "iam:ResyncMFADevice",
                          + "iam:ListMFADevices",
                          + "iam:EnableMFADevice",
                          + "iam:DeactivateMFADevice",
                        ]
                      + Effect   = "Allow"
                      + Resource = [
                          + "arn:aws:iam::348685125169:user/*/${aws:username}",
                          + "arn:aws:iam::348685125169:user/${aws:username}",
                        ]
                      + Sid      = "AllowManageOwnUserMFA"
                    },
                ]
              + Version   = "2012-10-17"
            }
        )
      + policy_id        = (known after apply)
      + tags             = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all         = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
    }

  # module.ebs_csi_irsa.aws_iam_policy.this[0] will be created
  + resource "aws_iam_policy" "this" {
      + arn              = (known after apply)
      + attachment_count = (known after apply)
      + description      = "Provides permissions to manage EBS volumes via the container storage interface driver"
      + id               = (known after apply)
      + name             = (known after apply)
      + name_prefix      = "EBS_CSI-"
      + path             = "/"
      + policy           = jsonencode(
            {
              + Statement = [
                  + {
                      + Action   = [
                          + "ec2:DescribeVolumesModifications",
                          + "ec2:DescribeVolumes",
                          + "ec2:DescribeTags",
                          + "ec2:DescribeSnapshots",
                          + "ec2:DescribeInstances",
                          + "ec2:DescribeAvailabilityZones",
                        ]
                      + Effect   = "Allow"
                      + Resource = "*"
                    },
                  + {
                      + Action   = [
                          + "ec2:ModifyVolume",
                          + "ec2:CreateSnapshot",
                        ]
                      + Effect   = "Allow"
                      + Resource = "arn:aws:ec2:*:*:volume/*"
                    },
                  + {
                      + Action   = [
                          + "ec2:DetachVolume",
                          + "ec2:AttachVolume",
                        ]
                      + Effect   = "Allow"
                      + Resource = [
                          + "arn:aws:ec2:*:*:volume/*",
                          + "arn:aws:ec2:*:*:instance/*",
                        ]
                    },
                  + {
                      + Action   = [
                          + "ec2:EnableFastSnapshotRestores",
                          + "ec2:CreateVolume",
                        ]
                      + Effect   = "Allow"
                      + Resource = "arn:aws:ec2:*:*:snapshot/*"
                    },
                  + {
                      + Action    = "ec2:CreateTags"
                      + Condition = {
                          + StringEquals = {
                              + "ec2:CreateAction" = [
                                  + "CreateVolume",
                                  + "CreateSnapshot",
                                ]
                            }
                        }
                      + Effect    = "Allow"
                      + Resource  = [
                          + "arn:aws:ec2:*:*:volume/*",
                          + "arn:aws:ec2:*:*:snapshot/*",
                        ]
                    },
                  + {
                      + Action   = "ec2:DeleteTags"
                      + Effect   = "Allow"
                      + Resource = [
                          + "arn:aws:ec2:*:*:volume/*",
                          + "arn:aws:ec2:*:*:snapshot/*",
                        ]
                    },
                  + {
                      + Action    = "ec2:CreateVolume"
                      + Condition = {
                          + StringLike = {
                              + "aws:RequestTag/ebs.csi.aws.com/cluster" = "true"
                            }
                        }
                      + Effect    = "Allow"
                      + Resource  = "arn:aws:ec2:*:*:volume/*"
                    },
                  + {
                      + Action    = "ec2:CreateVolume"
                      + Condition = {
                          + StringLike = {
                              + "aws:RequestTag/CSIVolumeName" = "*"
                            }
                        }
                      + Effect    = "Allow"
                      + Resource  = "arn:aws:ec2:*:*:volume/*"
                    },
                  + {
                      + Action    = "ec2:DeleteVolume"
                      + Condition = {
                          + StringLike = {
                              + "aws:ResourceTag/ebs.csi.aws.com/cluster" = "true"
                            }
                        }
                      + Effect    = "Allow"
                      + Resource  = "arn:aws:ec2:*:*:volume/*"
                    },
                  + {
                      + Action    = "ec2:DeleteVolume"
                      + Condition = {
                          + StringLike = {
                              + "aws:ResourceTag/CSIVolumeName" = "*"
                            }
                        }
                      + Effect    = "Allow"
                      + Resource  = "arn:aws:ec2:*:*:volume/*"
                    },
                  + {
                      + Action    = "ec2:DeleteVolume"
                      + Condition = {
                          + StringLike = {
                              + "ec2:ResourceTag/kubernetes.io/created-for/pvc/name" = "*"
                            }
                        }
                      + Effect    = "Allow"
                      + Resource  = "arn:aws:ec2:*:*:volume/*"
                    },
                  + {
                      + Action    = "ec2:CreateSnapshot"
                      + Condition = {
                          + StringLike = {
                              + "aws:RequestTag/CSIVolumeSnapshotName" = "*"
                            }
                        }
                      + Effect    = "Allow"
                      + Resource  = "arn:aws:ec2:*:*:snapshot/*"
                    },
                  + {
                      + Action    = "ec2:CreateSnapshot"
                      + Condition = {
                          + StringLike = {
                              + "aws:RequestTag/ebs.csi.aws.com/cluster" = "true"
                            }
                        }
                      + Effect    = "Allow"
                      + Resource  = "arn:aws:ec2:*:*:snapshot/*"
                    },
                  + {
                      + Action    = "ec2:DeleteSnapshot"
                      + Condition = {
                          + StringLike = {
                              + "aws:ResourceTag/CSIVolumeSnapshotName" = "*"
                            }
                        }
                      + Effect    = "Allow"
                      + Resource  = "arn:aws:ec2:*:*:snapshot/*"
                    },
                  + {
                      + Action    = "ec2:DeleteSnapshot"
                      + Condition = {
                          + StringLike = {
                              + "aws:ResourceTag/ebs.csi.aws.com/cluster" = "true"
                            }
                        }
                      + Effect    = "Allow"
                      + Resource  = "arn:aws:ec2:*:*:snapshot/*"
                    },
                ]
              + Version   = "2012-10-17"
            }
        )
      + policy_id        = (known after apply)
      + tags             = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all         = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
    }

  # module.eks.aws_cloudwatch_log_group.this[0] will be created
  + resource "aws_cloudwatch_log_group" "this" {
      + arn               = (known after apply)
      + id                = (known after apply)
      + log_group_class   = (known after apply)
      + name              = "/aws/eks/k8s-infra-kops-prow-build/cluster"
      + name_prefix       = (known after apply)
      + region            = "us-east-2"
      + retention_in_days = 30
      + skip_destroy      = false
      + tags              = {
          + "Name"       = "/aws/eks/k8s-infra-kops-prow-build/cluster"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all          = {
          + "Name"       = "/aws/eks/k8s-infra-kops-prow-build/cluster"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
    }

  # module.eks.aws_iam_policy.cluster_encryption[0] will be created
  + resource "aws_iam_policy" "cluster_encryption" {
      + arn              = (known after apply)
      + attachment_count = (known after apply)
      + description      = "Cluster encryption policy to allow cluster role to utilize CMK provided"
      + id               = (known after apply)
      + name             = (known after apply)
      + name_prefix      = "k8s-infra-kops-prow-build-cluster-ClusterEncryption"
      + path             = "/"
      + policy           = (known after apply)
      + policy_id        = (known after apply)
      + tags             = {
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all         = {
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
    }

  # module.eks.aws_iam_role.this[0] will be created
  + resource "aws_iam_role" "this" {
      + arn                   = (known after apply)
      + assume_role_policy    = jsonencode(
            {
              + Statement = [
                  + {
                      + Action    = [
                          + "sts:TagSession",
                          + "sts:AssumeRole",
                        ]
                      + Effect    = "Allow"
                      + Principal = {
                          + Service = "eks.amazonaws.com"
                        }
                      + Sid       = "EKSClusterAssumeRole"
                    },
                ]
              + Version   = "2012-10-17"
            }
        )
      + create_date           = (known after apply)
      + force_detach_policies = true
      + id                    = (known after apply)
      + managed_policy_arns   = (known after apply)
      + max_session_duration  = 3600
      + name                  = (known after apply)
      + name_prefix           = "k8s-infra-kops-prow-build-cluster-"
      + path                  = "/"
      + tags                  = {
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all              = {
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + unique_id             = (known after apply)

      + inline_policy (known after apply)
    }

  # module.eks.aws_iam_role_policy_attachment.cluster_encryption[0] will be created
  + resource "aws_iam_role_policy_attachment" "cluster_encryption" {
      + id         = (known after apply)
      + policy_arn = (known after apply)
      + role       = (known after apply)
    }

  # module.eks.aws_iam_role_policy_attachment.this["AmazonEKSClusterPolicy"] will be created
  + resource "aws_iam_role_policy_attachment" "this" {
      + id         = (known after apply)
      + policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
      + role       = (known after apply)
    }

  # module.eks.aws_security_group.cluster[0] will be created
  + resource "aws_security_group" "cluster" {
      + arn                    = (known after apply)
      + description            = "EKS cluster security group"
      + egress                 = (known after apply)
      + id                     = (known after apply)
      + ingress                = (known after apply)
      + name                   = (known after apply)
      + name_prefix            = "k8s-infra-kops-prow-build-cluster-"
      + owner_id               = (known after apply)
      + region                 = "us-east-2"
      + revoke_rules_on_delete = false
      + tags                   = {
          + "Name"       = "k8s-infra-kops-prow-build-cluster"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all               = {
          + "Name"       = "k8s-infra-kops-prow-build-cluster"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + vpc_id                 = (known after apply)
    }

  # module.eks.aws_security_group.node[0] will be created
  + resource "aws_security_group" "node" {
      + arn                    = (known after apply)
      + description            = "EKS node shared security group"
      + egress                 = (known after apply)
      + id                     = (known after apply)
      + ingress                = (known after apply)
      + name                   = (known after apply)
      + name_prefix            = "k8s-infra-kops-prow-build-node-"
      + owner_id               = (known after apply)
      + region                 = "us-east-2"
      + revoke_rules_on_delete = false
      + tags                   = {
          + "Name"                                            = "k8s-infra-kops-prow-build-node"
          + "githubRepo"                                      = "git.k8s.io/k8s.io"
          + "group"                                           = "sig-cluster-lifecycle"
          + "kubernetes.io/cluster/k8s-infra-kops-prow-build" = "owned"
          + "managed-by"                                      = "Terraform"
          + "region"                                          = "us-east-2"
          + "subproject"                                      = "kops"
        }
      + tags_all               = {
          + "Name"                                            = "k8s-infra-kops-prow-build-node"
          + "githubRepo"                                      = "git.k8s.io/k8s.io"
          + "group"                                           = "sig-cluster-lifecycle"
          + "kubernetes.io/cluster/k8s-infra-kops-prow-build" = "owned"
          + "managed-by"                                      = "Terraform"
          + "region"                                          = "us-east-2"
          + "subproject"                                      = "kops"
        }
      + vpc_id                 = (known after apply)
    }

  # module.eks.aws_security_group_rule.cluster["ingress_nodes_443"] will be created
  + resource "aws_security_group_rule" "cluster" {
      + description              = "Node groups to cluster API"
      + from_port                = 443
      + id                       = (known after apply)
      + protocol                 = "tcp"
      + region                   = "us-east-2"
      + security_group_id        = (known after apply)
      + security_group_rule_id   = (known after apply)
      + self                     = false
      + source_security_group_id = (known after apply)
      + to_port                  = 443
      + type                     = "ingress"
    }

  # module.eks.aws_security_group_rule.node["egress_all"] will be created
  + resource "aws_security_group_rule" "node" {
      + cidr_blocks              = [
          + "0.0.0.0/0",
        ]
      + description              = "Allow all egress"
      + from_port                = 0
      + id                       = (known after apply)
      + protocol                 = "-1"
      + region                   = "us-east-2"
      + security_group_id        = (known after apply)
      + security_group_rule_id   = (known after apply)
      + self                     = false
      + source_security_group_id = (known after apply)
      + to_port                  = 0
      + type                     = "egress"
    }

  # module.eks.aws_security_group_rule.node["ingress_cluster_443"] will be created
  + resource "aws_security_group_rule" "node" {
      + description              = "Cluster API to node groups"
      + from_port                = 443
      + id                       = (known after apply)
      + protocol                 = "tcp"
      + region                   = "us-east-2"
      + security_group_id        = (known after apply)
      + security_group_rule_id   = (known after apply)
      + self                     = false
      + source_security_group_id = (known after apply)
      + to_port                  = 443
      + type                     = "ingress"
    }

  # module.eks.aws_security_group_rule.node["ingress_cluster_4443_webhook"] will be created
  + resource "aws_security_group_rule" "node" {
      + description              = "Cluster API to node 4443/tcp webhook"
      + from_port                = 4443
      + id                       = (known after apply)
      + protocol                 = "tcp"
      + region                   = "us-east-2"
      + security_group_id        = (known after apply)
      + security_group_rule_id   = (known after apply)
      + self                     = false
      + source_security_group_id = (known after apply)
      + to_port                  = 4443
      + type                     = "ingress"
    }

  # module.eks.aws_security_group_rule.node["ingress_cluster_6443_webhook"] will be created
  + resource "aws_security_group_rule" "node" {
      + description              = "Cluster API to node 6443/tcp webhook"
      + from_port                = 6443
      + id                       = (known after apply)
      + protocol                 = "tcp"
      + region                   = "us-east-2"
      + security_group_id        = (known after apply)
      + security_group_rule_id   = (known after apply)
      + self                     = false
      + source_security_group_id = (known after apply)
      + to_port                  = 6443
      + type                     = "ingress"
    }

  # module.eks.aws_security_group_rule.node["ingress_cluster_8443_webhook"] will be created
  + resource "aws_security_group_rule" "node" {
      + description              = "Cluster API to node 8443/tcp webhook"
      + from_port                = 8443
      + id                       = (known after apply)
      + protocol                 = "tcp"
      + region                   = "us-east-2"
      + security_group_id        = (known after apply)
      + security_group_rule_id   = (known after apply)
      + self                     = false
      + source_security_group_id = (known after apply)
      + to_port                  = 8443
      + type                     = "ingress"
    }

  # module.eks.aws_security_group_rule.node["ingress_cluster_9443_webhook"] will be created
  + resource "aws_security_group_rule" "node" {
      + description              = "Cluster API to node 9443/tcp webhook"
      + from_port                = 9443
      + id                       = (known after apply)
      + protocol                 = "tcp"
      + region                   = "us-east-2"
      + security_group_id        = (known after apply)
      + security_group_rule_id   = (known after apply)
      + self                     = false
      + source_security_group_id = (known after apply)
      + to_port                  = 9443
      + type                     = "ingress"
    }

  # module.eks.aws_security_group_rule.node["ingress_cluster_kubelet"] will be created
  + resource "aws_security_group_rule" "node" {
      + description              = "Cluster API to node kubelets"
      + from_port                = 10250
      + id                       = (known after apply)
      + protocol                 = "tcp"
      + region                   = "us-east-2"
      + security_group_id        = (known after apply)
      + security_group_rule_id   = (known after apply)
      + self                     = false
      + source_security_group_id = (known after apply)
      + to_port                  = 10250
      + type                     = "ingress"
    }

  # module.eks.aws_security_group_rule.node["ingress_nodes_ephemeral"] will be created
  + resource "aws_security_group_rule" "node" {
      + description              = "Node to node ingress on ephemeral ports"
      + from_port                = 1025
      + id                       = (known after apply)
      + protocol                 = "tcp"
      + region                   = "us-east-2"
      + security_group_id        = (known after apply)
      + security_group_rule_id   = (known after apply)
      + self                     = true
      + source_security_group_id = (known after apply)
      + to_port                  = 65535
      + type                     = "ingress"
    }

  # module.eks.aws_security_group_rule.node["ingress_self_coredns_tcp"] will be created
  + resource "aws_security_group_rule" "node" {
      + description              = "Node to node CoreDNS"
      + from_port                = 53
      + id                       = (known after apply)
      + protocol                 = "tcp"
      + region                   = "us-east-2"
      + security_group_id        = (known after apply)
      + security_group_rule_id   = (known after apply)
      + self                     = true
      + source_security_group_id = (known after apply)
      + to_port                  = 53
      + type                     = "ingress"
    }

  # module.eks.aws_security_group_rule.node["ingress_self_coredns_udp"] will be created
  + resource "aws_security_group_rule" "node" {
      + description              = "Node to node CoreDNS UDP"
      + from_port                = 53
      + id                       = (known after apply)
      + protocol                 = "udp"
      + region                   = "us-east-2"
      + security_group_id        = (known after apply)
      + security_group_rule_id   = (known after apply)
      + self                     = true
      + source_security_group_id = (known after apply)
      + to_port                  = 53
      + type                     = "ingress"
    }

  # module.kops_ci_user.aws_iam_access_key.this_no_pgp[0] will be created
  + resource "aws_iam_access_key" "this_no_pgp" {
      + create_date                    = (known after apply)
      + encrypted_secret               = (known after apply)
      + encrypted_ses_smtp_password_v4 = (known after apply)
      + id                             = (known after apply)
      + key_fingerprint                = (known after apply)
      + secret                         = (sensitive value)
      + ses_smtp_password_v4           = (sensitive value)
      + status                         = "Active"
      + user                           = "kops-ci-user"
    }

  # module.kops_ci_user.aws_iam_user.this[0] will be updated in-place
  ~ resource "aws_iam_user" "this" {
        id                   = "kops-ci-user"
        name                 = "kops-ci-user"
      ~ tags                 = {
          + "Shared"     = "Ignore"
            "githubRepo" = "git.k8s.io/k8s.io"
            "group"      = "sig-cluster-lifecycle"
            "managed-by" = "Terraform"
          + "region"     = "us-east-2"
            "subproject" = "kops"
        }
      ~ tags_all             = {
          + "Shared"     = "Ignore"
          + "region"     = "us-east-2"
            # (4 unchanged elements hidden)
        }
        # (5 unchanged attributes hidden)
    }

  # module.vpc.data.aws_iam_policy_document.vpc_flow_log_cloudwatch[0] will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "vpc_flow_log_cloudwatch" {
      + id            = (known after apply)
      + json          = (known after apply)
      + minified_json = (known after apply)

      + statement {
          + actions   = [
              + "logs:CreateLogStream",
              + "logs:DescribeLogGroups",
              + "logs:DescribeLogStreams",
              + "logs:PutLogEvents",
            ]
          + effect    = "Allow"
          + resources = [
              + (known after apply),
            ]
          + sid       = "AWSVPCFlowLogsPushToCloudWatch"
        }
    }

  # module.vpc.aws_cloudwatch_log_group.flow_log[0] will be created
  + resource "aws_cloudwatch_log_group" "flow_log" {
      + arn               = (known after apply)
      + id                = (known after apply)
      + log_group_class   = (known after apply)
      + name              = (known after apply)
      + name_prefix       = (known after apply)
      + region            = "us-east-2"
      + retention_in_days = 30
      + skip_destroy      = false
      + tags              = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all          = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
    }

  # module.vpc.aws_default_network_acl.this[0] will be created
  + resource "aws_default_network_acl" "this" {
      + arn                    = (known after apply)
      + default_network_acl_id = (known after apply)
      + id                     = (known after apply)
      + owner_id               = (known after apply)
      + region                 = "us-east-2"
      + tags                   = {
          + "Name"       = "k8s-infra-kops-vpc-default"
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all               = {
          + "Name"       = "k8s-infra-kops-vpc-default"
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + vpc_id                 = (known after apply)

      + egress {
          + action          = "allow"
          + from_port       = 0
          + ipv6_cidr_block = "::/0"
          + protocol        = "-1"
          + rule_no         = 101
          + to_port         = 0
            # (1 unchanged attribute hidden)
        }
      + egress {
          + action          = "allow"
          + cidr_block      = "0.0.0.0/0"
          + from_port       = 0
          + protocol        = "-1"
          + rule_no         = 100
          + to_port         = 0
            # (1 unchanged attribute hidden)
        }

      + ingress {
          + action          = "allow"
          + from_port       = 0
          + ipv6_cidr_block = "::/0"
          + protocol        = "-1"
          + rule_no         = 101
          + to_port         = 0
            # (1 unchanged attribute hidden)
        }
      + ingress {
          + action          = "allow"
          + cidr_block      = "0.0.0.0/0"
          + from_port       = 0
          + protocol        = "-1"
          + rule_no         = 100
          + to_port         = 0
            # (1 unchanged attribute hidden)
        }
    }

  # module.vpc.aws_default_route_table.default[0] will be created
  + resource "aws_default_route_table" "default" {
      + arn                    = (known after apply)
      + default_route_table_id = (known after apply)
      + id                     = (known after apply)
      + owner_id               = (known after apply)
      + region                 = "us-east-2"
      + route                  = (known after apply)
      + tags                   = {
          + "Name"       = "k8s-infra-kops-vpc-default"
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all               = {
          + "Name"       = "k8s-infra-kops-vpc-default"
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + vpc_id                 = (known after apply)

      + timeouts {
          + create = "5m"
          + update = "5m"
        }
    }

  # module.vpc.aws_default_security_group.this[0] will be created
  + resource "aws_default_security_group" "this" {
      + arn                    = (known after apply)
      + description            = (known after apply)
      + egress                 = (known after apply)
      + id                     = (known after apply)
      + ingress                = (known after apply)
      + name                   = (known after apply)
      + name_prefix            = (known after apply)
      + owner_id               = (known after apply)
      + region                 = "us-east-2"
      + revoke_rules_on_delete = false
      + tags                   = {
          + "Name"       = "k8s-infra-kops-vpc-default"
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all               = {
          + "Name"       = "k8s-infra-kops-vpc-default"
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + vpc_id                 = (known after apply)
    }

  # module.vpc.aws_flow_log.this[0] will be created
  + resource "aws_flow_log" "this" {
      + arn                      = (known after apply)
      + iam_role_arn             = (known after apply)
      + id                       = (known after apply)
      + log_destination          = (known after apply)
      + log_destination_type     = "cloud-watch-logs"
      + log_format               = (known after apply)
      + max_aggregation_interval = 600
      + region                   = "us-east-2"
      + tags                     = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all                 = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + traffic_type             = "ALL"
      + vpc_id                   = (known after apply)
    }

  # module.vpc.aws_iam_policy.vpc_flow_log_cloudwatch[0] will be created
  + resource "aws_iam_policy" "vpc_flow_log_cloudwatch" {
      + arn              = (known after apply)
      + attachment_count = (known after apply)
      + id               = (known after apply)
      + name             = (known after apply)
      + name_prefix      = "vpc-flow-log-to-cloudwatch-"
      + path             = "/"
      + policy           = (known after apply)
      + policy_id        = (known after apply)
      + tags             = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all         = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
    }

  # module.vpc.aws_iam_role.vpc_flow_log_cloudwatch[0] will be created
  + resource "aws_iam_role" "vpc_flow_log_cloudwatch" {
      + arn                   = (known after apply)
      + assume_role_policy    = jsonencode(
            {
              + Statement = [
                  + {
                      + Action    = "sts:AssumeRole"
                      + Effect    = "Allow"
                      + Principal = {
                          + Service = "vpc-flow-logs.amazonaws.com"
                        }
                      + Sid       = "AWSVPCFlowLogsAssumeRole"
                    },
                ]
              + Version   = "2012-10-17"
            }
        )
      + create_date           = (known after apply)
      + force_detach_policies = false
      + id                    = (known after apply)
      + managed_policy_arns   = (known after apply)
      + max_session_duration  = 3600
      + name                  = (known after apply)
      + name_prefix           = "vpc-flow-log-role-"
      + path                  = "/"
      + tags                  = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all              = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + unique_id             = (known after apply)

      + inline_policy (known after apply)
    }

  # module.vpc.aws_iam_role_policy_attachment.vpc_flow_log_cloudwatch[0] will be created
  + resource "aws_iam_role_policy_attachment" "vpc_flow_log_cloudwatch" {
      + id         = (known after apply)
      + policy_arn = (known after apply)
      + role       = (known after apply)
    }

  # module.vpc.aws_vpc.this[0] will be created
  + resource "aws_vpc" "this" {
      + arn                                  = (known after apply)
      + cidr_block                           = (known after apply)
      + default_network_acl_id               = (known after apply)
      + default_route_table_id               = (known after apply)
      + default_security_group_id            = (known after apply)
      + dhcp_options_id                      = (known after apply)
      + enable_dns_hostnames                 = true
      + enable_dns_support                   = true
      + enable_network_address_usage_metrics = (known after apply)
      + id                                   = (known after apply)
      + instance_tenancy                     = "default"
      + ipv4_ipam_pool_id                    = (known after apply)
      + ipv6_association_id                  = (known after apply)
      + ipv6_cidr_block                      = (known after apply)
      + ipv6_cidr_block_network_border_group = (known after apply)
      + main_route_table_id                  = (known after apply)
      + owner_id                             = (known after apply)
      + region                               = "us-east-2"
      + tags                                 = {
          + "Name"       = "k8s-infra-kops-vpc"
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all                             = {
          + "Name"       = "k8s-infra-kops-vpc"
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
    }

  # module.vpc_cni_irsa.aws_iam_policy.this[0] will be created
  + resource "aws_iam_policy" "this" {
      + arn              = (known after apply)
      + attachment_count = (known after apply)
      + description      = "Provides the Amazon VPC CNI Plugin (amazon-vpc-cni-k8s) the permissions it requires to modify the IPv4/IPv6 address configuration on your EKS worker nodes"
      + id               = (known after apply)
      + name             = (known after apply)
      + name_prefix      = "VPC_CNI_IPv4-"
      + path             = "/"
      + policy           = jsonencode(
            {
              + Statement = [
                  + {
                      + Action   = [
                          + "ec2:UnassignPrivateIpAddresses",
                          + "ec2:ModifyNetworkInterfaceAttribute",
                          + "ec2:DetachNetworkInterface",
                          + "ec2:DescribeTags",
                          + "ec2:DescribeSubnets",
                          + "ec2:DescribeNetworkInterfaces",
                          + "ec2:DescribeInstances",
                          + "ec2:DescribeInstanceTypes",
                          + "ec2:DeleteNetworkInterface",
                          + "ec2:CreateNetworkInterface",
                          + "ec2:AttachNetworkInterface",
                          + "ec2:AssignPrivateIpAddresses",
                        ]
                      + Effect   = "Allow"
                      + Resource = "*"
                      + Sid      = "IPV4"
                    },
                  + {
                      + Action   = "ec2:CreateTags"
                      + Effect   = "Allow"
                      + Resource = "arn:aws:ec2:*:*:network-interface/*"
                      + Sid      = "CreateTags"
                    },
                ]
              + Version   = "2012-10-17"
            }
        )
      + policy_id        = (known after apply)
      + tags             = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all         = {
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
    }

  # module.vpc_endpoints.aws_security_group.this[0] will be created
  + resource "aws_security_group" "this" {
      + arn                    = (known after apply)
      + description            = "VPC endpoint security group"
      + egress                 = (known after apply)
      + id                     = (known after apply)
      + ingress                = (known after apply)
      + name                   = (known after apply)
      + name_prefix            = "k8s-infra-kops-vpc-endpoints-"
      + owner_id               = (known after apply)
      + region                 = "us-east-2"
      + revoke_rules_on_delete = false
      + tags                   = {
          + "Name"       = "k8s-infra-kops-vpc-endpoints-"
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + tags_all               = {
          + "Name"       = "k8s-infra-kops-vpc-endpoints-"
          + "Shared"     = "Ignore"
          + "githubRepo" = "git.k8s.io/k8s.io"
          + "group"      = "sig-cluster-lifecycle"
          + "managed-by" = "Terraform"
          + "region"     = "us-east-2"
          + "subproject" = "kops"
        }
      + vpc_id                 = (known after apply)
    }

  # module.vpc_endpoints.aws_security_group_rule.this["ingress_https"] will be created
  + resource "aws_security_group_rule" "this" {
      + cidr_blocks              = (known after apply)
      + description              = "HTTPS from VPC"
      + from_port                = 443
      + id                       = (known after apply)
      + protocol                 = "tcp"
      + region                   = "us-east-2"
      + security_group_id        = (known after apply)
      + security_group_rule_id   = (known after apply)
      + self                     = false
      + source_security_group_id = (known after apply)
      + to_port                  = 443
      + type                     = "ingress"
    }

  # module.eks.module.kms.data.aws_iam_policy_document.this[0] will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "this" {
      + id                        = (known after apply)
      + json                      = (known after apply)
      + minified_json             = (known after apply)
      + override_policy_documents = []
      + source_policy_documents   = []

      + statement {
          + actions   = [
              + "kms:*",
            ]
          + resources = [
              + "*",
            ]
          + sid       = "Default"

          + principals {
              + identifiers = [
                  + "arn:aws:iam::348685125169:root",
                ]
              + type        = "AWS"
            }
        }
      + statement {
          + actions   = [
              + "kms:CancelKeyDeletion",
              + "kms:Create*",
              + "kms:Delete*",
              + "kms:Describe*",
              + "kms:Disable*",
              + "kms:Enable*",
              + "kms:Get*",
              + "kms:ImportKeyMaterial",
              + "kms:List*",
              + "kms:Put*",
              + "kms:ReplicateKey",
              + "kms:Revoke*",
              + "kms:ScheduleKeyDeletion",
              + "kms:TagResource",
              + "kms:UntagResource",
              + "kms:Update*",
            ]
          + resources = [
              + "*",
            ]
          + sid       = "KeyAdministration"

          + principals {
              + identifiers = [
                  + "arn:aws:iam::348685125169:role/atlantis",
                ]
              + type        = "AWS"
            }
        }
      + statement {
          + actions   = [
              + "kms:Decrypt",
              + "kms:DescribeKey",
              + "kms:Encrypt",
              + "kms:GenerateDataKey*",
              + "kms:ReEncrypt*",
            ]
          + resources = [
              + "*",
            ]
          + sid       = "KeyUsage"

          + principals {
              + identifiers = [
                  + (known after apply),
                ]
              + type        = "AWS"
            }
        }
    }

  # module.eks.module.kms.aws_kms_alias.this["cluster"] will be created
  + resource "aws_kms_alias" "this" {
      + arn            = (known after apply)
      + id             = (known after apply)
      + name           = "alias/eks/k8s-infra-kops-prow-build"
      + name_prefix    = (known after apply)
      + region         = "us-east-2"
      + target_key_arn = (known after apply)
      + target_key_id  = (known after apply)
    }

  # module.eks.module.kms.aws_kms_key.this[0] will be created
  + resource "aws_kms_key" "this" {
      + arn                                = (known after apply)
      + bypass_policy_lockout_safety_check = false
      + customer_master_key_spec           = "SYMMETRIC_DEFAULT"
      + description                        = "k8s-infra-kops-prow-build cluster encryption key"
      + enable_key_rotation                = true
      + id                                 = (known after apply)
      + is_enabled                         = true
      + key_id                             = (known after apply)
      + key_usage                          = "ENCRYPT_DECRYPT"
      + multi_region                       = false
      + policy                             = (known after apply)
      + region                             = "us-east-2"
      + rotation_period_in_days            = (known after apply)
      + tags                               = {
          + "githubRepo"            = "git.k8s.io/k8s.io"
          + "group"                 = "sig-cluster-lifecycle"
          + "managed-by"            = "Terraform"
          + "region"                = "us-east-2"
          + "subproject"            = "kops"
          + "terraform-aws-modules" = "eks"
        }
      + tags_all                           = {
          + "githubRepo"            = "git.k8s.io/k8s.io"
          + "group"                 = "sig-cluster-lifecycle"
          + "managed-by"            = "Terraform"
          + "region"                = "us-east-2"
          + "subproject"            = "kops"
          + "terraform-aws-modules" = "eks"
        }
    }

Plan: 46 to add, 1 to change, 1 to destroy.

Changes to Outputs:
  - kops-ci-user-secret-key = (sensitive value) -> null
  - kops-ci_user-access-key = "AKIA3YUWJ5XPHTELQU62" -> null
╷
│ Warning: Missing required argument
│ 
│   with provider["registry.terraform.io/hashicorp/aws"].kops-infra-ci,
│   on providers.tf line 17, in provider "aws":
│   17: provider "aws" {
│ 
│ The argument "role_arn" is required, but no definition was found.
│ 
│ This will be an error in a future release.
╵
╷
│ Error: reading IAM OIDC Provider (arn:aws:iam::808842816990:oidc-provider/container.googleapis.com/v1/projects/k8s-prow/locations/us-central1-f/clusters/prow): operation error IAM: GetOpenIDConnectProvider, https response error StatusCode: 403, RequestID: 149a7894-7d3f-4feb-bfa8-2f5d08d8d895, api error AccessDenied: User: arn:aws:sts::348685125169:assumed-role/atlantis/1763470458794565738 is not authorized to access this resource
│ 
│   with aws_iam_openid_connect_provider.google_prow_idp,
│   on main.tf line 19, in resource "aws_iam_openid_connect_provider" "google_prow_idp":
│   19: resource "aws_iam_openid_connect_provider" "google_prow_idp" {
│ 
╵
╷
│ Error: failed to get shared config profile, kops-ci
│ 
│   with provider["registry.terraform.io/hashicorp/aws"].kops-local-ci,
│   on providers.tf line 26, in provider "aws":
│   26: provider "aws" {
│ 
╵
╷
│ Error: reading S3 Bucket (k8s-kops-ci-prow-state-store) policy: operation error S3: GetBucketPolicy, https response error StatusCode: 403, RequestID: PV7A6K8YAE7SBWBX, HostID: UqZYssYtaVzTJGRK16+qG0sHRuAcbn8bpY2ccFY/ALGrDqm11AnogprJ1iOoIMf85Z8HBbzZ3Nw=, api error AccessDenied: User: arn:aws:sts::348685125169:assumed-role/atlantis/1763470458794565738 is not authorized to perform: s3:GetBucketPolicy on resource: "arn:aws:s3:::k8s-kops-ci-prow-state-store" because no resource-based policy allows the s3:GetBucketPolicy action
│ 
│   with aws_s3_bucket.kops_state_store,
│   on s3.tf line 18, in resource "aws_s3_bucket" "kops_state_store":
│   18: resource "aws_s3_bucket" "kops_state_store" {
│ 
╵
╷
│ Error: reading S3 Bucket (k8s-kops-ci-prow) policy: operation error S3: GetBucketPolicy, https response error StatusCode: 403, RequestID: PV742N6VN2323WQY, HostID: O5oqEfmAZTH1ZTfXKQdfeQWYU5uQ4Alj42anpiqarkEx6w57lVdE1x/lLjEXCf9Fi9tkYwphKOvZLqN8+NTp7BrOgMKPrKIa51a2RiywlIc=, api error AccessDenied: User: arn:aws:sts::348685125169:assumed-role/atlantis/1763470458794565738 is not authorized to perform: s3:GetBucketPolicy on resource: "arn:aws:s3:::k8s-kops-ci-prow" because no resource-based policy allows the s3:GetBucketPolicy action
│ 
│   with aws_s3_bucket.kops_oidc_store,
│   on s3.tf line 57, in resource "aws_s3_bucket" "kops_oidc_store":
│   57: resource "aws_s3_bucket" "kops_oidc_store" {
│ 
╵
╷
│ Error: Invalid count argument
│ 
│   on .terraform/modules/vpc/main.tf line 146, in resource "aws_subnet" "public":
│  146:   count = local.create_public_subnets && (!var.one_nat_gateway_per_az || local.len_public_subnets >= length(var.azs)) ? local.len_public_subnets : 0
│ 
│ The "count" value depends on resource attributes that cannot be determined
│ until apply, so Terraform cannot predict how many instances will be
│ created. To work around this, use the -target argument to first apply only
│ the resources that the count depends on.
╵
╷
│ Error: Invalid count argument
│ 
│   on .terraform/modules/vpc/main.tf line 177, in resource "aws_route_table" "public":
│  177:   count = local.create_public_subnets ? local.num_public_route_tables : 0
│ 
│ The "count" value depends on resource attributes that cannot be determined
│ until apply, so Terraform cannot predict how many instances will be
│ created. To work around this, use the -target argument to first apply only
│ the resources that the count depends on.
╵
╷
│ Error: Invalid count argument
│ 
│   on .terraform/modules/vpc/main.tf line 294, in resource "aws_subnet" "private":
│  294:   count = local.create_private_subnets ? local.len_private_subnets : 0
│ 
│ The "count" value depends on resource attributes that cannot be determined
│ until apply, so Terraform cannot predict how many instances will be
│ created. To work around this, use the -target argument to first apply only
│ the resources that the count depends on.
╵
╷
│ Error: Invalid count argument
│ 
│   on .terraform/modules/vpc/main.tf line 325, in resource "aws_route_table" "private":
│  325:   count = local.create_private_subnets && local.max_subnet_length > 0 ? local.nat_gateway_count : 0
│ 
│ The "count" value depends on resource attributes that cannot be determined
│ until apply, so Terraform cannot predict how many instances will be
│ created. To work around this, use the -target argument to first apply only
│ the resources that the count depends on.
╵
╷
│ Error: Invalid count argument
│ 
│   on .terraform/modules/vpc/main.tf line 1161, in resource "aws_internet_gateway" "this":
│ 1161:   count = local.create_public_subnets && var.create_igw ? 1 : 0
│ 
│ The "count" value depends on resource attributes that cannot be determined
│ until apply, so Terraform cannot predict how many instances will be
│ created. To work around this, use the -target argument to first apply only
│ the resources that the count depends on.
╵
╷
│ Error: Invalid count argument
│ 
│   on .terraform/modules/vpc/main.tf line 1175, in resource "aws_egress_only_internet_gateway" "this":
│ 1175:   count = local.create_vpc && var.create_egress_only_igw && var.enable_ipv6 && local.max_subnet_length > 0 ? 1 : 0
│ 
│ The "count" value depends on resource attributes that cannot be determined
│ until apply, so Terraform cannot predict how many instances will be
│ created. To work around this, use the -target argument to first apply only
│ the resources that the count depends on.
╵

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/infra/aws Issues or PRs related to Kubernetes AWS infrastructure area/infra Infrastructure management, infrastructure design, code in infra/ cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants