Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CES-36] - Unify terraform state files location #1254

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

christian-calabrese
Copy link
Contributor

@christian-calabrese christian-calabrese commented Oct 22, 2024

Motivation and Context

Terraform state files are currently placed in different storage accounts.
Starting from io-infra, it is necessary to unify the location of those state files and place them in Italy North.

Major Changes

Pointing all the terraform configurations of io-infra to the storage account in Italy North called iopitntfst001

Dependencies

Testing

Documentation

Other Considerations

⚠️ It is important to run the following script to actually copy the state files to the new storage accounts before merging this PR:

#!/bin/bash

# Variables - Set your own values here
DEST_STORAGE_ACCOUNT_NAME="iopitntfst001"
DEST_CONTAINER_NAME="terraform-state"

# Array of state files with corresponding source storage account and container
# Format: "source_storage_account_name:source_container_name:state_file_name"
STATE_FILES=(
"tfinfprodio:terraform-state:io-infra.aks-platform-weu-beta.tfstate"
"tfinfprodio:terraform-state:io-infra.aks-platform-weu-prod01.tfstate"
"tfinfprodio:terraform-state:io-infra.aks-platform-weu-prod02.tfstate"
"tfinfprodio:terraform-state:io-infra.cgn.tfstate"
"tfinfprodio:terraform-state:io-infra.citizen-auth-app-weu-prod01.tfstate"
"tfinfprodio:terraform-state:io-infra.citizen-auth-common-prod.tfstate"
"tfinfprodio:terraform-state:io-infra.continua.tfstate"
"tfinfprodio:terraform-state:io-infra.elk-weu-beta.tfstate"
"tfinfprodio:terraform-state:io-infra.elt.tfstate"
"tfinfprodio:terraform-state:io-infra.eucovidcert.tfstate"
"tfinfprodio:terraform-state:io-infra.functions.tfstate"
"tfinfprodio:terraform-state:io-infra.ioweb-app-weu-prod01.tfstate"
"tfinfprodio:terraform-state:io-infra.ioweb-common-prod.tfstate"
"tfinfprodio:terraform-state:io-infra.messages-app-weu-beta.tfstate"
"tfinfprodio:terraform-state:io-infra.messages-app-weu-prod01.tfstate"
"tfinfprodio:terraform-state:io-infra.messages-app-weu-prod02.tfstate"
"tfinfprodio:terraform-state:io-infra.messages-common-prod.tfstate"
"tfinfprodio:terraform-state:io-infra.payments-app-weu-beta.tfstate"
"tfinfprodio:terraform-state:io-infra.payments-app-weu-prod01.tfstate"
"tfinfprodio:terraform-state:io-infra.payments-app-weu-prod02.tfstate"
"tfinfprodio:terraform-state:io-infra.payments-common-prod.tfstate"
"tfinfprodio:terraform-state:io-infra.profile-app-weu-beta.tfstate"
"tfinfprodio:terraform-state:io-infra.profile-app-weu-prod01.tfstate"
"tfinfprodio:terraform-state:io-infra.profile-app-weu-prod02.tfstate"
"tfinfprodio:terraform-state:io-infra.profile-common-prod.tfstate"
"tfinfprodio:terraform-state:io-infra.selfcare.tfstate"
"tfinfprodio:terraform-state:io-infra.github-runner.tfstate"
"tfappprodio:terraform-state:io-infra.identity.tfstate"
"tfappprodio:terraform-state:io-infra.load-test.tfstate"
"tfinfprodio:terraform-state:packer-terraform.tfstate"
"tfappprodio:terraform-state:io-infra.repository.tfstate"
)

# Install azcopy if not already installed
if ! command -v azcopy &> /dev/null
then
    echo "azcopy could not be found. Installing..."
    brew install azcopy
    echo "azcopy installed."
else
    echo "azcopy is already installed."
fi

# Login to destination storage account
echo "Logging in to destination storage account..."
# azcopy login

# Copy specified files from their respective source storage accounts to the destination
for entry in "${STATE_FILES[@]}"; do
    # Extract source account, container, and file name
    IFS=":" read -r SOURCE_STORAGE_ACCOUNT_NAME SOURCE_CONTAINER_NAME STATE_FILE <<< "$entry"

    # Login to the source storage account
    echo "Logging in to source storage account $SOURCE_STORAGE_ACCOUNT_NAME..."
    # azcopy login

    # Copy the file from source to destination
    echo "Copying $STATE_FILE from $SOURCE_STORAGE_ACCOUNT_NAME to $DEST_STORAGE_ACCOUNT_NAME..."
    azcopy copy "https://$SOURCE_STORAGE_ACCOUNT_NAME.blob.core.windows.net/$SOURCE_CONTAINER_NAME/$STATE_FILE" "https://$DEST_STORAGE_ACCOUNT_NAME.blob.core.windows.net/$DEST_CONTAINER_NAME/$STATE_FILE"
done

echo "Copy completed for selected files."

# List copied files in the destination to verify
echo "Verifying files in the destination storage account..."
azcopy list "https://$DEST_STORAGE_ACCOUNT_NAME.blob.core.windows.net/$DEST_CONTAINER_NAME"

echo "All done."

Copy link

github-actions bot commented Oct 22, 2024

Terraform Plan ('src/domains/eucovidcert/prod/westeurope') 📖

Terraform Plan
module.function_apps.data.azurerm_linux_function_app.function_services[0]: Still reading... [10s elapsed]
module.function_apps.data.azurerm_linux_function_app.function_services[1]: Still reading... [10s elapsed]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.function_apps.module.function_eucovidcert.azurerm_linux_function_app.this will be updated in-place
  ~ resource "azurerm_linux_function_app" "this" {
        id                                             = "/subscriptions/ec285037-c673-4f58-b594-d7c480da4e8b/resourceGroups/io-p-rg-eucovidcert/providers/Microsoft.Web/sites/io-p-eucovidcert-fn"
        name                                           = "io-p-eucovidcert-fn"
        tags                                           = {
            "CostCenter"                                     = "TS310 - PAGAMENTI & SERVIZI"
            "CreatedBy"                                      = "Terraform"
            "Environment"                                    = "Prod"
            "ManagementTeam"                                 = "IO Wallet"
            "Owner"                                          = "IO"
            "Source"                                         = "https://github.com/pagopa/io-infra/blob/main/src/domains/eucovidcert/prod/westeurope"
        }
        # (32 unchanged attributes hidden)

      ~ site_config {
          ~ health_check_eviction_time_in_min             = 0 -> 10
            # (30 unchanged attributes hidden)

            # (6 unchanged blocks hidden)
        }

        # (1 unchanged block hidden)
    }

  # module.function_apps.module.function_eucovidcert_staging_slot.azurerm_linux_function_app_slot.this will be updated in-place
  ~ resource "azurerm_linux_function_app_slot" "this" {
        id                                             = "/subscriptions/ec285037-c673-4f58-b594-d7c480da4e8b/resourceGroups/io-p-rg-eucovidcert/providers/Microsoft.Web/sites/io-p-eucovidcert-fn/slots/staging"
        name                                           = "staging"
      # Warning: this attribute value will no longer be marked as sensitive
      # after applying this change. The value is unchanged.
      ~ storage_account_name                           = (sensitive value)
        tags                                           = {
            "CostCenter"     = "TS310 - PAGAMENTI & SERVIZI"
            "CreatedBy"      = "Terraform"
            "Environment"    = "Prod"
            "ManagementTeam" = "IO Wallet"
            "Owner"          = "IO"
            "Source"         = "https://github.com/pagopa/io-infra/blob/main/src/domains/eucovidcert/prod/westeurope"
        }
        # (29 unchanged attributes hidden)

      ~ site_config {
          ~ health_check_eviction_time_in_min             = 0 -> 10
            # (31 unchanged attributes hidden)

            # (7 unchanged blocks hidden)
        }
    }

Plan: 0 to add, 2 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.

Copy link

sonarcloud bot commented Nov 5, 2024

Copy link

github-actions bot commented Nov 5, 2024

📖 Terraform Plan ('src/load-test/prod') - success

Terraform Plan
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # azurerm_load_test.weu_common will be created
  + resource "azurerm_load_test" "weu_common" {
      + data_plane_uri      = (known after apply)
      + description         = "A Load Test resource with access to IO Prod environment to test private components"
      + id                  = (known after apply)
      + location            = "westeurope"
      + name                = "io-p-weu-common-lt-01"
      + resource_group_name = "io-p-weu-load-test-rg-01"
      + tags                = {
          + "CostCenter"  = "TS310 - PAGAMENTI & SERVIZI"
          + "CreatedBy"   = "Terraform"
          + "Environment" = "Prod"
          + "Owner"       = "IO"
          + "Source"      = "https://github.com/pagopa/io-infra/blob/main/src/load-test/prod"
        }

      + identity {
          + principal_id = (known after apply)
          + tenant_id    = (known after apply)
          + type         = "SystemAssigned"
        }
    }

  # azurerm_private_dns_zone_virtual_network_link.azurewebsites_private_vnet_itn_common will be created
  + resource "azurerm_private_dns_zone_virtual_network_link" "azurewebsites_private_vnet_itn_common" {
      + id                    = (known after apply)
      + name                  = "io-p-weu-load-test-vnet-01"
      + private_dns_zone_name = "privatelink.azurewebsites.net"
      + registration_enabled  = false
      + resource_group_name   = "io-p-rg-common"
      + tags                  = {
          + "CostCenter"  = "TS310 - PAGAMENTI & SERVIZI"
          + "CreatedBy"   = "Terraform"
          + "Environment" = "Prod"
          + "Owner"       = "IO"
          + "Source"      = "https://github.com/pagopa/io-infra/blob/main/src/load-test/prod"
        }
      + virtual_network_id    = (known after apply)
    }

  # azurerm_resource_group.load_test will be created
  + resource "azurerm_resource_group" "load_test" {
      + id       = (known after apply)
      + location = "westeurope"
      + name     = "io-p-weu-load-test-rg-01"
      + tags     = {
          + "CostCenter"  = "TS310 - PAGAMENTI & SERVIZI"
          + "CreatedBy"   = "Terraform"
          + "Environment" = "Prod"
          + "Owner"       = "IO"
          + "Source"      = "https://github.com/pagopa/io-infra/blob/main/src/load-test/prod"
        }
    }

  # azurerm_subnet.weu_load_test will be created
  + resource "azurerm_subnet" "weu_load_test" {
      + address_prefixes                               = [
          + "10.40.0.0/26",
        ]
      + enforce_private_link_endpoint_network_policies = (known after apply)
      + enforce_private_link_service_network_policies  = (known after apply)
      + id                                             = (known after apply)
      + name                                           = "io-p-weu-load-test-snet-01"
      + private_endpoint_network_policies              = (known after apply)
      + private_endpoint_network_policies_enabled      = (known after apply)
      + private_link_service_network_policies_enabled  = (known after apply)
      + resource_group_name                            = "io-p-weu-load-test-rg-01"
      + virtual_network_name                           = "io-p-weu-load-test-vnet-01"
    }

  # azurerm_virtual_network.weu_load_test will be created
  + resource "azurerm_virtual_network" "weu_load_test" {
      + address_space       = [
          + "10.40.0.0/22",
        ]
      + dns_servers         = (known after apply)
      + guid                = (known after apply)
      + id                  = (known after apply)
      + location            = "westeurope"
      + name                = "io-p-weu-load-test-vnet-01"
      + resource_group_name = "io-p-weu-load-test-rg-01"
      + subnet              = (known after apply)
      + tags                = {
          + "CostCenter"  = "TS310 - PAGAMENTI & SERVIZI"
          + "CreatedBy"   = "Terraform"
          + "Environment" = "Prod"
          + "Owner"       = "IO"
          + "Source"      = "https://github.com/pagopa/io-infra/blob/main/src/load-test/prod"
        }
    }

  # azurerm_virtual_network_peering.weu_common will be created
  + resource "azurerm_virtual_network_peering" "weu_common" {
      + allow_forwarded_traffic      = false
      + allow_gateway_transit        = false
      + allow_virtual_network_access = true
      + id                           = (known after apply)
      + name                         = "io-p-vnet-common-to-io-p-weu-load-test-vnet-01-peer"
      + remote_virtual_network_id    = (known after apply)
      + resource_group_name          = "io-p-rg-common"
      + use_remote_gateways          = false
      + virtual_network_name         = "io-p-vnet-common"
    }

  # azurerm_virtual_network_peering.weu_load_test will be created
  + resource "azurerm_virtual_network_peering" "weu_load_test" {
      + allow_forwarded_traffic      = false
      + allow_gateway_transit        = false
      + allow_virtual_network_access = false
      + id                           = (known after apply)
      + name                         = "io-p-weu-load-test-vnet-01-to-io-p-vnet-common-peer"
      + remote_virtual_network_id    = "/subscriptions/ec285037-c673-4f58-b594-d7c480da4e8b/resourceGroups/io-p-rg-common/providers/Microsoft.Network/virtualNetworks/io-p-vnet-common"
      + resource_group_name          = "io-p-weu-load-test-rg-01"
      + use_remote_gateways          = false
      + virtual_network_name         = "io-p-weu-load-test-vnet-01"
    }

Plan: 7 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + load_test_weu_common = {
      + id           = (known after apply)
      + location     = "westeurope"
      + name         = "io-p-weu-common-lt-01"
      + principal_id = (known after apply)
    }

─────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.

Copy link

github-actions bot commented Nov 5, 2024

📖 Terraform Plan ('src/domains/elt/prod') - success

Terraform Plan
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.function_apps.module.function_elt.azurerm_linux_function_app.this will be updated in-place
  ~ resource "azurerm_linux_function_app" "this" {
      ~ app_settings                                   = {
          ~ "COSMOSDB_REPLICA_URI"                                                               = "https://io-p-cosmos-api.documents.azure.com:443/" -> "https://io-p-cosmos-api-northeurope.documents.azure.com:443/"
            # (110 unchanged elements hidden)
        }
        id                                             = "/subscriptions/ec285037-c673-4f58-b594-d7c480da4e8b/resourceGroups/io-p-elt-rg/providers/Microsoft.Web/sites/io-p-fn-elt"
        name                                           = "io-p-fn-elt"
        tags                                           = {
            "CostCenter"                                     = "TS310 - PAGAMENTI & SERVIZI"
            "CreatedBy"                                      = "Terraform"
            "Environment"                                    = "Prod"
            "ManagementTeam"                                 = "IO Platform"
            "Owner"                                          = "IO"
            "Source"                                         = "https://github.com/pagopa/io-infra/blob/main/src/domains/elt/prod"
        }
        # (31 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

  # module.function_apps.module.function_elt.azurerm_service_plan.this[0] will be updated in-place
  ~ resource "azurerm_service_plan" "this" {
        id                           = "/subscriptions/ec285037-c673-4f58-b594-d7c480da4e8b/resourceGroups/io-p-elt-rg/providers/Microsoft.Web/serverFarms/io-p-plan-fnelt"
      ~ maximum_elastic_worker_count = 3 -> 1
        name                         = "io-p-plan-fnelt"
        tags                         = {
            "CostCenter"     = "TS310 - PAGAMENTI & SERVIZI"
            "CreatedBy"      = "Terraform"
            "Environment"    = "Prod"
            "ManagementTeam" = "IO Platform"
            "Owner"          = "IO"
            "Source"         = "https://github.com/pagopa/io-infra/blob/main/src/domains/elt/prod"
        }
        # (10 unchanged attributes hidden)
    }

Plan: 0 to add, 2 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants