Skip to content

Commit

Permalink
DST-257: install infra temp (#3)
Browse files Browse the repository at this point in the history
Resolves #DST-257

Configured infra template
  • Loading branch information
ccheng26 authored Jun 24, 2024
1 parent 603efaf commit ce2baee
Show file tree
Hide file tree
Showing 185 changed files with 9,792 additions and 32 deletions.
6 changes: 1 addition & 5 deletions .dockleconfig
Original file line number Diff line number Diff line change
@@ -1,8 +1,4 @@
# This file is allows you to specify a list of files that is acceptable to Dockle
# To allow multiple files, use a list of names, example below. Make sure to remove the leading #
# DOCKLE_ACCEPT_FILES="file1,path/to/file2,file3/path,etc"
# https://github.com/goodwithtech/dockle#accept-suspicious-environment-variables--files--file-extensions
# DOCKLE_ACCEPT_FILES="file1,path/to/file2,file3/path,etc"

# The apiflask/settings file is a stub file that apiflask creates, and has no sensitive data in. We are ignoring it since it is unused
DOCKLE_ACCEPT_FILES=app/.venv/lib/python3.12/site-packages/apiflask/settings.py
# https://github.com/goodwithtech/dockle#accept-suspicious-environment-variables--files--file-extensions
58 changes: 58 additions & 0 deletions .github/actions/configure-aws-credentials/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: 'Configure AWS Credentials'
description: 'Configure AWS Credentials for a given application and |
environment so that the GitHub Actions workflow can access AWS resources. |
This is a wrapper around https://github.com/aws-actions/configure-aws-credentials |
that first determines the account, role, and region based on the |
account_names_by_environment configuration in app-config'
inputs:
app_name:
description: 'Name of application folder under /infra'
required: true
environment:
description: 'Name of environment (dev, staging, prod) that AWS resources live in, or "shared" for resources that are shared across environments'
required: true
runs:
using: "composite"
steps:
- name: Get AWS account authentication details (AWS account, IAM role, AWS region)
run: |
# Get AWS account authentication details (AWS account, IAM role, AWS region)
# associated with the application environment to figure out which AWS
# account to log into, which IAM role to assume, and which AWS region to use
echo "::group::AWS account authentication details"
terraform -chdir=infra/project-config init > /dev/null
terraform -chdir=infra/project-config apply -auto-approve > /dev/null
AWS_REGION=$(terraform -chdir=infra/project-config output -raw default_region)
echo "AWS_REGION=$AWS_REGION"
GITHUB_ACTIONS_ROLE_NAME=$(terraform -chdir=infra/project-config output -raw github_actions_role_name)
echo "GITHUB_ACTIONS_ROLE_NAME=$GITHUB_ACTIONS_ROLE_NAME"
terraform -chdir=infra/${{ inputs.app_name }}/app-config init > /dev/null
terraform -chdir=infra/${{ inputs.app_name }}/app-config apply -auto-approve > /dev/null
ACCOUNT_NAME=$(terraform -chdir=infra/${{ inputs.app_name }}/app-config output -json account_names_by_environment | jq -r .${{ inputs.environment }})
echo "ACCOUNT_NAME=$ACCOUNT_NAME"
# Get the account id associated with the account name extracting the
# ACCOUNT_ID part of the tfbackend file name which looks like
# <ACCOUNT_NAME>.<ACCOUNT_ID>.s3.tfbackend.
# The cut command splits the string with period as the delimeter and
# extracts the second field.
ACCOUNT_ID=$(ls infra/accounts/$ACCOUNT_NAME.*.s3.tfbackend | cut -d. -f2)
echo "ACCOUNT_ID=$ACCOUNT_ID"
AWS_ROLE_TO_ASSUME=arn:aws:iam::$ACCOUNT_ID:role/$GITHUB_ACTIONS_ROLE_NAME
echo "AWS_ROLE_TO_ASSUME=$AWS_ROLE_TO_ASSUME"
echo "::endgroup::"
echo "Setting env vars AWS_ROLE_TO_ASSUME and AWS_REGION..."
echo "AWS_ROLE_TO_ASSUME=$AWS_ROLE_TO_ASSUME" >> "$GITHUB_ENV"
echo "AWS_REGION=$AWS_REGION" >> "$GITHUB_ENV"
shell: bash
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v3
with:
role-to-assume: ${{ env.AWS_ROLE_TO_ASSUME }}
aws-region: ${{ env.AWS_REGION }}
19 changes: 3 additions & 16 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,15 @@
- [ ] Update PR Title to follow this pattern: `[INTENT]: [MESSAGE]`

> The title will become a one-line commit message in the git log, so be as concise and specific as possible -- refer to [How to Write a Git Commit Message](https://cbea.ms/git-commit/). Prepend [Conventional Commit](https://www.conventionalcommits.org/en/v1.0.0/#summary) intent (`fix:`, `feat:`, `chore:`, `ci:`, `docs:`, `style:`, `refactor:`, `perf:`, `test:`).
## Ticket

Resolves #{TICKET NUMBER or URL or description}

Resolves #{TICKET NUMBER OR URL}

## Changes

> What was added, updated, or removed in this PR.
> Prefer small PRs; try to limit to 300 lines of code changes
> * https://blog.logrocket.com/using-stacked-pull-requests-in-github/
> * https://opensource.com/article/18/6/anatomy-perfect-pull-request
> * https://developers.google.com/blockly/guides/modify/contribute/write_a_good_pr
## Context for reviewers

> Background context, more in-depth details of the implementation, and anything else you'd like to call out or ask reviewers.
> Add comments to your code under the "Files Changed" tab to explain complex logic or code
> * https://betterprogramming.pub/how-to-make-a-perfect-pull-request-3578fb4c112
> Testing instructions, background context, more in-depth details of the implementation, and anything else you'd like to call out or ask reviewers.
## Testing

> Provide evidence that the code works as expected. Explain what was done for testing and the results of the test plan. Include screenshots, [GIF demos](https://www.cockos.com/licecap/), shell commands or output to help show the changes working as expected. ProTip: you can drag and drop or paste images into this textbox.
> Provide evidence that the code works as expected. Explain what was done for testing and the results of the test plan. Include screenshots, [GIF demos](https://www.cockos.com/licecap/), shell commands or output to help show the changes working as expected. ProTip: you can drag and drop or paste images into this textbox.
47 changes: 47 additions & 0 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# CI/CD

The CI/CD for this project uses [reusable Github Actions workflows](https://docs.github.com/en/actions/using-workflows/reusing-workflows).

## 🧪 CI

### Per app workflows

Each app should have:

- `ci-[app_name]`: must be created; should run linting and testing
- `ci-[app_name]-vulnerability-scans`: calls `vulnerability-scans`
- Based on [ci-app-vulnerability-scans](https://github.com/navapbc/template-infra/blob/main/.github/workflows/ci-app-vulnerability-scans.yml)

### App-agnostic workflows

- [`ci-docs`](./ci-docs.yml): runs markdown linting on all markdown files in the file
- Configure in [markdownlint-config.json](./markdownlint-config.json)
- [`ci-infra`](./ci-infra.yml): run infrastructure CI checks

## 🚢 CD

Each app should have:

- `cd-[app_name]`: deploys an application
- Based on [`cd-app`](https://github.com/navapbc/template-infra/blob/main/.github/workflows/cd-app.yml)

The CD workflow uses these reusable workflows:

- [`deploy`](./deploy.yml): deploys an application
- [`database-migrations`](./database-migrations.yml): runs database migrations for an application
- [`build-and-publish`](./build-and-publish.yml): builds a container image for an application and publishes it to an image repository

```mermaid
graph TD
cd-app
deploy
database-migrations
build-and-publish
cd-app-->|calls|deploy-->|calls|database-migrations-->|calls|build-and-publish
```

## ⛑️ Helper workflows

- [`check-infra-auth`](./check-infra-auth.yml): verifes that the project's Github repo is able to connect to AWS

76 changes: 76 additions & 0 deletions .github/workflows/build-and-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
name: Build and publish
run-name: Build and publish ${{ inputs.app_name }}:${{ inputs.ref }}

on:
workflow_call:
inputs:
app_name:
description: "name of application folder under infra directory"
required: true
type: string
ref:
description: The branch, tag or SHA to checkout. When checking out the repository that triggered a workflow, this defaults to the reference or SHA for that event. Otherwise, use branch or tag that triggered the workflow run.
required: true
type: string
workflow_dispatch:
inputs:
app_name:
description: "name of application folder under infra directory"
required: true
type: string
ref:
description: The branch, tag or SHA to checkout. When checking out the repository that triggered a workflow, this defaults to the reference or SHA for that event. Otherwise, use branch or tag that triggered the workflow run.
required: true
type: string

jobs:
get-commit-hash:
name: Get commit hash
runs-on: ubuntu-latest
outputs:
commit_hash: ${{ steps.get-commit-hash.outputs.commit_hash }}
steps:
- uses: actions/checkout@v3
with:
ref: ${{ inputs.ref }}
- name: Get commit hash
id: get-commit-hash
run: |
COMMIT_HASH=$(git rev-parse ${{ inputs.ref }})
echo "Commit hash: $COMMIT_HASH"
echo "commit_hash=$COMMIT_HASH" >> "$GITHUB_OUTPUT"
build-and-publish:
name: Build and publish
runs-on: ubuntu-latest
needs: get-commit-hash
concurrency: ${{ github.workflow }}-${{ needs.get-commit-hash.outputs.commit_hash }}

permissions:
contents: read
id-token: write

steps:
- uses: actions/checkout@v3
with:
ref: ${{ inputs.ref }}

- name: Configure AWS credentials
uses: ./.github/actions/configure-aws-credentials
with:
app_name: ${{ inputs.app_name }}
environment: shared

- name: Check if image is already published
id: check-image-published
run: |
is_image_published=$(./bin/is-image-published "${{ inputs.app_name }}" "${{ inputs.ref }}")
echo "Is image published: $is_image_published"
echo "is_image_published=$is_image_published" >> "$GITHUB_OUTPUT"
- name: Build release
if: steps.check-image-published.outputs.IS_IMAGE_PUBLISHED == 'false'
run: make APP_NAME=${{ inputs.app_name }} release-build

- name: Publish release
if: steps.check-image-published.outputs.IS_IMAGE_PUBLISHED == 'false'
run: make APP_NAME=${{ inputs.app_name }} release-publish
33 changes: 33 additions & 0 deletions .github/workflows/cd-app.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Deploy App
# Need to set a default value for when the workflow is triggered from a git push
# which bypasses the default configuration for inputs
run-name: Deploy ${{ github.ref_name }} to App ${{ inputs.environment || 'dev' }}

on:
# !! Uncomment the following lines once you've set up the dev environment and ready to turn on continuous deployment
# push:
# branches:
# - "main"
# paths:
# - "app/**"
# - "bin/**"
# - "infra/**"
workflow_dispatch:
inputs:
environment:
description: "target environment"
required: true
default: "dev"
type: choice
options:
- dev
- staging
- prod

jobs:
deploy:
name: Deploy
uses: ./.github/workflows/deploy.yml
with:
app_name: "app"
environment: ${{ inputs.environment || 'dev' }}
30 changes: 30 additions & 0 deletions .github/workflows/check-infra-auth.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Check GitHub Actions AWS Authentication

on:
workflow_dispatch:
inputs:
aws_region:
description: AWS region
default: us-east-1
required: false
role_to_assume:
description: ARN of IAM role to assume
required: true

permissions:
contents: read
id-token: write

jobs:
caller-identity:
name: Check caller identity
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v3
with:
aws-region: ${{ inputs.aws_region }}
role-to-assume: ${{ inputs.role_to_assume }}
- run: aws sts get-caller-identity
26 changes: 26 additions & 0 deletions .github/workflows/ci-app-vulnerability-scans.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: CI Vulnerability Scans

on:
push:
branches:
- main
paths:
- app/**
- .grype.yml
- .hadolint.yaml
- .trivyignore
- .github/workflows/ci-app-vulnerability-scans.yml
pull_request:
paths:
- app/**
- .grype.yml
- .hadolint.yaml
- .trivyignore
- .github/workflows/ci-app-vulnerability-scans.yml

jobs:
vulnerability-scans:
name: Vulnerability Scans
uses: ./.github/workflows/vulnerability-scans.yml
with:
app_name: "app"
20 changes: 20 additions & 0 deletions .github/workflows/ci-docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: CI Documentation Checks

on:
push:
branches:
- main
pull_request:


jobs:
lint-markdown:
name: Lint markdown
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
# This is the GitHub Actions-friendly port of the linter used in the Makefile.
- uses: gaurav-nelson/[email protected]
with:
use-quiet-mode: 'yes' # errors only.
config-file: '.github/workflows/markdownlint-config.json'
50 changes: 50 additions & 0 deletions .github/workflows/ci-infra-service.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: CI Infra Service Checks

on:
# !! Uncomment to trigger automated infra tests once dev environment is set up
# push:
# branches:
# - main
# paths:
# - infra/*/service/**
# - infra/modules/**
# - infra/test/**
# - .github/workflows/ci-infra-service.yml
# pull_request:
# paths:
# - infra/*/service/**
# - infra/modules/**
# - infra/test/**
# - .github/workflows/ci-infra-service.yml
workflow_dispatch:

jobs:
infra-test-e2e:
name: Test service
runs-on: ubuntu-latest

permissions:
contents: read
id-token: write

steps:
- uses: actions/checkout@v3

- uses: hashicorp/setup-terraform@v2
with:
terraform_version: 1.8.3
terraform_wrapper: false

- uses: actions/setup-go@v3
with:
go-version: ">=1.19.0"

- name: Configure AWS credentials
uses: ./.github/actions/configure-aws-credentials
with:
app_name: app
# Run infra CI on dev environment
environment: dev

- name: Run Terratest
run: make infra-test-service
Loading

0 comments on commit ce2baee

Please sign in to comment.