docs(README): Add instructions for airgap deployment (#160)

* docs(README): Add instructions for airgap deployment

- Provide detailed steps for how to move Docker images across an airgap.
- The rest of the installation is the same.
- Also refactor the way we build certs into the Docker image. The old approach
  required editing the dockerfile, which was led certain people (i.e. myself) to
  accidentally check in the Dockerfile with their own issuer cert configured in
  it. The new approach passes the information in the environment.

Dockerfile

@@ -17,6 +17,14 @@ LABEL "org.opencontainers.image.source"="https://github.com/center-for-threat-in
 LABEL "org.opencontainers.image.description"="Threat Report ATT&CK Mapper"
 LABEL "org.opencontainers.image.license"="Apache-2.0"
 
+# Arguments
+ARG TRAM_CA_URL
+ARG TRAM_CA_THUMBPRINT
+
+# Change default shell to bash so that we can use pipes (|) safely. See:
+# https://github.com/hadolint/hadolint/wiki/DL4006
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
 # Install and update apt dependencies
 ENV DEBIAN_FRONTEND=noninteractive
 RUN apt-get update && \
@@ -31,11 +39,7 @@ RUN apt-get update && \
     python3-wheel && \
     rm -fr /var/lib/apt/lists/*
 
-# Add proxy settings, enterprise certs to prevent SSL issues.
-# Uncomment and update these lines as needed. There is an example
-# with a wget if you can/want to download the cert directly from your
-# organization, otherwise use the COPY command to move it into the
-# docker build, and call run the `update-ca-certificates` command.
+# Add proxy settings. Uncomment and update these lines as needed.
 #ENV proxy_host=${proxy_host:-proxy-server.my-organization.local} \
 #    proxy_port=${proxy_port:-80}
 #ENV http_proxy=http://${proxy_host}:${proxy_port} \
@@ -44,10 +48,19 @@ RUN apt-get update && \
 #ENV HTTP_PROXY=${http_proxy} \
 #    HTTPS_PROXY=${https_proxy} \
 #    NO_PROXY=${no_proxy}
-#COPY MY_ORG_ROOT.crt /usr/local/share/ca-certificates
-#RUN cd /usr/local/share/ca-certificates && \
-#    wget http://pki.my-organization.local/MY%20ORG%20ROOT.crt && \
-#    update-ca-certificates
+
+# Handle custom CA certificate, if specified.
+RUN if test -n "${TRAM_CA_URL}" -a -n "${TRAM_CA_THUMBPRINT}" ; then \
+        echo "Installing certificate authority from ${TRAM_CA_URL}" && \
+        curl -sk "${TRAM_CA_URL}" -o /usr/local/share/ca-certificates/tram_ca.crt && \
+        DOWNLOAD_CA_THUMBPRINT=$(openssl x509 -in /usr/local/share/ca-certificates/tram_ca.crt -fingerprint -noout | cut -d= -f2) && \
+        if test "${DOWNLOAD_CA_THUMBPRINT}" = "${TRAM_CA_THUMBPRINT}"; then \
+            update-ca-certificates; \
+        else \
+            printf "\n=====\nERROR\nExpected thumbprint: %s\nActual thumbprint:   %s\n=====\n" "${TRAM_CA_THUMBPRINT}" "${DOWNLOAD_CA_THUMBPRINT}"; \
+            exit 1; \
+        fi; \
+    fi
 
 RUN mkdir /tram && \
     python3 -m venv /tram/.venv && \

Makefile

@@ -49,9 +49,11 @@ pre-commit-run-all: venv .git/hooks/pre-commit ## Run pre-commit manually on all
 
 .PHONY: build-container
 build-container: venv ## Build container image
-	docker build -t $(APP_NAME):dev -t $(APP_NAME):$(TIMESTAMP)_$(GIT_HASH) -f Dockerfile . --label "org.opencontainers.image.revision=$(GIT_HASH)"
-	docker build -t $(APP_NAME)-nginx:dev -t $(APP_NAME)-nginx:$(TIMESTAMP)_$(GIT_HASH) -f docker/Dockerfile.nginx . --label "org.opencontainers.image.revision=$(GIT_HASH)"
-
+	docker build -t $(APP_NAME):dev -t $(APP_NAME):$(TIMESTAMP)_$(GIT_HASH) \
+		-f Dockerfile . --label "org.opencontainers.image.revision=$(GIT_HASH)" \
+		--build-arg TRAM_CA_URL=$(TRAM_CA_URL) --build-arg TRAM_CA_THUMBPRINT=$(TRAM_CA_THUMBPRINT)
+	docker build -t $(APP_NAME)-nginx:dev -t $(APP_NAME)-nginx:$(TIMESTAMP)_$(GIT_HASH) \
+		-f docker/Dockerfile.nginx . --label "org.opencontainers.image.revision=$(GIT_HASH)"
 
 .PHONY: start-container
 start-container: ## Start container via docker-compose, runs ctidorg/tram:latest image by default

README.md

@@ -21,8 +21,9 @@ to integrate ATT&CK more easily and consistently into their products.
 - [TRAM](#tram)
   - [Table of contents](#table-of-contents)
   - [Installation](#installation)
+    - [Air Gap Installation](#air-gap-installation)
   - [Installation Troubleshooting](#installation-troubleshooting)
-    - [[97438] Failed to execute script docker-compose](#97438-failed-to-execute-script-docker-compose)
+    - [\[97438\] Failed to execute script docker-compose](#97438-failed-to-execute-script-docker-compose)
   - [Report Troubleshooting](#report-troubleshooting)
     - [How long until my queued report is complete?](#how-long-until-my-queued-report-is-complete)
     - [Why is my report stuck in queued?](#why-is-my-report-stuck-in-queued)
@@ -31,6 +32,7 @@ to integrate ATT&CK more easily and consistently into their products.
   - [For Developers](#for-developers)
     - [Developer Setup](#developer-setup)
     - [Makefile Targets](#makefile-targets)
+    - [Custom CA Certificate](#custom-ca-certificate)
   - [Machine Learning Development](#machine-learning-development)
     - [Existing ML Models](#existing-ml-models)
     - [Creating Your Own ML Model](#creating-your-own-ml-model)
@@ -62,9 +64,58 @@ to integrate ATT&CK more easily and consistently into their products.
 password specified in docker-compose.yml
 ![image](https://user-images.githubusercontent.com/2951827/129959436-d36e8d1f-fe74-497e-b549-a74be8d140ca.png)
 
+### Air Gap Installation
+
+If you are unable to pull images from Docker Hub (i.e. due to corporate
+firewall, airgap, etc.), it is possible to download the images and move them
+onto the Docker host manually:
+
+1. Pull the images onto a machine that is able to access Docker Hub:
+
+    ```shell
+    $ docker pull ghcr.io/center-for-threat-informed-defense/tram:latest
+    $ docker pull ghcr.io/center-for-threat-informed-defense/tram-nginx:latest
+    ```
+
+2. Export the Docker images to compressed archive (`.tgz`) format:
+
+    ```shell
+    $ docker save ghcr.io/center-for-threat-informed-defense/tram:latest \
+        | gzip > tram-latest.tgz
+    $ docker save ghcr.io/center-for-threat-informed-defense/tram-nginx:latest \
+        | gzip > tram-nginx-latest.tgz
+    ```
+3. Confirm that the images were exported correctly.
+
+    ```shell
+    ls -lah tram*.tgz
+    -rw-r--r--  1 johndoe  wheel   345M Feb 24 12:56 tram-latest.tgz
+    -rw-r--r--  1 johndoe  wheel   9.4M Feb 24 12:57 tram-nginx-latest.tgz
+    ```
+
+4. Copy the images across the airgap.
+   - _This step will depend on your deployment environment, of course._
+
+5. Import the Docker images on the Docker host.
+
+    ```shell
+    $ docker load < tram-latest.tgz
+    $ docker load < tram-nginx-latest.tar.gz
+    ```
+
+6. Confirm that the images were loaded on the Docker host.
+
+    ```shell
+    $ docker images | grep tram
+    ghcr.io/center-for-threat-informed-defense/tram-nginx   latest    8fa8fb7801b9   2 weeks ago    23.5MB
+    ghcr.io/center-for-threat-informed-defense/tram         latest    d19b35523098   2 weeks ago    938MB
+    ```
+
+7. From this point, you can follow the main installation instructions above.
+
 ## Installation Troubleshooting
 
-### [97438] Failed to execute script docker-compose
+### \[97438\] Failed to execute script docker-compose
 
 If you see this stack trace:
 
@@ -215,13 +266,40 @@ containerized version is recommended for non-developers.
   - `make install-dev`
 - Manually run pre-commit hooks without performing a commit
   - `make pre-commit-run`
-- Build container image (By default, container is tagged with timestamp and git hash of codebase)
+- Build container image (By default, container is tagged with timestamp and git hash of codebase) _See note below about custom CA certificates in the Docker build.)_
   - `make build-container`
 - Run linting locally
   - `make lint`
 - Run unit tests, safety, and bandit locally
   - `make test`
 
+### Custom CA Certificate
+
+If you are building the container in an environment that intercepts SSL
+connections, you can specify a root CA certificate to inject into the container
+at build time. (This is only necessary for the TRAM application container. The
+TRAM Nginx container does not make outbound connections.)
+
+Export the following two variables in your environment.
+
+```shell
+$ export TRAM_CA_URL="http://your.domain.com/root.crt"
+$ export TRAM_CA_THUMBPRINT="C7:E0:F9:69:09:A4:A3:E7:A9:76:32:5F:68:79:9A:85:FD:F9:B3:BD"
+```
+
+The first variable is a URL to a PEM certificate containing a root certificate
+that you want to inject into the container. (If you use an `https` URL, then
+certificate checking is disabled.) The second variable is a SHA-1 certificate
+thumbprint that is used to verify that the correct certificate was downloaded.
+You can obtain the thumbprint with the following OpenSSL command:
+
+```shell
+$ openssl x509 -in <your-cert.crt> -fingerprint -noout
+SHA1 Fingerprint=C7:E0:F9:69:09:A4:A3:E7:A9:76:32:5F:68:79:9A:85:FD:F9:B3:BD
+```
+
+After exporting these two variables, you can run `make build-container` as usual
+and the TRAM container will contain your specified root certificate.
 
 ## Machine Learning Development