Overview

This is a proof of concept lab to explore attestation options for Hyperledger Avalon. For background on attestation see References.

The lab provides an end to end example of the code, components, and configuration for generating and verifying remote attestations. The code can be used to learn the call flow for ECDSA based attestation. It also provides dockerfiles that illustrate build and runtime dependencies. To a lesser degree, the project can also be used as a "known good" example when experimenting in new environments or different configuraitons.

The code is written first and foremost to make the attestation API readable and simple. It is not written to be fully secure or robust. You can treat this as a simplified example before looking at official SGX examples.

The main attestation logic is in App/App.cpp. It loads a minimal enclave and uses the SGX SDK methods for creating an ECDSA based attestation of that enclave.

The complement to the enclave application is the relying party's verifier in RelyingParty/Verifier.cpp. This program will read in a binary quote generated by the application and verify it using SGX DCAP libraries. Those libraries rely on a service deployed in the cloud environment or within your local enviroment.

Quote generation (creating an attestation) has been verified on Azure ACC nodes only. Quote verification has been verified with Intel open source reference components only.

The project also includes a Docker-based dev environment to facilitate building and some debugging. The app also works within that environment if the app is built in simulation mode. Verification will correctly return errors when given simulation quotes (because you should not trust simulated quotes). The container is meant for development not for deployment. It may be possible to modify the container for deployment by mapping in /dev/sgx and the quote provider library (libdcap_quoteprov.so) but this has not been tested.

Similarly the verifier may be run in a container or other environment supplying the required services and libraries to test the portability of the quote generated by the attestor application.

TODO

• Add report data to app and verifier.
• Add verifier using Quote Verification Enclave and/or discuss deployment models where Quote Verification Library alone makes sense (this is probably the most applicable model for Avalon).
• Print DCAP Codes nicely

Build

Docker

From project root

1. build the container

   ./build-docker

2. run the container - this drops you into a shell

   ./run-docker

3. build in simulation mode [default in the container] or hardware mode

   make clean && make

   or

   make clean && SGX_MODE=HW make

   This will produce an enclave application, attestor, built for simulation or hardware and a verifier which does not rely on hardware but does rely on DCAP services.

Ubuntu 18.04

1. Install machine per instructions in Cloud Provisioning

2. build in hardware mode (default) or simulation mode

   make clean && make

   or

   make clean && SGX_MODE=SIM make

Running

(Note: when run within the dev container only simulation mode binary of the attestor app will work. However if you build in HW mode the binary will work on Ubuntu natively.)

1. Create an attestation (also called generating a quote)

   ./attestor

   This outputs a binary file attestation.bytes.

2. Verify the attestation

   • Native execution

     ./verifier

     This consumes the attestation file and indicates whether it is acceptable. (Note you can also copy attestation.bytes to another host and run verify from the verification container which demonstrates remotely verifying the enclave using the intel reference implementation verification stack.)

   • Container

     1. Build verifier container

        ./build-docker-verifier

     2. Run verifier container

        ./run-docker-verifier

     3. If necessary copy attestation.bytes to the verifier's host

        I.e. if you are running the verifier on a different host than the attestor. The verifier will look for the attestation in the same directory as the verifier binary. The project root (e.g. /home/you/dancap) is mapped into the verifier container at /project/dancap.

     4. Run the verifier from the container's shell

        ./verifier

   Expect output to look like this:

   $ ./verifier
   Reading 4584 bytes... Read attestation file successfully.
   Success: Quote verification PASSED
   
   Verification result code: 0
   

   (The Verification result code may differ based on the platform patch level)

   If instead you get an error 19 (when running in the container):

   root@4f91bb8bbfd5:/project/dancap# ./verifier
   Reading 4580 bytes... Read attestation file successfully.
   ERROR: Quote verification FAILED with error: e019
   Verification error code: e006
   

   Please check your proxy settings and then make sure that the PCCS service is running:

   pm2 status

   If you don't see PCCS listed then you can start it as follows:

   cd /opt/intel/sgx-dcap-pccs/
   pm2 start pccs_server.config.js
   

   If you created the attestation using simulation expect an e01d (SGX_QL_QUOTE_FORMAT_UNSUPPORTED). This is expected because the attestation is just simulated and should not verify as an actual hardware attestation.

   root@aacd7eebf44a:/project/dancap# ./verifier
   Reading 1116 bytes... Read attestation file successfully.
   ERROR: Quote verification FAILED with error: e01d
   Verification error code: a006
   

Cloud Provisioning

Azure Confidential Compute

1. Provision an ACC node with

   • Ubuntu 18.04
   • Open Enclave SDK ** This is necessary to get the attestation dependencies installed correctly within Azure. See: https://www.intel.com/content/www/us/en/developer/articles/guide/get-started-with-azure-confidential-computing.html for more details.

2. Upgrade the machine apt-get update && apt-get upgrade

3. Install the Intel(r) SGX SDK

    sudo wget \
        https://download.01.org/intel-sgx/sgx-linux/2.9/distro/ubuntu18.04-server/sgx_linux_x64_sdk_2.9.100.2.bin \
        && sudo chmod +x sgx_linux_x64_sdk_2.9.100.2.bin \
        && echo "yes" | sudo ./sgx_linux_x64_sdk_2.9.100.2.bin \
        && sudo rm sgx_linux_x64_sdk_2.9.100.2.bin \
        && sudo ln -s /opt/intel/sgxsdk/lib64/libsgx_quote_ex_sim.so /opt/intel/sgxsdk/sdk_libs/ \
        && sudo sh -c 'echo ". /opt/intel/sgxsdk/environment" >> /etc/environment' 

1. Install the runtime quote (attestation) libraries

    sudo apt-get install libsgx-quote-ex

1. Set library search order to pick up Azure's quote provider library rather than the one available in the SGX common path. You can do this the right way by managing /etc/ld.so.conf Or you can do it the expedient way by removing the one we don't want.

   We want this one: $ /usr/lib/libdcap_quoteprov.so

   we do NOT want this one: $ /opt/intel/libsgx-enclave-common/aesm/libdcap_quoteprov.so

2. Finally add to your environment (e.g. /etc/environment or ~/.bash_profile)

   AZDCAP_COLLATERAL_VERSION=v2

Misc

The project has a helper function to print SGX error codes and messages. This helper function can be regenerated if the SGX SDK changes using the python script in App/:

App/generate_handle_sgx_error.py > App/handle_sgx_error.h

DCAP codes output by the verifier can be looked up here: https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteGeneration/quote_wrapper/common/inc/sgx_ql_lib_common.h

and here:

https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteVerification/QvE/Include/sgx_qve_header.h

References

• https://download.01.org/intel-sgx/dcap-1.1/linux/docs/Intel_SGX_DCAP_ECDSA_Orientation.pdf
• https://software.intel.com/en-us/blogs/2018/12/09/an-update-on-3rd-party-attestation
• https://01.org/intel-softwareguard-extensions/downloads/intel-sgx-dcap-1.6-release
• https://software.intel.com/sites/default/files/managed/f1/b8/intel-sgx-support-for-third-party-attestation.pdf
• https://github.com/intel/SGXDataCenterAttestationPrimitives