Skip to main content

cloud.gov Setup

Configuring Federalist in cloud.gov's GovCloud environment

Federalist is built to be deployed to the GovCloud region in cloud.gov. This guide is targeted at anyone who wishes to deploy the entire Federalist system to cloud.gov from the ground up, or anyone who wishes to get a better grasp of how the system is architected by looking at how it is deployed.

Before reading this guide, it may be helpful to refer to How Federalist Works to understand what components make up the Federalist architecture and how they are interrelated.

A Federalist deploy comprises of the following steps:

Deploying a private registry

Federalist's private registry stores the image that Federalist uses for its build containers. Under the hood it is a private Docker registry using an S3 storage driver with read-only mode enabled.

Push a registry to cloud.gov using the library/registry:2 image:

cf push federalist-registry -o library/registry:2

The registry needs an S3 bucket in which to store images. To this end a S3 service can be created:

cf create-service s3 basic federalist-registry-production-s3

After the registry and the S3 bucket are both up and running the registry's environment will need to be configured such that the registry uses the S3 bucket as a storage backend and is in read-only mode. This can be done by setting the registry environment as such:

REGISTRY_STORAGE: s3
REGISTRY_STORAGE_S3_ACCESSKEY: <ACCESS KEY FOR S3 SERVICE>
REGISTRY_STORAGE_S3_SECRETKEY: <SECRET KEY FOR S# SERVICE>
REGISTRY_STORAGE_S3_BUCKET: <BUCKET NAME FOR S3 SERVICE>
REGISTRY_STORAGE_S3_REGION: us-gov-west-1
REGISTRY_STORAGE_MAINTENANCE_READONLY: enabled: true

After setting the environment, the registry will need to be restaged to use the new configs:

cf restage federalist-registry

Pushing a federalist-garden-build image to the registry

Unfortunately pushing an image to the private registry is not as simple as building the image and pushing it to the registry. To push an image you must follow the following steps:

Building the image

The federalist-garden-build image can be built with docker. Clone the repo, cd into it and run docker build.

docker build --no-cache --tag federalist-garden-build .

The image should now appear in the list when you run docker images.

Scanning the image

Before the image can be pushed to the registry, it will need to be scanned by Clair. The steps to do this are:

Set some shell variables

Additionally, if you do not have your GOPATH set, you'll need to set that. Here is some documentation on how to set GOPATH.

Start up Clair with Docker and wait for vulnerability updates

We recommend starting Clair with Docker Compose. See the instructions at https://github.com/coreos/clair#docker-compose for how to set this up.

To start:

docker-compose up

It takes Clair a while (several hours) to update its vulnerability definitions and you might think it is being unresponsive. It is not finished updating until you see a message that looks something like:

{"Event":"adding metadata to vulnerabilities","Level":"info","Location":"updater.go:253","Time":"2017-06-23 19:46:09.100571"}

Use Clair CLI tool to scan the image

To actually scan the image we'll use the Clair analyze-local-images CLI tool.

First install the CLI tool according to the instructions at https://github.com/coreos/analyze-local-images#install

Clair's CLI tool copies the image to a tmp directory where Clair picks it up. In order for this to work, you need to change the value of the TMPDIR variable to /tmp: export TMPDIR=/tmp

Now, scan your image with analyze-local-images:

analyze-local-images <IMAGE_ID> > scan_<IMAGE_ID>_<DATE>.txt

where <IMAGE_ID> is the image ID for federalist-garden-build found by running docker images.

It can take a few minutes, but it should get back to you with a report about whatever vulnerabilities it finds. Save the results of the scan to the Federalist Clair Scans folder in Google Drive.

Pushing the image

Since the registry in cloud.gov is in read-only mode, it is not possible to push an image to it directly. In order to push an image to the registry a local registry with the same S3 storage driver is started and the image is pushed to that. The image is then saved in the S3 bucket where it is available to the remote registry.

We recommend using Docker Compose to run the local registry. To use Docker Compose you'll need to create a docker-compose.yml file. Here is an example of what a docker-compose.yml file would look like for a local registry:

registry:
  restart: always
  image: registry:2
  ports:
    - 5000:5000
  environment:
    REGISTRY_STORAGE: s3
    REGISTRY_STORAGE_S3_ACCESSKEY: <ACCESS KEY FOR S3 SERVICE>
    REGISTRY_STORAGE_S3_SECRETKEY: <SECRET KEY FOR S# SERVICE>
    REGISTRY_STORAGE_S3_BUCKET: <BUCKET NAME FOR S3 SERVICE>
    REGISTRY_STORAGE_S3_REGION: us-gov-west-1

After creating a docker-compose.yml, the registry can be started:

docker-compose up

By default the registry runs at localhost:5000. To push an image to the registry it will need to be tagged it with the URL for the local registry:

docker tag federalist-garden-build localhost:5000/federalist-garden-build

Finally, the image can be pushed to the local registry:

docker push localhost:5000/federalist-garden-build

Now the image should be available from the remote registry as <remote-registry-url>/federalist-garden-build.

To confirm that this worked you can pull the image from the remote registry and inspect it:

docker pull federalist-registry.fr.cloud.gov/federalist-garden-build
docker inspect federalist-registry.fr.cloud.gov/federalist-garden-build

Deploying build containers

Federalist's "build containers" are cloud.gov apps running with the federalist-garden-build image. The app needs to run without a healthcheck since it is not fitted to respond to HTTP requests.

To push the app use cf push:

cf push federalist-garden-build-1 --no-route -u none -o "federalist-registry.fr.cloud.gov/federalist-garden-build"

The first time the app is pushed it may fail to start. That is okay. The app cannot stand on its own without an environment provided by federalist / federalist-builder. It will be restarted with a valid environment when federalist-builder schedules a build on it.

Note that it is possible to push multiple build containers, as long as they have unique names.

Provisioning AWS resources

The next pieces of Federalist that need to be deployed will depend on certain AWS resources. That makes this a good time to provision the AWS resources that are needed:

How Federalist Works contains the details about how each of these should be configured.

Provisioning an RDS instance

The RDS instance can be provisioned with cloud.gov's RDS service broker and doesn't need any special configuration:

cf create-service aws-rds shared-psql federalist-production-rds

Provisioning a Redis instance

The Redis instance can be provisioned with cloud.gov's Redis service broker and doesn't need any special configuration:

cf create-service redis28 standard federalist-production-redis

Provisioning an S3 Bucket

The S3 bucket can be provisioned with cloud.gov's S3 service broker and requires some additional configuration:

cf create-service s3 basic-public federalist-production-s3

Configuration of the S3 bucket is done with the AWS CLI tool.

Once the bucket is created CORS will need to be enabled and configured to only serve GET requests that come from the main Federalist app's domain.

Next, the AWS CLI will need to be used to enable static website hosting in the bucket:

aws s3 website s3://${BUCKET_NAME}/site --index-document index.html

Provisioning an SQS queue

At this time cloud.gov does not have an SQS service broker. The SQS queue will need to be configured elsewhere.

Deploying federalist-builder

Note: federalist-builder is built to be deployed from TravisCI. The instructions for manually deploying federalist-builder are described below and are necessary for the initial deploy, but subsequent deploys should happen on Travis.

The first step to deploy federalist-builder is to configure the environment. The app's manifest.yml set's the app's environment. The manifest sets environment variables directly for non-secret configs. For secret configs it binds user-provided services.

federalist-builder has 2 user provided services that must be created before deploying in order for the app to work.

The first is federalist-ew-sqs-user which has the following values:

access_key: <AWS ACCESS KEY FOR SQS QUEUE>
secret_key: <AWS SECRET KEY FOR SQS QUEUE>

The second is federalist-deploy-user which has the following values:

DEPLOY_USER_USERNAME: <THE USERNAME OF THE FEDERALIST DEPLOY USER>
DEPLOY_USER_PASSWORD: <THE PASSWORD OF THE FEDERALIST DEPLOY USER>

The user provided services can be created with the Cloud Foundry CLI.

Once the user provided services are created deploying is a simple as running cf push.

Creating a GitHub OAuth Application

Federalist requires a GitHub OAuth application in order to work. The OAuth application should be setup under the GitHub organization that is deploying Federalist (e.g., the 18F org for any instance of Federalist used by 18F).

Adding an application is done within an organization's settings. Navigate to Settings > Developer Settings > OAuth Applications. Fill out all of the necessary fields. Make sure that the authorization URL is set to Federalist's OAuth authorization URL, e.g. https://federalist.18f.gov/auth.

When the application is created, save the client and secret keys to add to the federalist-<environment>-env user provided service as described below.

Deploying federalist

Note: federalist is built to be deployed from TravisCI. The instructions for manually deploying federalist are described below and are necessary for the initial deploy, but subsequent deploys should happen on Travis.

The first step to deploy federalist-builder is to configure the environment. The app's manifest.yml or staging_manifest.yml set's the app's environment. The manifest sets environment variables directly for non-secret configs. For secret configs it binds user-provided services.

The manifest specifies the following services which are provided by cloud.gov by cloud.gov service brokers:

In additional, there's a user provided service named federalist-production-env or federalist-staging-env depending on the environment. This user provided service should be created with the following values:

FEDERALIST_AWS_BUILD_KEY: <THE AWS ACCESS KEY FOR THE SQS QUEUE>
FEDERALIST_AWS_BUILD_SECRET: <THE AWS SECRET KEY FOR THE SQS QUEUE>
FEDERALIST_BUILD_CALLBACK: <CALLBACK URL FOR WHEN BUILDS ARE COMPLETE>
FEDERALIST_BUILD_TOKEN: <SECRET FOR AUTHENTICATING BUILD CALLBACK>
FEDERALIST_SESSION_SECRET": <SECRET FOR THE SAILS SESSION STORE>
FEDERALIST_SQS_QUEUE: <URL FOR SQS QUEUE>
GITHUB_CLIENT_CALLBACK_URL: <OAUTH CALLBACK URL FOR GITHUB AUTH>
GITHUB_CLIENT_ID: <CLIENT ID FOR GITHUB AUTH>
GITHUB_CLIENT_SECRET" <CLIENT SECRET FOR GITHUB AUTH>
GITHUB_WEBHOOK_SECRET: <SECRET FOR SIGNING GITHUB WEBHOOKS>
GITHUB_WEBHOOK_URL: <URL FOR GITHUB WEBHOOKS TO CALLBACK TO>

The user provided service can be created with the Cloud Foundry CLI.

Once the user provided services are created deploying is a simple as running cf push.

Federalist CDN Route

Federalist uses a CDN route service provided by cloud.gov to route traffic from the public domain to the application. To create such a service, run the following while targeting the appropriate space in cloud.gov:

cf create-user-provided-service cdn-route cdn-route federalist-route -c '{
  "domain": "federalist.18f.gov",
  "origin": "<URL WHERE FEDERALIST IS HOSTED>"
}'

Once this is done, running cf service federalist-route should give you a CloudFront domain name. Next, open a PR against the DNS repo to add a CNAME record for federalist.18f.gov with the CloudFront URL. Once the DNS changes propagate, Federalist should be available at federalist.18f.gov.

By default, the CDN caches error responses, so you will also need to work with cloud.gov support to change the mimimum error caching TTL to 0.

Rotating credentials

Federalist uses cloud.gov's space deployer service to deploy using CI. Additionally, the credentials for these deployer accounts are used by Federalist Builder to deploy build containers.

Occasionally, these credentials expire. When they do, it is necessary to regenerate the space deployer services and update the credentials. The credentials need to be updated in the user provided service and in Travis.

To update the credentials in Travis, go to the settings for federalist and federalist-builder. There the CF_USERNAME_STAGING, CF_USERNAME_PRODUCTION, CF_PASSWORD_STAGING, and CF_PASSWORD_PRODUCTION environment variables can be set to the correct values. This needs to be done for each app.

After the credentials are updated in CI, they need to be updated for the builder in staging and in production. The credentials live in a user provided service named federalist-deploy-user. That needs to be updated with new values for DEPLOY_USER_USERNAME and DEPLOY_USER_PASSWORD. To do that, see the docs on updating user provided services.