Skip to content

CI

ci

Our CI is a concourse deployment located at https://ci.cfdev.sh and the configuration files can be found at https://github.com/pivotal-cf/cfdev-ci. As of right now, the CI deployment has currently been spun down, but the CI repo contains everything needed to resurrect the pipelines and to build and test the products of CF Dev.

Tip

Additional information about CI jobs can be found here.

Pipelines

The pipelines are structured to reflect the main products consumed from the CF Dev team. The pipelines represent the OSS (cfdev) and closed-source (pcf) division between these products.

OSS

The OSS (cfdev) pipeline is responsible for building and testing the cfdev plugin, dependencies tarball and accessory binaries used for CF Dev. The dependencies tarball and accessory binaries are automatically available upon build via amazon s3. The cfdev plugin however is manually released to cf community plugins via a Github pull request to cloudfoundry/cli-plugin-repo. A Github Release is also made for the plugin when shipping.

Testing

The following sums up the testing jobs in the CFDEV pipeline:

  • acceptance (darwin, linux, windows) - Go through the lifecycle of the CF Dev plugin
  • acceptance-plus (darwin, linux, windows) - Go through the lifeycle of the CF Dev plugin, while exercising addtional features
  • meow (darwin) - Run CF Acceptance tests on a Cloud Foundry stood up by the CF Dev plugin

Releases

The following releases are made when the "ship-it" is invoked:

  • Github Release (cfdev plugin)
  • CF Community Plugins (cfdev plugin)

PCF

The closed-source (pcf) pipeline is responsible for building and testing the closed source dependencies tarball used for CF Dev. The dependencies tarball is only made available to Pivotal Network upon manually shipping.

Testing

The following sums up the testing jobs in the PCF pipeline:

  • acceptance (darwin, linux, windows) - Go through the lifecycle of the CF Dev plugin
  • acceptance-services (darwin, linux, windows) - Go through the lifeycle of the CF Dev plugin, while exercising addtional features and running smoke tests of packaged services
  • meow (darwin) - Run CF Acceptance tests on a Cloud Foundry stood up by the CF Dev plugin

Releases

The following releases are made when the "ship-it" is invoked:

  • Pivotal Network (dependencies tarball)

Meta

The Meta pipeline contains miscellanious jobs related to maintaining the deployment itself. These jobs include:

  • build-image - A job for automatically building the CI image pcfdev/ci-v2 when changes to the CI Dockerfile are pushed
  • build-runc-opsmanager - Provision a special flavor of opsmanager that extracts BOSH manifests and releases from tile deploys
  • ping-workers - A small job for keeping external workers talking to the main concourse cluster by executing a small task 2x a day

Deployment architecture

$ fly -t cfdev ws
name                                  containers  platform  tags              team  state    version  age
47c7c7a5-93d8-47b3-a1db-b78d0f800c0a  34          linux     none              none  running  2.2      261d
78f447f4-b74d-4a7e-a35a-374b8c9e14b2  45          linux     none              none  running  2.2      261d
linux-worker-1                        101         linux     linux-worker-1    none  running  2.2      261d
linux-worker-2                        6           linux     linux-worker-2    none  running  2.2      261d
macstadium-6                          1           darwin    macstadium-6      none  running  2.2      243d
macstadium-7                          3           darwin    macstadium-7      none  running  2.2      261d
macstadium-8                          1           darwin    macstadium-8      none  running  2.2      261d
macstadium-9                          3           darwin    macstadium-9      none  running  2.2      261d
windows-worker-1                      0           windows   windows-worker-1  none  running  2.2      199d
windows-worker-2                      0           windows   windows-worker-2  none  running  2.2      261d

The CF Dev Concourse is 1 BOSH concourse deployment that comprises of:

  • 2 linux containerized workers
  • 2 linux non-containerized external bare metal workers
  • 2 windows non-containerized external bare metal workers
  • 4 mac non-containerized external bare metal workers

For information on how to set up these workers, please reference the following docs:

Note

For information on how to access the Concourse BOSH director, please talk to a CF Dev team member

Updating pipelines

Updating the pipelines involves invoking the ./set-pipeline <PIPELINE-NAME> script. The script can be found in the CI repository. Logging into Lastpass is necessary before using.