Queue System Components

Queue mechanism components

The technical components that underpin the Queue System in Aegir:

  • A set of Docker container images which represent a reference implementation of the network model for a production Aegir deployment.
  • DDEV tooling to work with the backend dispatcherd container.
  • Drumkit targets for building and pulling the container images.
  • GitLabCI config to use the same images for our build pipelines.
  • Drush Aegir commands for posting input, logs, and exitcodes of Operations from dispatcherd.

Docker images

The core network model for an Aegir deployment is modeled by our DDEV localdev config. There are 3 main images we run (soon to be 4):

  • Web frontend (relayd)
  • Message queue service (rabbitmq)
  • Dispatcher backend (dispatcherd)
  • (Deployment target) (TBD)

Each of these images is built via Packer build scripts that handle building, tagging, and pushing the images. We delegate the actual provisioning of the contents of the contents of each image to a corresponding set of playbooks that in turn call tasks in an Ansible role, which allows them to be reused for provisioning a production Aegir environment.

Additionally, there is a small set of shell scripts which are called as part of the image build, doing things like installing apt packages or cleaning up unneeded components.

DDEV customization

We configure DDEV to use these images in the project using Docker Compose file(s); eg. .ddev/docker-compose.dispatcherd.yml which leverage the container images to provide service containers in DDEV. We also have a DDEV command to re-deploy and re-start the queue worker.

CI service images

The same container images are used in our GitLabCI config: .gitlab-ci.yml

Drumkit targets

Drumkit ties together these pieces. We have:

See Tutorial: Build container images for a quick introduction to these commands.