Optimize updates

Current architecture (Aegir 3)

Drupal platforms (codebases) are deployed under /var/aegir/platforms/. Sites are deployed using Drupal’s “multi-site” feature, at /var/aegir/platforms/PLATFORM_NAME/web/sites/SITE_NAME.

Current procedure

  1. put the site in maintenance mode
  2. take backup of site.
  3. Dump database
  4. Gzip tarball of db and site dir (including files and custom code)
  5. deploy new site on updated platform
  6. Unzip the backup to a temporary directory
  7. create a new db, and import the dump
  8. run update.php
  9. Rollback: delete temporary site
  10. Rewrite vhost to point to updated platform
  11. Delete old site.
  12. Take the updated site out of maintenance mode

This procedure is pessimistic about updates, and does not trust backup integrity. Thus, the site is offline throughout. The gzipping and unzipping is time-consuming and i/o intensive. So much so, that our worker daemon runs under ionice, and we run all tasks serially, to avoid simultaneous updates taking down a server. The database import is also time-consuming.

As a result of this pessimism, we incur significant downtime on every update, even when everything goes perfectly.

Proposed architecture (Ægir 5)

Drupal platforms are still deployed under /var/aegir/platforms/. However, sites are deployed at /var/aegir/sites/SITE_NAME. Again, using Drupal’s “multi-site” feature, we then symlink the site directory into the platform at /var/aegir/platforms/PLATFORM_NAME/web/sites/SITE_NAME.

Proposed procedure

  1. Take backup of site.
  2. Dump database
  3. Gzip tarball of db and site dir (including files and custom code)
  4. deploy test site
  5. Unzip the backup to a temporary path (e.g., /var/aegir/sites/SITE_NAME-test/)
  6. create a new db, and import the dump
  7. Symlink the test site into the updated platform.
  8. Run update.php
  9. (Optional) Run “update” suite of tests, if available.
  10. Delete test site
  11. Put the site in read-only mode, if available
  12. Fallback to using maintenance mode.
  13. Symlink the live site into the updated platform.
  14. Run update.php
  15. Rollback: restore database from backup
  16. Rewrite vhost to point to updated platform
  17. Take the updated site out of maintenance mode

This procedure is optimistic about updates, but still verifies every backup’s integrity. The site is only offline for the bare minimum to reliably run database updates. While the total time for the update is likely to be similar, downtime is minimized.