Architecting million dollar projects

Huge traffic, Transactions, CMS as a platform, Decoupled implementations

I was inspired by a webinar I recently attended, Architecting Growth–How to Crush Million Dollar Projects by Pantheon – a geek friendly managed host for enterprise WordPress and Drupal clients. This webinar is presented by their co-founder and discusses much of what they’ve learned providing infrastructure for larger scale projects.

This presentation covers their content management system (CMS) software-as-a-service (SaaS) delivery model in addressing scalability problems often seen in eCommerce, viral causes, campaigns, and news cycle use cases. Some challenges that arise from these use cases are:

  • Traffic spike handling
  • Transaction / processing growth handling
  • Expanding sites/properties that demand CMS as a platform
  • Pitfalls of coupled architectures

Conventional wisdom holds that CMS systems cannot well handle large scale traffic. They get into their methodology of combining heavy caching with horizontal scaling architecture and DevOps best practices to make the CMS powered enterprise work at scale.

Here’s a bit about their caching and scaling architecture:

  • Edge layer – secure socket layer (SSL) termination, content delivery network (CDN), cached static pages/posts, geographically disbursed
  • Service layer – Load balanced application nodes (web servers) on autoscale running NginX and PHP7
  • Micro-service layer – Caching with Redis objects/transients, micro-service database with replication for read optimization, home built Valhalla cloud file system optimized for content, and Apache Solr search indexing service

General recommendations for scaling CMS sites:

  • Test well in a environment that mirrors production including load testing and telemetry
  • Plan releases during off hours is a huge anti-pattern and is not advised, instead develop a solid release process
  • Support continuous integration (CI) by not letting your development get held back in the name of stability
  • Service oriented architecture that decouples applications as well as front and back ends and properties

Empower multiple teams that utilize common back end features with their various front end systems. A broader development team manages the core system while site specific development is managed by local developers. This continuous integration (CI) pattern of development uses a release flow that beings with an upstream core site, feeding individual development sites, then staged and released to canary production sites, then less customized production sites, followed by the heavier production sites.