eCommerce emergency preparedness

I was chatting with a lawyer friend of mine recently and he suggested the topic of emergency preparedness and management as we approach one of the year’s busiest shopping seasons, BFCM. No doubt, lawyers typically deal with things after they have become serious. I concur this is a great topic and is timely especially for at-risk online stores. Here goes…

Let’s begin with some of the things that can go wrong:

• Plugin, theme, or core software update breaks something, potentially checkout
• A link to a dev or staging site finds its way into the wild and orders trickle into the wrong database
• Traffic spike / concurrency hits some unexpected capacity limitation (memory, PHP worker shortage, disk space, timeouts, etc.)
• A service that you rely upon hits a usage limitation, such as a behind-the-scenes technical integration
• Web hosting service has an outage, usually quite temporary
• Payment provider has a service disruption, usually quite temporary
• A synchronous or render-blocking asset on the page is loading slowly, timing-out or crashing
• Marketing campaign contains wrong information or broken links
• Website gets compromised, indicated by content, malware or ads showing up
• Website gets compromised, denial of service or credit card guessing attack
• Disgruntled staff or vendor sabotages something or their account is hijacked

An ounce of prevention:

• Use quality managed hosting that takes responsibility for performance and security matters
• Automate backups and know when and how to restore from them, including granular restoration techniques
• Update your core, theme, and plugin software ahead of big events to ensure you have patched and supported versions in operation
• Know bottlenecks in your operation and work to automate them or have alternatives available
• Test your site’s PageSpeed / Lighthouse, response time (TTFB), render and load time KPIs to understand the weak points
• Test your site for Web Accessibility with the WAVE browser plugin and report errors to the appropriate parties
• Ensure all dev and staging environments (copies of the site) are offline, have a password wall, or redirect non admin traffic to production
• Check with your hosting service about anticipated traffic levels, ensure your provider and plan is adequate
• Review everybody with an administrator and shop manager account and their password strength, use a password strength requirement plugin (managed hosts usually have this built-in)
• Review fraud protection settings within your payment gateway services, such as velocity rules
• Have two or more payment providers active plus a backup or two ready to engage as needed
• Have brute-force login protection, such as JetPack or Limit Login Attempts plugin or firewall service, and lock-down login sources (IP addresses, XML-RPC logins, REST API, webhooks, etc.)
• Utilize a good firewall (WAF) such as Cloudflare Pro offering a rule engine with threat intelligence
• Increase manual testing of all critical workflows in production to ensure quality control escapes don’t happen at the wrong time

A pound of cure:

• Having a virtual “situation room” to communicate with critical parties during an event to ensure immediate and appropriate measures are engaged and panic doesn’t lead to poor decision making
• Viewing and understanding PHP error logs (sever level and the Woo Status area)
• Viewing and understanding browser console (JavaScript) errors
• Deactivating a problem plugin or disabling a problem feature within a plugin
• Swapping out payment gateways
• For shared environments contacting your host right away about an outage or slowdown
• For private servers (VPS) restarting your database service (MySQL), PHP-FPM service, or the operating system
• Activating reCaptcha service to enforce a layer of bot protection
• Enabling an auxiliary shipping or fulfillment service to handle overflows
• Granular restore from backup of specific orders, users, post types or settings
• Restoring the whole site from backup, which will lose all orders since the backup was taken
• Using the Store Notice setting to alert visitors about something important
• Mass emailing customers affected by a problem so they hopefully don’t worry too much or bombard customer support

Hopefully all this is good food for thought as you prepare for upcoming events. Shameless plug; It never hurts to bring in a technical consultant like myself to review things.