AutoScaling WordPress with Docker & AWS
Do you run your website with WordPress?
You run your website with WordPress and ask yourself: “How many concurrent visitors can it handle?” What if your site is an e-Commerce? According to Amazon.com you lose 1% of sales with every 100ms longer loading time. Today, customers expect your page to load in less than 3s – in their browser – not on your server.
Making your website fast for one visitor is relatively easy – use nginx with php7-fpm, cache static content (e.g. with varnish or memcached) and if latency is an issue because your visitors come from all over the world, use a CDN (Content-Delivery-Network) like CloudFlare or AKAMAI to bring your site as close as possible to the users.
Pssst, now’s when we add the fact that the Plesk platform includes all these technologies, enabling you to run your sites with unparalleled performance, in a few simple clicks. But if you expect lots of visitors on your website at the same time – which is what will ideally happen once your site becomes popular and successful – one server just might not be enough to handle all the requests.
You know when you’re at the supermarket and the line at the checkout is huge? If they’re in any way customer service oriented, they open a new checkout to distribute the load. What happens on a crowded Saturday when every last checkout is chock-a-block? People roll their eyes, sigh in despair and are suddenly very likely to visit a competitor next time. Not what we want, is it?
What about giving our customers a fast and reliable service which makes them leave with a smile and come back frequently because they felt oh-so-well-served?
Now we’re talking!
But how to tune WordPress to be able to handle massive parallel requests? That obviously requires several servers – like the checkout desks in the supermarket. But as seen in our crowded Saturday shopping experience, it might not be enough to simply add one or two servers. And adding 10 servers from the start could turn out tremendously expensive and ruin your business case.
What we need to achieve is that scaling your website works a lot like consuming power from the power supply!
If you have low traffic, you have low costs. When your traffic increases, infrastructure should automatically scale to handle the load. Ahhh bliss. This procedure leads to correlated costs which shouldn’t cause you any headaches as more traffic means more business. In other words, if you play your cards right and scale, increased costs for your servers shouldn’t hurt your revenues but quite the opposite. And if your traffic decreases, your server-related costs magically disappear too.
Excited to learn how all this funky stuff works?
Great! To make WordPress fast as a bullet, we need to accomplish the following steps:
- Setup an own database server with enough power on a separate machine.
- Move all static files to a file storage which is faster in delivering files.
- Setup a CDN in front of your site to bring at least static files close to your end users.
- Setup multiple servers with the exact same WordPress site (incl. configuration).
- Setup a load-balancer to distribute the load between these WP servers.
- Depending on how you want to make updates on your site, you can either.
- redeploy all instances to ship all your changes (better performance).
- or you use a shared filesystem that all instances use (slower, but easier to update).
Ramp up new instances automatically
But the king’s class is to actually ramp up new instances automatically driven by demand and ramp them down again when not needed.
How to accomplish that?
We need an infrastructure that allows, managed via APIs and with the capability of auto-scaling based on events (e.g. “high CPU consumption alarm”).
In our example, we use Amazon AWS since it is the most popular Cloud Service Provider based on amount of web-facing servers with the largest ecosystem. But Microsoft Azure and Google Compute Platform also have their strengths and can easily compete with AWS. Just pick one and you’re good to go. Again, Plesk runs smoothly on all major Cloud Service Providers and is available as an app on the AWS Marketplace.
Before going into the APIs, we should decide how we want to deploy WordPress on the servers. We do not want to deploy manually – we want to let the infrastructure auto-scale for us instead – which means the auto-scaling component decides when to add or remove servers. We could use Chef, Puppet, Ansible or simple bash scripts for this task, but our preference is to use Docker to simply package our WordPress including our website content and configuration fully separated from the infrastructure. And then just put this Docker image on each server and run it as a container. With this approach it is super simple to configure all we need once and reproduce deployments as often as we want with no effort.
Build a Docker Image
To build a Docker image you first need to describe it in a Dockerfile. Here you can see the Dockerfile that we’re using: https://github.com/plesk/wordpress-aws-scaler/blob/master/Dockerfile
We build our image with the latest WordPress version from wordpress.org by running:
$ docker build -t janloeffler/wordpress-aws-scaler:latest .
after building it, we need to push it to a Docker Registry – which is a file storage for Docker images. We use the official Docker Hub here:
$ docker push janloeffler/wordpress-aws-scaler:latest
You can easily run your image containing the WordPress locally to test it out. Be aware that you would have to specify parameters like database hostname and credentials, etc.
$ docker run -p 80:80 -p 443:443 -it janloeffler/wordpress-aws-scaler:latest
Now we need to get more provider specific since AWS, Microsoft Azure and Google Compute all have different APIs and call their services slightly different.
As said, we use AWS in our example:
Scaling – It’s all about APIs
AWS offers tons of REST APIs while each of them provides tons of API calls with again lots of parameters. Most of them are optional and can be used for flexible configuration. You can access these APIs either directly via REST http calls or by using the AWS CLI (https://docs.aws.amazon.com/cli/latest/userguide/installing.html) directly on your shell. In our example, we use the CLI for now which is a wrapper for the REST API and is easier to use for debugging.
For our super-fast auto-scaling WordPress we need the following APIs:
- EC2 (to manage virtual servers)
- S3 (to upload files to the file storage)
- S3api (to manage the file storage)
- RDS (to manage the database)
- ELB (to manage the load-balancer)
- AutoScaling (to configure auto-scaling)
- CloudWatch (to monitor load on our servers; required by auto-scaling)
- CloudFront (setup the Content-Delivery-Network)
- SNS (notification channel between monitoring and auto-scaling)
- Route53 (manage domains and DNS entries)
- IAM (manage access permissions of the infrastructure)
To give you an idea of the complexity – the EC2 API alone provides 210 API calls to manage compute resources on AWS.
To list all your EC2 instances in your AWS account you can simply run:
$ aws ec2 describe-instances
The result if all API calls is always represented as a JSON response. To automate AWS, you simply have to LOVE parsing JSON 😉
Since describing all required API calls would fill approximately 20 pages, we skip that and provide a solution to you that does the whole job of managing and auto-scaling WordPress with just 1!!! single command.
Good news – It’s OpenSource!
So check out the Plesk WordPress AWS Scaler! And YES! It is open-source:
Just download the repo to your local machine by cloning it:
$ git clone https://github.com/plesk/wordpress-aws-scaler.git
$ cd wordpress-aws-scaler
Now execute the Plesk WordPress AWS Scaler script to see its options:
$ sh manage-wordpress.sh
You can adjust the configuration to your needs, e.g.
- WordPress Site Title
- WordPress Admin Credentials
- E-Mail Address
- Domain Name
- New Relic License Key (for application performance management)
- EC2 & RDS configuration e.g. server sizes (here: instance types)
- … and much more
All of these parameters are optional and you can also create multiple config files for several WordPress sites in the same AWS accounts.
To create a new Auto-Scaling WordPress, simply execute:
$ sh manage-wordpress.sh create
to update all instances with a new version of your site:
$ sh manage-wordpress.sh update
to delete it incl. its data and all depending resources:
$ sh manage-wordpress.sh delete
And if you are interested in technical details, just open the file manage-WordPress.sh in your preferred IDE and have a look.
And to make it even easier for you to auto-scale WordPress we will launch a Plesk Extension soon. All you need to do is install this extension on your Plesk server and it’ll get the work done for you.
Ready to see and download the slides where this approach was presented at code.talks 2016 in Hamburg, Germany?
Don’t have Plesk yet? Download it here and try it out for free.
Happy scaling and stay Plesky!