Recommended AWS Cloud Services for Web Applications

#1

This article was written as a reply to @veekthur on the thread: Which Hosting company would you recommend for a music sharing/streaming website. The reply became too long so I decided to create a new topic for it so that others can easily discover it.

AWS is a behemoth of cloud services enabling millions of services around the world. These are my “personal” recommendations from my experience so far with AWS. First of all, let me say that I’m an AWS junkie. AWS offers more than I can even consume. I simply don’t have any reason to look elsewhere anymore.

HOSTING

I recommend AWS EC2. With EC2, you can spin off your own “free-for-a-year” web host within 2 minutes. EC2 comes with lots of preconfigured proprietary and community images which you may use to kickstart real quick. Or you may follow the long and tedious process of installing your own services on a fresh EC2 host - NOT recommended. Initially, I played around with Bitnami images. They are free to use, basic and compatible with AWS Free. Before, selecting an image, ensure that it contains your choice of web server, database server, backend cache server, etc.

For the server specifications, start with the free-tier server configuration (t1.micro instance). You may use the t1.micro instance/host for up 750 hours or one year free of charge while you are playing around with different images and doing your backend and front designs. When you launch your service, you can change the server configuration seamlessly to any compute-optimised or memory-optimised configuration to suit your hosting demands.

If you are Linux savvy or willing to learn Linux, I recommend that you use a Linux host, preferably RedHat or Ubuntu. Both have lots of support from their owners and the Linux community. DigitalOcean has lots of articles on Ubuntu which you may following while installing services on your EC2 host.
More reading:

  1. AWS Free Tier

WEB SERVER

I recommend that you run your sharing service on an NGINX web server. NGINX is generally more faster in serving your contents than APACHE. Most of the world’s top websites run on NGINX.
More reading:

  1. NGINX vs. Apache: Our View of a Decade-Old Question
  2. Nginx vs. Apache: Choosing a Linux web server

DATABASE SERVER

MYSQL is the de-facto database engine. You may host your MYSQL on the localhost (EC2 host) or remotely. If you are working with AWS, I will recommend that you use a remotely-hosted MYSQL server. That means that the MYSQL server should not be hosted on the EC2 host. This gives you the freedom to commission and decommission your EC2 hosts without worrying about your database. AWS’ solution for remote MYSQL server is the Amazon RDS. As with EC2, you can spin off an RDS instance within minutes and use it free of charge of up to 750 hours or one year.

To connect your RDS-hosted database server to your EC2-hosted web application, what you need is the database server endpoint (host URL) which can be found on the RDS backend.

##HTTP ACCELERATOR/ CACHE REVERSE PROXY
I recommend that you use Varnish as an http accelerator in order to assist the web server (NGINX) in serving your dynamic contents super fast. According to varnish-cache.org:

Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy. You install it in front of any server that speaks HTTP and configure it to cache the contents. Varnish Cache is really, really fast. It typically speeds up delivery with a factor of 300 - 1000x, depending on your architecture.

Varnish is easy to install via the repositories of Linux distros but you will need to configure your NGINX and Varnish to allow the later to serve upstream of the former. If you need an already installed and configured Varnish cache, spin off an EC2 instance from an Amazon Machine Image (AMI) pre-configured with Varnish. Bitnami images with Varnish might need you to still configure Varnish after installation. Their documentation on Varnish is straightforward though.
More reading:

  1. Varnish Software (Wikipedia)

DATABASE CACHING

A typical database server may receive the same query and return the same set of results over and over again. Database cache servers allows the storage of these similar sets of queries and results and directly serves them to the web server with little or no interaction with the database server. Memcached and Redis are the two most popular solutions for database caching. You may choose to host your database cache on your EC2 host or remotely via AWS Elasticache. On Elasticache, you can choose between Memcached and Redis and spin off a cache server using the t1.micro free-tier instance configuration.
More reading:

  1. Database Caching
  2. Databae Caching (Wikipedia)
  3. Introduction to Redis
  4. How to take advantage of Redis just adding it to your stack
  5. 11 Common Web Use Cases Solved In Redis
  6. What is Redis and what do I use it for? [closed]
  7. About Memcached
  8. Memcached: What is it and what does it do? (PHP Version)

OPCODE CACHING

According to wiki.typo3.org:

An opcode cache is a system to cache the result of the PHP code compilation to bytecode. This allows to use the bytecode instead of compiling on every request. Different opcode cache systems are available, mostly depending on used PHP version.

Common solutions for Opcode Caching include:

  1. APC
  2. XCache
  3. Eaccelerator (PHP <5.4)
  4. Zend OPCache (PHP >5.4).

These are usually installed on the localhost. Some AMIs come with preconfigured Opcode Cache Systems (commonly APC & Zend). Confirm before spinning off an EC2 instance with any particular AMI.
More reading:

  1. Magento Cache - confusion about Varnish, Redis, APC, Memcache

EDGE CACHING OR CONTENT CACHING

A typical web application might have static contents such as image, CSS, & JS, audio and video files. When requested, these files could be served directly from your web host or you could outsource that task to a Content Distribution Network (CDN) which will ensure that your static files are hosted on dedicated content servers around the world and that clients are served from the closest content server. These increases your site-load speed and decreases bandwidth usage on your web host.

Common CDNs include:

  1. Amazon CloudFront,
  2. Akamai
  3. MaxCDN
  4. CloudFlare,
  5. Limelight, etc.

I recommend the use of CloudFront especially if you are using AWS S3. CloudFront is still compatible with any other third-party host.

STORAGE

AWS allows up to 30GB of storage for a free-tier (t1.micro) instance. Configuring more storage space for your instance is expensive. Instead of adding more storage space to your web host, it is recommended that you setup AWS S3 buckets and use the buckets to serve your static contents. AWS S3 is also covered the the AWS Free policy.

If you need to synchronise your S3 buckets with folders on your EC2 web host, consider using s3fs-fuse. You may also configure CloudFront to serve your S3 buckets.

DNS

Amazon Route53 can offer highly-available DNS service for your web host, application servers and load balancers. If you have an EC2 web host, then it’s logically that you make Route53 your DNS provider. Route53 can also be used for purchasing domain names. AWS makes configuring Route53 very simple and intuitive.

SCALABILITY & AVAILABILITY CONSIDERATIONS

The biggest advantage in using AWS services is their high availability and scalability. Most AWS services are hosted on clusters of servers called Regions and mini-clusters within Regions known as Availability Zones. You may be required to choose a region and an availability zone closest to where most of your user might be. In the case of a global company, you can host different instances of your web hosts on different regions or availability zones to provide safety nets in the case of unavailability of a particular region or availability zone (Amazon says this hardly happens though).

Scalability of web applications on EC2 is achieved using Auto Scaling Groups and Elastic Load Balancers. Auto Scaling allows your web server, database server, or cache server to scale up and down depending on the load from users (traffic). Load Balancers allow traffic to be spread across multiple running instances of your web server and application server (which could be in the same region or across multiple regions).

Below is a 3-tier architecture featuring a typical usage of AWS services

If there is anything I didn’t put well, kindly point out for the sake of readers. You may shed more light on other services which I might have missed. Thank you.

A WishList For Santa TC-Radar To Give Us This Christmas For Being Good Members/Users
CRUDCAST Episode 8
Which Hosting company would you recommend for a music sharing/streaming website
#2

Wow, bro I appreciate this post. It’s a goldmine for someone like me. Thanks alot

#3

Nice Post.

A quick question, how can I sleep peacefully at night after I stitched all these services together? Especially when all I wanted to just do was to stream music? When my infrastructure have issues and I have to troubleshoot why my memory is balloning under low load, should I just ignore preparing my vision statement presentation slides and NewRelic the hell out of my cheaply put together infrastructure?

Oh well… I hope in the future someone can invent some sort of “managed service” that there is an agreeded SLA so that I can sleep peacefully at night. Infact it will be nice if the managed service actually runs on AWS.

Eureka! I am going start 4 companies now - Heroku, AppFog, Cloudinary and Memcachier. Because I am so generous, I will also advise AWS to start a service called Elastic Beanstalk.

Thanks for opening my eyes to these wonderful opportunities. Let me know your favourite charity so that I can donate some of the money I make in my new companies to them.

Thanks a lot once again.

#4

Lol @Tola

@Ndianabasi Great post. While reading, I was going to also suggest Elastic Beanstalk. For a newbie, it’s more straightforward, as it’s as easy as pushing your code and deploying to multiple configurable environments. The multiple environments mean you can have your upgrades tested, without affecting your live deployment until you are sure you want to.

#5

@Ndianabasi Thank you for this post. Do you have a blog or website that I can learn from? It’s been a while since someone posted something reasonable n knowledgeable on TC-Radar. Upon how great your post is, I only see a few likes but alot of comments. Smh. The other person who liked your post, diden’t even make a comment. WOW!!! Typical underrated post/thread/topic as always. I personally appreciate the patience, time and work you put in doing this. Happy Sunday my bro.

#6

Aww. Oh David!

#7

Dude just like and waka pass…

#8

@Ndianabasi very good post, an eye opener - good job
But em…kinda left a guy that just got a shared-linux-hosting from one of the popular hosting coy with basic features confused. All the services are great, but Where do u think he should start from?

#9

Hi @Tola. I understand where you are coming from. Your fears are quite understandable.

I recommended EC2 to @veekthur because he wants to run a music sharing service. This kind of service would need an easy-to-scale web architecture. Starting with a managed hosting account could be fast and easy now, but it will make you too comfortable. Then when your traffic surges, you will begin to encounter bottlenecks. In my opinion, it’s better to pass through the process of learning how to set up your own host and understand how to manage your services. It will pay off in the future.

AWS is cheap, but could also be expensive depending on how many services you are running, the configurations of your services, and your traffic demands. However, the beauty of AWS is that you are not charged a fixed rate unlike shared servers. AWS allows you to be charged based on actual usage. You could even shutdown or pause some services and won’t be charged during those period of idleness.

All services on AWS are managed services. So you can sleep and don’t worry about failure of your RDS cluster or DNS settings or Elasticache clusters or S3 buckets, etc. Even EC2 has most of its auxiliary services managed, e.g. Elastic Load Balancers, Network Interfaces, Elastic IPs, Security Groups, etc. However, for EC2, you are responsible for configuring your security groups (or firewall) and maintaining services running on your host.

EC2 can be used for any kind of Web Application whether a blog, mobile app, ecommerce store, etc, or Mail server. It offers you the freedom to choose what you want to run on your host. You have the freedom to choice the configuration of CPU, RAM, and OS needed for your services. However, this freedom comes with lots of responsibilities. So, if you are not ready to manage your own server, then a shared hosting account with any hosting provider could be sufficient for you.

#10

My very unpopular advice - focus on your day job and let the DevOps do thier job. This is because if you leave your database unpatched after a zero day attack, the Chinese and Russians will have you for dinner.

#11

Thank you very much for the kind words @davidsmith8900 and happy Sunday to you too. Back then, I used to maintain a blog on Autodesk Inventor, a Mechanical Engineering design application. If you are interested in that subject, check out http://inventortutorials.blogspot.com :smiley::smiley:. I’m an instructor by calling.

#12

Since this topic is about AWS, let me point out again that AWS services are managed services. For your database, RDS is automatically updated when there is a patch. You can schedule your own backup which are saved on EBS - a very persistent storage for AWS.

#13

@folorunsho1. If you need a Web server, then spin off an EC2 instance with a Bitnami image. SSH into your host using Putty, copy files to the host using WINSCP, Set up an Elastic IP address through the EC2 backend, configure your nginx or apache conf and virtual host files, and test the application using your Elastic IP address.

#14

Yep you are right about RDS but you also suggested that MySQL could be spun locally in EC2, which is way cheaper than RDS. That’s where the danger is to an untrained eye. This problem is not limited to just Amazon but to all IaaS providers including the well touted Digital Ocean - where people naively dump WordPress, Discourse, Magento and all other DIY open source software deployments.

#15

@Ndianabasi Thank you for the link, I appreciate it.

#16

I’m not sure when @Tola said managed hosting, he was referring to “shared” hosting. He did list the services he was referring to and they all have autoscaling capabilities built in. On the other hand, it’s very easy to mess up your EC2 instance with lots of unrelated junk in the same place and then make preparing for scale another assignment, and that’s where Elastic Beanstalk (another AWS product), AppFog, or Heroku come in.

#17

Good morning @xolubi.

I played around with Elastic Beanstalk when I started using AWS, but quickly abandoned it because I couldn’t achieve the things I needed to achieve fast through Beanstalk. Some many months after abandoning Beanstalk I went back to it again because of your last post. What I discovered yesterday enforced why I left it initially.

Elastic Beanstalk is a managed EC2 service i.e. Elastic Beanstalk environments are actually EC2 instances and will appear on your EC2 backend. Elastic Beanstalk allows you to quickly launch web-server environment tier or worker environment tier. The web server tier can be used to run web applications that handles HTTP(S) requests such as blogs, ecommerce stores, social media site, etc while the worker tier is used for running web application that handles background-processing jobs such as processing, queuing and sending out of email notifications on social network sites or sign-up pages.

Beanstalk offers the most basic server setup just sufficient to run your PHP, Python, Ruby or Java applications. On Beanstalk you cannot choice your own web server (Apache or NGINX). There are no options for installing addtional services such as web accelerators, database cache, opcode cache, or locally-hosted database server (Beanstalk only offers database service through RDS or DynamoDB). You cannot launch a Beanstalk environment with a pre-configured AMI containing your choice of database server, mail server, database cache, opcode cache, etc. All these can be achieved if you personally launch your own instance through EC2.

I remotely logged into an EC2 instance managed by Elastic Beanstalk through Putty and was warned that any changed made to the instance will be lost if the instance belonged to an Auto Scaling Group. This means that you are only allowed to manage Beanstalk instances through the Beanstalk backend and the only change you are permitted to carry out is uploading new versions of your application to your environment which will simply refresh to reflect the update. In this regard, Elastic Beanstalk behaves like a “shared hosting” where you are only allowed to upload your web application files to the server through the Cpanel.

For a developer who wants to run a complex web application such as ecommerce store, music-streaming service, etc., Elastic Beanstalk isn’t recommended. All the other advantages offered by Elastic Beanstalk such automatic load balancing and Auto Scale Group configuration can be done personally on EC2 with little research.

#18

The Twelve-Factor App :blush:

#19

Thanks. The link offers very interesting perspective. I wish I was a programmer but I’m just a non-techie looking for a powerful and highly-scalable means of hosting an already-developed and well-maintained web app - Magento. :stuck_out_tongue_winking_eye:

CRUDCAST Episode 8
#20

Constructing goal posts, are we? When did it become a “Magento” post rather than a general knowledge (with maybe a bias for a custom built music streaming service) one?