Creating basic AWS infrastructure is no big deal as long as it’s limited to replicating or restoring several EC2 instances and RDS databases.
The problem begins when your infrastructure expands to contain dozens of instances, databases, load balancers, subnets and other resources. As your environment gets bigger and more sophisticated, its maintenance and management are becoming more challenging.
What can you do to recreate your production infrastructure in a test environment quickly? Setting up everything from square one is time-consuming and may lead to unwanted differences between the environments while you need them to be the exact copies of each other.
You need a tool to model, set up, and automate the creation of all your infrastructure resources in AWS.
Meet AWS CloudFormation.
What Is AWS CloudFormation?
AWS CloudFormation is a tool that allows you to launch resources from scratch without any effort, using templates.
A template is a YAML or JSON document that describes all the resources and their properties. CloudFormation takes care of provisioning and configuring these resources for you.
With CloudFormation, you can reuse each template to set up your resources consistently and repeatedly. Just describe a resource once and then provision the same resource time and again in multiple regions.
How Does AWS CloudFormation Work?
The general principle behind AWS CloudFormation is pretty straightforward.
The core element is a template written in either YAML or JSON format. Every single resource you want to launch within a stack has to be described in the template file.
When you create a stack, AWS CloudFormation uses the underlying service calls to AWS to configure and provision your resources. When you delete a stack, all the contained resources are automatically terminated.
- CloudFormation may only perform actions that you are allowed to execute. If you want to launch or terminate an EC2 instance, the user that launches a template needs to be granted permissions by AWS Identity and Access Management (IAM) to do so.
- Each template has to be uploaded to the S3 bucket before Even if you specify a template file stored locally, it will be first uploaded to an automatically created S3 bucket. If there is already an existing bucket in place, it will be used instead.
- Once the template file is specified and uploaded, CloudFormation starts making a call to AWS services described in the template. When all resources are on, CloudFormation reports that the stack has been built and can be used. In case of any error in the template, it will be reported, and the process will be terminated so that you may fix the issue.
Working with CloudFormation: Updating Existing Stacks
Sometimes you will have to update a running stack. You can do it by modifying its template, without deleting the stack instance and creating a new one.
However, it’s vital that you understand how your changes will affect the running resources. Some of them can be updated, but some will be reset.
You can preview how the proposed stack changes may impact the running instances in the preview section:
In the above example, the Amazon Machine Image and instance type are to be changed, which requires the instance to rebuild.
Whenever you see ‘True’ in the Replacement column, this means that this resource will be replaced with a new one, so be extra cautious.
For example, if you wish to change some parameters of the RDS DB instance, CloudFormation may need to create a new database and delete the old one. Just imagine what would happen if you forgot to backup data from that database!
If you wish to maintain a resource after an update, keep it untouched in a new template. Modify only those resources that you need to be updated. Use the same values as the current stack configuration for resources and properties you are not updating.
What Are the Basic AWS CloudFormation Concepts?
You already know templates and stacks. These are the basic concepts of AWS CloudFormation.
AWS CloudFormation Templates
Whenever you create a stack, the resources within it are provisioned as described in the template. As I have mentioned above, an AWS CloudFormation template is a YAML or JSON formatted text file, which can be saved with any extension.
Let’s see the following, simple template:
AWSTemplateFormatVersion: 2010-09-09 Description: Template for EC2 instance Resources: MyEC2Instance: Type: AWS::EC2::Instance Properties: ImageId: ami-3bfab942 #This is an Amazon Linux AMI InstanceType: t2.micro KeyName: defaultAWSkey
The above template provisions an EC2 instance with ami-3bfab942 AMI, t2.micro instance type and defaultAWSkey key pair name. Properties not mentioned here will be set to default.
You don’t have to define properties in the template. Our template can get a little bit more advanced, and the properties can be referenced as input when the stack is created.
AWSTemplateFormatVersion: 2010-09-09 Description: Template for EC2 instance Parameters: Subnet: Description: The EC2 Instance will be launched in this subnet Type: AWS::EC2::Subnet::Id SecurityGroups: Description: Assign Security Groups for the instance Type: List<AWS::EC2::SecurityGroup::Id> AMIid: Description: Image ID Type: AWS::EC2::Image::Id Keypair: Description: Key pair name Type: AWS::EC2::KeyPair::KeyName InstanceType: Type: String Default: t2.micro AllowedValues: - t2.micro - t2.small - t2.medium Description: Enter t2.micro, t2.small or t2.medium. Default is t2.micro Resources: MyEC2Instance: Type: AWS::EC2::Instance Properties: ImageId: !Ref AMIid InstanceType: !Ref InstanceType KeyName: !Ref Keypair SecurityGroupIds: !Ref SecurityGroups SubnetId: !Ref Subnet
In this example, we define not only resources but also parameters that are to be passed to the template at runtime.
Then, in the Properties section, the ‘!Ref’ keyword signifies the reference to the specific parameter. For instance, !Ref InstanceType will take the InstanceType parameter and propagate it as the InstanceType value. InstanceType is a string with three allowed Values and t2.micro as the default value.
Launching this template will create a stack consisting of a single EC2 instance with properties defined at runtime. The possibility of specifying some values when the stack is created makes templates easier to reuse.
AWS CloudFormation Stacks
A stack is another fundamental concept in AWS CloudFormation. It is a single unit being a collection of resources. Resources are created, updated, and deleted by creating, updating and deleting stacks, and all the resources within a stack have to be defined by a template.
To provision these resources, you submit a template and create a stack. You can do it by using AWS CloudFormation console, API, or AWS CLI.
AWS CloudFormation Template Outline
Let’s briefly go through the CloudFormation template and take a look at its most important sections. You can find detailed information about each in the official CloudFormation documentation.
Format Version (optional)
Format Version specifies the template version. The current one is 2010-09-09.
This field describes your template.
Parameters specify values that you can pass into the template at runtime. You can later refer to the parameters in the Resources and Outputs sections.
The mapping of keys and associated values that can be used to specify conditional parameter values. For instance, with mappings, you can define AMIs available for a specific region.
Defines conditions that control whether particular resources are created or whether certain properties are assigned a value during the stack creation or update.
Resources specify the stack resources and their properties. You can refer to the resources in the Resources and Outputs sections. For example, you can define a SecurityGroup here and then refer to it in the EC2 instance definition.
Describes the values that are returned whenever you view your stack properties.
How to Create a Stack from AWS CloudFormation Templates?
It is time to show you how to create a stack from one of the templates shown above. The one that is more advanced.
- First, launch CloudFormation and click Create new stack.
- You can then design a template with the AWS CloudFormation Designer (which is NOT the optimal idea), select a sample template, upload your own template or specify an S3 template URL.
I am going to upload a template.
- In the next screen, you need to provide some details about your stack. Here you specify not only a name for your stack but also all parameters defined in the Parameters section of the template.
- Then you can set some options like Tags, Permissions, Alarms, Termination protection, etc. I will leave them at defaults for now.
- In the last screen, you can review your stack and eventually finalize the process.
- You will see the status and, in the panel below, all events related to the creation.
- When the creation is complete, the status will change to ‘CREATE_COMPLETE’ and you can start using your resources. To update the stack, choose Update Stack from the Actions menu and provide a new template for the existing stack.
- To delete a stack, choose Delete Stack, and all the resources related to it will be removed
More Use Cases!
AWS CloudFormation is such a powerful tool for automated resource creation and management that it’s impossible to list all its features and benefits in a single post.
If it sparked your interest (and I bet it did if you have anything to do with AWS), consider it as your homework to conduct further research.
What features of CloudFormation are worth investigating? Maybe nested templates? Or guidelines for preparation of an OS running on stack instances with the help of bootstrap helpers? What about integrating CloudFormation with Puppet or Chef to automate instance configuration?
There are infinite topics to examine. With AWS CloudFormation, you can prepare your environment according to your needs, and then hand it over to your customer or co-workers to be deployed and eventually deleted with just a few clicks. Amazing, isn’t it?