CloudFormation Intrinsic functions: These 4 are a must know!
I confess! I looked up what “Intrinsic” means in the dictionary. Merrian-Webster defines intrinsic as ” belonging to the essential nature“.
I finally settled for “built-in”. Intrinsic means built in!
CloudFormation Intrinsic functions just means built in functions.
What are Intrinsic functions in AWS CloudFormation?
Intrinsic functions are built-in functions used in the CloudFormation template to pass runtime values to parameters.
However intrinsic functions are not supported in all sections of the CloudFormation template. There are 3 scenarios when/where you can use CloudFormation Intrinsic functions.
- Conditions section: An optional section in the CloudFormation template, conditions is used to create stack resources when certain criteria is met.
- Resources section: The only required section in a CloudFormation template. It contains all the AWS resources you plan on deploying in your stack.
- Outputs section: Another optional section used to pass values to other stack templates or if you want to view resources and stack related info in AWS Management Console.
List of CloudFormation Intrinsic functions
There are a total of 17 Intrinsic functions, of which 4 are conditional functions. To limit the scope of this post we will be focusing on the 4 of the important ones:
- CloudFormation Intrinsic function Ref
- CloudFormation Intrinsic function If
- CloudFormation Intrinsic function Join
- CloudFormation Intrinsic function Sub
1. CloudFormation Intrinsic function Ref
Ref, short for Reference is used extensively in the Resources and Output sections of the CloudFormation template.
You can use the Ref intrinsic function in 2 contexts –
- Parameter: If the input to the Ref function is a parameter it returns the value of the parameter.
- Resource name: If the input to the Ref function is the logical name of an AWS resource, it returns the physical name of the resource.
This can sound a bit confusing. So let me clarify.
Every resource created in your stack needs to be uniquely identifiable. Certain AWS resources can be assigned a user-friendly name, called a Logical ID in your template.
This is great, but how do you identify resources without a Logical ID?
To avoid this confusion, CloudFormation, by default, assigns every resource a Physical ID. You then use the logical name in the Ref function to determine the physical name.
The declaration format for Ref Intrinsic function is as shown below.
JSON Format: { "Ref" : "LogicalName" }
YAML Format: !Ref LogicalName
Example on using CloudFormation Ref function as a Parameter
In the below example, we create an Amazon Relational Database Service (RDS) instance using a CloudFormation template.
CloudFormation Ref function is used as a parameter to pass Username and Password values from the Parameters section of the template to the Resources section of the template.
JSON Format { "Parameters": { "DBUser": { "NoEcho": "true", "Description" : "The database admin account username", "Type": "String", "MinLength": "1", "MaxLength": "16", "AllowedPattern" : "[a-zA-Z][a-zA-Z0-9]*", "ConstraintDescription" : "must begin with a letter and contain only alphanumeric characters." }, "DBPassword": { "NoEcho": "true", "Description" : "The database admin account password", "Type": "String", "MinLength": "8", "MaxLength": "41", "AllowedPattern" : "[a-zA-Z0-9]*", "ConstraintDescription" : "must contain only alphanumeric characters." } }, "Resources" : { "myDB" : { "Type" : "AWS::RDS::DBInstance", "Properties" : { "AllocatedStorage" : "100", "DBInstanceClass" : "db.t2.small", "Engine" : "MySQL", "Iops" : "1000", "MasterUsername" : { "Ref" : "DBUser" }, "MasterUserPassword" : { "Ref" : "DBPassword" } } } } }
YAML Format
Parameters:
DBUser:
NoEcho: 'true'
Description: The database admin account username
Type: String
MinLength: '1'
MaxLength: '16'
AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'
ConstraintDescription: must begin with a letter and contain only alphanumeric characters.
DBPassword:
NoEcho: 'true'
Description: The database admin account password
Type: String
MinLength: '8'
MaxLength: '41'
AllowedPattern: '[a-zA-Z0-9]*'
ConstraintDescription: must contain only alphanumeric characters.
Resources:
myDB:
Type: 'AWS::RDS::DBInstance'
Properties:
AllocatedStorage: '100'
DBInstanceClass: db.t2.small
Engine: MySQL
Iops: '1000'
MasterUsername: !Ref DBUser
MasterUserPassword: !Ref DBPassword
Example on using Ref function used as a Resource
This second example installs and deploys WordPress on an EC2 instance.
The CloudFormation Ref function is used as a resource to obtain the Physical ID from the Logical ID for the VPC.
JSON Format { "Parameters": { "VpcId": { "Type": "AWS::EC2::VPC::Id", "Description": "VpcId of your existing Virtual Private Cloud (VPC)", "ConstraintDescription": "must be the VPC Id of an existing Virtual Private Cloud." } }, "Resources": { "ALBTargetGroup": { "Type": "AWS::ElasticLoadBalancingV2::TargetGroup", "Properties": { "HealthCheckPath": "/wordpress/wp-admin/install.php", "HealthCheckIntervalSeconds": 10, "HealthCheckTimeoutSeconds": 5, "HealthyThresholdCount": 2, "Port": 80, "Protocol": "HTTP", "UnhealthyThresholdCount": 5, "VpcId": { "Ref": "VpcId" }, "TargetGroupAttributes": [ { "Key": "stickiness.enabled", "Value": "true" }, { "Key": "stickiness.type", "Value": "lb_cookie" }, { "Key": "stickiness.lb_cookie.duration_seconds", "Value": "30" } ] } } } }
YAML Format
Parameters:
VpcId:
Type: 'AWS::EC2::VPC::Id'
Description: VpcId of your existing Virtual Private Cloud (VPC)
ConstraintDescription: must be the VPC Id of an existing Virtual Private Cloud.
Resources:
ALBTargetGroup:
Type: 'AWS::ElasticLoadBalancingV2::TargetGroup'
Properties:
HealthCheckPath: /wordpress/wp-admin/install.php
HealthCheckIntervalSeconds: 10
HealthCheckTimeoutSeconds: 5
HealthyThresholdCount: 2
Port: 80
Protocol: HTTP
UnhealthyThresholdCount: 5
VpcId: !Ref VpcId
TargetGroupAttributes:
- Key: stickiness.enabled
Value: 'true'
- Key: stickiness.type
Value: lb_cookie
- Key: stickiness.lb_cookie.duration_seconds
Value: '30'
2. CloudFormation Intrinsic function If
Use the Fn::If intrinsic function to create AWS stack resources based on conditions.
This function is the same as an if…else statement. An ideal instance for using the If intrinsic function would be to create different EC2 instance types depending on the environment (dev, test, prd).
However, unlike the other conditional functions you do not use Fn:If in the Conditional section. You use it in the Resources section and the Output section as shown in the image below.
The syntax for Fn:If is as shown below.
JSON format: "Fn::If": [condition_name, value_if_true, value_if_false]
YAML format: !If [condition_name, value_if_true, value_if_false]
Example on using Fn:If intrinsic function
In this example, we create an Amazon Redshift cluster. The Fn::If intrinsic function is used in the Resources section to determine if it is a multi-node cluster based on the number of nodes.
The below is just an excerpt from the sample template to show how the function works. For the full template, click here.
JSON Format Resources": { "RedshiftCluster": { "Type": "AWS::Redshift::Cluster", "Properties": { "ClusterType": { "Ref": "ClusterType" }, "NumberOfNodes": { "Fn::If": [ "IsMultiNodeCluster", { "Ref": "NumberOfNodes" }, { "Ref": "AWS::NoValue" } ] }, "NodeType": { "Ref": "NodeType" }, "DBName": { "Ref": "DatabaseName" }, "MasterUsername": { "Ref": "MasterUsername" }, "MasterUserPassword": { "Ref": "MasterUserPassword" }, "ClusterParameterGroupName": { "Ref": "RedshiftClusterParameterGroup" } } } }
YAML Format Resources: RedshiftCluster: Type: 'AWS::Redshift::Cluster' Properties: ClusterType: !Ref ClusterType NumberOfNodes: !If - IsMultiNodeCluster - !Ref NumberOfNodes - !Ref 'AWS::NoValue' NodeType: !Ref NodeType DBName: !Ref DatabaseName MasterUsername: !Ref MasterUsername MasterUserPassword: !Ref MasterUserPassword ClusterParameterGroupName: !Ref RedshiftClusterParameterGroup
3. CloudFormation Intrinsic function Join
The CloudFormation Fn::Join intrinsic function creates a single output value in string format by combining multiple input values.
If you want the values in your output string separated by another value (delimiter) you can specify it. Otherwise, your values will be combined without a delimiter in between. Yes, not even white space!
The syntax for the Fn::Join intrinsic function is as shown below.
JSON format: {"Fn::Join" : [ "delimiter", [ comma-delimited list of values ] ] }
YAML format: !Join [ delimiter, [ comma-delimited list of values ] ]
Example on using Fn::Join
For the Fn::Join function example, we look at a sample template on creating an Amazon RDS instance.
The Fn::Join function is used in the Outputs section of the CloudFormation template to generate the JDBC Connection string. For the complete template, click here.
JSON Format "Outputs" : { "EC2Platform" : { "Description" : "Platform in which this stack is deployed", "Value" : { "Fn::If" : [ "Is-EC2-VPC", "EC2-VPC", "EC2-Classic" ]} }, "MasterJDBCConnectionString": { "Description" : "JDBC connection string for the master database", "Value" : { "Fn::Join": [ "", [ "jdbc:mysql://", { "Fn::GetAtt": [ "MasterDB", "Endpoint.Address" ] }, ":", { "Fn::GetAtt": [ "MasterDB", "Endpoint.Port" ] }, "/", { "Ref": "DBName" }]]} } }
YAML Format Outputs: EC2Platform: Description: Platform in which this stack is deployed Value: !If - Is-EC2-VPC - EC2-VPC - EC2-Classic MasterJDBCConnectionString: Description: JDBC connection string for the master database Value: !Join - '' - - 'jdbc:mysql://' - !GetAtt - MasterDB - Endpoint.Address - ':' - !GetAtt - MasterDB - Endpoint.Port - / - !Ref DBName
4. CloudFormation Intrinsic function Sub
The CloudFormation intrinsic function Sub is used to replace an input string with value(s). The syntax for Fn::Sub is as shown below.
JSON format
{"Fn::Sub" : [ String, { Var1Name: Var1Value, Var2Name: Var2Value } ] }
YAML format !Sub - String - Var1Name: Var1Value Var2Name: Var2Value
Pay close attention to the following points.
- You can define as many variable names as needed (Var1Name, Var2Name…….Var(n)Name).
- The variable value (VarValue), is specified after the colon.
- Every variable name and value combination should be specified as a key-value map.
- The input string denoted as “string” normally contains the variable (VarName) you need to replace.
- You have to specify the variable name in your string as “${VarName}” for both JSON and YAML formatted templates.
- For template parameter names, resource logical IDs and resource attributes do not use variables. They are already defined for you.
Example1: Intrinsic function Fn::Sub with variable map
The Sub intrinsic function can be tricky to understand. So, let’s look at a simple example first.
JSON Format { "Fn::Sub": [ "${Fruit1} and ${Fruit2}", { "Fruit1": "Apples", "Fruit2": "Oranges" } ] }
YAML Format !Sub - '${Fruit1} and ${Fruit2}' - Fruit1: Apples Fruit2: Oranges
In this example, we have a variable map with 2 variables – Fruit1 and Fruit2. The respective variable values are Apples and Oranges.
The input string in blue font generates the string value ” Apples and Oranges“.
This example was purely for illustration purposes only. For the most part you will use the Sub intrinsic function to substitute template parameter names, resource attributes and resource ID’s.
Example 2: Intrinsic function Sub without variable mapping
In this final example on Fn::Sub we create a file named setup.sql which contains the DDL to create a MySQL database and a user with permissions to the database.
DBName, DBUsername and DBPassword are template parameters defined in the Parameters section of the template.
JSON Format { "files": { "/tmp/setup.mysql": { "content": { "Fn::Sub": "CREATE DATABASE ${DBName}; \nCREATE USER '${DBUsername}'@'localhost' IDENTIFIED BY '${DBPassword}'; \nGRANT ALL ON ${DBName}.* TO '${DBUsername}'@'localhost'; \nFLUSH PRIVILEGES;\n" }, "mode": "000644", "owner": "root", "group": "root" } } }
YAML Format files: /tmp/setup.mysql: content: !Sub | CREATE DATABASE ${DBName}; CREATE USER '${DBUsername}'@'localhost' IDENTIFIED BY '${DBPassword}'; GRANT ALL ON ${DBName}.* TO '${DBUsername}'@'localhost'; FLUSH PRIVILEGES; mode: '000644' owner: root group: root
Related post: CloudFormation Parameters:Make your life simple
Summing things up
Sometime the big picture is overwhelming!
So, like I always say, start small! Focus on mastering these 4 CloudFormation Intrinsic functions. As you get better, your confidence will improve. You can then move on to mastering them all.
A few helpful hints and pointers:
- Try playing around in AWS CloudFormation Designer. It is a developer friendly graphics-based tool to create CloudFormation templates. You can drag and drop AWS resources and modify their properties to create stacks. The best part is you can view the JSON/YAML code being generated in real time.
- Understand Pseudo parameters before diving into Intrinsic functions. Pseudo parameters are predefined by AWS and are heavily used in combination with CloudFormation Intrinsic functions.
