Appearance
HCL is a configuration language designed to be both human- and machine-readable. HCL is an automation language that is used to manage, create, and release Infrastructure as Code. Based on a study conducted on the GitHub repository, HCL was the third highest in terms of language growth popularity in 2019, which indicates how important the HCL platform has become, which in turn was probably aided by HashiCorp applications like Terraform, Consul, and Vault. HCL is designed to ease the maintenance of cloud and multi-cloud hybrid solutions. The language is structural, with the goal of being easily understood by humans and machines. HCL is fully JSON-compatible and the language is intended to be used to build DevOps tools and servers, manage PKI passwords, and release Docker images. HCL gets its inspiration from libucl, the Nginx configuration, and other configuration languages.
libucl, the Universal Configuration Library Parser, is the main inspiration for the HCL language. As you will see, HCL uses a similar structure, and UCL is heavy inspired by Nginx configuration.
HCL comprises a family of DSLs. In this book, we will focus on HCL2, which emphasizes simplicity. HCL has a similar structure to JSON, which allows for a high probability of equivalence between JSON and HCL. HCL was designed to be JSON-compatible, and every HashiCorp product has a specific call for the relevant API and/or configuration. The entire product suite encompasses this basic syntax. Similar to other languages there are some primitive data types and structures:
These are the basic structures and data types that can be used to write HCL code. To create a variable, we can use this syntax: key = value (the space doesn’t matter). The key is the name of the value, and the value can be any of the primitive types such as string, number, or boolean:
description = "This is a String"
number = 1The string is defined using the double-quoted syntax and is based (only) on the UTF-8 character set: hello = "Hello World" It is not possible to create a multi-line string. To create multi-line strings, we need to use another syntax. To create a multi-line string, HCL uses a here document syntax, which is a multi-line string starting with << followed by the delimiter identifier (normally a three-letter word like EOF) and succeeded by the same delimiter identifier.
A here document is a file literal or input stream used in particular in the Unix system for adding a multi-line string in a piece of code. Typically this type of syntax starts with << EOF and ends with an EOF.
To create a multi-line string in this way, we can use any text as the delimiter identifier. In this case, it is EOF:
<< EOF
Hello
HCL
EOFIn HCL, all number data types have a default base of 10, which can be either of the following:
For example,
first=10
second=10.56The variable first is an integer number while the variable second is a float number. A number can be expressed in hexadecimal by adding the prefix 0x, octal using the prefix number 0, or scientific format using 1e10. For example, we can define the number data types as follows:
hexadecimal=0x1E
octal=07
scientific=2e15HCL supports tuple creation using the square brace syntax, for example:
array_test=["first",2,"third"]The value written in an array can be of different types. In the previous example, you can see two string data types and one number data type. In HCL, it is possible to create an array with any type of object:
test_array=[true,
<< ERRDOC
Hello
Array
ERRDOC,
"Test"]In HCL, objects and nested objects can be created using this syntax:
<type> <variable/object name> {...}:
provider "aws" {
description = "AWS server"
}
// We can use the same structure for the object to define an input variable:
variable "provider" {
name = "AWS"
}A Boolean variable in HCL follows the same rules of the other languages. The only value it can have is either true or false:
variable "active"{
value = True
}HCL is used for the majority of the use-case scenarios with the Terraform tool. This symbiosis has become a significant factor in the growth of the popularity of HCL. The HCL that is used to create a template can be translated into JSON by the parser, an important step for creating a valid and usable template for HIL. HIL, or HashiCorp Interpolating Language, is the language used for interpolating any string. It is primarily used in combination with HCL to use a variable defined in other parts of the configuration. HIL uses the syntax ${..} for interpolating the variable, as shown:
test = "Hello ${var.world}" The HIL is used to have something similar to a template. This language is mostly used in Terraform. The goal is to have a rich language definition of the infrastructure. The idea behind the creation of HIL was to extract the definition language used in Terraform and then clean it up to create a better and more powerful language. HIL has its own syntax, which it shares with HCL, such as comments, multi-line comments, Boolean, etc. With HIL, it is possible to create a function for the call, which can be used in the interpolation syntax of the function. This is, in turn, is called with the syntax func(arg1, arg2, ....). For example, we can create a function with the HIL in this way:
test = "${ func("Hello", ${hello.var})}"
HIL is utilized in more depth when we use Terraform and other software like Nomad.
You just got a concise introduction to HCL and HIL. But in order to progress beyond this point, you need to create a basic template to illustrate how both components work. HCL and HIL use the GPL language to create JSON code for the necessary configurations. JSON itself is quite capable of producing the necessary code or configurations so why are HCL/HIL needed? JSON lacks the ability in insert comments, which is essential for reviewing code or configurations, particularly for the massive infrastructure that HCL/HIL is aimed at.
HCL consists of three distinct, integrated sublanguages. All three work together to permit us to create the configuration file:
The structural language is used to define the overall structure, the hierarchical configuration structure, and its serialization. The three main parts for an HCL snippet are defined as bodies, the block, and attributes. The expression language is used to express the value of the attribute, which can be expressed in either of two ways: a literal or a derivation of another value. The template language is used to compose the value into a string. The template language uses one of the several types of expression defined in the expression language. When code is composed in HCL, all three sublanguages are normally used. The structural language is used at the top level to define the basis of the configuration file. The expression language and the template language can also be used in isolation or to implement a feature like a REPL, a debugger that can be integrated into more limited HCL syntaxes such as the JSON profile itself.
A fundamental part of every language is the syntax. Now we’ll introduce the basic grammar of HCL and the fundamental parts used to build the language. You’ve seen which data and type structures are allowed in the HCL language. Now we will delve deeper into syntax. The basic syntax in HCL has these basic rules:
Every name starting with an uppercase letter is a global production. This means it is common to all syntax specified in the document used to define the program. This is similar to a global variable in other languages.
Every name starting with a lowercase letter is a local production. This means it is valid only in the part of the document where it is defined. This is similar to a local variable in other languages.
Double quotes (“) or single quotes (‘) are used to mark a literal character sequence. This can be a punctuation marker or a keyword.
The default operator for combining items is the concatenation, the operator +.
The symbol | is a logical OR, which means one of the two operators, left or right, must be present.
The symbol * indicates a “zero or more” repetition of the item on the left. This means we can have a variable number of elements, with the minimum value of 0.
The symbol ? indicates one or more repetitions of the item to the left.
The parentheses, ( ) , are used to group items in order to apply the previous operator to them collectively.
These are the basic syntax notations used to define the grammar of the language. They are used in combination with the structure and data types for creating the configuration file(s). When a HCL configuration file is created, a certain set of rules are used to describe the syntax and grammar involved. There are three distinct sections of the file:
This structure defines the model of the configuration file. A configuration file is nothing more than a sequence of characters used to create the body. If we want to define a similar BNF grammar, we can define the basic structure for a configuration file as follows:
ConfigFile = Body;
Body = (Attribute | Block | OneLineBlock)*;
Attribute = Identifier "=" Expression Newline;
Block = Identifier (StringLit|Identifier)* "{" Newline Body "}"
Newline;
OneLineBlock = Identifier (StringLit|Identifier)* "{" (Identifier "="
Expression)? "}" Newline;A BNF (Backus-Naur Form) grammar is a notation technique used for free-form grammar. With this technique, we can define a new type of grammar for our own language. This is normally used when we create a new language, like HCL. The BNF is largely used when defining the language and is very helpful when we need to understand the language itself. There is a new version of the BNF called EBFN (Extend-Backus-Naur Form). The BNF is a simple language used in particular in the academic world. There is no unique definition and it is mostly used to describe metacode to be read to a human and is normally written on one line. The EBFN lets us write a more complex model representation of the code; it is possible to define a variable and function with a more complex syntax.
Identifiers are used to assign a name to a block, an attribute, or a variable. An identifier is a string of characters, beginning with a letter or a certain unambiguous punctuation token, followed by any number or letter of Unicode. The standard used to define an identifier is the Unicode standard, defined in the document UAX #31- Section 2. This document also defines the BNF grammar we can use to write our identifiers. The grammar is as follows:
<Identifier> := <Start> <Continue>* (<Medial> <Continue>+)*
To define an identifier, this notation is used:
Identifier = ID_Start (ID_Continue | '-')*;
where
ID_Start consists of sequence of Unicode letters and certain unambiguous punctuation.
ID_Continue defines a set of Unicode letters, combining marks and such, as defined in the Unicode standard.
In addition to the first two characters, ID_Start and ID_Continue, we use the character '–'; this character can also be used to define identifiers. The usage of the '–' character allows the identifier to have this character as part of the name.
There is no specific list of reserved words. This is because keywords change depending on the software used to configure.
In HCL, we have these operators:
+ && == < : { [ ( ${
- || != > ? } ] ) %{
* ! <= = . / >= => ,
% ...All of these operators are used for the logical, mathematical, and structural definitions of the language. As you write configuration files, this will become more apparent.
The numeric literal is used to define the structure of a number. The similar BNF notation for a numeric literal is as follows:
NumericLit = decimal+ ("." decimal+)? (expmark decimal+)?;
decimal = '0' .. '9';
expmark = ('e' | 'E') ("+" | "-")?;Using this syntax, we can define the number like an integer with a non-fractional part plus a fractional part, such as a float number and an exponent part. With the syntax defined previously, we can then write numbers like 0, 0.3, 1e-10, and -10
The expression sublanguage is used to create configuration files, specifying values within the attributes definition. The similar BNF specification for the expression is
Expression = (
ExprTerm |
Operation |
Conditional
);An expression is normally used to return a type; an expression can return any valid type. The ExprTerm is the operator used for the unary and binary expression. ExprTerm can act like a normal expression itself. The similar BNF syntax for the ExprTerm is
ExprTerm = (
LiteralValue |
CollectionValue |
TemplateExpr |
VariableExpr |
FunctionCall |
ForExpr |
ExprTerm Index |
ExprTerm GetAttr |
ExprTerm Splat |
"(" Expression ")"
);The ExprTerm defines a subset of a value and expression. If we use the characters ' and ', we can write a subexpression that follows exactly the same rules as the normal expression. The proceeding sections on values and expressions better illustrate these points. Expressions are used in HCL to define any single piece of the program. There are different types of expressions. One is the conditional expression: condition ? True : False
This expression uses a conditional expression to check a value and give a result of True or False. This is essentially an if...then or a for expression:
[for servers in var.list : lower(servers)]
The for...in is used to check every server in the list, and it’s written in all lowercase. There is another type of expression and you’ll see it in detail in the rest of the chapter.
A literal value represents the value and the type of a primitive. The literal value defines
The BNF definition for the LiteralValue is similar to
LiteralValue = (
NumericLit |
"true" |
"false" |
"null"
);The literal value does not directly define the string value. It is not directly defined in the LiteralValue. To overcome this, the user is allowed to create a string using the template language. The template can then be incorporated to create the string.
The collection value is used to create and define a collection. The BNF syntax to define collection value in comparison is
CollectionValue = tuple | object;
tuple = "[" (
(Expression ("," Expression)* ","?)?
) "]";
object = "{" (
(objectelem ("," objectelem)* ","?)?
) "}";
objectelem = (Identifier | Expression) "=" Expression;To create a tuple with an object, the tuple is enclosed within [], which is similar to an array. A set of objects can be created using the { and }. When we specify an object, we have the name and the value of the object. For example, we can define an object in this way: { foo = "Example" }
A template expression, TemplateExpr, embeds a program written in the sublanguage. The program is written as an expression and is normally used in particular in Terraform when we want to define a template.
A template in Terraform is an external file used in Terraform to dynamically load some resource. The template expression can be written in two different forms:
In the quoted template expression, any literal string sequence can be used within the template. It is possible to escape it using the backslash character, . The hereDoc template syntax allows for more flexibility during the creation of a template. This is the way to define the template in Terraform; for example, we can use a string form to define a template like this one:
data "template_file" "Initialize Consul Address" {
template = "${file("${path.module}/hcl_book.tpl")}"
vars = {
consul_address = "${aws_instance.consul.private_ip}"
}
}The following code calls a file named hcl_book.tpl. The tpl file is used to define the template expression. The hcl_book.tpl is written like
#!/bin/bash
echo "CONSUL_ADDRESS = ${consul_address}" > /tmp/iplistThe script reads the CONSUL_ADDRESS configured on the system and writes in the file iplist. The template file uses the double-quote syntax to define the template expression; the other form used to define the .tpl file is the hereDoc form. The hereDoc form is used to read the data from the template file. We can change the code in this way:
user_data = <<-EOT
echo "CONSUL_ADDRESS = ${aws_instance.consul.private_ip}" > /tmp/iplist
EOT.The VariableExpr is used to define the variable. The variable is in the expression and can be defined in its global scope. The BNF for a variable is very simple: VariableExpr = Identifier; This syntax is used to create the variable, which in turn allows it to be used in a configuration file.
A function is an operation identified usually with a symbolic name and it solves single or multiple operations as part of an algorithm returning a result. The namespace of the function is completely different from the namespace of the configuration file. Hence a function and variable can share the same name but this is not advisable because it can cause understandable challenges! A function is created using the FunctionCall syntax. The BNF syntax for the FunctionCall is
FunctionCall = Identifier "(" arguments ")";
Arguments = (
() ||
(Expression ("," Expression)* ("," | "...")?)
);A function is identified by the identifier, the name of the function. After the name are the characters ( and ), and in middle of the parenthesis are the arguments of the function. They are the parameters used in the function to produce the output we want. If the last argument is followed by ..., this means the final argument of the function must be evaluated like a tuple. HashiCorp define a lot of functions. They can be used to perform basic operations. For example, the function min() returns the minimum number: min(12,4,6)
A for expression is a new functionality of HCL2. It is used to read the value from a collection and build another collection. A for expression is used to read a value from a tuple or from an object collection. A for expression is used inside a tuple or an object to extract a set of values from the original tuple. For example, [ for value in ["a", "b"]: value] This code returns a new array called value with the value ["a","b"]. The position in an array starts at 1.
It is possible to use the for expression to read the array and have just the numerical position for the element. To create an array, we can use this syntax for the for expression: [for i, value in ["a","b"] : value
This syntax returns an array with these values: [1, 2]. It is possible to use an if statement to filter on the values selected in the for statement: [for i, value in ["a","b","c"] : value if i < 2 ] The code returns ["a","b"].
The last three components we need to analyze are the index, GetAttr, and splat. An index is used to return a value from a tuple, list, or map. It returns a single element from a collection of values. The expression is delimited by the square brackets, [ ], and identified by the expression. It is the key we need to get from the list. The key is applied to the list. In a scenario where the key is not present on this list, an error is returned. The GetAttr is an attribute access operator which returns a single value from an object. The syntax is an identifier followed by the name of the attributes we want to read from the object.
The last operator is the splat. It’s a unique type of operator which is used to access an element in a list. There are two types of splat operators:
The splat operator is used when shorthand for the normal operation is better suited, such as selecting a tuple for an object selection. The splat operator can have a "sugar syntax" for the for operator. A normal for loop in HCL has this syntax:
[for value in tuple: value.foo.bar[1]]
Using a splat operator, we can rewrite the for to get the same result. The splat operator can be rewritten in this way:
tuple[*].foo.bar[1]
In the second case, we used the full splat syntax. The operation is equal to [for value in tuple: value.foo.bar[1]].