Terraform backend-s in Unix/Linux

Terraform saves the state of the managed infrastructure and its configuration. The default state is stored locally in a JSON file named terraform.tfstate. From the first lift to the last plan change infrastructure every piece of data fills in the status file, allowing to map real resources to terraform configuration and track metadata.

“Backend” in Terraform is an abstraction that defines the processing condition and the way to perform certain operations, providing many important functions and is used to load state files when run “terraform apply”.

If you are using Terraform for a personal project, saving state in a local file terraform.tfstate works just fine. But if you want to use Terraform to your entire team for actual product you will come across several problems:

  • Shared storage is required for state files — to use Terraform to upgrade your infrastructure, each member of your team need access to the same file status Terraform. This means that you need to store these files in a common location.
  • The required file locking status as soon as data is transferred, you are faced with a new problem: lock. Without blocking, if two members of the team at the same time Terraform is used, you may encounter race conditions, since multiple processes simultaneously Terraform updates the state file, which leads to conflicts, data loss and file corruption conditions.
  • Requires data isolation. Ie for the prod will be used by some modules and state file for a different type ENV — other.

The types of the Terraform backend-ov:

  • AWS S3 — Standard (with locking via DynamoDB). Saves the state in the form of a specified key in a specified segment on Amazon S3. This backend also supports locking state and consistency checking using DynamoDB.
  • Consul — Standard (with locking). Saves state to the Consul in the form of K/V at the specified path.
  • terraform enterprise — Standard (non-blocking).
  • Remote State Storage is to use Terraform Cloud as the backend.
  • artifactory — Standard (no locking). Retains status as an artifact in Artifactory service.
  • azurerm — Standard (locking state). Saves the state as a BLOB with the given key in the “Blob Container” inside the “Blob Storage Account”. This backend also supports locking state and a consistency check using the “Azure Blob Storage”.
  • etcd — Standard (without Loka). Saves the state in etcd 2.x at a specified path.
  • etcdv3 — Standard (with lock). Saves the state in the vault etcd as K/V with the specified prefix.
  • gcs — Standard (Locke). Saves the state as an object in a custom prefix in a given segment in Google Cloud Storage (GCS). This backend also supports the lock state.
  • the http Standard (with optional lock). Saves the state using a simple REST client. The status is retrieved using GET, updated via POST and cleaned using DELETE. The method used for the updates is configurable.
  • manta — Standard (with locks in the most manta). Stores the state as an artifact in manta.
  • OSS — Standard (locked via TableStore). Saves the state as of a given key in a given segment in the vault Alibaba Cloud OSS. This backend also supports locking state and a consistency check through Alibaba Cloud Table Store, which you can enable by setting in the field tablestore_table an existing table name TableStore.
  • pg — Standard (with lock). Saves state to the database Postgres version 9.5 or later. This backend supports the lock state.
  • swift — Standard ( without the support of the blocking state). Stores the state as an artifact in Swift.

Not all of the examples I will touch upon in the illustrative examples, if you want to use — will add more. And now, here are the most basic and frequently used.

Terraform AWS S3 backend

Createopen the file:

$ vim backend.tf

Write in it:

terraform { backend "s3" { encrypt = true, bucket = "terraform-state" region = "us-west-2" key = "jenkins/jenkins-aor-nonprod.tfstate" shared_credentials_file = "$HOME/.aws/credentials" profile = "SSO_DCOPS" } required_providers { aws = ">= 2.14.0" } } 

Running init:

[email protected]  ~/Projects/work/Git_repo/garson_to_aws/tf/projects/nonprod/aor   master ●  terraform-0.11 init Initializing modules... - module.key_pair - module.alb - module.iam_asg - module.asg - module.efs_asg - module.iam_ecs - module.ecs_cluster Initializing the backend... Initializing provider plugins... The following providers do not have any version constraints in configuration, so the latest version was installed. To prevent automatic upgrades to new major versions that may contain breaking changes, it is recommended to add version = "..." constraints to the corresponding provider in configuration blocks, with the constraint strings suggested below. * provider.aws: version = "~> 2.33" * provider.template: version = "~> 2.1" Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All the Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary. ⚙ [email protected]  ~/Projects/work/Git_repo/garson_to_aws/tf/projects/nonprod/aor   master ● 

To make lock for TF, it is possible to use DynamoDB and store Loki for this, you need to create a table, e.g. “ship-jenkins-nonprod-tf”:

As a “Primary key” or “Partition key”, it should prescribe the “LockID”. Click on create. Now we need in the backend.tf add “dynamodb_table” field with your table name I have is “ship-jenkins-nonprod-tf”.

Open the file:

$ vim backend.tf

Write in it:

terraform { backend "s3" { encrypt = true, bucket = "terraform-state" region = "us-west-2" key = "jenkins/jenkins-aor-nonprod.tfstate" dynamodb_table = "ship-jenkins-nonprod-tf" shared_credentials_file = "$HOME/.aws/credentials" profile = "SSO_DCOPS" } required_providers { aws = ">= 2.14.0" } } 

On this everything.

Consul Terraform backend

Open the file:

$ vim backend.tf

Prescription:

terraform { backend "consul" { address = "server.consul.io" scheme = "https" path = "my_app_project/terraform.tfstate" } }

To connect the backend, you should run:

$ terraform init

You can use variables and override some values, for example:

$ terraform init  -backend-config="address=my_server_here.consul.io"  -backend-config="path=test_project/terraform.state"  -backend-config="scheme=https"

Something like that.

Terraform enterprise backend

Open the file:

$ vim backend.tf

Prescription:

terraform { backend "atlas" { name = "<ORGANIZATION>/<WORKSPACE>" address = "https://app.terraform.io" } }

To connect the backend, you should run:

$ terraform init

And so on, by analogy with other examples.

Terraform remote State Storage backend

Open the file:

$ vim backend.tf

Prescription:

terraform { backend "remote", { organization = "<ORG_NAME>" workspaces { name = "Example-Workspace" } } }

Still need a token that you will need to paste in:

$ vim ~/.terraformrc

In this file write:

credentials "app.terraform.io" { token = "token" }

You can find it on the official website (admin panel):

That’s all the article “Terraform backend-s in Unix/Linux” is completed.

Source: linux-notes.org

(Visited 6 times, 1 visits today)