Documentation generation for Terraform using python3 on Unix/linux

Long time did not write anything in Python, probably 6 months (came on a new project in the company). And then, the interest appeared to write the generated documentation for Terraform modules which I contribute. In General, I’m tired to write documentation based on describing in variable.tf and outputs.tf files. Figured that it really started to write….

For a start, you should install the module:

$ pip3 install pyhcl

Script in Python looks like this:

#!/usr/bin/env python3 # -*- coding: utf-8 -*- import argparse import time import hcl import os import os.path import json class Bgcolors: def __init__(self): self.get = { 'HEADER': '33[95m', 'OKBLUE': '33[94m', 'OKGREEN': '33[92m', 'WARNING': '33[93m', 'FAIL': '33[91m', 'ENDC': '33[0m', 'BOLD': '33[1m', 'UNDERLINE': '33[4m' } def header(m_dir, e_dir): if (m_dir is not None) and (e_dir is not None): dir_name = m_dir.strip().split('/')[-1] file_out = 'OUTPUT_{}.md'.format(dir_name) if os.path.isfile(file_out): os.remove(file_out) headers = """# Work with AWS {0} via A terraform terraform module for making {0}. ## Usage ---------------------- Import the module and retrieve with ``terraform get`` or ``terraform get --update``. Adding a module resource to your template, e.g. `main.tf`: """.format(dir_name.upper()) examples_dir = "{0}/{1}".format(e_dir, dir_name) examples_file = examples_dir + "/" + "main.tf" f_main = open(examples_file, "r") try: f = open(file_out, 'a') f.write(str(headers + "n")) f.write("``n") for x in f_main.readlines(): f.write(x) f.write("``nn") f.close() except ValueError: print('I cant write to [{}] file'.format(file_out)) else: print(Bgcolors().get['FAIL'], 'Please set/add [--mdir] or [--edir]', Bgcolors().get['ENDC']) print(Bgcolors().get['OKGREEN'], 'For help, use: script_name.py -h', Bgcolors().get['ENDC']) exit(1) return def header generate_inputs(m_dir): if m_dir is not None: tf_file_variables = m_dir + '/' + 'variables.tf' if os.path.isfile(tf_file_variables) is False: print(Bgcolors().get['FAIL'], 'File doesnt exist! Check PATH to module!', Bgcolors().get['ENDC']) print(Bgcolors().get['FAIL'], "You're trying to use [{}]".format(dir), Bgcolors().get['ENDC']) exit(0) dir_name = m_dir.strip().split('/')[-1] file_out = 'OUTPUT_{}.md'.format(dir_name) input_header = """## Module Input Variables ----------------------""" try: f = open(file_out, 'a') f.write(str(input_header + "n")) f.close() except ValueError: print('I cant write to [{}] file'.format(file_out)) with open(tf_file_variables, 'r') as fp_in: obj = hcl.load(fp_in) # print (json.dumps(obj, indent=4, sort_keys=True)) for variable in obj['variable']: description = obj['variable'][variable]['description'] default = obj['variable'][variable]['default'] line = '- `%s` - %s (`default = %s`)' % (str(variable), str(description), str(default)) try: f = open(file_out, 'a') f.write(str(line + 'n')) f.close() except ValueError: print('I cant write to [{}] file'.format(file_out)) else: print(Bgcolors().get['FAIL'], 'Please set/add [--mdir]', Bgcolors().get['ENDC']) print(Bgcolors().get['OKGREEN'], 'For help, use: script_name.py -h', Bgcolors().get['ENDC']) exit(1) return def generate_inputs generate_outputs(m_dir): assert isinstance(m_dir, object) if m_dir is not None: tf_file_output = str(m_dir) + '/' + 'outputs.tf' dir_name = str(m_dir).split('/')[-1] file_out = 'OUTPUT_{}.md'.format(dir_name) if os.path.isfile(tf_file_output): # print(tf_file_output) output_header = """ ## Module Output Variables ----------------------""" try: f = open(file_out, 'a') f.write(str(output_header + "n")) f.close() except ValueError: print('I cant write to [{}] file'.format(file_out)) with open(tf_file_output, 'r') as fp_out: obj = hcl.load(fp_out) # print (json.dumps(obj, indent=4, sort_keys=True)) for output in obj['output']: description = obj['output'][output]['description'] # print(description) if not description: description = '""' line = '- `%s` - %s' % (output, description) try: f = open(file_out, 'a') f.write(str(line + 'n')) f.close() except ValueError: print('I cant write to [{}] file'.format(file_out)) else: print(Bgcolors().get['FAIL'], 'Please set/add [--dir]', Bgcolors().get['ENDC']) print(Bgcolors().get['OKGREEN'], 'For help, use: script_name.py -h', Bgcolors().get['ENDC']) exit(1) return generate_outputs def footer(m_dir): if m_dir is not None: dir_name = m_dir.strip().split('/')[-1] file_out = 'OUTPUT_{}.md'.format(dir_name) authors = """ ## Authors ======= Created and maintained by [Vitaliy Natarov](https://github.com/SebastianUA) ([email protected]). License ======= The Apache 2 Licensed. See [LICENSE](https://github.com/SebastianUA/terraform/blob/master/LICENSE) for full details.""" try: f = open(file_out, 'a') f.write(str(authors + "n")) f.close() except ValueError: print('I cant write to [{}] file'.format(file_out)) print('Looks like that the [{0}] file has been created: {1}'.format(file_out, os.getcwd())) else: print(Bgcolors().get['FAIL'], 'Please set/add [--mdir]', Bgcolors().get['ENDC']) print(Bgcolors().get['OKGREEN'], 'For help, use: script_name.py -h', Bgcolors().get['ENDC']) exit(1) return footer def main(): start__time = time.time() parser = argparse.ArgumentParser(prog='python3 script_name.py -h', usage='python3 script_name.py {ARGS}', add_help=True, prefix_chars='--/', epilog="'created by Vitalii Natarov"') parser.add_argument('--version', action='version', version='v1.0.0') parser.add_argument('--md', '--mdir', dest='m_directory', help='Set the directory where the module exists', default=None, metavar='folder') parser.add_argument('--ed', '--edir', dest='e_directory', help='Set the directory where the example exists', default=None, metavar='folder') results = parser.parse_args() m_directory = results.m_directory # type: object e_directory = results.e_directory header(m_directory, e_directory) generate_inputs(m_directory) generate_outputs(m_directory) footer(m_directory) end__time = round(time.time() - start__time, 2) print("--- %s seconds ---" % end__time) print( Bgcolors().get['OKGREEN'], "============================================================", Bgcolors().get['ENDC']) print( Bgcolors().get['OKGREEN'], "==========================FINISHED==========================", Bgcolors().get['ENDC']) print( Bgcolors().get['OKGREEN'], "============================================================", Bgcolors().get['ENDC']) if __name__ == '__main__': main() 

Need to run this:

$ python3 generate_docs_pyhcl.py --mdir="/Users/captain/Projects/Terraform/aws/modules/nlb" --edir="/Users/captain/Projects/Terraform/aws/examples/nlb"

Where:

  • python3 — Binary with Python.
  • generate_docs_pyhcl.py — the Name of the script.
  • —mdir=”/Users/captain/Projects/Terraform/aws/modules/nlb” Is the path to podulo.
  • —edir=”/Users/captain/Projects/Terraform/aws/examples/nlb” Is the path where your projecteksempel.

The output will be like this:

Looks like that the [OUTPUT_nlb.md] file has been created: /Users/captain/Projects/Python/Terraform --- 0.04 seconds --- ============================================================ ==========================FINISHED========================== ============================================================

NOTE! This will only work on file at the feature description & default lines!

If you open the file that was generated, it looks like this:

# Work with AWS via A NLB terraform terraform module for making NLB. ## Usage ---------------------- Import the module and retrieve with ``terraform get`` or ``terraform get --update``. Adding a module resource to your template, e.g. `main.tf`: `` # # MAINTAINER: Vitaliy Natarov "[email protected]" # terraform { required_version = "> 0.9.0" } provider "aws" { region = "us-east-1" shared_credentials_file = "${pathexpand("~/.aws/credentials")}" profile = "default" } module "iam" { source = "../../modules/iam" name = "My-Security" region = "us-east-1" environment = "PROD" aws_iam_role-principals = [ "ec2.amazonaws.com", ] aws_iam_policy-actions = [ "cloudwatch:GetMetricStatistics", "logs:DescribeLogStreams", "logs:GetLogEvents", "elasticache:Describe*", "rds:Describe*", "rds:ListTagsForResource", "ec2:DescribeAccountAttributes", "ec2:DescribeAvailabilityZones", "ec2:DescribeSecurityGroups", "ec2:DescribeVpcs", "ec2:Owner", ] } module "vpc" { source = "../../modules/vpc" name = "My" environment = "PROD" # VPC instance_tenancy = "default" enable_dns_support = "true" enable_dns_hostnames = "true" assign_generated_ipv6_cidr_block = "false" enable_classiclink = "false" vpc_cidr = "172.31.0.0/16" private_subnet_cidrs = ["172.31.32.0/20"] public_subnet_cidrs = ["172.31.0.0/20"] availability_zones = ["us-east-1b"] enable_all_egress_ports = "true" allowed_ports = ["8080", "3306", "443", "80"] map_public_ip_on_launch = "true" #Internet-GateWay enable_internet_gateway = "true" #NAT enable_nat_gateway = "false" single_nat_gateway = "true" #VPN enable_vpn_gateway = "false" #DHCP enable_dhcp_options = "false" # EIP enable_eip = "false" } module "ec2" { source = "../../modules/ec2" name = "TEST-Machine" region = "us-east-1" environment = "PROD" number_of_instances = 2 ec2_instance_type = "t2.micro" enable_associate_public_ip_address = "true" disk_size = "8" tenancy = "${module.vpc.instance_tenancy}" iam_instance_profile = "${module.iam.instance_profile_id}" subnet_id = "${element(module.vpc.vpc-publicsubnet-ids, 0)}" #subnet_id = "${element(module.vpc.vpc-privatesubnet-ids, 0)}" #subnet_id = ["${element(module.vpc.vpc-privatesubnet-ids)}"] vpc_security_group_ids = ["${module.vpc.security_group_id}"] monitoring = "true" } module "nlb" { source = "../../modules/nlb" name = "Load-Balancer" region = "us-east-1" environment = "PROD" subnets = ["${module.vpc.vpc-privatesubnet-ids}"] vpc_id = "${module.vpc.vpc_id}" enable_deletion_protection = false backend_protocol = "TCP" alb_protocols = "TCP" #It's not working properly when use EC2... First of all, comment the line under this text. Run playbook. Uncomment that line. target_ids = ["${module.ec2.instance_ids}"] } `` ## Module Input Variables ---------------------- - `target_ids` - The ID of the target. This is the Instance ID for an instance, or the container ID for an ECS container. If the target type is ip, specify an IP address. (`default = []`) - `health_check_interval` - Interval in seconds on which the health check against the backend hosts is tried. (`default = 10`) - `name` - Name to be used on all resources as prefix (`default = TEST-NLB`) - `enable_deletion_protection` - If true, deletion of the load balancer will be disabled via the AWS API. This will prevent from deleting Terraform the load balancer. Defaults to false. (`default = False`) - `timeouts_create` - Used for Creating LB. Default = 10mins (`default = 10m`) - `certificate_arn` - The ARN of the SSL Certificate. e.g. 'arn:aws:iam::XXXXXXXXXXX:server-certificate/ProdServerCert' (`default = `) - `load_balancer_type` - The type of load balancer to create. Possible values are application or network. The default value is application. (`default = network`) - `region` - The region where to deploy this code (e.g. us-east-1). (`default = us-east-1`) - `vpc_id` - Set VPC ID for ?LB (`default = `) - `backend_protocol` - The protocol of the backend service speaks. Options: HTTP, HTTPS, TCP, SSL (secure tcp). (`default ' = HTTP`) - `name_prefix` - Creates a unique name beginning with the specified prefix. Conflicts with name (`default = nlb`) - `health_check_healthy_threshold` - positive Number of consecutive health checks before a backend instance is considered healthy. (`default = 3`) - `orchestration` - Type of orchestration (`default = Terraform`) - `health_check_unhealthy_threshold` - positive Number of consecutive health checks before a backend instance is considered unhealthy. (`default = 3`) - `lb_internal` - If true, the NLB will be an internal NLB (`default = False`) - `backend_port` - The port the service on the EC2 instances listen on. (`default = 80`) - `idle_timeout` - The time in seconds that the connection is allowed to be idle. Default: 60. (`default = 60`) - `health_check_port` - The port used by the health check if different from the traffic-port. (`default ' = traffic-port`) - `environment` - Environment for service (`default = STAGE`) - `timeouts_update` - Used for LB modifications. Default = 10mins (`default = 10m`) - `ip_address_type` - The type of IP addresses used by the subnets for your load balancer. The possible values are ipv4 and dualstack (`default = ipv4`) - `alb_protocols` - A protocol accepts the ALB. (e.g.: TCP) (default = TCP`) - `subnets` - A list of subnet IDs to attach to the NLB (`default = []`) - `createdby` - Created by (`default = Vitaliy Natarov`) - `target_type` - The type of target that you must specify when registering targets with this target group. The possible values are instance (targets are specified by the instance ID) or ip (targets are specified by IP address). The default is instance. Note that you can't specify targets for a target group using both instance IDs and IP addresses. If the target type is ip, specify the IP addresses from the subnets of the virtual private cloud (VPC) for the target group, the range of RFC 1918 (10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16), and the RFC 6598 range (100.64.0.0/10). You can't specify publicly routable IP addresses (`default = instance`) - `timeouts_delete` - Used for destroying LB LB. Default = 10mins (`default = 10m`) - `deregistration_delay` - The amount of time for Elastic Load Balancing to wait before changing the state of a target deregistering from draining to unused. The range is 0-3600 seconds. The default value is 300 seconds. (`default = 300`) ## Module Output Variables ---------------------- - `lb_id` - The ID of the lb's we created. - `lb_name` - "" - `lb_arn_suffix` - ARN suffix of our lb - can be used with CloudWatch `lb_listener_frontend_tcp_443_id` - The ID of the lb Listener we created. - `lb_dns_name` - The DNS name of the lb, presumably to be used with a CNAME friendlier. - `lb_listener_frontend_tcp_80_arn` - The ARN of the HTTPS lb Listener we created. - `target_group_arn` - ARN of the target group. Useful for passing to your Auto Scaling group module. - `lb_listener_frontend_tcp_80_id` - The ID of the lb Listener we created. - `lb_listener_frontend_tcp_443_arn` - The ARN of the HTTP lb Listener we created. - `lb_zone_id` - The zone_id of the lb to assist with creating DNS records. - `lb_arn` - ARN of the lb itself. Useful for debug output, for example when attaching a WAF. ## Authors ======= Created and maintained by [Vitaliy Natarov](https://github.com/SebastianUA) ([email protected]). License ======= The Apache 2 Licensed. See [LICENSE](https://github.com/SebastianUA/terraform/blob/master/LICENSE) for full details. 

Something like that! Not bad, right?

That’s all the article is “Generation of documentation for Terraform using python3 on Unix/linux” certified.

Source: linux-notes.org

(Visited 12 times, 1 visits today)