Terraform REST API Provider: A Pragmatic Approach to CRUD Operations

Terraform is renowned for its powerful capabilities in managing project infrastructure. It adeptly handles a wide range of use cases, from providing cloud services to provisioning databases. In this article, I will explore a scenario where we build a custom resource without creating our own Terraform provider. As it is not the most effective solution for specialized functionalities, I will offer a more suitable alternative.

Problem Statement

Consider a scenario where our project depends on a crucial resource that is essential for its operation. This resource is not intended to be managed directly by users. For instance, consider configuration entities that are utilized during the application's startup process and provide vital input for the entire system. I would prefer to avoid developing an administrative portal for CRUD operations or investing many hours in creating a custom Terraform provider.

Config Resource Description

To create a Config entity it requires:

MongoDB object, where the configuration data is maintained:

{
  "_id": "entity identifier",
  "name": "entity name",
  "revision": "entity revision",
  "type": "entity type",
  "status": "track entity state for each action, like create/update/delete",
  "fileUrl": "file url to s3 storage"
}

Configuration file stored on an S3 bucket.

Configuration is only accessible when the MongoDB object is linked to a file stored on an S3 bucket. The entire process is time-consuming and should be made asynchronous to enhance efficiency.

Solution

Explore a case where we have implemented two microservices:

• REST API service — tasked with managing CRUD operations via the /configs endpoints
• File Configuration service — dedicated to generating and storing files in an S3 bucket

Both services facilitate communication via a message queue. The REST API service only marks the configuration entity as valid once the file has been successfully uploaded. We have implemented the entire business logic; the next step involves its integration with Terraform.

For this particular application, I will utilize the restapi provider developed by Mastercard. This integration allows us to treat the configuration entity as we would any other infrastructure component within Terraform.

Terraform Code

The following configuration is our entry point, where we reference our server that performs the object creation. We specify the server address and the necessary information for the request.

# provider.tf
provider "restapi" {
  uri                  = "https://api.url.com"
  write_returns_object = true
  debug                = true

  headers = {
    "X-Auth-Token" = var.AUTH_TOKEN,
    "Content-Type" = "application/json"
  }
}

# main.tf
resource "restapi_object" "config_1" {
  path = "/api/objects"
  data = jsonencode(
    {
      "name" : "foo",
      "revision" : "1.0.2",
      "type" : 2,
    }
  )
}

Run terraform apply and let the magic happen.

Conclusions

This solution provides significant flexibility and allows for rapid implementation within our system. I can fully recommend it for domain-specific use cases. However, it is sometimes beneficial to create a custom Terraform provider, particularly for more generic functionalities. In the future, I will explore how to build a proper provider, covering the best practices.