Terraform Cloud Remote Private Registry Failing

Terraform Cloud Course
Lab - Private Module Registry

Public modules work perfectly fine within Terraform Cloud but not with Private Registry module. I validated this using CURL revealing error code 401 and Terraform trace debug logs. There seems to be a relay problem of the API token from managing private registry modules. I’ve reported this to Terraform Cloud support and waiting to hear back from Kode Kloud support team as well on replicating this issue.

Public Registry Module successfully passes with Terraform init

module "security-group" {
  source  = "terraform-aws-modules/security-group/aws"
  version = "4.16.0"
}

Private Registry Module fails after Terraform init

module "security-group" {
  source  = "app.terraform.io/terraform-cloud-labs/security-group/aws"
  version = "4.16.0"
}

CURL output below using environment variables to my org, workspace, module, provider and login credentials

curl -v -H "Authorization: Bearer ${TOKEN}" \
  https://${HOSTNAME}/api/registry/v1/modules/${ORG_NAME}/${MODULE}/${PROVIDER}/versions

*   Trying 75.2.98.97:443...
* Connected to app.terraform.io (75.2.98.97) port 443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
*  CAfile: /opt/local/share/curl/curl-ca-bundle.crt
*  CApath: none
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=app.terraform.io
*  start date: Dec 19 00:00:00 2021 GMT
*  expire date: Jan 17 23:59:59 2023 GMT
*  subjectAltName: host "app.terraform.io" matched cert's "app.terraform.io"
*  issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
*  SSL certificate verify ok.
<truncated for privacy>
< HTTP/2 401 
< date: Sun, 06 Nov 2022 05:14:12 GMT
< content-type: application/vnd.api+json; charset=utf-8
< content-length: 52
< cache-control: no-cache
< referrer-policy: strict-origin-when-cross-origin
< strict-transport-security: max-age=63072000; includeSubDomains; preload
< vary: Accept-Encoding
< vary: Accept, Origin
< x-content-type-options: nosniff
< x-download-options: noopen
< x-frame-options: SAMEORIGIN
< x-permitted-cross-domain-policies: none
< x-ratelimit-limit: 30
< x-ratelimit-remaining: 29
< x-ratelimit-reset: 0.215
< x-request-id: 9725597b-29c8-d668-1bb1-ee0ff3ccb6b3
< x-xss-protection: 1; mode=block
< 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection #0 to host app.terraform.io left intact
**{"errors":[{"status":"401","title":"unauthorized"}]}%** 

Trace log below
export TF_LOG=trace
export TF_LOG_PATH=./terraform.log
terraform init

output below

2022-11-06T10:51:52.465-0500 [DEBUG] found available version "4.8.0" for https:/terraform-cloud-labs/security-group/aws
2022-11-06T10:51:52.465-0500 [DEBUG] found available version "4.9.0" for https:/terraform-cloud-labs/security-group/aws
2022-11-06T10:51:52.465-0500 [DEBUG] found available version "4.11.0" for https:/terraform-cloud-labs/security-group/aws
2022-11-06T10:51:52.465-0500 [DEBUG] found available version "4.10.0" for https:/terraform-cloud-labs/security-group/aws
2022-11-06T10:51:52.465-0500 [DEBUG] found available version "4.12.0" for https:/terraform-cloud-labs/security-group/aws
2022-11-06T10:51:52.465-0500 [DEBUG] found available version "4.13.0" for https:/terraform-cloud-labs/security-group/aws
2022-11-06T10:51:52.465-0500 [DEBUG] found available version "4.13.1" for https:/terraform-cloud-labs/security-group/aws
2022-11-06T10:51:52.465-0500 [DEBUG] found available version "4.14.0" for https:/terraform-cloud-labs/security-group/aws
2022-11-06T10:51:52.465-0500 [DEBUG] found available version "4.15.0" for https:/terraform-cloud-labs/security-group/aws
2022-11-06T10:51:52.465-0500 [DEBUG] found available version "4.16.0" for https:/terraform-cloud-labs/security-group/aws
2022-11-06T10:51:52.467-0500 [DEBUG] looking up module location from "https://app.terraform.io/api/registry/v1/modules/terraform-cloud-labs/security-group/aws/4.8.0/download"
2022-11-06T10:51:52.467-0500 [DEBUG] GET https://app.terraform.io/api/registry/v1/modules/terraform-cloud-labs/security-group/aws/4.8.0/download
2022-11-06T10:51:52.467-0500 [TRACE] HTTP client GET request to https://app.terraform.io/api/registry/v1/modules/terraform-cloud-labs/security-group/aws/4.8.0/download
2022-11-06T10:51:52.544-0500 [ERROR] security-group from app.terraform.io/terraform-cloud-labs/security-group/aws 4.8.0: module "app.terraform.io/terraform-cloud-labs/security-group/aws" version "4.8.0" not found
2022-11-06T10:51:52.544-0500 [DEBUG] Module installer: begin vpc
2022-11-06T10:51:52.564-0500 [TRACE] ModuleInstaller: Module installer: vpc 3.18.1 already installed in .terraform/modules/vpc
2022-11-06T10:51:52.564-0500 [TRACE] modsdir: writing modules manifest to .terraform/modules/modules.json```

Hello Kode Kloud Support,

This seems like a common problem. Please replicate this issue and help us with a workaround solution. Thanks.

Hi @morgayle,

Thanks for reporting the issue with detailed information, I sent it to the KK development team to troubleshoot.

Thanks,
Trung.