Delay Schema Caching Until Resource Operations

[bflad]

Module version

v1.3.1

Description

Currently, this Go module will cache data source and managed resource (collectively called "resource" from now on) schemas when the GetProviderSchema RPC is called. This caching was introduced to prevent the multiple RPCs associated with a single resource from continually invoking the Schema method for no benefit as by Terraform's definition, schemas must be static. For large providers, this is very problematic as it will actively keep unused schema information that is likely not needed again in memory. Providers do not have an opt-out for this framework behavior.

Proposal

Rather than caching resource schemas during GetProviderSchema RPC, which in turn will keep all schemas in memory, only perform the caching when another resource-based operation RPC occurs. This should ensure that:

• Go garbage collection can occur after the GetProviderSchema RPC
• Actively used schemas still receive the slight performance benefit of caching rather than repeatedly invoking the Schema method

It'll be worth exploring with upstream if the schema fetching can be split across RPCs to prevent a large allocation for GetProviderSchema RPC response.

References

• tfsdk: Consider Caching Provider Defined Resources/Data Sources in Server #299
• [Bug]: Memory consumption has increased since v4.67.0 terraform-provider-aws#31722

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.