Providers, resources, and data sources have schemas. This document explores the options for how they relate to each other, and the types provider developers use to define them.
This problem is fairly symmetrical with respect to providers, resources, and data sources, so we choose to focus on resource schemas without loss of generality.
In helper/schema, Resource is a struct type with a field Schema, whose type is map[string]*Schema.
In the new framework, framework.Resource may be a struct type or interface: see the Structs and Interfaces design doc.
Since the schema is defined by the provider code but must be accessible from the framework code, this means that if framework.Resource is a struct, the schema can either be a field (framework.Resource.Schema), or a method (framework.Resource.GetSchema()), i.e. we could have
type Resource struct {
Schema *tfprotov6.Schema
}or
type Resource struct {
GetSchema func(context.Context) (*tfprotov6.Schema, []*tfprotov6.Diagnostic)
}If framework.Resource is an interface, the most straightforward option is to include a GetSchema() method which the provider code must implement, since the actual resource struct will be a provider-defined type:
type Resource interface {
GetSchema(ctx context.Context) (*tfprotov6.Schema, []*tfprotov6.Diagnostic)
}Another option is to define a framework.Schema type which providers can embed in their resource structs. For example, we could have the following framework code:
type Resource interface {
GetSchema(ctx context.Context) (*tfprotov6.Schema, []*tfprotov6.Diagnostic)
// other Resource methods
}
type Schema map[string]*tfprotov6.Schema
func (s Schema) GetSchema(_ context.Context) (*tfprotov6.Schema, []*tfprotov6.Diagnostic) {
// implement a conversion function here that converts from s to *tfprotov6.Schema
}Then in the provider code:
type myResource struct {
framework.Schema
}
func NewMyResource() framework.Resource {
return myResource{
"foo": *tfprotov6.Schema{},
"bar": *tfprotov6.Schema{},
}
}Here, provider developers still have the option of implementing myResource.GetSchema(), so myResource fulfils the framework.Resource interface. If instead of Schema.GetSchema() the framework defines Schema.getSchema(), an unexported method, the provider is forced to use the struct implementation.
This approach is simple and discoverable. The schema is a static piece of data (in conventional providers) known at compile time, so it is idiomatic that it be represented as a field on a struct.
While (depending on the type used to represent the schema - see below) it may not be apparent at compile time if a provider developer has forgotten to fill in a value for Resource.Schema, it will be obvious after the most basic of testing if this is the case.
This approach is also fairly simple and discoverable, and offers the option of returning an error or diag if the schema is not available. This may be a desirable feature for providers written using generated code, where generating the schema could throw an error.
A Resource struct could have a Schema field, and a GetSchema field with func type added later if needed. The framework could use whichever is set, either erroring if both are set or using a documented fallback mechanism to prefer one over the other.
Whatever the relationship of schema to resource, it is structured data that must be represented by a Go struct or map type. This section details two options for this: the existing Schema type in tfprotov6, and a proposed framework.Schema type.
The tfprotov6 (and tfprotov5) package in terraform-plugin-go provides a Schema type and related types: https://pkg.go.dev/github.com/hashicorp/terraform-plugin-go/tfprotov6#Schema
Resource schemas defined using these types can be quite verbose. For example, consider the following implementation of a resource with one attribute and one nested block, adapted from the sql_migrate resource from terraform-provider-sql:
var schema = &tfprotov5.Schema{
Block: &tfprotov5.SchemaBlock{
BlockTypes: []*tfprotov5.SchemaNestedBlock{
{
TypeName: "migration",
Nesting: tfprotov5.SchemaNestedBlockNestingModeList,
Block: &tfprotov5.SchemaBlock{
Attributes: []*tfprotov5.SchemaAttribute{
{
Name: "id",
Required: true,
Description: "Identifier can be any string to help identifying the migration in the source.",
DescriptionKind: tfprotov5.StringKindMarkdown,
Type: tftypes.String,
},
{
Name: "up",
Required: true,
Description: "The query to run when applying this migration.",
DescriptionKind: tfprotov5.StringKindMarkdown,
Type: tftypes.String,
},
{
Name: "down",
Required: true,
Description: "The query to run when undoing this migration.",
DescriptionKind: tfprotov5.StringKindMarkdown,
Type: tftypes.String,
},
},
},
},
},
Attributes: []*tfprotov5.SchemaAttribute{
&tfprotov5.SchemaAttribute{
Name: "complete_migrations",
Computed: true,
Description: "The completed migrations that have been run against your database. This list is used as " +
"storage to migrate down or as a trigger for downstream dependencies.",
DescriptionKind: tfprotov5.StringKindMarkdown,
Type: tftypes.List{
ElementType: tftypes.Object{
AttributeTypes: map[string]tftypes.Type{
"id": tftypes.String,
"up": tftypes.String,
"down": tftypes.String,
},
},
},
},
},
},
}Like helper/schema, the framework could wrap the tfprotov6.Schema types with helpers to reduce verbosity. This section illustrates how such a framework.Schema type might be used in provider code: the exact implementation itself is outside the scope of this design document. A minimally verbose option could look something like:
var schema = map[string]*framework.Schema{
"migration": {
Type: framework.NestedBlockType,
Nesting: framework.SchemaNestedBlockNestingModeList,
Attributes: {
"id": {
Required: true,
Description: "Identifier can be any string to help identifying the migration in the source.",
Type: framework.StringType,
},
"up": {
Required: true,
Description: "The query to run when applying this migration.",
Type: framework.StringType,
},
"down": {
Required: true,
Description: "The query to run when undoing this migration.",
Type: framework.StringType,
},
},
},
"complete_migrations": {
Computed: true,
Description: "The completed migrations that have been run against your database. This list is used as " +
"storage to migrate down or as a trigger for downstream dependencies.",
Type: framework.TypeList{
ElementType: framework.TypeObject{
AttributeTypes: map[string]framework.AttributeType{
"id": framework.TypeString,
"up": framework.TypeString,
"down": framework.TypeString,
},
},
},
},
}Types such as Schema and StringType would likely be exported by a package (like helper/schema), not the top-level framework package, whose name is used here for simplicity.
The work being done here by the framework.Schema type, from the provider developer's perspective, is:
- Remove the need for a
Blockfield in the schema struct for the root block, since this is unambiguous; - Use the attribute or nested block
Nameas a map key; - Handle defaults for fields such as
DescriptionKind(Nestingcould also be set appropriately); - Allow nested blocks to be defined in the same way as attributes with a special
NestedBlockType.
Apart from reducing verbosity, this approach has a number of benefits. Users will not have to import tftypes or tfprotov6 packages, removing the burden of having to understand why the terraform-plugin-go module exists separately from the framework. The framework schema types are more discoverable, being documented inside the framework code for framework users, not in plugin-go.
The API of the framework schema package for defining resource schemas (i.e. the functionality shown in the examples above) should not have to change unless there is a change in the Terraform protocol concerning schemas.
Compatibility can be evaluated by considering the following two situations.
Firstly, consider what happens if a backwards-compatible change is made in the Terraform protocol, such as the addition of SchemaObject in tfprotov6. In this case, the framework schema package should also be able to make the change in a backwards-compatible way, unless it concerns one of the assumptions made above (for example, that attributes/blocks will always have a unique Name that can be used as a map key). This seems unlikely, but it is possible: for example, a new BlockType is added which has no equivalent to Name or TypeName.
Secondly, consider what happens if a backwards-incompatible change is made in the Terraform protocol. In this case, a new Terraform protocol major version would be needed anyway, so the compatibility status of framework.Schema is no worse than tfprotov5.schema.
Note that in the case of Terraform protocol v5 and v6, despite a major version increase, the functional change made is backwards-compatible with respect to the schema, so tfprotov6.Schema is a backwards-compatible extension of tfprotov5.Schema. Wrapping these types in a framework.Schema type hides this complexity from users and saves them the work of updating their use of tfprotovN.Schema if something similar happens in the future.
The new framework should guide users toward the proper use of block and attribute types. For example, it should be obvious to users when they can use Optional, Computed, and Sensitive when using block, collection, and structural Terraform types. This is an infamous difficulty which helper/schema does not help users to navigate.
Like other implementation details of any framework.Schema type, the way the framework structures its types, or perhaps throws errors when an invalid combination of attributes and flags is attempted, is out of scope for this document - however, there may need to be some tradeoff here between brevity and helpfulness.
We have decided to ask users to implement a GetSchema method, on the theory
that we can always provide an embed or helper type for that behavior in the
future. We have also decided we would like to wrap the terraform-plugin-go Schema type in a
framework-specific type, to provide us with opportunities to abstract some of
the protocol verbosity away.