Skip to content

feat: added google_bigquery_table data source#14026

Merged
c2thorn merged 4 commits intoGoogleCloudPlatform:mainfrom
ramonvermeulen:ramon/22834-google-bigquery-table-data-source
May 22, 2025
Merged

feat: added google_bigquery_table data source#14026
c2thorn merged 4 commits intoGoogleCloudPlatform:mainfrom
ramonvermeulen:ramon/22834-google-bigquery-table-data-source

Conversation

@ramonvermeulen
Copy link
Copy Markdown
Contributor

@ramonvermeulen ramonvermeulen commented May 19, 2025

Closes hashicorp/terraform-provider-google#22834

  • First time doing a data source implementation

Release Note Template for Downstream PRs (will be copied)

See Write release notes for guidance.

Acceptance tests locally:

TF_ACC=1 TF_SCHEMA_PANIC_ON_ERROR=1 go test ./google/services/bigquery -v -run=TestAccDataSourceGoogleBigqueryTable_basic -timeout 240m -ldflags="-X=github.com/hashicorp/terraform-provider-google/version.ProviderVersion=acc"
=== RUN   TestAccDataSourceGoogleBigqueryTable_basic
=== PAUSE TestAccDataSourceGoogleBigqueryTable_basic
=== CONT  TestAccDataSourceGoogleBigqueryTable_basic
--- PASS: TestAccDataSourceGoogleBigqueryTable_basic (12.87s)
PASS
ok      github.com/hashicorp/terraform-provider-google/google/services/bigquery 13.550s

Would like to get some thoughts on the schema property, since it has a reference to itself. E.g. when the type = RECORD it has a nested schema, following the google docs this has a max depth of 15.

https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#tableschema
https://cloud.google.com/bigquery/docs/nested-repeated#limitations
How are generally slightly more complex data structures like this implemented in data sources?

Edit: Took a look at #13908 and implemented in a similar manner, handling schema just as the same type as in the original resource (in this case json as string).

Via

dsSchema := tpgresource.DatasourceSchemaFromResourceSchema(ResourceBigQueryTable().Schema)
`google_bigquery_table`

@modular-magician modular-magician added the awaiting-approval Pull requests that need reviewer's approval to run presubmit tests label May 19, 2025
@ramonvermeulen ramonvermeulen force-pushed the ramon/22834-google-bigquery-table-data-source branch from e09ead6 to 86d1c88 Compare May 20, 2025 09:43
@ramonvermeulen ramonvermeulen changed the title feat: added google_bigquery_table data source (WIP) feat: added google_bigquery_table data source May 20, 2025
@ramonvermeulen ramonvermeulen marked this pull request as ready for review May 20, 2025 10:24
@github-actions
Copy link
Copy Markdown

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

Googlers: For automatic test runs see go/terraform-auto-test-runs.

@c2thorn, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

@github-actions github-actions bot requested a review from c2thorn May 20, 2025 10:25
@ramonvermeulen
Copy link
Copy Markdown
Contributor Author

After final changes:

➜ make testacc TEST=./google/services/bigquery TESTARGS='-run=TestAccDataSourceGoogleBigqueryTable_basic'

TF_ACC=1 TF_SCHEMA_PANIC_ON_ERROR=1 go test ./google/services/bigquery -v -run=TestAccDataSourceGoogleBigqueryTable_basic -timeout 240m -ldflags="-X=github.com/hashicorp/terraform-provider-google/version.ProviderVersion=acc"
=== RUN   TestAccDataSourceGoogleBigqueryTable_basic
=== PAUSE TestAccDataSourceGoogleBigqueryTable_basic
=== CONT  TestAccDataSourceGoogleBigqueryTable_basic
--- PASS: TestAccDataSourceGoogleBigqueryTable_basic (13.03s)
PASS
ok      github.com/hashicorp/terraform-provider-google/google/services/bigquery 13.741s

@modular-magician modular-magician added service/bigquery and removed awaiting-approval Pull requests that need reviewer's approval to run presubmit tests labels May 22, 2025
@modular-magician
Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 324 insertions(+))
google-beta provider: Diff ( 4 files changed, 324 insertions(+))

@modular-magician
Copy link
Copy Markdown
Collaborator

Tests analytics

Total tests: 134
Passed tests: 123
Skipped tests: 10
Affected tests: 1

Click here to see the affected service packages
  • bigquery
#### Action taken
Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataSourceGoogleBigqueryTable_basic

Get to know how VCR tests work

@modular-magician
Copy link
Copy Markdown
Collaborator

🟢 Tests passed during RECORDING mode:
TestAccDataSourceGoogleBigqueryTable_basic [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.


🟢 All tests passed!

View the build log or the debug log for each test

Copy link
Copy Markdown
Member

@c2thorn c2thorn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the contribution @ramonvermeulen, this looks good to me.
Did you find the documentation to do this sufficient? https://googlecloudplatform.github.io/magic-modules/develop/add-handwritten-datasource/

@c2thorn c2thorn added this pull request to the merge queue May 22, 2025
Merged via the queue into GoogleCloudPlatform:main with commit 2a6c3c8 May 22, 2025
29 checks passed
@ramonvermeulen
Copy link
Copy Markdown
Contributor Author

thanks for the contribution @ramonvermeulen, this looks good to me. Did you find the documentation to do this sufficient? https://googlecloudplatform.github.io/magic-modules/develop/add-handwritten-datasource/

@c2thorn

Thanks for your review!

Yes, I think in general the mmv1 documentation is in quite a good shape. Maybe if it needs to improved, at point 2 I would add something related to the existence of tpgresource.DatasourceSchemaFromResourceSchema. For a lot of data sources I think this is really useful, unless you want a different Schema than the original resource.

NandiniAgrawal15 pushed a commit to NandiniAgrawal15/magic-modules that referenced this pull request Jun 5, 2025
BBBmau pushed a commit to BBBmau/magic-modules that referenced this pull request Jun 25, 2025
BBBmau pushed a commit to BBBmau/magic-modules that referenced this pull request Jun 25, 2025
jingqizz pushed a commit to jingqizz/magic-modules that referenced this pull request Jul 9, 2025
BBBmau pushed a commit to BBBmau/magic-modules that referenced this pull request Jul 28, 2025
BBBmau pushed a commit to BBBmau/magic-modules that referenced this pull request Jul 28, 2025
BBBmau pushed a commit to BBBmau/magic-modules that referenced this pull request Jul 28, 2025
BBBmau pushed a commit to BBBmau/magic-modules that referenced this pull request Jul 28, 2025
NandiniAgrawal15 pushed a commit to NandiniAgrawal15/magic-modules that referenced this pull request Sep 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enrich data.google_bigquery_tables to include table schemas

3 participants