Skip to content

Convert STAC Doc to rst#501

Merged
smk0033 merged 11 commits intodevelopfrom
stac-rst-update
Jun 30, 2025
Merged

Convert STAC Doc to rst#501
smk0033 merged 11 commits intodevelopfrom
stac-rst-update

Conversation

@smk0033
Copy link
Contributor

@smk0033 smk0033 commented Jun 3, 2025

Converted STAC Metadata Recs document from markdown to rst. Docs site currently does not support markdown

@smk0033 smk0033 requested review from hrodmn and wildintellect June 3, 2025 19:23
@smk0033 smk0033 self-assigned this Jun 3, 2025
@smk0033 smk0033 added the documentation Improvements or additions to documentation label Jun 3, 2025
Copy link
Contributor

@hrodmn hrodmn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I say in the thread below, I am in favor of pushing all of these fields as required (with a few exceptions).

I think it would be useful to include the icesat2-boreal-v2.1 metadata as an example (printed inline) since we went through that one very carefully and implemented the UMM-compliant fields. Let's hold that one up as the standard here - printing the JSON inline and linking to that repository should give users a good starting point for a new dataset.

@smk0033
Copy link
Contributor Author

smk0033 commented Jun 11, 2025

I started looking over some of the CEOS pages and left some rough notes in a comment under this issue, but it seems like there's a bit of alignment. Some questions:

  • CEOS has the full hierarchies listed for science keywords - is this something we want to recommend or implement?
  • CEOS recommends absolute href paths. I think we already do this, so I added a general note at the top
  • CEOS recommends having created, updated, and published datetime properties at the collection level. I think UMM requires provider dates the granule level, do we also want to add for the collection? It also seems like the ICESat item has a "created_datetime" but not updated or published unless I've overlooking it

There's a couple other things in the issue comment regarding band names and thumbnail image for the item
@wildintellect @hrodmn

@smk0033 smk0033 requested review from hrodmn and wildintellect June 11, 2025 18:44
@smk0033
Copy link
Contributor Author

smk0033 commented Jun 11, 2025

I also still need to look into UMM-Var😅

@wildintellect
Copy link
Collaborator

We decided in a meeting to address CEOS differences in the MAAP Data Requirements Document 1st, then port those changes to a future update of this page.

  • CEOS has the full hierarchies listed for science keywords - is this something we want to recommend or implement?
    Not unless CMR/UMM also requires this. We'd also have to look into how implement in STAC in a sane way that doesn't overly duplicate.
* CEOS recommends absolute href paths. I think we already do this, so I added a general note at the top

That makes sense, pretty sure we need absolute paths to find things in Buckets. There's a minor technical question as to where the paths get made that way. Example: in DPS output, the paths are relative inside the job, and the STAC ingestion tool @hrodmn made will convert them to absolute.

* CEOS recommends having created, updated, and published datetime properties at the collection level. I think UMM requires provider dates the granule level, do we also want to add for the collection? It also seems like the ICESat item has a "created_datetime" but not updated or published unless I've overlooking it

how does published differ from created?
Updated - we could do that to track if edits happen to the metadata, probably something our tools would autopopulate.

@smk0033
Copy link
Contributor Author

smk0033 commented Jun 11, 2025

CMR metadata does include the full hierarchy for science keywords

@smk0033
Copy link
Contributor Author

smk0033 commented Jun 11, 2025

Also the created/published is a good question. I'm assuming it'd probably be something along the lines of a STAC metadata record being created (maybe in a dev env), and then the publish date would be when the data/metadata is made available in prod?

Edit to add: UMM-G has create and insert, create being when the granule file was created and insert being when the granule file is entered into the database

@smk0033
Copy link
Contributor Author

smk0033 commented Jun 17, 2025

That makes sense, pretty sure we need absolute paths to find things in Buckets. There's a minor technical question as to where the paths get made that way. Example: in DPS output, the paths are relative inside the job, and the STAC ingestion tool @hrodmn made will convert them to absolute.

Ah, this was the comment I was talking about during backlog today! @hrodmn

I also left a couple comments above to your response @wildintellect

@hrodmn
Copy link
Contributor

hrodmn commented Jun 18, 2025

That makes sense, pretty sure we need absolute paths to find things in Buckets. There's a minor technical question as to where the paths get made that way. Example: in DPS output, the paths are relative inside the job, and the STAC ingestion tool @hrodmn made will convert them to absolute.

The answer is that hrefs must be absolute. There will be cases (like the DPS case that @wildintellect mentioned) where absolute hrefs are not known when the metadata are initially generated, but the full path to the assets is the most important component of published STAC metadata so it has to get in there before publication!

@hrodmn hrodmn self-requested a review June 18, 2025 16:06
Co-authored-by: Henry Rodman <henry.rodman@gmail.com>
@smk0033 smk0033 merged commit 99a33b0 into develop Jun 30, 2025
@smk0033 smk0033 deleted the stac-rst-update branch June 30, 2025 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants