Skip to content

fix: update repo structure validation logic to disallow false positives#10

Merged
buenos-nachos merged 15 commits into
mainfrom
mes/module-validation
May 2, 2025
Merged

fix: update repo structure validation logic to disallow false positives#10
buenos-nachos merged 15 commits into
mainfrom
mes/module-validation

Conversation

@buenos-nachos
Copy link
Copy Markdown
Contributor

@buenos-nachos buenos-nachos commented Apr 18, 2025

Splitting up the main validation PR I've got cooking up, so that it's easier to review.

Changes made

  • Split up previous validation steps into discrete "file structure validation" and "contributor README" validation steps
    • Added checks for ensuring that only specific directories could be allowed at the top of a user namespace
  • Moved the directories for the apache-airflow module (was not previously in a /modules directory, which the new checks did catch)
  • Moved validation to follow cmd directory structure
  • Started splitting up package across more "subdomain" files
  • Beefed up extractFrontmatter into separateFrontmatter – now returns out the frontmatter, as well as the README body
  • Removed all log.Panic calls
  • (2025-04-28) Updated schema to reflect new business requirements

@buenos-nachos buenos-nachos self-assigned this Apr 18, 2025
@buenos-nachos buenos-nachos requested a review from f0ssel April 22, 2025 16:31
Copy link
Copy Markdown
Contributor

@bcpeinhardt bcpeinhardt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😎

Comment on lines +21 to +24
// separateFrontmatter attempts to separate a README file's frontmatter content
// from the main README body, returning both values in that order. It does not
// validate whether the structure of the frontmatter is valid (i.e., that it's
// structured as YAML).
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Copy Markdown
Member

@f0ssel f0ssel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Comment on lines +74 to +96
const (
// validationPhaseFileStructureValidation indicates when the entire Registry
// directory is being verified for having all files be placed in the file
// system as expected.
validationPhaseFileStructureValidation validationPhase = iota

// validationPhaseFileLoad indicates when README files are being read from
// the file system
validationPhaseFileLoad

// validationPhaseReadmeParsing indicates when a README's frontmatter is
// being parsed as YAML. This phase does not include YAML validation.
validationPhaseReadmeParsing

// validationPhaseReadmeValidation indicates when a README's frontmatter is
// being validated as proper YAML with expected keys.
validationPhaseReadmeValidation

// validationPhaseAssetCrossReference indicates when a README's frontmatter
// is having all its relative URLs be validated for whether they point to
// valid resources.
validationPhaseAssetCrossReference
)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is common, but I personally prefer to use a string type for validationPhase and forgo the iota. You then don't need to convert to string with .String() and instead can just do string(phase) when needed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think my main worry with it was that it felt like the string approach meant that the custom type couldn't have a valid/meaningful zero value. Like, if I use custom strings for the main options, it's still possible to make a zero value version of the field, even though it shouldn't exist. Iota at least makes it so that the zero value is still valid (and represents the base role)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to go ahead with the current changes, but I'll bring it up in our 1:1. If you think it's still worth switching to strings, I can make a separate PR

github: nataindata
website: https://www.nataindata.com
status: community
status: partner
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@buenos-nachos buenos-nachos merged commit 45dc925 into main May 2, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants