Skip to content
This repository is currently being migrated. It's locked while the migration is in progress.

Design system content updates#4

Closed
emilywaggoner wants to merge 1 commit intodepartment-of-veterans-affairs:masterfrom
emilywaggoner:patch-1
Closed

Design system content updates#4
emilywaggoner wants to merge 1 commit intodepartment-of-veterans-affairs:masterfrom
emilywaggoner:patch-1

Conversation

@emilywaggoner
Copy link
Copy Markdown
Contributor

@emilywaggoner emilywaggoner commented Nov 1, 2018

Adding context to components: live examples, research, design decisions

@jaredcunha-usds
Copy link
Copy Markdown
Contributor

jaredcunha-usds commented Nov 2, 2018

Would we if we'd need to show the components in the wild, especially on components where someone would need to authenticate or perform some interaction. I think showing them in context is nice, but I'm wondering about how we address these instance, especially at scale.

  • What happens when a component is removed from the page? How would we keep on top of this?
  • What if a component example does not comport to usage guidelines (or gets modified in a way that doesn't)
  • What if the components are specific to authenticated users?

@emilywaggoner
Copy link
Copy Markdown
Contributor Author

emilywaggoner commented Nov 2, 2018

Yes I think this kind of context would require curation unless there is a way to pull back in live examples from the wild. Alternatively, we could use screenshots or animated gifs like the CMS design system does to demonstrate its responsiveness: https://design.cms.gov/guidelines/responsive/

@jaredcunha-usds
Copy link
Copy Markdown
Contributor

jaredcunha-usds commented Nov 2, 2018

Would it be helpful to show the component with something other than dummy text?

@jaredcunha-usds
Copy link
Copy Markdown
Contributor

jaredcunha-usds commented Dec 17, 2018

I'm going to close. We had something that came up that sort of came that leads me to think we should avoid real-world examples.

We had a debate on which alert boxes are expandable. The way the alerts are coded, any box can be expandable. There's just an additional script tag that needs to be included. However, some folks thought that perhaps only a certain type (warning) was expandable because that was the answer given. I'd prefer keeping examples generic because if we show too many examples with content, they might indirectly appear to establish rules that may not exist.

@kevwalsh kevwalsh mentioned this pull request May 25, 2022
Closed
@humancompanion-usds humancompanion-usds mentioned this pull request Feb 2, 2023
Closed
6 tasks
humancompanion-usds added a commit that referenced this pull request Jan 9, 2026
@humancompanion-usds @claude
docs: add comprehensive documentation for pattern adherence script
8fa03c4 
Addresses Copilot review comment #4 from PR #5447.

Adds detailed documentation to scripts/README.md covering:

**Overview**:
- Script purpose and usage
- CLI options (--vets-website-path)
- Performance comparison (local vs GitHub API)

**Technical Details**:
- Pattern discovery and detection strategies
- Pattern export mapping configuration
- Regex pattern matching for various naming conventions
- Special case handling (Signature pattern)
- Path extraction with leading slash support

**Operations**:
- How the script works (step-by-step)
- Output file formats (JSON + markdown)
- GitHub Actions workflow integration
- Sparse checkout optimization

**Maintenance**:
- Troubleshooting common issues
- Known limitations
- Future enhancement ideas
- Related documentation links

This provides future maintainers with complete context for
understanding and modifying the pattern adherence tracking system.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@humancompanion-usds humancompanion-usds mentioned this pull request Jan 9, 2026
Merged
humancompanion-usds added a commit that referenced this pull request Feb 11, 2026
@humancompanion-usds @claude
Automated Pattern Adherence Tracking System (#5447)
99335f3 
* feat: add pattern metadata parser

- Parse frontmatter from pattern markdown files
- Extract code-link URLs and file paths
- Add test for pattern parser

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: improve error handling in pattern parser

- Add try-catch for individual file read failures
- Validate code-link URLs before parsing
- Warn on malformed GitHub URLs
- Check patterns directory exists with helpful error

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat: add pattern file discovery

- Recursively scan pattern directories
- Filter out non-pattern files
- Report pattern statistics

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat: add product directory integration

- Fetch product-directory.json via gh CLI
- Filter to forms by analytics_category
- Extract relevant form metadata

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: improve robustness of product directory integration

- Add array validation before filtering products
- Make test assertion work for both category fields
- Add timeout to gh CLI command

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat: add vets-website import analysis

- Search GitHub code API for pattern imports
- Extract application names from file paths
- Handle rate limit and search failures gracefully

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: address security vulnerability and improve robustness in import analysis

- Fix command injection vulnerability with proper shell escaping
- Use path.parse() for robust file extension removal
- Add input validation for pattern code file parameter
- Add JSON parsing error handling
- Add timeout to prevent hanging

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat: build pattern-to-forms mapping

- Cross-reference importing apps with form products
- Calculate usage counts and compliance percentages
- Generate structured adherence data

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: improve robustness and performance of pattern mapping

- Add null check for pattern codeFile before analysis
- Parallelize pattern processing with Promise.all
- Add division-by-zero protection for compliance percentage
- Add JSDoc documentation for return type

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat: add report generation

- Generate JSON data files for consumption
- Create markdown report with summary and matrix
- Add main execution function
- Sort patterns by usage for readability

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* perf: optimize report generation

- Remove redundant pattern lookup in detailed section
- Use lookup map for O(1) form-pattern matrix checks
- Improve performance for large datasets

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* chore: add initial pattern adherence analysis results

- Generated pattern adherence JSON data
- Created markdown report with compliance summary
- Added implementation plan documentation

Analysis shows 9 codified patterns across 55 forms:
- Dates: 27% compliance (15 forms)
- Addresses: 20% compliance (11 forms)
- Names: 20% compliance (11 forms)
- Phone numbers: 0% compliance (needs investigation)

* feat: improve pattern detection and add local vets-website support

Improves pattern adherence detection accuracy and adds support for
analyzing a local vets-website repository to avoid GitHub API rate limits.

Key changes:
- Add --vets-website-path CLI flag to use local repo instead of GitHub API
- Add PATTERN_EXPORT_MAPPING to identify pattern-specific exports
- Rewrite findImporters() to detect barrel export patterns correctly
- Fix regex to match combined suffixes (e.g., ssnOrVaFileNumberNoHintUI)
- Add findJSFiles() helper for recursive local file discovery
- Support both local filesystem and GitHub API modes

This fixes the issue where patterns using combined suffixes like
"NoHintUI" or "NoHintSchema" were not being detected. Form 20-10206
now correctly shows SSN pattern usage.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* chore: update pattern adherence reports with accurate detection

Updates reports with complete pattern detection results using the
improved barrel export detection logic.

Key improvements over previous reports:
- Email: 0% → 42% (23 forms)
- Phone: 0% → 40% (22 forms)
- Names: 0% → 35% (19 forms)
- SSN: 0% → 35% (19 forms, including form 20-10206 ✓)
- Dates: 13% → 38% (21 forms)
- Relationship: 0% → 20% (11 forms)

Form 20-10206 now correctly shows SSN pattern usage, confirming
the regex fix for combined suffixes (NoHintUI, NoHintSchema) is
working as expected.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat: add special detection for Signature pattern (statementOfTruth)

Adds special handling for the Signature pattern which uses declarative
configuration rather than direct imports.

The Signature pattern (FormSignature.jsx) is unique:
- Located in components/ not web-component-patterns/
- Forms don't import it directly
- Forms use declarative `statementOfTruth:` config in preSubmitInfo
- Processed automatically by forms-system SubmitController

Detection approach:
- Search config/form.js files for `statementOfTruth` configuration
- Extract application name from file path
- Currently only works with local vets-website (--vets-website-path)

Results:
- Signature pattern: 0% → 25% (14 forms)
- Form 20-10206 now correctly shows Signature usage ✓
- Checked 92 form config files, found 30 matches across 8 apps

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat: add GitHub Actions workflow for weekly pattern adherence tracking

Adds automated weekly tracking of pattern adherence across VA.gov forms.

Workflow features:
- Runs weekly on Mondays at 6 AM UTC
- Uses sparse checkout (src/applications only) to optimize performance
- Analyzes 9 codified patterns across 55 forms
- Automatically creates PRs when data changes
- Includes manual workflow_dispatch trigger for testing

Technical approach:
- Sparse checkout of vets-website (only src/applications, ~200MB vs ~500MB)
- Uses --vets-website-path flag for local filesystem analysis
- Supports Signature pattern detection via statementOfTruth config
- Avoids GitHub API rate limits by using local files

Automation:
- Creates dated branch (pattern-adherence-update-YYYY-MM-DD)
- Commits JSON and markdown reports
- Opens PR with compliance summary
- Includes test plan checklist

This enables automated tracking of design system pattern adoption
without manual intervention.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: handle inconsistent leading slashes in path extraction

Addresses Copilot review comments #2 and #3 from PR #5447.

Issue:
Product directory data has inconsistent path formatting - some paths
have leading slashes ("/src/applications/dependents") while others
don't ("src/applications/caregivers"). This caused regex matching
failures when extracting application names.

Fix:
Updated all three path extraction points to handle optional leading slash:
- Line 305: findSignaturePattern() app extraction
- Line 457: findImporters() app extraction
- Line 513: buildPatternAdherence() app matching

Changed regex from:
  /^([^\/]+)/  or  /^src\/applications\//
to:
  /^\/?([^\/]+)/  and  /^\/?src\/applications\//

This ensures both formats are correctly processed:
  "simple-forms/..." -> "simple-forms" ✓
  "/simple-forms/..." -> "simple-forms" ✓
  "src/applications/caregivers" -> "caregivers" ✓
  "/src/applications/dependents" -> "dependents" ✓

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* docs: add comprehensive documentation for pattern adherence script

Addresses Copilot review comment #4 from PR #5447.

Adds detailed documentation to scripts/README.md covering:

**Overview**:
- Script purpose and usage
- CLI options (--vets-website-path)
- Performance comparison (local vs GitHub API)

**Technical Details**:
- Pattern discovery and detection strategies
- Pattern export mapping configuration
- Regex pattern matching for various naming conventions
- Special case handling (Signature pattern)
- Path extraction with leading slash support

**Operations**:
- How the script works (step-by-step)
- Output file formats (JSON + markdown)
- GitHub Actions workflow integration
- Sparse checkout optimization

**Maintenance**:
- Troubleshooting common issues
- Known limitations
- Future enhancement ideas
- Related documentation links

This provides future maintainers with complete context for
understanding and modifying the pattern adherence tracking system.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* security: prevent shell injection in GitHub API file fetching

Addresses security review comment from PR #5447.

Issue:
Line 434-437 used execSync with string interpolation, allowing
potential shell interpretation of special characters in filePath:
  execSync(`gh api repos/.../contents/.../\${filePath} --jq ...`)

If filePath contained shell metacharacters (e.g., spaces, quotes,
semicolons), they could be interpreted by the shell, creating a
potential command injection vector.

Fix:
Replace execSync with execFileSync and pass arguments as array:
  execFileSync('gh', ['api', 'repos/.../\${filePath}', '--jq', ...])

This prevents shell interpretation of filePath - it's passed directly
to the gh command as a single argument, not parsed by the shell.

Defense in depth:
While filePath comes from GitHub's Git Tree API (trusted source),
using execFileSync follows security best practices and prevents
any potential issues if the source changes.

Note: This code path is rarely used - the recommended approach is
--vets-website-path which uses local filesystem (no shell commands).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: correct checkout parameter in workflow (depth -> fetch-depth)

Fixes workflow failure in PR #5447.

Error:
  Unexpected input(s) 'depth', valid inputs are [..., fetch-depth, ...]

Issue:
The actions/checkout@v4 action uses 'fetch-depth' not 'depth' to
control shallow clone depth.

Fix:
Changed line 73 from:
  depth: 1
to:
  fetch-depth: 1

This allows the sparse checkout of vets-website to succeed, which
is required for the pattern adherence analysis to run.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: pass GITHUB_TOKEN for private repo access and use execFileSync

Fixes workflow error: "gh: Not Found (HTTP 404)" when fetching
product directory.

Issues:
1. product-directory repo is private (internal visibility)
2. execSync calls didn't pass GITHUB_TOKEN environment variable
3. Remaining execSync calls need security hardening

Changes:
1. Updated fetchProductDirectory() to pass GITHUB_TOKEN
2. Updated findImporters() GitHub API calls to pass GITHUB_TOKEN
3. Converted all remaining execSync to execFileSync for security

Why GITHUB_TOKEN is needed:
The product-directory repository is private with internal visibility.
Without authentication, gh CLI returns 404. The workflow sets
GITHUB_TOKEN=${{ secrets.VADS_WORKFLOWS }} which needs to be
explicitly passed to child processes via env option.

Security improvements:
All gh CLI invocations now use execFileSync instead of execSync:
- fetchProductDirectory (line 193)
- findImporters - get default branch (line 400)
- findImporters - get git tree (line 411)
- findImporters - get file content (line 435, already fixed)

This prevents shell interpretation of any variables and ensures
GITHUB_TOKEN is consistently passed for authenticated API access.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: use GH_TOKEN for gh CLI authentication in pattern adherence workflow

The gh CLI requires GH_TOKEN environment variable for authentication,
not GITHUB_TOKEN. This fixes all 4 locations where gh api is called:
- fetchProductDirectory() - fetching from product-directory repo
- findFormFiles() - fetching vets-website repo info and file tree
- findImporters() - fetching individual file contents

This should resolve the "Not Found (HTTP 404)" error when accessing
the private product-directory repository.

Note: The VADS_WORKFLOWS PAT must have access to both
vets-design-system-documentation and product-directory repositories.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: use unique branch names to avoid collisions on multiple runs

Add GitHub run ID to branch name to prevent conflicts when the workflow
runs multiple times on the same day. This allows for reruns and testing
without manual branch cleanup.

Branch name format: pattern-adherence-update-YYYY-MM-DD-<run_id>

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: remove pull_request triggers to prevent infinite loop

Removed pull_request triggers from automated workflows that create PRs.
The branches-ignore strategy does not work as expected because it filters
the base branch, not the head branch.

Problem: When these workflows run on PR events, they:
1. Generate/update data files
2. Create a new PR with the changes
3. This triggers the workflow again on the original PR
4. Creates another PR, causing an infinite loop

Solution: Only use schedule and workflow_dispatch triggers.
- schedule: Runs automatically on a set schedule
- workflow_dispatch: Allows manual triggering for testing

This prevents the loop while maintaining automated updates and
allowing controlled testing during development.

Affected workflows:
- pattern-adherence.yml: Removed pull_request trigger
- metrics-dashboard.yml: Removed pull_request trigger

Cleaned up 6 duplicate PRs created by the loop: #5451-5454, #5456-5457
Kept PR #5449 with the actual pattern adherence data.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Update scripts/collect-pattern-adherence.js

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: address Copilot security and code quality suggestions

- Add safeExecFileSync wrapper to validate arguments and prevent command
  injection from shell metacharacters
- Add escapeRegExp helper and escape exportName before use in RegExp
  constructor to prevent regex injection
- Replace pipe-delimited composite keys with nested Map/Set structure
  to avoid collisions if product/pattern names contain pipe characters
- Extract isFormProduct() helper to reduce duplication between main
  module and tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: address Copilot review comments on pattern adherence PR

- Remove | from safeExecFileSync denylist (safe with execFileSync, needed by jq)
- Fix GitHub API mode path handling with ltrimstr to strip src/applications/ prefix
- Update Node.js version from 18 to 20 to match .nvmrc
- Set process.exitCode = 1 in test runner catch handler for CI detectability
- Replace AI-specific instructions with tool-agnostic guidance in plan doc

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address second round of Copilot review comments

- Restore pull_request trigger on metrics-dashboard workflow (was
  unintentionally removed; now scoped to relevant paths)
- Use 2-segment app identifiers (e.g., simple-forms/20-10206) instead
  of single segment to avoid collapsing distinct forms under shared
  parent directories
- Update PR description to reflect workflow is included

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: revert pull_request trigger restoration on metrics-dashboard

The pull_request trigger was intentionally removed to prevent infinite
loops when the workflow commits updated metrics data back to PR branches.
Added an inline comment documenting this decision and called it out in
the PR description.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update docs/plans/2026-01-09-pattern-adherence-tracking.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Updated run of pattern adherence

* fix: address third round of Copilot review comments

- Rename test-pattern-parser.js to integration-test-pattern-parser.js
- Restore pull_request trigger with branches-ignore on metrics-dashboard
- Remove generated_at timestamp to avoid unnecessary weekly diffs
- Fix misleading code link in report (now links to GitHub source)
- Add separate Docs link for pattern permalink in report
- Fix plan doc architecture description: HTML → JSON + markdown
- Update plan doc references to renamed integration test file

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update scripts/README.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: address fourth round of Copilot review comments

- Add Jest unit tests for parsePatternMetadata, isFormProduct, and
  generateMarkdownReport (23 tests)
- Update README form matching docs to show 2-segment identifiers
- Remove stale generated_at from README JSON example
- Add token for vets-website checkout in pattern-adherence workflow
- Update PR description with current compliance data

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address fifth round of Copilot review comments

- Regenerate report and JSON data with fixed code (GitHub code links,
  no generated_at timestamp, separate Docs links)
- Update README form matching pseudo-code to show normalization
- Fix metrics-dashboard comment to match actual branches-ignore behavior

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@emilywaggoner @jaredcunha-usds