Skip to content

[curriculum-eval] curriculum-scorecard: overall page scores and trend analysis #2372

Closed as not planned
Closed as not planned
[curriculum-eval] curriculum-scorecard: overall page scores and trend analysis#2372
Labels
curriculumdocumentationImprovements or additions to documentationquality

Description

@github-actions
github-actions
bot
opened on Jul 30, 2026

Part Summary

Part Files Mean Score Std Dev
Part 1 — core path (lessons 00–14) 53 6.13 / 10.0 ±0.47
Part 2 — advanced (lessons 15+) 32 6.06 / 10.0 ±0.34
Overall corpus 86 6.10 / 10.0 ±0.43

Part 1 Scores (lessons 00–14)

File Overall Score
workshop/07d-confirm-model-access.md 5.29 / 10.0
workshop/side-quest-01-02-environment-reference.md 5.31 / 10.0
workshop/04-github-actions-intro.md 5.42 / 10.0
workshop/side-quest-11-06-anthropic-key.md 5.51 / 10.0
workshop/side-quest-05-03-two-file-structure.md 5.55 / 10.0
workshop/05-agentic-workflows-intro.md 5.57 / 10.0
workshop/side-quest-11-08-frontmatter-tools-outputs.md 5.64 / 10.0
workshop/side-quest-11-01b-workflow-structure.md 5.65 / 10.0
workshop/08-run-your-workflow.md 5.67 / 10.0
workshop/14b-pr-reviewer-workflow.md 5.69 / 10.0
workshop/side-quest-11-09-agent-session-phases.md 5.71 / 10.0
workshop/side-quest-13-04-token-optimization.md 5.71 / 10.0
workshop/side-quest-11-04-annotated-workflow.md 5.72 / 10.0
workshop/side-quest-06-03a-copilot-requests-permission.md 5.79 / 10.0
workshop/05b-agentic-workflows-security.md 5.83 / 10.0
workshop/side-quest-05-01-actions-power-user.md 5.87 / 10.0
workshop/side-quest-06-03-copilot-token.md 5.87 / 10.0
workshop/side-quest-05-02-aw-deep-dive.md 5.89 / 10.0
workshop/side-quest-10-02-jailbreak-brief.md 5.89 / 10.0
workshop/side-quest-11-05-event-triggers.md 5.89 / 10.0
workshop/side-quest-13-01-schedule-expressions.md 5.91 / 10.0
workshop/14-next-steps.md 5.93 / 10.0
workshop/side-quest-12-01-iterate-agent-output.md 5.93 / 10.0
workshop/side-quest-11-07-openai-key.md 5.95 / 10.0
workshop/side-quest-09-01f-debugging-checklist.md 6.01 / 10.0
workshop/09-agentic-editing.md 6.03 / 10.0
workshop/side-quest-08-01-codespaces-actions-write.md 6.05 / 10.0
workshop/08b-interpret-your-run.md 6.07 / 10.0
workshop/02a-setup-codespace.md 6.09 / 10.0
workshop/side-quest-10-01-agent-brief.md 6.11 / 10.0
workshop/side-quest-09-01-debug-output.md 6.15 / 10.0
workshop/side-quest-09-01a-pattern-long-plan-chain.md 6.20 / 10.0
workshop/side-quest-02-01-local-terminal.md 6.25 / 10.0
workshop/side-quest-01-01-terminal-basics.md 6.26 / 10.0
workshop/side-quest-09-01c-pattern-safe-output-blocked.md 6.35 / 10.0
workshop/side-quest-09-01d-pattern-permission-denied.md 6.37 / 10.0
workshop/side-quest-13-02-pr-summary-pattern.md 6.37 / 10.0
workshop/side-quest-09-01b-pattern-empty-results.md 6.39 / 10.0
workshop/side-quest-06-01-install-troubleshooting.md 6.43 / 10.0
workshop/side-quest-09-01e-pattern-done-no-write.md 6.43 / 10.0
workshop/side-quest-13-03-pr-checklist-pattern.md 6.47 / 10.0
workshop/side-quest-07-01-compile-workflow.md 6.51 / 10.0
workshop/side-quest-06-03c-copilot-github-token-ui-only.md 6.55 / 10.0
workshop/05c-agentic-workflows-practice.md 6.59 / 10.0
workshop/07-your-first-workflow.md 6.59 / 10.0
workshop/side-quest-11-03-better-prompts.md 6.61 / 10.0
workshop/side-quest-13-01-pr-labeler-pattern.md 6.67 / 10.0
workshop/side-quest-01-03-permission-errors.md 6.79 / 10.0
workshop/side-quest-06-03b-copilot-github-token.md 6.81 / 10.0
workshop/side-quest-06-04-install-local.md 6.83 / 10.0
workshop/side-quest-11-01-frontmatter-deep-dive.md 7.09 / 10.0
workshop/side-quest-06-02-cca-codespace.md 7.25 / 10.0
workshop/side-quest-11-02-yaml-frontmatter.md 7.25 / 10.0

Part 2 Scores (lessons 15+)

File Overall Score
workshop/side-quest-17-07-repo-poisoning.md 5.47 / 10.0
workshop/15-conditional-logic.md 5.53 / 10.0
workshop/side-quest-20-01-memory-patterns.md 5.59 / 10.0
workshop/16-connect-data-source.md 5.71 / 10.0
workshop/17-add-mcp-tools.md 5.75 / 10.0
workshop/side-quest-16-02-secrets-and-permissions.md 5.79 / 10.0
workshop/20-persistent-memory.md 5.81 / 10.0
workshop/side-quest-17-02-security-architecture.md 5.81 / 10.0
workshop/side-quest-17-03-prompt-injection.md 5.81 / 10.0
workshop/side-quest-16-03-token-exfiltration.md 5.85 / 10.0
workshop/26-manage-costs-and-budgets.md 5.87 / 10.0
workshop/side-quest-17-01-mcp-concepts.md 5.91 / 10.0
workshop/side-quest-17-04-permission-escalation.md 5.93 / 10.0
workshop/side-quest-17-06-output-injection.md 5.97 / 10.0
workshop/21-inline-sub-agents.md 6.01 / 10.0
workshop/side-quest-16-05-long-lived-credentials.md 6.01 / 10.0
workshop/side-quest-17-05-supply-chain-mcp.md 6.01 / 10.0
workshop/side-quest-25-01-audit-reference.md 6.01 / 10.0
workshop/side-quest-16-04-deterministic-vs-agentic-data-ops.md 6.03 / 10.0
workshop/side-quest-15-01-expressions-and-contexts.md 6.05 / 10.0
workshop/18-share-and-reuse.md 6.15 / 10.0
workshop/side-quest-24-01-runner-infrastructure.md 6.19 / 10.0
workshop/side-quest-15-02-chaining-conditions.md 6.25 / 10.0
workshop/22-error-handling-and-resilience.md 6.27 / 10.0
workshop/25-audit-and-observability.md 6.27 / 10.0
workshop/19-research-driven-training-node.md 6.37 / 10.0
workshop/27-evaluate-workflow-quality.md 6.37 / 10.0
workshop/side-quest-16-01-github-output.md 6.48 / 10.0
workshop/23-ab-experiments.md 6.55 / 10.0
workshop/side-quest-26-01-forecast-costs.md 6.55 / 10.0
workshop/24-self-hosted-runners.md 6.65 / 10.0
workshop/side-quest-21-01-sub-agent-syntax.md 6.99 / 10.0

Other Scores (no lesson number)

File Overall Score
workshop/side-quest-enterprise-setup.md 5.77 / 10.0

Score Trends (history window: 20 commits)

File Baseline Latest Delta Direction
workshop/02a-setup-codespace.md 8.84 6.09 -2.75 declining
workshop/04-github-actions-intro.md 8.12 5.38 -2.74 declining
workshop/05-agentic-workflows-intro.md 8.31 5.61 -2.70 declining
workshop/05b-agentic-workflows-security.md 8.60 5.83 -2.77 declining
workshop/05c-agentic-workflows-practice.md 9.08 6.59 -2.49 declining
workshop/07-your-first-workflow.md 9.34 6.57 -2.77 declining
workshop/07d-confirm-model-access.md 8.06 5.29 -2.77 declining
workshop/08-run-your-workflow.md 8.44 5.67 -2.77 declining
workshop/08b-interpret-your-run.md 8.82 6.07 -2.75 declining
workshop/09-agentic-editing.md 8.66 6.07 -2.59 declining
workshop/14-next-steps.md 8.70 5.93 -2.77 declining
workshop/14b-pr-reviewer-workflow.md 8.48 5.71 -2.77 declining
workshop/15-conditional-logic.md 8.30 5.53 -2.77 declining
workshop/16-connect-data-source.md 8.48 5.71 -2.77 declining
workshop/17-add-mcp-tools.md 8.52 5.75 -2.77 declining
workshop/18-share-and-reuse.md 8.90 6.15 -2.75 declining
workshop/19-research-driven-training-node.md 9.14 6.37 -2.77 declining
workshop/20-persistent-memory.md 8.58 5.81 -2.77 declining
workshop/21-inline-sub-agents.md 8.78 6.01 -2.77 declining
workshop/22-error-handling-and-resilience.md 9.14 6.27 -2.87 declining
workshop/23-ab-experiments.md 9.32 6.55 -2.77 declining
workshop/24-self-hosted-runners.md 8.85 6.65 -2.20 declining
workshop/25-audit-and-observability.md 9.04 6.27 -2.77 declining
workshop/26-manage-costs-and-budgets.md 8.62 5.87 -2.75 declining
workshop/27-evaluate-workflow-quality.md 9.12 6.37 -2.75 declining
workshop/side-quest-01-01-terminal-basics.md 9.00 6.26 -2.74 declining
workshop/side-quest-01-02-environment-reference.md 8.06 5.31 -2.75 declining
workshop/side-quest-01-03-permission-errors.md 9.56 6.79 -2.77 declining
workshop/side-quest-02-01-local-terminal.md 9.04 6.25 -2.79 declining
workshop/side-quest-05-01-actions-power-user.md 8.64 5.87 -2.77 declining
workshop/side-quest-05-02-aw-deep-dive.md 7.74 5.89 -1.85 declining
workshop/side-quest-06-01-install-troubleshooting.md 8.43 6.43 -2.00 declining
workshop/side-quest-06-02-cca-codespace.md 9.25 7.25 -2.00 declining
workshop/side-quest-06-03-copilot-token.md 8.62 5.87 -2.75 declining
workshop/side-quest-06-03a-copilot-requests-permission.md 8.54 5.79 -2.75 declining
workshop/side-quest-06-03b-copilot-github-token.md 9.58 6.81 -2.77 declining
workshop/side-quest-06-03c-copilot-github-token-ui-only.md 9.32 6.55 -2.77 declining
workshop/side-quest-06-04-install-local.md 9.60 6.83 -2.77 declining
workshop/side-quest-07-01-compile-workflow.md 8.53 6.51 -2.02 declining
workshop/side-quest-08-01-codespaces-actions-write.md 8.80 6.05 -2.75 declining
workshop/side-quest-09-01-debug-output.md 8.92 6.15 -2.77 declining
workshop/side-quest-09-01a-pattern-long-plan-chain.md 8.20 6.20 -2.00 declining
workshop/side-quest-09-01b-pattern-empty-results.md 8.41 6.39 -2.02 declining
workshop/side-quest-09-01c-pattern-safe-output-blocked.md 8.37 6.35 -2.02 declining
workshop/side-quest-09-01d-pattern-permission-denied.md 8.39 6.37 -2.02 declining
workshop/side-quest-09-01e-pattern-done-no-write.md 8.45 6.43 -2.02 declining
workshop/side-quest-09-01f-debugging-checklist.md 8.03 6.01 -2.02 declining
workshop/side-quest-10-01-agent-brief.md 8.86 6.11 -2.75 declining
workshop/side-quest-10-02-jailbreak-brief.md 8.66 5.89 -2.77 declining
workshop/side-quest-11-01-frontmatter-deep-dive.md 9.84 7.09 -2.75 declining
workshop/side-quest-11-01b-workflow-structure.md 8.42 5.65 -2.77 declining
workshop/side-quest-11-02-yaml-frontmatter.md 8.75 7.25 -1.50 declining
workshop/side-quest-11-03-better-prompts.md 9.38 6.61 -2.77 declining
workshop/side-quest-11-04-annotated-workflow.md 8.50 5.72 -2.78 declining
workshop/side-quest-11-05-event-triggers.md 8.66 5.89 -2.77 declining
workshop/side-quest-11-06-anthropic-key.md 8.26 5.51 -2.75 declining
workshop/side-quest-11-07-openai-key.md 8.70 5.95 -2.75 declining
workshop/side-quest-11-08-frontmatter-tools-outputs.md 8.41 5.64 -2.77 declining
workshop/side-quest-11-09-agent-session-phases.md 8.46 5.71 -2.75 declining
workshop/side-quest-12-01-iterate-agent-output.md 8.68 5.93 -2.75 declining
workshop/side-quest-13-01-pr-labeler-pattern.md 8.69 6.67 -2.02 declining
workshop/side-quest-13-01-schedule-expressions.md 8.66 5.91 -2.75 declining
workshop/side-quest-13-02-pr-summary-pattern.md 8.39 6.37 -2.02 declining
workshop/side-quest-13-03-pr-checklist-pattern.md 8.53 6.47 -2.06 declining
workshop/side-quest-13-04-token-optimization.md 8.46 5.71 -2.75 declining
workshop/side-quest-15-01-expressions-and-contexts.md 8.80 6.05 -2.75 declining
workshop/side-quest-15-02-chaining-conditions.md 9.02 6.25 -2.77 declining
workshop/side-quest-16-01-github-output.md 8.50 6.48 -2.02 declining
workshop/side-quest-16-02-secrets-and-permissions.md 8.54 5.79 -2.75 declining
workshop/side-quest-16-03-token-exfiltration.md 8.60 5.85 -2.75 declining
workshop/side-quest-16-04-deterministic-vs-agentic-data-ops.md 8.78 6.03 -2.75 declining
workshop/side-quest-16-05-long-lived-credentials.md 8.80 6.01 -2.79 declining
workshop/side-quest-17-01-mcp-concepts.md 8.64 5.91 -2.73 declining
workshop/side-quest-17-02-security-architecture.md 8.56 5.81 -2.75 declining
workshop/side-quest-17-03-prompt-injection.md 8.56 5.81 -2.75 declining
workshop/side-quest-17-04-permission-escalation.md 8.70 5.93 -2.77 declining
workshop/side-quest-17-05-supply-chain-mcp.md 8.76 6.01 -2.75 declining
workshop/side-quest-17-06-output-injection.md 8.74 5.97 -2.77 declining
workshop/side-quest-17-07-repo-poisoning.md 8.24 5.47 -2.77 declining
workshop/side-quest-20-01-memory-patterns.md 8.36 5.59 -2.77 declining
workshop/side-quest-21-01-sub-agent-syntax.md 9.76 6.99 -2.77 declining
workshop/side-quest-25-01-audit-reference.md 8.72 6.01 -2.71 declining
workshop/side-quest-26-01-forecast-costs.md 8.57 6.55 -2.02 declining
workshop/side-quest-enterprise-setup.md 8.52 5.77 -2.75 declining

Overall Trend

  • Baseline mean score: 8.70
  • Latest mean score: 6.10
  • Delta: -2.60
  • Direction: declining

Bloom's Taxonomy Distribution

Level Count
remember 5
understand 14
apply 48
analyze 7
evaluate 4
create 8

The corpus has healthy pyramid coverage: 14 understand steps, 48 apply steps, 7 analyze steps, 4 evaluate steps, and 8 create steps. Both has_min_understand_steps and has_min_analyze_steps are satisfied. No Bloom gap proposals are outstanding.

Corpus-Level Notes

  • Universal checkpoint gap (systemic): The corpus-wide score decline (−2.60 from baseline) is driven primarily by checkpoint_quality scoring 0.0 across the majority of pages — a structural shift affecting 70+ files simultaneously. The rubric awards full points only when has_checkpoint: true, which requires the canonical ## :white_check_mark: Checkpoint heading. Pages with checklist items outside that heading score 0.
  • Active learning density below target: Most pages score 2.6–5.0 on active learning (target: density ≥ 3.0). Reference and side-quest pages are most affected.
  • Scaffolding universally at 5.0: No page scores 10/10 on scaffolding. Pages have a :clipboard: Before You Start section but the rubric's has_prereq_section flag is not being set to true, suggesting either a detection heuristic mismatch or pages are missing the section entirely.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔬 Curriculum Quality Evaluator · 120.2 AIC · ⌖ 8.64 AIC · ⊞ 6.2K · ◷

  • expires on Jul 31, 2026, 1:26 PM UTC

Metadata

Metadata

Assignees

No one assigned

    Labels

    curriculumdocumentationImprovements or additions to documentationquality

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions