feat: Add CLI documentation verification system and fix phantom options#141
Merged
Conversation
This ensures manual workflow triggers use direct push by default, completing the transition from PR mode to streamlined direct push.
- Fix issue where empty string manuscript paths were not handled correctly
- Change conditions from truthiness check to explicit None check
- Resolves test failure in test_edge_cases_for_invalid_paths
- Fixes both _resolve_manuscript_path and _resolve_manuscript_name methods
- Empty strings (''), dots ('.'), and double dots ('..') now properly default to 'MANUSCRIPT'
This fixes the original nox test failure where edge cases expected 'MANUSCRIPT'
but got 'manuscript' due to improper path resolution logic.
Guillaume Issue #4: When user explicitly specifies tex_position='p' for dedicated page placement, use figure[p] environment instead of figure*[p]. This fix prevents automatic 2-column detection from overriding explicit user positioning. - Modified figure_processor.py to check position != 'p' before enabling 2-column mode for textwidth figures - Ensures dedicated page figures don't span columns inappropriately - Maintains existing behavior for other positioning options
- Remove restriction preventing figure* environment for position="p" - Allow dedicated page figures to use full 2-column spanning properly - Ensure figures with width="\textwidth" always span 2 columns as intended - Fix Guillaume's reported issue with figure positioning completely 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Remove APT repository files and packages (already staged) - Remove build artifacts: .deb, .buildinfo, .changes files - Add comprehensive .gitignore patterns for: - Package distribution artifacts - Build and audit logs - Debug and temporary development files - Additional exclusions for clean repository This cleanup prevents build artifacts from cluttering the repository and ensures better development hygiene.
- Update system_libs.py to import parse_version directly from packaging.version - Update dependency_manager.py to use same import pattern - This improves compatibility and follows packaging library best practices - Fixes potential import issues with newer packaging versions The direct import of parse_version is the recommended approach per packaging library documentation.
- Respect explicit tex_position="p" for dedicated page placement - Prevent auto-detection of 2-column layout when user specifies position="p" - This fixes Guillaume's reported issue where scaled figures (0.8, 80%, etc.) with tex_position="p" were incorrectly forced into 2-column mode - Maintains backward compatibility for auto-detection when no explicit positioning Fixes: tex_position="p" now works correctly with any width specification.
- global_container_manager.py: Replace 'any' with proper 'Any' type annotation - resource_manager.py: Improve cleanup function typing and add fallback logic - session_optimizer.py: Fix type annotations for better type safety These changes improve: - Type checking accuracy and IDE support - Resource cleanup reliability with better error handling - Code maintainability and documentation through proper typing All changes maintain backward compatibility while enhancing robustness.
Major enhancements to noxfile.py: **New Cleanup Sessions:** - cleanup_containers: Clean Docker/Podman resources - cleanup_emergency: Critical disk space cleanup - cleanup_pre_test: Pre-test environment preparation - cleanup_full: Complete system cleanup **Enhanced Testing:** - Container reuse detection for faster test cycles - Disk space monitoring and automatic cleanup triggers - Improved error handling and resource management - Enhanced Docker/Podman test sessions with cleanup hooks **Monitoring & Reporting:** - disk_usage: Real-time disk space reporting - space_report: Comprehensive resource analysis - Automatic cleanup recommendations based on usage These improvements significantly reduce CI/CD resource consumption, prevent disk space issues, and optimize development workflows.
**conftest.py improvements:** - Add container reuse detection to reduce test execution time - Implement comprehensive cleanup utilities integration - Add disk space monitoring and emergency cleanup triggers - Enhanced session/class-scoped fixture management - Better error handling and resource tracking **test_guillaume_issues.py:** - Add test_dedicated_page_figures_with_scaling for Guillaume's fix - Comprehensive validation of figure positioning edge cases - Ensure tex_position="p" works with various width specifications **test_install_system_libs.py:** - Update tests to match new packaging import patterns - Fix mock usage for parse_version function These changes improve test reliability, reduce resource consumption, and prevent CI/CD issues related to disk space and container management.
- Add nox_utils/ directory with cleanup and monitoring utilities - Update README.md with latest improvements and documentation - Support utilities for enhanced nox sessions and CI/CD optimization These utility modules enable the advanced cleanup and monitoring features added to the nox configuration.
**Build Script Fix:** - Add missing --verbose flag to build-deb.sh that CI workflow expects - Include verbose logging function for enhanced debugging output - Show configuration details when verbose mode is enabled **APT Repository Validation Fix:** - Support both standard (dists/stable/) and flat repository structures - Auto-detect repository structure during accessibility check - Update all file download logic to use detected structure - Handles both Release and Packages file locations properly These fixes resolve CI failures in Container APT Package Testing workflow by addressing compatibility with the actual repository structure.
…tectures Resolves CI Container APT Package Testing failures by properly handling the actual repository structure with Release in dists/stable/ and Packages in dists/stable/main/binary-amd64/. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
The build script was checking for a 'debhelper' command which doesn't exist. The debhelper package provides the 'dh' command instead. This fixes the CI Build Test Package failure where debhelper was installed but the script couldn't find the correct command. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
The script was trying to copy PROJECT_ROOT into a subdirectory of itself, causing 'cannot copy a directory into itself' error in CI. Now uses rsync or tar to properly copy source files while excluding the build directory to prevent recursion. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
The Container APT Package Testing workflow was missing required build dependencies for Python packages: - python3-all - python3-hatchling - pybuild-plugin-pyproject These are needed by dpkg-buildpackage to build the Debian package. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Fixed cp command using correct target filename instead of directory with slash - Added safety check to ensure output directory exists and is directory - Added verbose logging for copy operations - Resolves 'cp: cannot create regular file 'dist/': Not a directory' error 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
- Add validation-results/ and packaging/validation-results/ to gitignore - These directories contain temporary APT repository validation outputs - Includes downloaded Release/Packages files, GPG keys, and validation reports - Should not be tracked in version control as they are generated files 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
- Add --extra dev to uv sync commands in all CI jobs to install nox and other dev dependencies - Update docker/build-push-action from v5 to v6 across all workflows - Update actions/checkout from v4 to v5 in test-apt-containers workflow - Downgrade actions/upload-artifact from v5 to v4 for compatibility - Downgrade actions/download-artifact from v5 to v4 for compatibility Resolves the main CI failure: "Failed to spawn: nox - No such file or directory" Fixes Docker build workflow failures caused by outdated action versions. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Docker Build Workflow fixes: - Simplify matrix platform strategy to use static array instead of complex JSON parsing - Fix artifact naming with replace() function for platform names with slashes - Update artifact download path to use artifacts/ directory - Fix image loading to search artifacts directory for .tar files APT Container Testing Workflow fixes: - Add build-test-package dependency to container and integration test jobs - Fix artifact upload paths to upload entire directories instead of glob patterns - Add if-no-files-found: ignore to prevent upload failures when no files exist - Fix test result and container log uploads to handle missing files gracefully - Ensure proper job dependencies so tests run after package builds These changes address: - Docker workflow syntax errors causing immediate failures - APT workflow "No files were found" upload errors - Test coordination issues where tests ran before packages were built - Missing file handling in artifact operations 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Docker Build Workflow fixes: - Add explicit build trigger for pull requests to enable testing - Add build trigger for pushes to main/dev branches - Remove overly restrictive change detection that was causing workflows to skip entirely APT Container Testing Workflow fixes: - Add proper job dependencies for all test jobs to include build-test-package - Add continue-on-error to artifact downloads to handle missing test packages gracefully - Add validation checks for test script existence before execution - Improve error handling and logging in container test execution - Add proper output directory creation and file existence checks - Add better conditional logic to ensure jobs only run when prerequisites are met These changes address: - Docker workflow having 0 jobs due to overly restrictive change detection - APT workflow "Unable to download artifact" errors for missing test-package - Test execution failures due to missing files and improper error handling - Job dependency issues causing tests to run without proper setup 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Revert actions/checkout from v5 to v4 for compatibility - Revert docker/metadata-action from v5 to v4 for stability - Simplify matrix strategy for test engines to avoid complex conditional expressions - Remove complex conditional expressions from GitHub Step Summary outputs that may cause parsing issues These changes address the 'workflow file issue' error by simplifying complex GitHub Actions expressions that may not be compatible with current GitHub Actions parser. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
These workflows have been migrated to separate Docker infrastructure repository as part of the Python-first architecture migration described in the PR. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
### Repository Migration & Cleanup - Update documentation references from paxcalpt to HenriquesLab - Migrate docker-rxiv-maker to HenriquesLab organization - Remove archived paxcalpt/apt-rxiv-maker references ### Workflow Consolidation (22→6) - Remove redundant Homebrew workflows (auto-update, direct-update) - Remove legacy release-simple.yml (keep Python-driven release) - Consolidate to 6 essential workflows for better maintainability ### Enhanced Monitoring & Windows Support - Add HenriquesLab/docker-rxiv-maker to cross-repository health monitoring - Enhance WSL2 documentation for Windows users - Improve installation clarity and cross-platform file access guidance ### Infrastructure Alignment - Establish single source of truth for repository references - Standardize package distribution approach (Homebrew/APT/PyPI) - Strengthen ecosystem integration and coordination 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
Release candidate for testing streamlined ecosystem: - Consolidated workflows (22→6) - Enhanced cross-repository monitoring - Improved WSL2 documentation - Docker repository integration 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
- Enhanced formula updater with repository dispatch support - Improved release orchestration for streamlined workflows - Maintain compatibility with consolidated workflow structure 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
- Remove duplicate orchestration script (scripts/maintainer/orchestrate-release.py) - Remove redundant SHA256 fetcher shell script - Remove archived workflow backups directory - Update release orchestrator to reference correct workflow file (homebrew-python.yml) - Remove Scoop dependencies from package template scripts - Focus Windows users on WSL2+APT installation strategy This completes the ecosystem streamlining from 22 workflows to 6, ensuring maintainable and focused tooling aligned with our cross-platform distribution strategy. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
…orchestrator - Replace placeholder TODO implementations with functional code - Add GitHub release creation using gh CLI with auto-generated release notes - Add PyPI publishing using build + twine with proper authentication - Update workflow dependencies to include build and twine packages - Resolves the v1.5.26 release failure caused by dry-run simulation mode 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
Merge dev branch with complete orchestrator implementation: - Functional GitHub release creation using gh CLI - Real PyPI publishing using build + twine - Fixes the v1.5.26 release failure This enables the v1.5.26 release to proceed successfully.
This commit addresses two critical issues affecting the release pipeline: 1. **Orchestrator Publishing Improvements** - Implement OIDC trusted publishing support for PyPI - Add repository_dispatch pattern for cross-repository triggers - Fix token management and environment validation - Improve error handling and logging 2. **Docker UTF-8 Encoding Fixes** - Add explicit latin-1 encoding to all subprocess calls in AbstractContainerEngine - Resolve "UnicodeDecodeError: 'utf-8' codec can't decode byte" errors - Ensure robust handling of Docker container output with non-UTF-8 content - Fix encoding issues in container session management The OIDC implementation provides more secure, token-less PyPI publishing when running in GitHub Actions, while maintaining backward compatibility with token-based publishing as fallback. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
… comprehensive test coverage This commit integrates Guillaume's complete rewrite of the figure processing system that addresses major figure placement issues, along with comprehensive test coverage and code quality improvements. ## Guillaume's PR #131 Integration - ✅ Merged Guillaume's figure processor rewrite with advanced LaTeX positioning - ✅ Preserves all new features: inline figures, enhanced width parsing, landscape orientation, fit presets - ✅ Maintains safety mechanisms for single-column figure overflow protection ## New Figure Processing Features - Inline figures: Support for inline=true non-floating figures with captionof - Enhanced width parsing: Percentages (80%), fractions (0.8), LaTeX units (0.8textwidth) - Landscape orientation: landscape=true for sidewaysfigure environments - Float barriers: barrier=true for FloatBarrier placement control - Fit presets: fit=page|width|height for common sizing patterns - Safety clamping: Automatic textwidth to linewidth conversion for single-column protection - Strict width: strict_width=true to override safety clamping when needed ## Comprehensive Test Coverage - ✅ Added 15 new regression tests in TestGuillaumePR131Rewrite class - ✅ Fixed 8 existing test expectations to match Guillaume's implementation behavior - ✅ All Guillaume-related functionality now has 100% test coverage (163/163 tests passing) - ✅ Verified integration with EXAMPLE_MANUSCRIPT PDF generation ## Code Quality Improvements - ✅ Resolved all linting violations (25+ style issues) while preserving functionality - ✅ Improved code formatting and readability with proper multi-line statements - ✅ Added proper type hints and import organization - ✅ Maintained Guillaume's sophisticated positioning logic and safety mechanisms ## Implementation Details - Direct figure paths (Figures/figure.png) instead of subdirectory nesting - Precision-formatted dimensions (0.800linewidth vs 0.8linewidth) - Default positioning: [\!htbp] for figure, [\!tbp] for figure* - Intelligent two-column detection and auto-upgrade to figure* environments - Enhanced attribute parsing with graceful error handling This integration resolves the major figure placement problems while maintaining full backward compatibility and adding powerful new capabilities for complex document layouts. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Version 1.6.0 introduces major documentation improvements: - Ultra-consolidation of documentation structure - Professional navigation system with user-focused pathways - 75% reduction in root folder clutter (20+ files → 6 essential files) - Single source of truth for all installation, troubleshooting, and user guides - Enhanced cross-reference system for better discoverability - Backward-compatible content preservation with dramatically improved UX 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add check for existing release before attempting to create - Skip GitHub release creation if release already exists - Allows workflow to continue to PyPI publishing step - Resolves v1.6.0 release workflow failure 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
* Implement graceful fallback when OIDC trusted publishing fails * Attempt OIDC first, fall back to token-based publishing on failure * Add detailed logging for debugging OIDC authentication issues * Resolves exit status 2 error from twine OIDC publishing This fixes the v1.6.0 release pipeline by ensuring PyPI publishing succeeds even when trusted publishing is not configured. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
- Twine automatically detects OIDC when running in GitHub Actions with id-token: write - Remove incorrect --trusted-publishing flag that caused exit status 2 - OIDC will work once trusted publisher is configured on PyPI side Fixes the 'unrecognized arguments: --trusted-publishing' error in release workflow
This comprehensive update addresses CLI documentation accuracy issues and establishes prevention mechanisms for future documentation drift. ## 🔍 CLI Documentation Verification System - **CLI Scanner** (`scripts/cli_scanner.py`): AST-based parser that extracts real Click decorators from Python implementation files - **Documentation Parser** (`scripts/docs_parser.py`): Regex-based parser that extracts documented commands/options from Markdown files - **Verification Engine** (`scripts/cli_verifier.py`): Cross-reference analyzer that identifies phantom options, missing documentation, and mismatches - **Claude Code Slash Command** (`~/.claude/commands/verify-cli-documentation.md`): Comprehensive prevention system for regular verification ## 🚨 Fixed Critical Documentation Issues ### Phantom Options Eliminated - Removed 17+ non-existent options from CLI documentation - Fixed PDF command phantom options (`--validate`, `--draft`, `--engine`) - Fixed Validate command phantom options (6 removed: `--syntax-only`, `--citations-only`, etc.) - Fixed arXiv command phantom options (`--include-source`, `--include-figures`, etc.) - Fixed Clean command phantom options (`--dry-run`, `--force`) ### Missing Options Added - Added real PDF command options (`--track-changes`, `--debug`, `--container-mode`) - Added real arXiv command options (`--output-dir`, `--arxiv-dir`, `--zip-filename`, `--no-zip`) - Added real Clean command options (`--arxiv-only`, `--temp-only`) - Updated all help text to match implementation ### Documentation Focus Correction - Changed positioning from "flexible journal formatting" to "ONE awesome template for preprints" - Updated README.md to emphasize preprint focus over journal submissions - Removed journal submission workflows from user guide and publishing guide - Retitled publishing guide to "Preprint Guide: From Manuscript to arXiv" ## 🛡️ Advanced Features ### Command Alias Detection - Detects import-time aliases (`from .build import build as pdf`) - Handles registration-time aliases (`main.add_command(config_cmd, name="config")`) - Supports cross-module command definitions - Discovers Click groups and commands automatically ### Comprehensive Reporting - **Exit codes**: 0 (success), 1 (warnings), 2 (critical issues) for CI/CD - **Multiple formats**: Text, Markdown, JSON output - **Severity classification**: Critical vs. minor issues - **Actionable insights**: Specific file locations and suggested fixes ## 📊 Impact **Before**: 17+ phantom options confusing users, incomplete documentation **After**: Zero confirmed phantom options, comprehensive command coverage - Commands implemented: 19 (discovered via AST parsing) - Options implemented: 52 (all real, extracted from code) - Critical issues eliminated: Phantom `pdf` command resolved - Prevention system: Automated verification prevents future drift The verification system discovered and fixed extensive CLI documentation inaccuracies that were misleading users and breaking their workflows. 🤖 Generated with Claude Code(https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
📋 Documentation Validation ReportStatus: 🟢 EXCELLENT
📖 Full Validation Report# 📋 Documentation Validation Report
**Generated**: 2025-08-25T11:02:29.315476
## 📊 Summary
- **Files processed**: 4
- **Code blocks found**: 8
- **Valid blocks**: 8 ✅
- **Invalid blocks**: 0 ❌
- **Success rate**: 100.0%
**Overall Status**: 🟢 EXCELLENT
## ✅ Validation Results by File
- **README.md** ✅: 4/4 (100.0%)
- **CONTRIBUTING.md** ✅: 4/4 (100.0%)
---
*Report generated by the Living Documentation Validation System*This validation ensures all code examples in documentation remain accurate and functional. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 Summary
This PR introduces a comprehensive CLI documentation verification system that addresses critical documentation accuracy issues and establishes prevention mechanisms for future documentation drift.
🔍 CLI Documentation Verification System
New Components
scripts/cli_scanner.py): AST-based parser that extracts real Click decorators from Python implementation filesscripts/docs_parser.py): Regex-based parser that extracts documented commands/options from Markdown filesscripts/cli_verifier.py): Cross-reference analyzer that identifies phantom options, missing documentation, and mismatches~/.claude/commands/verify-cli-documentation.md): Comprehensive prevention system for regular verification🚨 Fixed Critical Documentation Issues
Phantom Options Eliminated ✅
--validate,--draft,--engine)--syntax-only,--citations-only, etc.)--include-source,--include-figures, etc.)--dry-run,--force)Missing Options Added ✅
--track-changes,--debug,--container-mode)--output-dir,--arxiv-dir,--zip-filename,--no-zip)--arxiv-only,--temp-only)Documentation Focus Correction ✅
🛡️ Advanced Features
Command Alias Detection
from .build import build as pdf)main.add_command(config_cmd, name="config"))Comprehensive Reporting
📊 Impact
🔧 Usage
Test plan
pdf(build) andconfig(config_cmd)The verification system discovered and fixed extensive CLI documentation inaccuracies that were misleading users and breaking their workflows. This establishes a foundation for maintaining accurate documentation going forward.
🤖 Generated with Claude Code