Skip to content

refactor: scripts/ into the installable smith package (src layout, uv)#10

Merged
araujof merged 10 commits into
mainfrom
refactor/smith-package
Jun 29, 2026
Merged

refactor: scripts/ into the installable smith package (src layout, uv)#10
araujof merged 10 commits into
mainfrom
refactor/smith-package

Conversation

@araujof

@araujof araujof commented Jun 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Summary

Repackages Smith's Python code from scripts/ (an ad-hoc import root held together by sys.path hacks) into a proper, publishable smith package using a src/ layout, with pyproject.toml at the repo root. Skills + CLI behavior are preserved.

What changed

  • src/smith/ package: moved cli.py + pipeline packages under src/smith/; all internal imports now use the smith.* root; removed the sys.path.insert(__file__/../../..) hacks. Entry point: smith = smith.cli:main.
  • Root pyproject.toml: src-layout discovery, declared [project.dependencies] (from the old requirements.txt) + a [dev] extra, package-data shipping the policy-testing harness and ares_config. requirements.txt is now a thin -e . pointer. Version → 0.2.0.dev0.
  • uv for package management: make install (uv venv + .[dev]), lint via pinned uvx ruff@0.15.20/black@26.5.1, and make package/publish → uv build/uv publish. CI reworked onto astral-sh/setup-uv.
  • OPA scorecard harness now ships inside the package (smith.policy_testing) and writes all generated output to a BASE_URL-relative dir (references/scorecard/, via TEST_OUTPUT_DIR) instead of scripts/tests/integration/ and the package dir. score_card.sh/coverage tooling are parametrized on ROOT + OUT_DIR. OPA-server lifecycle targets folded into the root Makefile; scripts/Makefile retired.
  • Legacy removal (unreachable from the CLI, unreferenced by guides/examples): kubectl/mcpgateway/beeai cluster, duplicate entry points (pipeline.py, tests/policy_testing.py), dead visualization/ package, the previous (non-functional) pytest suite, and stray upstream ARES repo scaffolding (incl. the .secrets.baseline that caused earlier gitleaks noise). ARES is the external ares-redteamer tool, located via ARES_HOME.
  • mcp_servers/ → examples/, with all references updated (.env_template, README, CLAUDE.md, guides, license-tool excludes, RagChatbot runtime paths, example READMEs).
  • Docs/CI/guides updated throughout for the new layout.

Verification

  • CI on this PR passed — all jobs green (Rego lint skipped, as configured).
  • make ci green (ruff/black over src/, OPA policy lint, SPDX headers).
  • make test live: OPA in Docker over 234 cases → scorecard + coverage written to references/scorecard/; the package dir is not written into.
  • uv build → wheel + sdist; twine check PASSED; wheel ships the harness + ares_config and excludes the ARES scaffolding and tests/.
  • Fresh-venv uv pip install -e . → smith --help and bare smith exit 0.
  • Example smoke tests: call-for-papers-mcp and car-price-mcp-main import cleanly; RagChatbot_MCPServer imports cleanly with no env (see fix below).

Also fixed (RagChatbot example)

  • Its RAG modules (rag_salary.py, rag_pipeline.py) now build the pipeline lazily on first query instead of at import, so rag_*/mcp_server import cleanly without RITS_*/BASE_URL set (previously crashed at import).
  • Wired RITS_API_KEY/RITS_BASE_URL into ChatOpenAI (they were read but never passed through).
  • Added the missing langchain-openai (and langchain-text-splitters) to its requirements.txt.

Follow-ups

  • tests/ is a placeholder; a real test suite is TODO.

araujof added 5 commits June 28, 2026 15:28
@araujof
refactor: remove legacy code and stray ARES scaffolding
ff23548 
Remove code unreachable from the smith CLI and unreferenced by any skill
guide/example, ahead of repackaging scripts/ into the smith package:

- kubectl/mcpgateway/beeai cluster (extract_commands/parse_commands/io/
  extract_case, red_feedback/tests, integration test_commands/test_pipeline,
  unit test_parser) — coupled to packages not in requirements.
- duplicate entry points (policy_agent/pipeline.py, tests/policy_testing.py).
- dead visualization/ package and the broken test_visualize_policy unit test.
- misc orphans (regal_update, extract_extensions, policy_diff tooling,
  reduce_improve/redundancy.sh).
- stray upstream ARES repo scaffolding under test_generation/ares/
  (.github, .pre-commit-config.yaml, .readthedocs.yaml, .secrets.baseline,
  .gitignore) — ARES is the external ares-redteamer tool, not vendored here.

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
@araujof
refactor: move scripts/ into src/smith package; pyproject to root
fa88eb1 
- Move cli.py + pipeline packages from scripts/ to a src/smith/ package
  (src layout); add smith.__init__ with __version__.
- Rewrite all internal imports to the smith.* root; drop the
  sys.path.insert(__file__/../../..) hacks (now a real installable package).
- Move the OPA scorecard harness into src/smith/policy_testing/.
- Move pyproject.toml to the repo root: src-layout discovery, entry point
  smith = smith.cli:main, declared [project.dependencies] (from
  requirements.txt) + [dev] extra, and package-data for the harness +
  ares_config inputs. Bump version to 0.2.0.dev0.
- tests/ is now a placeholder (README) pending proper tests.

Verified: uv pip install -e . succeeds; smith --help and bare smith exit 0.
Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
@araujof
refactor: BASE_URL-relative harness, uv Makefile, mcp_servers->examples
951f65b 
- policy_testing harness: route all generated outputs to a BASE_URL-relative
  dir (references/scorecard/, via TEST_OUTPUT_DIR) instead of the old
  scripts/tests/integration/ and the package dir. score_card.sh +
  convert_test_coverage.py + analyze_coverage.sh + coverage_analyzer.py are now
  parametrized on ROOT (skill root) and OUT_DIR; package scripts stay read-only.
- cli.py / run_policy_evaluation.py: drop base_url + "scripts/" coupling; run
  'make test' from BASE_URL; read scorecard from references/scorecard/.
- Root Makefile reworked for uv: install (uv venv + .[dev]), lint/format via
  pinned uvx ruff/black over src/, folded-in OPA-server + test targets driving
  the packaged harness, and package/dist/wheel/sdist/verify/publish[-test]/clean.
  Retire scripts/Makefile.
- Rename mcp_servers/ -> examples/; update .env_template, license_headers
  EXCLUDE_DIRS, extract_tools docstring, RagChatbot rag_*.py paths, and example
  READMEs. Fix license_headers REPO_ROOT depth for the src layout.

Verified live: make test runs OPA in Docker over 234 cases and writes the
scorecard + coverage to references/scorecard/ (package dir untouched);
make ci green; uv build produces a wheel that ships the harness + ares_config
and excludes the ARES scaffolding and tests/.

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
@araujof
docs+ares: repoint ARES to ARES_HOME; update docs/CI/guides for src l…
2e530df 
…ayout + uv

- attack.py: locate ARES via ARES_HOME (fallback in-tree), run with cwd=ares_home.
- .env_template: TEST_GENERATION_PATH -> src/smith/test_generation/; .gitignore
  ARES artifact path updated.
- README/CONTRIBUTING/CLAUDE.md: src layout + uv install, package/publish targets,
  packaged policy_testing harness, references/scorecard outputs, examples/ paths,
  updated repo-structure tree. requirements.txt is now a thin '-e .' pointer.
- Skill guides (opa_policy/*, test_generation/cross_validate): scorecard paths ->
  references/scorecard/.
- CI (ci.yml): uv-based (astral-sh/setup-uv); make lint/build/audit.
- CHANGELOG: add [Unreleased] entry for the refactor.

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
@araujof
docs: fix stale scripts/ paths in src docstrings
1739f16 
Update module-docstring invocation examples in license_headers.py,
validate_policy.py, and extract_tools.py to the src/smith/ layout.

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
@araujof araujof requested a review from dhl123 as a code owner June 28, 2026 20:35
@araujof araujof changed the title Refactor scripts/ into the installable smith package (src layout, uv) refactor: scripts/ into the installable smith package (src layout, uv) Jun 28, 2026
@araujof araujof added the enhancement New feature or request label Jun 28, 2026
@araujof araujof added this to the 0.1.1 milestone Jun 28, 2026
@araujof
fix(examples): make RagChatbot RAG modules import without env
4954ab9 
- Build the RAG pipeline (HF embeddings + ChatOpenAI + PDF index) lazily on
  first query instead of at import time, so rag_salary/rag_pipeline/mcp_server
  import cleanly without RITS_*/BASE_URL set (previously crashed at import).
- Wire RITS_API_KEY/RITS_BASE_URL into ChatOpenAI (they were read but never
  passed through).
- requirements.txt: add the missing langchain-openai (and langchain-text-splitters).

Verified: the three modules import with no env configured.
Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
@dhl123

dhl123 commented Jun 29, 2026

Copy link
Copy Markdown
Member

README.md

  • Fixed path: changed cd ../../../ to cd ../../../../

Examples' README.md

  • Fixed duplicate cd commands in examples/ instructions
  • Added note to clean references/test_cases folder before starting a new example

RagChatbot example

  • Added fastapi and uvicorn to requirements.txt

Makefile

  • Made venv creation idempotent (skip if .venv already exists)
  • Removed Regal linter prerequisite from OPA server startup / testing. Regal is configured to detect format issues, this can cause errors when running policy_testing on a generated but not formatted policy. These format problems should be fixed by the linting agent or run as a separate make lint-policy commands.

SKILL.md

  • Added prerequisite: activate .venv before running smith commands

opa/output folder

  • Added opa/output folder so patching agents can find their expected write path.

dhl123 added 4 commits June 29, 2026 15:25
@dhl123
duplicate venv checking when running make install, instruct smith to …
b0c424e 
…use existing venv when running smith CLI

Signed-off-by: Hailun Ding <hailun.ding@ibm.com>
@dhl123
add missing fastapi, uvicorn packages in requirements.txt, remove dup…
29b8ba9 
…licated cd commands, add clean setups when switching examples

Signed-off-by: Hailun Ding <hailun.ding@ibm.com>
@dhl123
remove regal in make test, updated readme in two examples, fixed cd p…
a3998bf 
…ath in README.md etc

Signed-off-by: Hailun Ding <hailun.ding@ibm.com>
@dhl123
add outputs folder in case red suggestion cannot find the writing path
efc2ff8 
Signed-off-by: Hailun Ding <hailun.ding@ibm.com>
@dhl123 dhl123 force-pushed the refactor/smith-package branch from c858a32 to efc2ff8 Compare June 29, 2026 19:26
@araujof araujof merged commit 58a26ea into main Jun 29, 2026
6 checks passed
@araujof araujof deleted the refactor/smith-package branch June 29, 2026 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dhl123 dhl123 dhl123 approved these changes

Assignees

@dhl123 dhl123

Labels

enhancement New feature or request

Projects

None yet

Milestone

0.1.1

Development

Successfully merging this pull request may close these issues.

2 participants

@araujof @dhl123