feat(marketplace): fold marketplace.yml into apm.yml's 'marketplace:' block (closes #722, implements #1036)

[danielmeppiel]

Summary

Folds publisher-side marketplace.yml into the existing consumer manifest apm.yml under a single marketplace: block, eliminating the dual-file authoring surface flagged in #722.

Closes #722. Implements #1036.

Why

apm.yml was the consumer surface (declares dependencies). marketplace.yml was the publisher surface (declares packages this repo offers). Two YAML files in the same repo with overlapping vocabulary (name, version, description, owner) created discoverability friction and template confusion.

A supply-chain analysis of 6 same-repo dual-role ecosystems (npm, Cargo, PyPI, Composer, Maven, Nix) shows all 6 use a single sectioned manifest (e.g. npm's publishConfig). The two-file pattern only appears when the publisher lives in a different repo (Homebrew taps, Helm). APM's prior shape was novel and lost the consolidation benefit.

What

• New marketplace: block in apm.yml carrying owner, build defaults, and packages.
• apm marketplace init|build|check now read/write apm.yml exclusively.
• apm marketplace migrate one-shot consolidates legacy marketplace.yml into apm.yml and removes the file.
• Legacy marketplace.yml still loads with a deprecation warning (one release).
• Both files together = hard error pointing at migrate.
• Local-path package sources (source: ./.github/plugins/foo) are now first-class — they were rejected by the previous SOURCE_RE.
• Tags emission is conditional (omitted when empty) to match the Anthropic canonical schema produced by hand-authored marketplace.json files like the one in microsoft/azure-skills.

How

• apm_cli/marketplace/yml_schema.py — new load_marketplace_from_apm_yml loader.
• apm_cli/commands/marketplace.py — init scaffolds apm.yml when absent and always injects the block; never falls back to marketplace.yml.
• apm_cli/marketplace/builder.py — conditional tags emission, local-path source handling.
• New tests:
  • tests/unit/marketplace/test_apm_yml_marketplace_loader.py
  • tests/unit/marketplace/test_local_path_compose.py
  • tests/unit/marketplace/test_migration_detection.py
  • tests/unit/commands/test_marketplace_init.py (rewritten for new behavior)

Validation

Cloned microsoft/azure-skills, appended a marketplace: block to its existing apm.yml per the new schema, ran apm marketplace build, and diffed against the repo's hand-authored .claude-plugin/marketplace.json — byte-for-byte match.

6745 passed, 1 warning, 27 subtests passed in 44.81s

CLI UX review

Captured the surface across 7 scenarios (init in empty repo / existing apm.yml / re-init guard / --force / legacy deprecation warning / both-files hard error / migrate). The cli-logging-expert flagged 3 issues, all addressed in commit 03f06cd7:

1. BLOCKER — apm marketplace migrate --yes crashed because the flag was named --force, while sibling commands (remove, publish) accept --yes. Fixed: --yes/-y is now an alias for --force on migrate.
2. HIGH — Init in an empty repo emitted two [+] lines for one atomic outcome. Collapsed to [+] Created apm.yml with 'marketplace:' block.
3. HIGH — Re-init against an existing block printed [x] (error) for what is a guard. Switched to [!] (warning).

Trade-offs

• One release of dual-load support. marketplace.yml still works with a deprecation warning. Slated for removal in the next minor.
• Schema is unchanged. This PR only touches the authoring INPUT surface. The generated marketplace.json remains the Anthropic canonical format.
• Single PR, ~800 LOC delta. Considered splitting (loader / commands / migrate) but the change is internally cohesive and easier to review as one unit.

How to test

# 1. Greenfield init
mkdir /tmp/mp && cd /tmp/mp
apm experimental enable marketplace-authoring
apm marketplace init --owner my-org
cat apm.yml  # marketplace: block present

# 2. Migration of legacy file
mkdir /tmp/mig && cd /tmp/mig
echo -e "name: app\nversion: 1.0.0\ndescription: x" > apm.yml
cat > marketplace.yml << 'YAML'
name: mp
version: 0.1.0
description: x
owner:
  name: acme
packages: []
YAML
apm marketplace migrate --yes
ls marketplace.yml  # gone
cat apm.yml  # marketplace: block populated

# 3. Real-repo build
git clone https://github.com/microsoft/azure-skills /tmp/az
cd /tmp/az
# (apm.yml already has the marketplace: block in this PR's validation)
apm marketplace build
diff <(jq -S . .claude-plugin/marketplace.json) <(jq -S . marketplace.json.generated)

[Copilot]

Pull request overview

This PR consolidates marketplace authoring configuration into a single apm.yml surface by introducing a top-level marketplace: block (with one-release legacy marketplace.yml support), adds a migration path, and expands package source support to include local ./... paths while keeping the generated marketplace.json Anthropic-compliant.

Changes:

• Add a shape-aware marketplace config loader (apm.yml block vs legacy marketplace.yml) plus a new apm marketplace migrate command.
• Update marketplace build/compose to support local-path packages and conditionally omit empty fields (e.g., tags; inherited top-level description/version).
• Rewrite/extend unit tests to cover apm.yml-block loading, local-path compose, and source detection/migration behavior.

Show a summary per file

File	Description
src/apm_cli/marketplace/yml_schema.py	Introduces MarketplaceConfig, apm.yml-block loader, local ./... sources, new defaults/override flags.
src/apm_cli/marketplace/migration.py	Adds config-source detection, smart loader w/ deprecation warning, and legacy->apm.yml migration helper.
src/apm_cli/marketplace/builder.py	Handles local-path entries, conditional field emission, and output resolution against project root.
src/apm_cli/marketplace/yml_editor.py	Makes the YAML editor shape-aware so it can operate on either legacy root keys or apm.yml marketplace: container.
src/apm_cli/marketplace/init_template.py	Adds a new rendered snippet for the apm.yml marketplace: block template.
src/apm_cli/marketplace/__init__.py	Re-exports new config/loader symbols.
src/apm_cli/commands/marketplace.py	Updates init/build/check/outdated/doctor/publish to use the smart loader; adds migrate; changes messaging.
src/apm_cli/commands/marketplace_plugin.py	Updates package-subcommands to locate the active config (apm.yml block vs legacy file) and hard-error on both.
src/apm_cli/commands/init.py	Adds apm init --marketplace to seed apm.yml with a marketplace authoring block.
tests/unit/marketplace/test_apm_yml_marketplace_loader.py	New tests for inheritance/override semantics in apm.yml marketplace block loader.
tests/unit/marketplace/test_migration_detection.py	New tests for config-source detection and smart loader behavior.
tests/unit/marketplace/test_local_path_compose.py	New tests for local-path package resolution + JSON compose output shape.
tests/unit/marketplace/test_yml_schema.py	Updates schema tests for local-path source acceptance and .. traversal rejection.
tests/unit/commands/test_marketplace_init.py	Rewritten tests reflecting apm.yml-block initialization behavior.
tests/unit/commands/test_marketplace_migrate.py	New CLI tests for the migration command.
tests/unit/commands/test_marketplace_build.py	Updates assertions/messages for new “config” wording.
tests/unit/commands/test_marketplace_check.py	Updates patches/mocks to use load_marketplace_config.
tests/unit/commands/test_marketplace_doctor.py	Updates expected doctor output text for new config naming.

Copilot's findings

• Files reviewed: 18/18 changed files
• Comments generated: 12

[Copilot]

This test no longer checks single-dot traversal; it validates that local-path sources like './acme' are accepted. Consider renaming the test function to reflect its new purpose (e.g. test_local_source_accepted) so failures are easier to interpret.

Suggested change

	def test_source_dot_traversal(self, tmp_path: Path):
	def test_local_source_accepted(self, tmp_path: Path):

[Copilot]

The init flow now calls _check_gitignore_for_marketplace_json(), but that helper's warning text still references tracking marketplace.yml. Since authoring config moved into apm.yml, please update the helper message (and any related guidance) to reference apm.yml + the generated marketplace.json path.

[Copilot]

This module-level docstring still implies the file only renders a standalone marketplace.yml scaffold, but it now also owns the apm.yml marketplace: block template. Consider updating the module docstring to reflect both templates so readers don't miss render_marketplace_block().

[Copilot]

LOCAL_SOURCE_RE is imported but never used here, which will fail unused-import linting. Either remove the import or use it to drive allow_current_dir behavior in _validate_source().

[Copilot]

DEPRECATION_MESSAGE is imported but never used in this file; please drop the unused import to keep linting clean.

Suggested change

DEPRECATION_MESSAGE,

[Copilot]

_has_marketplace_block() returns False on YAML parse errors, which makes detect_config_source()/load_marketplace_config() report "No marketplace config" even when apm.yml exists but is malformed. Consider surfacing a MarketplaceYmlError for unreadable apm.yml (or treating parse errors as "apm.yml present" so the real loader error is shown) to avoid misleading guidance.

Suggested change

	"""Return ``True`` when *apm_yml_path* exists and has ``marketplace:``."""
	if not apm_yml_path.exists():
	return False
	try:
	text = apm_yml_path.read_text(encoding="utf-8")
	data = yaml.safe_load(text)
	except (OSError, yaml.YAMLError):
	return False
	return isinstance(data, dict) and "marketplace" in data and \
	data["marketplace"] is not None
	"""Return ``True`` when *apm_yml_path* has a non-null ``marketplace:``.
	Missing files and valid YAML without a top-level ``marketplace`` block
	return ``False``. Read failures and YAML parse errors raise
	:class:`MarketplaceYmlError` so callers do not mistake malformed
	``apm.yml`` for an absent marketplace configuration.
	"""
	if not apm_yml_path.exists():
	return False
	try:
	text = apm_yml_path.read_text(encoding="utf-8")
	except OSError as exc:
	raise MarketplaceYmlError(
	f"Could not read {apm_yml_path.name}: {exc}"
	) from exc
	try:
	data = yaml.safe_load(text)
	except yaml.YAMLError as exc:
	raise MarketplaceYmlError(
	f"Invalid YAML in {apm_yml_path.name}: {exc}"
	) from exc
	return (
	isinstance(data, dict)
	and "marketplace" in data
	and data["marketplace"] is not None
	)

[Copilot]

migrate_marketplace_yml() assumes rt.load(apm_text) returns a mapping; if apm.yml is empty or not a YAML mapping, "marketplace" in apm_data will raise a TypeError. Add an explicit type check and raise MarketplaceYmlError with a clear message before proceeding.

[Copilot]

_is_apm_yml_with_marketplace() returns True even when the marketplace value is not a mapping. That can make _get_marketplace_container() return a non-dict and then container.get(...) will raise AttributeError. Tighten the check to require isinstance(data["marketplace"], dict) (or a ruamel CommentedMap).

[Copilot]

If apm.yml is empty or parses to a non-mapping, rt.load() may return None/other types and data["marketplace"] = ... will crash. Consider initializing data to an empty mapping when load returns None, and erroring out when it returns a non-mapping.

[Copilot]

The module docstring claims "rejection of unknown keys at both levels", but load_marketplace_from_apm_yml only enforces strict keys inside the marketplace: block (top-level apm.yml keys are intentionally not strict). Please update the docstring to match the actual behavior.

Suggested change

	of unknown keys at both levels.
	of unknown keys within the marketplace block.