Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,19 @@

## [Unreleased]

### Changed

- `POST /__aimock/reset` is now the canonical full reset and returns a plain `{ "reset": true }` with no deprecation header or body fields. `POST /__aimock/reset/journal` is unaffected.
- **`LLMock.reset()` and the control-API full reset are now one implementation.** Both call a shared `performFullReset`, which clears the fixtures array, the journal (entries _and_ per-test fixture match-counts, across every testId), video and fal.ai job/queue state, and the Gemini interaction/event-id counters. The in-process `reset()` previously left `veoVideoJobs`, `grokVideoJobs` and both Gemini counters populated, so an in-process test suite could see a stale Veo/Grok poll resolve after a reset and interaction ids that never restarted at `aimock-int-0`. `reset()` additionally clears the search / rerank / moderation fixtures, which are registrable only through `LLMock` and that the control API cannot reach.
- **Behavior change for multi-instance in-process users:** `reset()` now also clears state that is MODULE-GLOBAL, not per-instance — the Gemini interaction and event-id counters, and the fal.ai job/queue maps. With two `LLMock` instances live in one process, `a.reset()` rewinds the Gemini id sequence that instance `b` is mid-way through, so `b` re-emits `aimock-int-0` / `evt_1` — ids it has already handed out — and drops `b`'s in-flight fal jobs. The fal maps were already global before this change; the counters are newly reached. Give each instance its own process (or its own vitest worker) if that matters.
- `aimock-pytest`: `AIMockServer.reset()` and `.reset_fixtures()` now call `POST /__aimock/reset`. Both already performed a full reset, so the observable behavior is unchanged.
- **This does not yet avoid the deprecation warning.** `_version.py` pins `AIMOCK_VERSION = "1.38.0"`, and on that published server `/reset` is still the deprecated alias — so the client trips a deprecation on every reset until the pin moves. **Release follow-up (required):** bump `AIMOCK_VERSION` to the first npm release containing this change before publishing the next `aimock-pytest`. Until then the client is deprecation-clean only against a server built from this branch.

### Deprecated

- `POST /__aimock/reset/fixtures` — now a deprecated alias for `POST /__aimock/reset`. The name promised a fixtures-only reset while it always performed the full reset, so `/reset` is the honest route and the deprecation moves onto the alias: it still performs the same full reset but emits a `Deprecation: true` response header, `deprecated` / `deprecation` fields in the body, and a log warning. Use `POST /__aimock/reset` for a full reset, `POST /__aimock/reset/journal` for a journal-only one, or `DELETE /__aimock/fixtures` to clear fixtures and nothing else.
- The alias's **reset semantics are unchanged** — it clears exactly what it always did, so existing callers keep working. Its **response body is additively extended**: it now carries `deprecated` and `deprecation` alongside `reset`, plus the `Deprecation: true` header. A caller asserting strict equality on the old `{ "reset": true }` body will need to relax that assertion; a caller reading `body.reset` is unaffected.

## [1.38.0] - 2026-08-03

### Added
Expand Down
75 changes: 46 additions & 29 deletions docs/control-api/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -94,18 +94,21 @@ <h2>Route Overview</h2>
</tr>
<tr>
<td><code>POST</code></td>
<td><code>/__aimock/reset/fixtures</code></td>
<td>Full reset: fixtures + generation state + journal</td>
<td><code>/__aimock/reset</code></td>
<td>
Full reset: fixtures, journal entries, fixture match-counts (sequence position),
video + fal.ai job state, and the Gemini interaction / event-id counters
</td>
</tr>
<tr>
<td><code>POST</code></td>
<td><code>/__aimock/reset/journal</code></td>
<td>Clear only the request journal</td>
<td>Clear only the request journal entries</td>
</tr>
<tr>
<td><code>POST</code></td>
<td><code>/__aimock/reset</code></td>
<td><strong>Deprecated.</strong> Alias for <code>/reset/fixtures</code></td>
<td><code>/__aimock/reset/fixtures</code></td>
<td><strong>Deprecated.</strong> Alias for <code>/reset</code></td>
</tr>
<tr>
<td><code>POST</code></td>
Expand All @@ -118,11 +121,14 @@ <h2>Route Overview</h2>
<h2>Reset Routes</h2>
<p>
aimock keeps several kinds of in-memory state between requests: the loaded
<strong>fixtures</strong>, the per-provider <strong>generation state</strong> (video,
fal.ai, and Gemini counters), the <strong>fixture match-counts</strong> (sequence
position), and the <strong>request journal</strong> (recorded requests). The reset routes
let you clear these selectively &mdash; a full reset clears everything, while a journal
reset clears only the recorded requests and leaves the rest intact.
<strong>fixtures</strong>, the per-provider <strong>generation state</strong> (video and
fal.ai jobs, plus the Gemini interaction and event-id counters), the
<strong>fixture match-counts</strong> (sequence position), and the
<strong>request journal</strong> (recorded requests). Two routes clear it:
<code>POST /__aimock/reset</code> clears all of it, and
<code>POST /__aimock/reset/journal</code> clears only the recorded requests and leaves
everything else intact. A third, <code>POST /__aimock/reset/fixtures</code>, is a
deprecated alias for the full reset.
</p>

<div
Expand All @@ -136,23 +142,30 @@ <h2>Reset Routes</h2>
margin: 1.5rem 0;
"
>
<strong style="color: var(--warning)">The footgun this split fixes</strong> &mdash; a
caller that wanted a clean journal between test runs used to call
<code>POST /__aimock/reset</code> and unintentionally wiped the loaded fixtures too. Every
subsequent request then returned <code>no_fixture_match</code> until the server was
restarted. If you only want a clean read between runs, use
<strong style="color: var(--warning)">Pick the narrower route</strong> &mdash;
<code>POST /__aimock/reset</code> wipes the loaded fixtures along with everything else,
and what happens to the next request then depends on the mode. In replay mode it fails
with <code>404</code>, and in strict mode with <code>503</code> &mdash; both carry
<code>code: "no_fixture_match"</code>. But
<strong
>in record mode, with a provider key configured, an unmatched request is proxied to the
real provider</strong
>
&mdash; so a full reset mid-recording means live upstream calls and real spend, not an
error. If all you want is a clean read between test runs, use
<code>POST /__aimock/reset/journal</code> &mdash; it leaves your fixtures intact.
</div>

<h3>POST /__aimock/reset/fixtures</h3>
<h3>POST /__aimock/reset</h3>
<p>
<strong>Full reset.</strong> Clears the in-memory fixtures, the generation state (video /
fal.ai / Gemini counters), <em>and</em> the journal. Use this when you want the server
returned to a pristine, fixture-free state.
<strong>Full reset.</strong> Returns the server to a pristine, fixture-free state. It
clears the in-memory fixtures, the journal entries <em>and</em> the per-test fixture
match-counts (so sequenced fixtures rewind to their first response), the video and fal.ai
job and queue state, and the Gemini interaction and event-id counters.
</p>
<div class="code-block">
<div class="code-block-header">Full reset <span class="lang-tag">shell</span></div>
<pre><code>$ curl -X POST http://localhost:4010/__aimock/reset/fixtures</code></pre>
<pre><code>$ curl -X POST http://localhost:4010/__aimock/reset</code></pre>
</div>
<div class="code-block">
<div class="code-block-header">Response <span class="lang-tag">json</span></div>
Expand All @@ -177,25 +190,29 @@ <h3>POST /__aimock/reset/journal</h3>
<pre><code>{ <span class="prop">"reset"</span>: <span class="kw">true</span> }</code></pre>
</div>

<h3>POST /__aimock/reset <span style="color: var(--warning)">(Deprecated)</span></h3>
<h3>
POST /__aimock/reset/fixtures <span style="color: var(--warning)">(Deprecated)</span>
</h3>
<p>
<strong>Deprecated alias</strong> for <code>/__aimock/reset/fixtures</code>. It performs
the same full reset, but additionally sets a <code>Deprecation: true</code> response
header and adds <code>deprecated</code> / <code>deprecation</code> fields to the body.
Prefer the explicit <code>/reset/fixtures</code> or <code>/reset/journal</code> routes
&mdash; they make the intent (and blast radius) of the reset unambiguous.
<strong>Deprecated alias</strong> for <code>/__aimock/reset</code>. The name promises a
fixtures-only reset, but it performs the same full reset &mdash; journal, match-counts,
job state and counters all go with it. It additionally sets a
<code>Deprecation: true</code> response header and adds <code>deprecated</code> /
<code>deprecation</code> fields to the body. Use <code>/reset</code> for a full reset, or
<code>/reset/journal</code> for a journal-only one; to clear fixtures and nothing else,
use <code>DELETE /__aimock/fixtures</code>.
</p>
<div class="code-block">
<div class="code-block-header">Deprecated reset <span class="lang-tag">shell</span></div>
<pre><code>$ curl -i -X POST http://localhost:4010/__aimock/reset</code></pre>
<pre><code>$ curl -i -X POST http://localhost:4010/__aimock/reset/fixtures</code></pre>
</div>
<div class="code-block">
<div class="code-block-header">Response <span class="lang-tag">json</span></div>
<pre><code><span class="cm">// Deprecation: true (response header)</span>
{
<span class="prop">"reset"</span>: <span class="kw">true</span>,
<span class="prop">"deprecated"</span>: <span class="kw">true</span>,
<span class="prop">"deprecation"</span>: <span class="str">"POST /__aimock/reset is deprecated; use POST /__aimock/reset/fixtures (full reset) or POST /__aimock/reset/journal (journal only)"</span>
<span class="prop">"deprecation"</span>: <span class="str">"POST /__aimock/reset/fixtures is deprecated; use POST /__aimock/reset (full reset) or POST /__aimock/reset/journal (journal only)"</span>
}</code></pre>
</div>

Expand Down Expand Up @@ -253,7 +270,7 @@ <h3>POST /__aimock/fixtures</h3>
<h3>DELETE /__aimock/fixtures</h3>
<p>
Clears all registered fixtures. Generation state and the journal are left untouched. To
clear everything at once, use <code>/__aimock/reset/fixtures</code> instead.
clear everything at once, use <code>/__aimock/reset</code> instead.
</p>
<div class="code-block">
<div class="code-block-header">Clear fixtures <span class="lang-tag">shell</span></div>
Expand Down
7 changes: 4 additions & 3 deletions packages/aimock-pytest/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,11 @@ aimock.get_last_request() # most recent request or None
aimock.next_error(429, {"message": "Rate limited"})

# Reset
aimock.clear_fixtures() # remove all fixtures
aimock.reset_fixtures() # clear fixtures + generation state (and journal)
aimock.clear_fixtures() # remove all fixtures, nothing else
aimock.reset() # full reset: fixtures, journal entries + match-counts,
# video/fal job state, Gemini counters
aimock.reset_journal() # clear only the request journal (fixtures preserved)
aimock.reset() # alias for reset_fixtures()
aimock.reset_fixtures() # alias for reset() — a full reset, despite the name
```

## CLI Options
Expand Down
18 changes: 11 additions & 7 deletions packages/aimock-pytest/src/aimock_pytest/_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -324,15 +324,19 @@ def clear_fixtures(self) -> AIMockServer:
return self

def reset(self) -> AIMockServer:
"""Full reset: clear fixtures + generation state + journal (alias for
:meth:`reset_fixtures`)."""
return self.reset_fixtures()
"""Full reset via ``POST /__aimock/reset``: clears fixtures, journal
entries and fixture match-counts, video/fal job state, and the Gemini
counters."""
self._control_request("POST", "/reset", timeout=5).raise_for_status()
return self

def reset_fixtures(self) -> AIMockServer:
"""Clear fixtures + generation state (and journal) via
``POST /__aimock/reset/fixtures``."""
self._control_request("POST", "/reset/fixtures", timeout=5).raise_for_status()
return self
"""Alias for :meth:`reset` — a full reset, not a fixtures-only one.

The name is kept for compatibility; ``DELETE /__aimock/fixtures``
(:meth:`clear_fixtures`) is the fixtures-only call.
"""
return self.reset()

def reset_journal(self) -> AIMockServer:
"""Clear ONLY the request journal, leaving fixtures intact, via
Expand Down
76 changes: 76 additions & 0 deletions packages/aimock-pytest/tests/test_basic.py
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,82 @@ def test_reset_clears_fixtures(aimock):
assert r.status_code == 404


def _capture_control_posts(monkeypatch):
"""Spy on requests.post, recording every /__aimock/ control call.

Returns the list the spy appends ``(url, response)`` to. The real request
still goes out, so the recorded response is the SERVER's, not a stub.
"""
captured = []
real_post = requests.post

def spy(url, *args, **kwargs):
response = real_post(url, *args, **kwargs)
if "/__aimock/" in url:
captured.append((url, response))
return response

monkeypatch.setattr(requests, "post", spy)
return captured


def test_reset_targets_the_canonical_route_not_the_deprecated_alias(aimock, monkeypatch):
"""reset() must POST /__aimock/reset, which carries no deprecation signal.

Both routes perform the same full reset, so a functional assertion cannot
tell them apart. The deprecation signal can: the alias returns
``deprecated``/``deprecation`` in the body and a ``Deprecation`` header,
the canonical route returns neither.
"""
captured = _capture_control_posts(monkeypatch)
aimock.reset()

assert len(captured) == 1
url, response = captured[0]
assert url.endswith("/__aimock/reset")
assert response.status_code == 200

body = response.json()
assert body == {"reset": True}
assert "deprecated" not in body
assert "deprecation" not in body
assert "Deprecation" not in response.headers

# Anchor the discriminator: the alias DOES signal deprecation, so the
# assertions above genuinely distinguish the two routes rather than
# passing for both.
alias = requests.post(f"{aimock.base_url}/__aimock/reset/fixtures", timeout=5)
assert alias.status_code == 200
alias_body = alias.json()
assert alias_body["deprecated"] is True
# The discriminating substring: a self-referential message would read
# "use POST /__aimock/reset/fixtures (full reset)". A bare
# "POST /__aimock/reset" check would be satisfied by that too.
assert "use POST /__aimock/reset (full reset)" in alias_body["deprecation"]
assert alias.headers["Deprecation"] == "true"


def test_reset_fixtures_is_a_deprecation_free_alias_for_reset(aimock, monkeypatch):
"""reset_fixtures() delegates to reset(), so it too uses /__aimock/reset."""
aimock.on_message("test", {"content": "response"})

captured = _capture_control_posts(monkeypatch)
aimock.reset_fixtures()

assert len(captured) == 1
url, response = captured[0]
assert url.endswith("/__aimock/reset")
assert response.json() == {"reset": True}
assert "Deprecation" not in response.headers

# ...and it still performs the full reset its name promises.
r = requests.post(
f"{aimock.base_url}/v1/chat/completions",
json={"model": "gpt-4", "messages": [{"role": "user", "content": "test"}]},
)
assert r.status_code == 404


def test_reset_journal_preserves_fixtures(aimock):
"""reset_journal() clears the journal but leaves fixtures intact."""
aimock.on_message("hello", {"content": "Hi there!"})
Expand Down
Loading
Loading