Skip to content

fix(docs): unbreak published Fern site #615

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
andreatnvidia merged 4 commits into from
May 7, 2026
Merged

fix(docs): unbreak published Fern site #615

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
47d2cc6
fix(docs): shrink inline base64 images in notebook HTML outputs
lbliii May 7, 2026
afadf1c
fix(docs): migrate leftover MkDocs tab syntax to Fern Tabs
lbliii May 7, 2026
afa7f68
fix(docs): convert leftover MkDocs admonition to Fern Tip
lbliii May 7, 2026
8340272
Merge branch 'main' into lbliii/fix-fern-published-docs
andreatnvidia May 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion fern/components/notebooks/5-generating-images.json
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion fern/components/notebooks/5-generating-images.ts
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions fern/components/notebooks/6-editing-images-with-image-context.json
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions fern/components/notebooks/6-editing-images-with-image-context.ts
View file
Open in desktop

Large diffs are not rendered by default.

25 changes: 25 additions & 0 deletions fern/scripts/ipynb-to-fern-json.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,16 @@
re.IGNORECASE,
)

# Inline base64 PNG/JPEG embedded in IPython.display.HTML blobs. The image
# notebooks (5, 6) emit `<img src='data:image/png;base64,...'>` inside HTML
# outputs, which bypasses the `image/png` MIME path and so skips
# shrink_image_b64 — leaving multi-MB images in the .ts payload and breaking
# Fern's SSR bundler. Match here so the HTML branch can shrink them too.
INLINE_DATA_URI_RE = re.compile(
r"data:image/(png|jpe?g);base64,([A-Za-z0-9+/=\s]+?)(?=[\"'\s)])",
re.IGNORECASE,
Comment on lines +64 to +66

@greptile-apps greptile-apps Bot May 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Regex whitespace handling is self-contradicting

INLINE_DATA_URI_RE includes \s in the character class so the capture group can span whitespace-wrapped base64, and _sub even calls "".join(match.group(2).split()) to strip that whitespace. However, the lazy quantifier +? combined with \s in the lookahead (?=["'\s)]) means the match terminates at the first whitespace it encounters — so \s in the character class is never reachable and the whitespace-stripping in _sub is always a no-op. If a future notebook emits pretty-printed HTML with line-wrapped base64 (e.g. src='data:image/png;base64,\nAAA...), the regex captures only the empty prefix before the newline, passes garbage to shrink_image_b64, which fails silently and returns an empty string, and the replacement produces a broken data:image/jpeg;base64, URI.

Prompt To Fix With AI 
This is a comment left during a code review.
Path: fern/scripts/ipynb-to-fern-json.py
Line: 64-66

Comment:
**Regex whitespace handling is self-contradicting**

`INLINE_DATA_URI_RE` includes `\s` in the character class so the capture group can span whitespace-wrapped base64, and `_sub` even calls `"".join(match.group(2).split())` to strip that whitespace. However, the lazy quantifier `+?` combined with `\s` in the lookahead `(?=["'\s)])` means the match terminates at the *first* whitespace it encounters — so `\s` in the character class is never reachable and the whitespace-stripping in `_sub` is always a no-op. If a future notebook emits pretty-printed HTML with line-wrapped base64 (e.g. `src='data:image/png;base64,\nAAA...`), the regex captures only the empty prefix before the newline, passes garbage to `shrink_image_b64`, which fails silently and returns an empty string, and the replacement produces a broken `data:image/jpeg;base64,` URI.

How can I resolve this? If you propose a fix, please make it concise.

)


def get_language(metadata: dict) -> str:
info = metadata.get("kernelspec", {}) or {}
Expand Down Expand Up @@ -112,6 +122,20 @@ def shrink_image_b64(b64: str, max_dim: int = MAX_IMAGE_DIMENSION) -> tuple[str,
return b64, "image/png"


def shrink_inline_b64_in_html(html: str) -> str:
"""Replace each inline `data:image/...;base64,...` URI inside an HTML string
with a shrunk JPEG variant. IPython.display.HTML outputs in the image
notebooks embed full-resolution PNGs this way; without resizing, a single
cell can carry 2MB+ of base64."""

def _sub(match: re.Match[str]) -> str:
b64 = "".join(match.group(2).split())
shrunk, mime = shrink_image_b64(b64)
return f"data:{mime};base64,{shrunk}"

return INLINE_DATA_URI_RE.sub(_sub, html)


def extract_outputs(outputs: list) -> list[dict]:
result: list[dict] = []
for out in outputs:
Expand All @@ -133,6 +157,7 @@ def extract_outputs(outputs: list) -> list[dict]:
if isinstance(html, list):
html = "".join(html)
if html.strip():
html = shrink_inline_b64_in_html(html)
result.append({"type": "text", "data": html, "format": "html"})
elif "text/plain" in data:
text = data["text/plain"]
Expand Down
94 changes: 51 additions & 43 deletions fern/versions/v0.5.8/pages/concepts/agent-rollout-ingestion.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,68 +9,76 @@ position: 3

Use `AgentRolloutSeedSource` when you want to work from existing agent traces instead of traces captured during a Data Designer generation run.

=== "Claude Code"
<Tabs>
<Tab title="Claude Code">

Uses `~/.claude/projects` and `*.jsonl` by default.
Uses `~/.claude/projects` and `*.jsonl` by default.

```python
import data_designer.config as dd
```python
import data_designer.config as dd

seed_source = dd.AgentRolloutSeedSource(
format=dd.AgentRolloutFormat.CLAUDE_CODE,
)
```
seed_source = dd.AgentRolloutSeedSource(
format=dd.AgentRolloutFormat.CLAUDE_CODE,
)
```

=== "Codex"
</Tab>
<Tab title="Codex">

Uses `~/.codex/sessions` and `*.jsonl` by default.
Uses `~/.codex/sessions` and `*.jsonl` by default.

```python
import data_designer.config as dd
```python
import data_designer.config as dd

seed_source = dd.AgentRolloutSeedSource(
format=dd.AgentRolloutFormat.CODEX,
)
```
seed_source = dd.AgentRolloutSeedSource(
format=dd.AgentRolloutFormat.CODEX,
)
```

=== "Hermes Agent"
</Tab>
<Tab title="Hermes Agent">

Uses `~/.hermes/sessions` and `*.json*` by default so CLI session logs and gateway transcripts can coexist.
Uses `~/.hermes/sessions` and `*.json*` by default so CLI session logs and gateway transcripts can coexist.

```python
import data_designer.config as dd
```python
import data_designer.config as dd

seed_source = dd.AgentRolloutSeedSource(
format=dd.AgentRolloutFormat.HERMES_AGENT,
)
```
seed_source = dd.AgentRolloutSeedSource(
format=dd.AgentRolloutFormat.HERMES_AGENT,
)
```

=== "Pi Coding Agent"
</Tab>
<Tab title="Pi Coding Agent">

Uses `~/.pi/agent/sessions` and `*.jsonl` by default. Sessions are tree-structured JSONL files; the active conversation path is resolved automatically.
Uses `~/.pi/agent/sessions` and `*.jsonl` by default. Sessions are tree-structured JSONL files; the active conversation path is resolved automatically.

```python
import data_designer.config as dd
```python
import data_designer.config as dd

seed_source = dd.AgentRolloutSeedSource(
format=dd.AgentRolloutFormat.PI_CODING_AGENT,
)
```
seed_source = dd.AgentRolloutSeedSource(
format=dd.AgentRolloutFormat.PI_CODING_AGENT,
)
```

=== "ATIF"
</Tab>
<Tab title="ATIF">

ATIF requires an explicit `path`. See Harbor's [ATIF documentation](https://harborframework.com/docs/trajectory-format) for the format specification.
ATIF requires an explicit `path`. See Harbor's [ATIF documentation](https://harborframework.com/docs/trajectory-format) for the format specification.

```python
import data_designer.config as dd
```python
import data_designer.config as dd

seed_source = dd.AgentRolloutSeedSource(
format=dd.AgentRolloutFormat.ATIF,
path="/data/harbor/runs/swe-bench/job-042",
recursive=True,
file_pattern="trajectory*.json",
)
```
seed_source = dd.AgentRolloutSeedSource(
format=dd.AgentRolloutFormat.ATIF,
path="/data/harbor/runs/swe-bench/job-042",
recursive=True,
file_pattern="trajectory*.json",
)
```

</Tab>
</Tabs>

You can override `path` and `file_pattern` for any format when your rollout artifacts live outside the built-in defaults.

Expand Down
5 changes: 3 additions & 2 deletions fern/versions/v0.5.8/pages/concepts/models/default-model-settings.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,9 @@ When the Data Designer library or the CLI is initialized, default model configur
- **Model Configs**: `~/.data-designer/model_configs.yaml`
- **Model Providers**: `~/.data-designer/model_providers.yaml`

!!! tip Tip
While these files provide a convenient way to specify settings for your model providers and configuration you use most often, they can always be set programmatically in your SDG workflow.
<Tip>
While these files provide a convenient way to specify settings for your model providers and configuration you use most often, they can always be set programmatically in your SDG workflow.
</Tip>

You can customize the home directory location by setting the `DATA_DESIGNER_HOME` environment variable:

Expand Down
Loading