fix(slack): stop inlining PDFs as DataContent on vision-capable models - #638
Merged
Conversation
SlackThreadBindingActor was computing `inlinePdfs` by checking the
ModelModality.Image flag — conflating "model is vision-capable" with
"model accepts inline PDFs". When a user DMed a PDF to a Slack bot
backed by a llama.cpp / vLLM / Ollama endpoint via
OpenAiCompatibleChatClient, the PDF bytes were wrapped as a DataContent
and serialized as an `image_url` content part with a
`data:application/pdf;base64,...` URL. Every OpenAI-compatible upstream
rejects that with "Invalid url format: data:application/pdf;base64",
failing the turn before the agent ever got a chance to process the
file.
PDFs now stay path-only on all providers. The file still lands in
`{sessionDir}/inbox/` via the existing attachment pipeline and the
announcement line already points the agent at
`AttachmentNotes.ModelMissingPdf` — "use shell_execute (e.g.,
pdftotext) to extract text". The agent reads the file via tool calls
instead of the runtime trying to squeeze it through a wire format no
OpenAI-compatible endpoint supports.
Images are unchanged: vision-capable models still receive inline
`image_url` content parts, which is the only way they can be "seen" on
OpenAI-compatible wire formats.
Repros on session D0AC6CKBK5K/1776091017.523089 (2026-04-13).
- Delete `inlinePdfs` from both the live ingress path and
MergeGapWithLiveContents; simplify ResolveInlineDecision to two
parameters.
- Flip SlackAttachmentIngressTests.Pdf_in_dm_on_vision_capable_model_*
into a regression lock asserting no DataContent(application/pdf) ever
reaches the chat client.
Closes the urgent production incident. Follow-up work (enabling native
inline PDFs on providers that actually support them — Anthropic
document blocks, OpenAI file parts) is intentionally out of scope.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the urgent production incident where Slack PDF uploads to ArdyBot fail with
LLM provider error (500): Invalid url format: data:application/pdf;base64.Root cause:
SlackThreadBindingActorwas computinginlinePdfs = InputModalities.HasFlag(ModelModality.Image)— conflating "vision-capable" with "accepts inline PDFs". A PDF dropped into a Slack DM against a llama.cpp / vLLM / Ollama endpoint viaOpenAiCompatibleChatClientgot wrapped asDataContent(application/pdf), whichToMessageunconditionally serialized as animage_urlcontent part with adata:application/pdf;base64,...URL. No OpenAI-compatible upstream accepts that.Fix: PDFs are now unconditionally path-only on all providers. The file still lands in
{sessionDir}/inbox/and the announcement line already points the agent atAttachmentNotes.ModelMissingPdf— "current model has no native PDF support; use shell_execute (e.g., pdftotext) to extract text". The agent reads the file via tool calls instead of the runtime shoving bytes through a wire format the endpoint doesn't support.Images are unchanged: vision-capable models still inline as
image_url, because that's the only way they can be "seen" on OpenAI-compatible wire formats.Reproduces on session
D0AC6CKBK5K/1776091017.523089(2026-04-13).Changes
src/Netclaw.Channels.Slack/SlackThreadBindingActor.cs— deleteinlinePdfsfrom live ingress and historical backfill paths; simplifyResolveInlineDecisionto two parameters.src/Netclaw.Actors.Tests/Channels/SlackAttachmentIngressTests.cs— flipPdf_in_dm_on_vision_capable_model_*into a regression lock asserting noDataContent(application/pdf)ever reaches the chat client.Three prior PRs (#601, #607, #626) plumbed PDFs through ingress → scanner → inbox → LLM input assembly, which is what unmasked this far-end gating bug. None of them touched the modality gate or the provider client.
Out of scope (deliberate)
DataContentthrough the Anthropic SDK, OpenAIfilecontent parts) — those go through different client paths and need their own plumbing. Separate PR if/when someone wants that feature.Qwen3.5-27B-UD-Q4_K_XL.gguf's detected modalities are accurate. Orthogonal.Test plan
dotnet buildcleandotnet test— full suite, 2500+ tests pass including the flippedPdf_in_dm_is_saved_to_inbox_path_only_and_never_inlinedregressiondotnet slopwatch analyze— 0 issuesinlined="false"and the"current model has no native PDF support"note; agent runsshell_execute pdftotext ...againstinbox/Logo Design Services.pdfand answers the user. No 500.