Make Cloudflare fallback actually read knowledge-base URLs via server-side fetching

[Copilot]

The Cloudflare AI fallback was passing knowledge-base URLs as plain strings — the model had no way to read them, so answers relied on training data rather than the actual knowledge base. Any LLM can support URL-based grounding when page content is fetched server-side and injected into the prompt directly.

Changes

app/api/webhooks/reply/route.ts

• Added fetchUrlContent(url): fetches a URL (5 s timeout), extracts plain text via JSDOM (existing project dep, same approach as cleanBody), truncates to 4 000 chars
• Cloudflare research fallback now fetches up to 5 knowledge-base pages in parallel with Promise.allSettled and injects their text as prompt context — equivalent quality to Gemini's native urlContext tool
• Gracefully degrades to raw URL list if all fetches fail
• Named constants: MAX_FALLBACK_URLS, FALLBACK_URL_CONTENT_LENGTH, URL_FETCH_TIMEOUT_MS

README.md — Added a note explaining how the fallback handles URLs

Security

Initial regex-based HTML stripping triggered three CodeQL alerts (js/bad-tag-filter, js/incomplete-multi-character-sanitization ×2). Resolved by using JSDOM for parsing instead.

const dom = new JSDOM(html);
dom.window.document.querySelectorAll('script, style').forEach((el) => el.remove());
const text = (dom.window.document.body?.textContent ?? '')
  .replace(/\s+/g, ' ')
  .trim()
  .slice(0, FALLBACK_URL_CONTENT_LENGTH);

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
answerify	Ready	Preview, Comment	Mar 1, 2026 3:35pm