Skip to content

Protect real RAG search and answer APIs with authenticated owner scope #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
BigSimmo merged 7 commits into from
Jun 13, 2026
Merged

Protect real RAG search and answer APIs with authenticated owner scope #22

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
a7c45eb
fix: implement first planned fix batch
BigSimmo Jun 11, 2026
870b0ca
fix: implement first planned fix batch
BigSimmo Jun 11, 2026
8ae342e
fix: implement first planned fix batch
BigSimmo Jun 11, 2026
5952f1a
fix: implement first planned fix batch
BigSimmo Jun 11, 2026
a732265
fix: implement first planned fix batch
BigSimmo Jun 11, 2026
971ebe4
fix: implement first planned fix batch
BigSimmo Jun 11, 2026
ae29521
fix: gate private search UI and eval auth
BigSimmo Jun 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 26 additions & 1 deletion scripts/eval-search-api.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ loadEnvConfig(process.cwd());

type Args = {
baseUrl?: string;
authToken?: string;
limit?: number;
question?: string;
json: boolean;
Expand Down Expand Up @@ -42,6 +43,7 @@ function parseArgs(argv: string[]): Args {
if (!value || value.startsWith("--")) throw new Error(`Missing value for ${token}`);
index += 1;
if (token === "--base-url") args.baseUrl = value.replace(/\/+$/, "");
if (token === "--auth-token") args.authToken = value;
if (token === "--limit") args.limit = Number.parseInt(value, 10);
if (token === "--question") args.question = value;
}
Expand All @@ -56,6 +58,19 @@ function ensuredBaseUrl() {
}).trim();
}

function envAuthToken() {
return (
process.env.RAG_EVAL_API_AUTH_TOKEN?.trim() ||
process.env.RAG_EVAL_AUTH_TOKEN?.trim() ||
process.env.SUPABASE_ACCESS_TOKEN?.trim() ||

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 13, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid forwarding Supabase CLI tokens to the app

When a developer has the Supabase CLI configured in CI/local, SUPABASE_ACCESS_TOKEN is a personal access token for the Management API, not an Auth user JWT (Supabase docs describe it as a personal access token). This script now silently forwards that long-lived token as Authorization: Bearer to whichever --base-url is used and suppresses the no-token diagnostic, so real-mode evals still 401 while potentially exposing a management token to a preview/non-local URL. Please only auto-read the RAG-specific env vars/explicit flag, or otherwise require a real user session JWT.

Useful? React with 👍 / 👎.

""
);
}

function isLocalNoAuthEval() {
return process.env.LOCAL_NO_AUTH === "true" || process.env.NEXT_PUBLIC_LOCAL_NO_AUTH === "true";
}

function validate(result: ApiSearchEvalResult, testCase: ReturnType<typeof selectRagEvalCases>[number]) {
const failures: string[] = [];
if (!result.ok) failures.push(`HTTP ${result.status}`);
Expand All @@ -72,13 +87,17 @@ function validate(result: ApiSearchEvalResult, testCase: ReturnType<typeof selec
async function main() {
const args = parseArgs(process.argv.slice(2));
const baseUrl = args.baseUrl ?? ensuredBaseUrl();
const authToken = args.authToken?.trim() || envAuthToken();
const cases = selectRagEvalCases({ limit: args.limit, question: args.question });
const results: ApiSearchEvalResult[] = [];

for (const testCase of cases) {
const response = await fetch(`${baseUrl}/api/search`, {
method: "POST",
headers: { "Content-Type": "application/json" },
headers: {
"Content-Type": "application/json",
...(authToken ? { Authorization: `Bearer ${authToken}` } : {}),
},
body: JSON.stringify({ query: testCase.question, topK: 8 }),
});
const text = await response.text();
Expand All @@ -105,7 +124,13 @@ async function main() {
}

const thresholdFailures = results.filter((result) => result.failures.length > 0).map((result) => result.id);
const unauthorized = results.some((result) => result.status === 401);
if (args.json) console.log(JSON.stringify({ baseUrl, results, thresholdFailures }, null, 2));
if (unauthorized && !authToken && !isLocalNoAuthEval()) {
console.error(
"API search eval was rejected by the real-mode auth gate. Pass --auth-token or set RAG_EVAL_API_AUTH_TOKEN, RAG_EVAL_AUTH_TOKEN, or SUPABASE_ACCESS_TOKEN to a Supabase access token, or run the ensured local server with LOCAL_NO_AUTH=true and NEXT_PUBLIC_LOCAL_NO_AUTH=true.",
);
}
if (args.failOnThreshold && thresholdFailures.length > 0) process.exit(1);
}

Expand Down
10 changes: 9 additions & 1 deletion src/app/api/answer/route.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ import { jsonError, PublicApiError } from "@/lib/http";
import { consumePublicAnswerRateLimit } from "@/lib/public-rate-limit";
import { classifyRagQuery } from "@/lib/clinical-search";
import { buildSmartRagApiPlan } from "@/lib/smart-rag-api";
import { createAdminClient } from "@/lib/supabase/admin";
import * as serverAuth from "@/lib/supabase/auth";

export const runtime = "nodejs";

Expand Down Expand Up @@ -36,6 +38,9 @@ export async function POST(request: Request) {
});
}

const supabase = createAdminClient();
const user = await serverAuth.requireAuthenticatedUser(request, supabase);

const rateLimit = consumePublicAnswerRateLimit(request.headers);
if (rateLimit.limited) {
return NextResponse.json(
Expand All @@ -48,10 +53,13 @@ export async function POST(request: Request) {
query: body.query,
documentId: body.documentId,
documentIds: body.documentIds,
ownerId: undefined,
ownerId: user.id,
});
return NextResponse.json(answer);
} catch (error) {
if (error instanceof serverAuth.AuthenticationError) {
return serverAuth.unauthorizedResponse(error);
}
if (error instanceof z.ZodError) {
return jsonError(error, 400);
}
Expand Down
10 changes: 9 additions & 1 deletion src/app/api/answer/stream/route.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ import { answerQuestionWithScope, type AnswerProgressEvent } from "@/lib/rag";
import { classifyRagQuery } from "@/lib/clinical-search";
import { annotateSearchResults, buildEvidenceRelevance } from "@/lib/evidence-relevance";
import { buildSmartRagApiPlan } from "@/lib/smart-rag-api";
import { createAdminClient } from "@/lib/supabase/admin";
import { requireAuthenticatedUser } from "@/lib/supabase/auth";

export const runtime = "nodejs";

Expand Down Expand Up @@ -132,11 +134,17 @@ export async function POST(request: Request) {
const body = answerSchema.parse(await request.json());
if (isDemoMode()) return streamAnswer(body);

const supabase = createAdminClient();
const user = await requireAuthenticatedUser(request, supabase);

const rateLimit = consumePublicAnswerRateLimit(request.headers);
if (rateLimit.limited) return rateLimitStream(rateLimit);

return streamAnswer(body);
return streamAnswer(body, user.id);
} catch (error) {
if (error instanceof Error && error.name === "AuthenticationError") {
return Response.json({ error: "Authentication required." }, { status: 401 });
}
if (error instanceof z.ZodError) {
return jsonError(error, 400);
}
Expand Down
131 changes: 77 additions & 54 deletions src/app/api/search/route.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import { searchChunksWithTelemetry } from "@/lib/rag";
import { classifyRagQuery, normalizedClinicalSearchTokens } from "@/lib/clinical-search";
import { buildSmartRagApiPlan } from "@/lib/smart-rag-api";
import { createAdminClient } from "@/lib/supabase/admin";
import * as serverAuth from "@/lib/supabase/auth";
import { consumePublicSearchRateLimit } from "@/lib/public-rate-limit";
import type { ChunkImage, ClinicalSourceMetadata, SearchResult } from "@/lib/types";

Expand All @@ -28,10 +29,11 @@ const searchSchema = z.object({

type SearchRequestBody = z.infer<typeof searchSchema>;

const publicSearchInflight = new Map<string, Promise<unknown>>();
const scopedSearchInflight = new Map<string, Promise<unknown>>();

function publicSearchKey(body: SearchRequestBody) {
function scopedSearchKey(body: SearchRequestBody, ownerId: string) {
return JSON.stringify({
ownerId,
query: body.query.toLowerCase().replace(/\s+/g, " ").trim(),
topK: body.topK ?? null,
documentId: body.documentId ?? null,
Expand All @@ -42,14 +44,14 @@ function publicSearchKey(body: SearchRequestBody) {
});
}

async function coalescePublicSearch<T extends Record<string, unknown>>(key: string, producer: () => Promise<T>) {
const existing = publicSearchInflight.get(key) as Promise<T> | undefined;
async function coalesceScopedSearch<T extends Record<string, unknown>>(key: string, producer: () => Promise<T>) {
const existing = scopedSearchInflight.get(key) as Promise<T> | undefined;
if (existing) return { payload: await existing, coalesced: true };

const pending = producer().finally(() => {
publicSearchInflight.delete(key);
scopedSearchInflight.delete(key);
});
publicSearchInflight.set(key, pending);
scopedSearchInflight.set(key, pending);
return { payload: await pending, coalesced: false };
}

Expand Down Expand Up @@ -294,6 +296,7 @@ function candidatePromotions(query: string, results: SearchResult[]) {

function logWeakSearch(args: {
supabase: ReturnType<typeof createAdminClient>;
ownerId: string;
query: string;
queryClass: string;
route?: string | null;
Expand All @@ -317,7 +320,7 @@ function logWeakSearch(args: {
void args.supabase
.from("rag_query_misses")
.insert({
owner_id: null,
owner_id: args.ownerId,
query: args.query,
normalized_query: args.query.toLowerCase().replace(/\s+/g, " ").trim(),
query_class: args.queryClass,
Expand Down Expand Up @@ -356,7 +359,9 @@ function latencyBucket(ms: number) {
return "gte_5000ms";
}

function logPublicSearchObservation(args: {
function logSearchObservation(args: {
supabase: ReturnType<typeof createAdminClient>;
ownerId: string;
query: string;
results: SearchResult[];
payload: Record<string, unknown>;
Expand All @@ -371,48 +376,49 @@ function logPublicSearchObservation(args: {
const payloadBytes = Buffer.byteLength(JSON.stringify(args.payload), "utf8");
const topResult = args.results[0] ?? null;
const latencyMs = telemetryLatencyMs(telemetry);
await createAdminClient()
.from("rag_queries")
.insert({
owner_id: null,
query: args.query,
answer: null,
source_chunk_ids: args.results.map((result) => result.id),
model: "search",
metadata: {
event_type: args.failure ? "public_search_failure" : "public_search",
failure_code: args.failure?.code ?? null,
failure_cause_name: args.failure?.causeName ?? null,
failure_cause_message: args.failure?.causeMessage ?? null,
failure_sql_state: args.failure?.sqlState ?? null,
query_class: telemetry.query_class ?? null,
retrieval_strategy: telemetry.retrieval_strategy ?? null,
payload_bytes: payloadBytes,
result_count: args.results.length,
top_document_id: topResult?.document_id ?? null,
top_document_title: topResult?.title ?? null,
top_file_name: topResult?.file_name ?? null,
top_score: topResult ? (topResult.hybrid_score ?? topResult.similarity ?? null) : null,
latency_ms: latencyMs,
latency_bucket: latencyBucket(latencyMs),
search_cache_hit: telemetry.search_cache_hit ?? null,
embedding_skipped: telemetry.embedding_skipped ?? null,
},
});
await args.supabase.from("rag_queries").insert({
owner_id: args.ownerId,
query: args.query,
answer: null,
source_chunk_ids: args.results.map((result) => result.id),
model: "search",
metadata: {
event_type: args.failure ? "private_search_failure" : "private_search",
failure_code: args.failure?.code ?? null,
failure_cause_name: args.failure?.causeName ?? null,
failure_cause_message: args.failure?.causeMessage ?? null,
failure_sql_state: args.failure?.sqlState ?? null,
query_class: telemetry.query_class ?? null,
retrieval_strategy: telemetry.retrieval_strategy ?? null,
payload_bytes: payloadBytes,
result_count: args.results.length,
top_document_id: topResult?.document_id ?? null,
top_document_title: topResult?.title ?? null,
top_file_name: topResult?.file_name ?? null,
top_score: topResult ? (topResult.hybrid_score ?? topResult.similarity ?? null) : null,
latency_ms: latencyMs,
latency_bucket: latencyBucket(latencyMs),
search_cache_hit: telemetry.search_cache_hit ?? null,
embedding_skipped: telemetry.embedding_skipped ?? null,
},
});
} catch {
// Search telemetry must not affect the user-facing search path.
}
})();
}

async function buildPublicSearchPayload(body: SearchRequestBody) {
const supabase = createAdminClient();
async function buildScopedSearchPayload(
body: SearchRequestBody,
supabase: ReturnType<typeof createAdminClient>,
ownerId: string,
) {
const search = await searchChunksWithTelemetry({
query: body.query,
topK: body.mode === "documents" ? Math.max(body.topK ?? 12, Math.min(20, body.documentLimit)) : (body.topK ?? 8),
documentId: body.documentId,
documentIds: body.documentIds,
ownerId: undefined,
ownerId,
});
const resultLimit =
body.mode === "documents" ? Math.max(body.topK ?? 12, Math.min(20, body.documentLimit)) : (body.topK ?? 8);
Expand All @@ -421,7 +427,7 @@ async function buildPublicSearchPayload(body: SearchRequestBody) {
const relatedDocuments = body.includeRelatedDocuments
? await fetchRelatedDocuments({
supabase,
ownerId: undefined,
ownerId,
query: body.query,
results,
limit: body.mode === "documents" ? body.documentLimit : undefined,
Expand All @@ -444,6 +450,7 @@ async function buildPublicSearchPayload(body: SearchRequestBody) {
});
logWeakSearch({
supabase,
ownerId,
query: body.query,
queryClass,
route: body.mode,
Expand Down Expand Up @@ -497,7 +504,7 @@ async function buildPublicSearchPayload(body: SearchRequestBody) {
rrf_top_score: search.telemetry.rrf_top_score,
},
};
logPublicSearchObservation({ query: body.query, results, payload });
logSearchObservation({ supabase, ownerId, query: body.query, results, payload });
return payload;
}

Expand Down Expand Up @@ -531,6 +538,9 @@ function extractSqlState(error: unknown) {
}

export async function POST(request: Request) {
let supabase: ReturnType<typeof createAdminClient> | null = null;
let ownerId: string | null = null;

try {
const body = searchSchema.parse(await request.json());
if (isDemoMode()) {
Expand Down Expand Up @@ -564,6 +574,10 @@ export async function POST(request: Request) {
});
}

supabase = createAdminClient();
const user = await serverAuth.requireAuthenticatedUser(request, supabase);

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 13, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply rate limiting before auth validation

For requests that include any invalid Authorization: Bearer ... header, this call goes out to Supabase Auth and throws before the per-IP limiter at lines 581-582 is consumed, so those requests are never counted; the same auth-before-limit order was added to /api/answer and /api/answer/stream. A client can therefore send unlimited invalid-token attempts and force external auth validation on each request instead of receiving 429s from this endpoint. Move the cheap limiter before token validation, or add a separate auth-failure limiter, so authentication failures are throttled too.

Useful? React with 👍 / 👎.

ownerId = user.id;
Comment on lines +578 to +579

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Update existing unauthenticated route tests

Adding this auth gate changes the real-mode contract, but the existing tests/private-access-routes.test.ts suite still calls /api/search, /api/answer, and /api/answer/stream without an Authorization header and asserts successful unauthenticated behavior (for example the tests around allows unauthenticated search and answer without owner scope, public rate limiting, coalescing, and streaming without auth). With this line in place those requests return 401 before the mocked RAG functions run, so the normal npm test suite will fail unless those tests are updated or converted to demo/local-no-auth/authenticated scenarios.

Useful? React with 👍 / 👎.

Comment on lines +577 to +579

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pass auth when running API search evals

This new real-mode auth requirement also breaks the existing npm run eval:search:api workflow outside demo/local-no-auth mode: scripts/eval-search-api.ts posts to /api/search with only Content-Type, so every eval case now receives 401 and --fail-on-threshold will fail before measuring retrieval quality. The API eval script needs a way to supply a bearer token or otherwise opt into the same authenticated/local owner flow as the other RAG eval scripts.

Useful? React with 👍 / 👎.

Comment on lines +577 to +579

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Gate real-mode search on auth in the UI

After this auth check, the dashboard still enables the main search button in real mode whenever setup is ready because ClinicalDashboard passes realDataReady={canRunSearch} and canRunSearch does not include canUsePrivateApis. For an unsigned-in user with a configured real backend, submitting a search now just calls these endpoints without credentials, marks the session expired, and shows a 401 error instead of disabling the action or requiring sign-in first.

Useful? React with 👍 / 👎.


const rateLimit = consumePublicSearchRateLimit(request.headers);
if (rateLimit.limited) {
return NextResponse.json(
Expand All @@ -580,8 +594,10 @@ export async function POST(request: Request) {
);
}

const key = publicSearchKey(body);
const { payload, coalesced } = await coalescePublicSearch(key, () => buildPublicSearchPayload(body));
const key = scopedSearchKey(body, ownerId);
const { payload, coalesced } = await coalesceScopedSearch(key, () =>
buildScopedSearchPayload(body, supabase!, ownerId!),
);
return NextResponse.json({
...payload,
telemetry: {
Expand All @@ -590,6 +606,9 @@ export async function POST(request: Request) {
},
});
} catch (error) {
if (error instanceof serverAuth.AuthenticationError) {
return serverAuth.unauthorizedResponse(error);
}
if (error instanceof z.ZodError) {
return jsonError(error, 400);
}
Expand All @@ -603,17 +622,21 @@ export async function POST(request: Request) {
failure_code: code,
},
};
logPublicSearchObservation({
query: "unknown",
results: [],
payload: failurePayload,
failure: {
code,
causeName: error.name,
causeMessage: error.message,
sqlState: extractSqlState(error),
},
});
if (ownerId && supabase) {
logSearchObservation({
supabase,
ownerId,
query: "unknown",
results: [],
payload: failurePayload,
failure: {
code,
causeName: error.name,
causeMessage: error.message,
sqlState: extractSqlState(error),
},
});
}
return jsonError(
new PublicApiError("Search failed. Retry with a narrower question.", 500, {
code,
Expand Down
Loading
Loading