feat(usage): add cache analytics support

PossibleRefactoring.md

@@ -0,0 +1,95 @@
+# Possible Refactoring
+
+Ordered by lowest effort and lowest risk first.
+
+## 1. Remove dead `CacheTypeBoth`
+
+Effort: very low
+Risk: very low
+
+Why:
+- Defined in `internal/responsecache/semantic.go`.
+- No call sites found in the repo.
+
+How verified:
+- Symbol searched: `CacheTypeBoth`
+- Command: `rg -n "CacheTypeBoth"`
+
+Suggested action:
+- Delete the constant and let tests confirm nothing depended on it.
+
+## 2. Deduplicate the dashboard's empty `cacheOverview` object
+
+Effort: low
+Risk: very low
+
+Why:
+- The same shape is repeated in:
+  - `internal/admin/dashboard/static/js/dashboard.js`
+  - `internal/admin/dashboard/static/js/modules/usage.js`
+  - `internal/admin/dashboard/static/js/modules/execution-plans.js`
+
+Suggested action:
+- Keep a single `emptyCacheOverview()` factory and reuse it everywhere.
+
+## 3. Pick one owner for "cache overview is cached-only"
+
+Effort: low
+Risk: low
+
+Why:
+- The handler sets `CacheModeCached` in `internal/admin/handler.go`.
+- Each reader sets it again in:
+  - `internal/usage/reader_sqlite.go`
+  - `internal/usage/reader_postgresql.go`
+  - `internal/usage/reader_mongodb.go`
+- `GetCacheOverview()` already implies cached-only behavior.
+
+Suggested action:
+- Keep the override in one place only.
+- Prefer reader ownership so the behavior stays correct regardless of caller.
+
+## 4. Remove the legacy `ResponseCacheMiddleware.Middleware()` path
+
+Effort: medium
+Risk: medium
+
+Why:
+- Production flow now uses `HandleRequest()` from `internal/server/translated_inference_service.go`.
+- `.Middleware()` in `internal/responsecache/responsecache.go` is only referenced by tests.
+
+How verified:
+- Symbols searched: `Middleware()` and `HandleRequest(`
+- Commands:
+  - `rg -n "\\.Middleware\\(\\)" | sort`
+  - `rg -n "HandleRequest\\(" | sort`
+
+Suggested action:
+- Before deleting the compatibility wrapper, keep equivalent cache-hit and cache-miss coverage around `HandleRequest()`.
+- Existing tests in `internal/responsecache/handle_request_test.go` already cover core hit/miss flows and should be expanded first if wrapper-specific assertions are still needed.
+- Delete the compatibility wrapper.
+- Only remove `internal/responsecache/middleware_test.go` after `HandleRequest()`-level coverage fully preserves the hit/miss, response header/status, and cache population assertions currently carried by the middleware wrapper tests.
+
+## 5. Centralize cache-type vocabulary across packages
+
+Effort: medium to high
+Risk: medium
+
+Why:
+- Overlapping cache constants and normalization logic exist in:
+  - `internal/usage/cache_type.go`
+  - `internal/auditlog/auditlog.go`
+  - `internal/responsecache/semantic.go`
+- This increases the chance of drift when new cache types or modes are added.
+
+Suggested action:
+- Introduce a small shared internal package for cache semantics.
+- Do it only if it can be done without creating import cycles.
+
+## Recommended order
+
+1. Remove `CacheTypeBoth`.
+2. Deduplicate the dashboard empty-state object.
+3. Keep cached-only policy in one layer.
+4. Remove the legacy middleware path.
+5. Centralize cache semantics in a shared package.

internal/admin/dashboard/static/js/dashboard.js

@@ -46,6 +46,18 @@ function dashboard() {
         // Data
         summary: { total_requests: 0, total_input_tokens: 0, total_output_tokens: 0, total_tokens: 0, total_input_cost: null, total_output_cost: null, total_cost: null },
         daily: [],
+        cacheOverview: {
+            summary: {
+                total_hits: 0,
+                exact_hits: 0,
+                semantic_hits: 0,
+                total_input_tokens: 0,
+                total_output_tokens: 0,
+                total_tokens: 0,
+                total_saved_cost: null
+            },
+            daily: []
+        },
         models: [],
         categories: [],
         activeCategory: 'all',

internal/admin/dashboard/static/js/modules/charts.js

@@ -1,33 +1,61 @@
 (function(global) {
     function dashboardChartsModule() {
         return {
-            _overviewChartConfig(colors, labels, inputData, outputData) {
+            _overviewChartConfig(colors, labels, inputData, outputData, cacheInputData, cacheOutputData) {
+                const cacheEnabled = typeof this.cacheAnalyticsEnabled === 'function' && this.cacheAnalyticsEnabled();
+                const datasets = [
+                    {
+                        label: 'Input Tokens',
+                        data: inputData,
+                        borderColor: '#c2845a',
+                        backgroundColor: 'rgba(194, 132, 90, 0.1)',
+                        fill: true,
+                        tension: 0.3,
+                        pointRadius: 3,
+                        pointHoverRadius: 5
+                    },
+                    {
+                        label: 'Output Tokens',
+                        data: outputData,
+                        borderColor: '#7a9e7e',
+                        backgroundColor: 'rgba(122, 158, 126, 0.1)',
+                        fill: true,
+                        tension: 0.3,
+                        pointRadius: 3,
+                        pointHoverRadius: 5
+                    }
+                ];
+                if (cacheEnabled) {
+                    datasets.push(
+                        {
+                            label: 'Local Cache Input Tokens',
+                            data: cacheInputData,
+                            borderColor: '#5f7dcf',
+                            backgroundColor: 'rgba(95, 125, 207, 0.08)',
+                            fill: false,
+                            tension: 0.3,
+                            pointRadius: 2,
+                            pointHoverRadius: 4,
+                            borderDash: [6, 4]
+                        },
+                        {
+                            label: 'Local Cache Output Tokens',
+                            data: cacheOutputData,
+                            borderColor: '#b0677b',
+                            backgroundColor: 'rgba(176, 103, 123, 0.08)',
+                            fill: false,
+                            tension: 0.3,
+                            pointRadius: 2,
+                            pointHoverRadius: 4,
+                            borderDash: [6, 4]
+                        }
+                    );
+                }
                 return {
                     type: 'line',
                     data: {
                         labels: labels,
-                        datasets: [
-                            {
-                                label: 'Input Tokens',
-                                data: inputData,
-                                borderColor: '#c2845a',
-                                backgroundColor: 'rgba(194, 132, 90, 0.1)',
-                                fill: true,
-                                tension: 0.3,
-                                pointRadius: 3,
-                                pointHoverRadius: 5
-                            },
-                            {
-                                label: 'Output Tokens',
-                                data: outputData,
-                                borderColor: '#7a9e7e',
-                                backgroundColor: 'rgba(122, 158, 126, 0.1)',
-                                fill: true,
-                                tension: 0.3,
-                                pointRadius: 3,
-                                pointHoverRadius: 5
-                            }
-                        ]
+                        datasets: datasets
                     },
                     options: {
                         responsive: true,
@@ -182,7 +210,12 @@
                     const labels = filled.map((d) => d.date);
                     const inputData = filled.map((d) => d.input_tokens);
                     const outputData = filled.map((d) => d.output_tokens);
-                    const config = this._overviewChartConfig(colors, labels, inputData, outputData);
+                    const cacheByDate = {};
+                    const cacheDaily = this.fillMissingDays(this.cacheOverview && Array.isArray(this.cacheOverview.daily) ? this.cacheOverview.daily : []);
+                    cacheDaily.forEach((d) => { cacheByDate[d.date] = d; });
+                    const cacheInputData = labels.map((label) => (cacheByDate[label] && cacheByDate[label].input_tokens) || 0);
+                    const cacheOutputData = labels.map((label) => (cacheByDate[label] && cacheByDate[label].output_tokens) || 0);
+                    const config = this._overviewChartConfig(colors, labels, inputData, outputData, cacheInputData, cacheOutputData);
 
                     if (this.chart) {
                         this.chart.destroy();

internal/admin/dashboard/static/js/modules/execution-plans.js

@@ -61,6 +61,7 @@
 	                    'LOGGING_ENABLED',
 	                    'USAGE_ENABLED',
 	                    'GUARDRAILS_ENABLED',
+	                    'CACHE_ENABLED',
 	                    'REDIS_URL',
 	                    'SEMANTIC_CACHE_ENABLED'
 	                ];
@@ -80,6 +81,10 @@
 	            },
 
 	            executionPlanCacheVisible() {
+	                const explicit = this.executionPlanRuntimeFlag('CACHE_ENABLED');
+	                if (explicit !== '') {
+	                    return this.executionPlanRuntimeBooleanFlag('CACHE_ENABLED', false);
+	                }
 	                const redis = this.executionPlanRuntimeFlag('REDIS_URL');
 	                const semantic = this.executionPlanRuntimeFlag('SEMANTIC_CACHE_ENABLED');
 	                if (redis === '' && semantic === '') {
@@ -671,11 +676,32 @@
                         if (payload && typeof payload === 'object' && !Array.isArray(payload) && payload[key] !== undefined && payload[key] !== null) {
                             next[key] = String(payload[key]).trim();
                         }
-                    }
-                    this.executionPlanRuntimeConfig = next;
-                } catch (e) {
-                    console.error('Failed to fetch dashboard config:', e);
-                    this.executionPlanRuntimeConfig = {};
+	                    }
+	                    this.executionPlanRuntimeConfig = next;
+	                    if (typeof this.fetchCacheOverview === 'function') {
+	                        if (this.executionPlanCacheVisible()) {
+	                            this.fetchCacheOverview();
+	                        } else {
+	                            this.cacheOverview = {
+	                                summary: {
+	                                    total_hits: 0,
+	                                    exact_hits: 0,
+	                                    semantic_hits: 0,
+	                                    total_input_tokens: 0,
+	                                    total_output_tokens: 0,
+	                                    total_tokens: 0,
+	                                    total_saved_cost: null
+	                                },
+	                                daily: []
+	                            };
+	                            if (typeof this.renderChart === 'function') {
+	                                this.renderChart();
+	                            }
+	                        }
+	                    }
+	                } catch (e) {
+	                    console.error('Failed to fetch dashboard config:', e);
+	                    this.executionPlanRuntimeConfig = {};
                 } finally {
                     if (timeoutID !== null && typeof clearTimeout === 'function') {
                         clearTimeout(timeoutID);

internal/admin/dashboard/static/js/modules/usage.js

@@ -1,6 +1,27 @@
 (function(global) {
     function dashboardUsageModule() {
         return {
+            emptyCacheOverview() {
+                return {
+                    summary: {
+                        total_hits: 0,
+                        exact_hits: 0,
+                        semantic_hits: 0,
+                        total_input_tokens: 0,
+                        total_output_tokens: 0,
+                        total_tokens: 0,
+                        total_saved_cost: null
+                    },
+                    daily: []
+                };
+            },
+
+            cacheAnalyticsEnabled() {
+                return typeof this.executionPlanRuntimeBooleanFlag === 'function'
+                    ? this.executionPlanRuntimeBooleanFlag('CACHE_ENABLED', false)
+                    : false;
+            },
+
             _usageQueryStr() {
                 if (this.customStartDate && this.customEndDate) {
                     return 'start_date=' + this._formatDate(this.customStartDate) +
@@ -9,6 +30,61 @@
                 return 'days=' + this.days;
             },
 
+            async fetchCacheOverview() {
+                if (!this.cacheAnalyticsEnabled()) {
+                    this.cacheOverview = this.emptyCacheOverview();
+                    if (this.page === 'overview') this.renderChart();
+                    return;
+                }
+
+                const controller = typeof this._startAbortableRequest === 'function'
+                    ? this._startAbortableRequest('_cacheOverviewFetchController')
+                    : null;
+                const options = { headers: this.headers() };
+                if (controller) {
+                    options.signal = controller.signal;
+                }
+
+                try {
+                    let queryStr;
+                    if (this.customStartDate && this.customEndDate) {
+                        queryStr = 'start_date=' + this._formatDate(this.customStartDate) +
+                            '&end_date=' + this._formatDate(this.customEndDate);
+                    } else {
+                        queryStr = 'days=' + this.days;
+                    }
+                    queryStr += '&interval=' + this.interval;
+
+                    const res = await fetch('/admin/api/v1/cache/overview?' + queryStr, options);
+                    if (!this.handleFetchResponse(res, 'cache overview')) {
+                        this.cacheOverview = this.emptyCacheOverview();
+                        return;
+                    }
+                    const payload = await res.json();
+                    if (controller && controller.signal.aborted) {
+                        return;
+                    }
+                    this.cacheOverview = payload && typeof payload === 'object' ? payload : this.emptyCacheOverview();
+                    if (!this.cacheOverview.summary) {
+                        this.cacheOverview.summary = this.emptyCacheOverview().summary;
+                    }
+                    if (!Array.isArray(this.cacheOverview.daily)) {
+                        this.cacheOverview.daily = [];
+                    }
+                    if (this.page === 'overview') this.renderChart();
+                } catch (e) {
+                    if (typeof this._isAbortError === 'function' && this._isAbortError(e)) {
+                        return;
+                    }
+                    console.error('Failed to fetch cache overview:', e);
+                    this.cacheOverview = this.emptyCacheOverview();
+                } finally {
+                    if (typeof this._clearAbortableRequest === 'function') {
+                        this._clearAbortableRequest('_cacheOverviewFetchController', controller);
+                    }
+                }
+            },
+
             async fetchUsage() {
                 const controller = typeof this._startAbortableRequest === 'function'
                     ? this._startAbortableRequest('_usageFetchController')
@@ -48,6 +124,11 @@
                     this.summary = summary;
                     this.daily = daily;
                     this.renderChart();
+                    if (this.cacheAnalyticsEnabled()) {
+                        this.fetchCacheOverview();
+                    } else {
+                        this.cacheOverview = this.emptyCacheOverview();
+                    }
                     if (this.page === 'usage') this.fetchUsagePage();
                     if (this.page === 'audit-logs') this.fetchAuditLog(true);
                 } catch (e) {
@@ -63,7 +144,11 @@
             },
 
             async fetchUsagePage() {
-                await Promise.all([this.fetchModelUsage(), this.fetchUsageLog(true)]);
+                const requests = [this.fetchModelUsage(), this.fetchUsageLog(true)];
+                if (this.cacheAnalyticsEnabled()) {
+                    requests.push(this.fetchCacheOverview());
+                }
+                await Promise.all(requests);
                 this.renderBarChart();
             },