Add per-tool jq response filters for gateway tool results#5273
Conversation
Agent-Logs-Url: https://github.com/github/gh-aw-mcpg/sessions/7aabaf6a-7e6d-4110-a807-60dc34e41348 Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Agent-Logs-Url: https://github.com/github/gh-aw-mcpg/sessions/7aabaf6a-7e6d-4110-a807-60dc34e41348 Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a per-backend-tool jq “response shaping” feature to the gateway so large/redundant fields can be trimmed from tool results before inline return, payload storage/preview, and schema extraction—primarily to reduce token usage for high-volume GitHub MCP tools.
Changes:
- Introduces
tool_response_filtersper server (TOML + stdin JSON) and validates filters at config load by parsing/compiling jq. - Wires tool registration to wrap handlers with a filter-aware jq/payload middleware when a filter is configured.
- Adds middleware + config tests for filter compilation and basic filtering behavior.
Show a summary per file
| File | Description |
|---|---|
| internal/server/tool_registry.go | Looks up per-server per-tool filter config and chooses filtered vs unfiltered middleware wrapper. |
| internal/middleware/jqschema.go | Adds jq filter compilation/execution helpers and applies filtering in the tool result middleware pipeline. |
| internal/middleware/jqschema_test.go | Adds unit tests for applying filters and for filtered inline responses. |
| internal/config/validation.go | Validates tool_response_filters by parsing/compiling jq during config validation. |
| internal/config/validation_test.go | Tests valid/invalid/empty jq filter validation behavior. |
| internal/config/config_stdin.go | Adds stdin JSON support for tool_response_filters and passes it into ServerConfig. |
| internal/config/config_core.go | Adds ToolResponseFilters to ServerConfig and validates it on TOML load. |
| internal/config/config_core_test.go | Adds TOML load test coverage for tool_response_filters. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 8/8 changed files
- Comments generated: 2
| // stripExtensionFieldsForValidation returns a copy of the raw JSON with known gateway | ||
| // extension fields removed, so the copy can be validated against the upstream MCP Gateway | ||
| // schema. These fields are gateway-specific additions that are not part of the upstream | ||
| // schema definition, so they must be removed before schema validation to prevent spurious | ||
| // "additional properties" errors. | ||
| // | ||
| // Fields stripped: | ||
| // - Top-level "guards": gateway-specific guard configuration | ||
| // - Per-server "guard": reference to a named guard | ||
| // - Per-server "auth": upstream authentication configuration (OIDC etc.) | ||
| // | ||
| // Note: "guard-policies" and "registry" are already injected into the upstream schema | ||
| // by fetchAndFixSchema, so they do not need to be stripped here. | ||
| func stripExtensionFieldsForValidation(data []byte) ([]byte, error) { | ||
| var config map[string]interface{} | ||
| if err := json.Unmarshal(data, &config); err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| // Strip top-level "guards" extension field | ||
| delete(config, "guards") | ||
|
|
||
| // Strip per-server "guard" and "auth" extension fields | ||
| if servers, ok := config["mcpServers"].(map[string]interface{}); ok { | ||
| for _, server := range servers { | ||
| if serverMap, ok := server.(map[string]interface{}); ok { | ||
| delete(serverMap, "guard") | ||
| delete(serverMap, "auth") | ||
| delete(serverMap, "tool_response_filters") | ||
| } |
| } | ||
|
|
||
| filteredData, filterErr := applyCompiledToolResponseFilter(ctx, filterCode, data) | ||
| if filterErr != nil { | ||
| logger.LogWarn("payload", "Failed to apply tool response filter, returning original response: tool=%s, queryID=%s, error=%v", | ||
| toolName, queryID, filterErr) | ||
| return result, data | ||
| } | ||
|
|
||
| filteredResult, convertErr := mcp.ConvertToCallToolResult(filteredData) | ||
| if convertErr != nil { | ||
| logger.LogWarn("payload", "Failed to rebuild filtered tool response, returning original response: tool=%s, queryID=%s, error=%v", | ||
| toolName, queryID, convertErr) | ||
| return result, data | ||
| } | ||
|
|
|
@copilot address the review feedback |
Agent-Logs-Url: https://github.com/github/gh-aw-mcpg/sessions/9eb46543-9c2b-48fc-b346-5a0ee14e6b6d Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Agent-Logs-Url: https://github.com/github/gh-aw-mcpg/sessions/9eb46543-9c2b-48fc-b346-5a0ee14e6b6d Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Addressed in 2a4880b, with final cleanup in f7d01f5. The filter now handles MCP text-envelope payloads by parsing/filtering the first text JSON payload and rewriting both the outbound content and backing data consistently, and I updated the config_stdin comment to document stripping Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
Agent-Logs-Url: https:/-nolocalimports x_amd64/vet go ough x_amd64/compile x_amd64/vet --ve��` (dns block)
|
Some GitHub MCP tool responses carry large repeated fields that dominate token usage, notably
list_code_scanning_alertswhererule.helprepeats full CWE help text on every alert. This change adds a gateway-side, per-tool jq filter so responses can be trimmed before they reach the agent and before payload preview/schema generation runs.What changed
tool_response_filtersto per-server config for both TOML and stdin JSON configs.Config + fail-fast validation
Response pipeline
Result
Example:
That allows responses like:
[ { "number": 101, "rule": { "id": "go/sql-injection", "help": "very long repeated CWE help text" } } ]to be rewritten as:
[ { "number": 101, "rule": { "id": "go/sql-injection" } } ]Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
example.com/tmp/go-build302098926/b509/launcher.test /tmp/go-build302098926/b509/launcher.test -test.testlogfile=/tmp/go-build302098926/b509/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true ache/go/1.25.9/x-p ache/go/1.25.9/xgoogle.golang.org/grpc/internal/metadata x_amd64/asm(dns block)/tmp/go-build3754274398/b513/launcher.test /tmp/go-build3754274398/b513/launcher.test -test.testlogfile=/tmp/go-build3754274398/b513/testlog.txt -test.paniconexit0 -test.timeout=10m0s e=/t�� t0 m0s ash -errorsas -ifaceassert -nilfunc 098926/b514/importcfg -ato�� k/gh-aw-mcpg/gh-aw-mcpg/internal/logger/sanitizego1.25.9 k/gh-aw-mcpg/gh-aw-mcpg/internal/logger/sanitize-c=4 /opt/hostedtoolcache/go/1.25.9/x64/pkg/tool/linu-nolocalimports -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.25.9/x64/pkg/tool/linu/tmp/go-build2478948471/b264/_testmain.go(dns block)/tmp/go-build2307936139/b513/launcher.test /tmp/go-build2307936139/b513/launcher.test -test.testlogfile=/tmp/go-build2307936139/b513/testlog.txt -test.paniconexit0 -test.timeout=10m0s know�� known-linux-gnu/lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libobject-926daa94a00ee327.rlib known-linux-gnu/lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libmemchr-48d5b0db80402653.rlib known-linux-gnu/lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libaddr2line-3367f26bd486b29d.rlib known-linux-gnu/bash known-linux-gnu//usr/bin/runc known-linux-gnu/--version ive.12123747d8da05ed-cgu.00.rcgu.o ive.�� l/middleware/jqschema_test.go ive.12123747d8da05ed-cgu.03.rcgu.o x_amd64/compile ive.12123747d8dabash ive.12123747d8da--norc ive.12123747d8da--noprofile x_amd64/compile(dns block)invalid-host-that-does-not-exist-12345.com/tmp/go-build302098926/b491/config.test /tmp/go-build302098926/b491/config.test -test.testlogfile=/tmp/go-build302098926/b491/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true ternal/wasmdebug-p ternal/wasmdebuggoogle.golang.org/grpc/metadata x_amd64/compile(dns block)/tmp/go-build2478948471/b264/config.test /tmp/go-build2478948471/b264/config.test -test.testlogfile=/tmp/go-build2478948471/b264/testlog.txt -test.paniconexit0 -test.timeout=10m0s 0989�� uf@v1.36.11/reflect/protoreflect-errorsas uf@v1.36.11/reflect/protoreflect-ifaceassert x_amd64/vet --gdwarf-5 ernal/x -o x_amd64/vet rtcf�� 1.80.0/internal/-errorsas 1.80.0/internal/-ifaceassert 64/pkg/tool/linu-nilfunc --gdwarf-5 --64 -o 64/pkg/tool/linu-tests(dns block)/tmp/go-build2144274392/b264/config.test /tmp/go-build2144274392/b264/config.test -test.testlogfile=/tmp/go-build2144274392/b264/testlog.txt -test.paniconexit0 -test.timeout=10m0s .o .o .o .o .o .o 69327bc0f.build_0fe53110ba95802fc3eedafe42b5f4ea0d304545:internal/config/validation.go lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libstd-46d936097e8c5b85.rlib lib/�� lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libobject-926daa94a00ee327.rlib lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libmemchr-48d5b0db80402653.rlib lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libaddr2line-3367f26bd486b29d.rlib lib/rustlib/x86_/bin/sh lib/rustlib/x86_-c lib/rustlib/x86_echo " test-serena - Run Serena MCP Server tests (direct connection)" 05ed-cgu.00.rcgu.o(dns block)nonexistent.local/tmp/go-build302098926/b509/launcher.test /tmp/go-build302098926/b509/launcher.test -test.testlogfile=/tmp/go-build302098926/b509/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true ache/go/1.25.9/x-p ache/go/1.25.9/xgoogle.golang.org/grpc/internal/metadata x_amd64/asm(dns block)/tmp/go-build3754274398/b513/launcher.test /tmp/go-build3754274398/b513/launcher.test -test.testlogfile=/tmp/go-build3754274398/b513/testlog.txt -test.paniconexit0 -test.timeout=10m0s e=/t�� t0 m0s ash -errorsas -ifaceassert -nilfunc 098926/b514/importcfg -ato�� k/gh-aw-mcpg/gh-aw-mcpg/internal/logger/sanitizego1.25.9 k/gh-aw-mcpg/gh-aw-mcpg/internal/logger/sanitize-c=4 /opt/hostedtoolcache/go/1.25.9/x64/pkg/tool/linu-nolocalimports -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.25.9/x64/pkg/tool/linu/tmp/go-build2478948471/b264/_testmain.go(dns block)/tmp/go-build2307936139/b513/launcher.test /tmp/go-build2307936139/b513/launcher.test -test.testlogfile=/tmp/go-build2307936139/b513/testlog.txt -test.paniconexit0 -test.timeout=10m0s know�� known-linux-gnu/lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libobject-926daa94a00ee327.rlib known-linux-gnu/lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libmemchr-48d5b0db80402653.rlib known-linux-gnu/lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libaddr2line-3367f26bd486b29d.rlib known-linux-gnu/bash known-linux-gnu//usr/bin/runc known-linux-gnu/--version ive.12123747d8da05ed-cgu.00.rcgu.o ive.�� l/middleware/jqschema_test.go ive.12123747d8da05ed-cgu.03.rcgu.o x_amd64/compile ive.12123747d8dabash ive.12123747d8da--norc ive.12123747d8da--noprofile x_amd64/compile(dns block)slow.example.com/tmp/go-build302098926/b509/launcher.test /tmp/go-build302098926/b509/launcher.test -test.testlogfile=/tmp/go-build302098926/b509/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true ache/go/1.25.9/x-p ache/go/1.25.9/xgoogle.golang.org/grpc/internal/metadata x_amd64/asm(dns block)/tmp/go-build3754274398/b513/launcher.test /tmp/go-build3754274398/b513/launcher.test -test.testlogfile=/tmp/go-build3754274398/b513/testlog.txt -test.paniconexit0 -test.timeout=10m0s e=/t�� t0 m0s ash -errorsas -ifaceassert -nilfunc 098926/b514/importcfg -ato�� k/gh-aw-mcpg/gh-aw-mcpg/internal/logger/sanitizego1.25.9 k/gh-aw-mcpg/gh-aw-mcpg/internal/logger/sanitize-c=4 /opt/hostedtoolcache/go/1.25.9/x64/pkg/tool/linu-nolocalimports -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.25.9/x64/pkg/tool/linu/tmp/go-build2478948471/b264/_testmain.go(dns block)/tmp/go-build2307936139/b513/launcher.test /tmp/go-build2307936139/b513/launcher.test -test.testlogfile=/tmp/go-build2307936139/b513/testlog.txt -test.paniconexit0 -test.timeout=10m0s know�� known-linux-gnu/lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libobject-926daa94a00ee327.rlib known-linux-gnu/lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libmemchr-48d5b0db80402653.rlib known-linux-gnu/lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libaddr2line-3367f26bd486b29d.rlib known-linux-gnu/bash known-linux-gnu//usr/bin/runc known-linux-gnu/--version ive.12123747d8da05ed-cgu.00.rcgu.o ive.�� l/middleware/jqschema_test.go ive.12123747d8da05ed-cgu.03.rcgu.o x_amd64/compile ive.12123747d8dabash ive.12123747d8da--norc ive.12123747d8da--noprofile x_amd64/compile(dns block)this-host-does-not-exist-12345.com/tmp/go-build302098926/b518/mcp.test /tmp/go-build302098926/b518/mcp.test -test.testlogfile=/tmp/go-build302098926/b518/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true experimental.go internal.go x_amd64/compile -p crypto/ecdh -lang=go1.25 x_amd64/compile -I /opt/hostedtoolcache/go/1.25.9/x-p cfg x_amd64/vet --gdwarf-5 --64 -o x_amd64/vet(dns block)/tmp/go-build3754274398/b522/mcp.test /tmp/go-build3754274398/b522/mcp.test -test.testlogfile=/tmp/go-build3754274398/b522/testlog.txt -test.paniconexit0 -test.timeout=10m0s -ato�� -bool -buildtags bash -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.25.9/x--revs -ato�� -bool -buildtags x_amd64/link -errorsas -ifaceassert -nilfunc x_amd64/link(dns block)/tmp/go-build2307936139/b522/mcp.test /tmp/go-build2307936139/b522/mcp.test -test.testlogfile=/tmp/go-build2307936139/b522/testlog.txt -test.paniconexit0 -test.timeout=10m0s know�� known-linux-gnu/lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libobject-926daa94a00ee327.rlib known-linux-gnu/lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libmemchr-48d5b0db80402653.rlib known-linux-gnu/lib/rustlib/x86_64-REDACTED-linux-gnu/lib/libaddr2line-3367f26bd486b29d.rlib known-linux-gnu/runc jvhawl2nskseooqp--version z8tylr32cgwnziux.0lr3v9a.rcgu.o ruczucboywd9kbz6.0lr3v9a.rcgu.o aiea�� l/server/tool_registry.go p1agk9if6se1ud54.0lr3v9a.rcgu.o fc88gtczrpthgg1u.0lr3v9a.rcgu.o 6uc4ma8f604u2vi6bash 5vmlrtxer9j2lnp9--norc zucrirrh7hnf4z1l--noprofile o4qlefxtp7qruodt.0lr3v9a.rcgu.o(dns block)If you need me to access, download, or install something from one of these locations, you can either: