Skip to content

[test-improver] Improve tests for mcptest harness#2631

Merged
lpcox merged 1 commit into
mainfrom
test-improver/mcptest-harness-testify-252a4c365b026523
Mar 27, 2026
Merged

[test-improver] Improve tests for mcptest harness#2631
lpcox merged 1 commit into
mainfrom
test-improver/mcptest-harness-testify-252a4c365b026523

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

Test Improvements: harness_test.go

File Analyzed

  • Test File: internal/testutil/mcptest/harness_test.go
  • Package: internal/testutil/mcptest
  • Lines of Code: 275 → 197 (−78 lines of boilerplate)

Improvements Made

1. Better Testify Assertions

Replaced all manual if-block error checking with idiomatic testify calls:

  • require.NoError(t, err) instead of if err := ...; err != nil { t.Fatalf(...) }
  • require.Len(t, tools, 1) instead of if len(tools) != 1 { t.Errorf(...) }
  • assert.Equal(t, "test_echo", tools[0].Name) instead of nested if tools[0].Name != ...
  • assert.False(t, result.IsError) instead of if result.IsError { t.Error(...) }
  • require.True(t, ok, "Expected *sdk.TextContent") instead of if !ok { t.Error(...) }
  • assert.ElementsMatch(t, []string{"echo1", "echo2", "add"}, toolNames) instead of manual map-lookup loop
  • ✅ Added assert import alongside existing require

2. Increased Coverage

  • Added missing assertion on add tool result: assert.Equal(t, "8", textContent.Text) — the original test called the add tool but never verified the return value
  • require.Len on result.Content before accessing result.Content[0], preventing a potential index panic

3. Cleaner & More Stable Tests

  • ✅ Removed redundant if len(x) > 0 { ... } guards that were only needed because t.Errorf doesn't stop execution (now require.Len stops on failure)
  • ✅ Removed t.Logf("✓ ...") status messages that added noise without testing anything
  • ✅ Cleaner linear test flow without deeply nested conditionals
  • ✅ Consistent err := declaration pattern across all test functions

Why These Changes?

harness_test.go already imported require from testify but performed most assertions via manual if-blocks with t.Errorf/t.Fatalf. This pattern lets tests continue running after a failure, leading to confusing cascaded errors (e.g., an index-out-of-bounds panic after a length mismatch). Converting to testify stops tests at the right boundary, gives clearer failure messages, and removes substantial boilerplate.

The missing assertion on the add tool result was a genuine coverage gap — the test confirmed the tool call succeeded but never checked the computed value.


Generated by Test Improver Workflow
Focuses on better patterns, increased coverage, and more stable tests

Generated by Test Improver ·

- Replace if-block manual checks with require.NoError, require.Len,
  assert.Equal, assert.False, assert.ElementsMatch
- Remove redundant 'if len > 0' guards (now safe after require.Len)
- Remove t.Logf status log messages (not assertion output)
- Add missing assertion on add tool return value ("8")
- Use assert.ElementsMatch for unordered tool name comparison
- Add assert import alongside existing require import

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
lpcox added a commit that referenced this pull request Mar 27, 2026
…ctionPool.Get (#2634)

🤖 *This is an automated pull request from [Repo
Assist](https://github.com/github/gh-aw-mcpg/actions/runs/23636580547),
an AI assistant.*

## Summary

`SessionConnectionPool.Get` used an unsafe read-lock upgrade pattern
that introduced a data race:

```
RLock → read metadata → RUnlock → Lock → write metadata → Unlock → RLock
```

Between the manual `RUnlock` and subsequent `Lock`, another goroutine
could call `Delete` on the same key. If that happened, `Get` would
update and return a connection that had already been removed from the
pool (and potentially closed), violating the caller's assumption that
the returned connection is live.

The Go race detector (`go test -race`) would flag this as a data race on
`metadata.LastUsedAt`, `metadata.RequestCount`, and `metadata.State`.

## Root cause

The original code tried to optimise by using a read lock for the initial
map lookup, upgrading to a write lock only for the three metadata field
writes. This optimisation is **not safe** for `sync.RWMutex` because
Go's `RWMutex` does not support lock upgrading — you must release the
read lock before acquiring the write lock, creating an unavoidable race
window.

## Fix

Replace the entire operation with a single write lock (`p.mu.Lock()` /
`defer p.mu.Unlock()`). This is the standard, correct pattern for
read-then-update operations. Since every successful `Get` also performs
three writes to the metadata struct, the read-lock optimisation provided
minimal benefit and the lock contention difference is negligible.

## Changes

- `internal/launcher/connection_pool.go`: `Get` now uses `p.mu.Lock()` /
`defer p.mu.Unlock()` throughout, removing the manual lock upgrade.
- `internal/launcher/connection_pool_test.go`: Added
`TestConnectionPoolConcurrencyWithDeletes` which runs concurrent `Get`,
`Set`, and `Delete` operations to exercise the previously racy path. Run
with `go test -race` to confirm correctness.

## Trade-offs

The write lock means concurrent reads from different sessions are
serialised during `Get`. In practice this is unlikely to be a bottleneck
— the lock is held only for a map lookup and three field writes
(nanoseconds), and the actual MCP backend communication happens outside
the lock.

## Test Status

> ⚠️ Infrastructure note: the workflow environment cannot download the
Go 1.25.0 toolchain (network access to `proxy.golang.org` is blocked).
Tests could not be executed locally. The code change is mechanically
correct (removing the manual lock dance, keeping identical logic), and
the existing test suite is expected to pass in CI where the toolchain is
available.

Closes N/A (proactive bug fix — no linked issue)




> [!WARNING]
> <details>
> <summary><strong>⚠️ Firewall blocked 1 domain</strong></summary>
>
> The following domain was blocked by the firewall during workflow
execution:
>
> - `proxy.golang.org`
>
> To allow these domains, add them to the `network.allowed` list in your
workflow frontmatter:
>
> ```yaml
> network:
>   allowed:
>     - defaults
>     - "proxy.golang.org"
> ```
>
> See [Network
Configuration](https://github.github.com/gh-aw/reference/network/) for
more information.
>
> </details>


> [!NOTE]
> <details>
> <summary><b>🔒 Integrity filter blocked 14 items</b></summary>
>
> The following items were blocked because they don't meet the GitHub
integrity level.
>
> - #1711 `list_issues`: has lower integrity than agent
requires. The agent cannot read data with integrity below "merged".
> - #2568 `list_issues`: has lower integrity than agent
requires. The agent cannot read data with integrity below "merged".
> - #2626 `list_issues`: has lower integrity than agent
requires. The agent cannot read data with integrity below "merged".
> - #2628 `list_issues`: has lower integrity than agent
requires. The agent cannot read data with integrity below "merged".
> - #2629 `list_issues`: has lower integrity than agent
requires. The agent cannot read data with integrity below "merged".
> - #2630 `list_issues`: has lower integrity than agent
requires. The agent cannot read data with integrity below "merged".
> - #2632 `list_issues`: has lower integrity than agent
requires. The agent cannot read data with integrity below "merged".
> - #2633 `list_issues`: has lower integrity than agent
requires. The agent cannot read data with integrity below "merged".
> - [#2631](#2631)
`list_pull_requests`: has lower integrity than agent requires. The agent
cannot read data with integrity below "merged".
> - [#2577](#2577)
`list_pull_requests`: has lower integrity than agent requires. The agent
cannot read data with integrity below "merged".
> - [#2567](#2567)
`list_pull_requests`: has lower integrity than agent requires. The agent
cannot read data with integrity below "merged".
> - [#1711](#1711)
`search_issues`: has lower integrity than agent requires. The agent
cannot read data with integrity below "merged".
> - [#2278](#2278)
`issue_read`: has lower integrity than agent requires. The agent cannot
read data with integrity below "merged".
> - search_pull_requests `search_pull_requests`: has lower integrity
than agent requires. The agent cannot read data with integrity below
"merged".
>
> To allow these resources, lower `min-integrity` in your GitHub
frontmatter:
>
> ```yaml
> tools:
>   github:
>     min-integrity: approved  # merged | approved | unapproved | none
> ```
>
> </details>


> Generated by [Repo
Assist](https://github.com/github/gh-aw-mcpg/actions/runs/23636580547) ·
[◷](https://github.com/search?q=repo%3Agithub%2Fgh-aw-mcpg+%22gh-aw-workflow-id%3A+repo-assist%22&type=pullrequests)
>
> To install this [agentic
workflow](https://github.com/githubnext/agentics/tree/851905c06e905bf362a9f6cc54f912e3df747d55/workflows/repo-assist.md),
run
> ```
> gh aw add
githubnext/agentics@851905c
> ```

<!-- gh-aw-agentic-workflow: Repo Assist, engine: copilot, model: auto,
id: 23636580547, workflow_id: repo-assist, run:
https://github.com/github/gh-aw-mcpg/actions/runs/23636580547 -->

<!-- gh-aw-workflow-id: repo-assist -->
@lpcox lpcox marked this pull request as ready for review March 27, 2026 17:16
Copilot AI review requested due to automatic review settings March 27, 2026 17:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the internal/testutil/mcptest harness tests to reduce boilerplate and make failures clearer by standardizing on testify assertions.

Changes:

  • Replaced manual if/t.Errorf/t.Fatalf checks with require/assert helpers.
  • Added additional result validation (notably for the add tool) and safer length checks before indexing.
  • Simplified test flow by removing redundant guards and logging noise.
Comments suppressed due to low confidence (1)

internal/testutil/mcptest/harness_test.go:120

  • Same as above: if the tool returns IsError=true, prefer a require assertion so later content/type assertions don’t run and obscure the primary failure.
	require.NoError(t, err, "Failed to call add tool")
	assert.False(t, result.IsError)
	require.Len(t, result.Content, 1, "Expected exactly 1 content item")
	textContent, ok := result.Content[0].(*sdk.TextContent)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

t.Logf("✓ Tool executed correctly: %s", textContent.Text)
}
}
assert.False(t, result.IsError)

Copilot AI Mar 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

result.IsError indicates the tool reported an error; using assert.False allows the test to continue and can produce additional, less relevant failures. Use a require assertion here so the test stops immediately when the tool call reports an error.

This issue also appears on line 117 of the same file.

Suggested change
assert.False(t, result.IsError)
require.False(t, result.IsError)

Copilot uses AI. Check for mistakes.
@lpcox lpcox merged commit e424676 into main Mar 27, 2026
7 checks passed
@lpcox lpcox deleted the test-improver/mcptest-harness-testify-252a4c365b026523 branch March 27, 2026 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants