ROX-31495: Integration tests by janisz · Pull Request #50 · stackrox/stackrox-mcp

janisz · 2026-03-05T15:58:45Z

Description

Run integration tests against wiremock

Validation

CI + Coverage

github-actions · 2026-03-05T16:04:51Z

E2E Test Results

Commit: 46d7709
Workflow Run: View Details

=== Evaluation Summary ===

  ✓ list-clusters (assertions: 3/3)
  ✓ cve-detected-workloads (assertions: 3/3)
  ✓ cve-detected-clusters (assertions: 3/3)
  ✓ cve-nonexistent (assertions: 3/3)
  ✓ cve-cluster-does-exist (assertions: 3/3)
  ~ cve-cluster-does-not-exist (assertions: 2/3)
      - ToolsUsed: Required tool not called: server=stackrox-mcp, tool=, pattern=list_clusters
  ✓ cve-clusters-general (assertions: 3/3)
  ✓ cve-cluster-list (assertions: 3/3)
  ✓ cve-log4shell (assertions: 3/3)
  ✓ cve-multiple (assertions: 3/3)
  ✓ rhsa-not-supported (assertions: 2/2)

Tasks:      11/11 passed (100.00%)
Assertions: 31/32 passed (96.88%)
Agent used tokens:
  Input:  30462 tokens
  Output: 20362 tokens
Judge used tokens:
  Input:  9174 tokens
  Output: 13778 tokens

Signed-off-by: Tomasz Janiszewski <tomek@redhat.com> Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com> Co-authored-by: Mladen Todorovic <mtodor@gmail.com> Add Go integration tests for MCP server with WireMock Implements integration tests that verify MCP server functionality using stdio transport and WireMock as a mock StackRox Central backend. **Key Changes:** - Created integration test suite in `integration/` with build tag - Implemented stdio-based MCP client in `internal/testutil/mcp.go` - Added WireMock readiness check in `internal/testutil/wiremock.go` - Added test fixtures for expected WireMock response data - Updated Makefile with integration test targets - Updated GitHub Actions workflow to run integration tests in CI **Test Coverage:** - MCP protocol (initialize, list tools) - Tool invocations (list_clusters, get_deployments_for_cve, etc.) - Error handling (missing parameters) - Success and error scenarios Tests use stdio transport for simplicity and better control over the MCP server lifecycle. Each test starts a fresh MCP server subprocess and communicates via JSON-RPC over stdin/stdout. TODO: Fix WireMock request matching for CVE-2021-44228 deployment query Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> fix: Fix WireMock JSONPath patterns for deployment CVE queries Apply the same JSONPath fix from commit 01f58ab to deployments.json. The original mappings used $.query[?(@.query =~ ...)] which looked for a nested array structure, but gRPC protobuf-to-JSON conversion creates a simple object with a 'query' field (lowercase). Changed all deployment CVE mappings from: $.query[?(@.query =~ /.*CVE-XXX.*/)] to: $[?(@.query =~ /.*CVE-XXX.*/)] This fixes the TestIntegration_GetDeploymentsForCVE_Log4Shell test which was failing because WireMock couldn't match the gRPC requests. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> test: Unskip Log4Shell integration test The WireMock JSONPath fix for deployments.json resolves the issue that was causing this test to fail. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> refactor: Simplify integration tests and remove manual protoc installation This commit implements two major simplifications to the test infrastructure: 1. **Eliminate TestMain Pre-compilation Pattern**: - Extract main() body into new internal/app package with Run() function - Tests now call app.Run() in-process via io.Pipe() instead of subprocess - Removes 60+ lines of build/setup code from integration_test.go - Enables full code coverage (previously main.go had 0% coverage) - Faster test execution (no binary compilation overhead) - Better debugging (direct function calls vs exec) 2. **Remove Manual Protoc Installation from GitHub Actions**: - Delete manual protoc 3.20.1 download from workflow - Rely on Makefile's automatic protoc 32.1 installation - Single source of truth for protoc version - Eliminates version mismatch between CI and local dev Changes: - Create internal/app/app.go with Run() function extracted from main() - Update server.Start() to accept optional stdin/stdout parameters - Refactor testutil.NewMCPClient() to use ServerRunFunc callback - Remove TestMain, buildMCPBinary, global vars from integration tests - Update all integration test functions to use createMCPClient() helper - Remove "Install protoc" step from .github/workflows/test.yml Benefits: - 43 net lines removed (-161 +118) - Better code coverage - Simpler test maintenance - Aligned with Go testing best practices Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> fix: Add missing retry/timeout config to integration test setup The test config was missing RequestTimeout, MaxRetries, InitialBackoff, and MaxBackoff fields, causing the gRPC client to use zero values instead of the defaults (30s timeout, 3 retries). With MaxRetries=0, the retry loop never executed: `for attempt := range 0` This caused "Request failed after all retries ... attempts=0" warnings and empty responses from WireMock. Solution: Explicitly set the retry/timeout fields to match the defaults defined in internal/config/config.go. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> Signed-off-by: Tomasz Janiszewski <tomek@redhat.com> Add Go integration tests for MCP server with WireMock Implements integration tests that verify MCP server functionality using stdio transport and WireMock as a mock StackRox Central backend. **Key Changes:** - Created integration test suite in `integration/` with build tag - Implemented stdio-based MCP client in `internal/testutil/mcp.go` - Added WireMock readiness check in `internal/testutil/wiremock.go` - Added test fixtures for expected WireMock response data - Updated Makefile with integration test targets - Updated GitHub Actions workflow to run integration tests in CI **Test Coverage:** - MCP protocol (initialize, list tools) - Tool invocations (list_clusters, get_deployments_for_cve, etc.) - Error handling (missing parameters) - Success and error scenarios Tests use stdio transport for simplicity and better control over the MCP server lifecycle. Each test starts a fresh MCP server subprocess and communicates via JSON-RPC over stdin/stdout. TODO: Fix WireMock request matching for CVE-2021-44228 deployment query Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> fix: Fix WireMock JSONPath patterns for deployment CVE queries Apply the same JSONPath fix from commit 01f58ab to deployments.json. The original mappings used $.query[?(@.query =~ ...)] which looked for a nested array structure, but gRPC protobuf-to-JSON conversion creates a simple object with a 'query' field (lowercase). Changed all deployment CVE mappings from: $.query[?(@.query =~ /.*CVE-XXX.*/)] to: $[?(@.query =~ /.*CVE-XXX.*/)] This fixes the TestIntegration_GetDeploymentsForCVE_Log4Shell test which was failing because WireMock couldn't match the gRPC requests. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> test: Unskip Log4Shell integration test The WireMock JSONPath fix for deployments.json resolves the issue that was causing this test to fail. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> refactor: Simplify integration tests and remove manual protoc installation This commit implements two major simplifications to the test infrastructure: 1. **Eliminate TestMain Pre-compilation Pattern**: - Extract main() body into new internal/app package with Run() function - Tests now call app.Run() in-process via io.Pipe() instead of subprocess - Removes 60+ lines of build/setup code from integration_test.go - Enables full code coverage (previously main.go had 0% coverage) - Faster test execution (no binary compilation overhead) - Better debugging (direct function calls vs exec) 2. **Remove Manual Protoc Installation from GitHub Actions**: - Delete manual protoc 3.20.1 download from workflow - Rely on Makefile's automatic protoc 32.1 installation - Single source of truth for protoc version - Eliminates version mismatch between CI and local dev Changes: - Create internal/app/app.go with Run() function extracted from main() - Update server.Start() to accept optional stdin/stdout parameters - Refactor testutil.NewMCPClient() to use ServerRunFunc callback - Remove TestMain, buildMCPBinary, global vars from integration tests - Update all integration test functions to use createMCPClient() helper - Remove "Install protoc" step from .github/workflows/test.yml Benefits: - 43 net lines removed (-161 +118) - Better code coverage - Simpler test maintenance - Aligned with Go testing best practices Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> fix: Add missing retry/timeout config to integration test setup The test config was missing RequestTimeout, MaxRetries, InitialBackoff, and MaxBackoff fields, causing the gRPC client to use zero values instead of the defaults (30s timeout, 3 retries). With MaxRetries=0, the retry loop never executed: `for attempt := range 0` This caused "Request failed after all retries ... attempts=0" warnings and empty responses from WireMock. Solution: Explicitly set the retry/timeout fields to match the defaults defined in internal/config/config.go. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> Signed-off-by: Tomasz Janiszewski <tomek@redhat.com> cleanup Signed-off-by: Tomasz Janiszewski <tomek@redhat.com> Refactor integration tests and migrate to official MCP SDK This commit refactors the integration test suite and replaces the custom MCP client implementation with the official MCP Go SDK. Changes: 1. Table-Driven Tests Refactoring: - Consolidated 5 individual test functions into 2 table-driven tests - TestIntegration_ToolCalls: 4 successful tool call scenarios - TestIntegration_ToolCallErrors: error handling scenarios - Reduced code duplication by ~50 lines - Added helper functions: setupInitializedClient, callToolAndGetResult 2. Removed TestMain: - Eliminated WireMock readiness check from TestMain - Removed unused imports (fmt, os) - Simplified test setup (13 lines removed) 3. Migrated to Official MCP Go SDK: - Replaced custom internal/testutil/mcp.go (202 lines) - Created internal/testutil/mcp_client.go (141 lines) using SDK - Uses official mcp.Client and mcp.ClientSession - Proper type-safe content handling with *mcp.TextContent - Better error handling (protocol vs tool errors) Benefits: - Total code reduction: 99 lines removed - Better maintainability with table-driven tests - Future-proof with official SDK - Consistent with server-side SDK usage - All tests pass (3 test functions, 5 subtests total) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> fmt Signed-off-by: Tomasz Janiszewski <tomek@redhat.com>

Fixed 17 linter issues across multiple packages: - errcheck: Added error checks for Close() calls - gosec: Added security comment for test utility - lll: Split long function signature - nlreturn: Added blank lines before returns - wrapcheck: Wrapped external package errors with context - wsl: Fixed whitespace requirements - noctx: Used CommandContext instead of Command Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Split the Start function into smaller helper methods: - startStdio: handles stdio transport setup - startHTTP: handles HTTP server lifecycle This reduces the Start function from 61 to 11 lines, resolving the funlen linter error (max 60 lines). Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Signed-off-by: Tomasz Janiszewski <tomek@redhat.com>

codecov-commenter · 2026-03-09T17:42:55Z

Codecov Report

❌ Patch coverage is 8.41121% with 98 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.88%. Comparing base (9bf4946) to head (46d7709).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
internal/testutil/mcp_client.go	0.00%	65 Missing ⚠️
internal/app/app.go	16.00%	21 Missing ⚠️
internal/server/server.go	33.33%	9 Missing and 1 partial ⚠️
cmd/stackrox-mcp/main.go	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #50      +/-   ##
==========================================
- Coverage   78.37%   73.88%   -4.49%     
==========================================
  Files          27       30       +3     
  Lines        1216     1302      +86     
==========================================
+ Hits          953      962       +9     
- Misses        223      300      +77     
  Partials       40       40

Flag	Coverage Δ
integration	`73.88% <8.41%> (?)`
unit	`73.88% <8.41%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mtodor · 2026-03-12T16:40:49Z

internal/app/app.go

+)
+
+// getToolsets initializes and returns all available toolsets.
+func getToolsets(cfg *config.Config, c *client.Client) []toolsets.Toolset {


I think it would be ok to export this function.

Suggested change

func getToolsets(cfg *config.Config, c *client.Client) []toolsets.Toolset {

func GetToolsets(cfg *config.Config, c *client.Client) []toolsets.Toolset {

mtodor · 2026-03-12T16:46:38Z

cmd/stackrox-mcp/main_test.go

 )

+// getToolsets initializes and returns all available toolsets.
+func getToolsets(cfg *config.Config, c *client.Client) []toolsets.Toolset {


I think, we can delete TestGetToolsets, because getToolsets is moved to app.

Should we export getToolsets in app?

mtodor · 2026-03-12T16:49:56Z

internal/app/app_test.go

+	"github.com/stretchr/testify/require"
+)
+
+func TestGetToolsets(t *testing.T) {


We could just copy TestGetToolsets from main_test.go.

mtodor · 2026-03-12T16:52:25Z

internal/app/app.go

+
+// Run executes the MCP server with the given configuration and I/O streams.
+// If stdin/stdout are nil, os.Stdin/os.Stdout will be used.
+// This function is extracted from main() to allow tests to run the server in-process.


nitpick: I think, this is not relevant information

Suggested change

// This function is extracted from main() to allow tests to run the server in-process.

mtodor · 2026-03-12T16:57:41Z

internal/app/app.go

+	registry := toolsets.NewRegistry(cfg, getToolsets(cfg, stackroxClient))
+	srv := server.NewServer(cfg, registry)
+
+	err = stackroxClient.Connect(ctx)


Before, we have also used cancelable context here. Is there a reason why we are not doing that anymore?

mtodor · 2026-03-12T17:08:43Z

integration/fixtures.go

+
+package integration
+
+// Log4ShellFixture contains expected data from log4j_cve.json fixture.


nitpick: It's not easy to understand what file is refered here. It would be good to provide full path: wiremock/fixtures/deployments/log4j_cve.json

The same can be applied for AllClustersFixture comment.

mtodor · 2026-03-12T17:11:27Z

integration/integration_test.go

+}
+
+// createMCPClient is a helper function that creates an MCP client with the test configuration.
+func createMCPClient(t *testing.T) (*testutil.MCPTestClient, error) {


nitpick: let's group helper functions at the top of file.

mtodor · 2026-03-12T17:24:27Z

internal/testutil/mcp_client.go

+func RequireNoError(t *testing.T, result *mcp.CallToolResult) {
+	t.Helper()
+
+	if result.IsError {


nitpick: Let's reduce nesting levels:

if !result.IsError { return }

mtodor · 2026-03-12T17:25:34Z

internal/testutil/command.go

+)
+
+// RunCommand executes a shell command and returns the output and error.
+func RunCommand(command string) (string, error) {


I didn't find where are we using this. Could it be some leftover?

mtodor · 2026-03-12T17:31:58Z

integration/integration_test.go

+}
+
+// getTextContent extracts text from the first content item.
+func getTextContent(t *testing.T, result *mcp.CallToolResult) string {


We have some helpers in testutils, some are here. It would be good to unify them.

Change getToolsets to GetToolsets to make it available for use in test files outside the app package. Addresses PR #50 review comment #1 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Remove duplicate getToolsets function from main_test.go and use the exported app.GetToolsets instead. Also remove the duplicate TestGetToolsets test which will be enhanced in app_test.go. Addresses PR #50 review comment #2 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Replace simple test with comprehensive version that verifies specific toolset names are present. This test was previously in cmd/stackrox-mcp/main_test.go and is now in the proper location alongside the GetToolsets function. Addresses PR #50 review comment #3 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Remove implementation detail comment about stdin/stdout behavior that clutters the high-level API documentation. Addresses PR #50 review comment #4 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Remove parenthetical note that adds unnecessary context noise to the documentation. Addresses PR #50 review comment #6 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Remove parenthetical note that adds unnecessary context noise to the documentation. Addresses PR #50 review comment #7 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Remove parenthetical note that adds unnecessary context noise to the documentation. Addresses PR #50 review comment #8 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Add complete file paths to make it easier to locate the actual fixture files without searching. Includes paths for both Log4ShellFixture and AllClustersFixture. Addresses PR #50 review comment #9 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Reorganize helper functions to the top of the file after imports. New order: imports → helpers (createTestConfig, createMCPClient, setupInitializedClient, callToolAndGetResult, getTextContent) → tests. This improves code organization and readability. Addresses PR #50 review comment #10 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Reduce nesting level by using early return for the success case. This improves readability by handling the happy path first. Addresses PR #50 review comment #11 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Remove internal/testutil/command.go which contains the unused RunCommand function. Verified by searching the codebase - no actual usage found in any test or production code. Addresses PR #50 review comment #12 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Move cancellable context creation to the beginning of Run() so that the entire server lifecycle, including the client connection, can be properly cancelled. This allows graceful shutdown to interrupt the connection phase if needed. The signal handler is also set up earlier to ensure interrupts are caught as soon as possible. Addresses PR #50 review comment #5 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Move all integration test helper functions from integration package to internal/testutil/integration_helpers.go to avoid duplication and centralize test utilities. This improves code organization and makes helpers reusable across different test packages. Helpers moved: - createTestConfig -> CreateIntegrationTestConfig - createMCPClient -> CreateIntegrationMCPClient - setupInitializedClient -> SetupInitializedClient (made generic) - callToolAndGetResult -> CallToolAndGetResult - getTextContent -> GetTextContent The SetupInitializedClient is now generic and accepts a client creation function, making it reusable for different test scenarios. Addresses PR #50 review comment #13 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

janisz force-pushed the integration_tests branch from 42b1ca1 to 1d35a3d Compare March 6, 2026 14:26

janisz and others added 3 commits March 6, 2026 15:45

fix

630b3da

Signed-off-by: Tomasz Janiszewski <tomek@redhat.com>

janisz changed the title ~~Integration tests~~ ROX-31495: Integration tests Mar 9, 2026

janisz added 2 commits March 9, 2026 15:51

fix

e911b97

Signed-off-by: Tomasz Janiszewski <tomek@redhat.com>

fix

8550a0c

Signed-off-by: Tomasz Janiszewski <tomek@redhat.com>

janisz requested a review from mtodor March 9, 2026 17:27

mtodor reviewed Mar 12, 2026

View reviewed changes

janisz and others added 13 commits March 17, 2026 13:47

fix: export GetToolsets function for reusability

07f2802

Change getToolsets to GetToolsets to make it available for use in test files outside the app package. Addresses PR #50 review comment #1 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

docs: remove irrelevant stdin/stdout comment from app.go

f3c3f8e

Remove implementation detail comment about stdin/stdout behavior that clutters the high-level API documentation. Addresses PR #50 review comment #4 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

docs: remove '(production mode)' annotation from server.go

4aadc98

Remove parenthetical note that adds unnecessary context noise to the documentation. Addresses PR #50 review comment #6 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

docs: remove '(for testing)' annotation from server.go

8e73225

Remove parenthetical note that adds unnecessary context noise to the documentation. Addresses PR #50 review comment #7 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

docs: remove '(production)' annotation from server.go

46bb0fb

Remove parenthetical note that adds unnecessary context noise to the documentation. Addresses PR #50 review comment #8 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

refactor: simplify RequireNoError with early return pattern

8b90a9b

Reduce nesting level by using early return for the success case. This improves readability by handling the happy path first. Addresses PR #50 review comment #11 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ROX-31495: Integration tests#50

ROX-31495: Integration tests#50
janisz wants to merge 19 commits intomainfrom
integration_tests

janisz commented Mar 5, 2026

Uh oh!

github-actions bot commented Mar 5, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented Mar 9, 2026 •

edited

Loading

Uh oh!

mtodor Mar 12, 2026

Uh oh!

mtodor Mar 12, 2026

Uh oh!

mtodor Mar 12, 2026

Uh oh!

mtodor Mar 12, 2026

Uh oh!

mtodor Mar 12, 2026

Uh oh!

mtodor Mar 12, 2026

Uh oh!

mtodor Mar 12, 2026

Uh oh!

mtodor Mar 12, 2026

Uh oh!

mtodor Mar 12, 2026

Uh oh!

mtodor Mar 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	func getToolsets(cfg config.Config, c client.Client) []toolsets.Toolset {
	func GetToolsets(cfg config.Config, c client.Client) []toolsets.Toolset {


		package integration

		// Log4ShellFixture contains expected data from log4j_cve.json fixture.

Conversation

janisz commented Mar 5, 2026

Description

Validation

Uh oh!

github-actions bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Test Results

Uh oh!

codecov-commenter commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Mar 5, 2026 •

edited

Loading

codecov-commenter commented Mar 9, 2026 •

edited

Loading