-
Notifications
You must be signed in to change notification settings - Fork 1
Expand file tree
/
Copy pathintegration-testing.yaml
More file actions
206 lines (175 loc) · 8.3 KB
/
integration-testing.yaml
File metadata and controls
206 lines (175 loc) · 8.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
# Integration Testing Code Review Guidelines
# Comprehensive rules for test isolation, external dependencies, data management, environment, and performance
description: "Code review guidelines for integration testing covering test isolation, external dependencies, data management, and best practices"
globs:
- "**/*integration*"
- "**/*e2e*"
- "**/*test*"
- "**/*spec*"
- "**/*.test.ts"
- "**/*.test.js"
- "**/*_test.go"
- "**/*Test.java"
- "**/*IT.java"
- "**/*_spec.rb"
- "**/test_*.py"
rules:
# ============================================================================
# TEST ISOLATION RULES
# ============================================================================
- name: ensure-database-cleanup
description: >
Each test should clean up database state. Use transactions with rollback,
truncate tables, or delete test data after each test. Tests must not
leave data that affects other tests. Order-independent tests are essential.
severity: high
- name: use-test-containers
description: >
Use containerized databases and services (Testcontainers) for consistent
test environments. Containers ensure tests run against real services
without affecting shared environments. Each test run gets fresh instances.
severity: medium
- name: ensure-parallel-safety
description: >
Design tests to run safely in parallel. Use unique identifiers for test
data (UUIDs, timestamps). Avoid shared resources that cause conflicts.
Parallel tests speed up CI significantly but require careful design.
severity: high
- name: isolate-test-environments
description: >
Integration tests should not affect production or shared environments.
Use dedicated test databases, queues, and services. Never run tests
against production systems. Environment configuration must be separate.
severity: critical
- name: reset-external-state
description: >
Reset state of external services between tests: clear caches, reset
queues, restore files. External state accumulation causes flaky tests.
Automate state reset in test fixtures.
severity: high
# ============================================================================
# EXTERNAL DEPENDENCIES RULES
# ============================================================================
- name: use-service-mocking-strategically
description: >
Mock external services when: they're unreliable, slow, costly, or testing
specific responses. Use real services when: testing actual integration,
contract compliance, or realistic behavior. Document mocking decisions.
severity: medium
- name: implement-contract-testing
description: >
Use contract testing (Pact, Spring Cloud Contract) to verify integrations
without running actual services. Consumer-driven contracts ensure API
compatibility. Run contract tests in CI to catch breaking changes early.
severity: medium
- name: use-appropriate-test-doubles
description: >
Choose test doubles appropriately: use fakes for complex behavior,
stubs for simple responses, mocks when verifying interactions. Wire
mocks (MockServer, WireMock) for HTTP services. Match double type to need.
severity: medium
- name: handle-external-service-failures
description: >
Test behavior when external services fail, timeout, or return errors.
Integration tests should verify resilience patterns (retries, fallbacks,
circuit breakers). Don't only test happy paths.
severity: high
- name: manage-api-credentials-securely
description: >
Never commit API credentials in test code. Use environment variables,
secrets management, or test-specific credentials. Rotate test credentials
separately from production. Audit credential usage.
severity: critical
# ============================================================================
# DATA MANAGEMENT RULES
# ============================================================================
- name: use-test-data-factories
description: >
Use factory patterns (Factory Bot, ObjectMother) for creating test data.
Factories provide consistent, customizable data creation. Centralize
default values. Avoid duplicating data setup across tests.
severity: medium
- name: implement-database-seeding
description: >
Use database seeding for reference data needed by many tests. Seed
once at suite start, not per test. Keep seed data minimal and well-
documented. Version seed data with migrations.
severity: medium
- name: define-cleanup-strategies
description: >
Define clear cleanup strategy: transaction rollback (fastest), truncate
tables (thorough), delete specific records (targeted). Choose based on
test isolation needs and performance requirements.
severity: high
- name: avoid-production-data
description: >
Never use production data in tests. Generate synthetic data or anonymize
if production patterns needed. Production data has legal, privacy, and
security implications. Document data requirements.
severity: critical
- name: handle-test-data-dependencies
description: >
Manage test data dependencies explicitly. Create required parent records
before child records. Use factories that handle relationships. Document
data dependency chains.
severity: medium
# ============================================================================
# ENVIRONMENT RULES
# ============================================================================
- name: externalize-configuration
description: >
Use environment variables or config files for test environment settings.
No hardcoded hosts, ports, or credentials. Support different environments
(local, CI, staging). Document required configuration.
severity: high
- name: manage-secrets-properly
description: >
Use secrets management for test credentials. CI systems should inject
secrets securely. Never log secrets. Use different credentials for
test vs production. Rotate test credentials regularly.
severity: critical
- name: handle-network-considerations
description: >
Account for network latency, DNS resolution, and connectivity in tests.
Use appropriate timeouts. Handle network partitions in distributed tests.
CI runners may have different network characteristics.
severity: medium
- name: ensure-ci-cd-compatibility
description: >
Integration tests must run reliably in CI/CD pipelines. Avoid dependencies
on local resources. Handle Docker-in-Docker if needed. Monitor CI-specific
flakiness. Tests passing locally but failing in CI indicate environment issues.
severity: high
# ============================================================================
# PERFORMANCE RULES
# ============================================================================
- name: optimize-test-suite-speed
description: >
Keep integration test suite reasonably fast. Set time budgets per test.
Profile slow tests. Consider splitting into fast and slow suites. Long
test runs discourage running tests and slow feedback.
severity: medium
- name: implement-selective-testing
description: >
Support running subset of tests based on changed code. Tag tests by
feature/component. CI should run affected tests on PRs, full suite on merge.
Balance coverage with speed.
severity: medium
- name: cache-expensive-setup
description: >
Cache expensive setup operations (container startup, data loading) across
tests when safe. Use setup once patterns for suite-level fixtures. Balance
caching with isolation requirements.
severity: low
- name: parallelize-test-execution
description: >
Run independent tests in parallel. Use test framework parallelization
features. Ensure tests are parallel-safe before enabling. Parallel
execution can dramatically reduce total test time.
severity: medium
- name: use-appropriate-timeouts
description: >
Set appropriate timeouts for integration tests. Too short causes flakiness,
too long delays failure detection. Consider network latency and load.
Document timeout reasoning.
severity: medium