BLDX-432: YAML-driven datasource fixtures for E2E testing#1040
Open
fyzanshaik-atlan wants to merge 9 commits intomainfrom
Open
BLDX-432: YAML-driven datasource fixtures for E2E testing#1040fyzanshaik-atlan wants to merge 9 commits intomainfrom
fyzanshaik-atlan wants to merge 9 commits intomainfrom
Conversation
Add a fixtures package under test_utils/e2e/ that provides a reusable abstraction for managing data source lifecycle in E2E tests. This moves container orchestration (testcontainers), readiness checks, and env var injection out of individual apps and into the SDK. - DataSourceFixture ABC with setup/teardown/is_ready/get_env_vars contract - ContainerizedDataSourceFixture for Docker-backed sources (postgres, mysql, etc.) - HostedDataSourceFixture for non-containerizable sources (Snowflake, Redshift) - Pytest plugin (pytest11 entry point) for session-scoped lifecycle management - Optional testcontainers extra in pyproject.toml - Unit tests for all fixture classes and the plugin
fyzanshaik-atlan
requested review from
OnkarVO7,
cmgrote and
stefan-atlan
as code owners
📜 Docstring Coverage ReportRESULT: PASSED (minimum: 30.0%, actual: 77.9%) Detailed Coverage Report |
📦 Trivy Vulnerability Scan Results
Report SummaryCould not generate summary table (data length mismatch: 9 vs 8). Scan Result Detailsrequirements.txtuv.lock |
📦 Trivy Secret Scan Results
Report Summary
Scan Result Details✅ No secrets found during the scan for |
Collaborator
☂️ Python Coverage
Overall Coverage
New FilesNo new covered files... Modified FilesNo covered modified files...
|
|
🛠 Full Test Coverage Report: https://k.atlan.dev/coverage/application-sdk/pr/1040 |
|
🛠 Docs available at: https://k.atlan.dev/application-sdk/feat/datasource-fixtures |
📦 Example workflows test results
|
📦 Example workflows test results
|
📦 Example workflows test results
|
Replace the class-based ContainerizedDataSourceFixture and HostedDataSourceFixture with a declarative YAML-driven approach. App developers now declare a datasource.yaml instead of writing Python fixture subclasses. New modules: - schema.py: Pydantic models for YAML config validation - readiness.py: TCP readiness check for container startup - containerized.py: ContainerizedFixture using testcontainers GenericContainer - loader.py: Factory function to parse YAML and return fixture - plugin.py: Simplified pytest plugin activated via config._datasource_yaml Removes HostedFixture entirely — hosted databases use existing CI secret injection + expandvars pattern without SDK involvement.
fyzanshaik-atlan
changed the title
fyzanshaik-atlan
changed the title
Use tmp_path fixture instead of hardcoded Unix paths so tests pass on Windows where Path.resolve() prepends the drive letter.
…e positives TCP connect() alone can succeed when Docker's port proxy accepts the connection before the actual service is listening inside the container. The check now does a 1-byte recv() after connect to verify the service is actually holding the connection open.
…tials Replace hardcoded username/password/database/extra fields in ConnectionInfo with a single credentials dict pass-through from YAML. Env vars are now generated dynamically for all credential keys, uppercased automatically. This makes the fixture system work for any connector type (databases, dashboards, pipelines) without assuming a fixed credential shape.
fyzanshaik-atlan
added
the
e2e-testcontainers
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
fyzanshaik-atlan
force-pushed
the
feat/datasource-fixtures
branch
from
9072b32 to
f1f5dc6
Compare
fyzanshaik-atlan
removed
the
e2e-testcontainers
Contributor
Author
|
@greptile review |
Greptile SummaryThis PR introduces a declarative YAML-driven datasource fixture system for E2E testing, enabling app developers to provision containerized databases/services without writing Python code. Key changes:
Issues found:
Test coverage:
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| application_sdk/test_utils/e2e/fixtures/base.py | Defines ConnectionInfo and DataSourceFixture ABC with clean abstractions; wait_until_ready properly captures last error |
| application_sdk/test_utils/e2e/fixtures/schema.py | Pydantic models for YAML config with proper defaults and field descriptions; clean schema design |
| application_sdk/test_utils/e2e/fixtures/readiness.py | TCP readiness check with recv() to distinguish Docker proxy from real service; handles all connection states correctly |
| application_sdk/test_utils/e2e/fixtures/containerized.py | Container lifecycle management with volume mounting and env var generation; missing exception handling in teardown |
| application_sdk/test_utils/e2e/fixtures/loader.py | YAML loader factory with proper error handling for missing files and validation errors |
| application_sdk/test_utils/e2e/fixtures/plugin.py | Pytest plugin for fixture lifecycle; missing exception handling for setup failures could prevent cleanup |
Flowchart
flowchart TD
A[App's pytest_configure] -->|Sets _datasource_yaml| B[pytest_sessionstart hook]
B --> C[load_fixture_from_yaml]
C --> D[Parse YAML with Pydantic]
D --> E[Create ContainerizedFixture]
E --> F[fixture.setup]
F --> G[Start Docker container]
G --> H[Extract host:port]
H --> I[Create ConnectionInfo]
I --> J[wait_until_ready]
J --> K{check_tcp with recv}
K -->|Ready| L[Inject env vars to os.environ]
K -->|Not Ready| M[Retry with interval]
M --> K
L --> N[Store connection_info on config]
N --> O[Run tests]
O --> P[pytest_sessionfinish hook]
P --> Q[fixture.teardown]
Q --> R[Stop container]
R --> S[Clean up resources]
Last reviewed commit: b8b73d9
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+33
to
+38
| _active_fixture = load_fixture_from_yaml(yaml_path) | ||
| connection_info = _active_fixture.setup() | ||
|
|
||
| for key, value in _active_fixture.get_env_vars().items(): | ||
| os.environ[key] = value | ||
|
|
There was a problem hiding this comment.
Missing exception handling for fixture setup failures. If setup() or get_env_vars() raises an exception, the test session will fail to start but _active_fixture will still be set, preventing proper cleanup in pytest_sessionfinish().
Suggested change
| _active_fixture = load_fixture_from_yaml(yaml_path) | |
| connection_info = _active_fixture.setup() | |
| for key, value in _active_fixture.get_env_vars().items(): | |
| os.environ[key] = value | |
| try: | |
| _active_fixture = load_fixture_from_yaml(yaml_path) | |
| connection_info = _active_fixture.setup() | |
| for key, value in _active_fixture.get_env_vars().items(): | |
| os.environ[key] = value | |
| session.config._datasource_connection_info = connection_info # type: ignore[attr-defined] | |
| except Exception: | |
| if _active_fixture is not None: | |
| _active_fixture.teardown() | |
| _active_fixture = None | |
| raise |
Comment on lines
+74
to
+78
| def teardown(self) -> None: | ||
| """Stop and remove the container.""" | ||
| if self._container is not None: | ||
| self._container.stop() | ||
| self._container = None |
There was a problem hiding this comment.
Missing exception handling during container teardown. If stop() fails, container resources may leak and _container will remain set.
Suggested change
| def teardown(self) -> None: | |
| """Stop and remove the container.""" | |
| if self._container is not None: | |
| self._container.stop() | |
| self._container = None | |
| def teardown(self) -> None: | |
| """Stop and remove the container.""" | |
| if self._container is not None: | |
| try: | |
| self._container.stop() | |
| except Exception: | |
| pass # Best-effort cleanup | |
| finally: | |
| self._container = None | |
| self._connection_info = None |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Declarative YAML-driven datasource fixture system for E2E tests. App developers declare a
datasource.yamlwith image, port, env, volumes, and credentials — SDK handles container provisioning, readiness checks, env var injection, and teardown.Design
ConnectionInfoholdshost,port(from Docker), and a free-formcredentialsdict (from YAML). No hardcoded database fields — works for any connector type (databases, dashboards, pipelines).credentials: {api_token: x}withenv_prefix: E2E_SUPER→E2E_SUPER_API_TOKEN=x.ContainerizedFixturefor custom readiness → full customDataSourceFixtureas escape hatch.Module layout
Usage
Test plan