-
Notifications
You must be signed in to change notification settings - Fork 3
Enhancement test coverage #22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,6 +3,7 @@ performance_results | |
| # Temp files | ||
| */Debug | ||
| build/ | ||
| build_** | ||
| */build/ | ||
| *~ | ||
| */~ | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| # Code Coverage Notes | ||
|
|
||
| ## Coverage Calculation Differences Between CI Environments | ||
|
|
||
| ### Issue Background | ||
|
|
||
| The libgraph CI pipeline initially showed different coverage percentages between Ubuntu 22.04 and 24.04 environments due to different lcov configurations. | ||
|
|
||
| ### Root Cause | ||
|
|
||
| **lcov Version Differences:** | ||
| - Ubuntu 22.04: lcov 1.14 (from package lcov 1.15-1 but reports 1.14 due to known packaging issue) | ||
| - Ubuntu 24.04: lcov 2.1 (from package lcov 2.0-4ubuntu2) | ||
|
|
||
| **Original CI Configuration Differences:** | ||
| - Ubuntu 24.04: Used `--rc geninfo_unexecuted_blocks=1` flag (supported in lcov 2.0+) | ||
| - Ubuntu 22.04: Used basic lcov command without this flag (not supported in lcov 1.x) | ||
|
|
||
| ### Technical Impact | ||
|
|
||
| The `--rc geninfo_unexecuted_blocks=1` flag (available in lcov 2.0+) affects coverage calculation by: | ||
|
|
||
| 1. **Template Instantiation Coverage**: Includes unexecuted template code paths in coverage calculations | ||
| 2. **Inline Function Analysis**: More granular analysis of header-only library inline functions | ||
| 3. **Compiler Optimization Handling**: Different treatment of compiler-optimized or dead-code-eliminated blocks | ||
| 4. **Exception Path Coverage**: Stricter accounting of unreachable exception handling blocks | ||
|
|
||
| ### Solution | ||
|
|
||
| **Version-Aware Configuration (Applied to All Environments):** | ||
| ```bash | ||
| # Check lcov version and use appropriate flags | ||
| LCOV_VERSION=$(lcov --version 2>&1 | grep -oP 'lcov version \K[0-9]+\.[0-9]+' || echo "0.0") | ||
|
|
||
| # lcov 2.0+ supports geninfo_unexecuted_blocks, older versions (1.x) do not | ||
| if [ "$(printf '%s\n' "2.0" "$LCOV_VERSION" | sort -V | head -n1)" = "2.0" ]; then | ||
| # Use enhanced flags for lcov 2.0+ | ||
| lcov --directory ./build --capture --output-file ./build/coverage.info \ | ||
| --rc geninfo_unexecuted_blocks=1 --ignore-errors mismatch,negative,unused | ||
| else | ||
| # Use compatible flags for lcov 1.x | ||
| lcov --directory ./build --capture --output-file ./build/coverage.info \ | ||
| --ignore-errors mismatch,negative,unused | ||
| fi | ||
| ``` | ||
|
|
||
| ### Benefits | ||
|
|
||
| - **Version Compatibility**: CI works correctly on both lcov 1.x and 2.x installations | ||
| - **Consistent Reporting**: Coverage calculations are appropriate for each lcov version | ||
| - **Enhanced Analysis**: lcov 2.0+ environments get more comprehensive template-aware coverage analysis | ||
| - **Robust Error Handling**: Enhanced error handling for various lcov edge cases across versions | ||
| - **Backwards Compatibility**: Maintains functionality on older Ubuntu/lcov installations | ||
|
|
||
| ### Version-Specific Coverage Behavior | ||
|
|
||
| **lcov 2.0+ (Ubuntu 24.04):** | ||
| - More granular template instantiation coverage analysis | ||
| - Stricter accounting of inline functions in header-only libraries | ||
| - Enhanced compiler optimization and dead-code handling | ||
|
|
||
| **lcov 1.x (Ubuntu 22.04):** | ||
| - Traditional coverage analysis suitable for most projects | ||
| - Reliable baseline coverage metrics without advanced template handling | ||
|
|
||
| ### Recommendations for Other Projects | ||
|
|
||
| For C++ projects using CI across multiple Ubuntu versions: | ||
| - Implement version detection to use appropriate lcov flags | ||
| - Consider that lcov 2.0+ provides stricter analysis that may lower coverage percentages | ||
| - Test coverage thresholds should account for version-specific behavior differences | ||
| - Document expected coverage variations between lcov versions | ||
|
|
||
| ### Historical Context | ||
|
|
||
| This version-aware coverage solution was implemented in August 2025 to resolve CI discrepancies between Ubuntu 22.04 (lcov 1.14) and Ubuntu 24.04 (lcov 2.1) environments. The solution automatically detects lcov version and uses appropriate flags to ensure robust coverage reporting across different Ubuntu versions while maximizing analysis quality where possible. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -12,6 +12,8 @@ | |||||||||||||||||||||||||
| #define BFS_HPP | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| #include <limits> | ||||||||||||||||||||||||||
| #include <queue> | ||||||||||||||||||||||||||
| #include <unordered_set> | ||||||||||||||||||||||||||
| #include "graph/search/search_algorithm.hpp" | ||||||||||||||||||||||||||
| #include "graph/search/search_strategy.hpp" | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
|
|
@@ -163,6 +165,114 @@ class BFS final { | |||||||||||||||||||||||||
| SearchContext<State, Transition, StateIndexer> context; | ||||||||||||||||||||||||||
| return Search(graph.get(), context, start, goal); | ||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| /** | ||||||||||||||||||||||||||
| * @brief BFS traversal of all reachable vertices from start | ||||||||||||||||||||||||||
| * | ||||||||||||||||||||||||||
| * Performs breadth-first traversal starting from the given vertex, | ||||||||||||||||||||||||||
| * visiting all vertices reachable from the start vertex. | ||||||||||||||||||||||||||
| */ | ||||||||||||||||||||||||||
| template<typename State, typename Transition, typename StateIndexer, | ||||||||||||||||||||||||||
| typename VertexIdentifier> | ||||||||||||||||||||||||||
| static bool TraverseAll( | ||||||||||||||||||||||||||
| const Graph<State, Transition, StateIndexer>* graph, | ||||||||||||||||||||||||||
| SearchContext<State, Transition, StateIndexer>& context, | ||||||||||||||||||||||||||
| VertexIdentifier start) { | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| if (!graph) return false; | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| auto start_it = graph->FindVertex(start); | ||||||||||||||||||||||||||
| if (start_it == graph->vertex_end()) return false; | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| // Manual BFS traversal | ||||||||||||||||||||||||||
| using VertexIteratorType = decltype(start_it); | ||||||||||||||||||||||||||
| std::queue<VertexIteratorType> q; | ||||||||||||||||||||||||||
| std::unordered_set<int64_t> visited; | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| q.push(start_it); | ||||||||||||||||||||||||||
| visited.insert(start_it->vertex_id); | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| while (!q.empty()) { | ||||||||||||||||||||||||||
| auto current = q.front(); | ||||||||||||||||||||||||||
| q.pop(); | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| // Mark as visited in context | ||||||||||||||||||||||||||
| auto& current_info = context.GetSearchInfo(current); | ||||||||||||||||||||||||||
| current_info.SetChecked(true); | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| for (const auto& edge : current->edges_to) { | ||||||||||||||||||||||||||
| auto neighbor = edge.dst; | ||||||||||||||||||||||||||
| int64_t neighbor_id = neighbor->vertex_id; | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| if (visited.find(neighbor_id) == visited.end()) { | ||||||||||||||||||||||||||
| q.push(neighbor); | ||||||||||||||||||||||||||
| visited.insert(neighbor_id); | ||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
|
||||||||||||||||||||||||||
| // Use unified search logic with a goal predicate that never succeeds | |
| auto goal_predicate = [](const auto&) { return false; }; | |
| auto visit_callback = [&](const auto& vertex_it) { | |
| auto& info = context.GetSearchInfo(vertex_it); | |
| info.SetChecked(true); | |
| }; | |
| // Use BFS strategy | |
| SearchAlgorithm<State, Transition, StateIndexer>::template Search<BfsStrategy<State, Transition, StateIndexer>>( | |
| graph, context, start, goal_predicate, visit_callback); | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The TraverseAll method creates a BFS strategy but doesn't use the dummy_goal iterator correctly. The SearchAlgorithm::Search expects a valid goal iterator, but passing vertex_end() as a goal may not work as intended for traversing all vertices.