flowey: implement nix shell command wrapper#2825
flowey: implement nix shell command wrapper#2825justus-camp-microsoft merged 2 commits intomicrosoft:mainfrom
Conversation
038fc65 to
826744c
Compare
There was a problem hiding this comment.
Pull request overview
This PR implements a command wrapper system for flowey that enables wrapping shell commands in nix-shell --pure --run <cmd>. The implementation introduces FloweyShell and FloweyCmd as thin wrappers around xshell that support transparent command transformation while preserving all command builder methods (args, env vars, stdin, flags).
Changes:
- Adds new shell abstraction layer (
flowey_core/src/shell.rs) with FloweyShell, FloweyCmd, and CommandWrapperKind - Updates pipeline infrastructure to support per-job command wrappers with serialization through pipeline.json
- Modifies shell_cmd! macro to return FloweyCmd instead of xshell::Cmd
- Demonstrates usage in build_igvm.rs pipeline by wrapping commands with nix-shell
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| flowey/flowey_core/src/shell.rs | New shell abstraction module with FloweyShell/FloweyCmd wrappers and CommandWrapperKind enum; includes comprehensive test suite (16 tests) |
| flowey/flowey_core/src/pipeline.rs | Adds command_wrapper field to JobData and set_command_wrapper() method to PipelineJob |
| flowey/flowey_core/src/node.rs | Updates RustRuntimeServices to use FloweyShell instead of xshell::Shell; updates shell_cmd! macro to wrap commands |
| flowey/flowey_core/src/lib.rs | Exports new shell module |
| flowey/flowey/src/lib.rs | Re-exports shell module from flowey_core |
| flowey/flowey_cli/src/pipeline_resolver/generic.rs | Adds command_wrapper to ResolvedPipelineJob structure |
| flowey/flowey_cli/src/pipeline_resolver/direct_run.rs | Applies command wrapper to runtime services when executing jobs locally |
| flowey/flowey_cli/src/pipeline_resolver/ado_yaml.rs | Stores command wrapper in pipeline static DB for ADO backend |
| flowey/flowey_cli/src/pipeline_resolver/github_yaml/mod.rs | Stores command wrapper in pipeline static DB for GitHub backend |
| flowey/flowey_cli/src/cli/exec_snippet.rs | Retrieves and applies command wrapper from pipeline static DB when executing snippets; adds job_command_wrappers field to FloweyPipelineStaticDb |
| flowey/flowey_cli/src/pipeline_resolver/viz.rs | Ignores command_wrapper field in visualization destructuring patterns |
| flowey/flowey_hvlite/src/pipelines/build_igvm.rs | Example usage: wraps build-igvm job commands with NixShell referencing shell.nix |
| /// Wrap commands with `nix-shell --pure --run "..."`. | ||
| NixShell { | ||
| /// Optional path to a `shell.nix` file. If `None`, nix-shell | ||
| /// uses its default discovery (looking for `shell.nix` / | ||
| /// `default.nix` in the current directory). | ||
| path: Option<std::path::PathBuf>, | ||
| }, |
There was a problem hiding this comment.
The NixShell wrapper uses --pure flag which clears the environment before running the command. This means that environment variables set via FloweyCmd.env() will not be visible to the command running inside nix-shell, which could be surprising to users. Consider documenting this limitation in the doc comment, or providing guidance on how to pass environment variables into nix-shell (e.g., via shell.nix configuration or --keep flag).
826744c to
1b68d38
Compare
| pub fn ignore_status(mut self) -> Self { | ||
| self.ignore_status = true; |
There was a problem hiding this comment.
Why do we need ignore_status and set_ignore_status? This came up because I was trying to review the usage of pub for all these methods to make sure they are actually required.
There was a problem hiding this comment.
This is just me shadowing the builder methods on xshell::Cmd. I think we have some usages of this one specifically in the regen pipeline, but those don't use the FloweyShell so this is technically unused at the moment. I think it's unlikely for people to intuitively find underlying xshell methods that are missing from FloweyCmd to try to add them so imo it might be better to just go ahead and implement them all right off the bat (which is what I've done).
This PR enables wrapping commands per-job in flowey in order to wrap call commands in
nix-shell --pure --run <cmd>in some future jobs. Example usage would be (in a pipeline definition):