Log file corruption caused by fcli child processes

[rsenden]

Problem Description

When fcli is configured via FCLI_DEFAULT_* environment variables (e.g., FCLI_DEFAULT_LOG_FILE, FCLI_DEFAULT_LOG_LEVEL), these settings are inherited by child fcli processes spawned during execution. This causes multiple fcli processes to attempt writing to the same log file concurrently, resulting in:

• Log file corruption
• NUL characters (\0) appearing in the log
• Missing or truncated log entries
• Race conditions during file access

Note that this only applies to explicitly spawned processes; it doesn't apply to fcli invocations through FcliCommandExecutor, like run.fcli instructions in fcli actions. These run in the same JVM and share the same log appenders.

Root Cause

1. Parent fcli process sets logging configuration via environment variables
2. Child fcli processes (e.g., fcli --version during version detection, nested fcli tool install commands from actions) inherit these environment variables
3. All processes configure the same log file path
4. Multiple processes writing concurrently to the same file causes corruption

Current Workaround

None - users experiencing this issue must avoid using FCLI_DEFAULT_LOG_FILE when running fcli commands that spawn child processes (e.g., fcli tool fcli run, fcli tool install --copy-from).

Proposed Solutions

Option 1: Don't apply FCLI_DEFAULT_* environment variables in child processes

Maybe the best option, as usually these are meant to set default values for the fcli process that's explicitly invoked by the end user. Two sub-options:

• Pass a filtered set of environment variables to the ProcessBuilder that's used to start the child process, removing all FCLI_DEFAULT_* environment variables
• Have fcli ignore FCLI_DEFAULT_* environment variables if it detects an environment variable named FCLI_DISABLE_DEFAULT_ENV or similar; any ProcessBuilder that's used to start a child process would then be configured to set this environment variable for the child process

Option 2: Disable Logging in Child Processes (Easier)

• Introduce FCLI_DISABLE_LOGGING environment variable
• Set this variable when spawning child fcli processes (e.g., in ToolVersionDetector.tryExecute())
• Have fcli initialization check for this variable and skip all logging configuration if set
• Overrides any other logging configuration (file, level, etc.)

Pros:

• Simple to implement
• Prevents file conflicts
• Works for any nesting level

Cons:

• Lose child process logs entirely
• No visibility into what child processes are doing

Option 3: Aggregate Child Process Logs (More Complex)

Might be useful even if we also implement option 1 or 2:

• Capture log output from child fcli processes
• Insert captured log entries into parent's log file with appropriate markers (e.g., prefixed with child process identifier)
• Recursively works for multi-level nesting

Pros:

• Complete log visibility across all nested invocations
• Single coherent log file with full context
• Better debugging experience

Cons:

• More complex implementation
• Need to somehow collect log data from child process and pass to parent process
• Performance overhead from capturing and re-logging
• Need to handle edge cases (child process crashes, mixed output, etc.)

Option 4: Separate Log Files per Process

• Generate unique log file names for child processes (e.g., append PID or counter)
• Set FCLI_DEFAULT_LOG_FILE to child-specific path when spawning
• Optionally aggregate at end or leave separate

Pros:

• No file conflicts
• All logs preserved
• Easier than Option 2

Cons:

• Multiple log files to manage
• Harder to follow execution flow
• Need cleanup strategy for child log files

Implementation Considerations

• Nesting Depth: Solution must work for arbitrary nesting levels (action → tool install → version detection)
• Concurrency: Even within same JVM, different threads might spawn child processes
• Error Handling: Child process failures shouldn't corrupt parent logs
• Performance: Minimal overhead for spawning child processes
• Backward Compatibility: Don't break existing logging configurations

Code Locations

Key files involved:

• fcli-core/fcli-app/src/main/java/com/fortify/cli/app/runner/util/FortifyCLIDynamicInitializer.java - logging initialization
• fcli-core/fcli-tool/src/main/java/com/fortify/cli/tool/_common/helper/ToolVersionDetector.java - spawns external fcli processes
• fcli-core/fcli-common/src/main/java/com/fortify/cli/common/cli/util/FcliCommandExecutorFactory.java - in-JVM fcli command execution