feat: CI overhaul for nightlies and PRs

[redpanda-f]

Part of #72. Readies up CI for nightlies run and reporting to labelled issues in foc-devnet.

Depends on:

• feat: expose YCQL port in devnet-info export #88
• feat: add scenario test framework and prerequisites  #89
• feat: add Latesttag location resolver for init-time tag discovery #83 for resolving latesttag for stability runs
• feat: introduce notty flag for version command #82 for clean version info in reports
• feat: introduce lint.sh and pre-commit hooks for Rust and Python #81 for lint checks

[Copilot]

Pull request overview

This PR refactors the repository’s GitHub Actions CI into a reusable workflow that can be invoked by separate entrypoint workflows for pull requests and nightly runs, with optional scenario-test reporting to GitHub issues.

Changes:

• Convert CI “meat” into a reusable workflow_call workflow with inputs to control dependency versions and reporting.
• Add a PR-triggered workflow that runs linting and then invokes the reusable CI run with default settings.
• Add a nightly workflow (schedule + manual dispatch) that runs a stability/frontier matrix and enables issue reporting.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
.github/workflows/ci_run.yml	Introduces reusable CI run with inputs, caching tweaks, scenario execution, artifact upload, and optional issue reporting.
.github/workflows/ci_pull_request.yml	Adds PR workflow: lint + invoke reusable CI run (default config, reporting off).
.github/workflows/ci_nightly.yml	Adds nightly matrix workflow calling reusable CI run with different init_flags and reporting settings.

Comments suppressed due to low confidence (5)

.github/workflows/ci_run.yml:367

• Node/pnpm versions are set to floating values (node-version: lts/* and pnpm version: latest). This makes CI non-reproducible and can break unexpectedly when upstream releases change; pin these to known-good major/minor versions (or at least a major) to keep runs stable.
  .github/workflows/ci_run.yml:52
• The reusable workflow grants issues: write for the entire job even when enable_reporting is false (e.g., PR runs). Consider reducing token scope by moving issue-writing steps into a separate job/workflow that is only invoked when reporting is enabled, or by setting more restrictive permissions in callers when possible.
  .github/workflows/ci_run.yml:403
• The "Get CI job ID" step runs under if: always() but will still fail the job if the GitHub API call or jq processing errors (it uses curl -sSfL). To avoid flakiness/masking the real failure cause, make this step non-fatal (e.g., continue-on-error: true or handle curl/jq failures and just emit a warning).
  .github/workflows/ci_run.yml:400
• In the "Get CI job ID" step, the value written to $GITHUB_ENV uses $JOB_ID, but that variable is never set. This likely exports an empty GITHUB_CI_JOB_ID, defeating the purpose of resolving the numeric job ID for scenario reports.
  .github/workflows/ci_run.yml:408
• This workflow calls ./scripts/setup-scenarios-prerequisites.sh, but that file is not present in this PR branch (and there is no scripts/ directory). Unless this PR is rebased to include the dependency that adds the scenario prerequisites script, the CI run will fail before scenarios execute.