HYPERFLEET-488 - test: Add hyperfleet e2e CI job doc

Author: yingzhanredhat

hyperfleet/test-release/add-hyperfleet-e2e-ci-job-in-prow.md

@@ -0,0 +1,165 @@
+# Add Hyperfleet E2E CI Job in Prow
+
+
+**Metadata**
+- **Date:** 2026-01-27
+- **Authors:** Ying Zhang
+
+
+## Overview
+
+This document provides a comprehensive guide for adding and managing Hyperfleet Continuous Integration (CI) jobs in Prow. It covers job configuration, step registry setup, monitoring, triggering, and debugging procedures specific to HyperFleet E2E CI test workflows.
+
+## Implementation Steps
+
+### Job Configuration
+
+#### Init the job configuration in Prow
+
+For detailed instructions on initializing job configuration in Prow, please refer to the [Initial Setup](add-job-configuration-in-prow.md#initial-setup) section.
+
+#### Adding Step Registry Content
+
+According to the description in the [Test Step Registry](add-job-configuration-in-prow.md#test-step-registry) section, it needs to add the following files.
+
+For the Hyperfleet E2E test, create a new folder **e2e** under [openshift-hyperfleet](https://github.com/openshift/release/tree/master/ci-operator/step-registry/openshift-hyperfleet). The step files should be:
+```text
+ - openshift-hyperfleet-e2e-commands.sh
+ - openshift-hyperfleet-e2e-ref.yaml
+ - openshift-hyperfleet-e2e-ref.metadata.json
+ - OWNERS
+```
+
+#### Step Parameters Configuration
+
+**openshift-hyperfleet-e2e-ref.yaml**
+- Declaring step parameters: More detailed usage can refer to the [official doc](https://docs.ci.openshift.org/docs/architecture/step-registry/#declaring-step-parameters)
+  - Credentials  config: The secret has been added according to the [doc](prow-vault-access-management.md). For CI job, it requires gcloud credential to get the GKE cluster credential to deploy. We prepared a SA **hyperfleet-e2e** for it and store the credential under hyperfleet-e2e folder in Vault, so please don't delete it.
+  - Image: **hyperfleet-e2e** that is built from [Dockerfile](https://github.com/openshift/release/blob/master/ci-operator/config/openshift-hyperfleet/hyperfleet-e2e/openshift-hyperfleet-hyperfleet-e2e-main__e2e.yaml) 
+
+ ```yaml
+ref:
+  as: openshift-hyperfleet-e2e
+  from: hyperfleet-e2e
+  grace_period: 10m
+  commands: openshift-hyperfleet-e2e-commands.sh
+  resources:
+    requests:
+      cpu: 100m
+      memory: 300Mi
+  timeout: 1h0m0s
+  env:
+  - name: # environment variable name
+    default: # (optional) the value assigned if none is provided
+    documentation: # (optional) a textual description of the parameter. Markdown supported. 
+  credentials:
+  - collection: ""
+    namespace: test-credentials
+    name: hyperfleet-e2e
+    mount_path: /var/run/hyperfleet-e2e
+ ```
+
+**openshift-hyperfleet-e2e-commands.sh**
+- Define the shell script for the job. It depends on different business logic.
+  - In above step, it mounts the volume for credential. And the hcm-hyperfleet-e2e.json secret has been added in Vault, so it can get the value directly in the shell script
+
+```bash
+<Other steps>
+# HYPERFLEET_E2E_PATH is defined in above environment variable
+GCP_CREDENTIALS_FILE="${HYPERFLEET_E2E_PATH}/hcm-hyperfleet-e2e.json" 
+
+```
+
+**openshift-hyperfleet-e2e-ref.metadata.json**
+- Store the owners and path
+
+```json
+{
+  "path": "openshift-hyperfleet/e2e/openshift-hyperfleet-e2e-ref.yaml",
+  "owners": {
+    "approvers": [
+      "..."
+    ],
+    "reviewers": [
+      "..."
+    ]
+  }
+}
+```
+
+### Job URLs and Monitoring
+
+#### Finding Running Jobs
+
+To find and monitor running jobs on Prow:
+1. Navigate to the [Prow dashboard](https://prow.ci.openshift.org/).
+2. Use the filter bar to search for specific jobs:
+   - By job name: `*-hyperfleet-hyperfleet-e2e-main-e2e-*`
+   - By status: Add `&state=pending` or `&state=success` or `&state=failure` to the URL
+3. Click on any job to view detailed logs and execution information
+
+### Triggering Jobs
+
+#### Manual Trigger in Prow Dashboard
+
+According to the above step, find the job, then click the button under **Rerun** to trigger the job.
+
+#### Manual Trigger in a PR via GitHub Comment
+
+If you want to add some new code for the CI workflow, you can prepare a PR and trigger the job in the PR with the changed code via comment:
+
+```text
+/pj-rehearse <job_name>
+```
+
+### Creating Additional Jobs
+
+To add another job based on the existing CLM CI job:
+
+1. **Copy the existing job configuration**: Update the following:
+   ```yaml
+   - as: hyperfleet-e2e-nightly # job name
+     cron: 30 9 * * * # cron time
+     steps:
+       env:
+         <key>: <value>
+       test:
+       - ref: openshift-hyperfleet-e2e # step/chain/workflow name
+   ```
+
+2. **Modify the job name and parameters**
+   - Update the `as` field
+   - Adjust `cron` schedule (for periodic jobs)
+   - Modify environment variables as needed
+   - Update step registry references if different
+
+3. **Add the new job**
+
+   Run the command to generate and update the job configuration:
+   ```bash
+   make jobs
+   ```
+
+4. **Submit a PR with the new job configuration**
+
+### Debugging
+
+#### Viewing Job Logs
+
+1. Follow the [Finding Running Jobs](#finding-running-jobs) steps to locate the job
+2. Click the job link to jump to the detailed job page
+3. Click **Artifacts** to jump to the log page
+4. Navigate to the folder: **artifacts/hyperfleet-e2e-nightly/openshift-hyperfleet-e2e/**
+5. You can find the detailed logs under the **artifacts** folder
+
+
+## CI Jobs Example PRs
+
+- Initial CI jobs structure: [PR #73127](https://github.com/openshift/release/pull/73127)
+- Added hyperfleet-e2e test steps to CI Prow job: [PR #73973](https://github.com/openshift/release/pull/73973)
+
+## References
+
+- [Prow Documentation](https://docs.ci.openshift.org/)
+- [Step Registry Documentation](https://docs.ci.openshift.org/docs/architecture/step-registry/)
+- [OpenShift HyperFleet Repository](https://github.com/openshift/release/tree/master/ci-operator/step-registry/openshift-hyperfleet)