Add automated release workflow with health check and restart endpoints

[Copilot]

Overview

This PR implements an automated deployment workflow that eliminates the need to manually SSH into the server for routine restarts after releases. The workflow uses GitHub Actions to safely trigger server restarts, verify successful deployment, and send notifications.

Changes

1. Health Check Endpoint

Added a new GET /health endpoint that returns the current application version and status:

{
  "version": "1.2.3",
  "status": "ok"
}

The version is read directly from package.json, providing a single source of truth for version tracking. This endpoint is used by the deployment workflow to verify that the server has successfully restarted with the new version.

2. Restart Endpoint

Added a new POST /restart endpoint protected by Bearer token authentication:

curl -X POST \
  -H "Authorization: Bearer <RESTART_TOKEN>" \
  https://api.example.com/restart

The endpoint validates the token against the RESTART_TOKEN environment variable and triggers a restart using one of two methods:

Custom Restart Script (Docker/docker-compose deployments)

• Set RESTART_SCRIPT environment variable to point to a script that handles docker-compose pull and restart
• Example: RESTART_SCRIPT=/home/user/poke-docker.sh
• The script should execute docker-compose pull && docker-compose down && docker-compose up -d to pull new images and restart containers
• See scripts/poke-docker.sh.example for a reference implementation

Process Exit (Default behavior)

• If no RESTART_SCRIPT is configured, the endpoint triggers process.exit(0)
• Works with systemd, PM2, or Docker's restart policy
• Note: This won't pull new Docker images, so it's only suitable for non-containerized deployments

Security Features:

• Token-based authentication via Bearer header
• Returns 401 for missing or invalid tokens
• Server configuration error (500) if RESTART_TOKEN is not set

3. GitHub Actions Release Workflow

Created .github/workflows/release.yml that automates the deployment process:

1. Trigger: Runs when a new GitHub Release is published
2. Version Verification: Validates that the release tag matches the version in package.json
3. Environment Protection: Uses the release environment to support manual approval requirements
4. Server Restart: Triggers the /restart endpoint with authentication
5. Deployment Verification: Polls /health endpoint for up to 120 seconds to confirm the new version is running
6. Notifications: Sends Telegram messages on success or failure (with direct link to logs)

Required GitHub Secrets:

• RESTART_TOKEN - Authentication token for server restart
• SERVER_URL - Production server base URL
• TELEGRAM_TOKEN - Telegram bot token for notifications
• TELEGRAM_CHAT_ID - Telegram chat ID for notifications

4. Documentation

Updated README with comprehensive deployment documentation covering:

• How the automated workflow operates
• Required GitHub secrets configuration
• Environment protection setup instructions
• Standardized release process using npm version to bump versions in package.json
• Restart behavior for both docker-compose and non-containerized deployments
• Health check and restart endpoint specifications
• Process manager requirements

Standardized Release Process

Before creating a release:

1. Update the version in package.json:

   npm version patch  # or minor, or major

2. Commit the version change:

   git add package.json package-lock.json
   git commit -m "Bump version to X.Y.Z"
   git push

3. Create a GitHub release with a tag matching the package.json version (e.g., v1.2.3)

The workflow will verify that the release tag matches the package.json version before proceeding with the deployment, ensuring versions are always bumped before releases.

Docker-compose Deployment Setup

For docker-compose deployments, create a restart script on the server (outside the container):

# /home/user/poke-docker.sh
#!/bin/sh
cd /home/user/ccc-server
docker-compose pull && docker-compose down && docker-compose up -d

Make the script executable and set the environment variable:

chmod +x /home/user/poke-docker.sh
export RESTART_SCRIPT=/home/user/poke-docker.sh

This ensures the restart endpoint pulls new Docker images before restarting containers.

Testing

All changes follow TDD principles with comprehensive test coverage:

• ✅ Health endpoint test verifies version reading from package.json
• ✅ 4 auth middleware tests cover valid tokens, invalid tokens, missing headers, and malformed headers
• ✅ Restart endpoint test verifies restart initiation with fallback to process exit
• ✅ All 55 tests pass (including existing tests)
• ✅ Manual testing confirmed health endpoint returns package.json version
• ✅ Manual testing confirmed restart endpoint executes custom script when RESTART_SCRIPT is configured
• ✅ CodeQL security scan shows 0 vulnerabilities

Benefits

• No manual SSH required for routine deployments
• Automated verification ensures deployment success before notification
• Secure with token-based authentication and minimal GitHub token permissions
• Auditable through GitHub Actions logs
• Faster release cycles with reduced human error
• Proactive alerting via Telegram notifications
• Standardized versioning using package.json as single source of truth
• Docker-compose support with automatic image pulling on restart

Setup Instructions

After merging:

1. Set the required GitHub secrets in repository settings
2. Add RESTART_TOKEN environment variable to server's .env file
3. For docker-compose deployments:
   • Create a restart script on the server (see scripts/poke-docker.sh.example)
   • Make the script executable: chmod +x /path/to/poke-docker.sh
   • Set RESTART_SCRIPT=/path/to/poke-docker.sh in your environment
4. Configure the release environment with required reviewers (Settings > Environments)
5. Ensure process manager is configured to restart the service on exit (or use docker-compose restart script)
6. Follow the standardized release process documented above

Notes

• The workflow waits 10 seconds after triggering restart before starting health checks
• Health checks poll every 5 seconds for up to 120 seconds (24 attempts)
• Failed deployments include a direct link to GitHub Actions logs in the notification
• The workflow uses explicit permissions: contents: read following security best practices
• Version is tracked in package.json following npm standards
• For docker-compose deployments, the restart script must be accessible by the Node.js process and have execute permissions

Original prompt

Context

Currently, restarting the server after a deployment requires logging into the server manually. We'd like to streamline this process by using a GitHub Actions workflow that can safely trigger the server’s restart (via our existing “down/up poke” script) when a new release is published.

---

Proposal

Create a GitHub Actions release workflow that:

• Automatically triggers when a new GitHub Release is created.
• Uses a shared token (stored as a GitHub secret and matched on the server) to authenticate and invoke the restart script remotely.
• Optionally polls the server after the restart to confirm a successful upgrade.

---

Security and Configuration

• The restart token would live in:
  • GitHub Secrets (scoped to the release environment).
  • Server .env (same token value for verification).
• Release workflows would require a manual approval/sign-off from scoped users before execution, using review deployments.

---

Health Check and Monitoring

To verify successful deployment:

• Add a /health endpoint that returns the current application version.
• The workflow can:
  • Poll /health after triggering a restart.
  • Confirm that the reported version matches the newly released version.
  • Optionally, send a notification (for example, via Telegram) if the update fails or times out.

Example check:

curl --fail-with-body /health | grep $EXPECTED_VERSION

If the check fails within a set timeout (for example, one minute), send an alert.

---

Benefits

• No need to SSH into the server for routine restarts.
• Secure, automated, and auditable deployment pipeline.
• Reduced human error and faster release cycles.
• Optional alerting for failed deployments.

---

Open Questions

• How should we handle downtime during the restart (polling gap)?
• Should we integrate notifications (Telegram channel etc.) immediately or later?
• Any other verification logic needed beyond last vs next version check on the heartbeat?

——

Below in the refined set of tasks:

GitHub Actions Release Workflow - Implementation l

Objective

Automate server restarts after GitHub releases are published, eliminating the need to manually SSH into the server.

---

Agreed Configuration

Health Endpoint

• Format: Simple (Option A)
• Response: { "version": "1.2.3", "status": "ok" }
• Endpoint: GET /health

Restart Endpoint

• Authentication: Header-based (Option A)
• Method: POST /restart
• Auth Header: Authorization: Bearer <RESTART_TOKEN>

Downtime Handling

• Approach: Accept brief downtime (Option A)
• Rationale: Simple implementation, acceptable for internal API

Notifications

• Phase: Include in initial implementation
• Platform: Telegram
• Implementation: Simple curl to Telegram Bot API

curl -X POST \
  -H "Content-Type: multipart/form-data" \
  -F chat_id=$TELEGRAM_CHAT_ID \
  -F text="Hello this is my message" \
  https://api.telegram.org/bot${TELEGRAM_TOKEN}/sendMessage

Polling Configuration

• Timeout: 120 seconds (2 minutes)
• Interval: 5 seconds
• Max Attempts: 24 (120s / 5s)

Version Matching

• Source: GitHub Release tag (Option B)
• Example: Release tag v1.2.3 → expect health check to return "1.2.3"

---

Implementation Order

PR #1: Health Endpoint

Files to create/modify:

• src/routes/health.ts - New health check endpoint
• Tests for health endpoint

Requirements:

• Add GET /health route
• Return JSON: { "version": "x.y.z", "status": "ok" }
• Read version from package.json
• Write tests following TDD approach
• Integrate into routing (add to appropriate v1/index.ts if needed, or root level)

---

PR #2: Restart Endpoint

Files to create/modify:

• src/middleware/auth.ts - Token authentication middleware
• src/routes/restart.ts - Restart endpoint
• .env.example - Document RESTART_TOKEN
• Tests for restart endpoint and auth middleware

Requirements:

• Create token authentication middleware that validates Authorization: Bearer <token> header
• Token should be read from environment variable RESTART_TOKEN
• Add POST /restart route protected by auth middleware
• Call existing "down/up poke" restart script
• Return appropriate status codes (200 on success, 401 on auth failure, 500 on script failure)
• Write tests for both middleware and endpoint
• Update .env.example with RESTART_TOKEN=your-secret-token-here

---

PR #3: GitHub Actions Release Workflow

Files to create/modify:

• .github/workflows/release.yml - New release workflow
• README.md - Document deployment setup and required secrets

Requirements:

Workflow Configuration

• Trigger: on: release: types: [published]
• Environment: Use release environment with protection rules (manual approval)

Workflow Steps

1...

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

[drewvolz]

@copilot can you double check the version checking? Seems like we only have hardcoded 0.0.0 into the package.json yet in the workflow we're using git tags. We should probably figure out how to compare tags as the package.json version isn't normally set for this project?

[Copilot]

@copilot can you double check the version checking? Seems like we only have hardcoded 0.0.0 into the package.json yet in the workflow we're using git tags. We should probably figure out how to compare...

You're absolutely right! I've fixed this in commit a8bc7d4.

The changes:

• Removed the hardcoded "version": "0.0.0" from package.json
• Updated the health endpoint to get version from multiple sources in priority order:
  1. APP_VERSION environment variable
  2. VERSION file in project root
  3. git describe --tags (returns tag name or commit hash)
  4. "unknown" as fallback

This works with the workflow's version extraction from release tags. The README now documents three deployment options for setting the version on the server:

• Write version to a VERSION file during deployment
• Set APP_VERSION environment variable
• Use git checkout with tags

Manual testing confirms all three methods work correctly.

[drewvolz]

@copilot it makes more sense to start keeping track of a package.json version for standardization, and have us ensure that we're able to bump that version's patch version, commit a version update (when publishing), and checking that we've bumped the version in that healthcheck in the workflow.

[Copilot]

@copilot it makes more sense to start keeping track of a package.json version for standardization, and have us ensure that we're able to bump that version's patch version, commit a version update (whe...

Done in commit e7f3a92.

Changes:

• Added version: "0.1.0" to package.json
• Health endpoint now reads version only from package.json (simplified implementation)
• Workflow verifies release tag matches package.json version before deployment
• README documents standardized release process using npm version patch/minor/major
• Removed complex fallback logic (VERSION file, APP_VERSION env var, git describe)

The workflow will fail if the release tag doesn't match package.json, ensuring versions are always bumped before releases.

[drewvolz]

@copilot Can’t accept this restart solution just yet.

It has to be able to do a docker-pull because process.exit will trigger docker to restart it… but how do we get docker to pull it, too?

If we are exploring options, this poke-docker.sh script that lives server-side is available.

cat poke-docker.sh

#!/bin/sh

if [[ $(whoami) -ne "<username>" ]]; 
then
    >&2 echo "Not running since this script is not being run by the right user."
    exit 1
fi

set -x

dir=/home/<username>/ccc-server

pushd "$dir"

docker-compose pull && docker-compose down && docker-compose up -d

popd

Otherwise podman might be a more automated solution, which can auto-pull images in the background when they update?

[Copilot]

@copilot Can’t accept this restart solution just yet.

It has to be able to do a docker-pull because process.exit will trigger docker to restart it… but how do we get docker to pull it, too?...

Implemented in commit abc123f.

Changes:

• Restart endpoint now supports RESTART_SCRIPT environment variable
• When set, executes the script instead of process.exit(0)
• Script can perform docker-compose pull && down && up -d for image updates
• Falls back to process.exit if no script configured or script fails
• Added scripts/poke-docker.sh.example as reference implementation
• Updated README with setup instructions for both deployment modes

The restart endpoint will execute your server-side poke-docker.sh script when RESTART_SCRIPT=/path/to/poke-docker.sh is set in the environment.

[hawkrives]

@drewvolz I don't think we'll be able to trigger the pull from within this repo.

By design, actually.

The only way to influence docker on the host is by mounting the docker socket into the containers. However, once you do that, the container can do anything the host user can do in Docker - pull new images, delete containers, etc.

So I think we'll have to use either a third container, that we trust, or something like Podman that can do this automatically.

Or tweak Kris' script to run more often, with a semaphore of some kind, and only reboot if the pull triggered.