feat: add Grafana Mimir metrics infrastructure

[gsanchietti]

📋 Description

Integrates Grafana Mimir (metrics storage) and Grafana (dashboards) into the MY platform, enabling NethServer/NethSecurity systems to push Prometheus metrics via remote_write with per-organization isolation.

Architecture

NethServer  →  POST /mimir/api/v1/push  →  nginx  →  backend /api/mimir/*
                Basic Auth: system_key:system_secret          │
                                                              │ X-Scope-OrgID = organization_id
                                                              ▼
                                                           Mimir (pserv, 2-node cluster)

Browser  →  /grafana/  →  nginx  →  Grafana  →  backend /api/mimir  →  Mimir

Changes

Backend:

• backend/methods/mimir.go — wildcard proxy handler (ANY /api/mimir/*): HTTP Basic Auth (system_key + system_secret), Argon2id secret verification, X-Scope-OrgID injection, streaming reverse proxy with load balancing

Metrics stack:

• dedicated metrics VM: mimir1 + mimir2 (cluster)
• dedicated VM for grafana
• data, ruler and alertmanager configuration go to remote s3; only local per-tenant limits are deployed inside the mimir instance

Configuration (already in main):

• proxy/nginx.conf + nginx.conf.local — /mimir/ (→ backend, proxy_request_buffering off) and /grafana/ location blocks

Testing

Check for readiness:

curl -v -X POST -u NETH-xxx:yyy https://my-collect-qa-pr-41.onrender.com/api/services/mimir/ready

Write:

curl -v -X POST -u NETH-xxx:yyy https://my-collect-qa-pr-41.onrender.com/api/services/mimir/api/v1/push

Query:

curl -v  -u NETH-xxx:yyy 'https://my-collect-qa-pr-41.onrender.com/api/services/mimir/prometheus/api/v1/label/__name__/values?limit=40000&start=1771484100&end=1771487700

Grafana dashboard: https://my-grafana-qa.onrender.com/grafana

TODO

Possibile improvements:

• choose a set of labels for the hosts to identify them inside the queries, proposed:
  • hostname
  • system_key
• prepare common alert rules
• make sure to send few data from the hosts
• evaluate Grafana Oauth authentication with provisioned organizations (each organization can see only its own tenant datasource)
• expose mimir UI with authentication to evaluate cluster status and expose metrics to other prometheus
• switch to modern prometheus data format on host that sends data using remote write
• fix mimir and grafana versions so deployed vm will be reproducible

NethServer 8 integration

Changes for ns8-metrics:

diff -u ../bin/provision-prometheus.ori ../bin/provision-prometheus
--- ../bin/provision-prometheus.ori	2026-02-17 08:03:26.111372197 +0000
+++ ../bin/provision-prometheus	2026-02-18 16:23:52.234003166 +0000
@@ -63,6 +63,19 @@
         fp.write('        - localhost:9093\n')
         fp.write('rule_files:\n')
         fp.write('  - "/prometheus/rules.d/*.yml"\n')
+        fp.write('remote_write:\n')
+        fp.write('  - url: https://my-collect-qa-pr-41.onrender.com/api/services/mimir/api/v1/push\n')
+        fp.write('    basic_auth:\n')
+        fp.write('      username: NETH-xxx\n')
+        fp.write('      password: my_xxx\n')
+        fp.write('    write_relabel_configs:\n')
+        fp.write('      - action: replace\n')
+        fp.write('        target_label: system_key\n')
+        fp.write('        replacement: aa-bb-cc\n')
+        fp.write('      - action: replace\n')
+        fp.write('        target_label: hostname\n')
+        fp.write('        replacement: demo-heron2\n')
 
 def validate_and_generate_provider_configs(redis_client):
     for nkey in redis_client.scan_iter("node/*/vpn"):

NethSecurity integration

It requires telegraf binary with http and prometheusremotewrite plugins.

Config for telegraph:

[global_tags]

[agent]
  interval = "10s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = "0s"

[[inputs.cpu]]
  percpu = true
  totalcpu = true
  collect_cpu_time = false
  report_active = false
  core_tags = false

[[inputs.disk]]
  ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"]

[[inputs.diskio]]

[[inputs.kernel]]

[[inputs.mem]]

[[inputs.processes]]

[[inputs.system]]

[[outputs.http]]
  url = "http://mimir.gs.nethserver.net:9009/api/v1/push"
  data_format = "prometheusremotewrite"
  [outputs.http.headers]
     Content-Type = "application/x-protobuf"
     Content-Encoding = "snappy"
     X-Prometheus-Remote-Write-Version = "0.1.0"
     X-Scope-OrgID = "11"

🚀 Testing Environment

To trigger a fresh deployment of all services in the PR preview environment, comment:

update deploy

Automatic PR environments:

• Backend: https://my-backend-qa-pr-41.onrender.com
• Collect: https://my-collect-qa-pr-41.onrender.com
• Frontend: https://my-frontend-qa-pr-41.onrender.com
• Proxy: https://my-proxy-qa-pr-41.onrender.com

✅ Merge Checklist

Code Quality:

• 
• 
• 
• 

Builds:

• 
• 
• 
• 

[github-actions]

🔗 Redirect URIs Added to Logto

The following redirect URIs have been automatically added to the Logto application configuration:

Redirect URIs:

• https://my-frontend-qa-pr-41.onrender.com/login-redirect
• https://my-proxy-qa-pr-41.onrender.com/login-redirect

Post-logout redirect URIs:

• https://my-frontend-qa-pr-41.onrender.com/login
• https://my-proxy-qa-pr-41.onrender.com/login

These will be automatically removed when the PR is closed or merged.

[github-actions]

🤖 My API structural change detected

Preview documentation

Structural change details

Added (4)

• DELETE /api/services/mimir/{path}
• GET /api/services/mimir/{path}
• POST /api/services/mimir/{path}
• PUT /api/services/mimir/{path}

Powered by Bump.sh

[gsanchietti]

Replaced by #42

[github-actions]

🗑️ Redirect URIs Removed from Logto

The following redirect URIs have been automatically removed from the Logto application configuration:

Redirect URIs:

• https://my-frontend-qa-pr-41.onrender.com/login-redirect
• https://my-proxy-qa-pr-41.onrender.com/login-redirect

Post-logout redirect URIs:

• https://my-frontend-qa-pr-41.onrender.com/login
• https://my-proxy-qa-pr-41.onrender.com/login

Cleanup completed for PR #41.