From 789ec9eefc5c9db5c549f42ccc24066582800d4a Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Thu, 23 Apr 2026 13:48:42 +0200 Subject: [PATCH 01/41] feat(migration): cut appliance over to my collect MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes the migration started by the dual-send PR. After this commit, type=enterprise units talk directly to the my collect API with native my credentials; neither the legacy my.nethesis.it /isa/ and /api/ endpoints nor the /proxy/* translation routes remain on the hot path. type=community units are left on the legacy my.nethserver.com / backupd infrastructure — that is explicitly out of scope for the my migration. Credential rotation (existing enterprise units): - New /usr/sbin/migrate-to-my: idempotent one-shot, gated on type=enterprise. Calls the translation proxy's /proxy/credentials with the legacy Basic-Auth pair, reads back the mapped my system_key/system_secret and atomically rotates ns-plug.config. Preserves the legacy pair under legacy_system_id / legacy_secret for audit and manual rollback, (re)asserts collect_url because /etc/config/ns-plug is a conffile, and sets the migrated='1' marker that stops the helper from running again. - send-heartbeat / send-inventory / send-backup / remote-backup invoke migrate-to-my up front on the enterprise branch so the first successful cron tick flips a pre-migration unit over. Native my registration (fresh enterprise subscriptions): - /usr/sbin/register enterprise branch now POSTs my.nethesis.it/backend/api/systems/register with {system_secret: } and reads back system_key. The unit lands on the new my with its native credentials, collect_url is written alongside the legacy URLs, and migrated='1' is set so migrate-to-my is a no-op. Community register is untouched; it keeps calling my.nethserver.com /api/machine/info. - /usr/sbin/subscription-info enterprise branch reads from collect /info with the rotated credentials and emits a legacy-shaped envelope so ns.subscription info and the existing Vue UI keep parsing the same keys. The new my data model no longer tracks a subscription plan at the system level, so plan_name falls back to the organization name and valid_until is null (the UI treats that as "no expiration"). Single-path send scripts: - send-heartbeat enterprise: POST $collect_url/heartbeat with rotated Basic-Auth. The old my-old /isa/ primary + proxy shadow dual-send is gone; community continues on my.nethserver.com/api/machine/heartbeats/store. - send-inventory enterprise: POST $collect_url/inventory with a phonehome payload. The my-old /isa/ primary, the /api/systems/info registration-date refresh and the proxy shadow are gone; community continues on my.nethserver.com. - send-backup / remote-backup enterprise: single upload to $collect_url/backups with native creds, via remote-backup's upload/download/list/delete (which includes the backup UI mapping). Community remote-backup keeps the legacy $backup_url/$TYPE/api/v2/backup/ layout untouched. Config & packaging: - ns-plug.config.collect_url default added so fresh enterprise images carry the new endpoint; backup_url default restored so fresh community installs keep the legacy backupd URL. - DEPENDS: +jq (used by remote-backup's list parser, migrate-to-my and subscription-info). Failure mode: - A /proxy/credentials outage during an enterprise upgrade window leaves the unit on legacy credentials against collect, which returns 401. migrate-to-my is re-invoked every 10 minutes via send-heartbeat's cron entry, so the unit recovers automatically once the proxy is back up. Accepted trade-off: no dual-mode in the scripts; the simpler single-send path is preferred. --- packages/ns-api/files/ns.backup | 43 ++++--- packages/ns-phonehome/files/phonehome | 12 +- packages/ns-plug/Makefile | 3 +- packages/ns-plug/files/config | 1 + packages/ns-plug/files/migrate-to-my | 91 +++++++++++++++ packages/ns-plug/files/register | 22 +++- packages/ns-plug/files/remote-backup | 139 +++++++++++++++-------- packages/ns-plug/files/send-backup | 6 + packages/ns-plug/files/send-heartbeat | 44 +++++-- packages/ns-plug/files/send-inventory | 67 +++++++---- packages/ns-plug/files/subscription-info | 51 +++++++-- packages/ns-plug/files/unregister | 26 ++++- 12 files changed, 388 insertions(+), 117 deletions(-) create mode 100644 packages/ns-plug/files/migrate-to-my diff --git a/packages/ns-api/files/ns.backup b/packages/ns-api/files/ns.backup index 3cf1631ab..9049030da 100755 --- a/packages/ns-api/files/ns.backup +++ b/packages/ns-api/files/ns.backup @@ -169,21 +169,26 @@ elif cmd == 'call': elif action == 'registered-backup': if not os.path.exists(PASSPHRASE_PATH): - print(utils.validation_error('passphrase', 'missing')) - else: - try: - # create backup - file_name = create_backup() - backup_path = f'{DOWNLOAD_PATH}{file_name}' - # upload backup to server and remove it from filesystem - completed_process = subprocess.run(['/usr/sbin/remote-backup', 'upload', backup_path], check=True, - capture_output=True) - os.remove(backup_path) - print(json.dumps({'message': 'success'})) - except subprocess.CalledProcessError as error: - print(json.dumps(utils.generic_error(f'remote upload failed'))) - except RuntimeError as error: - print(json.dumps(utils.generic_error(error.args[0]))) + # Refuse the call before running sysupgrade/uploading, and emit + # valid JSON so the HTTP API wraps it as a 422 ValidationError + # the UI can render (the previous form printed a Python dict + # repr, which was silently dropped upstream and caused the run + # modal to stay open after a successful upload). + print(json.dumps(utils.validation_error('passphrase', 'missing'))) + sys.exit(0) + try: + # create backup + file_name = create_backup() + backup_path = f'{DOWNLOAD_PATH}{file_name}' + # upload backup to server and remove it from filesystem + completed_process = subprocess.run(['/usr/sbin/remote-backup', 'upload', backup_path], check=True, + capture_output=True) + os.remove(backup_path) + print(json.dumps({'message': 'success'})) + except subprocess.CalledProcessError as error: + print(json.dumps(utils.generic_error(f'remote upload failed'))) + except RuntimeError as error: + print(json.dumps(utils.generic_error(error.args[0]))) elif action == 'registered-restore': try: @@ -224,10 +229,12 @@ elif cmd == 'call': elif action == 'registered-delete-backup': try: data = json.load(sys.stdin) - p = subprocess.run(['/usr/sbin/remote-backup', 'delete', data['id']], + subprocess.run(['/usr/sbin/remote-backup', 'delete', data['id']], check=True, capture_output=True, text=True) - # return content - print(p.stdout) + # The remote side returns a structured JSON response; the UI + # only needs a success flag, matching the pattern of the + # other registered-* handlers (backup, restore). + print(json.dumps({'message': 'success'})) except subprocess.CalledProcessError as error: print(json.dumps(utils.generic_error('remote backup delete failed'))) except KeyError as error: diff --git a/packages/ns-phonehome/files/phonehome b/packages/ns-phonehome/files/phonehome index 08f64b71a..95e92e1e2 100755 --- a/packages/ns-phonehome/files/phonehome +++ b/packages/ns-phonehome/files/phonehome @@ -36,6 +36,15 @@ for func in dir(inventory): if func.startswith("info_"): info[func.removeprefix('info_')] = method(EUci()) +# Migration fingerprint. Populated only on enterprise units that went +# through migrate-to-my or the native my register — my uses this to +# track which units have already rotated off the translation proxy +# and decide when the proxy can be decommissioned. +migration = { + "from_legacy_system_id": u.get('ns-plug', 'config', 'legacy_system_id', default='') or None, + "migrated_at": u.get('ns-plug', 'config', 'migrated_at', default='') or None, +} + data = { "$schema": "https://schema.nethserver.org/facts/2022-12.json", "uuid": sid, @@ -61,7 +70,8 @@ data = { }, "pci": list(pci.values()), "mountpoints": mount_points, - "features": features + "features": features, + "migration": migration } } diff --git a/packages/ns-plug/Makefile b/packages/ns-plug/Makefile index 93d50ba8d..f4360e221 100644 --- a/packages/ns-plug/Makefile +++ b/packages/ns-plug/Makefile @@ -21,7 +21,7 @@ define Package/ns-plug CATEGORY:=NethSecurity TITLE:=NethSecurity controller client URL:=https://github.com/NethServer/nethsecurity-controller/ - DEPENDS:=+openvpn +lscpu +python3-nethsec +python3-yaml +telegraf +victoria-metrics + DEPENDS:=+openvpn +lscpu +python3-nethsec +python3-yaml +telegraf +victoria-metrics +jq PKGARCH:=all endef @@ -82,6 +82,7 @@ define Package/ns-plug/install $(INSTALL_BIN) ./files/ns-plug-alert-proxy $(1)/usr/sbin/ns-plug-alert-proxy $(INSTALL_BIN) ./files/distfeed-setup $(1)/usr/sbin/distfeed-setup $(INSTALL_BIN) ./files/apk-official $(1)/usr/sbin/apk-official + $(INSTALL_BIN) ./files/migrate-to-my $(1)/usr/sbin $(INSTALL_BIN) ./files/remote-backup $(1)/usr/sbin $(INSTALL_BIN) ./files/send-backup $(1)/usr/sbin $(INSTALL_BIN) ./files/send-heartbeat $(1)/usr/sbin diff --git a/packages/ns-plug/files/config b/packages/ns-plug/files/config index 9f32f0262..fd208ddf7 100644 --- a/packages/ns-plug/files/config +++ b/packages/ns-plug/files/config @@ -5,6 +5,7 @@ config main 'config' option unit_name '' option tls_verify '1' option backup_url 'https://backupd.nethesis.it' + option collect_url 'https://my.nethesis.it/collect/api/systems' option repository_url 'https://updates.nethsecurity.nethserver.org' option channel '' option tun_mtu '' diff --git a/packages/ns-plug/files/migrate-to-my b/packages/ns-plug/files/migrate-to-my new file mode 100644 index 000000000..c8f96d02d --- /dev/null +++ b/packages/ns-plug/files/migrate-to-my @@ -0,0 +1,91 @@ +#!/bin/sh + +# +# Copyright (C) 2026 Nethesis S.r.l. +# SPDX-License-Identifier: GPL-2.0-only +# + +# +# Idempotent one-shot migration from legacy my.nethesis.it / backupd +# credentials to the my collect native credentials, so the unit can +# authenticate directly against the my collect endpoints. +# +# Only enterprise units are migrated. Community (my.nethserver.com) +# has its own infrastructure and keeps using the legacy send-* +# endpoints — no credential rotation is applicable there. +# +# ns-plug.config.migrated='1' is the persistent marker. It is written +# by this script after a successful rotation, and also by +# /usr/sbin/register when it registers a fresh unit directly against +# the my collect endpoint (so a brand new install never triggers the +# rotation path and never hits /proxy/credentials with unmapped +# credentials). +# +# On the first successful invocation the script: +# 1. Calls the my translation proxy's /proxy/credentials endpoint +# with the legacy Basic-Auth pair and retrieves the mapped my +# system key / secret. +# 2. Writes the new credentials to ns-plug.config.system_id / secret +# and preserves the legacy pair under legacy_system_id / +# legacy_secret (for audit and manual rollback). +# 3. Re-asserts ns-plug.config.collect_url, because /etc/config/ +# ns-plug is a conffile: on registered units opkg keeps the +# user-modified copy across upgrades, so a new default alone +# would not reach them. +# 4. Sets the migrated='1' marker. +# +# The uci commit is atomic — a partial write cannot leave the unit in +# an inconsistent half-migrated state. +# + +# Marker: set only after a successful rotation or a native my register. +[ "$(uci -q get ns-plug.config.migrated)" = "1" ] && exit 0 + +# Community units stay on the legacy my.nethserver.com infrastructure. +TYPE=$(uci -q get ns-plug.config.type) +if [ "$TYPE" != "enterprise" ]; then + exit 0 +fi + +SYSTEM_ID=$(uci -q get ns-plug.config.system_id) +SYSTEM_SECRET=$(uci -q get ns-plug.config.secret) +if [ -z "$SYSTEM_ID" ] || [ -z "$SYSTEM_SECRET" ]; then + # Unregistered unit — nothing to migrate yet. + exit 0 +fi + +# Fetch the mapped my credentials via the translation proxy. +resp=$(/usr/bin/curl --silent --location-trusted --fail-with-body \ + --max-time 30 --retry 2 \ + --user "$SYSTEM_ID:$SYSTEM_SECRET" \ + https://my.nethesis.it/proxy/credentials 2>/dev/null) || { + logger -t migrate-to-my "credential fetch failed; will retry on next run" + exit 0 +} + +new_key=$(echo "$resp" | jq -r '.data.system_key // empty' 2>/dev/null) +new_secret=$(echo "$resp" | jq -r '.data.system_secret // empty' 2>/dev/null) +if [ -z "$new_key" ] || [ -z "$new_secret" ]; then + logger -t migrate-to-my "credentials missing in response" + exit 0 +fi + +# Timestamp the rotation so phonehome can publish the event and my +# can plot the fleet migration curve / decide when the translation +# proxy can be decommissioned. +migrated_at=$(date -u +%Y-%m-%dT%H:%M:%SZ) + +# Rotate atomically; legacy pair preserved for audit / rollback. +uci -q batch </dev/null) ;; enterprise) - url="https://my.nethesis.it/api/" + url="https://my.nethesis.it/backend/api/" - system_id=$(curl -s -m $timeout --retry 3 -L \ + register_resp=$(curl -s -m $timeout --retry 3 -L \ -H "Content-Type: application/json" -H "Accept: application/json" \ - -d '{"secret": "'$secret'"}' "${url}systems/info" | jq -r ".uuid" 2>/dev/null) + -d '{"system_secret": "'$secret'"}' "${url}systems/register") + + system_id=$(echo "$register_resp" | jq -r '.data.system_key // empty' 2>/dev/null) ;; *) exit_error "Invalid type '$type'" @@ -68,9 +75,12 @@ case "$type" in ;; enterprise) uci set ns-plug.config.type="enterprise" - uci set ns-plug.config.alerts_url="https://my.nethesis.it/isa/" uci set ns-plug.config.api_url="$url" - uci set ns-plug.config.inventory_url="https://my.nethesis.it/isa/inventory/store/" + uci set ns-plug.config.collect_url="https://my.nethesis.it/collect/api/systems" + # Native my register: no legacy credentials to rotate, so + # mark the unit as already migrated. migrate-to-my will be a + # no-op on every subsequent run. + uci set ns-plug.config.migrated="1" ;; esac diff --git a/packages/ns-plug/files/remote-backup b/packages/ns-plug/files/remote-backup index 9a12d0ef4..829ee40b3 100755 --- a/packages/ns-plug/files/remote-backup +++ b/packages/ns-plug/files/remote-backup @@ -6,8 +6,20 @@ # # -# Manage remote backup +# Manage configuration backups. # +# Enterprise units (type=enterprise) talk to my collect after the +# migrate-to-my credential rotation. Community units (type=community) +# keep using the legacy backupd.nethesis.it endpoint with the same +# URL layout they have always used — backupd still accepts both +# tenants behind the $TYPE/api/v2/backup/ path. +# +# Pipefail so the curl exit status survives the jq stage in `list`; +# without it a HTTP error on the server would be masked by a successful +# jq parse and ns.backup would report success to the UI. +# + +set -o pipefail function exit_error { >&2 echo "[ERROR] $@" @@ -15,68 +27,105 @@ function exit_error { } function help { - >&2 echo "Usage: $0 " + >&2 echo "Usage: $0 " >&2 echo "Commands:" - >&2 echo " - list: retrieve the list of available backups from remote server" - >&2 echo " - download [output]: download the given backup, if 'output' is empty downloaded file will be named as as 'file'" - >&2 echo " - upload : upload the given backup" + >&2 echo " - list: fetch the list of backups stored for this system" + >&2 echo " - download [output]: download the backup ; defaults to writing to a file named " + >&2 echo " - upload : upload a backup file" + >&2 echo " - delete : remove the backup " } SYSTEM_ID=$(uci -q get ns-plug.config.system_id) SYSTEM_SECRET=$(uci -q get ns-plug.config.secret) TYPE=$(uci -q get ns-plug.config.type) -URL=$(uci -q get ns-plug.config.backup_url) -if [ -z "$SYSTEM_ID" ] || [ -z "$SYSTEM_SECRET" ] || [ -z "$URL" ]; then - exit_error "System ID, system secret or backup url not found. Please configure ns-plug." +if [ -z "$SYSTEM_ID" ] || [ -z "$SYSTEM_SECRET" ]; then + exit_error "System ID or system secret not found. Please configure ns-plug." +fi + +cmd=${1:-list} + +if [ "$TYPE" = "enterprise" ]; then + /usr/sbin/migrate-to-my + + COLLECT_URL=$(uci -q get ns-plug.config.collect_url) + if [ -z "$COLLECT_URL" ]; then + exit_error "Collect URL not set. Pre-migration unit — retry later." + fi + + BASE="$COLLECT_URL/backups" + # --fail-with-body: exit 22 on 4xx/5xx while still writing the body + # so the caller can inspect the error payload. + curl_args="--silent --location-trusted --fail-with-body --user $SYSTEM_ID:$SYSTEM_SECRET" + + case "$cmd" in + list) + # my returns {code, message, data: {backups: [...]}} on + # success. Unwrap `data` so ns.backup can pass it through as + # {values: } without double-nesting. Fall back to an + # empty list on failure. + response=$(curl $curl_args "$BASE") + echo "$response" | jq 'if .data and (.data.backups // empty) then .data else {backups: []} end' + ;; + download) + file=$2 + [ -z "$file" ] && exit_error "No file specified" + output=${3-$file} + curl $curl_args -o "$output" "$BASE/$file" + ;; + upload) + file=$2 + [ -z "$file" ] && exit_error "No file specified" + curl $curl_args -X POST \ + -H "Content-Type: application/octet-stream" \ + -H "X-Filename: $(basename "$file")" \ + --data-binary "@$file" \ + "$BASE" + ;; + delete) + file=$2 + [ -z "$file" ] && exit_error "No file specified" + curl $curl_args -X DELETE "$BASE/$file" + ;; + *) + help + ;; + esac + + exit $? +fi + +# Community (legacy): backupd.nethesis.it with the /$TYPE/api/v2/backup/ +# URL layout. Unchanged from the pre-migration behaviour. +URL=$(uci -q get ns-plug.config.backup_url) +if [ -z "$URL" ]; then + exit_error "Backup URL not set. Please configure ns-plug." fi curl_args="--silent --location-trusted --user $SYSTEM_ID:$SYSTEM_SECRET" base_url="$URL/$TYPE/api/v2/backup/" -cmd=${1:-list} - case "$cmd" in list) curl $curl_args $base_url ;; download) file=$2 - if [ -z "$file" ]; then - exit_error "No file specified" - fi + [ -z "$file" ] && exit_error "No file specified" output=${3-$file} curl $curl_args $base_url$file -J -o "$output" - ;; - upload) - file=$2 - if [ -z "$file" ]; then - exit_error "No file specified" - fi - curl $curl_args $base_url --upload-file $file - rc=$? - # Temporary dual-send to new my.nethesis.it via the translation - # proxy, same pattern used by send-heartbeat / send-inventory. - # To be removed once the migration is complete. - if [ "$TYPE" = "enterprise" ]; then - # Strip the directory path first to get just the filename - filename="${file##*/}" - curl $curl_args -X POST \ - -H "Content-Type: application/octet-stream" \ - -H "X-Filename: ${filename}" \ - --data-binary "@$file" https://my.nethesis.it/proxy/backup >/dev/null || : - fi - exit $rc - ;; - delete) - file=$2 - if [ -z "$file" ]; then - exit_error "No file specified" - fi - curl $curl_args -X DELETE $base_url$file - ;; - - *) - help - ;; + ;; + upload) + file=$2 + [ -z "$file" ] && exit_error "No file specified" + curl $curl_args $base_url --upload-file $file + ;; + delete) + file=$2 + [ -z "$file" ] && exit_error "No file specified" + curl $curl_args -X DELETE $base_url$file + ;; + *) + help + ;; esac diff --git a/packages/ns-plug/files/send-backup b/packages/ns-plug/files/send-backup index 707fdc439..bb16d2d72 100644 --- a/packages/ns-plug/files/send-backup +++ b/packages/ns-plug/files/send-backup @@ -45,6 +45,12 @@ if [ -z "$SYSTEM_ID" ] || [ -z "$SYSTEM_SECRET" ]; then exit 0 fi +# remote-backup handles the enterprise/community branching and calls +# migrate-to-my when needed; this script just prepares the payload and +# delegates the upload. An enterprise unit still waiting on the +# migration surfaces its error through remote-backup, which is caught +# by set -e above. + # Create the backup mkdir -p "$WORK_DIR" sysupgrade -q -k -b "$BACKUP" diff --git a/packages/ns-plug/files/send-heartbeat b/packages/ns-plug/files/send-heartbeat index 36b328349..8f4d02c0e 100755 --- a/packages/ns-plug/files/send-heartbeat +++ b/packages/ns-plug/files/send-heartbeat @@ -5,11 +5,19 @@ # SPDX-License-Identifier: GPL-2.0-only # -# Send the heartbeat +# Send the heartbeat. +# +# Enterprise units (type=enterprise) post to the my collect endpoint +# with the rotated my credentials; migrate-to-my runs up front so a +# unit upgraded from the legacy my.nethesis.it path transparently +# flips over on the first successful rotation. +# +# Community units (type=community) are left on the legacy +# my.nethserver.com heartbeat path — that infrastructure has no +# counterpart on the new my and is out of scope for this migration. SYSTEM_ID=$(uci -q get ns-plug.config.system_id) SYSTEM_SECRET=$(uci -q get ns-plug.config.secret) -URL=$(uci -q get ns-plug.config.alerts_url)"heartbeats/store" TYPE=$(uci -q get ns-plug.config.type) if [ -z "$SYSTEM_ID" ] || [ -z "$SYSTEM_SECRET" ]; then @@ -17,14 +25,26 @@ if [ -z "$SYSTEM_ID" ] || [ -z "$SYSTEM_SECRET" ]; then exit 0 fi -/usr/bin/curl -m 180 --retry 3 -L -s \ - --header "Authorization: token $SYSTEM_SECRET" --header "Content-Type: application/json" --header "Accept: application/json" \ - --data-raw '{"lk": "'$SYSTEM_ID'"}' "$URL" >/dev/null +case "$TYPE" in + enterprise) + /usr/sbin/migrate-to-my -# Temporary send data to new endpoint -# To be removed when the migration to new my.nethesis.it will be completed -if [ "$TYPE" = "enterprise" ]; then - /usr/bin/curl -m 180 --retry 3 -L -s -X POST \ - --user "$SYSTEM_ID:$SYSTEM_SECRET" https://my.nethesis.it/proxy/heartbeat >/dev/null - exit 0 -fi + COLLECT_URL=$(uci -q get ns-plug.config.collect_url) + if [ -z "$COLLECT_URL" ]; then + # Pre-migration — retry on next tick. + exit 0 + fi + + /usr/bin/curl -m 30 --retry 3 -L -s -X POST \ + --user "$SYSTEM_ID:$SYSTEM_SECRET" \ + "$COLLECT_URL/heartbeat" >/dev/null + ;; + community) + URL=$(uci -q get ns-plug.config.alerts_url)"heartbeats/store" + /usr/bin/curl -m 180 --retry 3 -L -s \ + --header "Authorization: token $SYSTEM_SECRET" \ + --header "Content-Type: application/json" \ + --header "Accept: application/json" \ + --data-raw '{"lk": "'$SYSTEM_ID'"}' "$URL" >/dev/null + ;; +esac diff --git a/packages/ns-plug/files/send-inventory b/packages/ns-plug/files/send-inventory index a670e9925..b6658f672 100755 --- a/packages/ns-plug/files/send-inventory +++ b/packages/ns-plug/files/send-inventory @@ -5,11 +5,16 @@ # SPDX-License-Identifier: GPL-2.0-only # -# Send the inventory +# Send the inventory. +# +# Enterprise units post to my collect with the rotated my credentials; +# community units keep using the legacy my.nethserver.com inventory +# path. See send-heartbeat for the rationale — the two flows are kept +# parallel so the community infrastructure is never touched by the my +# migration. SYSTEM_ID=$(uci -q get ns-plug.config.system_id) SYSTEM_SECRET=$(uci -q get ns-plug.config.secret) -URL=$(uci -q get ns-plug.config.inventory_url) TYPE=$(uci -q get ns-plug.config.type) if [ -z "$SYSTEM_ID" ] || [ -z "$SYSTEM_SECRET" ]; then @@ -17,28 +22,40 @@ if [ -z "$SYSTEM_ID" ] || [ -z "$SYSTEM_SECRET" ]; then exit 0 fi -echo "{\"data\": {\"lk\": \"$SYSTEM_ID\", \"data\": $(/usr/sbin/inventory) }}" | \ -/usr/bin/curl -m 180 --retry 5 -L -s \ - --header "Authorization: token $SYSTEM_SECRET" --header "Content-Type: application/json" --header "Accept: application/json" \ - --data-binary @- "$URL" > /dev/null - -if [ $? -ne 0 ]; then - status="error" -else - status="success" -fi +status="error" + +case "$TYPE" in + enterprise) + /usr/sbin/migrate-to-my + + COLLECT_URL=$(uci -q get ns-plug.config.collect_url) + if [ -z "$COLLECT_URL" ]; then + # Pre-migration — retry on next tick. + exit 0 + fi + + /usr/sbin/phonehome | /usr/bin/curl -m 180 --retry 3 -L -s -X POST \ + --user "$SYSTEM_ID:$SYSTEM_SECRET" \ + -H "Content-Type: application/json" \ + --data-binary @- "$COLLECT_URL/inventory" >/dev/null + + if [ $? -eq 0 ]; then + status="success" + fi + ;; + community) + URL=$(uci -q get ns-plug.config.inventory_url) + echo "{\"data\": {\"lk\": \"$SYSTEM_ID\", \"data\": $(/usr/sbin/inventory) }}" | \ + /usr/bin/curl -m 180 --retry 5 -L -s \ + --header "Authorization: token $SYSTEM_SECRET" \ + --header "Content-Type: application/json" \ + --header "Accept: application/json" \ + --data-binary @- "$URL" > /dev/null + + if [ $? -eq 0 ]; then + status="success" + fi + ;; +esac echo '{"status": "'$status'", "last_attempt": "'$(date -Iseconds)'"}' > /tmp/inventory-sent.json - -if [ "$TYPE" = "enterprise" ]; then - # Update registration date - /usr/bin/curl -m 180 --retry 5 -L -s \ - --header "Content-Type: application/json" --header "Accept: application/json" \ - -d '{"secret":"'$SYSTEM_SECRET'"}' https://my.nethesis.it/api/systems/info >/dev/null - - # Temporary send data to new endpoint - # To be removed when the migration to new my.nethesis.it will be completed - /usr/sbin/phonehome | /usr/bin/curl -m 180 --retry 3 -L -s --user "$SYSTEM_ID:$SYSTEM_SECRET" \ - -H "Content-Type: application/json" \ - --data-binary @- https://my.nethesis.it/proxy/inventory >/dev/null || : -fi diff --git a/packages/ns-plug/files/subscription-info b/packages/ns-plug/files/subscription-info index d78d3cbdd..e0d60ecad 100755 --- a/packages/ns-plug/files/subscription-info +++ b/packages/ns-plug/files/subscription-info @@ -6,8 +6,16 @@ # # -# Retrieve subscription information -# The script takes an optional timeout parameter +# Retrieve subscription information. +# +# Enterprise units query the my collect /info endpoint with their +# native credentials and synthesise a payload shaped like the legacy +# my-old /api/systems/info response the UI expects. The subscription +# plan / expiration fields are left empty — the new my data model +# no longer tracks them at the system level; the ns.subscription +# info handler falls back to "-" / 0 / "active" in those cases. +# +# Community units keep hitting my.nethserver.com as before. # timeout=${1:-20} @@ -15,20 +23,49 @@ timeout=${1:-20} system_id=$(uci -q get ns-plug.config.system_id) if [ -z "$system_id" ]; then - # no subscription echo '{"uuid": ""}' exit 0 fi type=$(uci -q get ns-plug.config.type) secret=$(uci -q get ns-plug.config.secret) -url=$(uci -q get ns-plug.config.api_url | sed 's/\/$//') if [ "$type" = "enterprise" ]; then - curl -f -s -m $timeout --retry-delay 1 --retry 2 -L \ - -H "Content-Type: application/json" -H "Accept: application/json" \ - -d '{"secret": "'$secret'"}' "$url/systems/info" + /usr/sbin/migrate-to-my + + collect_url=$(uci -q get ns-plug.config.collect_url) + if [ -z "$collect_url" ]; then + # Pre-migration unit — nothing to report yet; caller falls + # back to a default payload built from ns-plug.config. + exit 1 + fi + + # /info returns {code, message, data: {system_id, registered, + # registered_at, suspended, organization: {name, ...}, ...}}. + # Translate to the legacy shape the UI/info handler parses. + resp=$(/usr/bin/curl -f -s -m $timeout --retry-delay 1 --retry 2 -L \ + -H "Accept: application/json" \ + --user "$system_id:$secret" \ + "$collect_url/info") || exit $? + + jq -c ' + .data as $s | + { + uuid: ($s.system_id // ""), + id: ($s.system_key // ""), + subscription: { + status: (if $s.registered and (($s.suspended // false) | not) then "valid" else "invalid" end), + valid_until: null, + subscription_plan: { name: ($s.organization.name // "-") } + } + } + ' </dev/null +# Release the legacy slot on my-old for migrated enterprise units — +# the /api/Utils/freekey PHP endpoint still exists on my-ent and lets +# the old dashboard record the unit as gone. Native enterprise units +# have no legacy slot to release (registered directly on my collect), +# and community has no freekey equivalent on dartagnan. +if [ "$TYPE" = "enterprise" ]; then + LEGACY_ID=$(uci -q get ns-plug.config.legacy_system_id) + LEGACY_SECRET=$(uci -q get ns-plug.config.legacy_secret) + if [ -n "$LEGACY_ID" ] && [ -n "$LEGACY_SECRET" ]; then + curl -s -m 180 --retry 3 -L \ + -H "Content-type: application/json" -H "Accept: application/json" \ + -d "{\"lk\":\"$LEGACY_ID\",\"secret\":\"$LEGACY_SECRET\"}" \ + https://my.nethesis.it/api/Utils/freekey >/dev/null + fi +fi # Reset ns-plug configuration uci set ns-plug.config.type="" @@ -28,6 +42,14 @@ uci set ns-plug.config.inventory_url="" uci set ns-plug.config.system_id="" uci set ns-plug.config.secret="" uci set ns-plug.config.repository_url="https://updates.nethsecurity.nethserver.org/$(cat /etc/repo-channel)" +# Drop the enterprise migration fingerprint so a subsequent register +# starts from a clean slate and migrate-to-my can run again if the +# unit re-registers with legacy credentials. +uci -q delete ns-plug.config.collect_url +uci -q delete ns-plug.config.migrated +uci -q delete ns-plug.config.migrated_at +uci -q delete ns-plug.config.legacy_system_id +uci -q delete ns-plug.config.legacy_secret # Save config uci commit ns-plug From c7487fa915df703ab3b2204b4c3c24bb876cf121 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Wed, 17 Jun 2026 10:37:00 +0200 Subject: [PATCH 02/41] feat(migration): switch alerts to native my collect after cutover vmalert.initd: when migrated=1 and collect_url is set, derive the native Mimir alertmanager URL from collect_url and notify with the rotated credentials; pre-cutover units stay on /proxy/alerts. migrate-to-my: reload vmalert right after the rotation so the alert path flips together with the credentials instead of at the next reload. --- packages/ns-plug/files/migrate-to-my | 8 ++++ packages/victoria-metrics/files/vmalert.initd | 37 +++++++++++++------ 2 files changed, 34 insertions(+), 11 deletions(-) diff --git a/packages/ns-plug/files/migrate-to-my b/packages/ns-plug/files/migrate-to-my index c8f96d02d..d22a5f6a5 100644 --- a/packages/ns-plug/files/migrate-to-my +++ b/packages/ns-plug/files/migrate-to-my @@ -87,5 +87,13 @@ set ns-plug.config.migrated_at=$migrated_at commit ns-plug EOI +# Repoint vmalert at the native Mimir endpoint right away. vmalert reloads on +# ns-plug changes, but migrate-to-my does not trigger reload_config, so force a +# targeted reload here: the running vmalert otherwise keeps its boot-time +# (legacy) creds and POSTs to /proxy/alerts until the next reload. Reloading +# now makes the alert path flip together with the credential rotation. Safe +# no-op if victoria-metrics is not installed. +/etc/init.d/vmalert reload 2>/dev/null || true + logger -t migrate-to-my "migrated to my collect credentials" exit 0 diff --git a/packages/victoria-metrics/files/vmalert.initd b/packages/victoria-metrics/files/vmalert.initd index 3abf258f9..a907c75b5 100644 --- a/packages/victoria-metrics/files/vmalert.initd +++ b/packages/victoria-metrics/files/vmalert.initd @@ -19,25 +19,40 @@ start_service() { config_get datasource_url main datasource_url "http://localhost:8428" config_get http_listen_addr main http_listen_addr "127.0.0.1:8081" - # Forward alerts to the new my.nethesis.it during the migration window, - # mirroring send-heartbeat / send-inventory: enterprise systems POST to the - # credential-translation proxy at my.nethesis.it/proxy/alerts using the - # ns-plug credentials (system_id:secret), which the proxy maps to the new my - # credentials. vmalert appends /api/v2/alerts to the notifier URL. The my - # switch-off release will repoint this to the native collect path. - local system_id system_secret system_type alerts_disabled notifier_url notifier_user notifier_pass + # Forward alerts to the new my for enterprise systems. Two windows: + # - Pre-cutover (migrated!=1): POST to the credential-translation proxy at + # my.nethesis.it/proxy/alerts with the still-legacy ns-plug credentials + # (system_id:secret); the proxy maps them to the new my credentials. + # - Post-cutover (migrated=1 + collect_url): migrate-to-my has rotated the + # credentials to native my credentials, so POST straight to the native + # Mimir alertmanager derived from collect_url — the legacy-only proxy + # would 401 the rotated credentials. vmalert appends /api/v2/alerts, so + # the notifier URL is the alertmanager base WITHOUT that suffix. + # Opt-out (disable_my_alerts=1) keeps the unit on the local alert-proxy only. + local system_id system_secret system_type alerts_disabled migrated collect_url + local notifier_url notifier_user notifier_pass config_load ns-plug 2>/dev/null && { config_get system_id config system_id "" config_get system_secret config secret "" config_get system_type config type "" + config_get migrated config migrated "" + config_get collect_url config collect_url "" + # opt-out: set ns-plug.config.disable_my_alerts=1 for alert-proxy only mode + config_get_bool alerts_disabled config disable_my_alerts 0 } - if [ "$system_type" = "enterprise" ] && [ -n "$system_id" ] && [ -n "$system_secret" ]; then - notifier_url="https://my.nethesis.it/proxy/alerts" + notifier_url="" + if [ "$system_type" = "enterprise" ] && [ -n "$system_id" ] && [ -n "$system_secret" ] && [ "$alerts_disabled" = "0" ]; then + if [ "$migrated" = "1" ] && [ -n "$collect_url" ]; then + # Native my Mimir alertmanager, derived from collect_url, e.g. + # https://my.nethesis.it/collect/api/systems + # -> https://my.nethesis.it/collect/api/services/mimir/alertmanager + notifier_url="${collect_url%%/collect/*}/collect/api/services/mimir/alertmanager" + else + notifier_url="https://my.nethesis.it/proxy/alerts" + fi notifier_user="$system_id" notifier_pass="$system_secret" - else - notifier_url="" fi procd_open_instance From 98d61afb35709af9cb96ed25817a724eabb6a7ef Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Fri, 19 Jun 2026 09:45:31 +0200 Subject: [PATCH 03/41] feat(migration): point my endpoints at my-proxy-prod.onrender.com Reach the new my via the Render prod proxy so migrated units work before the my.nethesis.it DNS flip; reverts at the flip. --- packages/ns-plug/files/config | 2 +- packages/ns-plug/files/migrate-to-my | 4 ++-- packages/ns-plug/files/register | 4 ++-- packages/ns-plug/files/unregister | 2 +- packages/victoria-metrics/files/vmalert.initd | 6 +++--- 5 files changed, 9 insertions(+), 9 deletions(-) diff --git a/packages/ns-plug/files/config b/packages/ns-plug/files/config index fd208ddf7..71e511bbc 100644 --- a/packages/ns-plug/files/config +++ b/packages/ns-plug/files/config @@ -5,7 +5,7 @@ config main 'config' option unit_name '' option tls_verify '1' option backup_url 'https://backupd.nethesis.it' - option collect_url 'https://my.nethesis.it/collect/api/systems' + option collect_url 'https://my-proxy-prod.onrender.com/collect/api/systems' option repository_url 'https://updates.nethsecurity.nethserver.org' option channel '' option tun_mtu '' diff --git a/packages/ns-plug/files/migrate-to-my b/packages/ns-plug/files/migrate-to-my index d22a5f6a5..9b88551ad 100644 --- a/packages/ns-plug/files/migrate-to-my +++ b/packages/ns-plug/files/migrate-to-my @@ -58,7 +58,7 @@ fi resp=$(/usr/bin/curl --silent --location-trusted --fail-with-body \ --max-time 30 --retry 2 \ --user "$SYSTEM_ID:$SYSTEM_SECRET" \ - https://my.nethesis.it/proxy/credentials 2>/dev/null) || { + https://my-proxy-prod.onrender.com/proxy/credentials 2>/dev/null) || { logger -t migrate-to-my "credential fetch failed; will retry on next run" exit 0 } @@ -81,7 +81,7 @@ set ns-plug.config.legacy_system_id=$SYSTEM_ID set ns-plug.config.legacy_secret=$SYSTEM_SECRET set ns-plug.config.system_id=$new_key set ns-plug.config.secret=$new_secret -set ns-plug.config.collect_url=https://my.nethesis.it/collect/api/systems +set ns-plug.config.collect_url=https://my-proxy-prod.onrender.com/collect/api/systems set ns-plug.config.migrated=1 set ns-plug.config.migrated_at=$migrated_at commit ns-plug diff --git a/packages/ns-plug/files/register b/packages/ns-plug/files/register index ac6d3a666..1d877c416 100755 --- a/packages/ns-plug/files/register +++ b/packages/ns-plug/files/register @@ -47,7 +47,7 @@ case "$type" in "${url}machine/info" | jq -r ".uuid" 2>/dev/null) ;; enterprise) - url="https://my.nethesis.it/backend/api/" + url="https://my-proxy-prod.onrender.com/backend/api/" register_resp=$(curl -s -m $timeout --retry 3 -L \ -H "Content-Type: application/json" -H "Accept: application/json" \ @@ -76,7 +76,7 @@ case "$type" in enterprise) uci set ns-plug.config.type="enterprise" uci set ns-plug.config.api_url="$url" - uci set ns-plug.config.collect_url="https://my.nethesis.it/collect/api/systems" + uci set ns-plug.config.collect_url="https://my-proxy-prod.onrender.com/collect/api/systems" # Native my register: no legacy credentials to rotate, so # mark the unit as already migrated. migrate-to-my will be a # no-op on every subsequent run. diff --git a/packages/ns-plug/files/unregister b/packages/ns-plug/files/unregister index 21041c4dd..5fe069815 100755 --- a/packages/ns-plug/files/unregister +++ b/packages/ns-plug/files/unregister @@ -30,7 +30,7 @@ if [ "$TYPE" = "enterprise" ]; then curl -s -m 180 --retry 3 -L \ -H "Content-type: application/json" -H "Accept: application/json" \ -d "{\"lk\":\"$LEGACY_ID\",\"secret\":\"$LEGACY_SECRET\"}" \ - https://my.nethesis.it/api/Utils/freekey >/dev/null + https://my-proxy-prod.onrender.com/api/Utils/freekey >/dev/null fi fi diff --git a/packages/victoria-metrics/files/vmalert.initd b/packages/victoria-metrics/files/vmalert.initd index a907c75b5..0151c2702 100644 --- a/packages/victoria-metrics/files/vmalert.initd +++ b/packages/victoria-metrics/files/vmalert.initd @@ -45,11 +45,11 @@ start_service() { if [ "$system_type" = "enterprise" ] && [ -n "$system_id" ] && [ -n "$system_secret" ] && [ "$alerts_disabled" = "0" ]; then if [ "$migrated" = "1" ] && [ -n "$collect_url" ]; then # Native my Mimir alertmanager, derived from collect_url, e.g. - # https://my.nethesis.it/collect/api/systems - # -> https://my.nethesis.it/collect/api/services/mimir/alertmanager + # https://my-proxy-prod.onrender.com/collect/api/systems + # -> https://my-proxy-prod.onrender.com/collect/api/services/mimir/alertmanager notifier_url="${collect_url%%/collect/*}/collect/api/services/mimir/alertmanager" else - notifier_url="https://my.nethesis.it/proxy/alerts" + notifier_url="https://my-proxy-prod.onrender.com/proxy/alerts" fi notifier_user="$system_id" notifier_pass="$system_secret" From 4bb31f4c7ca4684546b562dfe860268278e78396 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Mon, 29 Jun 2026 12:41:15 +0200 Subject: [PATCH 04/41] fix(alert): drop disable_my_alerts opt-out MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Align with main cf2d3052 — the option was added during a transition phase and is no longer required. The my-cutover branch predated that removal, so the rebase re-introduced it; remove it again to avoid resurrecting code dropped from main. --- packages/victoria-metrics/files/vmalert.initd | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/packages/victoria-metrics/files/vmalert.initd b/packages/victoria-metrics/files/vmalert.initd index 0151c2702..c3e35cbbd 100644 --- a/packages/victoria-metrics/files/vmalert.initd +++ b/packages/victoria-metrics/files/vmalert.initd @@ -28,8 +28,7 @@ start_service() { # Mimir alertmanager derived from collect_url — the legacy-only proxy # would 401 the rotated credentials. vmalert appends /api/v2/alerts, so # the notifier URL is the alertmanager base WITHOUT that suffix. - # Opt-out (disable_my_alerts=1) keeps the unit on the local alert-proxy only. - local system_id system_secret system_type alerts_disabled migrated collect_url + local system_id system_secret system_type migrated collect_url local notifier_url notifier_user notifier_pass config_load ns-plug 2>/dev/null && { config_get system_id config system_id "" @@ -37,12 +36,10 @@ start_service() { config_get system_type config type "" config_get migrated config migrated "" config_get collect_url config collect_url "" - # opt-out: set ns-plug.config.disable_my_alerts=1 for alert-proxy only mode - config_get_bool alerts_disabled config disable_my_alerts 0 } notifier_url="" - if [ "$system_type" = "enterprise" ] && [ -n "$system_id" ] && [ -n "$system_secret" ] && [ "$alerts_disabled" = "0" ]; then + if [ "$system_type" = "enterprise" ] && [ -n "$system_id" ] && [ -n "$system_secret" ]; then if [ "$migrated" = "1" ] && [ -n "$collect_url" ]; then # Native my Mimir alertmanager, derived from collect_url, e.g. # https://my-proxy-prod.onrender.com/collect/api/systems From 7546fd66bb2542edcdfda9ea93e57e27cabfef32 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Mon, 29 Jun 2026 12:54:12 +0200 Subject: [PATCH 05/41] build(ns-ui): bundle nethsecurity-ui#746 backup UI for the cutover MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pin ns-ui to the my-native backup UI (nethsecurity-ui#746) so a migrated appliance can browse/restore the backups it now sends to my collect. Temporary git-ref pin — replace with the released tag before merging (see Makefile note). --- packages/ns-ui/Makefile | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index 5403c6c47..16d7b483b 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -12,7 +12,12 @@ PKG_RELEASE:=1 PKG_SOURCE_PROTO:=git PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git -PKG_SOURCE_VERSION:=$(PKG_VERSION) +# TEMP (my cutover): pinned to nethsecurity-ui#746 (feat/backup-my-api) so the image +# bundles the my-native backup UI alongside this appliance cutover. The UI reads the +# new collect backup payload; on a device still on backupd it would break, hence it must +# ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) +# and bump PKG_VERSION to the nethsecurity-ui release that carries #746. +PKG_SOURCE_VERSION:=661fd064ac5e24a7552b81a2916c0dbf327b1c87 PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From 3f81c9a3e6804d98ff512fe50e6c3b61e5fa3e87 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Mon, 29 Jun 2026 15:59:52 +0200 Subject: [PATCH 06/41] feat(subscription): expose organization and enterprise plan for migrated units subscription-info threads organization.name explicitly; ns.subscription info returns it as the organization and defaults the plan to "Nethesis Enterprise" for enterprise units. Community units keep the real plan name untouched. --- packages/ns-api/files/ns.subscription | 9 +++++++++ packages/ns-plug/files/subscription-info | 1 + 2 files changed, 10 insertions(+) diff --git a/packages/ns-api/files/ns.subscription b/packages/ns-api/files/ns.subscription index dacadeb2c..c8f5c294e 100755 --- a/packages/ns-api/files/ns.subscription +++ b/packages/ns-api/files/ns.subscription @@ -61,6 +61,15 @@ def info(): type = u.get('ns-plug', 'config', 'type', default='') ret = {"server_id": data["id"], "systemd_id": data["uuid"], "plan": data["subscription"]["subscription_plan"]["name"], "expiration": expiration, "active": active, "type": type} + + # The new my has no per-system commercial plan. For enterprise units expose + # the organization explicitly (subscription-info threads organization.name) + # and default the plan label to "Nethesis Enterprise". Community units keep + # the real plan name untouched. + if type == "enterprise": + ret["organization"] = data.get("organization", "") + ret["plan"] = "Nethesis Enterprise" + return ret diff --git a/packages/ns-plug/files/subscription-info b/packages/ns-plug/files/subscription-info index e0d60ecad..363f2ff67 100755 --- a/packages/ns-plug/files/subscription-info +++ b/packages/ns-plug/files/subscription-info @@ -53,6 +53,7 @@ if [ "$type" = "enterprise" ]; then { uuid: ($s.system_id // ""), id: ($s.system_key // ""), + organization: ($s.organization.name // ""), subscription: { status: (if $s.registered and (($s.suspended // false) | not) then "valid" else "invalid" end), valid_until: null, From 9f00477a25f8ff69e1c86376d555e1b824441bf8 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Mon, 29 Jun 2026 15:59:52 +0200 Subject: [PATCH 07/41] build(ns-ui): bump pin to include the subscription view fix (nethsecurity-ui#746) --- packages/ns-ui/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index 16d7b483b..3b805961c 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=661fd064ac5e24a7552b81a2916c0dbf327b1c87 +PKG_SOURCE_VERSION:=bc03487db4673e9a497cd79b9de65addb39392f0 PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From 8149f943d0482fa6a610bd2ae7307587755c5224 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Tue, 30 Jun 2026 11:06:24 +0200 Subject: [PATCH 08/41] fix(subscription): surface already-registered on re-register register exits 2 on the backend 409 (the my system key is one-shot and never freed) and ns.subscription returns 'system_already_registered' instead of collapsing every failure into the generic 'invalid_secret_or_server_not_found', so the UI can explain a new system is needed. --- packages/ns-api/files/ns.subscription | 20 ++++++++++++-------- packages/ns-plug/files/register | 14 +++++++++++++- 2 files changed, 25 insertions(+), 9 deletions(-) diff --git a/packages/ns-api/files/ns.subscription b/packages/ns-api/files/ns.subscription index c8f5c294e..3f983f86c 100755 --- a/packages/ns-api/files/ns.subscription +++ b/packages/ns-api/files/ns.subscription @@ -20,17 +20,21 @@ def register(args): secret = args["secret"] - try: - subprocess.run(["/usr/sbin/register", "enterprise", secret, '5'], check=True, capture_output=True) + enterprise = subprocess.run(["/usr/sbin/register", "enterprise", secret, '5'], capture_output=True) + if enterprise.returncode == 0: return {"result": "success"} - except: - pass - try: - subprocess.run(["/usr/sbin/register", "community", secret, '5'], check=True, capture_output=True) + # Exit code 2: the system is already registered on my and its key cannot be + # reused (one-shot registration). Report it as-is instead of falling back to + # community, so the UI can guide the user to create a new system. + if enterprise.returncode == 2: + return utils.generic_error("system_already_registered") + + community = subprocess.run(["/usr/sbin/register", "community", secret, '5'], capture_output=True) + if community.returncode == 0: return {"result": "success"} - except: - return utils.generic_error("invalid_secret_or_server_not_found") + + return utils.generic_error("invalid_secret_or_server_not_found") def unregister(): try: diff --git a/packages/ns-plug/files/register b/packages/ns-plug/files/register index 1d877c416..33fe153ff 100755 --- a/packages/ns-plug/files/register +++ b/packages/ns-plug/files/register @@ -49,9 +49,21 @@ case "$type" in enterprise) url="https://my-proxy-prod.onrender.com/backend/api/" - register_resp=$(curl -s -m $timeout --retry 3 -L \ + register_resp=$(curl -s -m $timeout --retry 3 -L -w '\n%{http_code}' \ -H "Content-Type: application/json" -H "Accept: application/json" \ -d '{"system_secret": "'$secret'"}' "${url}systems/register") + http_code=$(echo "$register_resp" | sed -n '$p') + register_resp=$(echo "$register_resp" | sed '$d') + + # A system key is one-shot: once a system is registered on my it + # cannot be re-registered, because the key is never freed (licensing + # safeguard). The backend answers 409 in that case; surface it with a + # dedicated exit code (2) so ns.subscription can show a specific message + # instead of the generic "invalid secret / server not found". + if [ "$http_code" = "409" ]; then + >&2 echo "[ERROR] system already registered on my" + exit 2 + fi system_id=$(echo "$register_resp" | jq -r '.data.system_key // empty' 2>/dev/null) ;; From f58ea23e486d18963895a9125374cf611cb9c118 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Tue, 30 Jun 2026 11:09:35 +0200 Subject: [PATCH 09/41] chore(ns-ui): bump pin to nethsecurity-ui#746 head (899b407) Align the bundled ns-ui with the re-register UX fix in nethsecurity-ui#746. --- packages/ns-ui/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index 3b805961c..ced3012de 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=bc03487db4673e9a497cd79b9de65addb39392f0 +PKG_SOURCE_VERSION:=899b407dc9ab774fc1b45e4b727c9c3b4bdf1198 PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From 688672bbfa7157c26ac72cee79dc047872adb48a Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Tue, 30 Jun 2026 12:22:00 +0200 Subject: [PATCH 10/41] chore(ns-ui): bump pin to nethsecurity-ui#746 head (7402e3b) Realign the bundled ns-ui after the prettier formatting fix. --- packages/ns-ui/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index ced3012de..c8054eaa9 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=899b407dc9ab774fc1b45e4b727c9c3b4bdf1198 +PKG_SOURCE_VERSION:=7402e3b0969c1ba933cff3e874469a7d5b6a165a PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From 27881a87d23e1f345af103197b85f90acad9f480 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Tue, 30 Jun 2026 13:46:54 +0200 Subject: [PATCH 11/41] feat(subscription): expose system_url for enterprise units MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ns.subscription info now returns system_url (my-proxy-prod.onrender.com/systems/) for enterprise, so the UI can link the System ID to the portal — parity with ns8. --- packages/ns-api/files/ns.subscription | 1 + 1 file changed, 1 insertion(+) diff --git a/packages/ns-api/files/ns.subscription b/packages/ns-api/files/ns.subscription index 3f983f86c..44d8b44ad 100755 --- a/packages/ns-api/files/ns.subscription +++ b/packages/ns-api/files/ns.subscription @@ -73,6 +73,7 @@ def info(): if type == "enterprise": ret["organization"] = data.get("organization", "") ret["plan"] = "Nethesis Enterprise" + ret["system_url"] = f"https://my-proxy-prod.onrender.com/systems/{data['uuid']}" return ret From 8f1123eafc485904897154f80dfa0e577f3099d5 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Tue, 30 Jun 2026 13:46:54 +0200 Subject: [PATCH 12/41] chore(ns-ui): bump pin to nethsecurity-ui#746 head (4a0f10b) --- packages/ns-ui/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index c8054eaa9..72257dcd1 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=7402e3b0969c1ba933cff3e874469a7d5b6a165a +PKG_SOURCE_VERSION:=4a0f10b68f8b4be6e719fb24ae638263da001fc5 PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From 11bbba3376b1c1505e4a94d878f71230f21fc787 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Tue, 30 Jun 2026 14:24:40 +0200 Subject: [PATCH 13/41] feat(subscription): expose community system_url too (parity with ns8) Community units now get system_url=my.nethserver.com/servers/ so the UI links the System ID for community as well, matching the enterprise/ns8 behavior. --- packages/ns-api/files/ns.subscription | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/packages/ns-api/files/ns.subscription b/packages/ns-api/files/ns.subscription index 44d8b44ad..3d37207b7 100755 --- a/packages/ns-api/files/ns.subscription +++ b/packages/ns-api/files/ns.subscription @@ -74,6 +74,11 @@ def info(): ret["organization"] = data.get("organization", "") ret["plan"] = "Nethesis Enterprise" ret["system_url"] = f"https://my-proxy-prod.onrender.com/systems/{data['uuid']}" + else: + # Community: link the system to its my.nethserver.com page, matching ns8. + sub_id = (data.get("subscription") or {}).get("id") + if sub_id: + ret["system_url"] = f"https://my.nethserver.com/servers/{sub_id}" return ret From 80834dd81ca4780bdb3b0dcd3784b2464f9246be Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Tue, 7 Jul 2026 11:06:00 +0200 Subject: [PATCH 14/41] fix(subscription): send heartbeat before inventory on register register sent the inventory before the heartbeat, so a freshly-registered unit stayed unknown/pending on my until the (slower) inventory completed. Send the near-instant heartbeat first so the system flips to active right away; the inventory follows. Registration result is still reported on the heartbeat, as before. --- packages/ns-plug/files/register | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/packages/ns-plug/files/register b/packages/ns-plug/files/register index 33fe153ff..2e1b2b723 100755 --- a/packages/ns-plug/files/register +++ b/packages/ns-plug/files/register @@ -101,10 +101,12 @@ uci set ns-plug.config.secret="$secret" uci set ns-plug.config.repository_url="https://$system_id:$secret@distfeed.nethesis.it/repository/$type/nethsecurity" uci commit ns-plug reload_config -# Register the machine by sending the inventory for the first time -send-inventory +# Send the heartbeat first: it is near-instant and flips the system to +# "active" on my right away; the (slower) inventory follows. Registration +# success is still reported on the heartbeat result, as before. send-heartbeat exit_code=$? +send-inventory # Execute register hooks for script in $(find /usr/share/ns-plug/hooks/register -maxdepth 1 -executable -type f,l | sort); do From c76230501f1f3241765f28e35d95eacee5063e71 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Wed, 8 Jul 2026 12:13:57 +0200 Subject: [PATCH 15/41] feat(ns-api): dedalo login via My Nethesis device pairing ns.dedalo gains oidc-start/oidc-poll: the unit starts a device pairing on the hotspot manager, hands the verification URL to the browser and polls for the session token, then connects exactly like the password login did. Remote curl calls get timeouts; account name and user are stored in uci so the UI can show who the unit is linked to, and are cleared on password login and unregister. --- packages/ns-api/files/ns.dedalo | 112 +++++++++++++++++++++++++++++--- 1 file changed, 102 insertions(+), 10 deletions(-) diff --git a/packages/ns-api/files/ns.dedalo b/packages/ns-api/files/ns.dedalo index 210257c11..b91e7d51c 100755 --- a/packages/ns-api/files/ns.dedalo +++ b/packages/ns-api/files/ns.dedalo @@ -19,6 +19,7 @@ from euci import EUci tmp_dir = "/var/run/" token_file = f"{tmp_dir}/dedalo_token" +pairing_file = f"{tmp_dir}/dedalo_pairing.json" opts = ["network", "hotspot_id", "unit_name", "unit_description", "interface"] ## Utilities @@ -45,17 +46,10 @@ def setup(u): def login(args): u = EUci() try: - p = subprocess.run(['curl', '-L', '--url', f'https://{args["host"]}/api/login', '--header', 'Content-Type: application/json', '--data-binary', json.dumps(args)], check=True, capture_output=True, text=True) + p = subprocess.run(['curl', '-L', '-m', '15', '--connect-timeout', '5', '--url', f'https://{args["host"]}/api/login', '--header', 'Content-Type: application/json', '--data-binary', json.dumps(args)], check=True, capture_output=True, text=True) resp = json.loads(p.stdout) if 'token' in resp: - setup(u) - u.set("dedalo", "config", "splash_page", f'http://{args["host"]}/wings') - u.set("dedalo", "config", "aaa_url", f'https://{args["host"]}/wax/aaa') - u.set("dedalo", "config", "api_url", f'https://{args["host"]}/api') - u.commit("dedalo") - os.makedirs(tmp_dir, exist_ok = True) - with open(token_file, "w") as fp: - fp.write(resp["token"]) + _connect_to_host(u, args["host"], resp["token"]) return {"response": "success"} else: return utils.generic_error("login_failed") @@ -63,6 +57,87 @@ def login(args): print(e, file=sys.stderr) return {"success": False} + +def _connect_to_host(u, host, token, account_name="", account_user=""): + # same side effects as a successful password login: point the unit at + # the chosen hotspot manager and store the session token; the account + # info (from OIDC pairing) is kept to show who the unit is linked to + setup(u) + u.set("dedalo", "config", "splash_page", f'http://{host}/wings') + u.set("dedalo", "config", "aaa_url", f'https://{host}/wax/aaa') + u.set("dedalo", "config", "api_url", f'https://{host}/api') + for opt, value in (("account_name", account_name), ("account_user", account_user)): + if value: + u.set("dedalo", "config", opt, value) + else: + try: + u.delete("dedalo", "config", opt) + except: + pass + u.commit("dedalo") + os.makedirs(tmp_dir, exist_ok = True) + with open(token_file, "w") as fp: + fp.write(token) + +def oidc_start(args): + host = args.get("host") or "my.nethspot.com" + u = EUci() + unit_name = u.get("dedalo", "config", "unit_name", default="") + if not unit_name: + with open('/proc/sys/kernel/hostname', 'r') as fp: + unit_name = fp.read().strip() + try: + p = subprocess.run(['curl', '-s', '-L', '-m', '15', '--connect-timeout', '5', '-X', 'POST', '-w', '\n%{http_code}', '--url', f'https://{host}/api/auth/oidc/device/start', '--header', 'Content-Type: application/json', '--data-binary', json.dumps({"unit_name": unit_name})], check=True, capture_output=True, text=True) + body, _, http_code = p.stdout.rpartition('\n') + if http_code == '404': + # hotspot manager without OIDC device pairing support + return utils.generic_error("oidc_not_supported") + resp = json.loads(body) + except Exception as e: + print(e, file=sys.stderr) + return utils.generic_error("pairing_start_failed") + if 'device_code' not in resp or 'verification_url' not in resp: + return utils.generic_error("pairing_start_failed") + # the device_code stays on the unit: the browser only ever sees the + # verification_url (carrying the public pair_id) + os.makedirs(tmp_dir, exist_ok = True) + fd = os.open(pairing_file, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600) + with os.fdopen(fd, 'w') as fp: + json.dump({"host": host, "device_code": resp["device_code"]}, fp) + return { + "verification_url": resp["verification_url"], + "expires_in": resp.get("expires_in", 600), + "interval": resp.get("interval", 2), + } + +def oidc_poll(): + u = EUci() + try: + with open(pairing_file, 'r') as fp: + pairing = json.load(fp) + except: + return utils.generic_error("no_pairing_in_progress") + host = pairing["host"] + try: + p = subprocess.run(['curl', '-s', '-L', '-m', '15', '--connect-timeout', '5', '--url', f'https://{host}/api/auth/oidc/device/poll', '--header', 'Content-Type: application/json', '--data-binary', json.dumps({"device_code": pairing["device_code"]})], check=True, capture_output=True, text=True) + resp = json.loads(p.stdout) + except Exception as e: + # transient error talking to the hotspot manager: keep polling + print(e, file=sys.stderr) + return {"status": "pending"} + status = resp.get("status", "") + if status == "ready": + os.remove(pairing_file) + _connect_to_host(u, host, resp["token"], resp.get("account_name", ""), resp.get("logged_by", "")) + return {"status": "success", "account_name": resp.get("account_name", "")} + if status == "failed": + os.remove(pairing_file) + return {"status": "failed", "error": resp.get("error", "unknown")} + if status == "expired": + os.remove(pairing_file) + return {"status": "expired"} + return {"status": "pending"} + def list_sessions(): process = subprocess.run(["/usr/bin/dedalo", "query", "list"], capture_output=True, text=True) if not process.stdout: @@ -131,7 +206,7 @@ def list_parents(): u = EUci() try: api_url = u.get("dedalo", "config", "api_url") - p = subprocess.run(['curl', '-L', '-s', '--url', f'{api_url}/hotspots', '--header', f"Token: {_get_token()}"], capture_output=True, text=True) + p = subprocess.run(['curl', '-L', '-s', '-m', '15', '--connect-timeout', '5', '--url', f'{api_url}/hotspots', '--header', f"Token: {_get_token()}"], capture_output=True, text=True) resp = json.loads(p.stdout) for p in resp["data"]: parents.append({"id": p["id"], "name": p["name"], "description": p["description"]}) @@ -148,6 +223,12 @@ def unregister(): except Exception as e: print(e, file=sys.stderr) return utils.generic_error("unregister_failed") + try: + u.delete("dedalo", "config", "account_name") + u.delete("dedalo", "config", "account_user") + u.commit("dedalo") + except: + pass try: firewall.delete_linked_sections(EUci(), "dedalo/config") subprocess.run(["/sbin/ifdown", "dedalo"], capture_output=True, check=True) @@ -178,6 +259,10 @@ def get_configuration(): with open('/proc/sys/kernel/hostname', 'r') as fp: ret["unit_name"] = fp.read().strip() ret["connected"] = os.path.exists(token_file) + ret["account_name"] = u.get("dedalo", "config", "account_name", default="") + ret["account_user"] = u.get("dedalo", "config", "account_user", default="") + api_url = u.get("dedalo", "config", "api_url", default="") + ret["manager_host"] = api_url.replace("https://", "").replace("/api", "") return {"configuration": ret} def set_configuration(args): @@ -259,6 +344,8 @@ cmd = sys.argv[1] if cmd == 'list': print(json.dumps({ "login": {"host": "my.nethspot.com", "username": "myuser", "password": "mypassword"}, + "oidc-start": {"host": "my.nethspot.com"}, + "oidc-poll": {}, "list-sessions": {}, "list-parents": {}, "list-devices": {}, @@ -285,6 +372,11 @@ else: elif action == "login": args = json.loads(sys.stdin.read()) ret = login(args) + elif action == "oidc-start": + args = json.loads(sys.stdin.read()) + ret = oidc_start(args) + elif action == "oidc-poll": + ret = oidc_poll() elif action == "set-configuration": args = json.loads(sys.stdin.read()) ret = set_configuration(args) From 8bb5e967d2fb6d692f82bd000ac690fe088bd9dd Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Wed, 8 Jul 2026 12:13:57 +0200 Subject: [PATCH 16/41] chore(ns-ui): bump pin to the My Nethesis hotspot login --- packages/ns-ui/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index 72257dcd1..d5e0001b3 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=4a0f10b68f8b4be6e719fb24ae638263da001fc5 +PKG_SOURCE_VERSION:=9209ee9bdc426f109700df33bdd9ebdaf5c3ac3c PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From 185ec3212218fb662e4c713fd9cd64c8f3746263 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Wed, 8 Jul 2026 17:49:53 +0200 Subject: [PATCH 17/41] chore(ns-ui): bump pin to the hotspot manager link --- packages/ns-ui/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index d5e0001b3..82edf0ea6 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=9209ee9bdc426f109700df33bdd9ebdaf5c3ac3c +PKG_SOURCE_VERSION:=29d98833ea1dd94c2916abfd17f70b3f973c38ed PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From e11c355491f6f4aa3d77d883ac995fa76fc42eb1 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Thu, 9 Jul 2026 10:39:58 +0200 Subject: [PATCH 18/41] chore(ns-ui): bump pin to the refined My Nethesis login UX --- packages/ns-ui/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index 82edf0ea6..a52ebd1ba 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=29d98833ea1dd94c2916abfd17f70b3f973c38ed +PKG_SOURCE_VERSION:=536b2fd2b10bcc29735c73a5d91b26d41c8f9ab5 PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From efc963cf49846022dc950627c786546424db7b87 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Thu, 9 Jul 2026 12:08:39 +0200 Subject: [PATCH 19/41] fix(ns-plug): log send failures to syslog --- packages/ns-plug/files/remote-backup | 5 +++-- packages/ns-plug/files/send-backup | 7 ++++++- packages/ns-plug/files/send-heartbeat | 8 ++++++-- packages/ns-plug/files/send-inventory | 10 ++++++++-- 4 files changed, 23 insertions(+), 7 deletions(-) diff --git a/packages/ns-plug/files/remote-backup b/packages/ns-plug/files/remote-backup index 829ee40b3..b54d087d3 100755 --- a/packages/ns-plug/files/remote-backup +++ b/packages/ns-plug/files/remote-backup @@ -55,8 +55,9 @@ if [ "$TYPE" = "enterprise" ]; then BASE="$COLLECT_URL/backups" # --fail-with-body: exit 22 on 4xx/5xx while still writing the body - # so the caller can inspect the error payload. - curl_args="--silent --location-trusted --fail-with-body --user $SYSTEM_ID:$SYSTEM_SECRET" + # so the caller can inspect the error payload. --show-error keeps + # curl's own message (DNS, TLS, timeout) on stderr despite --silent. + curl_args="--silent --show-error --location-trusted --fail-with-body --user $SYSTEM_ID:$SYSTEM_SECRET" case "$cmd" in list) diff --git a/packages/ns-plug/files/send-backup b/packages/ns-plug/files/send-backup index bb16d2d72..988eafd97 100644 --- a/packages/ns-plug/files/send-backup +++ b/packages/ns-plug/files/send-backup @@ -29,7 +29,12 @@ send() { if [ -s "$PASSPHRASE" ]; then # send encrypted backup gpg --batch -c --yes --passphrase-file "$PASSPHRASE" "$BACKUP" - remote-backup upload "$BACKUP.gpg" + # cron discards job output: surface upload failures in syslog. + # The md5 marker is not saved, so the next run retries. + if ! err=$(remote-backup upload "$BACKUP.gpg" 2>&1); then + logger -t send-backup "backup upload failed: $err" + exit 1 + fi mv "$MD5" "$MD5_LAST" else # password not set, abort upload diff --git a/packages/ns-plug/files/send-heartbeat b/packages/ns-plug/files/send-heartbeat index 8f4d02c0e..a7706e750 100755 --- a/packages/ns-plug/files/send-heartbeat +++ b/packages/ns-plug/files/send-heartbeat @@ -35,9 +35,13 @@ case "$TYPE" in exit 0 fi - /usr/bin/curl -m 30 --retry 3 -L -s -X POST \ + # cron discards job output: surface failures (DNS, TLS, + # timeouts, HTTP errors) in syslog via logger. + err=$(/usr/bin/curl -m 30 --retry 3 -L -sSf -X POST \ --user "$SYSTEM_ID:$SYSTEM_SECRET" \ - "$COLLECT_URL/heartbeat" >/dev/null + -o /dev/null -w 'HTTP %{http_code}' \ + "$COLLECT_URL/heartbeat" 2>&1) || \ + logger -t send-heartbeat "heartbeat send failed: $err" ;; community) URL=$(uci -q get ns-plug.config.alerts_url)"heartbeats/store" diff --git a/packages/ns-plug/files/send-inventory b/packages/ns-plug/files/send-inventory index b6658f672..eee533c32 100755 --- a/packages/ns-plug/files/send-inventory +++ b/packages/ns-plug/files/send-inventory @@ -34,13 +34,19 @@ case "$TYPE" in exit 0 fi - /usr/sbin/phonehome | /usr/bin/curl -m 180 --retry 3 -L -s -X POST \ + # cron discards job output: surface failures (DNS, TLS, + # timeouts, HTTP errors) in syslog via logger. + err=$(/usr/sbin/phonehome | /usr/bin/curl -m 180 --retry 3 -L -sSf -X POST \ --user "$SYSTEM_ID:$SYSTEM_SECRET" \ -H "Content-Type: application/json" \ - --data-binary @- "$COLLECT_URL/inventory" >/dev/null + --data-binary @- \ + -o /dev/null -w 'HTTP %{http_code}' \ + "$COLLECT_URL/inventory" 2>&1) if [ $? -eq 0 ]; then status="success" + else + logger -t send-inventory "inventory send failed: $err" fi ;; community) From f05dd7425ebad93ebc2fe1a8868647fd74010742 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Thu, 9 Jul 2026 14:49:18 +0200 Subject: [PATCH 20/41] fix(ns-api): return expected register/oidc errors as validation errors --- packages/ns-api/files/ns.dedalo | 14 ++++++++++---- packages/ns-api/files/ns.subscription | 8 +++++--- 2 files changed, 15 insertions(+), 7 deletions(-) diff --git a/packages/ns-api/files/ns.dedalo b/packages/ns-api/files/ns.dedalo index b91e7d51c..546eb7f53 100755 --- a/packages/ns-api/files/ns.dedalo +++ b/packages/ns-api/files/ns.dedalo @@ -89,15 +89,18 @@ def oidc_start(args): try: p = subprocess.run(['curl', '-s', '-L', '-m', '15', '--connect-timeout', '5', '-X', 'POST', '-w', '\n%{http_code}', '--url', f'https://{host}/api/auth/oidc/device/start', '--header', 'Content-Type: application/json', '--data-binary', json.dumps({"unit_name": unit_name})], check=True, capture_output=True, text=True) body, _, http_code = p.stdout.rpartition('\n') + # Expected outcomes of the user-provided host (manager without OIDC + # support, wrong/unreachable host) are validation errors: the UI + # shows them inline without the global error toast. if http_code == '404': # hotspot manager without OIDC device pairing support - return utils.generic_error("oidc_not_supported") + return utils.validation_error("host", "oidc_not_supported") resp = json.loads(body) except Exception as e: print(e, file=sys.stderr) - return utils.generic_error("pairing_start_failed") + return utils.validation_error("host", "pairing_start_failed") if 'device_code' not in resp or 'verification_url' not in resp: - return utils.generic_error("pairing_start_failed") + return utils.validation_error("host", "pairing_start_failed") # the device_code stays on the unit: the browser only ever sees the # verification_url (carrying the public pair_id) os.makedirs(tmp_dir, exist_ok = True) @@ -132,7 +135,10 @@ def oidc_poll(): return {"status": "success", "account_name": resp.get("account_name", "")} if status == "failed": os.remove(pairing_file) - return {"status": "failed", "error": resp.get("error", "unknown")} + # NB: don't name the key "error" — a top-level "error" key makes + # nethsecurity-api reply 500 (application-error convention) and the + # failed status would never reach the UI as data. + return {"status": "failed", "reason": resp.get("error", "unknown")} if status == "expired": os.remove(pairing_file) return {"status": "expired"} diff --git a/packages/ns-api/files/ns.subscription b/packages/ns-api/files/ns.subscription index 3d37207b7..3dc66039e 100755 --- a/packages/ns-api/files/ns.subscription +++ b/packages/ns-api/files/ns.subscription @@ -25,10 +25,12 @@ def register(args): return {"result": "success"} # Exit code 2: the system is already registered on my and its key cannot be - # reused (one-shot registration). Report it as-is instead of falling back to - # community, so the UI can guide the user to create a new system. + # reused (one-shot registration). Report it as a validation error on the + # secret (expected outcome of user input, no global error toast) instead of + # falling back to community, so the UI can guide the user to create a new + # system. if enterprise.returncode == 2: - return utils.generic_error("system_already_registered") + return utils.validation_error("secret", "system_already_registered") community = subprocess.run(["/usr/sbin/register", "community", secret, '5'], capture_output=True) if community.returncode == 0: From 367748ad9b5ba8c0ea21b3558346b286d2580345 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Thu, 9 Jul 2026 14:51:21 +0200 Subject: [PATCH 21/41] chore(ns-ui): bump pin to the inline login errors fix --- packages/ns-ui/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index a52ebd1ba..24bfac1b3 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=536b2fd2b10bcc29735c73a5d91b26d41c8f9ab5 +PKG_SOURCE_VERSION:=0ef3710d553fb883213ce85dbbfc5ffb865b4b5a PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From ee04169f785f5b3046c48e18bab5281a4d95e26d Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Mon, 13 Jul 2026 12:22:57 +0200 Subject: [PATCH 22/41] chore(ns-ui): bump pin to the session status card --- packages/ns-ui/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index 24bfac1b3..c94898036 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=0ef3710d553fb883213ce85dbbfc5ffb865b4b5a +PKG_SOURCE_VERSION:=11561d797cd17d8beadb457090e701eb95645be4 PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From 91a172b1417eeae53bb7ba0700aa3ab0a56f4e77 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Tue, 14 Jul 2026 14:05:44 +0200 Subject: [PATCH 23/41] feat(ns-api): expose the my system name in subscription info --- packages/ns-api/files/ns.subscription | 3 +++ packages/ns-plug/files/subscription-info | 1 + 2 files changed, 4 insertions(+) diff --git a/packages/ns-api/files/ns.subscription b/packages/ns-api/files/ns.subscription index 3dc66039e..82f12bdb3 100755 --- a/packages/ns-api/files/ns.subscription +++ b/packages/ns-api/files/ns.subscription @@ -74,6 +74,9 @@ def info(): # the real plan name untouched. if type == "enterprise": ret["organization"] = data.get("organization", "") + # The system name given on my at creation time (threaded by + # subscription-info from the collect /info payload). + ret["system_name"] = data.get("system_name", "") ret["plan"] = "Nethesis Enterprise" ret["system_url"] = f"https://my-proxy-prod.onrender.com/systems/{data['uuid']}" else: diff --git a/packages/ns-plug/files/subscription-info b/packages/ns-plug/files/subscription-info index 363f2ff67..59478f717 100755 --- a/packages/ns-plug/files/subscription-info +++ b/packages/ns-plug/files/subscription-info @@ -53,6 +53,7 @@ if [ "$type" = "enterprise" ]; then { uuid: ($s.system_id // ""), id: ($s.system_key // ""), + system_name: ($s.name // ""), organization: ($s.organization.name // ""), subscription: { status: (if $s.registered and (($s.suspended // false) | not) then "valid" else "invalid" end), From e9e91db07ccfc74dd63f1ab7806d10180508c50c Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Tue, 14 Jul 2026 14:06:26 +0200 Subject: [PATCH 24/41] chore(ns-ui): bump pin to the system name on subscription --- packages/ns-ui/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index c94898036..b9af8478a 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=11561d797cd17d8beadb457090e701eb95645be4 +PKG_SOURCE_VERSION:=22329204b8f9011f8295661505d9755fa2ab0364 PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From eaf28afa453f05f1c9b8f2f38677617a14a1b67a Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Mon, 20 Jul 2026 15:31:21 +0200 Subject: [PATCH 25/41] chore(ns-ui): bump pin to nethsecurity-ui#746 head (c6afb3a) Follow the rebase of nethsecurity-ui#746 onto its current main. --- packages/ns-ui/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index b9af8478a..d5d2385f3 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=22329204b8f9011f8295661505d9755fa2ab0364 +PKG_SOURCE_VERSION:=c6afb3a79d0db8bfb098ff23a56fa453e912b883 PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From 27d12c0b3a499c6d2becdc3ca7bb6bb84be889f0 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Mon, 20 Jul 2026 16:23:04 +0200 Subject: [PATCH 26/41] chore(ns-ui): bump pin to nethsecurity-ui#746 head (2cacc72) Follow the lint fix on nethsecurity-ui#746. --- packages/ns-ui/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index d5d2385f3..32053d22b 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=c6afb3a79d0db8bfb098ff23a56fa453e912b883 +PKG_SOURCE_VERSION:=2cacc724f98367ed39b04d496ca9cca42702825d PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From 2affdfd512386f3390e37fd23351be2209ae1e17 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Mon, 27 Jul 2026 11:55:28 +0200 Subject: [PATCH 27/41] chore(ns-ui): bump pin to nethsecurity-ui#746 head (5b489e2b) Follow the rebase of nethsecurity-ui#746 onto its current main. --- packages/ns-ui/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index 32053d22b..0c38c92ee 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=2cacc724f98367ed39b04d496ca9cca42702825d +PKG_SOURCE_VERSION:=5b489e2b2c4b5011975af4c91fd5236c2c9969ab PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From a56b828807b0b09e39260cb96fd38156368fb139 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Tue, 28 Jul 2026 10:26:46 +0200 Subject: [PATCH 28/41] chore(ns-ui): bump pin to nethsecurity-ui#746 head (f910296) Follow the hotspot connection UI restructure on nethsecurity-ui#746. --- packages/ns-ui/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index 0c38c92ee..f2bce6391 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=5b489e2b2c4b5011975af4c91fd5236c2c9969ab +PKG_SOURCE_VERSION:=f910296bcdab8ef0b21b2547462d07bc7b0b7008 PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From 2976303fa8ae16c149f21a880abd0ef935931fdf Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Wed, 26 Aug 2026 14:52:28 +0200 Subject: [PATCH 29/41] chore(ns-ui): bump pin to nethsecurity-ui#746 head (e17c6fa) --- packages/ns-ui/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index f2bce6391..fb992886d 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -8,7 +8,7 @@ include $(TOPDIR)/rules.mk PKG_NAME:=ns-ui # renovate: datasource=github-releases depName=NethServer/nethsecurity-ui PKG_VERSION:=2.23.4 -PKG_RELEASE:=1 +PKG_RELEASE:=2 PKG_SOURCE_PROTO:=git PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=f910296bcdab8ef0b21b2547462d07bc7b0b7008 +PKG_SOURCE_VERSION:=e17c6fa3a0d4e6bae71272aee177c2e50c486455 PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From 4385319cea8015b854a9a458c76b1279c5f8ddb8 Mon Sep 17 00:00:00 2001 From: Edoardo Spadoni Date: Thu, 27 Aug 2026 15:52:32 +0200 Subject: [PATCH 30/41] fix(migration): rotate the enterprise feed credentials too migrate-to-my rotated the my credentials but never re-ran the feed hooks, so the blocklists and the apk repository kept authenticating as the legacy system: they work only while the old my still answers for it. Re-run the feed hooks after the rotation. ts-ip added banip's ban_allowurl entry only when no bl.nethesis.it one was present, so an entry written before a rotation kept the old pair for good. Enforce a single entry holding the current credentials. --- packages/ns-plug/files/migrate-to-my | 10 ++++++++++ packages/ns-threat_shield/files/ts-ip | 13 +++++++++++-- 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/packages/ns-plug/files/migrate-to-my b/packages/ns-plug/files/migrate-to-my index 9b88551ad..b82655bb0 100644 --- a/packages/ns-plug/files/migrate-to-my +++ b/packages/ns-plug/files/migrate-to-my @@ -95,5 +95,15 @@ EOI # no-op if victoria-metrics is not installed. /etc/init.d/vmalert reload 2>/dev/null || true +# The enterprise feeds hold the credentials inside their URLs, written at +# registration: rotating the pair here would leave them authenticating as the +# legacy system, which works only until the old my goes away. Re-run just the +# feed hooks -- the other register hooks act on a first registration. +for feed_setup in /usr/sbin/ts-ip /usr/sbin/ts-dns /usr/sbin/distfeed-setup; do + [ -x "$feed_setup" ] && "$feed_setup" || true +done +/etc/init.d/banip reload 2>/dev/null || true +/etc/init.d/adblock reload 2>/dev/null || true + logger -t migrate-to-my "migrated to my collect credentials" exit 0 diff --git a/packages/ns-threat_shield/files/ts-ip b/packages/ns-threat_shield/files/ts-ip index 8acc85333..bd4fd7f36 100755 --- a/packages/ns-threat_shield/files/ts-ip +++ b/packages/ns-threat_shield/files/ts-ip @@ -33,8 +33,17 @@ TYPE=$(uci -q get ns-plug.config.type) if [ ! -z "$SYSTEM_SECRET" ] && [ ! -z "$SYSTEM_ID" ]; then jq -s '.[0] * .[1]' /etc/banip/banip.nethesis.feeds /etc/banip/banip.feeds \ | sed -e "s/__USER__/$SYSTEM_ID/" -e "s/__PASSWORD__/$SYSTEM_SECRET/" -e "s/__TYPE__/$TYPE/" > /etc/banip/banip.custom.feeds - if ! uci -q get banip.global.ban_allowurl | grep -q bl.nethesis.it; then - uci add_list banip.global.ban_allowurl="https://$SYSTEM_ID:$SYSTEM_SECRET@bl.nethesis.it/plain/$TYPE/nethesis-blacklists/whitelist.global" + # The credentials are part of the URL, so an entry written before a key + # rotation keeps fetching as the old pair. Enforce one entry, the current + # one: compare the whole set, and del_list per entry (a multi-line value + # matches nothing). + allow_current="https://$SYSTEM_ID:$SYSTEM_SECRET@bl.nethesis.it/plain/$TYPE/nethesis-blacklists/whitelist.global" + allow_have=$(uci -q get banip.global.ban_allowurl | tr " " "\n" | grep bl.nethesis.it) + if [ "$allow_have" != "$allow_current" ]; then + for allow_stale in $allow_have; do + uci del_list banip.global.ban_allowurl="$allow_stale" + done + uci add_list banip.global.ban_allowurl="$allow_current" uci commit banip fi else From 1aaa7d4bece00e44da504455b44bed61524c51ee Mon Sep 17 00:00:00 2001 From: Tommaso Bailetti Date: Tue, 8 Sep 2026 10:11:19 +0200 Subject: [PATCH 31/41] addressed some styling/fix --- packages/ns-api/files/ns.backup | 36 ++++++++--------- packages/ns-api/files/ns.subscription | 7 +--- packages/ns-phonehome/files/phonehome | 12 +----- packages/ns-plug/files/config | 2 + packages/ns-plug/files/migrate-to-my | 9 +---- packages/ns-plug/files/register | 21 ++-------- packages/ns-plug/files/remote-backup | 39 +++++++----------- packages/ns-plug/files/send-backup | 9 +---- packages/ns-plug/files/send-heartbeat | 27 ++----------- packages/ns-plug/files/send-inventory | 40 ++++--------------- packages/ns-plug/files/subscription-info | 28 +++---------- packages/ns-plug/files/unregister | 13 +----- packages/victoria-metrics/files/vmalert.initd | 34 +++------------- 13 files changed, 63 insertions(+), 214 deletions(-) diff --git a/packages/ns-api/files/ns.backup b/packages/ns-api/files/ns.backup index 9049030da..85a13d7e5 100755 --- a/packages/ns-api/files/ns.backup +++ b/packages/ns-api/files/ns.backup @@ -169,26 +169,21 @@ elif cmd == 'call': elif action == 'registered-backup': if not os.path.exists(PASSPHRASE_PATH): - # Refuse the call before running sysupgrade/uploading, and emit - # valid JSON so the HTTP API wraps it as a 422 ValidationError - # the UI can render (the previous form printed a Python dict - # repr, which was silently dropped upstream and caused the run - # modal to stay open after a successful upload). - print(json.dumps(utils.validation_error('passphrase', 'missing'))) - sys.exit(0) - try: - # create backup - file_name = create_backup() - backup_path = f'{DOWNLOAD_PATH}{file_name}' - # upload backup to server and remove it from filesystem - completed_process = subprocess.run(['/usr/sbin/remote-backup', 'upload', backup_path], check=True, - capture_output=True) - os.remove(backup_path) - print(json.dumps({'message': 'success'})) - except subprocess.CalledProcessError as error: - print(json.dumps(utils.generic_error(f'remote upload failed'))) - except RuntimeError as error: - print(json.dumps(utils.generic_error(error.args[0]))) + print(utils.validation_error('passphrase', 'missing')) + else: + try: + # create backup + file_name = create_backup() + backup_path = f'{DOWNLOAD_PATH}{file_name}' + # upload backup to server and remove it from filesystem + completed_process = subprocess.run(['/usr/sbin/remote-backup', 'upload', backup_path], check=True, + capture_output=True) + os.remove(backup_path) + print(json.dumps({'message': 'success'})) + except subprocess.CalledProcessError as error: + print(json.dumps(utils.generic_error(f'remote upload failed'))) + except RuntimeError as error: + print(json.dumps(utils.generic_error(error.args[0]))) elif action == 'registered-restore': try: @@ -231,6 +226,7 @@ elif cmd == 'call': data = json.load(sys.stdin) subprocess.run(['/usr/sbin/remote-backup', 'delete', data['id']], check=True, capture_output=True, text=True) + # FIXME: check api # The remote side returns a structured JSON response; the UI # only needs a success flag, matching the pattern of the # other registered-* handlers (backup, restore). diff --git a/packages/ns-api/files/ns.subscription b/packages/ns-api/files/ns.subscription index 82f12bdb3..0cb913b39 100755 --- a/packages/ns-api/files/ns.subscription +++ b/packages/ns-api/files/ns.subscription @@ -24,11 +24,7 @@ def register(args): if enterprise.returncode == 0: return {"result": "success"} - # Exit code 2: the system is already registered on my and its key cannot be - # reused (one-shot registration). Report it as a validation error on the - # secret (expected outcome of user input, no global error toast) instead of - # falling back to community, so the UI can guide the user to create a new - # system. + # Exit code 2: the system is already registered on my. if enterprise.returncode == 2: return utils.validation_error("secret", "system_already_registered") @@ -72,6 +68,7 @@ def info(): # the organization explicitly (subscription-info threads organization.name) # and default the plan label to "Nethesis Enterprise". Community units keep # the real plan name untouched. + # FIXME: check with upstream if type == "enterprise": ret["organization"] = data.get("organization", "") # The system name given on my at creation time (threaded by diff --git a/packages/ns-phonehome/files/phonehome b/packages/ns-phonehome/files/phonehome index 95e92e1e2..08f64b71a 100755 --- a/packages/ns-phonehome/files/phonehome +++ b/packages/ns-phonehome/files/phonehome @@ -36,15 +36,6 @@ for func in dir(inventory): if func.startswith("info_"): info[func.removeprefix('info_')] = method(EUci()) -# Migration fingerprint. Populated only on enterprise units that went -# through migrate-to-my or the native my register — my uses this to -# track which units have already rotated off the translation proxy -# and decide when the proxy can be decommissioned. -migration = { - "from_legacy_system_id": u.get('ns-plug', 'config', 'legacy_system_id', default='') or None, - "migrated_at": u.get('ns-plug', 'config', 'migrated_at', default='') or None, -} - data = { "$schema": "https://schema.nethserver.org/facts/2022-12.json", "uuid": sid, @@ -70,8 +61,7 @@ data = { }, "pci": list(pci.values()), "mountpoints": mount_points, - "features": features, - "migration": migration + "features": features } } diff --git a/packages/ns-plug/files/config b/packages/ns-plug/files/config index 71e511bbc..592860445 100644 --- a/packages/ns-plug/files/config +++ b/packages/ns-plug/files/config @@ -6,6 +6,8 @@ config main 'config' option tls_verify '1' option backup_url 'https://backupd.nethesis.it' option collect_url 'https://my-proxy-prod.onrender.com/collect/api/systems' +# FIXME: add it ot migrate-to-my + option notifier_url 'https://my-proxy-prod.onrender.com/collect/api/services/mimir/alertmanager' option repository_url 'https://updates.nethsecurity.nethserver.org' option channel '' option tun_mtu '' diff --git a/packages/ns-plug/files/migrate-to-my b/packages/ns-plug/files/migrate-to-my index b82655bb0..ef9ff138d 100644 --- a/packages/ns-plug/files/migrate-to-my +++ b/packages/ns-plug/files/migrate-to-my @@ -42,8 +42,7 @@ [ "$(uci -q get ns-plug.config.migrated)" = "1" ] && exit 0 # Community units stay on the legacy my.nethserver.com infrastructure. -TYPE=$(uci -q get ns-plug.config.type) -if [ "$TYPE" != "enterprise" ]; then +if [ "$(uci -q get ns-plug.config.type)" != "enterprise" ]; then exit 0 fi @@ -70,11 +69,6 @@ if [ -z "$new_key" ] || [ -z "$new_secret" ]; then exit 0 fi -# Timestamp the rotation so phonehome can publish the event and my -# can plot the fleet migration curve / decide when the translation -# proxy can be decommissioned. -migrated_at=$(date -u +%Y-%m-%dT%H:%M:%SZ) - # Rotate atomically; legacy pair preserved for audit / rollback. uci -q batch <&2 echo "[ERROR] system already registered on my" exit 2 @@ -89,9 +80,6 @@ case "$type" in uci set ns-plug.config.type="enterprise" uci set ns-plug.config.api_url="$url" uci set ns-plug.config.collect_url="https://my-proxy-prod.onrender.com/collect/api/systems" - # Native my register: no legacy credentials to rotate, so - # mark the unit as already migrated. migrate-to-my will be a - # no-op on every subsequent run. uci set ns-plug.config.migrated="1" ;; esac @@ -101,12 +89,9 @@ uci set ns-plug.config.secret="$secret" uci set ns-plug.config.repository_url="https://$system_id:$secret@distfeed.nethesis.it/repository/$type/nethsecurity" uci commit ns-plug reload_config -# Send the heartbeat first: it is near-instant and flips the system to -# "active" on my right away; the (slower) inventory follows. Registration -# success is still reported on the heartbeat result, as before. send-heartbeat -exit_code=$? send-inventory +exit_code=$? # Execute register hooks for script in $(find /usr/share/ns-plug/hooks/register -maxdepth 1 -executable -type f,l | sort); do diff --git a/packages/ns-plug/files/remote-backup b/packages/ns-plug/files/remote-backup index b54d087d3..a7593b97d 100755 --- a/packages/ns-plug/files/remote-backup +++ b/packages/ns-plug/files/remote-backup @@ -6,17 +6,7 @@ # # -# Manage configuration backups. -# -# Enterprise units (type=enterprise) talk to my collect after the -# migrate-to-my credential rotation. Community units (type=community) -# keep using the legacy backupd.nethesis.it endpoint with the same -# URL layout they have always used — backupd still accepts both -# tenants behind the $TYPE/api/v2/backup/ path. -# -# Pipefail so the curl exit status survives the jq stage in `list`; -# without it a HTTP error on the server would be masked by a successful -# jq parse and ns.backup would report success to the UI. +# Manage remote backup # set -o pipefail @@ -46,19 +36,11 @@ fi cmd=${1:-list} if [ "$TYPE" = "enterprise" ]; then - /usr/sbin/migrate-to-my - - COLLECT_URL=$(uci -q get ns-plug.config.collect_url) - if [ -z "$COLLECT_URL" ]; then - exit_error "Collect URL not set. Pre-migration unit — retry later." - fi - - BASE="$COLLECT_URL/backups" - # --fail-with-body: exit 22 on 4xx/5xx while still writing the body - # so the caller can inspect the error payload. --show-error keeps - # curl's own message (DNS, TLS, timeout) on stderr despite --silent. + BASE="$(uci -q get ns-plug.config.collect_url)/backups" curl_args="--silent --show-error --location-trusted --fail-with-body --user $SYSTEM_ID:$SYSTEM_SECRET" + + # FIXME: check with upstream case "$cmd" in list) # my returns {code, message, data: {backups: [...]}} on @@ -112,20 +94,27 @@ case "$cmd" in ;; download) file=$2 - [ -z "$file" ] && exit_error "No file specified" + if [ -z "$file" ]; then + exit_error "No file specified" + fi output=${3-$file} curl $curl_args $base_url$file -J -o "$output" ;; upload) file=$2 - [ -z "$file" ] && exit_error "No file specified" + if [ -z "$file" ]; then + exit_error "No file specified" + fi curl $curl_args $base_url --upload-file $file ;; delete) file=$2 - [ -z "$file" ] && exit_error "No file specified" + if [ -z "$file" ]; then + exit_error "No file specified" + fi curl $curl_args -X DELETE $base_url$file ;; + *) help ;; diff --git a/packages/ns-plug/files/send-backup b/packages/ns-plug/files/send-backup index 988eafd97..8aa52057b 100644 --- a/packages/ns-plug/files/send-backup +++ b/packages/ns-plug/files/send-backup @@ -29,8 +29,7 @@ send() { if [ -s "$PASSPHRASE" ]; then # send encrypted backup gpg --batch -c --yes --passphrase-file "$PASSPHRASE" "$BACKUP" - # cron discards job output: surface upload failures in syslog. - # The md5 marker is not saved, so the next run retries. + # to surface error log from cron if ! err=$(remote-backup upload "$BACKUP.gpg" 2>&1); then logger -t send-backup "backup upload failed: $err" exit 1 @@ -50,12 +49,6 @@ if [ -z "$SYSTEM_ID" ] || [ -z "$SYSTEM_SECRET" ]; then exit 0 fi -# remote-backup handles the enterprise/community branching and calls -# migrate-to-my when needed; this script just prepares the payload and -# delegates the upload. An enterprise unit still waiting on the -# migration surfaces its error through remote-backup, which is caught -# by set -e above. - # Create the backup mkdir -p "$WORK_DIR" sysupgrade -q -k -b "$BACKUP" diff --git a/packages/ns-plug/files/send-heartbeat b/packages/ns-plug/files/send-heartbeat index a7706e750..15eef50ea 100755 --- a/packages/ns-plug/files/send-heartbeat +++ b/packages/ns-plug/files/send-heartbeat @@ -5,16 +5,7 @@ # SPDX-License-Identifier: GPL-2.0-only # -# Send the heartbeat. -# -# Enterprise units (type=enterprise) post to the my collect endpoint -# with the rotated my credentials; migrate-to-my runs up front so a -# unit upgraded from the legacy my.nethesis.it path transparently -# flips over on the first successful rotation. -# -# Community units (type=community) are left on the legacy -# my.nethserver.com heartbeat path — that infrastructure has no -# counterpart on the new my and is out of scope for this migration. +# Send the heartbeat SYSTEM_ID=$(uci -q get ns-plug.config.system_id) SYSTEM_SECRET=$(uci -q get ns-plug.config.secret) @@ -27,21 +18,9 @@ fi case "$TYPE" in enterprise) - /usr/sbin/migrate-to-my - - COLLECT_URL=$(uci -q get ns-plug.config.collect_url) - if [ -z "$COLLECT_URL" ]; then - # Pre-migration — retry on next tick. - exit 0 - fi - - # cron discards job output: surface failures (DNS, TLS, - # timeouts, HTTP errors) in syslog via logger. - err=$(/usr/bin/curl -m 30 --retry 3 -L -sSf -X POST \ + /usr/bin/curl -m 30 --retry 3 -L -sSf -X POST \ --user "$SYSTEM_ID:$SYSTEM_SECRET" \ - -o /dev/null -w 'HTTP %{http_code}' \ - "$COLLECT_URL/heartbeat" 2>&1) || \ - logger -t send-heartbeat "heartbeat send failed: $err" + "$(uci -q get ns-plug.config.collect_url)/heartbeat" >/dev/null ;; community) URL=$(uci -q get ns-plug.config.alerts_url)"heartbeats/store" diff --git a/packages/ns-plug/files/send-inventory b/packages/ns-plug/files/send-inventory index eee533c32..ece3a8398 100755 --- a/packages/ns-plug/files/send-inventory +++ b/packages/ns-plug/files/send-inventory @@ -5,13 +5,7 @@ # SPDX-License-Identifier: GPL-2.0-only # -# Send the inventory. -# -# Enterprise units post to my collect with the rotated my credentials; -# community units keep using the legacy my.nethserver.com inventory -# path. See send-heartbeat for the rationale — the two flows are kept -# parallel so the community infrastructure is never touched by the my -# migration. +# Send the inventory SYSTEM_ID=$(uci -q get ns-plug.config.system_id) SYSTEM_SECRET=$(uci -q get ns-plug.config.secret) @@ -26,42 +20,24 @@ status="error" case "$TYPE" in enterprise) - /usr/sbin/migrate-to-my - - COLLECT_URL=$(uci -q get ns-plug.config.collect_url) - if [ -z "$COLLECT_URL" ]; then - # Pre-migration — retry on next tick. - exit 0 - fi - - # cron discards job output: surface failures (DNS, TLS, - # timeouts, HTTP errors) in syslog via logger. - err=$(/usr/sbin/phonehome | /usr/bin/curl -m 180 --retry 3 -L -sSf -X POST \ + /usr/sbin/phonehome | /usr/bin/curl -m 180 --retry 3 -L -sSf -X POST \ --user "$SYSTEM_ID:$SYSTEM_SECRET" \ -H "Content-Type: application/json" \ --data-binary @- \ - -o /dev/null -w 'HTTP %{http_code}' \ - "$COLLECT_URL/inventory" 2>&1) - - if [ $? -eq 0 ]; then - status="success" - else - logger -t send-inventory "inventory send failed: $err" - fi + "$(uci -q get ns-plug.config.collect_url)/inventory" > /dev/null ;; community) - URL=$(uci -q get ns-plug.config.inventory_url) echo "{\"data\": {\"lk\": \"$SYSTEM_ID\", \"data\": $(/usr/sbin/inventory) }}" | \ /usr/bin/curl -m 180 --retry 5 -L -s \ --header "Authorization: token $SYSTEM_SECRET" \ --header "Content-Type: application/json" \ --header "Accept: application/json" \ - --data-binary @- "$URL" > /dev/null - - if [ $? -eq 0 ]; then - status="success" - fi + --data-binary @- "$(uci -q get ns-plug.config.inventory_url)" > /dev/null ;; esac +if [ $? -eq 0 ]; then + status="success" +fi + echo '{"status": "'$status'", "last_attempt": "'$(date -Iseconds)'"}' > /tmp/inventory-sent.json diff --git a/packages/ns-plug/files/subscription-info b/packages/ns-plug/files/subscription-info index 59478f717..dee2816e2 100755 --- a/packages/ns-plug/files/subscription-info +++ b/packages/ns-plug/files/subscription-info @@ -6,16 +6,8 @@ # # -# Retrieve subscription information. -# -# Enterprise units query the my collect /info endpoint with their -# native credentials and synthesise a payload shaped like the legacy -# my-old /api/systems/info response the UI expects. The subscription -# plan / expiration fields are left empty — the new my data model -# no longer tracks them at the system level; the ns.subscription -# info handler falls back to "-" / 0 / "active" in those cases. -# -# Community units keep hitting my.nethserver.com as before. +# Retrieve subscription information +# The script takes an optional timeout parameter # timeout=${1:-20} @@ -23,6 +15,7 @@ timeout=${1:-20} system_id=$(uci -q get ns-plug.config.system_id) if [ -z "$system_id" ]; then + # no subscription echo '{"uuid": ""}' exit 0 fi @@ -31,22 +24,11 @@ type=$(uci -q get ns-plug.config.type) secret=$(uci -q get ns-plug.config.secret) if [ "$type" = "enterprise" ]; then - /usr/sbin/migrate-to-my - - collect_url=$(uci -q get ns-plug.config.collect_url) - if [ -z "$collect_url" ]; then - # Pre-migration unit — nothing to report yet; caller falls - # back to a default payload built from ns-plug.config. - exit 1 - fi - - # /info returns {code, message, data: {system_id, registered, - # registered_at, suspended, organization: {name, ...}, ...}}. - # Translate to the legacy shape the UI/info handler parses. + # TODO: Controllare risposta con la UI resp=$(/usr/bin/curl -f -s -m $timeout --retry-delay 1 --retry 2 -L \ -H "Accept: application/json" \ --user "$system_id:$secret" \ - "$collect_url/info") || exit $? + "$(uci -q get ns-plug.config.collect_url)/info") || exit $? jq -c ' .data as $s | diff --git a/packages/ns-plug/files/unregister b/packages/ns-plug/files/unregister index 5fe069815..45379596c 100755 --- a/packages/ns-plug/files/unregister +++ b/packages/ns-plug/files/unregister @@ -11,19 +11,14 @@ SYSTEM_ID=$(uci -q get ns-plug.config.system_id) SYSTEM_SECRET=$(uci -q get ns-plug.config.secret) -TYPE=$(uci -q get ns-plug.config.type) if [ -z "$SYSTEM_ID" ] || [ -z "$SYSTEM_SECRET" ]; then # System ID and System secret not found, configure ns-plug to enable it exit 0 fi -# Release the legacy slot on my-old for migrated enterprise units — -# the /api/Utils/freekey PHP endpoint still exists on my-ent and lets -# the old dashboard record the unit as gone. Native enterprise units -# have no legacy slot to release (registered directly on my collect), -# and community has no freekey equivalent on dartagnan. -if [ "$TYPE" = "enterprise" ]; then +# Release the legacy slot on my-old for migrated enterprise units +if [ "$(uci -q get ns-plug.config.type)" = "enterprise" ]; then LEGACY_ID=$(uci -q get ns-plug.config.legacy_system_id) LEGACY_SECRET=$(uci -q get ns-plug.config.legacy_secret) if [ -n "$LEGACY_ID" ] && [ -n "$LEGACY_SECRET" ]; then @@ -42,12 +37,8 @@ uci set ns-plug.config.inventory_url="" uci set ns-plug.config.system_id="" uci set ns-plug.config.secret="" uci set ns-plug.config.repository_url="https://updates.nethsecurity.nethserver.org/$(cat /etc/repo-channel)" -# Drop the enterprise migration fingerprint so a subsequent register -# starts from a clean slate and migrate-to-my can run again if the -# unit re-registers with legacy credentials. uci -q delete ns-plug.config.collect_url uci -q delete ns-plug.config.migrated -uci -q delete ns-plug.config.migrated_at uci -q delete ns-plug.config.legacy_system_id uci -q delete ns-plug.config.legacy_secret diff --git a/packages/victoria-metrics/files/vmalert.initd b/packages/victoria-metrics/files/vmalert.initd index c3e35cbbd..c17321129 100644 --- a/packages/victoria-metrics/files/vmalert.initd +++ b/packages/victoria-metrics/files/vmalert.initd @@ -19,38 +19,14 @@ start_service() { config_get datasource_url main datasource_url "http://localhost:8428" config_get http_listen_addr main http_listen_addr "127.0.0.1:8081" - # Forward alerts to the new my for enterprise systems. Two windows: - # - Pre-cutover (migrated!=1): POST to the credential-translation proxy at - # my.nethesis.it/proxy/alerts with the still-legacy ns-plug credentials - # (system_id:secret); the proxy maps them to the new my credentials. - # - Post-cutover (migrated=1 + collect_url): migrate-to-my has rotated the - # credentials to native my credentials, so POST straight to the native - # Mimir alertmanager derived from collect_url — the legacy-only proxy - # would 401 the rotated credentials. vmalert appends /api/v2/alerts, so - # the notifier URL is the alertmanager base WITHOUT that suffix. - local system_id system_secret system_type migrated collect_url - local notifier_url notifier_user notifier_pass + # Forward alerts to the new my for enterprise systems. + local system_id system_secret system_type notifier_url notifier_user notifier_pass config_load ns-plug 2>/dev/null && { config_get system_id config system_id "" config_get system_secret config secret "" config_get system_type config type "" - config_get migrated config migrated "" - config_get collect_url config collect_url "" + config_get notifier_url config notifier_url "" } - - notifier_url="" - if [ "$system_type" = "enterprise" ] && [ -n "$system_id" ] && [ -n "$system_secret" ]; then - if [ "$migrated" = "1" ] && [ -n "$collect_url" ]; then - # Native my Mimir alertmanager, derived from collect_url, e.g. - # https://my-proxy-prod.onrender.com/collect/api/systems - # -> https://my-proxy-prod.onrender.com/collect/api/services/mimir/alertmanager - notifier_url="${collect_url%%/collect/*}/collect/api/services/mimir/alertmanager" - else - notifier_url="https://my-proxy-prod.onrender.com/proxy/alerts" - fi - notifier_user="$system_id" - notifier_pass="$system_secret" - fi procd_open_instance procd_set_param command $PROG @@ -69,10 +45,10 @@ start_service() { local pass_file="/var/run/vmalert/notifier.pass" mkdir -p /var/run/vmalert chmod 700 /var/run/vmalert - ( umask 077; printf '%s' "$notifier_pass" > "$pass_file" ) + ( umask 077; printf '%s' "$system_secret" > "$pass_file" ) procd_append_param command -notifier.url="$notifier_url" - procd_append_param command -notifier.basicAuth.username="$notifier_user" + procd_append_param command -notifier.basicAuth.username="$system_id" procd_append_param command -notifier.basicAuth.passwordFile="$pass_file" fi From a05e6e3914e768dfc72a02d55d2ebde14ed4ee19 Mon Sep 17 00:00:00 2001 From: Tommaso Bailetti Date: Tue, 8 Sep 2026 10:16:12 +0200 Subject: [PATCH 32/41] running migrate on package install --- packages/ns-plug/Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/packages/ns-plug/Makefile b/packages/ns-plug/Makefile index f4360e221..974a9b82d 100644 --- a/packages/ns-plug/Makefile +++ b/packages/ns-plug/Makefile @@ -47,6 +47,7 @@ if [ -z "$${IPKG_INSTROOT}" ]; then /etc/init.d/ns-plug-alert-proxy enable /etc/init.d/ns-plug-alert-proxy restart /etc/init.d/vmalert reload + /usr/libexec/migrate-to-my fi exit 0 endef From a096454fd4de17c2d7309854c38496d3e2030fc4 Mon Sep 17 00:00:00 2001 From: Tommaso Bailetti Date: Tue, 8 Sep 2026 10:22:00 +0200 Subject: [PATCH 33/41] removed proxy alert --- AGENTS.md | 2 +- packages/ns-ha/README.md | 3 +- packages/ns-plug/Makefile | 6 - packages/ns-plug/files/ns-plug-alert-proxy | 182 ------------------ .../ns-plug/files/ns-plug-alert-proxy.init | 30 --- packages/telegraf/README.md | 1 - packages/telegraf/files/telegraf-services | 3 +- packages/victoria-metrics/README.md | 58 +----- packages/victoria-metrics/files/vmalert.initd | 3 - 9 files changed, 5 insertions(+), 283 deletions(-) delete mode 100644 packages/ns-plug/files/ns-plug-alert-proxy delete mode 100644 packages/ns-plug/files/ns-plug-alert-proxy.init diff --git a/AGENTS.md b/AGENTS.md index a0e4eb0e7..bc77854c5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -57,7 +57,7 @@ recent host Ruby (4.x), so the site is built in a `ruby:3.3` container and the - `builder/apply-patches.sh` strips the `patches/` prefix and applies each patch into the matching upstream source directory. - `files/` is the rootfs overlay for the final image. `files/etc/uci-defaults` holds first-boot defaults. - Runtime web stack: **nginx** serves `ns-ui` from `/www-ns` and proxies `/api/` → **ns-api-server** on `127.0.0.1:8090`; `ns-api-server` handles auth/JWT and forwards calls to ubus/rpcd handlers. -- System monitoring alerts, including HA alerts, follow the Telegraf → Victoria Metrics/vmalert → `ns-plug-alert-proxy` path rather than sending legacy portal alerts directly from service scripts. +- System monitoring alerts, including HA alerts, follow the Telegraf → Victoria Metrics/vmalert -> Mimir. - Many local packages are thin wrappers around upstream code. When changing behavior in one of those areas, inspect the matching upstream repo first and treat the local package as integration glue. --- diff --git a/packages/ns-ha/README.md b/packages/ns-ha/README.md index 7ee191088..9b87024d9 100644 --- a/packages/ns-ha/README.md +++ b/packages/ns-ha/README.md @@ -366,8 +366,7 @@ Keepalived Statistics: HA alerts are evaluated by **vmalert** from metrics exported by `/usr/libexec/telegraf-ha-alert`. The collector and HA alert rules are installed by the always-present `telegraf` and -`victoria-metrics` packages. When alerts fire, `ns-plug-alert-proxy` forwards the legacy HA alert IDs -to the monitoring portal if the machine has a valid registration. +`victoria-metrics` packages. Available alerts are: diff --git a/packages/ns-plug/Makefile b/packages/ns-plug/Makefile index 974a9b82d..1ef7be8d4 100644 --- a/packages/ns-plug/Makefile +++ b/packages/ns-plug/Makefile @@ -44,8 +44,6 @@ if [ -z "$${IPKG_INSTROOT}" ]; then /etc/init.d/cron restart /usr/libexec/ns-plug/40_ns-plug_mwan_hooks /etc/init.d/ns-plug restart - /etc/init.d/ns-plug-alert-proxy enable - /etc/init.d/ns-plug-alert-proxy restart /etc/init.d/vmalert reload /usr/libexec/migrate-to-my fi @@ -59,8 +57,6 @@ if [ -z "$${IPKG_INSTROOT}" ]; then crontab -l | grep -v "/usr/sbin/send-inventory" | sort | uniq | crontab - crontab -l | grep -v "/usr/sbin/send-heartbeat" | sort | uniq | crontab - sed -i '/\/usr\/libexec\/ns-plug\/mwan-hooks/d' /etc/mwan3.user - /etc/init.d/ns-plug-alert-proxy stop - /etc/init.d/ns-plug-alert-proxy disable fi exit 0 endef @@ -78,9 +74,7 @@ define Package/ns-plug/install $(INSTALL_DIR) $(1)/usr/libexec/ns-plug $(INSTALL_DIR) $(1)/usr/libexec/mwan-hooks $(INSTALL_BIN) ./files/ns-plug.init $(1)/etc/init.d/ns-plug - $(INSTALL_BIN) ./files/ns-plug-alert-proxy.init $(1)/etc/init.d/ns-plug-alert-proxy $(INSTALL_BIN) ./files/ns-plug $(1)/usr/sbin/ns-plug - $(INSTALL_BIN) ./files/ns-plug-alert-proxy $(1)/usr/sbin/ns-plug-alert-proxy $(INSTALL_BIN) ./files/distfeed-setup $(1)/usr/sbin/distfeed-setup $(INSTALL_BIN) ./files/apk-official $(1)/usr/sbin/apk-official $(INSTALL_BIN) ./files/migrate-to-my $(1)/usr/sbin diff --git a/packages/ns-plug/files/ns-plug-alert-proxy b/packages/ns-plug/files/ns-plug-alert-proxy deleted file mode 100644 index b1167c7b3..000000000 --- a/packages/ns-plug/files/ns-plug-alert-proxy +++ /dev/null @@ -1,182 +0,0 @@ -#!/usr/bin/python3 - -# -# Copyright (C) 2026 Nethesis S.r.l. -# SPDX-License-Identifier: GPL-2.0-only -# - -""" -Alert proxy: receives Alertmanager-like notifications from vmalert and -forwards selected alerts to the legacy my.nethesis.it / my.nethserver.com -monitoring portals. - -Only the following alerts are forwarded: - - WanDown → wan::down - - DiskSpaceCritical → df:root:percent_bytes:free (path=/) - df:boot:percent_bytes:free (path=/boot) - - BackupEncryptionDisabled → backup:config:notencrypted - - StorageStatus → storage:status - - HaPrimaryFailed → ha:primary:failed - - HaSyncFailed → ha:sync:failed - -All other alerts are silently dropped. -If the machine is not registered (no system_id/secret in UCI), all alerts -are silently dropped. - -Firing/resolved state is determined from the Alertmanager-standard endsAt -field: if endsAt is in the future (or zero/missing) the alert is FAILURE; -if endsAt is in the past the alert is OK. -""" - -import json -import logging -import re -import sys -import time -import urllib.request -from datetime import datetime, timezone -from http.server import BaseHTTPRequestHandler, HTTPServer -from socketserver import ThreadingMixIn -from euci import EUci - -LISTEN_ADDR = "127.0.0.1" -LISTEN_PORT = 9095 - -_DISK_PATH_MAP = { - "/": "df:root:percent_bytes:free", - "/boot": "df:boot:percent_bytes:free", -} - -_ZERO_TIME = "0001-01-01T00:00:00Z" -# vmalert uses nanosecond precision; strip to microseconds for Python parsing -_NANO_RE = re.compile(r"(\.\d{6})\d+(Z|[+-]\d{2}:\d{2})$") - - -def _is_firing(alert): - """Return True if the alert is currently firing based on endsAt.""" - ends_at_str = alert.get("endsAt", "") - if not ends_at_str or ends_at_str == _ZERO_TIME: - return True - ends_at_str = _NANO_RE.sub(r"\1\2", ends_at_str) - ends_at_str = ends_at_str.replace("Z", "+00:00") - try: - ends_at = datetime.fromisoformat(ends_at_str) - return ends_at > datetime.now(timezone.utc) - except Exception: - return True - - -def _map_alert_id(alert_name, labels): - """Return the legacy alert_id string, or None if the alert is not mapped.""" - if alert_name == "WanDown": - iface = labels.get("interface", "unknown") - return f"wan:{iface}:down" - if alert_name == "DiskSpaceCritical": - path = labels.get("path", "") - return _DISK_PATH_MAP.get(path) - if alert_name == "BackupEncryptionDisabled": - return "backup:config:notencrypted" - if alert_name == "StorageStatus": - return "storage:status" - if alert_name == "HaPrimaryFailed": - return "ha:primary:failed" - if alert_name == "HaSyncFailed": - return "ha:sync:failed" - return None - - -def _send_alert(system_id, secret, alerts_url, alert_id, status, retry=3): - url = alerts_url.rstrip("/") + "/alerts/store" - payload = json.dumps( - {"lk": system_id, "alert_id": alert_id, "status": status} - ).encode() - req = urllib.request.Request( - url, - data=payload, - method="POST", - headers={ - "Authorization": f"token {secret}", - "Content-Type": "application/json", - "Accept": "application/json", - }, - ) - try: - with urllib.request.urlopen(req, timeout=60) as resp: - logging.debug(f"Alert sent: {alert_id} {status} → {resp.status}") - except Exception as ex: - if retry > 0: - logging.warning(f"Alert send failed: {alert_id} {ex} — retrying in 20s") - time.sleep(20) - _send_alert(system_id, secret, alerts_url, alert_id, status, retry - 1) - else: - logging.warning(f"Alert send aborted: {alert_id} {ex}") - - -class _AlertHandler(BaseHTTPRequestHandler): - def log_message(self, format, *args): - # Suppress access log - pass - - def do_GET(self): - self.send_response(200) - self.end_headers() - - def do_POST(self): - if self.system_id is None or self.secret is None or self.alerts_url is None: - logging.debug("Alert dropped (not registered): no system_id or secret") - self.send_response(200) - self.end_headers() - return - try: - length = int(self.headers.get("Content-Length", 0)) - body = self.rfile.read(length) - data = json.loads(body) - except Exception as ex: - self.send_response(400) - self.end_headers() - self.wfile.write(str(ex).encode()) - return - - if type(data) is list: - alerts = data - else: - alerts = data.get("alerts", []) - for alert in alerts: - labels = alert.get("labels", {}) - alert_name = labels.get("alertname", "") - legacy_status = "FAILURE" if _is_firing(alert) else "OK" - - alert_id = _map_alert_id(alert_name, labels) - if not alert_id: - logging.debug(f"Alert dropped (no mapping): {alert_name} {labels}") - continue - - _send_alert(self.system_id, self.secret, self.alerts_url, alert_id, legacy_status) - - self.send_response(200) - self.end_headers() - - def __init__(self, *args, **kwargs): - uci = EUci() - self.system_id = uci.get("ns-plug", "config", "system_id", default=None) - self.secret = uci.get("ns-plug", "config", "secret", default=None) - self.alerts_url = uci.get("ns-plug", "config", "alerts_url", default=None) - super().__init__(*args, **kwargs) - - -class _ThreadingHTTPServer(ThreadingMixIn, HTTPServer): - daemon_threads = True - - -def main(): - uci = EUci() - loglevel_str = uci.get("ns-plug", "config", "alert_proxy_loglevel", default="warning") - loglevel = getattr(logging, loglevel_str.upper(), logging.WARNING) - logging.basicConfig(level=loglevel, format="%(message)s", stream=sys.stderr) - server = _ThreadingHTTPServer((LISTEN_ADDR, LISTEN_PORT), _AlertHandler) - logging.info(f"alert-proxy listening on {LISTEN_ADDR}:{LISTEN_PORT}") - server.serve_forever() - - -if __name__ == "__main__": - main() diff --git a/packages/ns-plug/files/ns-plug-alert-proxy.init b/packages/ns-plug/files/ns-plug-alert-proxy.init deleted file mode 100644 index 38ce01012..000000000 --- a/packages/ns-plug/files/ns-plug-alert-proxy.init +++ /dev/null @@ -1,30 +0,0 @@ -#!/bin/sh /etc/rc.common - -# -# Copyright (C) 2026 Nethesis S.r.l. -# SPDX-License-Identifier: GPL-2.0-only -# - -START=95 -STOP=4 -USE_PROCD=1 - -start_service() { - procd_open_instance - procd_set_param stdout 1 - procd_set_param stderr 1 - procd_set_param command '/usr/sbin/ns-plug-alert-proxy' - procd_set_param respawn 3600 5 0 - procd_close_instance -} - -service_triggers() -{ - procd_add_reload_trigger "ns-plug" -} - -reload_service() -{ - stop - start -} diff --git a/packages/telegraf/README.md b/packages/telegraf/README.md index b6cb55a73..70284b418 100644 --- a/packages/telegraf/README.md +++ b/packages/telegraf/README.md @@ -65,7 +65,6 @@ ns-clm ns-flashstart ns-flows ns-plug -ns-plug-alert-proxy ns-stats ns-ui odhcpd diff --git a/packages/telegraf/files/telegraf-services b/packages/telegraf/files/telegraf-services index 1eff7a2e9..1bcf41391 100644 --- a/packages/telegraf/files/telegraf-services +++ b/packages/telegraf/files/telegraf-services @@ -22,6 +22,7 @@ import json import subprocess import sys + MONITORED_SERVICES = { "conntrackd", "cron", @@ -38,7 +39,6 @@ MONITORED_SERVICES = { "ns-flashstart", "ns-flows", "ns-plug", - "ns-plug-alert-proxy", "ns-stats", "ns-ui", "odhcpd", @@ -56,6 +56,7 @@ MONITORED_SERVICES = { # Excluded service: adblock + def get_service_list(): result = subprocess.run( ["ubus", "call", "service", "list"], diff --git a/packages/victoria-metrics/README.md b/packages/victoria-metrics/README.md index 6dcc24bff..f9ac75873 100644 --- a/packages/victoria-metrics/README.md +++ b/packages/victoria-metrics/README.md @@ -137,72 +137,16 @@ The engine processes an active incident through three phases: Because real-world server incidents do not align perfectly with the monitoring engine's internal execution clock, notifications feature a variable delay window of 5 to 10 minutes from the actual start of the incident. -## Forwarding alerts to my.nethesis.it - -[my](https://github.com/NethServer/my/) uses Grafana Mimir as a multi-tenant -alertmanager for cloud-side alert processing. Enterprise systems forward their -alerts to it automatically, mirroring `send-heartbeat` / `send-inventory`: -vmalert POSTs alerts to the credential-translation proxy at -`https://my.nethesis.it/proxy/alerts` using the ns-plug credentials -(`system_id` / `secret`), which the proxy maps to the new my credentials before -forwarding them to the Mimir alertmanager. No manual configuration is needed — -it is enabled whenever `ns-plug.config.type` is `enterprise` and the system is -registered (`system_id` / `secret` set). vmalert always also notifies the local -ns-plug-alert-proxy (`http://127.0.0.1:9095`), which handles the legacy path and -unregistered machines. - -By default, ns-plug-alert proxy logs only when an alert can't be forwarded to legacy my.nethesis.it. -To increase verbosity and debug all communications with the portal, -set `ns-plug.config.alert_proxy_loglevel` to `info` or `debug` and restart ns-plug-alert-proxy: -```bash -uci set ns-plug.config.alert_proxy_loglevel='debug' -uci commit ns-plug -/etc/init.d/ns-plug-alert-proxy restart -``` - -> Migration note: the my switch-off release will repoint this from -> `/proxy/alerts` to the native collect endpoint -> (`/collect/api/services/mimir/alertmanager`) with rotated credentials. - ## Alert notifications System alerts are handled by vmalert (Victoria Metrics alert evaluation engine) which evaluates alert rules against metrics collected by telegraf. -When a rule transitions from `Pending` to `Firing`, vmalert sends an Alertmanager notification to the following endponts: -- ns-plug-alert-proxy, listening on port 9095, which forwards only some alerts to the legacy monitoring portal -- https://my.nethesis.it/proxy/alerts, wich forwards all alerts to the new Mimir alertmanager +When a rule transitions from `Pending` to `Firing`, vmalert sends to remote Mimir instance if active subscription is present. vmalert sends a notification for firing alerts every `interval`, set to 5 minutes for most alerts, until the alert resolves. When the alert resolves, vmalert sends 4 notifications at 5-minute intervals to ensure the resolution is received by the alertmanager (or the proxy) even if the first notification is lost. -**Migration note** - -When legacy my.nethesis.it will be replaced with the new one: -- remove ns-plug-alert-proxy from the system (caveat: also my.nethserver.com will not receive alerts anymore) -- change vmalert configuration to send alerts directly to the new Mimir alertmanager endpoint: replace `/proxy/alerts` - with the native collect endpoint `/collect/api/services/mimir/alertmanager` with rotated credentials. - -### ns-plug-alert-proxy - -The proxy forwards only the following legacy alerts: -| Alert | Condition | Legacy alert_id | -|---|---|---| -| `WanDown` | WAN interface offline for 2m | `wan::down` | -| `DiskSpaceCritical` | Disk usage > 90% for 2m | `df:root:percent_bytes:free` or `df:boot:percent_bytes:free` | -| `StorageStatus` | Storage status is error | `storage:status` | -| `HaPrimaryFailed` | Backup node became master | `ha:primary:failed` | -| `HaSyncFailed` | HA sync failure detected on the primary node | `ha:sync:failed` | - -All other alert are silently dropped by the proxy. -If the machine does not have a subscription, all alerts are silently dropped. - -The proxy starts automatically at boot regardless of registration state. -By default, firing/resolved state is determined from the Alertmanager-standard `endsAt` field: -if `endsAt` is in the future (or zero/missing) a **FAILURE** is sent; if `endsAt` is in -the past an **OK** is sent. HA recovery/failover event alerts override this default mapping so -they can keep the legacy `ha:primary:failed` semantics. - ## Alert history The `vmalert` alerts keeps the state of all active alerts inside VictoriaMetrics using the remote-write protocol. diff --git a/packages/victoria-metrics/files/vmalert.initd b/packages/victoria-metrics/files/vmalert.initd index c17321129..6a73abdee 100644 --- a/packages/victoria-metrics/files/vmalert.initd +++ b/packages/victoria-metrics/files/vmalert.initd @@ -36,9 +36,6 @@ start_service() { procd_append_param command -remoteRead.url="$datasource_url" procd_append_param command -remoteWrite.url="$datasource_url" - # Always notify the local alert-proxy (handles unregistered machines gracefully) - procd_append_param command -notifier.url="http://127.0.0.1:9095" - # Also forward alerts to my.nethesis.it for registered enterprise systems if [ -n "$notifier_url" ]; then # Avoid leaking secret inside command line From 5ce39d746a0d7c1fc08b56b9973190d0393ba7c0 Mon Sep 17 00:00:00 2001 From: Tommaso Bailetti Date: Tue, 8 Sep 2026 10:56:00 +0200 Subject: [PATCH 34/41] fixes --- packages/ns-plug/files/config | 3 +-- packages/ns-plug/files/migrate-to-my | 1 + packages/ns-plug/files/register | 1 + packages/ns-plug/files/unregister | 4 +--- packages/victoria-metrics/files/vmalert.initd | 2 ++ 5 files changed, 6 insertions(+), 5 deletions(-) diff --git a/packages/ns-plug/files/config b/packages/ns-plug/files/config index 592860445..38537c6ab 100644 --- a/packages/ns-plug/files/config +++ b/packages/ns-plug/files/config @@ -6,8 +6,7 @@ config main 'config' option tls_verify '1' option backup_url 'https://backupd.nethesis.it' option collect_url 'https://my-proxy-prod.onrender.com/collect/api/systems' -# FIXME: add it ot migrate-to-my - option notifier_url 'https://my-proxy-prod.onrender.com/collect/api/services/mimir/alertmanager' + option notifier_url '' option repository_url 'https://updates.nethsecurity.nethserver.org' option channel '' option tun_mtu '' diff --git a/packages/ns-plug/files/migrate-to-my b/packages/ns-plug/files/migrate-to-my index ef9ff138d..b5e3421bd 100644 --- a/packages/ns-plug/files/migrate-to-my +++ b/packages/ns-plug/files/migrate-to-my @@ -76,6 +76,7 @@ set ns-plug.config.legacy_secret=$SYSTEM_SECRET set ns-plug.config.system_id=$new_key set ns-plug.config.secret=$new_secret set ns-plug.config.collect_url=https://my-proxy-prod.onrender.com/collect/api/systems +set ns-plug.config.notifier_url=https://my-proxy-prod.onrender.com/collect/api/services/mimir/alertmanager set ns-plug.config.migrated=1 commit ns-plug EOI diff --git a/packages/ns-plug/files/register b/packages/ns-plug/files/register index 99bd144d8..5b7871bfb 100755 --- a/packages/ns-plug/files/register +++ b/packages/ns-plug/files/register @@ -80,6 +80,7 @@ case "$type" in uci set ns-plug.config.type="enterprise" uci set ns-plug.config.api_url="$url" uci set ns-plug.config.collect_url="https://my-proxy-prod.onrender.com/collect/api/systems" + uci set ns-plug.config.notifier_url=https://my-proxy-prod.onrender.com/collect/api/services/mimir/alertmanager uci set ns-plug.config.migrated="1" ;; esac diff --git a/packages/ns-plug/files/unregister b/packages/ns-plug/files/unregister index 45379596c..4c223d6ca 100755 --- a/packages/ns-plug/files/unregister +++ b/packages/ns-plug/files/unregister @@ -31,12 +31,10 @@ fi # Reset ns-plug configuration uci set ns-plug.config.type="" -uci set ns-plug.config.alerts_url="" -uci set ns-plug.config.api_url="" -uci set ns-plug.config.inventory_url="" uci set ns-plug.config.system_id="" uci set ns-plug.config.secret="" uci set ns-plug.config.repository_url="https://updates.nethsecurity.nethserver.org/$(cat /etc/repo-channel)" +uci set ns-plug.config.notifier_url="" uci -q delete ns-plug.config.collect_url uci -q delete ns-plug.config.migrated uci -q delete ns-plug.config.legacy_system_id diff --git a/packages/victoria-metrics/files/vmalert.initd b/packages/victoria-metrics/files/vmalert.initd index 6a73abdee..08d189bce 100644 --- a/packages/victoria-metrics/files/vmalert.initd +++ b/packages/victoria-metrics/files/vmalert.initd @@ -47,6 +47,8 @@ start_service() { procd_append_param command -notifier.url="$notifier_url" procd_append_param command -notifier.basicAuth.username="$system_id" procd_append_param command -notifier.basicAuth.passwordFile="$pass_file" + else + procd_append_param command -notifier.blackhole fi procd_set_param stdout 1 From b4e045a4c1b448d127ddccb027043e810c10b347 Mon Sep 17 00:00:00 2001 From: Tommaso Bailetti Date: Tue, 8 Sep 2026 10:59:57 +0200 Subject: [PATCH 35/41] removed fixmes --- packages/ns-api/files/ns.backup | 4 ---- packages/ns-api/files/ns.subscription | 5 ----- packages/ns-plug/files/remote-backup | 9 +-------- 3 files changed, 1 insertion(+), 17 deletions(-) diff --git a/packages/ns-api/files/ns.backup b/packages/ns-api/files/ns.backup index 85a13d7e5..9dc4fe242 100755 --- a/packages/ns-api/files/ns.backup +++ b/packages/ns-api/files/ns.backup @@ -226,10 +226,6 @@ elif cmd == 'call': data = json.load(sys.stdin) subprocess.run(['/usr/sbin/remote-backup', 'delete', data['id']], check=True, capture_output=True, text=True) - # FIXME: check api - # The remote side returns a structured JSON response; the UI - # only needs a success flag, matching the pattern of the - # other registered-* handlers (backup, restore). print(json.dumps({'message': 'success'})) except subprocess.CalledProcessError as error: print(json.dumps(utils.generic_error('remote backup delete failed'))) diff --git a/packages/ns-api/files/ns.subscription b/packages/ns-api/files/ns.subscription index 0cb913b39..29cdd1e11 100755 --- a/packages/ns-api/files/ns.subscription +++ b/packages/ns-api/files/ns.subscription @@ -64,11 +64,6 @@ def info(): ret = {"server_id": data["id"], "systemd_id": data["uuid"], "plan": data["subscription"]["subscription_plan"]["name"], "expiration": expiration, "active": active, "type": type} - # The new my has no per-system commercial plan. For enterprise units expose - # the organization explicitly (subscription-info threads organization.name) - # and default the plan label to "Nethesis Enterprise". Community units keep - # the real plan name untouched. - # FIXME: check with upstream if type == "enterprise": ret["organization"] = data.get("organization", "") # The system name given on my at creation time (threaded by diff --git a/packages/ns-plug/files/remote-backup b/packages/ns-plug/files/remote-backup index a7593b97d..e588fd80b 100755 --- a/packages/ns-plug/files/remote-backup +++ b/packages/ns-plug/files/remote-backup @@ -39,16 +39,9 @@ if [ "$TYPE" = "enterprise" ]; then BASE="$(uci -q get ns-plug.config.collect_url)/backups" curl_args="--silent --show-error --location-trusted --fail-with-body --user $SYSTEM_ID:$SYSTEM_SECRET" - - # FIXME: check with upstream case "$cmd" in list) - # my returns {code, message, data: {backups: [...]}} on - # success. Unwrap `data` so ns.backup can pass it through as - # {values: } without double-nesting. Fall back to an - # empty list on failure. - response=$(curl $curl_args "$BASE") - echo "$response" | jq 'if .data and (.data.backups // empty) then .data else {backups: []} end' + curl $curl_args "$BASE" ;; download) file=$2 From 16180d24989ae7907a5a5b9738da57881baa2a9a Mon Sep 17 00:00:00 2001 From: Tommaso Bailetti Date: Tue, 8 Sep 2026 14:24:19 +0200 Subject: [PATCH 36/41] make migration script to over to libexec, added return codes and api --- packages/ns-api/files/ns.subscription | 10 +++++++++- packages/ns-plug/Makefile | 2 +- packages/ns-plug/files/migrate-to-my | 10 +++++----- 3 files changed, 15 insertions(+), 7 deletions(-) diff --git a/packages/ns-api/files/ns.subscription b/packages/ns-api/files/ns.subscription index 29cdd1e11..1ebd6ac7d 100755 --- a/packages/ns-api/files/ns.subscription +++ b/packages/ns-api/files/ns.subscription @@ -11,6 +11,7 @@ import sys import json import subprocess from datetime import datetime +from time import sleep from nethsec import utils from euci import EUci @@ -106,7 +107,8 @@ if cmd == 'list': "unregister": {}, "info": {}, "inventory-status": {}, - "send-inventory": {} + "send-inventory": {}, + "migrate": {}, })) elif cmd == 'call': action = sys.argv[2] @@ -121,5 +123,11 @@ elif cmd == 'call': ret = inventory_status() elif action == "send-inventory": ret = send_inventory() + elif action == "migrate": + try: + subprocess.run(["/usr/libexec/migrate-to-my"], check=True, capture_output=True) + ret = {} + except subprocess.CalledProcessError: + ret = utils.generic_error("failed to migrate") print(json.dumps(ret)) diff --git a/packages/ns-plug/Makefile b/packages/ns-plug/Makefile index 1ef7be8d4..44745d2ec 100644 --- a/packages/ns-plug/Makefile +++ b/packages/ns-plug/Makefile @@ -77,7 +77,7 @@ define Package/ns-plug/install $(INSTALL_BIN) ./files/ns-plug $(1)/usr/sbin/ns-plug $(INSTALL_BIN) ./files/distfeed-setup $(1)/usr/sbin/distfeed-setup $(INSTALL_BIN) ./files/apk-official $(1)/usr/sbin/apk-official - $(INSTALL_BIN) ./files/migrate-to-my $(1)/usr/sbin + $(INSTALL_BIN) ./files/migrate-to-my $(1)/usr/libexec $(INSTALL_BIN) ./files/remote-backup $(1)/usr/sbin $(INSTALL_BIN) ./files/send-backup $(1)/usr/sbin $(INSTALL_BIN) ./files/send-heartbeat $(1)/usr/sbin diff --git a/packages/ns-plug/files/migrate-to-my b/packages/ns-plug/files/migrate-to-my index b5e3421bd..ccf00532d 100644 --- a/packages/ns-plug/files/migrate-to-my +++ b/packages/ns-plug/files/migrate-to-my @@ -58,15 +58,15 @@ resp=$(/usr/bin/curl --silent --location-trusted --fail-with-body \ --max-time 30 --retry 2 \ --user "$SYSTEM_ID:$SYSTEM_SECRET" \ https://my-proxy-prod.onrender.com/proxy/credentials 2>/dev/null) || { - logger -t migrate-to-my "credential fetch failed; will retry on next run" - exit 0 + echo "credential fetch failed; will retry on next run" + exit 1 } new_key=$(echo "$resp" | jq -r '.data.system_key // empty' 2>/dev/null) new_secret=$(echo "$resp" | jq -r '.data.system_secret // empty' 2>/dev/null) if [ -z "$new_key" ] || [ -z "$new_secret" ]; then - logger -t migrate-to-my "credentials missing in response" - exit 0 + echo "credentials missing in response" + exit 2 fi # Rotate atomically; legacy pair preserved for audit / rollback. @@ -99,5 +99,5 @@ done /etc/init.d/banip reload 2>/dev/null || true /etc/init.d/adblock reload 2>/dev/null || true -logger -t migrate-to-my "migrated to my collect credentials" +echo "migrated to my collect credentials" exit 0 From 6d0fbd7fd60b40be70cbd10ad411c7bad0942342 Mon Sep 17 00:00:00 2001 From: Tommaso Bailetti Date: Tue, 8 Sep 2026 14:52:41 +0200 Subject: [PATCH 37/41] added migrated attribute in subscription info --- packages/ns-api/files/ns.subscription | 2 ++ 1 file changed, 2 insertions(+) diff --git a/packages/ns-api/files/ns.subscription b/packages/ns-api/files/ns.subscription index 1ebd6ac7d..371ea5f50 100755 --- a/packages/ns-api/files/ns.subscription +++ b/packages/ns-api/files/ns.subscription @@ -78,6 +78,8 @@ def info(): if sub_id: ret["system_url"] = f"https://my.nethserver.com/servers/{sub_id}" + ret["migrated"] = u.get('ns-plug', 'config', 'migrated', dtype=bool, default=False) + return ret From 501c89765befa622c814eece1707e56ce43e5ce6 Mon Sep 17 00:00:00 2001 From: Tommaso Bailetti Date: Tue, 8 Sep 2026 14:54:58 +0200 Subject: [PATCH 38/41] bumped ui --- packages/ns-ui/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index fb992886d..562efb598 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -17,7 +17,7 @@ PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git # new collect backup payload; on a device still on backupd it would break, hence it must # ship together with this PR. BEFORE MERGING: revert to PKG_SOURCE_VERSION:=$(PKG_VERSION) # and bump PKG_VERSION to the nethsecurity-ui release that carries #746. -PKG_SOURCE_VERSION:=e17c6fa3a0d4e6bae71272aee177c2e50c486455 +PKG_SOURCE_VERSION:=6ef65208782728b320421f6e6d2bceea1b1da611 PKG_SOURCE_SUBDIR:=nethsecurity-ui-$(PKG_SOURCE_VERSION) PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_SOURCE_SUBDIR) PKG_MIRROR_HASH:=skip From 5927a97f63947bb7a0ef4990f36fd85bc5cb3fd0 Mon Sep 17 00:00:00 2001 From: Tommaso Bailetti Date: Tue, 8 Sep 2026 15:15:29 +0200 Subject: [PATCH 39/41] removed comment --- packages/ns-plug/files/subscription-info | 1 - 1 file changed, 1 deletion(-) diff --git a/packages/ns-plug/files/subscription-info b/packages/ns-plug/files/subscription-info index dee2816e2..8044984dd 100755 --- a/packages/ns-plug/files/subscription-info +++ b/packages/ns-plug/files/subscription-info @@ -24,7 +24,6 @@ type=$(uci -q get ns-plug.config.type) secret=$(uci -q get ns-plug.config.secret) if [ "$type" = "enterprise" ]; then - # TODO: Controllare risposta con la UI resp=$(/usr/bin/curl -f -s -m $timeout --retry-delay 1 --retry 2 -L \ -H "Accept: application/json" \ --user "$system_id:$secret" \ From 1e338a0e6e15384fad09aeabcc7b135068fd2a43 Mon Sep 17 00:00:00 2001 From: Tommaso Bailetti Date: Tue, 8 Sep 2026 16:14:45 +0200 Subject: [PATCH 40/41] added extra vmalert flags --- packages/victoria-metrics/README.md | 15 +++++++++++++++ packages/victoria-metrics/files/vmalert.initd | 10 +++++++++- 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/packages/victoria-metrics/README.md b/packages/victoria-metrics/README.md index f9ac75873..a32894a03 100644 --- a/packages/victoria-metrics/README.md +++ b/packages/victoria-metrics/README.md @@ -46,6 +46,21 @@ config victoriametrics 'main' - `storage_path`: Where to store metrics data (default: `/var/lib/victoriametrics`, auto-detects `/mnt/data/victoriametrics` if available) - `retention_period`: How long to keep metrics (`1d`, `7d`, `30d`, `1y`, etc.) (default: `7d`, auto-detects `1y` if not set) +### Advanced: Extra vmalert Flags + +`/etc/config/vmalert`'s `main` section accepts an `additional_parameters` list: extra `vmalert` +CLI flags, one per entry, appended after every other flag on the command line. + +For example, to point vmalert at an external Alertmanager notifier: + +```bash +uci add_list vmalert.main.additional_parameters='-notifier.url=' +uci commit vmalert +reload_config +``` + +See the [vmalert documentation](https://docs.victoriametrics.com/vmalert/) for the full flag list. + ### Accessing the Web UI By default the server is accessible only on localhost for security. diff --git a/packages/victoria-metrics/files/vmalert.initd b/packages/victoria-metrics/files/vmalert.initd index 08d189bce..18d9bd382 100644 --- a/packages/victoria-metrics/files/vmalert.initd +++ b/packages/victoria-metrics/files/vmalert.initd @@ -12,6 +12,10 @@ USE_PROCD=1 PROG="/usr/bin/vmalert" RULE_DIR="/etc/vmalert/rules" +function append_params() { + procd_append_param command $@ +} + start_service() { config_load vmalert 2>/dev/null || true @@ -50,7 +54,11 @@ start_service() { else procd_append_param command -notifier.blackhole fi - + + # config_load ns-plug above discards vmalert's parsed UCI state + config_load vmalert 2>/dev/null || true + config_list_foreach main additional_parameters append_params + procd_set_param stdout 1 procd_set_param stderr 1 procd_set_param respawn 3600 5 5 From 93cd987c230268ddc9ce45e26ea5706f81f38234 Mon Sep 17 00:00:00 2001 From: Tommaso Bailetti Date: Tue, 8 Sep 2026 16:35:09 +0200 Subject: [PATCH 41/41] bumped versions --- packages/ns-api/Makefile | 2 +- packages/ns-plug/Makefile | 2 +- packages/ns-threat_shield/Makefile | 2 +- packages/ns-ui/Makefile | 4 ++-- packages/telegraf/Makefile | 2 +- packages/victoria-metrics/Makefile | 2 +- 6 files changed, 7 insertions(+), 7 deletions(-) diff --git a/packages/ns-api/Makefile b/packages/ns-api/Makefile index e1579e0aa..db4b2d56a 100644 --- a/packages/ns-api/Makefile +++ b/packages/ns-api/Makefile @@ -6,7 +6,7 @@ include $(TOPDIR)/rules.mk PKG_NAME:=ns-api -PKG_VERSION:=3.7.2 +PKG_VERSION:=3.8.0_beta PKG_RELEASE:=1 PKG_BUILD_DIR:=$(BUILD_DIR)/ns-api-$(PKG_VERSION) diff --git a/packages/ns-plug/Makefile b/packages/ns-plug/Makefile index 44745d2ec..4a6c76bce 100644 --- a/packages/ns-plug/Makefile +++ b/packages/ns-plug/Makefile @@ -6,7 +6,7 @@ include $(TOPDIR)/rules.mk PKG_NAME:=ns-plug -PKG_VERSION:=1.3.2 +PKG_VERSION:=1.4.0_beta PKG_RELEASE:=1 PKG_BUILD_DIR:=$(BUILD_DIR)/ns-plug-$(PKG_VERSION) diff --git a/packages/ns-threat_shield/Makefile b/packages/ns-threat_shield/Makefile index d0b116059..6c964c0e4 100644 --- a/packages/ns-threat_shield/Makefile +++ b/packages/ns-threat_shield/Makefile @@ -7,7 +7,7 @@ include $(TOPDIR)/rules.mk PKG_NAME:=ns-threat_shield -PKG_VERSION:=1.0.2 +PKG_VERSION:=1.0.3_beta PKG_RELEASE:=1 PKG_BUILD_DIR:=$(BUILD_DIR)/ns-threat_shield-$(PKG_VERSION) diff --git a/packages/ns-ui/Makefile b/packages/ns-ui/Makefile index 562efb598..852668498 100644 --- a/packages/ns-ui/Makefile +++ b/packages/ns-ui/Makefile @@ -7,8 +7,8 @@ include $(TOPDIR)/rules.mk PKG_NAME:=ns-ui # renovate: datasource=github-releases depName=NethServer/nethsecurity-ui -PKG_VERSION:=2.23.4 -PKG_RELEASE:=2 +PKG_VERSION:=2.24.0_beta +PKG_RELEASE:=1 PKG_SOURCE_PROTO:=git PKG_SOURCE_URL:=https://github.com/NethServer/nethsecurity-ui.git diff --git a/packages/telegraf/Makefile b/packages/telegraf/Makefile index c8facc1db..827999560 100644 --- a/packages/telegraf/Makefile +++ b/packages/telegraf/Makefile @@ -8,7 +8,7 @@ include $(TOPDIR)/rules.mk PKG_NAME:=telegraf # renovate: datasource=github-tags depName=influxdata/telegraf PKG_VERSION:=1.39.1 -PKG_RELEASE:=3 +PKG_RELEASE:=4 PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz PKG_SOURCE_URL:=https://codeload.github.com/influxdata/telegraf/tar.gz/v$(PKG_VERSION)? diff --git a/packages/victoria-metrics/Makefile b/packages/victoria-metrics/Makefile index 77d9fc84a..15de80400 100644 --- a/packages/victoria-metrics/Makefile +++ b/packages/victoria-metrics/Makefile @@ -8,7 +8,7 @@ include $(TOPDIR)/rules.mk PKG_NAME:=victoria-metrics # renovate: datasource=github-tags depName=VictoriaMetrics/VictoriaMetrics PKG_VERSION:=1.146.0 -PKG_RELEASE:=1 +PKG_RELEASE:=2 PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz PKG_SOURCE_URL:=https://codeload.github.com/VictoriaMetrics/VictoriaMetrics/tar.gz/v$(PKG_VERSION)?