From f382b1ec7b956aa5db94e52cec7edd2a9c630df4 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 13:55:50 +0300 Subject: [PATCH 001/157] docs(fork): state that this is an unofficial fork, and how it is changed Adds FORK.md (why the fork exists, the red-then-green rule, the base) and NOTICE (Evolution Foundation's attribution from upstream main, plus the modification notice Apache 2.0 section 4 asks for). Co-Authored-By: Claude Opus 5.5 --- FORK.md | 45 +++++++++++++++++++++++++++++++++++++++++++++ NOTICE | 26 ++++++++++++++++++++++++++ README.md | 2 ++ 3 files changed, 73 insertions(+) create mode 100644 FORK.md create mode 100644 NOTICE diff --git a/FORK.md b/FORK.md new file mode 100644 index 0000000000..578a7023e4 --- /dev/null +++ b/FORK.md @@ -0,0 +1,45 @@ +# This fork + +An unofficial fork of [Evolution API](https://github.com/evolution-foundation/evolution-api) +2.3.7. It is not endorsed by Evolution Foundation, and +the image it builds is not an official Evolution API build: it carries its own +name of its own, not Evolution's. + +## Why it exists + +Evolution API has no test suite. Every bug below shipped because nothing could +catch it, and a consumer that needed a fix could only patch the minified bundle +with string replacements. This fork shows the other way: the same code, a test +harness that runs Evolution's own TypeScript source against the real Baileys, +and every change made test first. + +## The rule: red, then green + +Every change in this fork is two commits, in this order: + +1. `test: ...`, a test that reproduces the bug and **fails** on the code as it + is. CI runs on that commit, so its red run is the proof that the test sees + the bug. +2. `fix: ...` (or `feat:`), the smallest change that makes that test pass, + with the rest of the suite still green. + +A fix without a failing test first is not merged here. See `AGENTS.md`. + +## Base + +- Source: the `2.3.7` tag (`cd800f29`). Upstream `main` has the same code. + Upstream `develop` (2.4.0) is not used: it requires licence activation + against Evolution Foundation's server and breaks `POST /instance/create`. +- Baileys pinned to `7.0.0-rc14` (2.3.7 ships rc.9, which is inside the range + of CVE-2026-48063). + +## Changes from 2.3.7 + +The list grows with each red and green pair. `git log 2.3.7..` is the source of +truth, and `git diff 2.3.7 --stat` lists every file modified from the original. + +## Licence + +Evolution API is licensed under the Apache License 2.0 with additional +conditions (`LICENSE`), which this fork keeps unchanged. `NOTICE` carries +Evolution Foundation's attribution and states the modifications. diff --git a/NOTICE b/NOTICE new file mode 100644 index 0000000000..6a6a89ad29 --- /dev/null +++ b/NOTICE @@ -0,0 +1,26 @@ +Evolution API +Copyright 2026 Evolution Foundation + +This product includes software developed by Evolution Foundation +(https://evolutionfoundation.com.br). + +Trademark notice +"Evolution Foundation", "Evolution" and "Evolution API" are trademarks of +Evolution Foundation. The Evolution API logo, wordmark, and visual identity +are governed by the Trademark and Brand Assets Policy included in this +repository (TRADEMARKS.md). + +Third-party attributions + +This product includes software derived from CodeChat WhatsApp API +(https://github.com/code-chat-br/whatsapp-api), originally licensed under MIT. +The CodeChat project implemented the Baileys library +(https://github.com/WhiskeySockets/Baileys), which Evolution API also uses for +its WhatsApp Web integration. + +Modifications + +This is an unofficial fork of Evolution API 2.3.7, maintained by Almog Cohen +(https://github.com/AlmogCohen/evolution-api). It is not endorsed +by Evolution Foundation. The changes are described in FORK.md, and each one is +a separate commit with the test that proves it. diff --git a/README.md b/README.md index eb7e638c16..7d02d4534c 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,5 @@ +> **Unofficial fork** of Evolution API 2.3.7, made test first. Not endorsed by Evolution Foundation. See [FORK.md](FORK.md) for what changed and why. +

Evolution Api

From fadd8bd76d0ba6f84794b7385b0b1a460df0ab0a Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 13:56:41 +0300 Subject: [PATCH 002/157] build(deps): pin baileys 7.0.0-rc14 2.3.7 ships 7.0.0-rc.9, which is inside the range of CVE-2026-48063 (message and history-sync spoofing, rc.1 to rc11, GHSA-qvv5-jq5g-4cgg). rc13 is also the first to learn @lid to phone mappings from the history sync. Pinned exactly: an RC is not a range. Co-Authored-By: Claude Opus 5.5 --- package-lock.json | 768 +++++++++++++++++----------------------------- package.json | 4 +- 2 files changed, 288 insertions(+), 484 deletions(-) diff --git a/package-lock.json b/package-lock.json index c45e8fef38..2fb0832437 100644 --- a/package-lock.json +++ b/package-lock.json @@ -21,7 +21,7 @@ "amqplib": "^0.10.5", "audio-decode": "^2.2.3", "axios": "^1.7.9", - "baileys": "7.0.0-rc.9", + "baileys": "7.0.0-rc14", "class-validator": "^0.14.1", "compression": "^1.7.5", "cors": "^2.8.5", @@ -829,9 +829,9 @@ } }, "node_modules/@borewit/text-codec": { - "version": "0.2.0", - "resolved": "https://registry.npmjs.org/@borewit/text-codec/-/text-codec-0.2.0.tgz", - "integrity": "sha512-X999CKBxGwX8wW+4gFibsbiNdwqmdQEXmUejIWaIqdrHBgS5ARIOOeyiQbHjP9G58xVEPcuvP6VwwH3A0OFTOA==", + "version": "0.2.2", + "resolved": "https://registry.npmjs.org/@borewit/text-codec/-/text-codec-0.2.2.tgz", + "integrity": "sha512-DDaRehssg1aNrH4+2hnj1B7vnUGEjU6OIlyRdkMd0aUdIUvKXrJfXsy8LVtXAy7DRvYVluWbMspsRhz2lcW0mQ==", "license": "MIT", "funding": { "type": "github", @@ -2372,17 +2372,17 @@ } }, "node_modules/@jimp/core": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/core/-/core-1.6.0.tgz", - "integrity": "sha512-EQQlKU3s9QfdJqiSrZWNTxBs3rKXgO2W+GxNXDtwchF3a4IqxDheFX1ti+Env9hdJXDiYLp2jTRjlxhPthsk8w==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/core/-/core-1.6.1.tgz", + "integrity": "sha512-+BoKC5G6hkrSy501zcJ2EpfnllP+avPevcBfRcZe/CW+EwEfY6X1EZ8QWyT7NpDIvEEJb1fdJnMMfUnFkxmw9A==", "license": "MIT", "dependencies": { - "@jimp/file-ops": "1.6.0", - "@jimp/types": "1.6.0", - "@jimp/utils": "1.6.0", + "@jimp/file-ops": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", "await-to-js": "^3.0.0", "exif-parser": "^0.1.12", - "file-type": "^16.0.0", + "file-type": "^21.3.3", "mime": "3" }, "engines": { @@ -2402,14 +2402,14 @@ } }, "node_modules/@jimp/diff": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/diff/-/diff-1.6.0.tgz", - "integrity": "sha512-+yUAQ5gvRC5D1WHYxjBHZI7JBRusGGSLf8AmPRPCenTzh4PA+wZ1xv2+cYqQwTfQHU5tXYOhA0xDytfHUf1Zyw==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/diff/-/diff-1.6.1.tgz", + "integrity": "sha512-YkKDPdHjLgo1Api3+Bhc0GLAygldlpt97NfOKoNg1U6IUNXA6X2MgosCjPfSBiSvJvrrz1fsIR+/4cfYXBI/HQ==", "license": "MIT", "dependencies": { - "@jimp/plugin-resize": "1.6.0", - "@jimp/types": "1.6.0", - "@jimp/utils": "1.6.0", + "@jimp/plugin-resize": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", "pixelmatch": "^5.3.0" }, "engines": { @@ -2417,23 +2417,23 @@ } }, "node_modules/@jimp/file-ops": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/file-ops/-/file-ops-1.6.0.tgz", - "integrity": "sha512-Dx/bVDmgnRe1AlniRpCKrGRm5YvGmUwbDzt+MAkgmLGf+jvBT75hmMEZ003n9HQI/aPnm/YKnXjg/hOpzNCpHQ==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/file-ops/-/file-ops-1.6.1.tgz", + "integrity": "sha512-T+gX6osHjprbDRad0/B71Evyre7ZdVY1z/gFGEG9Z8KOtZPKboWvPeP2UjbZYWQLy9UKCPQX1FNAnDiOPkJL7w==", "license": "MIT", "engines": { "node": ">=18" } }, "node_modules/@jimp/js-bmp": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/js-bmp/-/js-bmp-1.6.0.tgz", - "integrity": "sha512-FU6Q5PC/e3yzLyBDXupR3SnL3htU7S3KEs4e6rjDP6gNEOXRFsWs6YD3hXuXd50jd8ummy+q2WSwuGkr8wi+Gw==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/js-bmp/-/js-bmp-1.6.1.tgz", + "integrity": "sha512-xzWzNT4/u5zGrTT3Tme9sGU7YzIKxi13+BCQwLqACbt5DXf9SAfdzRkopZQnmDko+6In5nqaT89Gjs43/WdnYQ==", "license": "MIT", "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/types": "1.6.0", - "@jimp/utils": "1.6.0", + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", "bmp-ts": "^1.0.9" }, "engines": { @@ -2441,13 +2441,13 @@ } }, "node_modules/@jimp/js-gif": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/js-gif/-/js-gif-1.6.0.tgz", - "integrity": "sha512-N9CZPHOrJTsAUoWkWZstLPpwT5AwJ0wge+47+ix3++SdSL/H2QzyMqxbcDYNFe4MoI5MIhATfb0/dl/wmX221g==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/js-gif/-/js-gif-1.6.1.tgz", + "integrity": "sha512-YjY2W26rQa05XhanYhRZ7dingCiNN+T2Ymb1JiigIbABY0B28wHE3v3Cf1/HZPWGu0hOg36ylaKgV5KxF2M58w==", "license": "MIT", "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/types": "1.6.0", + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", "gifwrap": "^0.10.1", "omggif": "^1.0.10" }, @@ -2456,13 +2456,13 @@ } }, "node_modules/@jimp/js-jpeg": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/js-jpeg/-/js-jpeg-1.6.0.tgz", - "integrity": "sha512-6vgFDqeusblf5Pok6B2DUiMXplH8RhIKAryj1yn+007SIAQ0khM1Uptxmpku/0MfbClx2r7pnJv9gWpAEJdMVA==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/js-jpeg/-/js-jpeg-1.6.1.tgz", + "integrity": "sha512-HT9H3yOmlOFzYmdI15IYdfy6ggQhSRIaHeA+OTJSEORXBqEo97sUZu/DsgHIcX5NJ7TkJBTgZ9BZXsV6UbsyMg==", "license": "MIT", "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/types": "1.6.0", + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", "jpeg-js": "^0.4.4" }, "engines": { @@ -2470,13 +2470,13 @@ } }, "node_modules/@jimp/js-png": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/js-png/-/js-png-1.6.0.tgz", - "integrity": "sha512-AbQHScy3hDDgMRNfG0tPjL88AV6qKAILGReIa3ATpW5QFjBKpisvUaOqhzJ7Reic1oawx3Riyv152gaPfqsBVg==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/js-png/-/js-png-1.6.1.tgz", + "integrity": "sha512-SZ/KVhI5UjcSzzlXsXdIi/LhJ7UShf2NkMOtVrbZQcGzsqNtynAelrOXeoTxcanfVqmNhAoVHg8yR2cYoqrYjA==", "license": "MIT", "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/types": "1.6.0", + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", "pngjs": "^7.0.0" }, "engines": { @@ -2484,13 +2484,13 @@ } }, "node_modules/@jimp/js-tiff": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/js-tiff/-/js-tiff-1.6.0.tgz", - "integrity": "sha512-zhReR8/7KO+adijj3h0ZQUOiun3mXUv79zYEAKvE0O+rP7EhgtKvWJOZfRzdZSNv0Pu1rKtgM72qgtwe2tFvyw==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/js-tiff/-/js-tiff-1.6.1.tgz", + "integrity": "sha512-jDG/eJquID1M4MBlKMmDRBmz2TpXMv7TUyu2nIRUxhlUc2ogC82T+VQUkca9GJH1BBJ9dx5sSE5dGkWNjIbZxw==", "license": "MIT", "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/types": "1.6.0", + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", "utif2": "^4.1.0" }, "engines": { @@ -2498,13 +2498,13 @@ } }, "node_modules/@jimp/plugin-blit": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-blit/-/plugin-blit-1.6.0.tgz", - "integrity": "sha512-M+uRWl1csi7qilnSK8uxK4RJMSuVeBiO1AY0+7APnfUbQNZm6hCe0CCFv1Iyw1D/Dhb8ph8fQgm5mwM0eSxgVA==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-blit/-/plugin-blit-1.6.1.tgz", + "integrity": "sha512-MwnI7C7K81uWddY9FLw1fCOIy6SsPIUftUz36Spt7jisCn8/40DhQMlSxpxTNelnZb/2SnloFimQfRZAmHLOqQ==", "license": "MIT", "dependencies": { - "@jimp/types": "1.6.0", - "@jimp/utils": "1.6.0", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", "zod": "^3.23.8" }, "engines": { @@ -2512,25 +2512,25 @@ } }, "node_modules/@jimp/plugin-blur": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-blur/-/plugin-blur-1.6.0.tgz", - "integrity": "sha512-zrM7iic1OTwUCb0g/rN5y+UnmdEsT3IfuCXCJJNs8SZzP0MkZ1eTvuwK9ZidCuMo4+J3xkzCidRwYXB5CyGZTw==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-blur/-/plugin-blur-1.6.1.tgz", + "integrity": "sha512-lIo7Tzp5jQu30EFFSK/phXANK3citKVEjepDjQ6ljHoIFtuMRrnybnmI2Md24ulvWlDaz+hh3n6qrMb8ydwhZQ==", "license": "MIT", "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/utils": "1.6.0" + "@jimp/core": "1.6.1", + "@jimp/utils": "1.6.1" }, "engines": { "node": ">=18" } }, "node_modules/@jimp/plugin-circle": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-circle/-/plugin-circle-1.6.0.tgz", - "integrity": "sha512-xt1Gp+LtdMKAXfDp3HNaG30SPZW6AQ7dtAtTnoRKorRi+5yCJjKqXRgkewS5bvj8DEh87Ko1ydJfzqS3P2tdWw==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-circle/-/plugin-circle-1.6.1.tgz", + "integrity": "sha512-kK1PavY6cKHNNKce37vdV4Tmpc1/zDKngGoeOV3j+EMatoHFZUinV3s6F9aWryPs3A0xhCLZgdJ6Zeea1d5LCQ==", "license": "MIT", "dependencies": { - "@jimp/types": "1.6.0", + "@jimp/types": "1.6.1", "zod": "^3.23.8" }, "engines": { @@ -2538,14 +2538,14 @@ } }, "node_modules/@jimp/plugin-color": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-color/-/plugin-color-1.6.0.tgz", - "integrity": "sha512-J5q8IVCpkBsxIXM+45XOXTrsyfblyMZg3a9eAo0P7VPH4+CrvyNQwaYatbAIamSIN1YzxmO3DkIZXzRjFSz1SA==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-color/-/plugin-color-1.6.1.tgz", + "integrity": "sha512-LtUN1vAP+LRlZAtTNVhDRSiXx+26Kbz3zJaG6a5k59gQ95jgT5mknnF8lxkHcqJthM4MEk3/tPxkdJpEybyF/A==", "license": "MIT", "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/types": "1.6.0", - "@jimp/utils": "1.6.0", + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", "tinycolor2": "^1.6.0", "zod": "^3.23.8" }, @@ -2554,16 +2554,16 @@ } }, "node_modules/@jimp/plugin-contain": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-contain/-/plugin-contain-1.6.0.tgz", - "integrity": "sha512-oN/n+Vdq/Qg9bB4yOBOxtY9IPAtEfES8J1n9Ddx+XhGBYT1/QTU/JYkGaAkIGoPnyYvmLEDqMz2SGihqlpqfzQ==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-contain/-/plugin-contain-1.6.1.tgz", + "integrity": "sha512-m0qhrfA8jkTqretGv4w+T/ADFR4GwBpE0sCOC2uJ0dzr44/ddOMsIdrpi89kabqYiPYIrxkgdCVCLm3zn1Vkkg==", "license": "MIT", "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/plugin-blit": "1.6.0", - "@jimp/plugin-resize": "1.6.0", - "@jimp/types": "1.6.0", - "@jimp/utils": "1.6.0", + "@jimp/core": "1.6.1", + "@jimp/plugin-blit": "1.6.1", + "@jimp/plugin-resize": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", "zod": "^3.23.8" }, "engines": { @@ -2571,15 +2571,15 @@ } }, "node_modules/@jimp/plugin-cover": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-cover/-/plugin-cover-1.6.0.tgz", - "integrity": "sha512-Iow0h6yqSC269YUJ8HC3Q/MpCi2V55sMlbkkTTx4zPvd8mWZlC0ykrNDeAy9IJegrQ7v5E99rJwmQu25lygKLA==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-cover/-/plugin-cover-1.6.1.tgz", + "integrity": "sha512-hZytnsth0zoll6cPf434BrT+p/v569Wr5tyO6Dp0dH1IDPhzhB5F38sZGMLDo7bzQiN9JFVB3fxkcJ/WYCJ3Mg==", "license": "MIT", "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/plugin-crop": "1.6.0", - "@jimp/plugin-resize": "1.6.0", - "@jimp/types": "1.6.0", + "@jimp/core": "1.6.1", + "@jimp/plugin-crop": "1.6.1", + "@jimp/plugin-resize": "1.6.1", + "@jimp/types": "1.6.1", "zod": "^3.23.8" }, "engines": { @@ -2587,14 +2587,14 @@ } }, "node_modules/@jimp/plugin-crop": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-crop/-/plugin-crop-1.6.0.tgz", - "integrity": "sha512-KqZkEhvs+21USdySCUDI+GFa393eDIzbi1smBqkUPTE+pRwSWMAf01D5OC3ZWB+xZsNla93BDS9iCkLHA8wang==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-crop/-/plugin-crop-1.6.1.tgz", + "integrity": "sha512-EerRSLlclXyKDnYc/H9w/1amZW7b7v3OGi/VlerPd2M/pAu5X8TkyYWtfqYCXnNp1Ixtd8oCo9zGfY9zoXT4rg==", "license": "MIT", "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/types": "1.6.0", - "@jimp/utils": "1.6.0", + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", "zod": "^3.23.8" }, "engines": { @@ -2602,13 +2602,13 @@ } }, "node_modules/@jimp/plugin-displace": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-displace/-/plugin-displace-1.6.0.tgz", - "integrity": "sha512-4Y10X9qwr5F+Bo5ME356XSACEF55485j5nGdiyJ9hYzjQP9nGgxNJaZ4SAOqpd+k5sFaIeD7SQ0Occ26uIng5Q==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-displace/-/plugin-displace-1.6.1.tgz", + "integrity": "sha512-K07QVl7xQwIfD6KfxRV/c3E9e7ZBXxUXdWuvoTWcKHL2qV48MOF5Nqbz/aJW4ThnQARIsxvYlZjPFiqkCjlU+g==", "license": "MIT", "dependencies": { - "@jimp/types": "1.6.0", - "@jimp/utils": "1.6.0", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", "zod": "^3.23.8" }, "engines": { @@ -2616,25 +2616,25 @@ } }, "node_modules/@jimp/plugin-dither": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-dither/-/plugin-dither-1.6.0.tgz", - "integrity": "sha512-600d1RxY0pKwgyU0tgMahLNKsqEcxGdbgXadCiVCoGd6V6glyCvkNrnnwC0n5aJ56Htkj88PToSdF88tNVZEEQ==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-dither/-/plugin-dither-1.6.1.tgz", + "integrity": "sha512-+2V+GCV2WycMoX1/z977TkZ8Zq/4MVSKElHYatgUqtwXMi2fDK2gKYU2g9V39IqFvTJsTIsK0+58VFz/ROBVew==", "license": "MIT", "dependencies": { - "@jimp/types": "1.6.0" + "@jimp/types": "1.6.1" }, "engines": { "node": ">=18" } }, "node_modules/@jimp/plugin-fisheye": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-fisheye/-/plugin-fisheye-1.6.0.tgz", - "integrity": "sha512-E5QHKWSCBFtpgZarlmN3Q6+rTQxjirFqo44ohoTjzYVrDI6B6beXNnPIThJgPr0Y9GwfzgyarKvQuQuqCnnfbA==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-fisheye/-/plugin-fisheye-1.6.1.tgz", + "integrity": "sha512-XtS5ZyoZ0vxZxJ6gkqI63SivhtI58vX95foMPM+cyzYkRsJXMOYCr8DScxF5bp4Xr003NjYm/P+7+08tibwzHA==", "license": "MIT", "dependencies": { - "@jimp/types": "1.6.0", - "@jimp/utils": "1.6.0", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", "zod": "^3.23.8" }, "engines": { @@ -2642,12 +2642,12 @@ } }, "node_modules/@jimp/plugin-flip": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-flip/-/plugin-flip-1.6.0.tgz", - "integrity": "sha512-/+rJVDuBIVOgwoyVkBjUFHtP+wmW0r+r5OQ2GpatQofToPVbJw1DdYWXlwviSx7hvixTWLKVgRWQ5Dw862emDg==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-flip/-/plugin-flip-1.6.1.tgz", + "integrity": "sha512-ws38W/sGj7LobNRayQ83garxiktOyWxM5vO/y4a/2cy9v65SLEUzVkrj+oeAaUSSObdz4HcCEla7XtGlnAGAaA==", "license": "MIT", "dependencies": { - "@jimp/types": "1.6.0", + "@jimp/types": "1.6.1", "zod": "^3.23.8" }, "engines": { @@ -2655,20 +2655,20 @@ } }, "node_modules/@jimp/plugin-hash": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-hash/-/plugin-hash-1.6.0.tgz", - "integrity": "sha512-wWzl0kTpDJgYVbZdajTf+4NBSKvmI3bRI8q6EH9CVeIHps9VWVsUvEyb7rpbcwVLWYuzDtP2R0lTT6WeBNQH9Q==", - "license": "MIT", - "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/js-bmp": "1.6.0", - "@jimp/js-jpeg": "1.6.0", - "@jimp/js-png": "1.6.0", - "@jimp/js-tiff": "1.6.0", - "@jimp/plugin-color": "1.6.0", - "@jimp/plugin-resize": "1.6.0", - "@jimp/types": "1.6.0", - "@jimp/utils": "1.6.0", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-hash/-/plugin-hash-1.6.1.tgz", + "integrity": "sha512-sZt6ZcMX6i8vFWb4GYnw0pR/o9++ef0dTVcboTB5B/g7nrxCODIB4wfEkJ/YqZM5wUvol77K1qeS0/rVO6z21A==", + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/js-bmp": "1.6.1", + "@jimp/js-jpeg": "1.6.1", + "@jimp/js-png": "1.6.1", + "@jimp/js-tiff": "1.6.1", + "@jimp/plugin-color": "1.6.1", + "@jimp/plugin-resize": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", "any-base": "^1.1.0" }, "engines": { @@ -2676,12 +2676,12 @@ } }, "node_modules/@jimp/plugin-mask": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-mask/-/plugin-mask-1.6.0.tgz", - "integrity": "sha512-Cwy7ExSJMZszvkad8NV8o/Z92X2kFUFM8mcDAhNVxU0Q6tA0op2UKRJY51eoK8r6eds/qak3FQkXakvNabdLnA==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-mask/-/plugin-mask-1.6.1.tgz", + "integrity": "sha512-SIG0/FcmEj3tkwFxc7fAGLO8o4uNzMpSOdQOhbCgxefQKq5wOVMk9BQx/sdMPBwtMLr9WLq0GzLA/rk6t2v20A==", "license": "MIT", "dependencies": { - "@jimp/types": "1.6.0", + "@jimp/types": "1.6.1", "zod": "^3.23.8" }, "engines": { @@ -2689,16 +2689,16 @@ } }, "node_modules/@jimp/plugin-print": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-print/-/plugin-print-1.6.0.tgz", - "integrity": "sha512-zarTIJi8fjoGMSI/M3Xh5yY9T65p03XJmPsuNet19K/Q7mwRU6EV2pfj+28++2PV2NJ+htDF5uecAlnGyxFN2A==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-print/-/plugin-print-1.6.1.tgz", + "integrity": "sha512-BYVz/X3Xzv8XYilVeDy11NOp0h7BTDjlOtu0BekIFHP1yHVd24AXNzbOy52XlzYZWQ0Dl36HOHEpl/nSNrzc6w==", "license": "MIT", "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/js-jpeg": "1.6.0", - "@jimp/js-png": "1.6.0", - "@jimp/plugin-blit": "1.6.0", - "@jimp/types": "1.6.0", + "@jimp/core": "1.6.1", + "@jimp/js-jpeg": "1.6.1", + "@jimp/js-png": "1.6.1", + "@jimp/plugin-blit": "1.6.1", + "@jimp/types": "1.6.1", "parse-bmfont-ascii": "^1.0.6", "parse-bmfont-binary": "^1.0.6", "parse-bmfont-xml": "^1.1.6", @@ -2710,9 +2710,9 @@ } }, "node_modules/@jimp/plugin-quantize": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-quantize/-/plugin-quantize-1.6.0.tgz", - "integrity": "sha512-EmzZ/s9StYQwbpG6rUGBCisc3f64JIhSH+ncTJd+iFGtGo0YvSeMdAd+zqgiHpfZoOL54dNavZNjF4otK+mvlg==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-quantize/-/plugin-quantize-1.6.1.tgz", + "integrity": "sha512-J2En9PLURfP+vwYDtuZ9T8yBW6BWYZBScydAjRiPBmJfEhTcNQqiiQODrZf7EqbbX/Sy5H6dAeRiqkgoV9N6Ww==", "license": "MIT", "dependencies": { "image-q": "^4.0.0", @@ -2723,13 +2723,13 @@ } }, "node_modules/@jimp/plugin-resize": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-resize/-/plugin-resize-1.6.0.tgz", - "integrity": "sha512-uSUD1mqXN9i1SGSz5ov3keRZ7S9L32/mAQG08wUwZiEi5FpbV0K8A8l1zkazAIZi9IJzLlTauRNU41Mi8IF9fA==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-resize/-/plugin-resize-1.6.1.tgz", + "integrity": "sha512-CLkrtJoIz2HdWnpYiN6p8KYcPc00rCH/SUu6o+lfZL05Q4uhecJlnvXuj9x+U6mDn3ldPmJj6aZqMHuUJzdVqg==", "license": "MIT", "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/types": "1.6.0", + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", "zod": "^3.23.8" }, "engines": { @@ -2737,16 +2737,16 @@ } }, "node_modules/@jimp/plugin-rotate": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-rotate/-/plugin-rotate-1.6.0.tgz", - "integrity": "sha512-JagdjBLnUZGSG4xjCLkIpQOZZ3Mjbg8aGCCi4G69qR+OjNpOeGI7N2EQlfK/WE8BEHOW5vdjSyglNqcYbQBWRw==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-rotate/-/plugin-rotate-1.6.1.tgz", + "integrity": "sha512-nOjVjbbj705B02ksysKnh0POAwEBXZtJ9zQ5qC+X7Tavl3JNn+P3BzQovbBxLPSbUSld6XID9z5ijin4PtOAUg==", "license": "MIT", "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/plugin-crop": "1.6.0", - "@jimp/plugin-resize": "1.6.0", - "@jimp/types": "1.6.0", - "@jimp/utils": "1.6.0", + "@jimp/core": "1.6.1", + "@jimp/plugin-crop": "1.6.1", + "@jimp/plugin-resize": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", "zod": "^3.23.8" }, "engines": { @@ -2754,16 +2754,16 @@ } }, "node_modules/@jimp/plugin-threshold": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/plugin-threshold/-/plugin-threshold-1.6.0.tgz", - "integrity": "sha512-M59m5dzLoHOVWdM41O8z9SyySzcDn43xHseOH0HavjsfQsT56GGCC4QzU1banJidbUrePhzoEdS42uFE8Fei8w==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-threshold/-/plugin-threshold-1.6.1.tgz", + "integrity": "sha512-JOKv9F8s6tnVLf4sB/2fF0F339EFnHvgEdFYugO6VhowKLsap0pEZmLyE/DlRnYtIj2RddHZVxVMp/eKJ04l2Q==", "license": "MIT", "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/plugin-color": "1.6.0", - "@jimp/plugin-hash": "1.6.0", - "@jimp/types": "1.6.0", - "@jimp/utils": "1.6.0", + "@jimp/core": "1.6.1", + "@jimp/plugin-color": "1.6.1", + "@jimp/plugin-hash": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", "zod": "^3.23.8" }, "engines": { @@ -2771,9 +2771,9 @@ } }, "node_modules/@jimp/types": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/types/-/types-1.6.0.tgz", - "integrity": "sha512-7UfRsiKo5GZTAATxm2qQ7jqmUXP0DxTArztllTcYdyw6Xi5oT4RaoXynVtCD4UyLK5gJgkZJcwonoijrhYFKfg==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/types/-/types-1.6.1.tgz", + "integrity": "sha512-leI7YbveTNi565m910XgIOwXyuu074H5qazAD1357HImJSv2hqxnWXpwxQbadGWZ7goZRYBDZy5lpqud0p7q5w==", "license": "MIT", "dependencies": { "zod": "^3.23.8" @@ -2783,12 +2783,12 @@ } }, "node_modules/@jimp/utils": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/@jimp/utils/-/utils-1.6.0.tgz", - "integrity": "sha512-gqFTGEosKbOkYF/WFj26jMHOI5OH2jeP1MmC/zbK6BF6VJBf8rIC5898dPfSzZEbSA0wbbV5slbntWVc5PKLFA==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/utils/-/utils-1.6.1.tgz", + "integrity": "sha512-veFPRd93FCnS7AgmCkPgARVGoDRrJ9cm1ujuNyA+UfQ5VKbED2002sm5XfFLFwTsKC8j04heTrwe+tU1dluXOw==", "license": "MIT", "dependencies": { - "@jimp/types": "1.6.0", + "@jimp/types": "1.6.1", "tinycolor2": "^1.6.0" }, "engines": { @@ -3578,25 +3578,24 @@ "license": "BSD-3-Clause" }, "node_modules/@protobufjs/codegen": { - "version": "2.0.4", - "resolved": "https://registry.npmjs.org/@protobufjs/codegen/-/codegen-2.0.4.tgz", - "integrity": "sha512-YyFaikqM5sH0ziFZCN3xDC7zeGaB/d0IUb9CATugHWbd1FRFwWwt4ld4OYMPWu5a3Xe01mGAULCdqhMlPl29Jg==", + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/@protobufjs/codegen/-/codegen-2.0.5.tgz", + "integrity": "sha512-zgXFLzW3Ap33e6d0Wlj4MGIm6Ce8O89n/apUaGNB/jx+hw+ruWEp7EwGUshdLKVRCxZW12fp9r40E1mQrf/34g==", "license": "BSD-3-Clause" }, "node_modules/@protobufjs/eventemitter": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/@protobufjs/eventemitter/-/eventemitter-1.1.0.tgz", - "integrity": "sha512-j9ednRT81vYJ9OfVuXG6ERSTdEL1xVsNgqpkxMsbIabzSo3goCjDIveeGv5d03om39ML71RdmrGNjG5SReBP/Q==", + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@protobufjs/eventemitter/-/eventemitter-1.1.1.tgz", + "integrity": "sha512-vW1GmwMZNnL+gMRaovlh9yZX74kc+TTU3FObkkurpMaRtBfLP3ldjS9KQWlwZgraRE0+dheEEoAxdzcJQ8eXZg==", "license": "BSD-3-Clause" }, "node_modules/@protobufjs/fetch": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/@protobufjs/fetch/-/fetch-1.1.0.tgz", - "integrity": "sha512-lljVXpqXebpsijW71PZaCYeIcE5on1w5DlQy5WH6GLbFryLUrBD4932W/E2BSpfRJWseIL4v/KPgBFxDOIdKpQ==", + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@protobufjs/fetch/-/fetch-1.1.1.tgz", + "integrity": "sha512-GpptLrs57adMSuHi3VNj0mAF8dwh36LMaYF6XyJ6JMWlVsc+t42tm1HSEDmOs3A8fC9yyeisgLhsTVQokOZ0zw==", "license": "BSD-3-Clause", "dependencies": { - "@protobufjs/aspromise": "^1.1.1", - "@protobufjs/inquire": "^1.1.0" + "@protobufjs/aspromise": "^1.1.1" } }, "node_modules/@protobufjs/float": { @@ -3605,12 +3604,6 @@ "integrity": "sha512-Ddb+kVXlXst9d+R9PfTIxh1EdNkgoRe5tOX6t01f1lYWOvJnSPDBlG241QLzcyPdoNTsblLUdujGSE4RzrTZGQ==", "license": "BSD-3-Clause" }, - "node_modules/@protobufjs/inquire": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/@protobufjs/inquire/-/inquire-1.1.0.tgz", - "integrity": "sha512-kdSefcPdruJiFMVSbn801t4vFK7KB/5gd2fYvrxhuJYg8ILrmn9SKSX2tZdV6V+ksulWqS7aXjBcRXl3wHoD9Q==", - "license": "BSD-3-Clause" - }, "node_modules/@protobufjs/path": { "version": "1.1.2", "resolved": "https://registry.npmjs.org/@protobufjs/path/-/path-1.1.2.tgz", @@ -3624,9 +3617,9 @@ "license": "BSD-3-Clause" }, "node_modules/@protobufjs/utf8": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/@protobufjs/utf8/-/utf8-1.1.0.tgz", - "integrity": "sha512-Vvn3zZrhQZkkBE8LSuW3em98c0FwgO4nxzv6OdSxPKJIEKY2bGbHn+mhGIPerzI4twdxaP8/0+06HBpwf345Lw==", + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/utf8/-/utf8-1.1.2.tgz", + "integrity": "sha512-b1UQwcEZ4yCnMCD8DAL1VlbvBJE9/IX4FTIp7BG1xYpf29SLazLSrqUkj4w7Y5y7cCVP6E5tcqqcI0xemPkHug==", "license": "BSD-3-Clause" }, "node_modules/@redis/bloom": { @@ -4758,34 +4751,6 @@ "url": "https://github.com/sponsors/Borewit" } }, - "node_modules/@tokenizer/inflate/node_modules/@borewit/text-codec": { - "version": "0.1.1", - "resolved": "https://registry.npmjs.org/@borewit/text-codec/-/text-codec-0.1.1.tgz", - "integrity": "sha512-5L/uBxmjaCIX5h8Z+uu+kA9BQLkc/Wl06UGR5ajNRxu+/XjonB5i8JpgFMrPj3LXTCPA0pv8yxUvbUi+QthGGA==", - "license": "MIT", - "funding": { - "type": "github", - "url": "https://github.com/sponsors/Borewit" - } - }, - "node_modules/@tokenizer/inflate/node_modules/token-types": { - "version": "6.1.1", - "resolved": "https://registry.npmjs.org/token-types/-/token-types-6.1.1.tgz", - "integrity": "sha512-kh9LVIWH5CnL63Ipf0jhlBIy0UsrMj/NJDfpsy1SqOXlLKEVyXXYrnFxFT1yOOYVGBSApeVnjPw/sBz5BfEjAQ==", - "license": "MIT", - "dependencies": { - "@borewit/text-codec": "^0.1.0", - "@tokenizer/token": "^0.3.0", - "ieee754": "^1.2.1" - }, - "engines": { - "node": ">=14.16" - }, - "funding": { - "type": "github", - "url": "https://github.com/sponsors/Borewit" - } - }, "node_modules/@tokenizer/token": { "version": "0.3.0", "resolved": "https://registry.npmjs.org/@tokenizer/token/-/token-0.3.0.tgz", @@ -4895,12 +4860,6 @@ "dev": true, "license": "MIT" }, - "node_modules/@types/long": { - "version": "4.0.2", - "resolved": "https://registry.npmjs.org/@types/long/-/long-4.0.2.tgz", - "integrity": "sha512-MqTGEo5bj5t157U6fA/BiDynNkn0YknVdh48CMPkTSpFTVmvao5UQmm7uEF6xBEo7qIMAlY/JSleYaE6VOdpaA==", - "license": "MIT" - }, "node_modules/@types/mime": { "version": "4.0.0", "resolved": "https://registry.npmjs.org/@types/mime/-/mime-4.0.0.tgz", @@ -5814,21 +5773,22 @@ } }, "node_modules/baileys": { - "version": "7.0.0-rc.9", - "resolved": "https://registry.npmjs.org/baileys/-/baileys-7.0.0-rc.9.tgz", - "integrity": "sha512-Txd2dZ9MHbojvsHckeuCnAKPO/bQjKxua/0tQSJwOKXffK5vpS82k4eA/Nb46K0cK0Bx+fyY0zhnQHYMBriQcw==", + "version": "7.0.0-rc14", + "resolved": "https://registry.npmjs.org/baileys/-/baileys-7.0.0-rc14.tgz", + "integrity": "sha512-pewtrljhWx5JTUBvvkXZz1fL3JPiwzjBsnhx/DWf2LWBx1cZNH7J/sF22xDehba5mySWUQU4a1Pwt7lWARUxaA==", "hasInstallScript": true, "license": "MIT", "dependencies": { "@cacheable/node-cache": "^1.4.0", "@hapi/boom": "^9.1.3", "async-mutex": "^0.5.0", - "libsignal": "git+https://github.com/whiskeysockets/libsignal-node.git", + "libsignal": "^6.0.0", "lru-cache": "^11.1.0", - "music-metadata": "^11.7.0", + "music-metadata": "^11.12.3", "p-queue": "^9.0.0", "pino": "^9.6", - "protobufjs": "^7.2.4", + "protobufjs": "^7.5.6", + "whatsapp-rust-bridge": "0.5.4", "ws": "^8.13.0" }, "engines": { @@ -5836,7 +5796,7 @@ }, "peerDependencies": { "audio-decode": "^2.1.3", - "jimp": "^1.6.0", + "jimp": "^1.6.1", "link-preview-js": "^3.0.0", "sharp": "*" }, @@ -5877,6 +5837,7 @@ "version": "1.5.1", "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz", "integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==", + "dev": true, "funding": [ { "type": "github", @@ -8336,15 +8297,6 @@ "integrity": "sha512-GWkBvjiSZK87ELrYOSESUYeVIc9mvLLf/nXalMOS5dYrgZq9o5OVkbZAVM06CVxYsCwH9BDZFPlQTlPA1j4ahA==", "license": "MIT" }, - "node_modules/events": { - "version": "3.3.0", - "resolved": "https://registry.npmjs.org/events/-/events-3.3.0.tgz", - "integrity": "sha512-mQw+2fkQbALzQ7V0MY0IqdnXNOeTtP4r0lN9z7AAawCXgqea7bDii20AYrIBrFd/Hx0M2Ocz6S111CaFkUcb0Q==", - "license": "MIT", - "engines": { - "node": ">=0.8.x" - } - }, "node_modules/exif-parser": { "version": "0.1.12", "resolved": "https://registry.npmjs.org/exif-parser/-/exif-parser-0.1.12.tgz", @@ -8629,17 +8581,18 @@ } }, "node_modules/file-type": { - "version": "16.5.4", - "resolved": "https://registry.npmjs.org/file-type/-/file-type-16.5.4.tgz", - "integrity": "sha512-/yFHK0aGjFEgDJjEKP0pWCplsPFPhwyfwevf/pVxiN0tmE4L9LmwWxWukdJSHdoCli4VgQLehjJtwQBnqmsKcw==", + "version": "21.3.4", + "resolved": "https://registry.npmjs.org/file-type/-/file-type-21.3.4.tgz", + "integrity": "sha512-Ievi/yy8DS3ygGvT47PjSfdFoX+2isQueoYP1cntFW1JLYAuS4GD7NUPGg4zv2iZfV52uDyk5w5Z0TdpRS6Q1g==", "license": "MIT", "dependencies": { - "readable-web-to-node-stream": "^3.0.0", - "strtok3": "^6.2.4", - "token-types": "^4.1.1" + "@tokenizer/inflate": "^0.4.1", + "strtok3": "^10.3.4", + "token-types": "^6.1.1", + "uint8array-extras": "^1.4.0" }, "engines": { - "node": ">=10" + "node": ">=20" }, "funding": { "url": "https://github.com/sindresorhus/file-type?sponsor=1" @@ -10351,38 +10304,38 @@ "license": "ISC" }, "node_modules/jimp": { - "version": "1.6.0", - "resolved": "https://registry.npmjs.org/jimp/-/jimp-1.6.0.tgz", - "integrity": "sha512-YcwCHw1kiqEeI5xRpDlPPBGL2EOpBKLwO4yIBJcXWHPj5PnA5urGq0jbyhM5KoNpypQ6VboSoxc9D8HyfvngSg==", - "license": "MIT", - "dependencies": { - "@jimp/core": "1.6.0", - "@jimp/diff": "1.6.0", - "@jimp/js-bmp": "1.6.0", - "@jimp/js-gif": "1.6.0", - "@jimp/js-jpeg": "1.6.0", - "@jimp/js-png": "1.6.0", - "@jimp/js-tiff": "1.6.0", - "@jimp/plugin-blit": "1.6.0", - "@jimp/plugin-blur": "1.6.0", - "@jimp/plugin-circle": "1.6.0", - "@jimp/plugin-color": "1.6.0", - "@jimp/plugin-contain": "1.6.0", - "@jimp/plugin-cover": "1.6.0", - "@jimp/plugin-crop": "1.6.0", - "@jimp/plugin-displace": "1.6.0", - "@jimp/plugin-dither": "1.6.0", - "@jimp/plugin-fisheye": "1.6.0", - "@jimp/plugin-flip": "1.6.0", - "@jimp/plugin-hash": "1.6.0", - "@jimp/plugin-mask": "1.6.0", - "@jimp/plugin-print": "1.6.0", - "@jimp/plugin-quantize": "1.6.0", - "@jimp/plugin-resize": "1.6.0", - "@jimp/plugin-rotate": "1.6.0", - "@jimp/plugin-threshold": "1.6.0", - "@jimp/types": "1.6.0", - "@jimp/utils": "1.6.0" + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/jimp/-/jimp-1.6.1.tgz", + "integrity": "sha512-hNQh6rZtWfSVWSNVmvq87N5BPJsNH7k7I7qyrXf9DOma9xATQk3fsyHazCQe51nCjdkoWdTmh0vD7bjVSLoxxw==", + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/diff": "1.6.1", + "@jimp/js-bmp": "1.6.1", + "@jimp/js-gif": "1.6.1", + "@jimp/js-jpeg": "1.6.1", + "@jimp/js-png": "1.6.1", + "@jimp/js-tiff": "1.6.1", + "@jimp/plugin-blit": "1.6.1", + "@jimp/plugin-blur": "1.6.1", + "@jimp/plugin-circle": "1.6.1", + "@jimp/plugin-color": "1.6.1", + "@jimp/plugin-contain": "1.6.1", + "@jimp/plugin-cover": "1.6.1", + "@jimp/plugin-crop": "1.6.1", + "@jimp/plugin-displace": "1.6.1", + "@jimp/plugin-dither": "1.6.1", + "@jimp/plugin-fisheye": "1.6.1", + "@jimp/plugin-flip": "1.6.1", + "@jimp/plugin-hash": "1.6.1", + "@jimp/plugin-mask": "1.6.1", + "@jimp/plugin-print": "1.6.1", + "@jimp/plugin-quantize": "1.6.1", + "@jimp/plugin-resize": "1.6.1", + "@jimp/plugin-rotate": "1.6.1", + "@jimp/plugin-threshold": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1" }, "engines": { "node": ">=18" @@ -10610,51 +10563,13 @@ "license": "MIT" }, "node_modules/libsignal": { - "name": "@whiskeysockets/libsignal-node", - "version": "2.0.1", - "resolved": "git+ssh://git@github.com/whiskeysockets/libsignal-node.git#1c30d7d7e76a3b0aa120b04dc6a26f5a12dccf67", + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/libsignal/-/libsignal-6.0.0.tgz", + "integrity": "sha512-d/5V3YFtDljbFMufz4ncyUYGYhJl+vzAe+c2EFFBQ6bz1h8Q3IOMEGXYMzlibU60I+e8GagMMpji18iez3P1hA==", "license": "GPL-3.0", "dependencies": { "curve25519-js": "^0.0.4", - "protobufjs": "6.8.8" - } - }, - "node_modules/libsignal/node_modules/@types/node": { - "version": "10.17.60", - "resolved": "https://registry.npmjs.org/@types/node/-/node-10.17.60.tgz", - "integrity": "sha512-F0KIgDJfy2nA3zMLmWGKxcH2ZVEtCZXHHdOQs2gSaQ27+lNeEfGxzkIw90aXswATX7AZ33tahPbzy6KAfUreVw==", - "license": "MIT" - }, - "node_modules/libsignal/node_modules/long": { - "version": "4.0.0", - "resolved": "https://registry.npmjs.org/long/-/long-4.0.0.tgz", - "integrity": "sha512-XsP+KhQif4bjX1kbuSiySJFNAehNxgLb6hPRGJ9QsUr8ajHkuXGdrHmFUTUUXhDwVX2R5bY4JNZEwbUiMhV+MA==", - "license": "Apache-2.0" - }, - "node_modules/libsignal/node_modules/protobufjs": { - "version": "6.8.8", - "resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-6.8.8.tgz", - "integrity": "sha512-AAmHtD5pXgZfi7GMpllpO3q1Xw1OYldr+dMUlAnffGTAhqkg72WdmSY71uKBF/JuyiKs8psYbtKrhi0ASCD8qw==", - "hasInstallScript": true, - "license": "BSD-3-Clause", - "dependencies": { - "@protobufjs/aspromise": "^1.1.2", - "@protobufjs/base64": "^1.1.2", - "@protobufjs/codegen": "^2.0.4", - "@protobufjs/eventemitter": "^1.1.0", - "@protobufjs/fetch": "^1.1.0", - "@protobufjs/float": "^1.0.2", - "@protobufjs/inquire": "^1.1.0", - "@protobufjs/path": "^1.1.2", - "@protobufjs/pool": "^1.1.0", - "@protobufjs/utf8": "^1.1.0", - "@types/long": "^4.0.0", - "@types/node": "^10.1.0", - "long": "^4.0.0" - }, - "bin": { - "pbjs": "bin/pbjs", - "pbts": "bin/pbts" + "protobufjs": "^7.5.5" } }, "node_modules/lilconfig": { @@ -11232,12 +11147,16 @@ } }, "node_modules/media-typer": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/media-typer/-/media-typer-1.1.0.tgz", - "integrity": "sha512-aisnrDP4GNe06UcKFnV5bfMNPBUw4jsLGaWwWfnH3v02GnBuXX2MCVn5RbrWo0j3pczUilYblq7fQ7Nw2t5XKw==", + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/media-typer/-/media-typer-2.0.0.tgz", + "integrity": "sha512-kOy3OxT2HH39N70UnKgu4NWDZjLOz8W/mfyvniHjRH/DrL3f2pOfvWQ4p60offbbtDAnXWp0v9LfMIqMec269Q==", "license": "MIT", "engines": { - "node": ">= 0.8" + "node": ">=18" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/express" } }, "node_modules/mediainfo.js": { @@ -11656,9 +11575,9 @@ } }, "node_modules/music-metadata": { - "version": "11.10.3", - "resolved": "https://registry.npmjs.org/music-metadata/-/music-metadata-11.10.3.tgz", - "integrity": "sha512-j0g/x4cNNZW6I5gdcPAY+GFkJY9WHTpkFDMBJKQLxJQyvSfQbXm57fTE3haGFFuOzCgtsTd4Plwc49Sn9RacDQ==", + "version": "11.16.1", + "resolved": "https://registry.npmjs.org/music-metadata/-/music-metadata-11.16.1.tgz", + "integrity": "sha512-uR/mHK6eyfl8h3jVCobF5b38Mfg1IDJMiBxIVbmm5F+G1OXEFcUpQHveO84tJBAqvY93GCRY2R4kDyx7s06Rug==", "funding": [ { "type": "github", @@ -11671,80 +11590,32 @@ ], "license": "MIT", "dependencies": { - "@borewit/text-codec": "^0.2.0", + "@borewit/text-codec": "^0.2.2", "@tokenizer/token": "^0.3.0", - "content-type": "^1.0.5", + "content-type": "^2.1.0", "debug": "^4.4.3", - "file-type": "^21.1.1", - "media-typer": "^1.1.0", - "strtok3": "^10.3.4", - "token-types": "^6.1.1", - "uint8array-extras": "^1.5.0" + "file-type": "^21.3.4", + "media-typer": "^2.0.0", + "strtok3": "^10.3.5", + "token-types": "^6.1.2", + "uint8array-extras": "^1.5.0", + "win-guid": "^0.2.1" }, "engines": { "node": ">=18" } }, - "node_modules/music-metadata/node_modules/file-type": { - "version": "21.1.1", - "resolved": "https://registry.npmjs.org/file-type/-/file-type-21.1.1.tgz", - "integrity": "sha512-ifJXo8zUqbQ/bLbl9sFoqHNTNWbnPY1COImFfM6CCy7z+E+jC1eY9YfOKkx0fckIg+VljAy2/87T61fp0+eEkg==", - "license": "MIT", - "dependencies": { - "@tokenizer/inflate": "^0.4.1", - "strtok3": "^10.3.4", - "token-types": "^6.1.1", - "uint8array-extras": "^1.4.0" - }, - "engines": { - "node": ">=20" - }, - "funding": { - "url": "https://github.com/sindresorhus/file-type?sponsor=1" - } - }, - "node_modules/music-metadata/node_modules/strtok3": { - "version": "10.3.4", - "resolved": "https://registry.npmjs.org/strtok3/-/strtok3-10.3.4.tgz", - "integrity": "sha512-KIy5nylvC5le1OdaaoCJ07L+8iQzJHGH6pWDuzS+d07Cu7n1MZ2x26P8ZKIWfbK02+XIL8Mp4RkWeqdUCrDMfg==", + "node_modules/music-metadata/node_modules/content-type": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/content-type/-/content-type-2.1.0.tgz", + "integrity": "sha512-mj7UPXE0jaqaOsukNZRUEfEi2AcL7C/vwmwcHV0O97eO1E1pxBZuyjlZrx5seTaNBg1U6+o35wpa35Qfcc+7ag==", "license": "MIT", - "dependencies": { - "@tokenizer/token": "^0.3.0" - }, "engines": { "node": ">=18" }, "funding": { - "type": "github", - "url": "https://github.com/sponsors/Borewit" - } - }, - "node_modules/music-metadata/node_modules/token-types": { - "version": "6.1.1", - "resolved": "https://registry.npmjs.org/token-types/-/token-types-6.1.1.tgz", - "integrity": "sha512-kh9LVIWH5CnL63Ipf0jhlBIy0UsrMj/NJDfpsy1SqOXlLKEVyXXYrnFxFT1yOOYVGBSApeVnjPw/sBz5BfEjAQ==", - "license": "MIT", - "dependencies": { - "@borewit/text-codec": "^0.1.0", - "@tokenizer/token": "^0.3.0", - "ieee754": "^1.2.1" - }, - "engines": { - "node": ">=14.16" - }, - "funding": { - "type": "github", - "url": "https://github.com/sponsors/Borewit" - } - }, - "node_modules/music-metadata/node_modules/token-types/node_modules/@borewit/text-codec": { - "version": "0.1.1", - "resolved": "https://registry.npmjs.org/@borewit/text-codec/-/text-codec-0.1.1.tgz", - "integrity": "sha512-5L/uBxmjaCIX5h8Z+uu+kA9BQLkc/Wl06UGR5ajNRxu+/XjonB5i8JpgFMrPj3LXTCPA0pv8yxUvbUi+QthGGA==", - "license": "MIT", - "funding": { - "type": "github", - "url": "https://github.com/sponsors/Borewit" + "type": "opencollective", + "url": "https://opencollective.com/express" } }, "node_modules/mute-stream": { @@ -12576,19 +12447,6 @@ "integrity": "sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==", "license": "MIT" }, - "node_modules/peek-readable": { - "version": "4.1.0", - "resolved": "https://registry.npmjs.org/peek-readable/-/peek-readable-4.1.0.tgz", - "integrity": "sha512-ZI3LnwUv5nOGbQzD9c2iDG6toheuXSZP5esSHBjopsXH4dg19soufvpUGA3uohi5anFtGb2lhAVdHzH6R/Evvg==", - "license": "MIT", - "engines": { - "node": ">=8" - }, - "funding": { - "type": "github", - "url": "https://github.com/sponsors/Borewit" - } - }, "node_modules/perfect-debounce": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/perfect-debounce/-/perfect-debounce-1.0.0.tgz", @@ -12957,15 +12815,6 @@ } } }, - "node_modules/process": { - "version": "0.11.10", - "resolved": "https://registry.npmjs.org/process/-/process-0.11.10.tgz", - "integrity": "sha512-cdGef/drWFoydD1JsMzuFf8100nZl+GT+yacc2bEced5f9Rjk4z+WtFUTBu9PhOi9j/jfmBPu0mMEY4wIdAF8A==", - "license": "MIT", - "engines": { - "node": ">= 0.6.0" - } - }, "node_modules/process-warning": { "version": "5.0.0", "resolved": "https://registry.npmjs.org/process-warning/-/process-warning-5.0.0.tgz", @@ -12983,24 +12832,23 @@ "license": "MIT" }, "node_modules/protobufjs": { - "version": "7.5.4", - "resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.5.4.tgz", - "integrity": "sha512-CvexbZtbov6jW2eXAvLukXjXUW1TzFaivC46BpWc/3BpcCysb5Vffu+B3XHMm8lVEuy2Mm4XGex8hBSg1yapPg==", + "version": "7.6.6", + "resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.6.6.tgz", + "integrity": "sha512-dYDWdjSl5RNb7SgPxGQcRU+GtvP7s2fpkrY0r432PcOIaZ0/rBcxEZnQN67iJhFuQiVw754JDoPruPCNdGsbjg==", "hasInstallScript": true, "license": "BSD-3-Clause", "dependencies": { "@protobufjs/aspromise": "^1.1.2", "@protobufjs/base64": "^1.1.2", - "@protobufjs/codegen": "^2.0.4", - "@protobufjs/eventemitter": "^1.1.0", - "@protobufjs/fetch": "^1.1.0", + "@protobufjs/codegen": "^2.0.5", + "@protobufjs/eventemitter": "^1.1.1", + "@protobufjs/fetch": "^1.1.1", "@protobufjs/float": "^1.0.2", - "@protobufjs/inquire": "^1.1.0", "@protobufjs/path": "^1.1.2", "@protobufjs/pool": "^1.1.0", - "@protobufjs/utf8": "^1.1.0", + "@protobufjs/utf8": "^1.1.1", "@types/node": ">=13.7.0", - "long": "^5.0.0" + "long": "^5.3.2" }, "engines": { "node": ">=12.0.0" @@ -13411,62 +13259,6 @@ "node": ">= 6" } }, - "node_modules/readable-web-to-node-stream": { - "version": "3.0.4", - "resolved": "https://registry.npmjs.org/readable-web-to-node-stream/-/readable-web-to-node-stream-3.0.4.tgz", - "integrity": "sha512-9nX56alTf5bwXQ3ZDipHJhusu9NTQJ/CVPtb/XHAJCXihZeitfJvIRS4GqQ/mfIoOE3IelHMrpayVrosdHBuLw==", - "license": "MIT", - "dependencies": { - "readable-stream": "^4.7.0" - }, - "engines": { - "node": ">=8" - }, - "funding": { - "type": "github", - "url": "https://github.com/sponsors/Borewit" - } - }, - "node_modules/readable-web-to-node-stream/node_modules/buffer": { - "version": "6.0.3", - "resolved": "https://registry.npmjs.org/buffer/-/buffer-6.0.3.tgz", - "integrity": "sha512-FTiCpNxtwiZZHEZbcbTIcZjERVICn9yq/pDFkTl95/AxzD1naBctN7YO68riM/gLSDY7sdrMby8hofADYuuqOA==", - "funding": [ - { - "type": "github", - "url": "https://github.com/sponsors/feross" - }, - { - "type": "patreon", - "url": "https://www.patreon.com/feross" - }, - { - "type": "consulting", - "url": "https://feross.org/support" - } - ], - "license": "MIT", - "dependencies": { - "base64-js": "^1.3.1", - "ieee754": "^1.2.1" - } - }, - "node_modules/readable-web-to-node-stream/node_modules/readable-stream": { - "version": "4.7.0", - "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-4.7.0.tgz", - "integrity": "sha512-oIGGmcpTLwPga8Bn6/Z75SVaH1z5dUut2ibSyAMVhmUggWpmDn2dapB0n7f8nwaSiRtepAsfJyfXIO5DCVAODg==", - "license": "MIT", - "dependencies": { - "abort-controller": "^3.0.0", - "buffer": "^6.0.3", - "events": "^3.3.0", - "process": "^0.11.10", - "string_decoder": "^1.3.0" - }, - "engines": { - "node": "^12.22.0 || ^14.17.0 || >=16.0.0" - } - }, "node_modules/readdirp": { "version": "4.1.2", "resolved": "https://registry.npmjs.org/readdirp/-/readdirp-4.1.2.tgz", @@ -14176,9 +13968,9 @@ "license": "ISC" }, "node_modules/simple-xml-to-json": { - "version": "1.2.3", - "resolved": "https://registry.npmjs.org/simple-xml-to-json/-/simple-xml-to-json-1.2.3.tgz", - "integrity": "sha512-kWJDCr9EWtZ+/EYYM5MareWj2cRnZGF93YDNpH4jQiHB+hBIZnfPFSQiVMzZOdk+zXWqTZ/9fTeQNu2DqeiudA==", + "version": "1.2.7", + "resolved": "https://registry.npmjs.org/simple-xml-to-json/-/simple-xml-to-json-1.2.7.tgz", + "integrity": "sha512-mz9VXphOxQWX3eQ/uXCtm6upltoN0DLx8Zb5T4TFC4FHB7S9FDPGre8CfLWqPWQQH/GrQYd2AXhhVM5LDpYx6Q==", "license": "MIT", "engines": { "node": ">=20.12.2" @@ -14640,16 +14432,15 @@ "license": "MIT" }, "node_modules/strtok3": { - "version": "6.3.0", - "resolved": "https://registry.npmjs.org/strtok3/-/strtok3-6.3.0.tgz", - "integrity": "sha512-fZtbhtvI9I48xDSywd/somNqgUHl2L2cstmXCCif0itOf96jeW18MBSyrLuNicYQVkvpOxkZtkzujiTJ9LW5Jw==", + "version": "10.3.5", + "resolved": "https://registry.npmjs.org/strtok3/-/strtok3-10.3.5.tgz", + "integrity": "sha512-ki4hZQfh5rX0QDLLkOCj+h+CVNkqmp/CMf8v8kZpkNVK6jGQooMytqzLZYUVYIZcFZ6yDB70EfD8POcFXiF5oA==", "license": "MIT", "dependencies": { - "@tokenizer/token": "^0.3.0", - "peek-readable": "^4.1.0" + "@tokenizer/token": "^0.3.0" }, "engines": { - "node": ">=10" + "node": ">=18" }, "funding": { "type": "github", @@ -14914,16 +14705,17 @@ } }, "node_modules/token-types": { - "version": "4.2.1", - "resolved": "https://registry.npmjs.org/token-types/-/token-types-4.2.1.tgz", - "integrity": "sha512-6udB24Q737UD/SDsKAHI9FCRP7Bqc9D/MQUV02ORQg5iskjtLJlZJNdN4kKtcdtwCeWIwIHDGaUsTsCCAa8sFQ==", + "version": "6.1.2", + "resolved": "https://registry.npmjs.org/token-types/-/token-types-6.1.2.tgz", + "integrity": "sha512-dRXchy+C0IgK8WPC6xvCHFRIWYUbqqdEIKPaKo/AcTUNzwLTK6AH7RjdLWsEZcAN/TBdtfUw3PYEgPr5VPr6ww==", "license": "MIT", "dependencies": { + "@borewit/text-codec": "^0.2.1", "@tokenizer/token": "^0.3.0", "ieee754": "^1.2.1" }, "engines": { - "node": ">=10" + "node": ">=14.16" }, "funding": { "type": "github", @@ -15944,6 +15736,12 @@ "integrity": "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ==", "license": "BSD-2-Clause" }, + "node_modules/whatsapp-rust-bridge": { + "version": "0.5.4", + "resolved": "https://registry.npmjs.org/whatsapp-rust-bridge/-/whatsapp-rust-bridge-0.5.4.tgz", + "integrity": "sha512-yYO1qSs0Fe7tGtnxOFHomocUD6IZtoAgmA4oDFyGIRZ67D3QZk3w7swA6XXFXNQngiyrg2k7tul6IrM3eUFh7A==", + "license": "MIT" + }, "node_modules/whatwg-url": { "version": "5.0.0", "resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-5.0.0.tgz", @@ -16064,6 +15862,12 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/win-guid": { + "version": "0.2.1", + "resolved": "https://registry.npmjs.org/win-guid/-/win-guid-0.2.1.tgz", + "integrity": "sha512-gEIQU4mkgl2OPeoNrWflcJFJ3Ae2BPd4eCsHHA/XikslkIVms/nHhvnvzIZV7VLmBvtFlDOzLt9rrZT+n6D67A==", + "license": "MIT" + }, "node_modules/word-wrap": { "version": "1.2.5", "resolved": "https://registry.npmjs.org/word-wrap/-/word-wrap-1.2.5.tgz", diff --git a/package.json b/package.json index 56e32fcc8a..369296c3cd 100644 --- a/package.json +++ b/package.json @@ -77,7 +77,7 @@ "amqplib": "^0.10.5", "audio-decode": "^2.2.3", "axios": "^1.7.9", - "baileys": "7.0.0-rc.9", + "baileys": "7.0.0-rc14", "class-validator": "^0.14.1", "compression": "^1.7.5", "cors": "^2.8.5", @@ -87,10 +87,10 @@ "eventemitter2": "^6.4.9", "express": "^4.21.2", "express-async-errors": "^3.1.1", + "fetch-socks": "^1.3.2", "fluent-ffmpeg": "^2.1.3", "form-data": "^4.0.1", "https-proxy-agent": "^7.0.6", - "fetch-socks": "^1.3.2", "i18next": "^23.7.19", "jimp": "^1.6.0", "json-schema": "^0.4.0", From a52aaf0a94790148b3bfbf32dcded0e9056a346c Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 13:57:51 +0300 Subject: [PATCH 003/157] test: a vitest harness that runs Evolution's source against the real Baileys Evolution has no test suite, and test/ has been in .gitignore since the first commit. This adds one: vitest 4.1.11 runs the TypeScript source directly, the real BaileysStartupService is driven through Baileys' real event buffer, and Prisma and the server module are replaced by in-memory fakes. - Baileys resolves from the installed package (package.json pins it), and deep imports reach the same build. BAILEYS_DIR points the suite at another build; BAILEYS_EXPECT fails the run when the resolved version differs. - Tests start from a production configuration (the minimal profile: only the instance stored, local cache, WARN and above). A test that needs storage asks for the stored profile. Co-Authored-By: Claude Opus 5.5 --- .gitignore | 2 +- package-lock.json | 1186 +++++++++++++++++++++++++- package.json | 8 +- test/global-setup.ts | 15 + test/harness/baileys-version.test.ts | 22 + test/harness/service.test.ts | 36 + test/helpers/app-state.ts | 45 + test/helpers/baileys-fixtures.ts | 30 + test/helpers/baileys-service.ts | 58 ++ test/helpers/capture-output.ts | 28 + test/helpers/fake-prisma.ts | 85 ++ test/helpers/fake-server-module.ts | 14 + test/helpers/http-app.ts | 20 + test/helpers/profiles.ts | 48 ++ test/helpers/socket-history.ts | 65 ++ test/setup.ts | 9 + vitest.config.mts | 38 + 17 files changed, 1664 insertions(+), 45 deletions(-) create mode 100644 test/global-setup.ts create mode 100644 test/harness/baileys-version.test.ts create mode 100644 test/harness/service.test.ts create mode 100644 test/helpers/app-state.ts create mode 100644 test/helpers/baileys-fixtures.ts create mode 100644 test/helpers/baileys-service.ts create mode 100644 test/helpers/capture-output.ts create mode 100644 test/helpers/fake-prisma.ts create mode 100644 test/helpers/fake-server-module.ts create mode 100644 test/helpers/http-app.ts create mode 100644 test/helpers/profiles.ts create mode 100644 test/helpers/socket-history.ts create mode 100644 test/setup.ts create mode 100644 vitest.config.mts diff --git a/.gitignore b/.gitignore index 768d8afa41..27f2df0d3d 100644 --- a/.gitignore +++ b/.gitignore @@ -34,7 +34,7 @@ lerna-debug.log* # Project related /instances/* !/instances/.gitkeep -/test/ + /src/env.yml /store *.env diff --git a/package-lock.json b/package-lock.json index 2fb0832437..506d676f9c 100644 --- a/package-lock.json +++ b/package-lock.json @@ -97,7 +97,8 @@ "prettier": "^3.4.2", "tsconfig-paths": "^4.2.0", "tsx": "^4.20.5", - "typescript": "^5.7.2" + "typescript": "^5.7.2", + "vitest": "^4.1.11" } }, "node_modules/@adiwajshing/keyed-db": { @@ -3414,6 +3415,16 @@ "@opentelemetry/api": "^1.1.0" } }, + "node_modules/@oxc-project/types": { + "version": "0.151.0", + "resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.151.0.tgz", + "integrity": "sha512-J1yXrIlNDZVzE3ada310xeAw7nH8yCAyLPuUIsjKatFPmfn5bS1oW+cM+QsGOtVWd5nhSpbwZWx/rue+r5Z+PA==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/oxc-project" + } + }, "node_modules/@paralleldrive/cuid2": { "version": "2.3.1", "resolved": "https://registry.npmjs.org/@paralleldrive/cuid2/-/cuid2-2.3.1.tgz", @@ -3681,6 +3692,286 @@ "@redis/client": "^1.0.0" } }, + "node_modules/@rolldown/binding-android-arm-eabi": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm-eabi/-/binding-android-arm-eabi-1.2.11.tgz", + "integrity": "sha512-A5kXfGKvKWWZE0TtPrfsvT+q4Y5d1QG8gGUzpYjGydM+fARM9MuX90PrXYXe0XbsDVgyxxNzHo6giCj90bsFNw==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-android-arm64": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.2.11.tgz", + "integrity": "sha512-z6cTycz+iJ4PVkuL4HHW4DfTfoeU/2nqYYuSOrTmH7yHK5Y0LCOnA03V4ZNxavyVaU1oOqUgIg2klN/s+USGOA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-darwin-arm64": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.2.11.tgz", + "integrity": "sha512-jShvqNtP6vDC6/A5JOAzbVV+DkgHqhl/ScVCJEbt+TUY6QYz7YnXcrg3sLtFBniro0f/Ld50ZwCWA6f7KYD1nQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-darwin-x64": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.2.11.tgz", + "integrity": "sha512-f2i2xiNWq1Z1l2++q2fuhZRdLAT3aqxD6vRNm1RAxpUoBcdqNB3C0s1Bt+K+PbEx2F5F4gQp6hqKkphCY/xF9w==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-freebsd-x64": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.2.11.tgz", + "integrity": "sha512-4Ir5FSOKIAMr4r0kExpt1s3bMgzJU3rA45AYOHtQpls0oNeqcYBKrWMlckrYH4KCfGLfkfn1tN1dmZPMVsdXow==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-linux-arm-gnueabihf": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.2.11.tgz", + "integrity": "sha512-/gnRDM+39BROzAN/k1OZjDPnDMcZxB/0EUxKjONO5yVkNEvlsoMDrxGNKgZi/ttFriS2gwlDNzB65pvNbFOXIQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-linux-arm64-gnu": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.2.11.tgz", + "integrity": "sha512-PFaK8HwvAHbaKbBcDNQihjMKYvFnA5hiENx/l5tphTDz1E0WFp32l0A7aq7lyUwGsRw/xSrNIy/gIK4thrSCrw==", + "cpu": [ + "arm64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-linux-arm64-musl": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.2.11.tgz", + "integrity": "sha512-AskzJUIKRLPxkruR1wLKewGbOw+EYfU/9lOrBFj4AFrEA8hPpKFnODWNu2WLaNs0QNkEb9QIJufmVZZIL/bJlg==", + "cpu": [ + "arm64" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-linux-ppc64-gnu": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.2.11.tgz", + "integrity": "sha512-qlUGAheh2yh8afH7QBgx0PrRHN85hKnNd78x8MeMhXivuevgd8vgf6/CstOzmNKY/lLTHvNTrPy98cLnAugzJw==", + "cpu": [ + "ppc64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-linux-s390x-gnu": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.2.11.tgz", + "integrity": "sha512-secpEad+0vCbSfn8upFySkDskv+bGPk3THSDS9Y89yc4rb4kzqHp8Dmyd9BkQW4SnhNXBZCl/6CrO//hZahNJQ==", + "cpu": [ + "s390x" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-linux-x64-gnu": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.2.11.tgz", + "integrity": "sha512-mOVBT3dPpkWm8XBWPmU4bf+U6dYDLeMo/9ojUmis4N0L5uu10qra5vOyngZ7/PSdoE4G9KvRt4bloRxNjLas7A==", + "cpu": [ + "x64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-linux-x64-musl": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.2.11.tgz", + "integrity": "sha512-Is78i9A8Ui4SqcxUwFJ9uMmjDn58IbVTjFWYdQestFEgeuEmHMLGNriXnVJKkwG2YiZjw8cP0zCTyDMdDGtOOg==", + "cpu": [ + "x64" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-openharmony-arm64": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.2.11.tgz", + "integrity": "sha512-dUCXneZ87INUMyQ0D+C0HrEBNUPNXHaPmU5GTjyKTJEiussw9Kaj5Ln8UztPe4epV/ffvgNBEadksdYhmW6xJA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-win32-arm64-msvc": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.2.11.tgz", + "integrity": "sha512-jByxb6qfd+bH1xUd0qnfFnb17i9sWBPY2tOavJ0l3tdr3OTu+Kvtm8cd/JV5nFt657b1VqGltxg9olOEfofXWw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-win32-x64-msvc": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.2.11.tgz", + "integrity": "sha512-/PzKqzAJ03i19oy2ItPvyvaVjOjBCNnfaJs8yvUdGBKmiESgnrJSQ2awd81QzFbbnAmu7YO9ZnJrDCb9VSJPRA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/pluginutils": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.1.tgz", + "integrity": "sha512-2j9bGt5Jh8hj+vPtgzPtl72j0yRxHAyumoo6TNfAjsLB04UtpSvPbPcDcBMxz7n+9CYB0c1GxQFxYRg2jimqGw==", + "dev": true, + "license": "MIT" + }, "node_modules/@rollup/rollup-android-arm-eabi": { "version": "4.53.3", "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.53.3.tgz", @@ -4680,9 +4971,9 @@ "license": "MIT" }, "node_modules/@standard-schema/spec": { - "version": "1.0.0", - "resolved": "https://registry.npmjs.org/@standard-schema/spec/-/spec-1.0.0.tgz", - "integrity": "sha512-m2bOd0f2RT9k8QJx1JN85cZYyH1RqFBdlwtkSlf4tBDYLCiiZnv1fIIwacK6cqwXavOydf0NPToMQgpKq+dVlA==", + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/@standard-schema/spec/-/spec-1.1.0.tgz", + "integrity": "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==", "license": "MIT" }, "node_modules/@thi.ng/bitstream": { @@ -4768,6 +5059,17 @@ "@types/node": "*" } }, + "node_modules/@types/chai": { + "version": "5.2.3", + "resolved": "https://registry.npmjs.org/@types/chai/-/chai-5.2.3.tgz", + "integrity": "sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/deep-eql": "*", + "assertion-error": "^2.0.1" + } + }, "node_modules/@types/compression": { "version": "1.8.1", "resolved": "https://registry.npmjs.org/@types/compression/-/compression-1.8.1.tgz", @@ -4807,6 +5109,13 @@ "@types/node": "*" } }, + "node_modules/@types/deep-eql": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/@types/deep-eql/-/deep-eql-4.0.2.tgz", + "integrity": "sha512-c9h9dVVMigMPc4bwTvC5dxqtqJZwQPePsWjPlpSOnojbor6pGqdk541lfA7AqFQr5pB1BRdq0juY9db81BwyFw==", + "dev": true, + "license": "MIT" + }, "node_modules/@types/estree": { "version": "1.0.8", "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.8.tgz", @@ -5273,6 +5582,119 @@ "dev": true, "license": "ISC" }, + "node_modules/@vitest/expect": { + "version": "4.1.11", + "resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.11.tgz", + "integrity": "sha512-VX2x5vNJXET47KAFzwERI+KRMtTTCSWTfSMKsW7JsUsXV4psq++e3DvZpuTDOpHcxytiDs6p2nhVb2tVDiiUYw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@standard-schema/spec": "^1.1.0", + "@types/chai": "^5.2.2", + "@vitest/spy": "4.1.11", + "@vitest/utils": "4.1.11", + "chai": "^6.2.2", + "tinyrainbow": "^3.1.0" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/@vitest/mocker": { + "version": "4.1.11", + "resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-4.1.11.tgz", + "integrity": "sha512-2XJVD55d1o5AZous5CCGKS74g/riOj9odEt2bQpCVZeblHyHdnMeFl4jl0XjU21stf4mbjUkew2eXQZt65g5CQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vitest/spy": "4.1.11", + "estree-walker": "^3.0.3", + "magic-string": "^0.30.21" + }, + "funding": { + "url": "https://opencollective.com/vitest" + }, + "peerDependencies": { + "msw": "^2.4.9", + "vite": "^6.0.0 || ^7.0.0 || ^8.0.0" + }, + "peerDependenciesMeta": { + "msw": { + "optional": true + }, + "vite": { + "optional": true + } + } + }, + "node_modules/@vitest/pretty-format": { + "version": "4.1.11", + "resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-4.1.11.tgz", + "integrity": "sha512-yiZzPbGTS9Sr/JpFl8zHrcIkAofNbFV6k21vIgQN/cY/oxZeXhJv5sc/MBJ5jFKWmWs+oJHw0UXLZjmf931+Vw==", + "dev": true, + "license": "MIT", + "dependencies": { + "tinyrainbow": "^3.1.0" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/@vitest/runner": { + "version": "4.1.11", + "resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-4.1.11.tgz", + "integrity": "sha512-LztvUgdwMNJMIkj3hQnnxiC2Xy1zNxq928W/xhjCLaNCzqTZOudjwbQf6v9IntZGPw132i2Lq2rgTRZHD3JHNw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vitest/utils": "4.1.11", + "pathe": "^2.0.3" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/@vitest/snapshot": { + "version": "4.1.11", + "resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-4.1.11.tgz", + "integrity": "sha512-pN7ikn1ON7h8ee4gIAp4AzyK+zBtJPzVbqOgu5LCEh4VaJVbPQcgYQYJIMGQPXVeJJq1fnfazis7a5pFNPahog==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vitest/pretty-format": "4.1.11", + "@vitest/utils": "4.1.11", + "magic-string": "^0.30.21", + "pathe": "^2.0.3" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/@vitest/spy": { + "version": "4.1.11", + "resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-4.1.11.tgz", + "integrity": "sha512-apNa/prQy2qCeywhnixOHPRCgGNhvg7T4Dapfl1GahLp/R+uhBm5cPyFoNVyqsNd2h1nJxL6BqqdIjiABL60YA==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/@vitest/utils": { + "version": "4.1.11", + "resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-4.1.11.tgz", + "integrity": "sha512-zTCVGpyFsGWBhllOyKlTw/vnr6D9qxsfSDyfbyZmTyjHw5N/VuvzHpHoQjm2ZJzn4RJgx5w4r7V0er69CmLgPQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vitest/pretty-format": "4.1.11", + "convert-source-map": "^2.0.0", + "tinyrainbow": "^3.1.0" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, "node_modules/@wasm-audio-decoders/common": { "version": "9.0.7", "resolved": "https://registry.npmjs.org/@wasm-audio-decoders/common/-/common-9.0.7.tgz", @@ -5657,6 +6079,16 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/assertion-error": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/assertion-error/-/assertion-error-2.0.1.tgz", + "integrity": "sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=12" + } + }, "node_modules/async": { "version": "0.2.10", "resolved": "https://registry.npmjs.org/async/-/async-0.2.10.tgz", @@ -6182,6 +6614,16 @@ "node": ">=6" } }, + "node_modules/chai": { + "version": "6.2.2", + "resolved": "https://registry.npmjs.org/chai/-/chai-6.2.2.tgz", + "integrity": "sha512-NUPRluOfOiTKBKvWPtSD4PhFvWCqOi0BGStNWs57X9js7XGTprSmFoz5F0tWhR4WPjNeR9jXqdC7/UpSJTnlRg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=18" + } + }, "node_modules/chalk": { "version": "5.6.2", "resolved": "https://registry.npmjs.org/chalk/-/chalk-5.6.2.tgz", @@ -6673,6 +7115,13 @@ "node": ">=16" } }, + "node_modules/convert-source-map": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/convert-source-map/-/convert-source-map-2.0.0.tgz", + "integrity": "sha512-Kvp459HrV2FEJ1CAsi1Ku+MY3kasH19TFykTz2xWmMeq6bk2NU3XXvfJ+Q61m0xktWwt+1HSYf3JZsTms3aRJg==", + "dev": true, + "license": "MIT" + }, "node_modules/cookie": { "version": "0.7.1", "resolved": "https://registry.npmjs.org/cookie/-/cookie-0.7.1.tgz", @@ -7533,6 +7982,13 @@ "node": ">= 0.4" } }, + "node_modules/es-module-lexer": { + "version": "2.3.2", + "resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-2.3.2.tgz", + "integrity": "sha512-poHGpORABojJJucnV9KbOavETW8lBVnphkW77ER5/BQ5Fz7oXSoCNek7IH3vR5nRjdsEz926ibFYX8KtLQmdyw==", + "dev": true, + "license": "MIT" + }, "node_modules/es-object-atoms": { "version": "1.1.1", "resolved": "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.1.tgz", @@ -8257,6 +8713,16 @@ "node": ">=4.0" } }, + "node_modules/estree-walker": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/estree-walker/-/estree-walker-3.0.3.tgz", + "integrity": "sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/estree": "^1.0.0" + } + }, "node_modules/esutils": { "version": "2.0.3", "resolved": "https://registry.npmjs.org/esutils/-/esutils-2.0.3.tgz", @@ -8315,6 +8781,16 @@ "node": ">=0.10.0" } }, + "node_modules/expect-type": { + "version": "1.4.0", + "resolved": "https://registry.npmjs.org/expect-type/-/expect-type-1.4.0.tgz", + "integrity": "sha512-KfYbmpRm0VbLjEvVa9yGwCi9GI34xvi7A/HXYWQO65CSD2u3MczUJSuwXKFIxlGsgBQizV9q5J9NHj4VG0n+pA==", + "dev": true, + "license": "Apache-2.0", + "engines": { + "node": ">=12.0.0" + } + }, "node_modules/express": { "version": "4.21.2", "resolved": "https://registry.npmjs.org/express/-/express-4.21.2.tgz", @@ -9043,7 +9519,7 @@ "version": "4.13.0", "resolved": "https://registry.npmjs.org/get-tsconfig/-/get-tsconfig-4.13.0.tgz", "integrity": "sha512-1VKTZJCwBrvbd+Wn3AOgQP/2Av+TfTCOlE4AcRJE72W1ksZXbAx8PPBR9RzgTeSPzlPMHrbANMH3LbltH73wxQ==", - "devOptional": true, + "dev": true, "license": "MIT", "dependencies": { "resolve-pkg-maps": "^1.0.0" @@ -10542,34 +11018,307 @@ "@keyv/serialize": "^1.1.1" } }, - "node_modules/levn": { - "version": "0.4.1", - "resolved": "https://registry.npmjs.org/levn/-/levn-0.4.1.tgz", - "integrity": "sha512-+bT2uH4E5LGE7h/n3evcS/sQlJXCpIp6ym8OWJ5eV6+67Dsql/LaaT7qJBAt2rzfoa/5QBGBhxDix1dMt2kQKQ==", + "node_modules/levn": { + "version": "0.4.1", + "resolved": "https://registry.npmjs.org/levn/-/levn-0.4.1.tgz", + "integrity": "sha512-+bT2uH4E5LGE7h/n3evcS/sQlJXCpIp6ym8OWJ5eV6+67Dsql/LaaT7qJBAt2rzfoa/5QBGBhxDix1dMt2kQKQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "prelude-ls": "^1.2.1", + "type-check": "~0.4.0" + }, + "engines": { + "node": ">= 0.8.0" + } + }, + "node_modules/libphonenumber-js": { + "version": "1.12.29", + "resolved": "https://registry.npmjs.org/libphonenumber-js/-/libphonenumber-js-1.12.29.tgz", + "integrity": "sha512-P2aLrbeqHbmh8+9P35LXQfXOKc7XJ0ymUKl7tyeyQjdRNfzunXWxQXGc4yl3fUf28fqLRfPY+vIVvFXK7KEBTw==", + "license": "MIT" + }, + "node_modules/libsignal": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/libsignal/-/libsignal-6.0.0.tgz", + "integrity": "sha512-d/5V3YFtDljbFMufz4ncyUYGYhJl+vzAe+c2EFFBQ6bz1h8Q3IOMEGXYMzlibU60I+e8GagMMpji18iez3P1hA==", + "license": "GPL-3.0", + "dependencies": { + "curve25519-js": "^0.0.4", + "protobufjs": "^7.5.5" + } + }, + "node_modules/lightningcss": { + "version": "1.33.0", + "resolved": "https://registry.npmjs.org/lightningcss/-/lightningcss-1.33.0.tgz", + "integrity": "sha512-WkUDrojuJs0xkgGf2udWxa3yGBRxPtxUkB79i6aCZLRgc7PM8fZe9TosfPDcvEpQZbuFASnHYmRLBLUbmLOIIA==", + "dev": true, + "license": "MPL-2.0", + "dependencies": { + "detect-libc": "^2.0.3" + }, + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + }, + "optionalDependencies": { + "lightningcss-android-arm64": "1.33.0", + "lightningcss-darwin-arm64": "1.33.0", + "lightningcss-darwin-x64": "1.33.0", + "lightningcss-freebsd-x64": "1.33.0", + "lightningcss-linux-arm-gnueabihf": "1.33.0", + "lightningcss-linux-arm64-gnu": "1.33.0", + "lightningcss-linux-arm64-musl": "1.33.0", + "lightningcss-linux-x64-gnu": "1.33.0", + "lightningcss-linux-x64-musl": "1.33.0", + "lightningcss-win32-arm64-msvc": "1.33.0", + "lightningcss-win32-x64-msvc": "1.33.0" + } + }, + "node_modules/lightningcss-android-arm64": { + "version": "1.33.0", + "resolved": "https://registry.npmjs.org/lightningcss-android-arm64/-/lightningcss-android-arm64-1.33.0.tgz", + "integrity": "sha512-gEpRTalKdosp4Bb8qWtc2iOgE5SeIHlpS1up9bFq2wAyYhl1UdTObYiHe98zEM9SQvSoqQZ1IQD0JNpg3Ml5pg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-darwin-arm64": { + "version": "1.33.0", + "resolved": "https://registry.npmjs.org/lightningcss-darwin-arm64/-/lightningcss-darwin-arm64-1.33.0.tgz", + "integrity": "sha512-Sciaz8eenNTKn9b3t7+xr0ipTp9YxKQY4npwQ3mrRuL0BAVHBLyZxofhaKBAVtzmtRZ/zTyo0/to4B1uWG/Djg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-darwin-x64": { + "version": "1.33.0", + "resolved": "https://registry.npmjs.org/lightningcss-darwin-x64/-/lightningcss-darwin-x64-1.33.0.tgz", + "integrity": "sha512-Z5UPAxzrjlWNNyGy6i65cJzzvgJ5D3T6wMvs+gWpY9d7qRhANrxqAp6LhxIgZhWEw18RfJTGcRxjuLIBr+m8XQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-freebsd-x64": { + "version": "1.33.0", + "resolved": "https://registry.npmjs.org/lightningcss-freebsd-x64/-/lightningcss-freebsd-x64-1.33.0.tgz", + "integrity": "sha512-QQM/Ti/hQajJwCY+RiWuCZ9sdtI/XQk7nDK5vC8kkdwixezOlDgvDx7+RT+QjK6FcFT4MpsuoBnHIo/O3StRRg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-linux-arm-gnueabihf": { + "version": "1.33.0", + "resolved": "https://registry.npmjs.org/lightningcss-linux-arm-gnueabihf/-/lightningcss-linux-arm-gnueabihf-1.33.0.tgz", + "integrity": "sha512-N7FVBe6iS24MlM6R/4RBTxGhQheZGs7tiQ9U32UtF75NzP5Q7xWPRqLBCKxlRQRk3rY1jCIPLzx7WzOhuUIRLQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-linux-arm64-gnu": { + "version": "1.33.0", + "resolved": "https://registry.npmjs.org/lightningcss-linux-arm64-gnu/-/lightningcss-linux-arm64-gnu-1.33.0.tgz", + "integrity": "sha512-j2v/itmy4HlNxlc6voKXYgBqNi0Ng2LShg4z7GufpEgs05P+2suBVyi9I6YHq5uoVFx9ETin3eCEhLVyXGQnKg==", + "cpu": [ + "arm64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MPL-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-linux-arm64-musl": { + "version": "1.33.0", + "resolved": "https://registry.npmjs.org/lightningcss-linux-arm64-musl/-/lightningcss-linux-arm64-musl-1.33.0.tgz", + "integrity": "sha512-yiO5ROMuYQgXbC60yjZU5CYSFZGKXL0HFATXt9mHJn1+zW55oCtMI9NfcVhYLMFDL7gV7oBPon/EmMMGg2OvtQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MPL-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-linux-x64-gnu": { + "version": "1.33.0", + "resolved": "https://registry.npmjs.org/lightningcss-linux-x64-gnu/-/lightningcss-linux-x64-gnu-1.33.0.tgz", + "integrity": "sha512-ar+Ju7LmcN0Jo4FpL4hpFybwNG9/3A/Br5KW2n2jyODg3MEZXaDYADdemoNS+BDNfMgKvylJLj4S5tyRActuAg==", + "cpu": [ + "x64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MPL-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-linux-x64-musl": { + "version": "1.33.0", + "resolved": "https://registry.npmjs.org/lightningcss-linux-x64-musl/-/lightningcss-linux-x64-musl-1.33.0.tgz", + "integrity": "sha512-RYiYbkokw0trfKqqzfF55lginwEPrD3OJDfTuJzFs1MK6iFnDenaz1fqLLtX4ITG3OktJQXOeTaw1awrBAlZPw==", + "cpu": [ + "x64" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MPL-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-win32-arm64-msvc": { + "version": "1.33.0", + "resolved": "https://registry.npmjs.org/lightningcss-win32-arm64-msvc/-/lightningcss-win32-arm64-msvc-1.33.0.tgz", + "integrity": "sha512-1K+MPfLSFVpphzpdbfkhlWk6wBrTObBzS2T6db10PNOZgR9GoVsAWzwNyuhUYYbTp23j+4RrncfujZ4uAzXvwA==", + "cpu": [ + "arm64" + ], "dev": true, - "license": "MIT", - "dependencies": { - "prelude-ls": "^1.2.1", - "type-check": "~0.4.0" - }, + "license": "MPL-2.0", + "optional": true, + "os": [ + "win32" + ], "engines": { - "node": ">= 0.8.0" + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" } }, - "node_modules/libphonenumber-js": { - "version": "1.12.29", - "resolved": "https://registry.npmjs.org/libphonenumber-js/-/libphonenumber-js-1.12.29.tgz", - "integrity": "sha512-P2aLrbeqHbmh8+9P35LXQfXOKc7XJ0ymUKl7tyeyQjdRNfzunXWxQXGc4yl3fUf28fqLRfPY+vIVvFXK7KEBTw==", - "license": "MIT" - }, - "node_modules/libsignal": { - "version": "6.0.0", - "resolved": "https://registry.npmjs.org/libsignal/-/libsignal-6.0.0.tgz", - "integrity": "sha512-d/5V3YFtDljbFMufz4ncyUYGYhJl+vzAe+c2EFFBQ6bz1h8Q3IOMEGXYMzlibU60I+e8GagMMpji18iez3P1hA==", - "license": "GPL-3.0", - "dependencies": { - "curve25519-js": "^0.0.4", - "protobufjs": "^7.5.5" + "node_modules/lightningcss-win32-x64-msvc": { + "version": "1.33.0", + "resolved": "https://registry.npmjs.org/lightningcss-win32-x64-msvc/-/lightningcss-win32-x64-msvc-1.33.0.tgz", + "integrity": "sha512-OlEICDx/Xl0FqSp4bry8zFnCvGpig3Gl4gCquvYwHuqJKEC1+n9NgDniFvqHGmMv1ZkqDJrDqKKSykTDX+ehuA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" } }, "node_modules/lilconfig": { @@ -11649,6 +12398,25 @@ "url": "https://github.com/sindresorhus/nano-spawn?sponsor=1" } }, + "node_modules/nanoid": { + "version": "3.3.19", + "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.19.tgz", + "integrity": "sha512-Y2tUNy4ouw6tq5oDSKeQYGOyhkUBhNOcGV/02KC+6kd9eDGqdZd++mjMiIDilrBYvjEnCYvVtsuHCuP+okSfug==", + "dev": true, + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/ai" + } + ], + "license": "MIT", + "bin": { + "nanoid": "bin/nanoid.cjs" + }, + "engines": { + "node": "^10 || ^12 || ^13.7 || ^14 || >=15.0.1" + } + }, "node_modules/nats": { "version": "2.29.3", "resolved": "https://registry.npmjs.org/nats/-/nats-2.29.3.tgz", @@ -11913,6 +12681,20 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/obug": { + "version": "2.2.1", + "resolved": "https://registry.npmjs.org/obug/-/obug-2.2.1.tgz", + "integrity": "sha512-XrsrhT5sybtKI6wakr2SPOlGZWWYbUXZ7a0jT8/QOeAPau+1X/bSegNe5YR75oJmEZQbKningirmGOEJCIk61Q==", + "dev": true, + "funding": [ + "https://github.com/sponsors/sxzz", + "https://opencollective.com/debug" + ], + "license": "MIT", + "engines": { + "node": ">=12.20.0" + } + }, "node_modules/ogg-opus-decoder": { "version": "1.7.3", "resolved": "https://registry.npmjs.org/ogg-opus-decoder/-/ogg-opus-decoder-1.7.3.tgz", @@ -12670,6 +13452,35 @@ "node": ">= 0.4" } }, + "node_modules/postcss": { + "version": "8.5.28", + "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.28.tgz", + "integrity": "sha512-RRuzqDtt5Y9h3quz5hWhK+TPnsmVs6WwSU6LkJMeY4HstUEDuYTG8UJSdawMRzmzAtV+KEoG8N3Qg2qLy5vM/A==", + "dev": true, + "funding": [ + { + "type": "opencollective", + "url": "https://opencollective.com/postcss/" + }, + { + "type": "tidelift", + "url": "https://tidelift.com/funding/github/npm/postcss" + }, + { + "type": "github", + "url": "https://github.com/sponsors/ai" + } + ], + "license": "MIT", + "dependencies": { + "nanoid": "^3.3.18", + "picocolors": "^1.1.1", + "source-map-js": "^1.2.1" + }, + "engines": { + "node": "^10 || ^12 || >=14" + } + }, "node_modules/postcss-load-config": { "version": "6.0.1", "resolved": "https://registry.npmjs.org/postcss-load-config/-/postcss-load-config-6.0.1.tgz", @@ -13434,7 +14245,7 @@ "version": "1.0.0", "resolved": "https://registry.npmjs.org/resolve-pkg-maps/-/resolve-pkg-maps-1.0.0.tgz", "integrity": "sha512-seS2Tj26TBVOC2NIc2rOe2y2ZO7efxITtLZcGSOnHHNOQ7CkiUBfw0Iw2ck6xkIhPwLhKNLS8BO+hEpngQlqzw==", - "devOptional": true, + "dev": true, "license": "MIT", "funding": { "url": "https://github.com/privatenumber/resolve-pkg-maps?sponsor=1" @@ -13489,6 +14300,40 @@ "url": "https://github.com/sponsors/isaacs" } }, + "node_modules/rolldown": { + "version": "1.2.11", + "resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.2.11.tgz", + "integrity": "sha512-qpSwIyz0jHQq5qXBTNxFmE6664rJ7O+4TvPFOiOaBSrz8IOHc1koKKSqTM2H6u1UG1+TveuC6vaDHKXFOvb1Kw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@oxc-project/types": "=0.151.0", + "@rolldown/pluginutils": "^1.0.0" + }, + "bin": { + "rolldown": "bin/cli.mjs" + }, + "engines": { + "node": "^20.19.0 || >=22.12.0" + }, + "optionalDependencies": { + "@rolldown/binding-android-arm-eabi": "1.2.11", + "@rolldown/binding-android-arm64": "1.2.11", + "@rolldown/binding-darwin-arm64": "1.2.11", + "@rolldown/binding-darwin-x64": "1.2.11", + "@rolldown/binding-freebsd-x64": "1.2.11", + "@rolldown/binding-linux-arm-gnueabihf": "1.2.11", + "@rolldown/binding-linux-arm64-gnu": "1.2.11", + "@rolldown/binding-linux-arm64-musl": "1.2.11", + "@rolldown/binding-linux-ppc64-gnu": "1.2.11", + "@rolldown/binding-linux-s390x-gnu": "1.2.11", + "@rolldown/binding-linux-x64-gnu": "1.2.11", + "@rolldown/binding-linux-x64-musl": "1.2.11", + "@rolldown/binding-openharmony-arm64": "1.2.11", + "@rolldown/binding-win32-arm64-msvc": "1.2.11", + "@rolldown/binding-win32-x64-msvc": "1.2.11" + } + }, "node_modules/rollup": { "version": "4.53.3", "resolved": "https://registry.npmjs.org/rollup/-/rollup-4.53.3.tgz", @@ -13960,6 +14805,13 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/siginfo": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/siginfo/-/siginfo-2.0.0.tgz", + "integrity": "sha512-ybx0WO1/8bSBLEWXZvEd7gMW3Sn3JFlW3TvX1nREbDLRNQNaeNN8WK0meBwPdAaOI7TtRRRJn/Es1zhrrCHu7g==", + "dev": true, + "license": "ISC" + }, "node_modules/signal-exit": { "version": "3.0.7", "resolved": "https://registry.npmjs.org/signal-exit/-/signal-exit-3.0.7.tgz", @@ -14204,6 +15056,16 @@ "node": ">= 12" } }, + "node_modules/source-map-js": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz", + "integrity": "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==", + "dev": true, + "license": "BSD-3-Clause", + "engines": { + "node": ">=0.10.0" + } + }, "node_modules/split-on-first": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/split-on-first/-/split-on-first-1.1.0.tgz", @@ -14222,6 +15084,13 @@ "node": ">= 10.x" } }, + "node_modules/stackback": { + "version": "0.0.2", + "resolved": "https://registry.npmjs.org/stackback/-/stackback-0.0.2.tgz", + "integrity": "sha512-1XMJE5fQo1jGH6Y/7ebnwPOBEkIEnT4QF32d5R1+VXdXveM0IBMJt8zfaxX1P3QhVwrYe+576+jkANtSS2mBbw==", + "dev": true, + "license": "MIT" + }, "node_modules/statuses": { "version": "2.0.1", "resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.1.tgz", @@ -14231,6 +15100,13 @@ "node": ">= 0.8" } }, + "node_modules/std-env": { + "version": "4.2.0", + "resolved": "https://registry.npmjs.org/std-env/-/std-env-4.2.0.tgz", + "integrity": "sha512-oCUKSupKTHX53EyjDtuZQ64pjLJ6yYCtpmEw0goYxtjG9KpbRe8KAsl2tBUGU9DyMcJ0RwJ8GqJAFzMXcXW1Rw==", + "dev": true, + "license": "MIT" + }, "node_modules/stop-iteration-iterator": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/stop-iteration-iterator/-/stop-iteration-iterator-1.1.0.tgz", @@ -14609,6 +15485,13 @@ "readable-stream": "3" } }, + "node_modules/tinybench": { + "version": "2.9.0", + "resolved": "https://registry.npmjs.org/tinybench/-/tinybench-2.9.0.tgz", + "integrity": "sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg==", + "dev": true, + "license": "MIT" + }, "node_modules/tinycolor2": { "version": "1.6.0", "resolved": "https://registry.npmjs.org/tinycolor2/-/tinycolor2-1.6.0.tgz", @@ -14625,13 +15508,13 @@ } }, "node_modules/tinyglobby": { - "version": "0.2.15", - "resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.15.tgz", - "integrity": "sha512-j2Zq4NyQYG5XMST4cbs02Ak8iJUdxRM0XI5QyxXuZOzKOINmWurp3smXu3y5wDcJrptwpSjgXHzIQxR0omXljQ==", + "version": "0.2.17", + "resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.17.tgz", + "integrity": "sha512-wXR/dYpcqKmfWpEdZjiKJOwCNFndD0DMnrW/cYjVGttEkBfVgcLFHoNrlj47mjOVic9yyNu65alsgF4NQyTa2g==", "license": "MIT", "dependencies": { "fdir": "^6.5.0", - "picomatch": "^4.0.3" + "picomatch": "^4.0.4" }, "engines": { "node": ">=12.0.0" @@ -14658,9 +15541,9 @@ } }, "node_modules/tinyglobby/node_modules/picomatch": { - "version": "4.0.3", - "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz", - "integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==", + "version": "4.0.7", + "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.7.tgz", + "integrity": "sha512-qcJu88Q2IWqJsDD529JKMdwGm/dvInW4HvQnRwiH9JtihJvzGOscDtHE3x1pBKeUOTysQ8kVmLnJ2kJu7yhcGA==", "license": "MIT", "engines": { "node": ">=12" @@ -14669,6 +15552,16 @@ "url": "https://github.com/sponsors/jonschlinkert" } }, + "node_modules/tinyrainbow": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/tinyrainbow/-/tinyrainbow-3.1.1.tgz", + "integrity": "sha512-yau8yJdTt989Mm0Bd/236QnzEiPf2xLLTqUZRUJOo/3CB078LSwzei343DgtJVmfJKJE3TMINY1u42SQsP6mXw==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=14.0.0" + } + }, "node_modules/tmp": { "version": "0.0.33", "resolved": "https://registry.npmjs.org/tmp/-/tmp-0.0.33.tgz", @@ -14849,7 +15742,7 @@ "version": "4.20.6", "resolved": "https://registry.npmjs.org/tsx/-/tsx-4.20.6.tgz", "integrity": "sha512-ytQKuwgmrrkDTFP4LjR0ToE2nqgy886GpvRSpU0JAnrdBYppuY5rLkRUYPU1yCryb24SsKBTL/hlDQAEFVwtZg==", - "devOptional": true, + "dev": true, "license": "MIT", "dependencies": { "esbuild": "~0.25.0", @@ -15311,7 +16204,7 @@ "version": "0.25.12", "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.25.12.tgz", "integrity": "sha512-bbPBYYrtZbkt6Os6FiTLCTFxvq4tt3JKall1vRwshA3fdVztsLAatFaZobhkBC8/BrPetoa0oksYoKXoG4ryJg==", - "devOptional": true, + "dev": true, "hasInstallScript": true, "license": "MIT", "bin": { @@ -15497,7 +16390,7 @@ "version": "5.9.3", "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", - "devOptional": true, + "dev": true, "license": "Apache-2.0", "bin": { "tsc": "bin/tsc", @@ -15699,6 +16592,200 @@ "node": ">= 0.8" } }, + "node_modules/vite": { + "version": "8.3.1", + "resolved": "https://registry.npmjs.org/vite/-/vite-8.3.1.tgz", + "integrity": "sha512-/bvH9E9tmCXRGp2uXY3WbOldqpTwFkbha/8ANaEQ6VkxhH60KyqLwgZq6lG2y+4uT55x9+9eUHMpQ7uGnOCKjA==", + "dev": true, + "license": "MIT", + "dependencies": { + "lightningcss": "^1.33.0", + "picomatch": "^4.0.7", + "postcss": "^8.5.28", + "rolldown": "~1.2.9", + "tinyglobby": "^0.2.17" + }, + "bin": { + "vite": "bin/vite.js" + }, + "engines": { + "node": "^20.19.0 || >=22.12.0" + }, + "funding": { + "url": "https://github.com/vitejs/vite?sponsor=1" + }, + "optionalDependencies": { + "fsevents": "~2.3.3" + }, + "peerDependencies": { + "@types/node": "^20.19.0 || >=22.12.0", + "@vitejs/devtools": "^0.7.1", + "esbuild": "^0.27.0 || ^0.28.0", + "jiti": ">=1.21.0", + "less": "^4.0.0", + "sass": "^1.70.0", + "sass-embedded": "^1.70.0", + "stylus": ">=0.54.8", + "sugarss": "^5.0.0", + "terser": "^5.16.0", + "tsx": "^4.8.1", + "yaml": "^2.4.2" + }, + "peerDependenciesMeta": { + "@types/node": { + "optional": true + }, + "@vitejs/devtools": { + "optional": true + }, + "esbuild": { + "optional": true + }, + "jiti": { + "optional": true + }, + "less": { + "optional": true + }, + "sass": { + "optional": true + }, + "sass-embedded": { + "optional": true + }, + "stylus": { + "optional": true + }, + "sugarss": { + "optional": true + }, + "terser": { + "optional": true + }, + "tsx": { + "optional": true + }, + "yaml": { + "optional": true + } + } + }, + "node_modules/vite/node_modules/picomatch": { + "version": "4.0.7", + "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.7.tgz", + "integrity": "sha512-qcJu88Q2IWqJsDD529JKMdwGm/dvInW4HvQnRwiH9JtihJvzGOscDtHE3x1pBKeUOTysQ8kVmLnJ2kJu7yhcGA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=12" + }, + "funding": { + "url": "https://github.com/sponsors/jonschlinkert" + } + }, + "node_modules/vitest": { + "version": "4.1.11", + "resolved": "https://registry.npmjs.org/vitest/-/vitest-4.1.11.tgz", + "integrity": "sha512-fhACrNXUidIbGSBr5FlbuBkO7VWC1ZyLl0DO4CU2DrQoAPxX84Ysxs+HeGQpii5lZWV1Q4gBZTTu49mF+A6Edw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vitest/expect": "4.1.11", + "@vitest/mocker": "4.1.11", + "@vitest/pretty-format": "4.1.11", + "@vitest/runner": "4.1.11", + "@vitest/snapshot": "4.1.11", + "@vitest/spy": "4.1.11", + "@vitest/utils": "4.1.11", + "es-module-lexer": "^2.0.0", + "expect-type": "^1.3.0", + "magic-string": "^0.30.21", + "obug": "^2.1.1", + "pathe": "^2.0.3", + "picomatch": "^4.0.3", + "std-env": "^4.0.0-rc.1", + "tinybench": "^2.9.0", + "tinyexec": "^1.0.2", + "tinyglobby": "^0.2.15", + "tinyrainbow": "^3.1.0", + "vite": "^6.0.0 || ^7.0.0 || ^8.0.0", + "why-is-node-running": "^2.3.0" + }, + "bin": { + "vitest": "vitest.mjs" + }, + "engines": { + "node": "^20.0.0 || ^22.0.0 || >=24.0.0" + }, + "funding": { + "url": "https://opencollective.com/vitest" + }, + "peerDependencies": { + "@edge-runtime/vm": "*", + "@opentelemetry/api": "^1.9.0", + "@types/node": "^20.0.0 || ^22.0.0 || >=24.0.0", + "@vitest/browser-playwright": "4.1.11", + "@vitest/browser-preview": "4.1.11", + "@vitest/browser-webdriverio": "4.1.11", + "@vitest/coverage-istanbul": "4.1.11", + "@vitest/coverage-v8": "4.1.11", + "@vitest/ui": "4.1.11", + "happy-dom": "*", + "jsdom": "*", + "vite": "^6.0.0 || ^7.0.0 || ^8.0.0" + }, + "peerDependenciesMeta": { + "@edge-runtime/vm": { + "optional": true + }, + "@opentelemetry/api": { + "optional": true + }, + "@types/node": { + "optional": true + }, + "@vitest/browser-playwright": { + "optional": true + }, + "@vitest/browser-preview": { + "optional": true + }, + "@vitest/browser-webdriverio": { + "optional": true + }, + "@vitest/coverage-istanbul": { + "optional": true + }, + "@vitest/coverage-v8": { + "optional": true + }, + "@vitest/ui": { + "optional": true + }, + "happy-dom": { + "optional": true + }, + "jsdom": { + "optional": true + }, + "vite": { + "optional": false + } + } + }, + "node_modules/vitest/node_modules/picomatch": { + "version": "4.0.7", + "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.7.tgz", + "integrity": "sha512-qcJu88Q2IWqJsDD529JKMdwGm/dvInW4HvQnRwiH9JtihJvzGOscDtHE3x1pBKeUOTysQ8kVmLnJ2kJu7yhcGA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=12" + }, + "funding": { + "url": "https://github.com/sponsors/jonschlinkert" + } + }, "node_modules/wcwidth": { "version": "1.0.1", "resolved": "https://registry.npmjs.org/wcwidth/-/wcwidth-1.0.1.tgz", @@ -15862,6 +16949,23 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/why-is-node-running": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/why-is-node-running/-/why-is-node-running-2.3.0.tgz", + "integrity": "sha512-hUrmaWBdVDcxvYqnyh09zunKzROWjbZTiNy8dBEjkS7ehEDQibXJ7XvlmtbwuTclUiIyN+CyXQD4Vmko8fNm8w==", + "dev": true, + "license": "MIT", + "dependencies": { + "siginfo": "^2.0.0", + "stackback": "0.0.2" + }, + "bin": { + "why-is-node-running": "cli.js" + }, + "engines": { + "node": ">=8" + } + }, "node_modules/win-guid": { "version": "0.2.1", "resolved": "https://registry.npmjs.org/win-guid/-/win-guid-0.2.1.tgz", @@ -16024,7 +17128,7 @@ "version": "2.8.1", "resolved": "https://registry.npmjs.org/yaml/-/yaml-2.8.1.tgz", "integrity": "sha512-lcYcMxX2PO9XMGvAJkJ3OsNMw+/7FKes7/hgerGUYWIoWu5j/+YQqcZr5JnPZWzOsEBgMbSbiSTn/dv/69Mkpw==", - "devOptional": true, + "dev": true, "license": "ISC", "bin": { "yaml": "bin.mjs" diff --git a/package.json b/package.json index 369296c3cd..b34e0134d5 100644 --- a/package.json +++ b/package.json @@ -9,7 +9,7 @@ "start": "tsx ./src/main.ts", "start:prod": "node dist/main", "dev:server": "tsx watch ./src/main.ts", - "test": "tsx watch ./test/all.test.ts", + "test": "vitest run", "lint": "eslint --fix --ext .ts src", "lint:check": "eslint --ext .ts src", "commit": "cz", @@ -20,7 +20,8 @@ "db:studio": "node runWithProvider.js \"npx prisma studio --schema ./prisma/DATABASE_PROVIDER-schema.prisma\"", "db:migrate:dev": "node runWithProvider.js \"rm -rf ./prisma/migrations && cp -r ./prisma/DATABASE_PROVIDER-migrations ./prisma/migrations && npx prisma migrate dev --schema ./prisma/DATABASE_PROVIDER-schema.prisma && cp -r ./prisma/migrations/* ./prisma/DATABASE_PROVIDER-migrations\"", "db:migrate:dev:win": "node runWithProvider.js \"xcopy /E /I prisma\\DATABASE_PROVIDER-migrations prisma\\migrations && npx prisma migrate dev --schema prisma\\DATABASE_PROVIDER-schema.prisma\"", - "prepare": "husky" + "prepare": "husky", + "test:watch": "vitest" }, "repository": { "type": "git", @@ -153,6 +154,7 @@ "prettier": "^3.4.2", "tsconfig-paths": "^4.2.0", "tsx": "^4.20.5", - "typescript": "^5.7.2" + "typescript": "^5.7.2", + "vitest": "^4.1.11" } } diff --git a/test/global-setup.ts b/test/global-setup.ts new file mode 100644 index 0000000000..41e473dd84 --- /dev/null +++ b/test/global-setup.ts @@ -0,0 +1,15 @@ +// Say which Baileys every run tested, and fail the run when it is not the one +// the job expected (BAILEYS_EXPECT), so a green run can never be a run against +// the wrong version. +import { readFileSync } from 'node:fs'; +import { createRequire } from 'node:module'; +import { dirname, join } from 'node:path'; + +export default function () { + const require = createRequire(import.meta.url); + const dir = process.env.BAILEYS_DIR ?? dirname(require.resolve('baileys/package.json')); + const { version } = JSON.parse(readFileSync(join(dir, 'package.json'), 'utf8')); + console.log(`[harness] baileys ${version} from ${dir}`); + const expected = process.env.BAILEYS_EXPECT; + if (expected && expected !== version) throw new Error(`BAILEYS_EXPECT=${expected}, but the tests resolve baileys ${version} (${dir})`); +} diff --git a/test/harness/baileys-version.test.ts b/test/harness/baileys-version.test.ts new file mode 100644 index 0000000000..26fca432cf --- /dev/null +++ b/test/harness/baileys-version.test.ts @@ -0,0 +1,22 @@ +// The harness tests the Baileys that package.json pins, and a deep import +// reaches the same build as the package itself. +import { readFileSync } from 'node:fs'; +import { join } from 'node:path'; + +import { describe, expect, it } from 'vitest'; + +const pinned = JSON.parse(readFileSync(join(__dirname, '../../package.json'), 'utf8')).dependencies.baileys; +const resolved = process.env.BAILEYS_RESOLVED_DIR!; +const version = JSON.parse(readFileSync(join(resolved, 'package.json'), 'utf8')).version; + +describe('harness: Baileys resolution', () => { + it.runIf(!process.env.BAILEYS_DIR)('tests the version package.json pins', () => { + expect(version).toBe(pinned); + }); + + it('resolves deep imports from the same build', async () => { + const deep = await import('baileys/lib/Utils/event-buffer.js' as any); + const top = await import('baileys'); + expect(deep.makeEventBuffer).toBe((top as any).makeEventBuffer); + }); +}); diff --git a/test/harness/service.test.ts b/test/harness/service.test.ts new file mode 100644 index 0000000000..fdfea1c2ab --- /dev/null +++ b/test/harness/service.test.ts @@ -0,0 +1,36 @@ +// The harness drives Evolution's real BaileysStartupService through Baileys' +// real event buffer, under a configuration profile. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { beforeEach, describe, expect, it } from 'vitest'; + +import { deliver, makeService } from '../helpers/baileys-service'; +import { emitted } from '../helpers/fake-server-module'; + +const incoming = { + key: { remoteJid: '972500000001@s.whatsapp.net', fromMe: false, id: '3EB0AAAAAAAAAAAAAAAA' }, + message: { conversation: 'hello' }, + messageTimestamp: 1_700_000_000, + pushName: 'Sender', +}; + +describe('harness: an incoming text message', () => { + beforeEach(() => void emitted.splice(0)); + + it('reaches the messages.upsert webhook, and is not stored under the minimal profile', async () => { + const { service, prisma, ev } = await makeService({ profile: 'minimal' }); + await deliver(service, ev, { 'messages.upsert': { messages: [incoming], type: 'notify' } }); + const upsert = emitted.find((e) => e.event === 'messages.upsert'); + expect(upsert?.data?.key?.id).toBe(incoming.key.id); + expect(upsert?.data?.message?.conversation).toBe('hello'); + expect(prisma.message.rows).toHaveLength(0); + }); + + it('is stored under the stored profile', async () => { + const { service, prisma, ev } = await makeService({ profile: 'stored' }); + await deliver(service, ev, { 'messages.upsert': { messages: [incoming], type: 'notify' } }); + expect(prisma.message.rows.map((r: any) => r.key?.id)).toEqual([incoming.key.id]); + }); +}); diff --git a/test/helpers/app-state.ts b/test/helpers/app-state.ts new file mode 100644 index 0000000000..476fd2948a --- /dev/null +++ b/test/helpers/app-state.ts @@ -0,0 +1,45 @@ +// Build a real encrypted app-state patch with Baileys' own encoder, and decode it +// with whatever key the code under test hands back. No WhatsApp involved. +import { randomBytes } from 'node:crypto'; + +import { decodePatches, encodeSyncdPatch, newLTHashState, proto } from 'baileys'; + +export function freshAppStateKey() { + const keyId = randomBytes(6).toString('base64'); + // As Baileys receives it in APP_STATE_SYNC_KEY_SHARE: a decoded proto message. + const key = proto.Message.AppStateSyncKeyData.decode( + proto.Message.AppStateSyncKeyData.encode({ + keyData: randomBytes(32), + fingerprint: { rawId: 1, currentIndex: 0, deviceIndexes: [0] }, + timestamp: Date.now(), + }).finish(), + ); + return { keyId, key }; +} + +/** A contact saved on the phone, as the phone syncs it to a linked device. */ +export async function contactPatch(keyId: string, key: any, jid: string, fullName: string) { + const { patch } = await encodeSyncdPatch( + { + type: 'critical_unblock_low', + index: ['contact', jid], + syncAction: { contactAction: { fullName } }, + apiVersion: 2, + operation: proto.SyncdMutation.SyncdOperation.SET, + } as any, + keyId, + newLTHashState(), + async () => key, + ); + return { ...patch, version: { version: 1 } }; +} + +/** Names Baileys recovers from the patch; an Error when it throws (rc9), [] when it skips (rc13+). */ +export async function decodeContactNames(patch: any, getKey: (id: string) => Promise) { + try { + const { mutationMap } = await decodePatches('critical_unblock_low', [patch], newLTHashState(), getKey, {}, 0); + return Object.values(mutationMap).map((m: any) => m.syncAction.value?.contactAction?.fullName); + } catch (error) { + return error as Error; + } +} diff --git a/test/helpers/baileys-fixtures.ts b/test/helpers/baileys-fixtures.ts new file mode 100644 index 0000000000..183dd26a03 --- /dev/null +++ b/test/helpers/baileys-fixtures.ts @@ -0,0 +1,30 @@ +// Fixtures are WhatsApp-side inputs (a HistorySync proto, an app-state sync +// action). The Baileys version under test turns them into events, so each +// version hands Evolution exactly what it would in production, no more. +import { processHistoryMessage, processSyncAction, proto } from 'baileys'; + +import { WUID } from './baileys-service'; + +/** What Baileys emits as messaging-history.set for this HistorySync payload. */ +export function historyEvent(sync: Record) { + const data: any = (processHistoryMessage as any)(proto.HistorySync.fromObject(sync)); + return { ...data, isLatest: true }; +} + +/** The events Baileys emits for one decoded app-state mutation. */ +export function syncActionEvents(index: string[], action: Record) { + const events: Record = {}; + const collector: any = { emit: (name: string, data: any) => void (events[name] = data) }; + const syncAction = { index, syncAction: { value: proto.SyncActionValue.fromObject({ timestamp: 1_700_000_000, ...action }) } }; + (processSyncAction as any)(syncAction, collector, { id: WUID, name: 'Me' }, undefined, undefined); + return events; +} + +export const msg = (remoteJid: string, id: string, text: string, extra: Record = {}) => ({ + message: { + key: { remoteJid, fromMe: false, id }, + message: { conversation: text }, + messageTimestamp: 1_700_000_000, + ...extra, + }, +}); diff --git a/test/helpers/baileys-service.ts b/test/helpers/baileys-service.ts new file mode 100644 index 0000000000..597f8581ce --- /dev/null +++ b/test/helpers/baileys-service.ts @@ -0,0 +1,58 @@ +// Build a BaileysStartupService with fakes around it, and a fake socket whose +// event emitter is the REAL Baileys event buffer, so eventHandler() receives +// batches the way it does in production. +import EventEmitter2 from 'eventemitter2'; +import { makeEventBuffer } from 'baileys'; +import P from 'pino'; + +import { fakePrisma } from './fake-prisma'; +import { applyProfile, type Profile } from './profiles'; + +export const WUID = '972500000000@s.whatsapp.net'; + +export async function makeService(opts: { profile?: Profile } = {}) { + applyProfile(opts.profile ?? 'minimal'); + const { BaileysStartupService } = await import('@api/integrations/channel/whatsapp/whatsapp.baileys.service'); + const { CacheService } = await import('@api/services/cache.service'); + const { LocalCache } = await import('@cache/localcache'); + const { ConfigService } = await import('@config/env.config'); + const configService = new ConfigService(); + const prisma = fakePrisma(); + const cache = new CacheService(new LocalCache(configService, 'instance')); + const baileysCache = new CacheService(new LocalCache(configService, 'baileys')); + const service: any = new BaileysStartupService(configService, new EventEmitter2(), prisma, cache, null as any, baileysCache, null as any); + service.setInstance({ instanceName: 'test', instanceId: 'inst-1', integration: 'WHATSAPP-BAILEYS' }); + const ev = makeEventBuffer(P({ level: 'silent' }) as any); + service.client = { + ev, + user: { id: WUID }, + profilePictureUrl: async () => undefined, + signalRepository: { lidMapping: { getPNForLID: async () => undefined, getLIDForPN: async () => undefined } }, + }; + service.instance.wuid = WUID; + // eventHandler() saves creds on every creds.update (Baileys emits one per history batch). + service.instance.authState = { saveCreds: async () => undefined }; + return { service, prisma, ev }; +} + +/** Wire eventHandler() to the fake socket and emit a batch through the real event buffer. */ +export async function deliver(service: any, ev: any, events: Record, opts: { buffered?: boolean } = {}) { + if (!service.__wired) { + service.eventHandler(); + service.__wired = true; + } + // Baileys emits history and most message events inside a buffered function + // (upsertMessage is ev.createBufferedFunction), so buffered is the production path. + if (opts.buffered !== false) ev.buffer(); + for (const [name, payload] of Object.entries(events)) ev.emit(name, payload); + if (opts.buffered !== false) await ev.flush(); + await settle(service); +} + +/** eventHandler() chains every batch on eventProcessingQueue; handlers fire async work after it. */ +export async function settle(service: any) { + for (let i = 0; i < 5; i++) { + await service.eventProcessingQueue; + await new Promise((r) => setTimeout(r, 5)); + } +} diff --git a/test/helpers/capture-output.ts b/test/helpers/capture-output.ts new file mode 100644 index 0000000000..0d580fc061 --- /dev/null +++ b/test/helpers/capture-output.ts @@ -0,0 +1,28 @@ +// Everything the process prints while `fn` runs: console.* and raw stdout/stderr. +export async function captureOutput(fn: () => Promise) { + const out: string[] = []; + const fmt = (args: any[]) => args.map((a) => (typeof a === 'string' ? a : safe(a))).join(' '); + const names = ['log', 'info', 'warn', 'error', 'debug'] as const; + const saved = names.map((n) => console[n]); + const stdout = process.stdout.write.bind(process.stdout); + const stderr = process.stderr.write.bind(process.stderr); + names.forEach((n) => (console[n] = (...a: any[]) => void out.push(fmt(a)))); + (process.stdout as any).write = (c: any) => (out.push(String(c)), true); + (process.stderr as any).write = (c: any) => (out.push(String(c)), true); + try { + await fn(); + } finally { + names.forEach((n, i) => (console[n] = saved[i])); + (process.stdout as any).write = stdout; + (process.stderr as any).write = stderr; + } + return out.join('\n'); +} + +function safe(v: any) { + try { + return JSON.stringify(v, (_k, x) => (typeof x === 'bigint' ? String(x) : x)); + } catch { + return String(v); + } +} diff --git a/test/helpers/fake-prisma.ts b/test/helpers/fake-prisma.ts new file mode 100644 index 0000000000..be03a1bc15 --- /dev/null +++ b/test/helpers/fake-prisma.ts @@ -0,0 +1,85 @@ +// An in-memory stand-in for the parts of PrismaRepository the Baileys handlers use. +// Rows live in plain arrays so a test can assert on what Evolution stored. +type Row = Record; + +const matches = (row: Row, where: Row = {}) => + Object.entries(where).every(([k, v]) => { + if (k === 'remoteJid_instanceId') return row.remoteJid === v.remoteJid && row.instanceId === v.instanceId; + if (v && typeof v === 'object' && !Array.isArray(v)) { + if ('in' in v) return (v.in as any[]).includes(row[k]); + if ('path' in v) return true; // JSON path filters: not modelled, treated as match + return matches(row[k] ?? {}, v); + } + return row[k] === v; + }); + +function table(name: string, uniqueKey: (r: Row) => string | undefined) { + const rows: Row[] = []; + const find = (where?: Row) => rows.filter((r) => matches(r, where)); + const t = { + rows, + findMany: async (args: Row = {}) => find(args.where), + findFirst: async (args: Row = {}) => find(args.where)[0] ?? null, + findUnique: async (args: Row = {}) => find(args.where)[0] ?? null, + count: async (args: Row = {}) => find(args.where).length, + create: async ({ data }: Row) => (rows.push({ id: `${name}-${rows.length + 1}`, ...data }), rows[rows.length - 1]), + createMany: async ({ data, skipDuplicates }: Row) => { + let count = 0; + for (const d of [].concat(data)) { + const key = uniqueKey(d); + if (skipDuplicates && key && rows.some((r) => uniqueKey(r) === key)) continue; + rows.push({ id: `${name}-${rows.length + 1}`, ...d }); + count++; + } + return { count }; + }, + update: async ({ where, data }: Row) => { + const r = find(where)[0]; + if (r) Object.assign(r, strip(data)); + return r; + }, + updateMany: async ({ where, data }: Row) => { + const hit = find(where); + hit.forEach((r) => Object.assign(r, strip(data))); + return { count: hit.length }; + }, + upsert: async ({ where, create, update }: Row) => { + const r = find(where)[0]; + if (r) return Object.assign(r, strip(update)); + return t.create({ data: create }); + }, + delete: async ({ where }: Row) => { + const i = rows.findIndex((r) => matches(r, where)); + return i >= 0 ? rows.splice(i, 1)[0] : null; + }, + deleteMany: async ({ where }: Row = {}) => { + const hit = find(where); + hit.forEach((r) => rows.splice(rows.indexOf(r), 1)); + return { count: hit.length }; + }, + }; + return t; +} + +// Prisma ignores undefined fields in an update; so do we. +const strip = (data: Row) => Object.fromEntries(Object.entries(data ?? {}).filter(([, v]) => v !== undefined)); + +export function fakePrisma() { + const byJid = (r: Row) => (r.remoteJid ? `${r.remoteJid}|${r.instanceId}` : undefined); + const db: any = { + contact: table('contact', byJid), + chat: table('chat', byJid), + message: table('message', (r) => r.key?.id && `${r.key.id}|${r.instanceId}`), + messageUpdate: table('messageUpdate', () => undefined), + setting: table('setting', (r) => r.instanceId), + session: table('session', (r) => r.sessionId), + label: table('label', () => undefined), + isOnWhatsapp: table('isOnWhatsapp', (r) => r.remoteJid), + instance: table('instance', (r) => r.id), + }; + db.$transaction = async (ops: any) => (typeof ops === 'function' ? ops(db) : Promise.all(ops)); + db.$queryRaw = async () => []; + db.$executeRaw = async () => 0; + db.$queryRawUnsafe = async () => []; + return db; +} diff --git a/test/helpers/fake-server-module.ts b/test/helpers/fake-server-module.ts new file mode 100644 index 0000000000..069da72f3c --- /dev/null +++ b/test/helpers/fake-server-module.ts @@ -0,0 +1,14 @@ +// Replaces @api/server.module, which builds the whole application at import +// (Prisma client, monitor, every controller). Handlers reach three things +// through it: eventManager (webhooks and other transports), chatbotController +// and waMonitor. Tests read what was emitted from `emitted`. +import { vi } from 'vitest'; + +import { fakePrisma } from './fake-prisma'; + +export const emitted: { event: string; data: any; extra?: any }[] = []; +export const eventManager = { emit: vi.fn(async (e: any) => void emitted.push({ event: e.event, data: e.data, extra: e.extra })) }; +export const chatbotController = { emit: vi.fn(async () => undefined) }; +export const waMonitor = { waInstances: {} }; +export const prismaRepository = fakePrisma(); +export const cache = undefined; diff --git a/test/helpers/http-app.ts b/test/helpers/http-app.ts new file mode 100644 index 0000000000..5c96b3df48 --- /dev/null +++ b/test/helpers/http-app.ts @@ -0,0 +1,20 @@ +// T3: Evolution's real router and real Prisma against a throwaway Postgres. +// main.ts builds the app inside bootstrap() and does not export it, so this +// mirrors the parts a route needs: JSON body parsing, the router, and the +// error handler's status mapping (main.ts, "app.use((err, req, res, next) ..."). +import type { AddressInfo } from 'node:net'; + +export async function startApp() { + const express = (await import('express')).default; + const { router } = await import('@api/routes/index.router'); + const app = express(); + app.use(express.json({ limit: '10mb' })); + app.use('/', router); + app.use((err: any, _req: any, res: any, _next: any) => + res.status(err?.status || 500).json({ status: err?.status || 500, error: err?.error, response: { message: err?.message } }), + ); + const server = app.listen(0, '127.0.0.1'); + await new Promise((r) => server.once('listening', r)); + const base = `http://127.0.0.1:${(server.address() as AddressInfo).port}`; + return { base, close: () => new Promise((r) => server.close(r)) }; +} diff --git a/test/helpers/profiles.ts b/test/helpers/profiles.ts new file mode 100644 index 0000000000..9f6dd66a1f --- /dev/null +++ b/test/helpers/profiles.ts @@ -0,0 +1,48 @@ +// Configuration profiles. Evolution's behaviour branches on these flags, so a +// test that only ever runs with storage on says nothing about a deployment +// that has it off, and the reverse. +// +// `minimal` is a production configuration that keeps its own data outside +// Evolution: only the instance is stored, local cache, no Redis, no S3, WARN and above. +// `stored` turns on everything Evolution can store, as a default install does. +export const PROFILES = { + minimal: { + DATABASE_SAVE_DATA_INSTANCE: 'true', + DATABASE_SAVE_DATA_NEW_MESSAGE: 'false', + DATABASE_SAVE_MESSAGE_UPDATE: 'false', + DATABASE_SAVE_DATA_CONTACTS: 'false', + DATABASE_SAVE_DATA_CHATS: 'false', + DATABASE_SAVE_DATA_LABELS: 'false', + DATABASE_SAVE_DATA_HISTORIC: 'false', + DATABASE_SAVE_IS_ON_WHATSAPP: 'false', + DATABASE_DELETE_MESSAGE: 'true', + CACHE_REDIS_ENABLED: 'false', + CACHE_LOCAL_ENABLED: 'true', + S3_ENABLED: 'false', + LOG_LEVEL: 'ERROR,WARN', + LOG_BAILEYS: 'error', + }, + stored: { + DATABASE_SAVE_DATA_INSTANCE: 'true', + DATABASE_SAVE_DATA_NEW_MESSAGE: 'true', + DATABASE_SAVE_MESSAGE_UPDATE: 'true', + DATABASE_SAVE_DATA_CONTACTS: 'true', + DATABASE_SAVE_DATA_CHATS: 'true', + DATABASE_SAVE_DATA_LABELS: 'true', + DATABASE_SAVE_DATA_HISTORIC: 'true', + DATABASE_SAVE_IS_ON_WHATSAPP: 'true', + DATABASE_DELETE_MESSAGE: 'true', + CACHE_REDIS_ENABLED: 'false', + CACHE_LOCAL_ENABLED: 'true', + S3_ENABLED: 'false', + LOG_LEVEL: 'ERROR,WARN', + LOG_BAILEYS: 'error', + }, +} as const; + +export type Profile = keyof typeof PROFILES; + +/** Set every flag of the profile, so nothing leaks from the previous test. */ +export function applyProfile(profile: Profile) { + Object.assign(process.env, PROFILES[profile]); +} diff --git a/test/helpers/socket-history.ts b/test/helpers/socket-history.ts new file mode 100644 index 0000000000..554c568a51 --- /dev/null +++ b/test/helpers/socket-history.ts @@ -0,0 +1,65 @@ +// History exactly as the socket delivers it at link time: Baileys' own +// processMessage handles a HISTORY_SYNC_NOTIFICATION (inline payload, no download) +// inside ev.createBufferedFunction, as Socket/chats.js upsertMessage does, with a REAL +// signal repository. So the event Evolution sees is whatever this Baileys version +// really emits (rc13+: mappings stored in the repository, then dropped by the buffer). +import { deflateSync } from 'node:zlib'; + +import * as B from 'baileys'; +import P from 'pino'; + +const logger: any = P({ level: 'silent' }); + +export async function realSignalRepository() { + const { makeLibSignalRepository } = await import('baileys/lib/Signal/libsignal.js' as any); + const mem: Record> = {}; + const store = { + get: async (type: string, ids: string[]) => Object.fromEntries(ids.map((id) => [id, mem[type]?.[id]])), + set: async (data: any) => { + for (const t in data) + for (const id in data[t]) { + mem[t] ??= {}; + data[t][id] == null ? delete mem[t][id] : (mem[t][id] = data[t][id]); + } + }, + }; + const keys = (B as any).addTransactionCapability((B as any).makeCacheableSignalKeyStore(store, logger), logger, { + maxCommitRetries: 1, + delayBetweenTriesMs: 1, + }); + const creds = (B as any).initAuthCreds(); + return { repo: makeLibSignalRepository({ creds, keys }, logger, async () => undefined), keys, creds }; +} + +export async function socketHistory(ev: any, client: any, sync: Record) { + const { default: processMessage } = await import('baileys/lib/Utils/process-message.js' as any); + const { proto } = B as any; + const hs = proto.HistorySync.fromObject(sync); + const inline = deflateSync(Buffer.from(proto.HistorySync.encode(hs).finish())); + const msg = { + key: { remoteJid: '972500000000@s.whatsapp.net', fromMe: true, id: 'HS1' }, + messageTimestamp: 1_700_000_001, + message: { + protocolMessage: { + type: proto.Message.ProtocolMessage.Type.HISTORY_SYNC_NOTIFICATION, + historySyncNotification: { syncType: hs.syncType, chunkOrder: 1, progress: 100, initialHistBootstrapInlinePayload: inline }, + }, + }, + }; + const ctx = { + shouldProcessHistoryMsg: true, + ev, + logger, + options: {}, + placeholderResendCache: undefined, + getMessage: async () => undefined, + creds: { ...client.__creds, me: { id: '972500000000:3@s.whatsapp.net', lid: '999999999999999:3@lid', name: 'Me' }, processedHistoryMessages: [] }, + keyStore: client.__keys, + signalRepository: client.signalRepository, + }; + await ev.createBufferedFunction(async () => { + await processMessage(msg, ctx); + })(); + // createBufferedFunction flushes on a 100ms timer when it is the only buffer. + await new Promise((r) => setTimeout(r, 250)); +} diff --git a/test/setup.ts b/test/setup.ts new file mode 100644 index 0000000000..3ca8318bb5 --- /dev/null +++ b/test/setup.ts @@ -0,0 +1,9 @@ +// Every test starts from the `minimal` production configuration (test/helpers/profiles.ts). +// A test that needs Evolution to store data asks for the `stored` profile. +import { applyProfile } from './helpers/profiles'; + +process.env.DATABASE_PROVIDER ??= 'postgresql'; +process.env.DATABASE_CONNECTION_URI ??= 'postgresql://unused:unused@127.0.0.1:1/unused'; +process.env.CHATWOOT_ENABLED ??= 'false'; +process.env.TELEMETRY_ENABLED ??= 'false'; +applyProfile('minimal'); diff --git a/vitest.config.mts b/vitest.config.mts new file mode 100644 index 0000000000..b330774f15 --- /dev/null +++ b/vitest.config.mts @@ -0,0 +1,38 @@ +// Evolution's own TypeScript source, run by vitest, against the Baileys that +// npm installed (package.json pins it). BAILEYS_DIR points the same tests at +// another Baileys build, for trying a newer release before bumping the pin. +import { createRequire } from 'node:module'; +import { dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { defineConfig } from 'vitest/config'; + +const root = fileURLToPath(new URL('.', import.meta.url)); +const require = createRequire(import.meta.url); +const baileysDir = process.env.BAILEYS_DIR ?? dirname(require.resolve('baileys/package.json')); +process.env.BAILEYS_RESOLVED_DIR = baileysDir; + +export default defineConfig({ + resolve: { + // Both the package and deep imports (baileys/lib/...) resolve to ONE directory, + // so a test can never mix two Baileys versions. + alias: [ + { find: /^baileys$/, replacement: `${baileysDir}/lib/index.js` }, + { find: /^baileys\/(.*)$/, replacement: `${baileysDir}/$1` }, + { find: /^@api\/(.*)$/, replacement: `${root}src/api/$1` }, + { find: /^@cache\/(.*)$/, replacement: `${root}src/cache/$1` }, + { find: /^@config\/(.*)$/, replacement: `${root}src/config/$1` }, + { find: /^@exceptions$/, replacement: `${root}src/exceptions` }, + { find: /^@libs\/(.*)$/, replacement: `${root}src/libs/$1` }, + { find: /^@utils\/(.*)$/, replacement: `${root}src/utils/$1` }, + { find: /^@validate\/(.*)$/, replacement: `${root}src/validate/$1` }, + ], + }, + test: { + include: ['test/**/*.test.ts'], + globalSetup: ['test/global-setup.ts'], + setupFiles: ['test/setup.ts'], + environment: 'node', + pool: 'forks', + testTimeout: 20000, + }, +}); From c5d815b4ce9b964f4e2a8e3e6ac53dfa9f7e5338 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 13:57:52 +0300 Subject: [PATCH 004/157] ci: run the fork's tests on every commit Typecheck and the vitest suite on the pinned Baileys, on every push to a fork/ branch and on pull requests. A weekly job runs the same suite against the newest Baileys release. The upstream workflows are left as they are: none of them triggers on fork/ branches. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/fork-tests.yml | 57 ++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 .github/workflows/fork-tests.yml diff --git a/.github/workflows/fork-tests.yml b/.github/workflows/fork-tests.yml new file mode 100644 index 0000000000..e99183f39b --- /dev/null +++ b/.github/workflows/fork-tests.yml @@ -0,0 +1,57 @@ +# The fork's tests. Every commit on a fork/ branch runs them, so a test: +# commit shows its red run and the fix: commit after it shows green (FORK.md). +name: fork tests + +on: + push: + branches: ['fork/**'] + pull_request: + schedule: + # Weekly, against the newest Baileys release, so a new version is a + # diff to read before anyone bumps the pin. + - cron: '0 6 * * 1' + workflow_dispatch: + +permissions: + contents: read + +jobs: + test: + name: typecheck and tests (pinned Baileys) + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: 24 + cache: npm + - run: npm ci + - run: npm run db:generate + env: + DATABASE_PROVIDER: postgresql + - run: npx tsc --noEmit + - name: tests + run: | + expected=$(node -p "require('./package.json').dependencies.baileys") + BAILEYS_EXPECT="$expected" npx vitest run + + baileys-latest: + name: tests against the newest Baileys + if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: 24 + cache: npm + - run: npm ci + - run: npm install --no-save baileys@latest + - run: npm run db:generate + env: + DATABASE_PROVIDER: postgresql + - name: tests + run: | + installed=$(node -p "require('baileys/package.json').version") + echo "Testing baileys $installed" + BAILEYS_EXPECT="$installed" npx vitest run From fa3d836644151116af996306a9f619a649ab6434 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 13:58:07 +0300 Subject: [PATCH 005/157] docs(fork): the working rules, test first AGENTS.md gains a first section for this fork: every change starts with a failing test committed alone, then the fix; how the harness runs; profiles; what to assert. CLAUDE.md points at it. These are this fork's rules, not Evolution's. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 41 +++++++++++++++++++++++++++++++++++++++++ CLAUDE.md | 2 ++ 2 files changed, 43 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 143e6c748e..35a4c0b47d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,3 +1,44 @@ +# Working in this fork (read first) + +This is an unofficial fork of Evolution API 2.3.7 (`FORK.md`). +These rules are this fork's own standard. They are not Evolution's, and they +do not go into pull requests offered to evolution-foundation/evolution-api. + +## Test first, always: red, then green + +1. Write the test that reproduces the bug or specifies the behaviour, and run + it against the code as it is. It must FAIL, for the reason the bug gives. + A test that passes before the fix proves nothing: rewrite it. +2. Commit the test alone: `test: `. CI runs on that commit, + and its red run is the evidence. +3. Make the smallest change that turns it green, with the rest of the suite + still green, and commit it: `fix: ...` or `feat: ...`. Push the two commits + one at a time, so CI records each. + +Never change a test to make it pass, and never skip or delete a failing test +without saying why in the commit. + +## How the tests run + +- `npm test` runs vitest over `test/**/*.test.ts`. The tests run Evolution's + TypeScript source, not the bundle, against the Baileys that `package.json` + pins (`BAILEYS_DIR` points them at another build). +- A fixture is a WhatsApp-side input (a `HistorySync` proto, an app-state + action, an encrypted patch), turned into events by the Baileys version under + test. Hand-written events are allowed only where Baileys has no builder. +- `test/helpers/baileys-service.ts` builds the real `BaileysStartupService` + with an in-memory Prisma and the real Baileys event buffer; `deliver()` + sends a batch the way the socket does (buffered). +- Every test runs under a configuration profile (`test/helpers/profiles.ts`). + The default is `minimal`, a production configuration that stores only the instance; a test + of storage uses `stored`. A behaviour that depends on a flag is tested + under both. +- Assert what a consumer observes: the webhook payload, the stored row, the + socket call, the HTTP answer. Assert exact fields, not substrings. +- No test touches WhatsApp, a real account or the network beyond localhost. + +--- + # Evolution API - AI Agent Guidelines This document provides comprehensive guidelines for AI agents (Claude, GPT, Cursor, etc.) working with the Evolution API codebase. diff --git a/CLAUDE.md b/CLAUDE.md index 7c0e87a040..9e073c2181 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,3 +1,5 @@ +**This is a fork: read the first section of `AGENTS.md` (test first, red then green) before changing anything.** + # CLAUDE.md This file provides comprehensive guidance to Claude AI when working with the Evolution API codebase. From eb94ff73752a0892c71ed8cdde62e738c757ff0c Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:02:20 +0300 Subject: [PATCH 006/157] test: app-state sync keys survive a reload in every auth store Each of the three auth stores (prisma, redis-db, provider-files) saves a real app-state sync key, is opened again as after a restart, and decodes a real encrypted contact patch with the key it reads back. Fails today: the stores revive the key with AppStateSyncKeyData.create(), which leaves keyData a base64 string, so the patch is skipped and no name comes back. Co-Authored-By: Claude Opus 5.5 --- test/unit/auth-state-app-state-key.test.ts | 112 +++++++++++++++++++++ 1 file changed, 112 insertions(+) create mode 100644 test/unit/auth-state-app-state-key.test.ts diff --git a/test/unit/auth-state-app-state-key.test.ts b/test/unit/auth-state-app-state-key.test.ts new file mode 100644 index 0000000000..9dd037139a --- /dev/null +++ b/test/unit/auth-state-app-state-key.test.ts @@ -0,0 +1,112 @@ +// Every auth-state store Evolution ships revives an app-state sync key with +// AppStateSyncKeyData.create(). After a JSON round trip (a restart, or any read +// back from storage) the key's bytes are a base64 STRING, because JSON.stringify +// calls the proto's toJSON before BufferJSON.replacer sees it. create() keeps the +// string; fromObject() turns it back into bytes (Baileys' own store, +// lib/Utils/use-multi-file-auth-state.js). With the string, Baileys derives the +// wrong keys and skips every app-state patch: saved contact names, labels, mutes, +// archives, all silently gone after the first restart. +// +// Each test saves a real key through the real store, opens the store AGAIN (as a +// restart does) and decodes a real encrypted contact patch with the key it reads. +import { rmSync } from 'node:fs'; +import { createServer } from 'node:http'; +import type { AddressInfo } from 'node:net'; + +import { afterAll, describe, expect, it, vi } from 'vitest'; + +import { contactPatch, decodeContactNames, freshAppStateKey } from '../helpers/app-state'; + +// The Prisma store keeps creds in the session table (the fake Prisma's) and, with +// Redis off, keys in files under INSTANCE_DIR (a temp dir here, not the repo). +const tmp = await vi.hoisted(async () => { + const { mkdtempSync } = await import('node:fs'); + const { tmpdir } = await import('node:os'); + const { join } = await import('node:path'); + return mkdtempSync(join(tmpdir(), 'evo-auth-')); +}); +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('@config/path.config', async (importOriginal) => ({ ...(await importOriginal()), INSTANCE_DIR: tmp })); + +const JID = '972500000001@s.whatsapp.net'; +const NAME = 'Dana Levi'; + +type Open = () => Promise<{ keys: any }>; + +/** Save a key through one opening of the store, decode a patch with what a second opening reads. */ +async function saveReloadDecode(open: Open) { + const { keyId, key } = freshAppStateKey(); + const patch = await contactPatch(keyId, key, JID, NAME); + await (await open()).keys.set({ 'app-state-sync-key': { [keyId]: key } }); + const reopened = await open(); + const getKey = async (id: string) => (await reopened.keys.get('app-state-sync-key', [id]))[id]; + return decodeContactNames(patch, getKey); +} + +afterAll(() => rmSync(tmp, { recursive: true, force: true })); + +describe('an app-state sync key survives a reload in every auth store', () => { + it('prisma store (creds in the session table, keys in local files)', async () => { + const { default: useMultiFileAuthStatePrisma } = await import('@utils/use-multi-file-auth-state-prisma'); + const open = async () => (await useMultiFileAuthStatePrisma('prisma-session', null as any)).state; + expect(await saveReloadDecode(open)).toEqual([NAME]); + }); + + it('redis-db store (over Evolution cache engine, which serialises as Redis does)', async () => { + const { CacheService } = await import('@api/services/cache.service'); + const { LocalCache } = await import('@cache/localcache'); + const { ConfigService } = await import('@config/env.config'); + const { useMultiFileAuthStateRedisDb } = await import('@utils/use-multi-file-auth-state-redis-db'); + const configService = new ConfigService(); + // A new CacheService per opening, over the same engine, as after a restart with Redis. + const open = async () => + (await useMultiFileAuthStateRedisDb('redis-session', new CacheService(new LocalCache(configService, 'auth-test')))) + .state; + expect(await saveReloadDecode(open)).toEqual([NAME]); + }); + + it('provider-files store (over a local file server that stores what it is sent)', async () => { + // The file provider stores the posted `data` (a JSON string) and serves it back as JSON. + const files = new Map(); + const server = createServer((req, res) => { + let body = ''; + req.on('data', (c) => (body += c)); + req.on('end', () => { + const path = req.url ?? ''; + if (req.method === 'POST') { + const parsed = body ? JSON.parse(body) : {}; + if (typeof parsed.data === 'string') files.set(path, parsed.data); + res.setHeader('content-type', 'application/json'); + return res.end('{}'); + } + if (req.method === 'GET' && files.has(path)) { + res.setHeader('content-type', 'application/json'); + return res.end(files.get(path)); + } + if (req.method === 'DELETE') return files.delete(path), res.end('{}'); + res.statusCode = 404; + res.end('{}'); + }); + }); + server.listen(0, '127.0.0.1'); + await new Promise((r) => server.once('listening', r)); + try { + Object.assign(process.env, { + PROVIDER_ENABLED: 'true', + PROVIDER_HOST: '127.0.0.1', + PROVIDER_PORT: String((server.address() as AddressInfo).port), + PROVIDER_PREFIX: 'test', + }); + const { ConfigService } = await import('@config/env.config'); + const { ProviderFiles } = await import('@api/provider/sessions'); + const { AuthStateProvider } = await import('@utils/use-multi-file-auth-state-provider-files'); + const open = async () => + (await new AuthStateProvider(new ProviderFiles(new ConfigService())).authStateProvider('provider-session')).state; + expect(await saveReloadDecode(open)).toEqual([NAME]); + expect([...files.keys()].some((k) => k.includes('app-state-sync-key-'))).toBe(true); + } finally { + for (const k of ['PROVIDER_ENABLED', 'PROVIDER_HOST', 'PROVIDER_PORT', 'PROVIDER_PREFIX']) delete process.env[k]; + await new Promise((r) => server.close(r)); + } + }); +}); From 4ffaf1e2fac5323e7828a1f129b1e32ca2195be5 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:02:34 +0300 Subject: [PATCH 007/157] fix(auth-state): reload app-state sync keys with fromObject After a JSON round trip an app-state sync key's keyData is a base64 string. AppStateSyncKeyData.create() keeps it a string, so Baileys derives the wrong keys and skips every app-state patch after a restart. fromObject() decodes it back to bytes, as Baileys' own multi-file store does. Applied to the prisma, redis-db and provider-files stores. Co-Authored-By: Claude Opus 5.5 --- src/utils/use-multi-file-auth-state-prisma.ts | 2 +- src/utils/use-multi-file-auth-state-provider-files.ts | 2 +- src/utils/use-multi-file-auth-state-redis-db.ts | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/utils/use-multi-file-auth-state-prisma.ts b/src/utils/use-multi-file-auth-state-prisma.ts index d090780234..bd11afa2fe 100644 --- a/src/utils/use-multi-file-auth-state-prisma.ts +++ b/src/utils/use-multi-file-auth-state-prisma.ts @@ -182,7 +182,7 @@ export default async function useMultiFileAuthStatePrisma( ids.map(async (id) => { let value = await readData(`${type}-${id}`); if (type === 'app-state-sync-key' && value) { - value = proto.Message.AppStateSyncKeyData.create(value); + value = proto.Message.AppStateSyncKeyData.fromObject(value); } data[id] = value; diff --git a/src/utils/use-multi-file-auth-state-provider-files.ts b/src/utils/use-multi-file-auth-state-provider-files.ts index eecc3100e5..157918169d 100644 --- a/src/utils/use-multi-file-auth-state-provider-files.ts +++ b/src/utils/use-multi-file-auth-state-provider-files.ts @@ -116,7 +116,7 @@ export class AuthStateProvider { ids.map(async (id) => { let value = await readData(`${type}-${id}`); if (type === 'app-state-sync-key' && value) { - value = proto.Message.AppStateSyncKeyData.create(value); + value = proto.Message.AppStateSyncKeyData.fromObject(value); } data[id] = value; diff --git a/src/utils/use-multi-file-auth-state-redis-db.ts b/src/utils/use-multi-file-auth-state-redis-db.ts index e0981c700c..4d5d5a5457 100644 --- a/src/utils/use-multi-file-auth-state-redis-db.ts +++ b/src/utils/use-multi-file-auth-state-redis-db.ts @@ -61,7 +61,7 @@ export async function useMultiFileAuthStateRedisDb( ids.map(async (id) => { let value = await readData(`${type}-${id}`); if (type === 'app-state-sync-key' && value) { - value = proto.Message.AppStateSyncKeyData.create(value); + value = proto.Message.AppStateSyncKeyData.fromObject(value); } data[id] = value; From 22b0eda2d920fbdf45f6856c00cc51fa0f62e9b1 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:04:47 +0300 Subject: [PATCH 008/157] test: a webhook subscribed to GROUP_UPDATE receives group updates Baileys emits groups.update, which every transport upper-cases to GROUPS_UPDATE before comparing it with the subscription. /webhook/set (and every //set) accepts only GROUP_UPDATE, and the global env config is keyed GROUP_UPDATE, so a subscription to group updates never matches. Through the real WebhookController to a local HTTP destination: subscribed as GROUP_UPDATE (the spelling /webhook/set accepts) or GROUPS_UPDATE, per instance and global, the event must arrive, and other events must be unaffected. The same checks run through the websocket, rabbitmq, nats, sqs, kafka and pusher controllers with recording clients. Fails today for GROUP_UPDATE, per instance in all seven transports and global in webhook, rabbitmq, nats, kafka and pusher. Co-Authored-By: Claude Opus 5.5 --- test/events/group-update-transports.test.ts | 151 ++++++++++++++++++++ test/events/group-update-webhook.test.ts | 93 ++++++++++++ 2 files changed, 244 insertions(+) create mode 100644 test/events/group-update-transports.test.ts create mode 100644 test/events/group-update-webhook.test.ts diff --git a/test/events/group-update-transports.test.ts b/test/events/group-update-transports.test.ts new file mode 100644 index 0000000000..773b7ebdca --- /dev/null +++ b/test/events/group-update-transports.test.ts @@ -0,0 +1,151 @@ +// The same two spellings of the group update event (see group-update-webhook.test.ts) +// in every other transport. Each one upper-cases `groups.update` to GROUPS_UPDATE +// and compares it with the instance's stored events (which //set only +// accepts as GROUP_UPDATE) and, where it has one, with its global env config +// (keyed GROUP_UPDATE, except SQS). The real controllers run; only the client +// each one publishes through (socket.io, AMQP channel, NATS, SQS, Kafka, Pusher) +// is a recording fake, so nothing leaves the process. +import { beforeEach, describe, expect, it, vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +const GROUP = { id: '120363000000000001@g.us', subject: 'Synthetic group' }; +const monitor: any = { waInstances: { test: { instanceId: 'inst-1' } } }; +const emitData = (event: string) => ({ + instanceName: 'test', + origin: 'test', + event, + data: event === 'groups.update' ? GROUP : { key: { id: 'X' } }, + serverUrl: 'http://127.0.0.1', + dateTime: '2026-09-27T00:00:00.000Z', + sender: '972500000000@s.whatsapp.net', + apiKey: null, + local: true, +}); + +// Each transport: its table and config section, how to build it with a recording +// client, and how to read the event names it published from that client. +type Transport = { + name: string; + config?: string; + build: (prisma: any) => Promise<{ ctrl: any; published: () => string[] }>; +}; + +const transports: Transport[] = [ + { + name: 'websocket', + build: async (prisma) => { + const { WebsocketController } = await import('@api/integrations/event/websocket/websocket.controller'); + const sent: string[] = []; + const ctrl: any = new WebsocketController(prisma, monitor); + ctrl.io = { emit: (e: string) => sent.push(e), of: () => ({ emit: (e: string) => sent.push(e) }) }; + return { ctrl, published: () => sent }; + }, + }, + { + name: 'rabbitmq', + config: 'RABBITMQ', + build: async (prisma) => { + const { RabbitmqController } = await import('@api/integrations/event/rabbitmq/rabbitmq.controller'); + const sent: string[] = []; + const ctrl: any = new RabbitmqController(prisma, monitor); + ctrl.amqpChannel = { + assertExchange: async () => undefined, + assertQueue: async () => undefined, + bindQueue: async () => undefined, + publish: async (_x: string, _k: string, body: Buffer) => void sent.push(JSON.parse(body.toString()).event), + }; + return { ctrl, published: () => sent }; + }, + }, + { + name: 'nats', + config: 'NATS', + build: async (prisma) => { + const { NatsController } = await import('@api/integrations/event/nats/nats.controller'); + const sent: string[] = []; + const ctrl: any = new NatsController(prisma, monitor); + ctrl.natsClient = { publish: (_s: string, body: Uint8Array) => void sent.push(JSON.parse(Buffer.from(body).toString()).event) }; + return { ctrl, published: () => sent }; + }, + }, + { + name: 'sqs', + build: async (prisma) => { + const { SqsController } = await import('@api/integrations/event/sqs/sqs.controller'); + const sent: string[] = []; + const ctrl: any = new SqsController(prisma, monitor); + ctrl.sqs = { sendMessage: (p: any) => void sent.push(JSON.parse(p.MessageBody).event) }; + return { ctrl, published: () => sent }; + }, + }, + { + name: 'kafka', + config: 'KAFKA', + build: async (prisma) => { + const { KafkaController } = await import('@api/integrations/event/kafka/kafka.controller'); + const sent: string[] = []; + const ctrl: any = new KafkaController(prisma, monitor); + ctrl.producer = { send: async (r: any) => void sent.push(JSON.parse(r.messages[0].value).event) }; + return { ctrl, published: () => sent }; + }, + }, + { + name: 'pusher', + config: 'PUSHER', + build: async (prisma) => { + const { PusherController } = await import('@api/integrations/event/pusher/pusher.controller'); + const sent: string[] = []; + const client = { trigger: (_c: string, _e: string, d: any) => void sent.push(d.event) }; + const ctrl: any = new PusherController(prisma, monitor); + ctrl.pusherClients = { test: client }; + ctrl.globalPusherClient = client; + return { ctrl, published: () => sent }; + }, + }, +]; + +async function emitThrough(t: Transport, events: string[] | null, names: string[]) { + const row = events && { enabled: true, events, appId: 'app', key: 'k', secret: 's', cluster: 'eu', useTLS: true }; + const prisma: any = { [t.name]: { findUnique: async () => row }, instance: { findMany: async () => [] } }; + const { ctrl, published } = await t.build(prisma); + ctrl.status = true; // the transport is ENABLED (read from env at construction) + for (const event of names) await ctrl.emit(emitData(event)); + return published(); +} + +describe.each(transports)('$name', (t) => { + beforeEach(async () => void (await import('@config/env.config'))); + + it.each(['GROUP_UPDATE', 'GROUPS_UPDATE'])('per instance, subscribed as %s, receives groups.update', async (name) => { + expect(await emitThrough(t, [name], ['groups.update'])).toEqual(['groups.update']); + }); + + it('per instance, other subscriptions are unchanged and do not bring group updates', async () => { + const events = ['messages.upsert', 'groups.upsert', 'group-participants.update', 'groups.update']; + expect(await emitThrough(t, ['MESSAGES_UPSERT', 'GROUPS_UPSERT', 'GROUP_PARTICIPANTS_UPDATE'], events)).toEqual([ + 'messages.upsert', + 'groups.upsert', + 'group-participants.update', + ]); + expect(await emitThrough(t, ['GROUP_UPDATE'], ['messages.upsert', 'groups.upsert'])).toEqual([]); + }); + + if (t.config) { + it('global, enabled by its GROUPS_UPDATE env var (config key GROUP_UPDATE), receives groups.update', async () => { + const { configService } = await import('@config/env.config'); + const section = configService.get(t.config!); + const saved = structuredClone(section); + if ('GLOBAL_ENABLED' in section) section.GLOBAL_ENABLED = true; + else section.GLOBAL.ENABLED = true; + section.EVENTS = Object.assign(section.EVENTS, Object.fromEntries(Object.keys(section.EVENTS).map((k) => [k, false]))); + section.EVENTS.GROUP_UPDATE = true; + try { + expect(await emitThrough(t, null, ['groups.update', 'messages.upsert'])).toEqual(['groups.update']); + } finally { + Object.assign(section, saved); + Object.assign(section.EVENTS, saved.EVENTS); + } + }); + } +}); diff --git a/test/events/group-update-webhook.test.ts b/test/events/group-update-webhook.test.ts new file mode 100644 index 0000000000..0a7cad4430 --- /dev/null +++ b/test/events/group-update-webhook.test.ts @@ -0,0 +1,93 @@ +// Group updates have two spellings. Baileys emits `groups.update`, which every +// transport upper-cases to GROUPS_UPDATE before comparing it with what the +// instance subscribed to. But the only spelling /webhook/set (and every other +// //set) accepts is GROUP_UPDATE (EventController.events), and the +// global env config is keyed GROUP_UPDATE too. So a webhook subscribed to group +// updates, per instance or global, never receives one, and GROUP_UPDATE is the +// spelling a client can store. +// +// Through the real WebhookController, to a real HTTP destination on 127.0.0.1. +import { createServer } from 'node:http'; +import type { AddressInfo } from 'node:net'; + +import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +const GROUP = { id: '120363000000000001@g.us', subject: 'Synthetic group' }; + +describe('a webhook subscribed to group updates', () => { + const hits: { path: string; body: any }[] = []; + let base: string; + const server = createServer((req, res) => { + let body = ''; + req.on('data', (c) => (body += c)); + req.on('end', () => (hits.push({ path: req.url ?? '', body: JSON.parse(body) }), res.end('ok'))); + }); + + beforeAll(async () => { + server.listen(0, '127.0.0.1'); + await new Promise((r) => server.once('listening', r)); + base = `http://127.0.0.1:${(server.address() as AddressInfo).port}`; + }); + afterAll(() => new Promise((r) => server.close(r))); + beforeEach(() => void hits.splice(0)); + + /** Emit one event through a WebhookController whose instance row subscribes to `events` (null: no row). */ + async function emit(events: string[] | null, event: string, data: any = GROUP) { + const { WebhookController } = await import('@api/integrations/event/webhook/webhook.controller'); + const row = events && { enabled: true, events, url: `${base}/hook`, headers: {}, webhookBase64: false, webhookByEvents: false }; + const prisma: any = { webhook: { findUnique: async () => row, findFirst: async () => row } }; + const monitor: any = { waInstances: { test: { instanceId: 'inst-1' } } }; + await new WebhookController(prisma, monitor).emit({ + instanceName: 'test', + origin: 'test', + event, + data, + serverUrl: 'http://127.0.0.1', + dateTime: '2026-09-27T00:00:00.000Z', + sender: '972500000000@s.whatsapp.net', + apiKey: null, + local: true, + }); + } + + it.each(['GROUP_UPDATE', 'GROUPS_UPDATE'])('per instance, subscribed as %s, receives groups.update', async (name) => { + await emit([name], 'groups.update'); + expect(hits.map((h) => h.path)).toEqual(['/hook']); + expect(hits[0].body.event).toBe('groups.update'); + expect(hits[0].body.instance).toBe('test'); + expect(hits[0].body.data).toEqual(GROUP); + }); + + it('per instance, a group update subscription does not bring other events', async () => { + await emit(['GROUP_UPDATE'], 'messages.upsert', { key: { id: 'X' } }); + await emit(['GROUP_UPDATE'], 'groups.upsert'); + await emit(['GROUP_UPDATE'], 'group-participants.update'); + expect(hits).toEqual([]); + }); + + it('per instance, other subscriptions are unchanged and do not bring group updates', async () => { + const subscribed = ['MESSAGES_UPSERT', 'GROUPS_UPSERT', 'GROUP_PARTICIPANTS_UPDATE']; + for (const event of ['messages.upsert', 'groups.upsert', 'group-participants.update', 'groups.update']) + await emit(subscribed, event); + expect(hits.map((h) => h.body.event)).toEqual(['messages.upsert', 'groups.upsert', 'group-participants.update']); + }); + + it('global, enabled by WEBHOOK_EVENTS_GROUPS_UPDATE (config key GROUP_UPDATE), receives groups.update', async () => { + const { configService } = await import('@config/env.config'); + const webhook = configService.get('WEBHOOK'); + const saved = structuredClone(webhook); + Object.assign(webhook.GLOBAL, { ENABLED: true, URL: `${base}/global`, WEBHOOK_BY_EVENTS: false }); + webhook.EVENTS = Object.fromEntries(Object.keys(webhook.EVENTS).map((k) => [k, false])); + webhook.EVENTS.GROUP_UPDATE = true; + try { + await emit(null, 'groups.update'); + await emit(null, 'messages.upsert', { key: { id: 'X' } }); + } finally { + Object.assign(webhook, saved); + } + expect(hits.map((h) => [h.path, h.body.event])).toEqual([['/global', 'groups.update']]); + expect(hits[0].body.data).toEqual(GROUP); + }); +}); From b0eddeda60d8438566de726f5f155cba76472e96 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:05:23 +0300 Subject: [PATCH 009/157] fix(webhook): accept both spellings of the group update event The event is groups.update (GROUPS_UPDATE once upper-cased), but the //set schemas store GROUP_UPDATE and the global env configs are keyed GROUP_UPDATE (except SQS's), so group updates reached no subscriber. A shared isSubscribed() in event.controller.ts accepts both spellings and replaces the identical comparisons in the webhook, websocket, rabbitmq, sqs, nats, kafka and pusher controllers, per instance and global. No stored value is renamed, so existing subscriptions keep working. Co-Authored-By: Claude Opus 5.5 --- src/api/integrations/event/event.controller.ts | 14 ++++++++++++++ .../integrations/event/kafka/kafka.controller.ts | 6 +++--- src/api/integrations/event/nats/nats.controller.ts | 6 +++--- .../integrations/event/pusher/pusher.controller.ts | 6 +++--- .../event/rabbitmq/rabbitmq.controller.ts | 6 +++--- src/api/integrations/event/sqs/sqs.controller.ts | 4 ++-- .../event/webhook/webhook.controller.ts | 6 +++--- .../event/websocket/websocket.controller.ts | 4 ++-- 8 files changed, 33 insertions(+), 19 deletions(-) diff --git a/src/api/integrations/event/event.controller.ts b/src/api/integrations/event/event.controller.ts index 39b52184bf..a714686a48 100644 --- a/src/api/integrations/event/event.controller.ts +++ b/src/api/integrations/event/event.controller.ts @@ -17,6 +17,20 @@ export type EmitData = { extra?: Record; }; +// Group updates have two spellings. Baileys emits groups.update, GROUPS_UPDATE once +// upper-cased, but every //set schema (EventController.events) and every +// global env config except SQS's stores it as GROUP_UPDATE. Both are accepted and +// neither is renamed, so a subscription stored under either name keeps working. +const EVENT_ALIASES: Record = { GROUPS_UPDATE: ['GROUPS_UPDATE', 'GROUP_UPDATE'] }; + +/** Whether `subscribed` (an instance's stored event names, or a global config's flags) includes the event `we`. */ +export function isSubscribed(subscribed: unknown, we: string): boolean { + const names = EVENT_ALIASES[we] ?? [we]; + if (Array.isArray(subscribed)) return names.some((name) => subscribed.includes(name)); + if (subscribed && typeof subscribed === 'object') return names.some((name) => !!subscribed[name]); + return false; +} + export interface EventControllerInterface { set(instanceName: string, data: any): Promise; get(instanceName: string): Promise; diff --git a/src/api/integrations/event/kafka/kafka.controller.ts b/src/api/integrations/event/kafka/kafka.controller.ts index 543c759ad5..7df6076d87 100644 --- a/src/api/integrations/event/kafka/kafka.controller.ts +++ b/src/api/integrations/event/kafka/kafka.controller.ts @@ -4,7 +4,7 @@ import { configService, Kafka, Log } from '@config/env.config'; import { Logger } from '@config/logger.config'; import { Consumer, ConsumerConfig, Kafka as KafkaJS, KafkaConfig, Producer, ProducerConfig } from 'kafkajs'; -import { EmitData, EventController, EventControllerInterface } from '../event.controller'; +import { EmitData, EventController, EventControllerInterface, isSubscribed } from '../event.controller'; export class KafkaController extends EventController implements EventControllerInterface { private kafkaClient: KafkaJS | null = null; @@ -299,7 +299,7 @@ export class KafkaController extends EventController implements EventControllerI const messageValue = JSON.stringify(message); // Instance-specific events - if (instanceKafka?.enabled && this.producer && Array.isArray(kafkaLocal) && kafkaLocal.includes(we)) { + if (instanceKafka?.enabled && this.producer && isSubscribed(kafkaLocal, we)) { const topicName = this.getTopicName(event, false, instanceName); let retry = 0; @@ -345,7 +345,7 @@ export class KafkaController extends EventController implements EventControllerI } // Global events - if (kafkaGlobal && kafkaEvents[we] && this.producer) { + if (kafkaGlobal && isSubscribed(kafkaEvents, we) && this.producer) { const topicName = this.getTopicName(event, true); let retry = 0; diff --git a/src/api/integrations/event/nats/nats.controller.ts b/src/api/integrations/event/nats/nats.controller.ts index 1ff4fbae89..5b0e759233 100644 --- a/src/api/integrations/event/nats/nats.controller.ts +++ b/src/api/integrations/event/nats/nats.controller.ts @@ -4,7 +4,7 @@ import { configService, Log, Nats } from '@config/env.config'; import { Logger } from '@config/logger.config'; import { connect, NatsConnection, StringCodec } from 'nats'; -import { EmitData, EventController, EventControllerInterface } from '../event.controller'; +import { EmitData, EventController, EventControllerInterface, isSubscribed } from '../event.controller'; export class NatsController extends EventController implements EventControllerInterface { public natsClient: NatsConnection | null = null; @@ -78,7 +78,7 @@ export class NatsController extends EventController implements EventControllerIn // Instância específica if (instanceNats?.enabled) { - if (Array.isArray(natsLocal) && natsLocal.includes(we)) { + if (isSubscribed(natsLocal, we)) { const subject = `${instanceName}.${event.toLowerCase()}`; try { @@ -98,7 +98,7 @@ export class NatsController extends EventController implements EventControllerIn } // Global - if (natsGlobal && natsEvents[we]) { + if (natsGlobal && isSubscribed(natsEvents, we)) { try { const subject = prefixKey ? `${prefixKey}.${event.toLowerCase()}` : event.toLowerCase(); diff --git a/src/api/integrations/event/pusher/pusher.controller.ts b/src/api/integrations/event/pusher/pusher.controller.ts index 045f7cc4f7..bbb2d23e4b 100644 --- a/src/api/integrations/event/pusher/pusher.controller.ts +++ b/src/api/integrations/event/pusher/pusher.controller.ts @@ -6,7 +6,7 @@ import { configService, Log, Pusher as ConfigPusher } from '@config/env.config'; import { Logger } from '@config/logger.config'; import Pusher from 'pusher'; -import { EmitData, EventController, EventControllerInterface } from '../event.controller'; +import { EmitData, EventController, EventControllerInterface, isSubscribed } from '../event.controller'; export class PusherController extends EventController implements EventControllerInterface { private readonly logger = new Logger('PusherController'); private pusherClients: { [instanceName: string]: Pusher } = {}; @@ -162,7 +162,7 @@ export class PusherController extends EventController implements EventController } if (local && instance && instance.enabled) { const pusherLocalEvents = instance.events; - if (Array.isArray(pusherLocalEvents) && pusherLocalEvents.includes(we)) { + if (isSubscribed(pusherLocalEvents, we)) { if (enabledLog) { this.logger.log({ local: `${origin}.sendData-Pusher`, @@ -188,7 +188,7 @@ export class PusherController extends EventController implements EventController } if (this.pusherConfig.GLOBAL?.ENABLED) { const globalEvents = this.pusherConfig.EVENTS; - if (globalEvents[we]) { + if (isSubscribed(globalEvents, we)) { if (enabledLog) { this.logger.log({ local: `${origin}.sendData-Pusher-Global`, diff --git a/src/api/integrations/event/rabbitmq/rabbitmq.controller.ts b/src/api/integrations/event/rabbitmq/rabbitmq.controller.ts index b4625508be..5b070ec1a8 100644 --- a/src/api/integrations/event/rabbitmq/rabbitmq.controller.ts +++ b/src/api/integrations/event/rabbitmq/rabbitmq.controller.ts @@ -4,7 +4,7 @@ import { configService, Log, Rabbitmq } from '@config/env.config'; import { Logger } from '@config/logger.config'; import * as amqp from 'amqplib/callback_api'; -import { EmitData, EventController, EventControllerInterface } from '../event.controller'; +import { EmitData, EventController, EventControllerInterface, isSubscribed } from '../event.controller'; export class RabbitmqController extends EventController implements EventControllerInterface { public amqpChannel: amqp.Channel | null = null; @@ -245,7 +245,7 @@ export class RabbitmqController extends EventController implements EventControll }; if (instanceRabbitmq?.enabled && this.amqpChannel) { - if (Array.isArray(rabbitmqLocal) && rabbitmqLocal.includes(we)) { + if (isSubscribed(rabbitmqLocal, we)) { const exchangeName = instanceName ?? rabbitmqExchangeName; let retry = 0; @@ -298,7 +298,7 @@ export class RabbitmqController extends EventController implements EventControll } } - if (rabbitmqGlobal && rabbitmqEvents[we] && this.amqpChannel) { + if (rabbitmqGlobal && isSubscribed(rabbitmqEvents, we) && this.amqpChannel) { const exchangeName = rabbitmqExchangeName; let retry = 0; diff --git a/src/api/integrations/event/sqs/sqs.controller.ts b/src/api/integrations/event/sqs/sqs.controller.ts index 2b0398ef21..d3788aebde 100644 --- a/src/api/integrations/event/sqs/sqs.controller.ts +++ b/src/api/integrations/event/sqs/sqs.controller.ts @@ -5,7 +5,7 @@ import { CreateQueueCommand, DeleteQueueCommand, ListQueuesCommand, SQS } from ' import { configService, HttpServer, Log, S3, Sqs } from '@config/env.config'; import { Logger } from '@config/logger.config'; -import { EmitData, EventController, EventControllerInterface } from '../event.controller'; +import { EmitData, EventController, EventControllerInterface, isSubscribed } from '../event.controller'; import { EventDto } from '../event.dto'; export class SqsController extends EventController implements EventControllerInterface { @@ -119,7 +119,7 @@ export class SqsController extends EventController implements EventControllerInt } } - if (Array.isArray(sqsEvents) && sqsEvents.includes(we)) { + if (isSubscribed(sqsEvents, we)) { const prefixName = sqsConfig.GLOBAL_ENABLED ? sqsConfig.GLOBAL_PREFIX_NAME : instanceName; const eventFormatted = sqsConfig.GLOBAL_ENABLED && sqsConfig.GLOBAL_FORCE_SINGLE_QUEUE diff --git a/src/api/integrations/event/webhook/webhook.controller.ts b/src/api/integrations/event/webhook/webhook.controller.ts index 7f1dd8dc0e..44956936c9 100644 --- a/src/api/integrations/event/webhook/webhook.controller.ts +++ b/src/api/integrations/event/webhook/webhook.controller.ts @@ -8,7 +8,7 @@ import { Logger } from '@config/logger.config'; import axios, { AxiosInstance } from 'axios'; import * as jwt from 'jsonwebtoken'; -import { EmitData, EventController, EventControllerInterface } from '../event.controller'; +import { EmitData, EventController, EventControllerInterface, isSubscribed } from '../event.controller'; export class WebhookController extends EventController implements EventControllerInterface { private readonly logger = new Logger('WebhookController'); @@ -103,7 +103,7 @@ export class WebhookController extends EventController implements EventControlle }; if (local && instance?.enabled) { - if (Array.isArray(webhookLocal) && webhookLocal.includes(we)) { + if (isSubscribed(webhookLocal, we)) { let baseURL: string; if (instance?.webhookByEvents) { @@ -150,7 +150,7 @@ export class WebhookController extends EventController implements EventControlle } if (webhookConfig.GLOBAL?.ENABLED) { - if (webhookConfig.EVENTS[we]) { + if (isSubscribed(webhookConfig.EVENTS, we)) { let globalURL = webhookConfig.GLOBAL.URL; if (webhookConfig.GLOBAL.WEBHOOK_BY_EVENTS) { diff --git a/src/api/integrations/event/websocket/websocket.controller.ts b/src/api/integrations/event/websocket/websocket.controller.ts index 3c763f08d6..595aaa991c 100644 --- a/src/api/integrations/event/websocket/websocket.controller.ts +++ b/src/api/integrations/event/websocket/websocket.controller.ts @@ -5,7 +5,7 @@ import { Logger } from '@config/logger.config'; import { Server } from 'http'; import { Server as SocketIO } from 'socket.io'; -import { EmitData, EventController, EventControllerInterface } from '../event.controller'; +import { EmitData, EventController, EventControllerInterface, isSubscribed } from '../event.controller'; export class WebsocketController extends EventController implements EventControllerInterface { private io: SocketIO; @@ -156,7 +156,7 @@ export class WebsocketController extends EventController implements EventControl return; } - if (Array.isArray(instance?.events) && instance?.events.includes(configEv)) { + if (isSubscribed(instance?.events, configEv)) { this.socket.of(`/${instanceName}`).emit(event, message); if (logEnabled) { From b588821966a98559105c6aa13d5b4928a3b683a1 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:06:11 +0300 Subject: [PATCH 010/157] test: getMessage answers a miss with nothing, not an empty message Baileys calls the socket's getMessage to answer a retry request for a message this instance sent once the message has left its recent-message cache, and relays any truthy answer under the original id. Evolution answers a database miss with { conversation: '' }. Under the minimal profile (no messages stored) every lookup misses. The test asks for an unknown id under both profiles and for a sent message under minimal, and expects undefined; a message stored under the stored profile must still come back as stored. Fails today with { conversation: '' }. Co-Authored-By: Claude Opus 5.5 --- test/handlers/get-message-miss.test.ts | 56 ++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) create mode 100644 test/handlers/get-message-miss.test.ts diff --git a/test/handlers/get-message-miss.test.ts b/test/handlers/get-message-miss.test.ts new file mode 100644 index 0000000000..1e495ec6c8 --- /dev/null +++ b/test/handlers/get-message-miss.test.ts @@ -0,0 +1,56 @@ +// Evolution hands Baileys its getMessage (the socket config's `getMessage`), and +// Baileys calls it to answer a recipient's retry request for a message this +// instance sent, when the message is no longer in its own recent-message cache. +// Baileys relays whatever comes back if it is truthy, under the original message +// id; only a falsy answer (its own default is `async () => undefined`) means "not +// available, send nothing" (lib/Socket/messages-recv.js, sendMessagesAgain). +// Evolution answers a database miss with `{ conversation: '' }`, so the recipient +// is sent an empty message. A deployment that stores no messages (for example +// DATABASE_SAVE_DATA_NEW_MESSAGE=false, the `minimal` profile) misses every time. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { describe, expect, it } from 'vitest'; + +import { deliver, makeService } from '../helpers/baileys-service'; +import type { Profile } from '../helpers/profiles'; + +const CHAT = '972500000001@s.whatsapp.net'; + +/** getMessage's query over the fake's rows: WHERE "instanceId" = $1 AND "key"->>'id' = $2. */ +function answerFromRows(prisma: any) { + prisma.$queryRaw = async (_sql: TemplateStringsArray, instanceId: string, id: string) => + prisma.message.rows.filter((r: any) => r.instanceId === instanceId && r.key?.id === id); +} + +describe('getMessage, as Baileys calls it to answer a retry request', () => { + it.each(['minimal', 'stored'])('under the %s profile, answers a message it does not have with undefined', async (profile) => { + const { service, prisma } = await makeService({ profile }); + answerFromRows(prisma); + const answer = await service.getMessage({ remoteJid: CHAT, fromMe: true, id: '3EB0MISSING0000000001' }); + expect(answer).toBeUndefined(); + }); + + it('under the minimal profile, a message the instance sent is not stored, so a retry for it finds nothing', async () => { + const { service, prisma, ev } = await makeService({ profile: 'minimal' }); + answerFromRows(prisma); + const key = { remoteJid: CHAT, fromMe: true, id: '3EB0SENT0000000000001' }; + await deliver(service, ev, { + 'messages.upsert': { messages: [{ key, message: { conversation: 'the real text' }, messageTimestamp: 1_700_000_000 }], type: 'notify' }, + }); + expect(prisma.message.rows).toHaveLength(0); + expect(await service.getMessage(key)).toBeUndefined(); + }); + + it('under the stored profile, a stored message is still returned as stored, so Baileys can resend it', async () => { + const { service, prisma, ev } = await makeService({ profile: 'stored' }); + answerFromRows(prisma); + const key = { remoteJid: CHAT, fromMe: true, id: '3EB0SENT0000000000002' }; + await deliver(service, ev, { + 'messages.upsert': { messages: [{ key, message: { conversation: 'the real text' }, messageTimestamp: 1_700_000_000 }], type: 'notify' }, + }); + expect(prisma.message.rows.map((r: any) => r.key?.id)).toEqual([key.id]); + expect(await service.getMessage(key)).toEqual({ conversation: 'the real text' }); + }); +}); From 26b9bfa12247e78d1471f5ccfc23ccbe682480f6 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:06:32 +0300 Subject: [PATCH 011/157] fix(baileys): answer a getMessage miss with undefined A lookup that finds no row used to throw on webMessageInfo[0].message and be answered by the catch with { conversation: '' }, which Baileys relays as the retried message under the original id. Return undefined when nothing is stored, which Baileys reads as "not available" and sends nothing. A lookup that throws (database error) still gets the old answer. Co-Authored-By: Claude Opus 5.5 --- .../channel/whatsapp/whatsapp.baileys.service.ts | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 60e857fcc1..81f4cfa0cd 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -529,6 +529,12 @@ export class BaileysStartupService extends ChannelStartupService { AND "key"->>'id' = ${key.id} `) as proto.IWebMessageInfo[]; + // Not stored: answer undefined. Baileys calls this to answer a retry request + // and relays any truthy answer; only a falsy one means "not available". + if (!webMessageInfo?.length) { + return undefined; + } + if (full) { return webMessageInfo[0]; } From e98168a992c2c9dbebbdc91cc4ffa820eee07bf5 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:02:37 +0300 Subject: [PATCH 012/157] test: forward every @lid to phone mapping Baileys learns, on contacts.upsert Baileys (rc13+) learns which phone number each private @lid belongs to and hands the mappings over as lidPnMappings on messaging-history.set and as lid-mapping.update events. Evolution 2.3.7 reads neither, so a consumer never learns that an @lid chat is a known phone number. The test drives history through the production path (Baileys' own processMessage inside ev.createBufferedFunction, a real signal repository) and asserts the exact CONTACTS_UPSERT items a consumer reads a mapping from: { remoteJid: , pushName: null, lid: , phoneNumber: , instanceId }. On the unchanged code all seven cases fail with no mapping item emitted at all (expected [ ...mappings ], received []), because nothing forwards either source: - (a) an @lid-keyed conversation, mapped by phoneNumberToLidMappings or its own pnJid - (b) a phone-keyed conversation whose lidJid is set - (c) a mapping with no conversation of its own - (d) the mapping in a later, and in an earlier, history batch than its conversation - (e) a live lid-mapping.update from pnForLidChatAction - the negative control (no mapping for an unmapped @lid, no unrelated pair linked), whose positive half expects the two real pairs Forwarding the event field alone is not enough: Baileys' event buffer drops lidPnMappings when it flushes a buffered messaging-history.set (rc14 lib/Utils/event-buffer.js, consolidateEvents). Checked against the spike fix that recovers mappings by asking the signal repository for each @lid chat or contact id: (a), (e) and the earlier-batch (d) pass, while (b), (c), the later-batch (d) and the negative control's unconversationed pair still fail. Co-Authored-By: Claude Opus 5.5 --- test/handlers/lid-mapping.test.ts | 126 ++++++++++++++++++++++++++++++ 1 file changed, 126 insertions(+) create mode 100644 test/handlers/lid-mapping.test.ts diff --git a/test/handlers/lid-mapping.test.ts b/test/handlers/lid-mapping.test.ts new file mode 100644 index 0000000000..5b03de8902 --- /dev/null +++ b/test/handlers/lid-mapping.test.ts @@ -0,0 +1,126 @@ +// WhatsApp addresses many personal chats by a private @lid, and Baileys (rc13+) +// learns which phone number each @lid belongs to: from the history sync +// (`lidPnMappings` on messaging-history.set, built from phoneNumberToLidMappings +// and from each conversation's pnJid / lidJid) and live (`lid-mapping.update`). +// A consumer learns a mapping from one CONTACTS_UPSERT item: +// { remoteJid: , pushName: null, lid: , phoneNumber: , instanceId } +// +// History runs through the production path: Baileys' own processMessage inside +// ev.createBufferedFunction, with a real signal repository. That matters because +// Baileys' event buffer drops `lidPnMappings` when it flushes a buffered +// messaging-history.set (lib/Utils/event-buffer.js, consolidateEvents), so a fix +// that only reads the field off the event is not enough. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { beforeEach, describe, expect, it } from 'vitest'; + +import { msg, syncActionEvents } from '../helpers/baileys-fixtures'; +import { deliver, makeService, settle } from '../helpers/baileys-service'; +import { emitted } from '../helpers/fake-server-module'; +import { realSignalRepository, socketHistory } from '../helpers/socket-history'; + +const INITIAL_BOOTSTRAP = 0; +const RECENT = 3; + +const lid = (n: number) => `1000000000000${String(n).padStart(2, '0')}@lid`; +const pn = (n: number) => `9725000000${String(n).padStart(2, '0')}@s.whatsapp.net`; + +/** The item a consumer reads a mapping from. */ +const mapping = (n: number) => ({ remoteJid: pn(n), pushName: null, lid: lid(n), phoneNumber: pn(n), instanceId: 'inst-1' }); + +/** Every CONTACTS_UPSERT item, as the webhook carries it. */ +const contactItems = () => + emitted.filter((e) => e.event === 'contacts.upsert').flatMap((e) => [].concat(e.data ?? []) as any[]); + +/** The CONTACTS_UPSERT items that carry a mapping, ordered by lid. */ +const mappingItems = () => + contactItems() + .filter((item) => item && ('lid' in item || 'phoneNumber' in item)) + .sort((a, b) => String(a.lid).localeCompare(String(b.lid))); + +/** A service wired to a real signal repository, fed history the way the socket feeds it. */ +async function linkTime() { + const { service, ev } = await makeService(); + const { repo, keys, creds } = await realSignalRepository(); + Object.assign(service.client, { signalRepository: repo, __keys: keys, __creds: creds }); + service.eventHandler(); + service.__wired = true; + const history = async (sync: Record) => { + await socketHistory(ev, service.client, { syncType: INITIAL_BOOTSTRAP, progress: 100, ...sync }); + await settle(service); + }; + return { service, ev, history }; +} + +describe('@lid to phone mappings reach CONTACTS_UPSERT', () => { + beforeEach(() => void emitted.splice(0)); + + it('(a) a conversation keyed by the @lid, mapped by phoneNumberToLidMappings or by its own pnJid', async () => { + const { history } = await linkTime(); + await history({ + conversations: [ + { id: lid(1), messages: [msg(lid(1), 'A1', 'hi')] }, + { id: lid(2), pnJid: pn(2), messages: [msg(lid(2), 'A2', 'hello')] }, + ], + phoneNumberToLidMappings: [{ lidJid: lid(1), pnJid: pn(1) }], + }); + expect(mappingItems()).toEqual([mapping(1), mapping(2)]); + }); + + it('(b) a conversation keyed by the phone number, whose lidJid is set', async () => { + const { history } = await linkTime(); + await history({ conversations: [{ id: pn(3), lidJid: lid(3), messages: [msg(pn(3), 'B1', 'hi')] }] }); + expect(mappingItems()).toEqual([mapping(3)]); + }); + + it('(c) a mapping in phoneNumberToLidMappings with no conversation of its own', async () => { + const { history } = await linkTime(); + await history({ + conversations: [{ id: '120363000000000001@g.us', messages: [msg('120363000000000001@g.us', 'C1', 'group')] }], + phoneNumberToLidMappings: [{ lidJid: lid(4), pnJid: pn(4) }], + }); + expect(mappingItems()).toEqual([mapping(4)]); + }); + + it('(d) the mapping arrives in a later history batch than its conversation', async () => { + const { history } = await linkTime(); + await history({ conversations: [{ id: lid(5), messages: [msg(lid(5), 'D1', 'hi')] }] }); + await history({ syncType: RECENT, conversations: [], phoneNumberToLidMappings: [{ lidJid: lid(5), pnJid: pn(5) }] }); + expect(mappingItems()).toEqual([mapping(5)]); + }); + + it('(d) the mapping arrives in an earlier history batch than its conversation', async () => { + const { history } = await linkTime(); + await history({ conversations: [], phoneNumberToLidMappings: [{ lidJid: lid(6), pnJid: pn(6) }] }); + await history({ syncType: RECENT, conversations: [{ id: lid(6), messages: [msg(lid(6), 'D2', 'hi')] }] }); + expect(mappingItems()).toEqual([mapping(6)]); + }); + + it('(e) live, from the phone (pnForLidChatAction -> lid-mapping.update)', async () => { + const { service, ev } = await makeService(); + const events = syncActionEvents(['pnForLidChat', lid(7)], { pnForLidChatAction: { pnJid: pn(7) } }); + expect(Object.keys(events)).toContain('lid-mapping.update'); + await deliver(service, ev, events); + expect(mappingItems()).toEqual([mapping(7)]); + }); + + it('invents no mapping for an unmapped @lid, and links no unrelated pair', async () => { + const { history } = await linkTime(); + await history({ + conversations: [ + { id: lid(8), messages: [msg(lid(8), 'N1', 'nobody told us who I am')] }, + { id: lid(9), messages: [msg(lid(9), 'N2', 'hi')] }, + ], + phoneNumberToLidMappings: [ + { lidJid: lid(9), pnJid: pn(9) }, + { lidJid: lid(10), pnJid: pn(10) }, + ], + }); + expect(mappingItems()).toEqual([mapping(9), mapping(10)]); + const items = contactItems(); + expect(items.filter((i) => i.lid === lid(8) || (i.remoteJid === lid(8) && i.phoneNumber))).toEqual([]); + expect(items.filter((i) => (i.lid === lid(9) && i.phoneNumber !== pn(9)) || (i.lid === lid(10) && i.phoneNumber !== pn(10)))).toEqual([]); + }); +}); From 71ba3e8faa43dd6495b9532e8a4aa74c809ce55e Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:04:27 +0300 Subject: [PATCH 013/157] feat(baileys): forward @lid to phone mappings on contacts.upsert Every @lid to phone mapping Baileys learns now reaches the consumer as one CONTACTS_UPSERT item: { remoteJid: , pushName: null, lid: , phoneNumber: , instanceId }. Pairs are oriented, normalized to user jids and de-duplicated within a batch; anything that is not an @lid and a phone jid is skipped. - Live mappings: lid-mapping.update is not a bufferable event, so it reaches ev.process intact and is forwarded there. - History mappings: Baileys puts every mapping of a batch on messaging-history.set as lidPnMappings (phoneNumberToLidMappings, plus each conversation's pnJid or lidJid, lib/Utils/history.js:45-78), but history is processed inside ev.createBufferedFunction and the buffer drops the field when it consolidates the batch (lib/Utils/event-buffer.js:543-556). In the test's buffered path the handler receives lidPnMappings undefined in every batch. So eventHandler() now wraps client.ev.emit once per socket and reads lidPnMappings where Baileys emits it, before the buffer sees it, queued on eventProcessingQueue like every other event. Plain TypeScript, Baileys is not patched, and nothing is looked up. Chosen because it is the only option that covers every case. Rejected: - Reading lid / phoneNumber off the contacts Baileys builds from history: they survive the buffer, but a contact carries only its own conversation's lidJid or pnJid (history.js:56-62). A mapping that exists only in phoneNumberToLidMappings (history.js:45-49) has no contact, so an @lid conversation mapped that way, a mapping with no conversation, and a mapping in a later batch than its conversation are all lost. - Looking the mapping up in the signal repository: getPNForLID is local (cache, then the keys store, lib/Signal/lid-mapping.js:199-270), but getLIDForPN falls back to a USYNC query on a miss (lid-mapping.js:155-176), so a phone-keyed conversation cannot be resolved without the network, and the key store has no way to enumerate its entries, so a mapping with no conversation id to ask about, or one arriving in a batch with no conversations, cannot be found at all. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 57 +++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 81f4cfa0cd..0e05e6b571 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -113,6 +113,7 @@ import makeWASocket, { isJidBroadcast, isJidGroup, isJidNewsletter, + isLidUser, isPnUser, jidNormalizedUser, makeCacheableSignalKeyStore, @@ -929,6 +930,56 @@ export class BaileysStartupService extends ChannelStartupService { }, }; + /** + * Tell consumers which phone number a private @lid belongs to: one CONTACTS_UPSERT + * item per pair, { remoteJid: , pushName: null, lid, phoneNumber: }. + */ + private lidMappingHandle(mappings: { lid?: string; pn?: string }[]) { + const seen = new Set(); + const contacts = []; + for (const m of mappings ?? []) { + let [lid, pn] = [m?.lid, m?.pn]; + if (isLidUser(pn) && isPnUser(lid)) [lid, pn] = [pn, lid]; + if (!isLidUser(lid) || !isPnUser(pn)) continue; + [lid, pn] = [jidNormalizedUser(lid), jidNormalizedUser(pn)]; + if (!lid || !pn || seen.has(`${lid}|${pn}`)) continue; + seen.add(`${lid}|${pn}`); + contacts.push({ remoteJid: pn, pushName: null, lid, phoneNumber: pn, instanceId: this.instanceId }); + } + if (contacts.length) { + this.sendDataWebhook(Events.CONTACTS_UPSERT, contacts); + } + } + + /** + * Baileys hands over the mappings it learns from a history sync as `lidPnMappings` + * on messaging-history.set, but history is processed inside a buffered function and + * the event buffer drops that field when it consolidates the batch (Baileys + * lib/Utils/event-buffer.js, consolidateEvents). So read it where Baileys emits it, + * before the buffer does. Every mapping in the batch is there, whether it came from + * phoneNumberToLidMappings or from a conversation's pnJid or lidJid, and nothing is + * looked up, so no network query can follow. + */ + private tapHistoryLidMappings() { + const ev = this.client.ev as any; + if (ev.__lidPnMappingsTap) return; + const emit = ev.emit.bind(ev); + ev.emit = (event: string, data: any) => { + if (event === 'messaging-history.set' && data?.lidPnMappings?.length) { + const mappings = [...data.lidPnMappings]; + this.eventProcessingQueue = this.eventProcessingQueue.then(() => { + try { + if (!this.endSession) this.lidMappingHandle(mappings); + } catch (error) { + this.logger.error(error); + } + }); + } + return emit(event, data); + }; + ev.__lidPnMappingsTap = true; + } + private readonly messageHandle = { 'messaging-history.set': async ({ messages, @@ -1879,6 +1930,8 @@ export class BaileysStartupService extends ChannelStartupService { }; private eventHandler() { + this.tapHistoryLidMappings(); + this.client.ev.process(async (events) => { this.eventProcessingQueue = this.eventProcessingQueue.then(async () => { try { @@ -1913,6 +1966,10 @@ export class BaileysStartupService extends ChannelStartupService { this.instance.authState.saveCreds(); } + if (events['lid-mapping.update']) { + this.lidMappingHandle([events['lid-mapping.update']]); + } + if (events['messaging-history.set']) { const payload = events['messaging-history.set']; await this.messageHandle['messaging-history.set'](payload); From 9e958c1857e7172a7c640c9b4ea06e715b45c2b1 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:03:09 +0300 Subject: [PATCH 014/157] test: say on contacts.upsert whether a name is the one the owner saved Co-Authored-By: Claude Opus 5.5 --- test/handlers/saved-names.test.ts | 117 ++++++++++++++++++++++++++++++ 1 file changed, 117 insertions(+) create mode 100644 test/handlers/saved-names.test.ts diff --git a/test/handlers/saved-names.test.ts b/test/handlers/saved-names.test.ts new file mode 100644 index 0000000000..e95dcbb834 --- /dev/null +++ b/test/handlers/saved-names.test.ts @@ -0,0 +1,117 @@ +// A contact has two names: the one the phone's owner SAVED in their address +// book (Baileys `name`, from an app-state contact action) and the contact's own +// PROFILE name (the message stanza's pushName, Baileys `notify`). Evolution +// folds both into one `pushName` on contacts.upsert, so a consumer cannot tell +// them apart and a profile name that arrives later replaces the saved one. +// Each contacts.upsert item says `saved: true` only when the name is certainly +// the saved one, and `saved: false` otherwise. +// +// Where the names come from in Baileys 7.0.0-rc14: +// - contactAction (lib/Utils/sync-action-utils.js processContactAction) and +// lidContactAction (lib/Utils/chat-utils.js) emit contacts.upsert with +// name = fullName || firstName || username, and `username` alongside. A name +// equal to the username is the contact's handle, not a saved name. +// - history (lib/Utils/history.js) builds name = displayName || name || +// username for every conversation, and PUSH_NAME syncs carry only `notify`: +// neither is certainly a saved name. +// - a live message (lib/Socket/chats.js upsertMessage) carries the sender's +// pushName, and emits contacts.update [{ id, notify, verifiedName }]. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { beforeEach, describe, expect, it } from 'vitest'; + +import { historyEvent, msg, syncActionEvents } from '../helpers/baileys-fixtures'; +import { deliver, makeService } from '../helpers/baileys-service'; +import { emitted } from '../helpers/fake-server-module'; + +const ALPHA = '972500000001@s.whatsapp.net'; +const BRAVO = '972500000002@s.whatsapp.net'; +const CHARLIE = '972500000003@s.whatsapp.net'; +const DELTA = '972500000004@s.whatsapp.net'; +const ECHO_LID = '100000000005@lid'; +const INITIAL_BOOTSTRAP = 0; +const PUSH_NAME = 4; + +const upserts = (jid: string) => + emitted + .filter((e) => e.event === 'contacts.upsert') + .flatMap((e) => [].concat(e.data)) + .filter((c: any) => c?.remoteJid === jid); + +/** An item from Evolution's contacts.upsert handler (Baileys contacts.upsert, or history). */ +const item = (remoteJid: string, pushName: string, saved: boolean) => ({ + remoteJid, + pushName, + profilePicUrl: null, + instanceId: 'inst-1', + saved, +}); + +/** An item from Evolution's messages.upsert handler, which looks the picture up (none here). */ +const fromMessage = (remoteJid: string, pushName: string, saved: boolean) => ({ + ...item(remoteJid, pushName, saved), + profilePicUrl: undefined, +}); + +/** What the socket emits for an incoming text: the message, and the sender's profile name (chats.js upsertMessage). */ +const incoming = (jid: string, id: string, pushName: string) => ({ + 'messages.upsert': { messages: [{ ...msg(jid, id, 'hi').message, pushName }], type: 'notify' }, + 'contacts.update': [{ id: jid, notify: pushName, verifiedName: undefined }], +}); + +describe('contacts.upsert says whether a name is the one the owner saved', () => { + beforeEach(() => void emitted.splice(0)); + + it('app-state contact action: the address-book name is saved, a bare username is not', async () => { + const { service, ev } = await makeService(); + await deliver(service, ev, syncActionEvents(['contact', ALPHA], { contactAction: { fullName: 'Alpha Saved', firstName: 'Alpha' } })); + await deliver(service, ev, syncActionEvents(['contact', BRAVO], { contactAction: { firstName: 'Bravo' } })); + await deliver(service, ev, syncActionEvents(['contact', CHARLIE], { contactAction: { username: 'charlie.handle' } })); + await deliver(service, ev, syncActionEvents(['lid_contact', ECHO_LID], { lidContactAction: { fullName: 'Echo Saved' } })); + + expect(upserts(ALPHA)).toEqual([item(ALPHA, 'Alpha Saved', true)]); + expect(upserts(BRAVO)).toEqual([item(BRAVO, 'Bravo', true)]); + expect(upserts(CHARLIE)).toEqual([item(CHARLIE, 'charlie.handle', false)]); + expect(upserts(ECHO_LID)).toEqual([item(ECHO_LID, 'Echo Saved', true)]); + }); + + it('a live message: the sender profile name is kept, and is not saved', async () => { + const { service, ev } = await makeService(); + await deliver(service, ev, incoming(DELTA, '3EB0DDDDDDDDDDDDDDD1', 'delta profile')); + + expect(upserts(DELTA)).toEqual([fromMessage(DELTA, 'delta profile', false)]); + }); + + it('history: display names, usernames, chat names and push names are never marked saved', async () => { + const { service, ev } = await makeService(); + await deliver(service, ev, { + 'messaging-history.set': historyEvent({ + syncType: INITIAL_BOOTSTRAP, + progress: 100, + conversations: [ + { id: ALPHA, displayName: 'Alpha Display', messages: [msg(ALPHA, 'A1', 'hi')] }, + { id: BRAVO, username: 'bravo.handle', messages: [msg(BRAVO, 'B1', 'hi')] }, + { id: CHARLIE, name: 'Charlie Chat', messages: [msg(CHARLIE, 'C1', 'hi')] }, + ], + }), + }); + await deliver(service, ev, { + 'messaging-history.set': historyEvent({ syncType: PUSH_NAME, pushnames: [{ id: DELTA, pushname: 'delta profile' }] }), + }); + + expect(upserts(ALPHA)).toEqual([item(ALPHA, 'Alpha Display', false)]); + expect(upserts(BRAVO)).toEqual([item(BRAVO, 'bravo.handle', false)]); + expect(upserts(CHARLIE)).toEqual([item(CHARLIE, 'Charlie Chat', false)]); + expect(upserts(DELTA)).toEqual([item(DELTA, 'delta profile', false)]); + }); + + it('the saved name and a later profile name for the same person stay distinguishable', async () => { + const { service, ev } = await makeService(); + await deliver(service, ev, syncActionEvents(['contact', ALPHA], { contactAction: { fullName: 'Alpha Saved', firstName: 'Alpha' } })); + await deliver(service, ev, incoming(ALPHA, '3EB0AAAAAAAAAAAAAAA1', 'alpha profile')); + + expect(upserts(ALPHA)).toEqual([item(ALPHA, 'Alpha Saved', true), fromMessage(ALPHA, 'alpha profile', false)]); + }); +}); From f7b5606226f2b6969d88308888537803a9bc6766 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:03:33 +0300 Subject: [PATCH 015/157] feat(baileys): mark saved contact names on contacts.upsert, only when certain Each contacts.upsert webhook item now carries saved. It is true only for a name from an app-state contact action (fullName or firstName), and false for history names (displayName, chat name or username), push names, a message's pushName, and a contact action name that is just the username. Every existing field is unchanged, and saved is not written to the stored row. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 23 +++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 0e05e6b571..e7e92d0d89 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -813,7 +813,12 @@ export class BaileysStartupService extends ChannelStartupService { }; private readonly contactHandle = { - 'contacts.upsert': async (contacts: Contact[]) => { + // `saved` on each contacts.upsert item says the name is certainly the one the + // owner saved in their address book. In Baileys only an app-state contact + // action emits contacts.upsert, with name = fullName || firstName || username, + // so a name equal to the username is a handle, not a saved name. Callers that + // cannot be sure (history) pass saved: false. + 'contacts.upsert': async (contacts: (Contact & { saved?: boolean })[]) => { try { const contactsRaw: any = contacts.map((contact) => ({ remoteJid: contact.id, @@ -823,7 +828,13 @@ export class BaileysStartupService extends ChannelStartupService { })); if (contactsRaw.length > 0) { - this.sendDataWebhook(Events.CONTACTS_UPSERT, contactsRaw); + this.sendDataWebhook( + Events.CONTACTS_UPSERT, + contactsRaw.map((raw, i) => ({ + ...raw, + saved: contacts[i].saved ?? (!!contacts[i].name && contacts[i].name !== contacts[i].username), + })), + ); if (this.configService.get('DATABASE').SAVE_DATA.CONTACTS) await this.prismaRepository.contact.createMany({ data: contactsRaw, skipDuplicates: true }); @@ -1125,7 +1136,10 @@ export class BaileysStartupService extends ChannelStartupService { } await this.contactHandle['contacts.upsert']( - contacts.filter((c) => !!c.notify || !!c.name).map((c) => ({ id: c.id, name: c.name ?? c.notify })), + contacts + .filter((c) => !!c.notify || !!c.name) + // A history name is displayName || name || username, and a push name is the profile name. + .map((c) => ({ id: c.id, name: c.name ?? c.notify, saved: false })), ); contacts = undefined; @@ -1598,7 +1612,8 @@ export class BaileysStartupService extends ChannelStartupService { continue; } - this.sendDataWebhook(Events.CONTACTS_UPSERT, contactRaw); + // A message's pushName is the sender's own profile name, never a saved one. + this.sendDataWebhook(Events.CONTACTS_UPSERT, { ...contactRaw, saved: false }); if (this.configService.get('DATABASE').SAVE_DATA.CONTACTS) await this.prismaRepository.contact.upsert({ From dd11ebe75933a8610060f4cf2097026f3eb1e043 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:05:16 +0300 Subject: [PATCH 016/157] test: nothing a message carries reaches the logs under LOG_LEVEL=ERROR,WARN Drives each path that handles a message, contact or call with a distinctive text, phone number and push name, under the minimal and stored profiles, and asserts none of the three is printed. captureOutput now also catches direct writes to fd 1 and 2, which is how pino (the logger Evolution hands Baileys) prints. Paths that leak on 2.3.7: - ordinary incoming message and placeholder resend: console.log(messageRaw) prints text, sender number and push name; the resend also prints the whole message after "Message received from phone" - on-demand history sync: console.log dumps every message in the batch - message that failed to decrypt ("No session record"): Baileys' error log "failed to decrypt message" prints the key with the sender's JID, and Evolution's WARN "Message ignored with messageStubParameters" prints the whole message with the JID and push name - Baileys "error in handling message": prints the stanza (from, notify, body) through the pino logger Evolution configures - incoming call: console.log of the CB:call and CB:ack,class:call stanzas prints the caller's JID and push name - group metadata cache lookup: console.log "Cache request for group" prints the group JID, which embeds the creator's number in old-format ids - group participants update when the lookup fails: ERROR prints the group JID - raw node through baileysSendNode: console.log prints the whole stanza - stored profile only: "Original message not found for update" WARN prints the key, and "Chat insert record ignored" prints the chat JID - status update under minimal: a bare console.log "CACHE:" bypasses LOG_LEVEL Guards that already pass: a history batch and an outgoing text message. Co-Authored-By: Claude Opus 5.5 --- test/handlers/log-privacy.test.ts | 226 ++++++++++++++++++++++++++++++ test/helpers/capture-output.ts | 24 +++- 2 files changed, 249 insertions(+), 1 deletion(-) create mode 100644 test/handlers/log-privacy.test.ts diff --git a/test/handlers/log-privacy.test.ts b/test/handlers/log-privacy.test.ts new file mode 100644 index 0000000000..62dc873485 --- /dev/null +++ b/test/handlers/log-privacy.test.ts @@ -0,0 +1,226 @@ +// A deployment can run Evolution with LOG_LEVEL=ERROR,WARN and LOG_BAILEYS=error +// on the promise that payloads never reach the logs. Each case drives one path +// that handles a message, a contact or a call with a distinctive text, phone +// number and push name, and asserts that none of the three is printed: +// not by Evolution's logger, not by a bare console call, and not by the pino +// logger Evolution hands Baileys. +import { vi } from 'vitest'; + +const sockets = vi.hoisted(() => ({ make: undefined as undefined | ((config: any) => any) })); + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); +// createClient() builds the socket with makeWASocket; the fake returns the +// harness client plus a ws emitter, and keeps the config (and so the logger) +// Evolution passed. +vi.mock('baileys', async (importOriginal) => { + const real: any = await importOriginal(); + const makeWASocket = (config: any) => sockets.make!(config); + return { ...real, default: makeWASocket, makeWASocket }; +}); + +import { EventEmitter } from 'node:events'; + +import { binaryNodeToString, decryptMessageNode } from 'baileys'; +import { describe, expect, it } from 'vitest'; + +import { historyEvent, msg } from '../helpers/baileys-fixtures'; +import { deliver, makeService, settle, WUID } from '../helpers/baileys-service'; +import { captureOutput } from '../helpers/capture-output'; +import type { Profile } from '../helpers/profiles'; + +const PHONE = '972509876543'; +const TEXT = 'Zq7 the private sentence Zq7'; +const NAME = 'Dana Kfirovich'; +const SENDER = `${PHONE}@s.whatsapp.net`; +// An old-format group id embeds its creator's phone number. +const GROUP = `${PHONE}-1600000000@g.us`; +const SECRETS = { text: 'Zq7', phone: PHONE, name: NAME }; + +/** Each secret that appears in the output, with the line it appeared on (truncated). */ +function leaks(out: string) { + const found: string[] = []; + for (const [what, secret] of Object.entries(SECRETS)) { + const line = out.split('\n').find((l) => l.includes(secret)); + if (line !== undefined) found.push(`${what}: ${line.replace(/\x1b\[[0-9;]*m/g, '').trim().slice(0, 160)}`); + } + return found; +} + +const incoming = (id: string, extra: Record = {}) => ({ ...msg(SENDER, id, TEXT).message, pushName: NAME, ...extra }); + +/** Run createClient() against the fake socket, so the ws handlers and the Baileys logger are Evolution's own. */ +async function connect(service: any, ev: any) { + const client = service.client; + let config: any; + sockets.make = (c: any) => ((config = c), { ...client, ev, ws: new EventEmitter() }); + service.defineAuthState = async () => ({ state: { creds: {}, keys: {} }, saveCreds: async () => undefined }); + service.localSettings ??= {}; + await service.createClient(); + service.__wired = true; // createClient() already called eventHandler() + return { socket: service.client, config }; +} + +describe.each(['minimal', 'stored'] as Profile[])('nothing a message carries reaches the logs (profile %s)', (profile) => { + it('an ordinary incoming message', async () => { + const { service, ev } = await makeService({ profile }); + const out = await captureOutput(() => deliver(service, ev, { 'messages.upsert': { messages: [incoming('M1')], type: 'notify' } })); + expect(leaks(out)).toEqual([]); + }); + + it('an incoming message whose push name renames a known chat, when the chat update fails', async () => { + const { service, prisma, ev } = await makeService({ profile }); + prisma.chat.rows.push({ id: 'chat-1', remoteJid: SENDER, instanceId: 'inst-1', name: 'Old name' }); + prisma.chat.update = async () => { + throw new Error('database unavailable'); + }; + const out = await captureOutput(() => deliver(service, ev, { 'messages.upsert': { messages: [incoming('M2')], type: 'notify' } })); + expect(leaks(out)).toEqual([]); + }); + + it('a message the phone re-sent on request (placeholder resend)', async () => { + const { service, ev } = await makeService({ profile }); + const out = await captureOutput(() => + deliver(service, ev, { 'messages.upsert': { messages: [incoming('M3')], type: 'notify', requestId: 'R1' } }), + ); + expect(leaks(out)).toEqual([]); + }); + + it('a history batch', async () => { + const { service, ev } = await makeService({ profile }); + const event = historyEvent({ + syncType: 3, // RECENT + conversations: [{ id: SENDER, name: NAME, messages: [msg(SENDER, 'H1', TEXT, { pushName: NAME })] }], + pushnames: [{ id: SENDER, pushname: NAME }], + }); + const out = await captureOutput(() => deliver(service, ev, { 'messaging-history.set': event })); + expect(leaks(out)).toEqual([]); + }); + + it('an on-demand history sync', async () => { + const { service, ev } = await makeService({ profile }); + const event = historyEvent({ + syncType: 6, // ON_DEMAND + conversations: [{ id: SENDER, messages: [msg(SENDER, 'H2', TEXT, { pushName: NAME })] }], + }); + const out = await captureOutput(() => deliver(service, ev, { 'messaging-history.set': event })); + expect(leaks(out)).toEqual([]); + }); + + it('a message that failed to decrypt (No session record), through Baileys and then Evolution', async () => { + const { service, ev } = await makeService({ profile }); + const { socket, config } = await connect(service, ev); + const stanza = { + tag: 'message', + attrs: { from: SENDER, id: 'D1', t: '1700000000', type: 'text', notify: NAME }, + content: [{ tag: 'enc', attrs: { v: '2', type: 'msg' }, content: new Uint8Array([1, 2, 3]) }], + }; + const repository = { + lidMapping: { getLIDForPN: async () => null, storeLIDPNMappings: async () => undefined }, + decryptMessage: async () => { + throw new Error('No session record'); + }, + }; + const out = await captureOutput(async () => { + const { fullMessage, decrypt } = decryptMessageNode(stanza as any, socket.user.id, undefined as any, repository as any, config.logger); + await decrypt(); + expect(fullMessage.messageStubParameters).toEqual(['No session record']); + await deliver(service, ev, { 'messages.upsert': { messages: [fullMessage], type: 'notify' } }); + await new Promise((r) => setTimeout(r, 20)); // pino's async write + }); + expect(leaks(out)).toEqual([]); + }); + + it('a message Baileys fails to handle (its "error in handling message" log)', async () => { + const { service, ev } = await makeService({ profile }); + const { config } = await connect(service, ev); + const node = { tag: 'message', attrs: { from: SENDER, id: 'E1', notify: NAME }, content: [{ tag: 'body', attrs: {}, content: TEXT }] }; + // The call Baileys makes at messages-recv.js:1436 (rc14). + const out = await captureOutput(async () => { + config.logger.error({ error: new Error('boom'), node: binaryNodeToString(node as any) }, 'error in handling message'); + await new Promise((r) => setTimeout(r, 20)); + }); + expect(leaks(out)).toEqual([]); + }); + + it('a message status update', async () => { + const { service, ev } = await makeService({ profile }); + const out = await captureOutput(() => + deliver(service, ev, { 'messages.update': [{ key: { remoteJid: SENDER, fromMe: true, id: `S-${profile}` }, update: { status: 4 } }] }), + ); + expect(leaks(out)).toEqual([]); + // Nothing about an ordinary status update is an error or a warning. + if (profile === 'minimal') expect(out).toBe(''); + }); + + it('an incoming call (the raw call stanzas and the call event)', async () => { + const { service, ev } = await makeService({ profile }); + const { socket } = await connect(service, ev); + const offer = { + tag: 'call', + attrs: { from: SENDER, id: 'C1', t: '1700000000', notify: NAME }, + content: [{ tag: 'offer', attrs: { 'call-id': 'CALL1', 'call-creator': SENDER }, content: undefined }], + }; + const ack = { tag: 'ack', attrs: { from: SENDER, id: 'C1', class: 'call', type: 'offer' } }; + const call = [{ chatId: SENDER, from: SENDER, id: 'CALL1', date: new Date(1_700_000_000_000), offline: false, status: 'offer', isVideo: false, isGroup: false }]; + const out = await captureOutput(async () => { + socket.ws.emit('CB:call', offer); + socket.ws.emit('CB:ack,class:call', ack); + await deliver(service, ev, { call }); + }); + expect(leaks(out)).toEqual([]); + }); + + it('a group metadata cache lookup (miss, then hit)', async () => { + const { service } = await makeService({ profile }); + service.client.groupMetadata = async (id: string) => ({ id, subject: TEXT, participants: [{ id: SENDER, admin: null }] }); + const out = await captureOutput(async () => { + await service.getGroupMetadataCache(GROUP); + await service.getGroupMetadataCache(GROUP); + }); + expect(leaks(out)).toEqual([]); + }); + + it('a group participants update whose participant lookup fails', async () => { + const { service, ev } = await makeService({ profile }); + service.client.groupMetadata = async () => { + throw new Error('item-not-found'); + }; + const out = await captureOutput(() => + deliver(service, ev, { 'group-participants.update': { id: `${PHONE}-1600000001@g.us`, author: SENDER, participants: [SENDER], action: 'add' } }), + ); + expect(leaks(out)).toEqual([]); + }); + + it('an outgoing text message', async () => { + const { service } = await makeService({ profile }); + Object.assign(service.client, { + onWhatsApp: async (...jids: string[]) => jids.map((jid) => ({ exists: true, jid })), + sendMessage: async (jid: string, content: any) => ({ + key: { remoteJid: jid, fromMe: true, id: 'OUT1' }, + message: { conversation: content.text ?? TEXT }, + messageTimestamp: 1_700_000_000, + status: 1, + }), + presenceSubscribe: async () => undefined, + sendPresenceUpdate: async () => undefined, + }); + let sent: any; + const out = await captureOutput(async () => { + sent = await service.textMessage({ number: PHONE, text: TEXT }); + await settle(service); + }); + expect(sent?.key?.id).toBe('OUT1'); + expect(leaks(out)).toEqual([]); + }); + + it('a raw node sent through baileysSendNode', async () => { + const { service } = await makeService({ profile }); + service.client.sendNode = async () => undefined; + const stanza = { tag: 'message', attrs: { to: SENDER, id: 'N1' }, content: [{ tag: 'body', attrs: {}, content: TEXT }] }; + const out = await captureOutput(() => service.baileysSendNode(stanza)); + expect(leaks(out)).toEqual([]); + }); +}); diff --git a/test/helpers/capture-output.ts b/test/helpers/capture-output.ts index 0d580fc061..8200bc848c 100644 --- a/test/helpers/capture-output.ts +++ b/test/helpers/capture-output.ts @@ -1,4 +1,9 @@ -// Everything the process prints while `fn` runs: console.* and raw stdout/stderr. +// Everything the process prints while `fn` runs: console.* and raw +// stdout/stderr, including direct writes to file descriptors 1 and 2 (pino, +// which Baileys logs through, writes to the fd with sonic-boom and never +// touches process.stdout). +import fs from 'node:fs'; + export async function captureOutput(fn: () => Promise) { const out: string[] = []; const fmt = (args: any[]) => args.map((a) => (typeof a === 'string' ? a : safe(a))).join(' '); @@ -6,15 +11,32 @@ export async function captureOutput(fn: () => Promise) { const saved = names.map((n) => console[n]); const stdout = process.stdout.write.bind(process.stdout); const stderr = process.stderr.write.bind(process.stderr); + const fsWrite = fs.write; + const fsWriteSync = fs.writeSync; + const isStd = (fd: any) => fd === 1 || fd === 2; names.forEach((n) => (console[n] = (...a: any[]) => void out.push(fmt(a)))); (process.stdout as any).write = (c: any) => (out.push(String(c)), true); (process.stderr as any).write = (c: any) => (out.push(String(c)), true); + (fs as any).writeSync = (fd: any, data: any, ...rest: any[]) => { + if (!isStd(fd)) return (fsWriteSync as any)(fd, data, ...rest); + out.push(String(data)); + return typeof data === 'string' ? Buffer.byteLength(data) : data.length; + }; + (fs as any).write = (fd: any, data: any, ...rest: any[]) => { + if (!isStd(fd)) return (fsWrite as any)(fd, data, ...rest); + out.push(String(data)); + const cb = rest.find((r) => typeof r === 'function'); + const n = typeof data === 'string' ? Buffer.byteLength(data) : data.length; + if (cb) process.nextTick(cb, null, n, data); + }; try { await fn(); } finally { names.forEach((n, i) => (console[n] = saved[i])); (process.stdout as any).write = stdout; (process.stderr as any).write = stderr; + (fs as any).write = fsWrite; + (fs as any).writeSync = fsWriteSync; } return out.join('\n'); } From c52cab49993c56de28c92e2b776add748de0aff0 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:07:27 +0300 Subject: [PATCH 017/157] fix(baileys): keep message content, numbers and names out of the logs A deployment with LOG_LEVEL=ERROR,WARN still printed message text, phone numbers and push names: bare console.log dumps that ignore LOG_LEVEL, WARN and ERROR lines that serialised whole messages or keys, and the pino logger handed to Baileys, which at level error prints message keys, sender JIDs and whole stanzas. - Baileys gets makeBaileysLogger (src/utils/log-privacy.ts): the same level, but every logged object is reduced to message id, chat type, message type, the session-record flag, counts and the error's name and message, with anything shaped like a JID or a phone number masked. Used for the socket, the signal key store and media downloads. - Removed console.log(messageRaw); the full payload stays available at VERBOSE through the existing logger.verbose. - On-demand history, "Message received from phone", CB:call and CB:ack,class:call, CACHE, group cache lookups and baileysSendNode now go through the logger (INFO or VERBOSE) with ids and counts only. - "Message ignored with messageStubParameters" prints id, chat type and the matched decrypt error instead of the whole message. - "Original message not found for update" prints the message id, not the key; "Chat insert record ignored" prints the chat type, not the JID; the group participants error drops the group JID. Webhook payloads and every other behaviour are unchanged. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 75 ++++++++++--------- src/utils/log-privacy.ts | 61 +++++++++++++++ 2 files changed, 99 insertions(+), 37 deletions(-) create mode 100644 src/utils/log-privacy.ts diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index e7e92d0d89..f8a88d383c 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -82,6 +82,7 @@ import { createId as cuid } from '@paralleldrive/cuid2'; import { Instance, Message } from '@prisma/client'; import { createJid } from '@utils/createJid'; import { fetchLatestWaWebVersion } from '@utils/fetchLatestWaWebVersion'; +import { jidKind, makeBaileysLogger } from '@utils/log-privacy'; import { makeProxyAgent, makeProxyAgentUndici } from '@utils/makeProxyAgent'; import { getOnWhatsappCache, saveOnWhatsappCache } from '@utils/onWhatsappCache'; import { status } from '@utils/renderStatus'; @@ -146,7 +147,6 @@ import NodeCache from 'node-cache'; import cron from 'node-cron'; import { release } from 'os'; import { join } from 'path'; -import P from 'pino'; import qrcode, { QRCodeToDataURLOptions } from 'qrcode'; import qrcodeTerminal from 'qrcode-terminal'; import sharp from 'sharp'; @@ -645,11 +645,11 @@ export class BaileysStartupService extends ChannelStartupService { const socketConfig: UserFacingSocketConfig = { ...options, version, - logger: P({ level: this.logBaileys }), + logger: makeBaileysLogger(this.logBaileys), printQRInTerminal: false, auth: { creds: this.instance.authState.state.creds, - keys: makeCacheableSignalKeyStore(this.instance.authState.state.keys, P({ level: 'error' }) as any), + keys: makeCacheableSignalKeyStore(this.instance.authState.state.keys, makeBaileysLogger('error') as any), }, msgRetryCounterCache: this.msgRetryCounterCache, generateHighQualityLinkPreview: true, @@ -711,13 +711,13 @@ export class BaileysStartupService extends ChannelStartupService { this.eventHandler(); this.client.ws.on('CB:call', (packet) => { - console.log('CB:call', packet); + this.logger.verbose(`CB:call id=${packet?.attrs?.id ?? ''}`); const payload = { event: 'CB:call', packet: packet }; this.sendDataWebhook(Events.CALL, payload, true, ['websocket']); }); this.client.ws.on('CB:ack,class:call', (packet) => { - console.log('CB:ack,class:call', packet); + this.logger.verbose(`CB:ack,class:call id=${packet?.attrs?.id ?? ''}`); const payload = { event: 'CB:ack,class:call', packet: packet }; this.sendDataWebhook(Events.CALL, payload, true, ['websocket']); }); @@ -1009,7 +1009,7 @@ export class BaileysStartupService extends ChannelStartupService { }) => { try { if (syncType === proto.HistorySync.HistorySyncType.ON_DEMAND) { - console.log('received on-demand history sync, messages=', messages); + this.logger.info(`received on-demand history sync, messages=${messages.length}`); } console.log( `recv ${chats.length} chats, ${contacts.length} contacts, ${messages.length} msgs (is latest: ${isLatest}, progress: ${progress}%), type: ${syncType}`, @@ -1156,21 +1156,20 @@ export class BaileysStartupService extends ChannelStartupService { ) => { try { for (const received of messages) { - if ( - received?.messageStubParameters?.some?.((param) => - [ - 'No matching sessions found for message', - 'Bad MAC', - 'failed to decrypt message', - 'SessionError', - 'Invalid PreKey ID', - 'No session record', - 'No session found to decrypt message', - 'Message absent from node', - ].some((err) => param?.includes?.(err)), - ) - ) { - this.logger.warn(`Message ignored with messageStubParameters: ${JSON.stringify(received, null, 2)}`); + const decryptFailure = [ + 'No matching sessions found for message', + 'Bad MAC', + 'failed to decrypt message', + 'SessionError', + 'Invalid PreKey ID', + 'No session record', + 'No session found to decrypt message', + 'Message absent from node', + ].find((err) => received?.messageStubParameters?.some?.((param) => param?.includes?.(err))); + if (decryptFailure) { + this.logger.warn( + `Message ignored with messageStubParameters: id=${received.key?.id}, chat=${jidKind(received.key?.remoteJid)}, reason=${decryptFailure}`, + ); continue; } if (received.message?.conversation || received.message?.extendedTextMessage?.text) { @@ -1181,7 +1180,7 @@ export class BaileysStartupService extends ChannelStartupService { console.log('requested placeholder resync, id=', messageId); } else if (requestId) { - console.log('Message received from phone, id=', requestId, received); + this.logger.info(`Message received from phone, id=${requestId}, message id=${received.key?.id}`); } if (text == 'onDemandHistSync') { @@ -1267,7 +1266,9 @@ export class BaileysStartupService extends ChannelStartupService { data: { name: received.pushName }, }); } catch { - console.log(`Chat insert record ignored: ${received.key.remoteJid} - ${this.instanceId}`); + this.logger.warn( + `Chat insert record ignored: ${jidKind(received.key.remoteJid)} chat - ${this.instanceId}`, + ); } } } @@ -1519,7 +1520,7 @@ export class BaileysStartupService extends ChannelStartupService { { key: received.key, message: received?.message }, 'buffer', {}, - { logger: P({ level: 'error' }) as any, reuploadRequest: this.client.updateMediaMessage }, + { logger: makeBaileysLogger('error') as any, reuploadRequest: this.client.updateMediaMessage }, ); if (buffer) { @@ -1530,7 +1531,7 @@ export class BaileysStartupService extends ChannelStartupService { { key: received.key, message: received?.message }, 'buffer', {}, - { logger: P({ level: 'error' }) as any, reuploadRequest: this.client.updateMediaMessage }, + { logger: makeBaileysLogger('error') as any, reuploadRequest: this.client.updateMediaMessage }, ); if (buffer) { @@ -1549,8 +1550,6 @@ export class BaileysStartupService extends ChannelStartupService { if (messageRaw.key.remoteJid?.includes('@lid') && messageRaw.key.remoteJidAlt) { messageRaw.key.remoteJid = messageRaw.key.remoteJidAlt; } - console.log(messageRaw); - this.sendDataWebhook(Events.MESSAGES_UPSERT, messageRaw); await chatbotController.emit({ @@ -1642,7 +1641,7 @@ export class BaileysStartupService extends ChannelStartupService { const cached = await this.baileysCache.get(updateKey); const secondsSinceEpoch = Math.floor(Date.now() / 1000); - console.log('CACHE:', { cached, updateKey, messageTimestamp: update.messageTimestamp, secondsSinceEpoch }); + this.logger.verbose({ cached, updateKey, messageTimestamp: update.messageTimestamp, secondsSinceEpoch }); if ( (update.messageTimestamp && update.messageTimestamp === cached) || @@ -1718,7 +1717,7 @@ export class BaileysStartupService extends ChannelStartupService { findMessage = messages[0] || null; if (!findMessage?.id) { - this.logger.warn(`Original message not found for update. Skipping. Key: ${JSON.stringify(key)}`); + this.logger.warn(`Original message not found for update. Skipping. Message id: ${key.id}`); continue; } message.messageId = findMessage.id; @@ -1791,7 +1790,9 @@ export class BaileysStartupService extends ChannelStartupService { try { await this.prismaRepository.chat.update({ where: { id: existingChat.id }, data: chatToInsert }); } catch { - console.log(`Chat insert record ignored: ${chatToInsert.remoteJid} - ${chatToInsert.instanceId}`); + this.logger.warn( + `Chat insert record ignored: ${jidKind(chatToInsert.remoteJid)} chat - ${chatToInsert.instanceId}`, + ); } } } @@ -1872,7 +1873,7 @@ export class BaileysStartupService extends ChannelStartupService { this.sendDataWebhook(Events.GROUP_PARTICIPANTS_UPDATE, enhancedParticipantsUpdate); } catch (error) { this.logger.error( - `Failed to resolve participant data for GROUP_PARTICIPANTS_UPDATE webhook: ${error.message} | Group: ${participantsUpdate.id} | Participants: ${participantsUpdate.participants.length}`, + `Failed to resolve participant data for GROUP_PARTICIPANTS_UPDATE webhook: ${error.message} | Participants: ${participantsUpdate.participants.length}`, ); // Fallback - envia sem conversão this.sendDataWebhook(Events.GROUP_PARTICIPANTS_UPDATE, participantsUpdate); @@ -2594,7 +2595,7 @@ export class BaileysStartupService extends ChannelStartupService { { key: messageRaw.key, message: messageRaw?.message }, 'buffer', {}, - { logger: P({ level: 'error' }) as any, reuploadRequest: this.client.updateMediaMessage }, + { logger: makeBaileysLogger('error') as any, reuploadRequest: this.client.updateMediaMessage }, ); if (buffer) { @@ -2605,7 +2606,7 @@ export class BaileysStartupService extends ChannelStartupService { { key: messageRaw.key, message: messageRaw?.message }, 'buffer', {}, - { logger: P({ level: 'error' }) as any, reuploadRequest: this.client.updateMediaMessage }, + { logger: makeBaileysLogger('error') as any, reuploadRequest: this.client.updateMediaMessage }, ); if (buffer) { @@ -3978,7 +3979,7 @@ export class BaileysStartupService extends ChannelStartupService { { key: msg?.key, message: msg?.message }, 'buffer', {}, - { logger: P({ level: 'error' }) as any, reuploadRequest: this.client.updateMediaMessage }, + { logger: makeBaileysLogger('error') as any, reuploadRequest: this.client.updateMediaMessage }, ); } catch { this.logger.error('Download Media failed, trying to retry in 5 seconds...'); @@ -4387,7 +4388,7 @@ export class BaileysStartupService extends ChannelStartupService { if ((cacheConf?.REDIS?.ENABLED && cacheConf?.REDIS?.URI !== '') || cacheConf?.LOCAL?.ENABLED) { if (await groupMetadataCache?.has(groupJid)) { - console.log(`Cache request for group: ${groupJid}`); + this.logger.verbose('Cache request for group: found'); const meta = await groupMetadataCache.get(groupJid); if (Date.now() - meta.timestamp > 3600000) { @@ -4397,7 +4398,7 @@ export class BaileysStartupService extends ChannelStartupService { return meta.data; } - console.log(`Cache request for group: ${groupJid} - not found`); + this.logger.verbose('Cache request for group: not found'); return await this.updateGroupMetadataCache(groupJid); } @@ -4938,7 +4939,7 @@ export class BaileysStartupService extends ChannelStartupService { } public async baileysSendNode(stanza: any) { - console.log('stanza', JSON.stringify(stanza)); + this.logger.verbose(`stanza ${stanza?.tag ?? ''}`); const response = await this.client.sendNode(stanza); return response; diff --git a/src/utils/log-privacy.ts b/src/utils/log-privacy.ts new file mode 100644 index 0000000000..d47edc5d24 --- /dev/null +++ b/src/utils/log-privacy.ts @@ -0,0 +1,61 @@ +import P from 'pino'; + +// What a log line may say about a message: its id, what kind of chat it is +// in, why something failed, and counts. Never its text, a phone number or JID, +// or a push name. + +/** The kind of chat a JID names, without the number in it. */ +export function jidKind(jid: unknown): string { + if (typeof jid !== 'string' || !jid) return 'unknown'; + if (jid.endsWith('@g.us')) return 'group'; + if (jid.endsWith('@lid') || jid.endsWith('@hosted.lid')) return 'lid'; + if (jid.endsWith('@s.whatsapp.net') || jid.endsWith('@c.us') || jid.endsWith('@hosted')) return 'user'; + if (jid.endsWith('@broadcast')) return 'broadcast'; + if (jid.endsWith('@newsletter')) return 'newsletter'; + return 'other'; +} + +/** Mask anything in a diagnostic string that looks like a JID or a phone number. */ +export function scrub(value: unknown): string { + return String(value ?? '') + .replace(/[^\s'"<>=,;:()[\]{}]+@[^\s'"<>=,;:()[\]{}]+/g, '[jid]') + .replace(/\d{6,}/g, '[number]'); +} + +const PRIMITIVE_FIELDS = ['messageType', 'isSessionRecordError', 'opName', 'count', 'attempt', 'retryCount']; +const ERROR_FIELDS = ['err', 'error', 'ackErr']; + +/** + * The fields of a Baileys log object that are safe to print. Baileys logs + * message keys, sender and author JIDs and whole stanzas (from, notify, body) + * at level error, e.g. "failed to decrypt message" and "error in handling + * message", so everything else is dropped. + */ +export function safeLogFields(obj: Record): Record { + const out: Record = {}; + if (typeof obj?.key?.id === 'string') out.messageId = obj.key.id; + const chat = obj?.key?.remoteJid ?? obj?.jid ?? obj?.sender; + if (chat) out.chatType = jidKind(chat); + for (const field of PRIMITIVE_FIELDS) { + const v = obj?.[field]; + if (typeof v === 'number' || typeof v === 'boolean') out[field] = v; + else if (typeof v === 'string') out[field] = scrub(v).slice(0, 80); + } + // Under `error`, not `err`: pino's err serializer would rebuild the object. + const e = ERROR_FIELDS.map((field) => obj?.[field]).find((v) => v && typeof v === 'object'); + if (e) out.error = { name: scrub(e.name ?? 'Error'), message: scrub(e.message).slice(0, 200) }; + return out; +} + +/** The pino logger Evolution hands Baileys: the level asked for, and only bounded fields. */ +export function makeBaileysLogger(level: string) { + return P({ + level, + formatters: { log: (obj) => safeLogFields(obj) }, + hooks: { + logMethod(args, method) { + return method.apply(this, args.map((a) => (typeof a === 'string' ? scrub(a) : a)) as Parameters); + }, + }, + }); +} From 5e2059e851f948a9695f4897cf2fec3e1f1a9f2c Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:03:16 +0300 Subject: [PATCH 018/157] test: media downloads leave through the instance's proxy A local CDN and local HTTP CONNECT and SOCKS5 proxies on 127.0.0.1, with the instance's proxy set the way /proxy/set and a connect set it. Downloads via getBase64FromMediaMessage (the /chat/getBase64FromMediaMessage route and the S3 storage path) and the messages.upsert webhook base64 path must reach the CDN through the proxy. On the current code they reach it directly: the socket gets the proxy, but Baileys downloads with fetch and only a `dispatcher` in the download options routes it. Co-Authored-By: Claude Opus 5.5 --- test/helpers/fake-prisma.ts | 1 + test/helpers/local-net.ts | 112 ++++++++++++++++++++++++++++ test/proxy/media-download.test.ts | 117 ++++++++++++++++++++++++++++++ 3 files changed, 230 insertions(+) create mode 100644 test/helpers/local-net.ts create mode 100644 test/proxy/media-download.test.ts diff --git a/test/helpers/fake-prisma.ts b/test/helpers/fake-prisma.ts index be03a1bc15..473fbd30cf 100644 --- a/test/helpers/fake-prisma.ts +++ b/test/helpers/fake-prisma.ts @@ -76,6 +76,7 @@ export function fakePrisma() { label: table('label', () => undefined), isOnWhatsapp: table('isOnWhatsapp', (r) => r.remoteJid), instance: table('instance', (r) => r.id), + proxy: table('proxy', (r) => r.instanceId), }; db.$transaction = async (ops: any) => (typeof ops === 'function' ? ops(db) : Promise.all(ops)); db.$queryRaw = async () => []; diff --git a/test/helpers/local-net.ts b/test/helpers/local-net.ts new file mode 100644 index 0000000000..0decdc5137 --- /dev/null +++ b/test/helpers/local-net.ts @@ -0,0 +1,112 @@ +// A media CDN and the two kinds of proxy Evolution supports (HTTP CONNECT and +// SOCKS5), all on 127.0.0.1. Each proxy records every connection it relays and +// refuses any target that is not 127.0.0.1, so nothing a test does can leave +// the machine through it. +import http from 'node:http'; +import net from 'node:net'; + +export type Listening = { port: number; log: string[]; close: () => Promise }; + +const LOCAL = '127.0.0.1'; + +function track(server: net.Server) { + const sockets = new Set(); + server.on('connection', (s) => { + sockets.add(s); + s.on('close', () => sockets.delete(s)); + }); + return () => + new Promise((resolve) => { + sockets.forEach((s) => s.destroy()); + server.close(() => resolve()); + }); +} + +async function listen(server: net.Server): Promise { + await new Promise((resolve) => server.listen(0, LOCAL, resolve)); + return (server.address() as net.AddressInfo).port; +} + +/** Serves `files` (path -> bytes) and logs `METHOD path` for every request. */ +export async function startCdn(files: Record): Promise { + const log: string[] = []; + const server = http.createServer((req, res) => { + log.push(`${req.method} ${req.url}`); + const body = files[req.url ?? '']; + if (!body) return void res.writeHead(404).end(); + res.writeHead(200, { 'content-type': 'application/octet-stream', 'content-length': body.length }).end(body); + }); + const close = track(server); + return { port: await listen(server), log, close }; +} + +function pipeTo(client: net.Socket, host: string, port: number, onOpen: () => void) { + const upstream = net.connect(port, host, () => { + onOpen(); + upstream.pipe(client); + client.pipe(upstream); + }); + upstream.on('error', () => client.destroy()); + client.on('error', () => upstream.destroy()); +} + +/** An HTTP proxy: CONNECT tunnels, and absolute-URI forwarding. Logs `CONNECT host:port` / `GET url`. */ +export async function startHttpProxy(): Promise { + const log: string[] = []; + const server = http.createServer((req, res) => { + log.push(`${req.method} ${req.url}`); + const target = new URL(req.url ?? ''); + if (target.hostname !== LOCAL) return void res.writeHead(403).end(); + const upstream = http.request( + { host: LOCAL, port: target.port, path: target.pathname + target.search, method: req.method, headers: req.headers }, + (up) => { + res.writeHead(up.statusCode ?? 502, up.headers); + up.pipe(res); + }, + ); + upstream.on('error', () => res.destroy()); + req.pipe(upstream); + }); + server.on('connect', (req, client: net.Socket, head) => { + log.push(`CONNECT ${req.url}`); + const [host, port] = (req.url ?? '').split(':'); + if (host !== LOCAL) return void client.end('HTTP/1.1 403 Forbidden\r\n\r\n'); + pipeTo(client, host, Number(port), () => { + client.write('HTTP/1.1 200 Connection Established\r\n\r\n'); + if (head?.length) client.unshift(head); + }); + }); + const close = track(server); + return { port: await listen(server), log, close }; +} + +/** A SOCKS5 proxy, no authentication, CONNECT only. Logs `SOCKS5 host:port`. */ +export async function startSocks5Proxy(): Promise { + const log: string[] = []; + const server = net.createServer((client) => { + client.once('data', (greeting) => { + if (greeting[0] !== 5) return void client.destroy(); + client.write(Buffer.from([5, 0])); + client.once('data', (req) => { + let host: string; + let offset: number; + if (req[3] === 1) { + host = [...req.subarray(4, 8)].join('.'); + offset = 8; + } else if (req[3] === 3) { + const len = req[4]; + host = req.subarray(5, 5 + len).toString(); + offset = 5 + len; + } else return void client.destroy(); + const port = req.readUInt16BE(offset); + log.push(`SOCKS5 ${host}:${port}`); + const reply = (code: number) => Buffer.from([5, code, 0, 1, 0, 0, 0, 0, 0, 0]); + if (req[1] !== 1 || host !== LOCAL) return void client.end(reply(2)); + pipeTo(client, host, port, () => client.write(reply(0))); + }); + }); + client.on('error', () => client.destroy()); + }); + const close = track(server); + return { port: await listen(server), log, close }; +} diff --git a/test/proxy/media-download.test.ts b/test/proxy/media-download.test.ts new file mode 100644 index 0000000000..443fd38e9c --- /dev/null +++ b/test/proxy/media-download.test.ts @@ -0,0 +1,117 @@ +// A deployment can give each linked account its own proxy exit (Evolution's per-instance +// proxy, /proxy/set, the Proxy table). The socket leaves through it; media +// downloads must leave through the same exit, or the account's media is fetched +// from the server's own IP while its messages come from the person's. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { readFile, rm } from 'node:fs/promises'; + +import { encryptedStream } from 'baileys'; +import { getGlobalDispatcher, MockAgent, setGlobalDispatcher } from 'undici'; +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from 'vitest'; + +import { deliver, makeService } from '../helpers/baileys-service'; +import { emitted } from '../helpers/fake-server-module'; +import { type Listening, startCdn, startHttpProxy, startSocks5Proxy } from '../helpers/local-net'; + +const PLAIN = Buffer.from('a photo, as the person sent it '.repeat(200)); +const PATH = '/v/t62.7118-24/media.enc'; + +let cdn: Listening; +let mediaKey: Uint8Array; + +// Refuse any connection that is not to 127.0.0.1, whatever path a request takes. +const previousDispatcher = getGlobalDispatcher(); +const guard = new MockAgent(); +guard.disableNetConnect(); +guard.enableNetConnect((host: string) => host.startsWith('127.0.0.1:')); + +beforeAll(async () => { + setGlobalDispatcher(guard); + // Encrypt with Baileys' own upload path, so the CDN serves exactly what WhatsApp would. + const enc = await encryptedStream(PLAIN, 'image', {}); + mediaKey = enc.mediaKey; + const body = await readFile(enc.encFilePath); + await rm(enc.encFilePath, { force: true }); + cdn = await startCdn({ [PATH]: body }); +}); + +afterAll(async () => { + await cdn.close(); + setGlobalDispatcher(previousDispatcher); + await guard.close(); +}); + +const proxies: Listening[] = []; +beforeEach(() => { + cdn.log.splice(0); + emitted.splice(0); +}); +afterEach(async () => { + await Promise.all(proxies.splice(0).map((p) => p.close())); +}); + +const imageMessage = () => ({ + key: { remoteJid: '972500000001@s.whatsapp.net', fromMe: false, id: '3EB0BBBBBBBBBBBBBBBB' }, + message: { + imageMessage: { + url: `http://127.0.0.1:${cdn.port}${PATH}`, + mediaKey, + mimetype: 'image/jpeg', + fileLength: PLAIN.length, + }, + }, + messageTimestamp: 1_700_000_000, + pushName: 'Sender', +}); + +/** Set the instance's proxy the way /proxy/set does, then load it the way a connect does. */ +async function serviceBehind(protocol: 'http' | 'socks5') { + const proxy = protocol === 'http' ? await startHttpProxy() : await startSocks5Proxy(); + proxies.push(proxy); + const made = await makeService(); + await made.service.setProxy({ + enabled: true, + host: '127.0.0.1', + port: String(proxy.port), + protocol, + username: '', + password: '', + }); + await made.service.loadProxy(); + return { ...made, proxy }; +} + +const tunnelled = (protocol: 'http' | 'socks5') => + protocol === 'http' ? `CONNECT 127.0.0.1:${cdn.port}` : `SOCKS5 127.0.0.1:${cdn.port}`; + +describe('media downloads leave through the instance proxy', () => { + it('control: with no proxy, getBase64FromMediaMessage fetches the CDN directly', async () => { + const { service } = await makeService(); + const media = await service.getBase64FromMediaMessage({ message: imageMessage() }); + expect(Buffer.from(media.base64, 'base64').equals(PLAIN)).toBe(true); + expect(cdn.log).toEqual([`GET ${PATH}`]); + }); + + for (const protocol of ['http', 'socks5'] as const) { + it(`getBase64FromMediaMessage (/chat/getBase64FromMediaMessage, and S3 storage) goes through a ${protocol} proxy`, async () => { + const { service, proxy } = await serviceBehind(protocol); + const media = await service.getBase64FromMediaMessage({ message: imageMessage() }); + expect(Buffer.from(media.base64, 'base64').equals(PLAIN)).toBe(true); + expect(proxy.log).toEqual([tunnelled(protocol)]); + expect(cdn.log).toEqual([`GET ${PATH}`]); + }); + } + + it('the messages.upsert webhook base64 download goes through the proxy', async () => { + const { service, ev, proxy } = await serviceBehind('http'); + Object.assign(service.localWebhook, { enabled: true, webhookBase64: true }); + await deliver(service, ev, { 'messages.upsert': { messages: [imageMessage()], type: 'notify' } }); + const upsert = emitted.find((e) => e.event === 'messages.upsert'); + expect(Buffer.from(upsert?.data?.message?.base64 ?? '', 'base64').equals(PLAIN)).toBe(true); + expect(proxy.log).toEqual([tunnelled('http')]); + expect(cdn.log).toEqual([`GET ${PATH}`]); + }); +}); From 2c2193fa46a3620c30adc2dcaf1f382d55efb3c3 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:04:39 +0300 Subject: [PATCH 019/157] fix(baileys): download media through the instance's proxy Baileys 7 downloads media with fetch, which ignores the socket's agent and fetchAgent and routes only through a `dispatcher` in the download options. Every downloadMediaMessage and downloadContentFromMessage call in the Baileys service (getBase64FromMediaMessage and its fallback, and the webhook base64 downloads in messages.upsert and after sending) now passes the instance's proxy as an undici dispatcher (ProxyAgent for http/https, fetch-socks for socks4/socks5), built by the existing makeProxyAgentUndici. It is the socket's own fetchAgent when the socket was created with one, so a proxyscrape exit is the same exit, and it is rebuilt when /proxy/set changes the proxy. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 47 ++++++++++++++++--- 1 file changed, 41 insertions(+), 6 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index f8a88d383c..5bc4ab27b1 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -252,6 +252,8 @@ export class BaileysStartupService extends ChannelStartupService { private endSession = false; private logBaileys = this.configService.get('LOG').BAILEYS; private eventProcessingQueue: Promise = Promise.resolve(); + // The dispatcher media downloads go through: the instance's proxy, the same exit as the socket. + private mediaProxy?: { key: string; dispatcher: ReturnType }; // Cache TTL constants (in seconds) private readonly MESSAGE_CACHE_TTL_SECONDS = 5 * 60; // 5 minutes - avoid duplicate message processing @@ -642,6 +644,8 @@ export class BaileysStartupService extends ChannelStartupService { } } + if (options?.fetchAgent) this.mediaProxy = { key: this.proxyKey(), dispatcher: options.fetchAgent }; + const socketConfig: UserFacingSocketConfig = { ...options, version, @@ -1519,7 +1523,7 @@ export class BaileysStartupService extends ChannelStartupService { const buffer = await downloadMediaMessage( { key: received.key, message: received?.message }, 'buffer', - {}, + this.mediaDownloadOptions(), { logger: makeBaileysLogger('error') as any, reuploadRequest: this.client.updateMediaMessage }, ); @@ -1530,7 +1534,7 @@ export class BaileysStartupService extends ChannelStartupService { const buffer = await downloadMediaMessage( { key: received.key, message: received?.message }, 'buffer', - {}, + this.mediaDownloadOptions(), { logger: makeBaileysLogger('error') as any, reuploadRequest: this.client.updateMediaMessage }, ); @@ -2594,7 +2598,7 @@ export class BaileysStartupService extends ChannelStartupService { const buffer = await downloadMediaMessage( { key: messageRaw.key, message: messageRaw?.message }, 'buffer', - {}, + this.mediaDownloadOptions(), { logger: makeBaileysLogger('error') as any, reuploadRequest: this.client.updateMediaMessage }, ); @@ -2605,7 +2609,7 @@ export class BaileysStartupService extends ChannelStartupService { const buffer = await downloadMediaMessage( { key: messageRaw.key, message: messageRaw?.message }, 'buffer', - {}, + this.mediaDownloadOptions(), { logger: makeBaileysLogger('error') as any, reuploadRequest: this.client.updateMediaMessage }, ); @@ -3913,6 +3917,37 @@ export class BaileysStartupService extends ChannelStartupService { return map[mediaType] || null; } + private proxyKey() { + const { protocol, host, port, username, password } = this.localProxy; + return JSON.stringify([protocol, host, port, username, password]); + } + + /** + * Options for Baileys' media downloads. Baileys downloads with fetch, which + * ignores the socket's `agent`/`fetchAgent` and routes only through a + * `dispatcher`, so without this the media leaves from the server's own IP + * while the messages leave through the instance's proxy. + */ + private mediaDownloadOptions() { + if (!this.localProxy?.enabled || !this.localProxy.host) return {}; + const key = this.proxyKey(); + if (this.mediaProxy?.key !== key) { + // A proxyscrape host is a list the socket picked one exit from; only the socket's own dispatcher is that exit. + if (this.localProxy.host.includes('proxyscrape')) return {}; + this.mediaProxy = { + key, + dispatcher: makeProxyAgentUndici({ + host: this.localProxy.host, + port: this.localProxy.port, + protocol: this.localProxy.protocol, + username: this.localProxy.username, + password: this.localProxy.password, + }), + }; + } + return { options: { dispatcher: this.mediaProxy.dispatcher } as RequestInit }; + } + public async getBase64FromMediaMessage(data: getBase64FromMediaMessageDto, getBuffer = false) { try { const m = data?.message; @@ -3978,7 +4013,7 @@ export class BaileysStartupService extends ChannelStartupService { buffer = await downloadMediaMessage( { key: msg?.key, message: msg?.message }, 'buffer', - {}, + this.mediaDownloadOptions(), { logger: makeBaileysLogger('error') as any, reuploadRequest: this.client.updateMediaMessage }, ); } catch { @@ -3995,7 +4030,7 @@ export class BaileysStartupService extends ChannelStartupService { url: `https://mmg.whatsapp.net${msg?.message?.[mediaType]?.directPath}`, }, await this.mapMediaType(mediaType), - {}, + this.mediaDownloadOptions(), ); const chunks = []; for await (const chunk of media) { From 37775a9cc438bbcc1e9b78d90434bcb61102ff1f Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:08:14 +0300 Subject: [PATCH 020/157] test: media uploads work on an instance with a proxy, and leave through it Runs Evolution's real connect with makeWASocket replaced by a spy, then runs Baileys' own getWAUploadToServer with the socket config Evolution built, against a local HTTPS media host (a test certificate added to the default CAs, verification on) through local HTTP CONNECT and SOCKS5 proxies. Today the upload fails on every host: Baileys uploads with http.request under Node and passes fetchAgent as its agent, and Evolution's fetchAgent is an undici dispatcher, which http.request refuses with ERR_INVALID_ARG_TYPE. A connection guard in the helpers refuses any socket not to the loopback address and records what tried. Co-Authored-By: Claude Opus 5.5 --- test/fixtures/tls/local.crt | 13 ++++ test/fixtures/tls/local.key | 5 ++ test/helpers/connect.ts | 46 ++++++++++++ test/helpers/fake-prisma.ts | 2 + test/helpers/local-net.ts | 83 ++++++++++++++++++--- test/proxy/media-upload.test.ts | 126 ++++++++++++++++++++++++++++++++ 6 files changed, 266 insertions(+), 9 deletions(-) create mode 100644 test/fixtures/tls/local.crt create mode 100644 test/fixtures/tls/local.key create mode 100644 test/helpers/connect.ts create mode 100644 test/proxy/media-upload.test.ts diff --git a/test/fixtures/tls/local.crt b/test/fixtures/tls/local.crt new file mode 100644 index 0000000000..52b8c1add4 --- /dev/null +++ b/test/fixtures/tls/local.crt @@ -0,0 +1,13 @@ +-----BEGIN CERTIFICATE----- +MIIB3TCCAYKgAwIBAgIUcpxdfmimev0xnVOBp30LsQ3fXmkwCgYIKoZIzj0EAwIw +HTEbMBkGA1UEAwwSY2lyY2xlcy10ZXN0LWxvY2FsMCAXDTI2MDkyNzExMDY0OFoY +DzIxMjYwOTAzMTEwNjQ4WjAdMRswGQYDVQQDDBJjaXJjbGVzLXRlc3QtbG9jYWww +WTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAARypC7eLcffgdSa94MD28JqueDQe+lq +FTwwhi4T+FXlPhytAbSgH5P8hBRWnmLWqbxISl3Bv9ybWhPzcVZR/xk0o4GdMIGa +MB0GA1UdDgQWBBRO2yAfPb8kGAXLFunbY4TswqvS9jAfBgNVHSMEGDAWgBRO2yAf +Pb8kGAXLFunbY4TswqvS9jAPBgNVHRMBAf8EBTADAQH/MEcGA1UdEQRAMD6HBH8A +AAGCEHdlYi53aGF0c2FwcC5jb22CGXJhdy5naXRodWJ1c2VyY29udGVudC5jb22C +CWxvY2FsaG9zdDAKBggqhkjOPQQDAgNJADBGAiEAmB9pBffyItARIOB/IxD/I6xd +WjNkCeYWmvsBbVjqHpYCIQCzGJGHTsKMIh+gJyPw879CfwLbeTJ30JqcXNaJp/zD +aw== +-----END CERTIFICATE----- diff --git a/test/fixtures/tls/local.key b/test/fixtures/tls/local.key new file mode 100644 index 0000000000..ef90c85876 --- /dev/null +++ b/test/fixtures/tls/local.key @@ -0,0 +1,5 @@ +-----BEGIN PRIVATE KEY----- +MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQg5EGYsjHtP6Aw3DcU +QSKXDpuyA0v0t9LVzD8gAM9PRqShRANCAARypC7eLcffgdSa94MD28JqueDQe+lq +FTwwhi4T+FXlPhytAbSgH5P8hBRWnmLWqbxISl3Bv9ybWhPzcVZR/xk0 +-----END PRIVATE KEY----- diff --git a/test/helpers/connect.ts b/test/helpers/connect.ts new file mode 100644 index 0000000000..a1a5e91647 --- /dev/null +++ b/test/helpers/connect.ts @@ -0,0 +1,46 @@ +// Run Evolution's real connect (connectToWhatsapp -> createClient) with the +// socket factory replaced, so a test can read the socket config Evolution +// builds without opening a WebSocket. The test file mocks `baileys` so that +// makeWASocket is `socketSpy`, which returns fakeSocket(). +import { EventEmitter } from 'node:events'; + +import { initAuthCreds, makeEventBuffer } from 'baileys'; +import P from 'pino'; + +import { makeService } from './baileys-service'; + +export function fakeSocket() { + return { + ev: makeEventBuffer(P({ level: 'silent' }) as any), + ws: new EventEmitter(), + end: () => undefined, + }; +} + +export type ProxyProtocol = 'http' | 'socks5'; + +/** Set the proxy the way /proxy/set does, then connect the way Evolution does. Returns the socket config. */ +export async function connectBehind(socketSpy: { mock: { calls: any[][] } }, proxy?: { protocol: ProxyProtocol; port: number }) { + const { service } = await makeService(); + if (proxy) { + await service.setProxy({ + enabled: true, + host: '127.0.0.1', + port: String(proxy.port), + protocol: proxy.protocol, + username: '', + password: '', + }); + } + // The auth state is files on disk under the instances directory; nothing about it touches the network. + service.defineAuthState = async () => ({ + state: { creds: initAuthCreds(), keys: { get: async () => ({}), set: async () => undefined } }, + saveCreds: async () => undefined, + removeCreds: async () => undefined, + }); + const before = socketSpy.mock.calls.length; + await service.connectToWhatsapp(); + const config = socketSpy.mock.calls[before]?.[0]; + if (!config) throw new Error('connectToWhatsapp did not create a socket'); + return { service, config }; +} diff --git a/test/helpers/fake-prisma.ts b/test/helpers/fake-prisma.ts index 473fbd30cf..d35b70e411 100644 --- a/test/helpers/fake-prisma.ts +++ b/test/helpers/fake-prisma.ts @@ -77,6 +77,8 @@ export function fakePrisma() { isOnWhatsapp: table('isOnWhatsapp', (r) => r.remoteJid), instance: table('instance', (r) => r.id), proxy: table('proxy', (r) => r.instanceId), + webhook: table('webhook', (r) => r.instanceId), + chatwoot: table('chatwoot', (r) => r.instanceId), }; db.$transaction = async (ops: any) => (typeof ops === 'function' ? ops(db) : Promise.all(ops)); db.$queryRaw = async () => []; diff --git a/test/helpers/local-net.ts b/test/helpers/local-net.ts index 0decdc5137..96345a26f5 100644 --- a/test/helpers/local-net.ts +++ b/test/helpers/local-net.ts @@ -1,9 +1,12 @@ // A media CDN and the two kinds of proxy Evolution supports (HTTP CONNECT and // SOCKS5), all on 127.0.0.1. Each proxy records every connection it relays and -// refuses any target that is not 127.0.0.1, so nothing a test does can leave -// the machine through it. +// refuses any target that is not 127.0.0.1 (or one it was told to remap to a +// local server), so nothing a test does can leave the machine through it. +import { readFileSync } from 'node:fs'; import http from 'node:http'; +import https from 'node:https'; import net from 'node:net'; +import tls from 'node:tls'; export type Listening = { port: number; log: string[]; close: () => Promise }; @@ -50,8 +53,17 @@ function pipeTo(client: net.Socket, host: string, port: number, onOpen: () => vo client.on('error', () => upstream.destroy()); } +/** `host:port` the client asked for -> the 127.0.0.1 `host:port` to connect instead, or undefined to refuse. */ +type Remap = Record; +function resolveTarget(target: string, remap: Remap = {}): [string, number] | undefined { + const to = remap[target] ?? target; + const i = to.lastIndexOf(':'); + const host = to.slice(0, i); + return host === LOCAL ? [host, Number(to.slice(i + 1))] : undefined; +} + /** An HTTP proxy: CONNECT tunnels, and absolute-URI forwarding. Logs `CONNECT host:port` / `GET url`. */ -export async function startHttpProxy(): Promise { +export async function startHttpProxy(opts: { remap?: Remap } = {}): Promise { const log: string[] = []; const server = http.createServer((req, res) => { log.push(`${req.method} ${req.url}`); @@ -69,9 +81,9 @@ export async function startHttpProxy(): Promise { }); server.on('connect', (req, client: net.Socket, head) => { log.push(`CONNECT ${req.url}`); - const [host, port] = (req.url ?? '').split(':'); - if (host !== LOCAL) return void client.end('HTTP/1.1 403 Forbidden\r\n\r\n'); - pipeTo(client, host, Number(port), () => { + const target = resolveTarget(req.url ?? '', opts.remap); + if (!target) return void client.end('HTTP/1.1 403 Forbidden\r\n\r\n'); + pipeTo(client, target[0], target[1], () => { client.write('HTTP/1.1 200 Connection Established\r\n\r\n'); if (head?.length) client.unshift(head); }); @@ -81,7 +93,7 @@ export async function startHttpProxy(): Promise { } /** A SOCKS5 proxy, no authentication, CONNECT only. Logs `SOCKS5 host:port`. */ -export async function startSocks5Proxy(): Promise { +export async function startSocks5Proxy(opts: { remap?: Remap } = {}): Promise { const log: string[] = []; const server = net.createServer((client) => { client.once('data', (greeting) => { @@ -101,8 +113,9 @@ export async function startSocks5Proxy(): Promise { const port = req.readUInt16BE(offset); log.push(`SOCKS5 ${host}:${port}`); const reply = (code: number) => Buffer.from([5, code, 0, 1, 0, 0, 0, 0, 0, 0]); - if (req[1] !== 1 || host !== LOCAL) return void client.end(reply(2)); - pipeTo(client, host, port, () => client.write(reply(0))); + const target = resolveTarget(`${host}:${port}`, opts.remap); + if (req[1] !== 1 || !target) return void client.end(reply(2)); + pipeTo(client, target[0], target[1], () => client.write(reply(0))); }); }); client.on('error', () => client.destroy()); @@ -110,3 +123,55 @@ export async function startSocks5Proxy(): Promise { const close = track(server); return { port: await listen(server), log, close }; } + +const TLS_DIR = new URL('../fixtures/tls/', import.meta.url); +/** A self-signed test certificate for 127.0.0.1, web.whatsapp.com and raw.githubusercontent.com. */ +export const testTls = { + key: readFileSync(new URL('local.key', TLS_DIR)), + cert: readFileSync(new URL('local.crt', TLS_DIR)), +}; + +/** Trust the test certificate process-wide (added to the default CAs, verification stays on). Returns undo. */ +export function trustTestCertificate() { + const before = tls.getCACertificates('default'); + tls.setDefaultCACertificates([...before, testTls.cert.toString()]); + return () => tls.setDefaultCACertificates(before); +} + +/** An HTTPS server on 127.0.0.1 with the test certificate. Logs `METHOD path` for every request. */ +export async function startHttpsServer( + handler: (req: http.IncomingMessage, body: Buffer, res: http.ServerResponse) => void, +): Promise { + const log: string[] = []; + const server = https.createServer(testTls, (req, res) => { + log.push(`${req.method} ${req.url}`); + const chunks: Buffer[] = []; + req.on('data', (c) => chunks.push(c)); + req.on('end', () => handler(req, Buffer.concat(chunks), res)); + }); + const close = track(server); + return { port: await listen(server), log, close }; +} + +/** + * Refuse every socket connection that is not to the loopback address, in this + * process, whatever library opens it (http, https, axios, undici, ws). The + * refused `host:port`s are recorded, so a test can say what tried to leave. + * Returns undo. + */ +export function loopbackOnly() { + const refused: string[] = []; + const original = net.Socket.prototype.connect; + net.Socket.prototype.connect = function (this: net.Socket, ...args: any[]) { + const first = Array.isArray(args[0]) ? args[0][0] : args[0]; + const opts = typeof first === 'object' && first !== null ? first : { port: first, host: args[1] }; + const host = opts.path ? LOCAL : (opts.host ?? 'localhost'); + if (!['127.0.0.1', 'localhost', '::1'].includes(host)) { + refused.push(`${host}:${opts.port}`); + process.nextTick(() => this.destroy(new Error(`test: refused a connection to ${host}:${opts.port}`))); + return this; + } + return original.apply(this, args as any); + } as any; + return { refused, restore: () => void (net.Socket.prototype.connect = original) }; +} diff --git a/test/proxy/media-upload.test.ts b/test/proxy/media-upload.test.ts new file mode 100644 index 0000000000..c715ed8dea --- /dev/null +++ b/test/proxy/media-upload.test.ts @@ -0,0 +1,126 @@ +// Sending media: Baileys uploads the encrypted file to a WhatsApp media host +// with the socket config Evolution built. On an instance with a proxy the upload +// must work, and must leave through that proxy like the rest of the account. +import { vi } from 'vitest'; + +const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + return { ...orig, default: socketSpy, makeWASocket: socketSpy }; +}); +// Pinned here: this file is about uploads (the version fetch has its own test). +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); + +import { createHash } from 'node:crypto'; +import { mkdtemp, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { DEFAULT_CONNECTION_CONFIG, getWAUploadToServer } from 'baileys'; +import P from 'pino'; +import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest'; + +import { connectBehind, fakeSocket, type ProxyProtocol } from '../helpers/connect'; +import { + type Listening, + loopbackOnly, + startHttpProxy, + startHttpsServer, + startSocks5Proxy, + trustTestCertificate, +} from '../helpers/local-net'; + +socketSpy.mockImplementation(fakeSocket); + +const ENCRYPTED = Buffer.from('an encrypted photo, as Baileys uploads it '.repeat(300)); +const SHA_B64 = createHash('sha256').update(ENCRYPTED).digest('base64'); + +let guard: ReturnType; +let untrust: () => void; +let mediaHost: Listening; +let received: Buffer[]; +let dir: string; +let filePath: string; +const proxies: Listening[] = []; + +beforeAll(async () => { + guard = loopbackOnly(); + untrust = trustTestCertificate(); + mediaHost = await startHttpsServer((req, body, res) => { + received.push(body); + res.writeHead(200, { 'content-type': 'application/json' }); + res.end(JSON.stringify({ url: `https://127.0.0.1:${mediaHost.port}/m/1`, direct_path: '/m/1' })); + }); + dir = await mkdtemp(join(tmpdir(), 'evo-upload-')); + filePath = join(dir, 'image-enc'); + await writeFile(filePath, ENCRYPTED); +}); + +afterAll(async () => { + await mediaHost.close(); + await rm(dir, { recursive: true, force: true }); + untrust(); + guard.restore(); +}); + +afterEach(async () => { + await Promise.all(proxies.splice(0).map((p) => p.close())); +}); + +// Baileys logs each host's failure at warn and throws only "failed on all hosts"; keep the cause. +const warnings: string[] = []; +const logger = Object.assign(P({ level: 'silent' }), { + warn: (obj: any, msg?: string) => void warnings.push(`${msg}: ${String(obj?.trace ?? '').split('\n')[0]}`), +}); + +/** Baileys' own upload, built from the socket config the way makeWASocket builds it. */ +function uploaderFor(config: any) { + const merged = { ...DEFAULT_CONNECTION_CONFIG, ...config, logger }; + return getWAUploadToServer(merged, async () => ({ + hosts: [{ hostname: `127.0.0.1:${mediaHost.port}`, maxContentLengthBytes: 1e9 }], + auth: 'test-auth', + ttl: 3600, + fetchDate: new Date(), + })); +} + +async function upload(config: any) { + received = []; + mediaHost.log.splice(0); + warnings.splice(0); + return uploaderFor(config)(filePath, { mediaType: 'image', fileEncSha256B64: SHA_B64, timeoutMs: 10_000 }).catch( + (error) => { + throw new Error(`${error.message}. ${warnings.join(' | ')}`); + }, + ); +} + +describe('media uploads on an instance with a proxy', () => { + it('control: with no proxy, the upload reaches the media host directly', async () => { + const { config } = await connectBehind(socketSpy); + const result = await upload(config); + expect(result).toEqual({ mediaUrl: `https://127.0.0.1:${mediaHost.port}/m/1`, directPath: '/m/1', meta_hmac: undefined, fbid: undefined, ts: undefined }); + expect(Buffer.concat(received).equals(ENCRYPTED)).toBe(true); + }); + + for (const protocol of ['http', 'socks5'] as ProxyProtocol[]) { + it(`works, and leaves through a ${protocol} proxy`, async () => { + const proxy = protocol === 'http' ? await startHttpProxy() : await startSocks5Proxy(); + proxies.push(proxy); + const { config } = await connectBehind(socketSpy, { protocol, port: proxy.port }); + const result = await upload(config); + expect(result).toEqual({ mediaUrl: `https://127.0.0.1:${mediaHost.port}/m/1`, directPath: '/m/1', meta_hmac: undefined, fbid: undefined, ts: undefined }); + expect(Buffer.concat(received).equals(ENCRYPTED)).toBe(true); + expect(mediaHost.log).toHaveLength(1); + expect(proxy.log).toEqual([protocol === 'http' ? `CONNECT 127.0.0.1:${mediaHost.port}` : `SOCKS5 127.0.0.1:${mediaHost.port}`]); + }); + } + + it('nothing tried to leave 127.0.0.1', () => { + expect(guard.refused).toEqual([]); + }); +}); From c5435efe2e433e5c13109a30c371199bfd276e47 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:08:31 +0300 Subject: [PATCH 021/157] fix(baileys): give Baileys an http agent for uploads through the proxy Under Node, Baileys uploads media with http.request and passes the socket's fetchAgent as its agent. Evolution set fetchAgent to an undici dispatcher, which http.request refuses, so every media send on an instance with a proxy failed with "Media upload failed on all hosts". fetchAgent is now the same kind of agent as the websocket's (makeProxyAgent: https-proxy-agent or socks-proxy-agent), and the undici dispatcher is kept for downloads only. The proxy is resolved once per connect, so a proxyscrape exit stays one exit for the socket, uploads and downloads. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 36 +++++++++---------- 1 file changed, 17 insertions(+), 19 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 5bc4ab27b1..40b5cceb68 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -613,38 +613,36 @@ export class BaileysStartupService extends ChannelStartupService { if (this.localProxy?.enabled) { this.logger.info('Proxy enabled: ' + this.localProxy?.host); + let proxy: Parameters[0]; + if (this.localProxy?.host?.includes('proxyscrape')) { try { const response = await axios.get(this.localProxy?.host); const text = response.data; const proxyUrls = text.split('\r\n'); const rand = Math.floor(Math.random() * Math.floor(proxyUrls.length)); - const proxyUrl = 'http://' + proxyUrls[rand]; - options = { agent: makeProxyAgent(proxyUrl), fetchAgent: makeProxyAgentUndici(proxyUrl) }; + proxy = 'http://' + proxyUrls[rand]; } catch { this.localProxy.enabled = false; } } else { - options = { - agent: makeProxyAgent({ - host: this.localProxy.host, - port: this.localProxy.port, - protocol: this.localProxy.protocol, - username: this.localProxy.username, - password: this.localProxy.password, - }), - fetchAgent: makeProxyAgentUndici({ - host: this.localProxy.host, - port: this.localProxy.port, - protocol: this.localProxy.protocol, - username: this.localProxy.username, - password: this.localProxy.password, - }), + proxy = { + host: this.localProxy.host, + port: this.localProxy.port, + protocol: this.localProxy.protocol, + username: this.localProxy.username, + password: this.localProxy.password, }; } - } - if (options?.fetchAgent) this.mediaProxy = { key: this.proxyKey(), dispatcher: options.fetchAgent }; + if (proxy) { + // Baileys uploads with http.request under Node, which takes an http agent + // (fetchAgent), not an undici dispatcher. Downloads use fetch, which takes + // only a dispatcher: see mediaDownloadOptions. All three share one exit. + options = { agent: makeProxyAgent(proxy), fetchAgent: makeProxyAgent(proxy) }; + this.mediaProxy = { key: this.proxyKey(), dispatcher: makeProxyAgentUndici(proxy) }; + } + } const socketConfig: UserFacingSocketConfig = { ...options, From 5187ea2940bd5f2f71355cf094fbd021f1229f6d Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:09:17 +0300 Subject: [PATCH 022/157] test: the WhatsApp Web version fetch leaves through the instance's proxy Runs Evolution's real connect with a proxy set and makeWASocket replaced by a spy. The proxy maps web.whatsapp.com and raw.githubusercontent.com to local HTTPS servers, and the connection guard refuses anything else. Today both the sw.js request (axios) and the fallback to Baileys' version file (fetch) go straight out, so the guard records web.whatsapp.com:443 and raw.githubusercontent.com:443 and the proxy sees nothing. Co-Authored-By: Claude Opus 5.5 --- test/proxy/version-fetch.test.ts | 105 +++++++++++++++++++++++++++++++ 1 file changed, 105 insertions(+) create mode 100644 test/proxy/version-fetch.test.ts diff --git a/test/proxy/version-fetch.test.ts b/test/proxy/version-fetch.test.ts new file mode 100644 index 0000000000..343b08cf33 --- /dev/null +++ b/test/proxy/version-fetch.test.ts @@ -0,0 +1,105 @@ +// Every connect starts by asking WhatsApp Web for its current version +// (fetchLatestWaWebVersion: web.whatsapp.com/sw.js, falling back to Baileys' +// version file on GitHub). On an instance with a proxy that request must leave +// through the proxy too, or every account's connect is announced from the +// server's own IP. The proxy here maps those two hosts to local HTTPS servers. +import { vi } from 'vitest'; + +const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + return { ...orig, default: socketSpy, makeWASocket: socketSpy }; +}); + +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from 'vitest'; + +import { connectBehind, fakeSocket, type ProxyProtocol } from '../helpers/connect'; +import { + type Listening, + loopbackOnly, + startHttpProxy, + startHttpsServer, + startSocks5Proxy, + trustTestCertificate, +} from '../helpers/local-net'; + +socketSpy.mockImplementation(fakeSocket); + +const BAILEYS_DEFAULTS = '/WhiskeySockets/Baileys/master/src/Defaults/index.ts'; + +let guard: ReturnType; +let untrust: () => void; +let web: Listening; +let github: Listening; +let swHasRevision = true; +const proxies: Listening[] = []; + +beforeAll(async () => { + guard = loopbackOnly(); + untrust = trustTestCertificate(); + web = await startHttpsServer((req, _body, res) => { + if (req.url !== '/sw.js') return void res.writeHead(404).end(); + res.writeHead(200, { 'content-type': 'text/javascript' }); + res.end(swHasRevision ? 'self.__swData=JSON.parse("{\\"client_revision\\":1027654321}");' : 'self.__swData={};'); + }); + github = await startHttpsServer((req, _body, res) => { + if (req.url !== BAILEYS_DEFAULTS) return void res.writeHead(404).end(); + res.writeHead(200, { 'content-type': 'text/plain' }); + res.end(['// 1', '// 2', '// 3', '// 4', '// 5', '// 6', 'const version = [2, 3000, 1011111111]', ''].join('\n')); + }); +}); + +afterAll(async () => { + await web.close(); + await github.close(); + untrust(); + guard.restore(); +}); + +beforeEach(() => { + swHasRevision = true; + guard.refused.splice(0); + web.log.splice(0); + github.log.splice(0); +}); + +afterEach(async () => { + await Promise.all(proxies.splice(0).map((p) => p.close())); +}); + +async function proxyFor(protocol: ProxyProtocol) { + const remap = { + 'web.whatsapp.com:443': `127.0.0.1:${web.port}`, + 'raw.githubusercontent.com:443': `127.0.0.1:${github.port}`, + }; + const proxy = protocol === 'http' ? await startHttpProxy({ remap }) : await startSocks5Proxy({ remap }); + proxies.push(proxy); + return proxy; +} + +const via = (protocol: ProxyProtocol, host: string) => (protocol === 'http' ? `CONNECT ${host}:443` : `SOCKS5 ${host}:443`); + +describe('the WhatsApp Web version fetch on connect', () => { + for (const protocol of ['http', 'socks5'] as ProxyProtocol[]) { + it(`leaves through a ${protocol} proxy`, async () => { + const proxy = await proxyFor(protocol); + const { config } = await connectBehind(socketSpy, { protocol, port: proxy.port }); + expect(guard.refused).toEqual([]); + expect(proxy.log).toEqual([via(protocol, 'web.whatsapp.com')]); + expect(web.log).toEqual(['GET /sw.js']); + expect(config.version).toEqual([2, 3000, 1027654321]); + }); + } + + it('falls back to Baileys version file through the proxy too', async () => { + swHasRevision = false; + const proxy = await proxyFor('http'); + const { config } = await connectBehind(socketSpy, { protocol: 'http', port: proxy.port }); + expect(guard.refused).toEqual([]); + expect(proxy.log).toEqual([via('http', 'web.whatsapp.com'), via('http', 'raw.githubusercontent.com')]); + expect(github.log).toEqual([`GET ${BAILEYS_DEFAULTS}`]); + expect(config.version).toEqual([2, 3000, 1011111111]); + }); +}); From 7435f41060cfbb298d9249c406b5c1d36f20b613 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:09:46 +0300 Subject: [PATCH 023/157] fix(baileys): fetch the WhatsApp Web version through the instance's proxy createClient asked web.whatsapp.com for its version (and, failing that, GitHub for Baileys' version file) before resolving the instance's proxy, so every account's connect was preceded by a request from the server's own IP. The proxy is now resolved first, the sw.js request (axios) goes through the socket's proxy agent with axios' own env proxy off, and the Baileys fallback (fetch) through the undici dispatcher. Without a proxy nothing changes. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 20 +++++++++++-------- src/utils/fetchLatestWaWebVersion.ts | 10 +++++++--- 2 files changed, 19 insertions(+), 11 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 40b5cceb68..682511b7bd 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -600,14 +600,6 @@ export class BaileysStartupService extends ChannelStartupService { this.logger.info(`Browser: ${browser}`); } - const baileysVersion = await fetchLatestWaWebVersion({}); - const version = baileysVersion.version; - const log = `Baileys version: ${version.join('.')}`; - - this.logger.info(log); - - this.logger.info(`Group Ignore: ${this.localSettings.groupsIgnore}`); - let options; if (this.localProxy?.enabled) { @@ -644,6 +636,18 @@ export class BaileysStartupService extends ChannelStartupService { } } + // The version request precedes every connect, so it leaves through the same exit as the socket. + const baileysVersion = await fetchLatestWaWebVersion( + options ? { httpsAgent: options.fetchAgent, proxy: false } : {}, + options ? ({ dispatcher: this.mediaProxy.dispatcher } as RequestInit) : {}, + ); + const version = baileysVersion.version; + const log = `Baileys version: ${version.join('.')}`; + + this.logger.info(log); + + this.logger.info(`Group Ignore: ${this.localSettings.groupsIgnore}`); + const socketConfig: UserFacingSocketConfig = { ...options, version, diff --git a/src/utils/fetchLatestWaWebVersion.ts b/src/utils/fetchLatestWaWebVersion.ts index 6dcfb797e5..6ba2892425 100644 --- a/src/utils/fetchLatestWaWebVersion.ts +++ b/src/utils/fetchLatestWaWebVersion.ts @@ -1,7 +1,11 @@ import axios, { AxiosRequestConfig } from 'axios'; import { fetchLatestBaileysVersion, WAVersion } from 'baileys'; -export const fetchLatestWaWebVersion = async (options: AxiosRequestConfig<{}>) => { +/** + * `options` go to the sw.js request (axios); `fallbackOptions` to Baileys' + * fallback fetch, which takes only an undici `dispatcher` for a proxy. + */ +export const fetchLatestWaWebVersion = async (options: AxiosRequestConfig<{}>, fallbackOptions: RequestInit = {}) => { try { const { data } = await axios.get('https://web.whatsapp.com/sw.js', { ...options, @@ -13,7 +17,7 @@ export const fetchLatestWaWebVersion = async (options: AxiosRequestConfig<{}>) = if (!match?.[1]) { return { - version: (await fetchLatestBaileysVersion()).version as WAVersion, + version: (await fetchLatestBaileysVersion(fallbackOptions)).version as WAVersion, isLatest: false, error: { message: 'Could not find client revision in the fetched content', @@ -29,7 +33,7 @@ export const fetchLatestWaWebVersion = async (options: AxiosRequestConfig<{}>) = }; } catch (error) { return { - version: (await fetchLatestBaileysVersion()).version as WAVersion, + version: (await fetchLatestBaileysVersion(fallbackOptions)).version as WAVersion, isLatest: false, error, }; From bc3567e281a1a599aeda06fad5d67f0f5094ada0 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:15:37 +0300 Subject: [PATCH 024/157] test: a connect never starts before the instance's proxy is loaded connectToWhatsapp starts loadProxy() without waiting for it and goes on to build the socket. With the Proxy row read taking a database round trip (50ms here), the socket config carries no proxy agent and the version fetch goes out directly: the connection guard records web.whatsapp.com and raw.githubusercontent.com, and a connection opened with the socket's agent goes straight out too. Covered for the first connect after boot and for a second connect of the same service (restart, reconnect after close), where loadProxy first sets enabled to false and so drops a proxy that was already loaded. Co-Authored-By: Claude Opus 5.5 --- test/helpers/connect.ts | 16 ++- test/proxy/connect-waits-for-proxy.test.ts | 126 +++++++++++++++++++++ 2 files changed, 136 insertions(+), 6 deletions(-) create mode 100644 test/proxy/connect-waits-for-proxy.test.ts diff --git a/test/helpers/connect.ts b/test/helpers/connect.ts index a1a5e91647..b2429397c9 100644 --- a/test/helpers/connect.ts +++ b/test/helpers/connect.ts @@ -17,6 +17,15 @@ export function fakeSocket() { }; } +/** The auth state is files on disk under the instances directory; nothing about it touches the network. */ +export function stubAuthState(service: any) { + service.defineAuthState = async () => ({ + state: { creds: initAuthCreds(), keys: { get: async () => ({}), set: async () => undefined } }, + saveCreds: async () => undefined, + removeCreds: async () => undefined, + }); +} + export type ProxyProtocol = 'http' | 'socks5'; /** Set the proxy the way /proxy/set does, then connect the way Evolution does. Returns the socket config. */ @@ -32,12 +41,7 @@ export async function connectBehind(socketSpy: { mock: { calls: any[][] } }, pro password: '', }); } - // The auth state is files on disk under the instances directory; nothing about it touches the network. - service.defineAuthState = async () => ({ - state: { creds: initAuthCreds(), keys: { get: async () => ({}), set: async () => undefined } }, - saveCreds: async () => undefined, - removeCreds: async () => undefined, - }); + stubAuthState(service); const before = socketSpy.mock.calls.length; await service.connectToWhatsapp(); const config = socketSpy.mock.calls[before]?.[0]; diff --git a/test/proxy/connect-waits-for-proxy.test.ts b/test/proxy/connect-waits-for-proxy.test.ts new file mode 100644 index 0000000000..e6e2da580b --- /dev/null +++ b/test/proxy/connect-waits-for-proxy.test.ts @@ -0,0 +1,126 @@ +// connectToWhatsapp loads the instance's proxy from the database and then +// builds the socket. If the socket is built before the Proxy row has been read, +// the whole account connects from the server's own IP. A real database round +// trip takes time, so the fake one here does too. +// +// Every way an instance connects goes through connectToWhatsapp: the boot +// auto-connect (monitor), /instance/create, /instance/connect, /instance/restart +// and the reconnect after a closed connection. The first connect of a fresh +// service is the boot; the second connect of the same service is a restart or +// a reconnect, where the proxy is already loaded and must stay in force. +import { vi } from 'vitest'; + +const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + return { ...orig, default: socketSpy, makeWASocket: socketSpy }; +}); + +import https from 'node:https'; + +import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; + +import { makeService } from '../helpers/baileys-service'; +import { fakeSocket, stubAuthState } from '../helpers/connect'; +import { + type Listening, + loopbackOnly, + startHttpProxy, + startHttpsServer, + trustTestCertificate, +} from '../helpers/local-net'; + +socketSpy.mockImplementation(fakeSocket); + +const PROXY_READ_MS = 50; + +let guard: ReturnType; +let untrust: () => void; +let web: Listening; +let proxy: Listening; + +beforeAll(async () => { + guard = loopbackOnly(); + untrust = trustTestCertificate(); + web = await startHttpsServer((req, _body, res) => { + res.writeHead(200, { 'content-type': 'text/javascript' }); + res.end('self.__swData=JSON.parse("{\\"client_revision\\":1027654321}");'); + }); + proxy = await startHttpProxy({ remap: { 'web.whatsapp.com:443': `127.0.0.1:${web.port}` } }); +}); + +afterAll(async () => { + await proxy.close(); + await web.close(); + untrust(); + guard.restore(); +}); + +beforeEach(() => { + guard.refused.splice(0); + proxy.log.splice(0); +}); + +/** A service as the monitor builds it on boot: the Proxy row is in the database, nothing is in memory yet. */ +async function bootedService() { + const { service, prisma } = await makeService(); + await prisma.proxy.create({ + data: { + instanceId: 'inst-1', + enabled: true, + host: '127.0.0.1', + port: String(proxy.port), + protocol: 'http', + username: '', + password: '', + }, + }); + const read = prisma.proxy.findUnique; + prisma.proxy.findUnique = async (args: any) => { + await new Promise((r) => setTimeout(r, PROXY_READ_MS)); + return read(args); + }; + stubAuthState(service); + return service; +} + +/** Connect, and return what the socket would do with the config Evolution gave it. */ +async function connect(service: any) { + const before = socketSpy.mock.calls.length; + await service.connectToWhatsapp(); + const config = socketSpy.mock.calls[before][0]; + // The WebSocket opens its connection with config.agent (Baileys passes it to ws); open one the same way. + const probe = await new Promise((resolve, reject) => + https + .get('https://web.whatsapp.com/sw.js', { agent: config.agent }, (res) => { + res.resume(); + res.on('end', () => resolve(`${res.statusCode}`)); + }) + .on('error', reject), + ).catch((e) => `error: ${e.message}`); + return { config, probe }; +} + +describe('a connect never starts before the instance proxy is loaded', () => { + it('on boot: the socket and the version fetch leave through the proxy', async () => { + const service = await bootedService(); + const { config, probe } = await connect(service); + expect(guard.refused).toEqual([]); + expect(proxy.log).toEqual(['CONNECT web.whatsapp.com:443', 'CONNECT web.whatsapp.com:443']); + expect(probe).toBe('200'); + expect(config.version).toEqual([2, 3000, 1027654321]); + }); + + it('on a restart or reconnect of a connected instance: still through the proxy', async () => { + const service = await bootedService(); + await connect(service); + guard.refused.splice(0); + proxy.log.splice(0); + const { probe } = await connect(service); + expect(guard.refused).toEqual([]); + expect(proxy.log).toEqual(['CONNECT web.whatsapp.com:443', 'CONNECT web.whatsapp.com:443']); + expect(probe).toBe('200'); + }); +}); From 1876a9b9be607960576acac06a3209110caab378 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:15:57 +0300 Subject: [PATCH 025/157] fix(baileys): wait for the proxy before connecting connectToWhatsapp now awaits loadProxy() before createClient, so the socket, the version fetch and media are built from the Proxy row rather than from whatever the read had reached. Every connect goes through here: the boot auto-connect, /instance/create, /instance/connect, /instance/restart, the reconnect after a closed connection and the Chatwoot reconnect. A failed Proxy read now fails the connect instead of connecting without the proxy. Co-Authored-By: Claude Opus 5.5 --- .../integrations/channel/whatsapp/whatsapp.baileys.service.ts | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 682511b7bd..d52fb243f5 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -738,7 +738,8 @@ export class BaileysStartupService extends ChannelStartupService { this.loadChatwoot(); this.loadSettings(); this.loadWebhook(); - this.loadProxy(); + // The socket, the version fetch and media all take their exit from localProxy: read it before connecting. + await this.loadProxy(); // Remontar o messageProcessor para garantir que está funcionando após reconexão this.messageProcessor.mount({ From 6f895ce7b0ec70be88e128d795e06b490e7af860 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:17:27 +0300 Subject: [PATCH 026/157] test: a connect uses the instance's stored settings, not defaults connectToWhatsapp starts loadSettings() without waiting for it and builds the socket from localSettings. With the Setting row read taking a database round trip (50ms here), the first connect after boot gets syncFullHistory and markOnlineOnConnect undefined, and shouldIgnoreJid keeps groups that the instance ignores. Two stored rows, since with full history on Evolution keeps groups even when groupsIgnore is set. The reconnect cases hold on the current code (loadSettings does not clear before reading) and guard that path. Co-Authored-By: Claude Opus 5.5 --- test/connect/stored-settings.test.ts | 98 ++++++++++++++++++++++++++++ 1 file changed, 98 insertions(+) create mode 100644 test/connect/stored-settings.test.ts diff --git a/test/connect/stored-settings.test.ts b/test/connect/stored-settings.test.ts new file mode 100644 index 0000000000..b32ce341d8 --- /dev/null +++ b/test/connect/stored-settings.test.ts @@ -0,0 +1,98 @@ +// connectToWhatsapp loads the instance's settings from the database and then +// builds the socket, which takes several of them as config: syncFullHistory +// (history at link time), groupsIgnore and readStatus (shouldIgnoreJid) and +// alwaysOnline (markOnlineOnConnect). If the socket is built before the Setting +// row has been read, it is built from defaults. A real database round trip +// takes time, so the fake one here does too. +import { vi } from 'vitest'; + +const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + return { ...orig, default: socketSpy, makeWASocket: socketSpy }; +}); +// This file is about settings; the version fetch has its own tests. +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); + +import { afterAll, beforeAll, describe, expect, it } from 'vitest'; + +import { makeService } from '../helpers/baileys-service'; +import { fakeSocket, stubAuthState } from '../helpers/connect'; +import { loopbackOnly } from '../helpers/local-net'; + +socketSpy.mockImplementation(fakeSocket); + +const SETTING_READ_MS = 50; +const GROUP = '120363000000000000@g.us'; +const DM = '972500000001@s.whatsapp.net'; + +let guard: ReturnType; +beforeAll(() => void (guard = loopbackOnly())); +afterAll(() => { + expect(guard.refused).toEqual([]); + guard.restore(); +}); + +/** A service as the monitor builds it on boot: the Setting row is in the database, nothing is in memory yet. */ +async function bootedService(settings: Record) { + const { service, prisma } = await makeService(); + await prisma.setting.create({ + data: { instanceId: 'inst-1', rejectCall: false, msgCall: '', readMessages: false, readStatus: false, ...settings }, + }); + const read = prisma.setting.findUnique; + prisma.setting.findUnique = async (args: any) => { + await new Promise((r) => setTimeout(r, SETTING_READ_MS)); + return read(args); + }; + stubAuthState(service); + return service; +} + +async function connect(service: any) { + const before = socketSpy.mock.calls.length; + await service.connectToWhatsapp(); + const config = socketSpy.mock.calls[before][0]; + return { + syncFullHistory: config.syncFullHistory, + markOnlineOnConnect: config.markOnlineOnConnect, + ignoresGroup: !!config.shouldIgnoreJid(GROUP), + ignoresDm: !!config.shouldIgnoreJid(DM), + }; +} + +// Evolution's own rule (createClient, shouldIgnoreJid): with full history on, groups are kept even when groupsIgnore is set. +const cases = [ + { + name: 'full history, groups ignored, always online', + settings: { syncFullHistory: true, groupsIgnore: true, alwaysOnline: true }, + expected: { syncFullHistory: true, markOnlineOnConnect: true, ignoresGroup: false, ignoresDm: false }, + }, + { + name: 'no full history, groups ignored, always online', + settings: { syncFullHistory: false, groupsIgnore: true, alwaysOnline: true }, + expected: { syncFullHistory: false, markOnlineOnConnect: true, ignoresGroup: true, ignoresDm: false }, + }, +]; + +describe('a connect uses the instance stored settings, not defaults', () => { + for (const { name, settings, expected } of cases) { + it(`on boot (${name})`, async () => { + const service = await bootedService(settings); + expect(await connect(service)).toEqual(expected); + }); + + // A reconnect comes to an instance that has been running, so the first connect's reads have landed. + // loadSettings, unlike loadProxy, does not clear the values before reading, so this case holds + // on the code before the fix too: it guards the reconnect path against a fix that breaks it. + it(`on a restart or reconnect (${name})`, async () => { + const service = await bootedService(settings); + await connect(service); + await new Promise((r) => setTimeout(r, SETTING_READ_MS * 2)); + expect(await connect(service)).toEqual(expected); + }); + } +}); From 452402ecd3afaa81a64342cb396adfc6bda26d02 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:17:41 +0300 Subject: [PATCH 027/157] fix(baileys): wait for settings before connecting connectToWhatsapp now awaits loadSettings() before createClient, so the socket is built from the instance's Setting row (syncFullHistory, groupsIgnore, readStatus, alwaysOnline, and the wavoip token read right after) rather than from defaults when the read is slower than the rest of the connect. Co-Authored-By: Claude Opus 5.5 --- .../integrations/channel/whatsapp/whatsapp.baileys.service.ts | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index d52fb243f5..4b8f5e3949 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -736,7 +736,8 @@ export class BaileysStartupService extends ChannelStartupService { public async connectToWhatsapp(number?: string): Promise { try { this.loadChatwoot(); - this.loadSettings(); + // The socket takes syncFullHistory, groupsIgnore, readStatus and alwaysOnline as config: read them first. + await this.loadSettings(); this.loadWebhook(); // The socket, the version fetch and media all take their exit from localProxy: read it before connecting. await this.loadProxy(); From 3ec7c2035dc11441ee126aae503759ec5b3a1647 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:29:31 +0300 Subject: [PATCH 028/157] test: a database error never replaces a linked session's credentials Co-Authored-By: Claude Opus 5.5 --- test/unit/auth-state-db-error.test.ts | 96 +++++++++++++++++++++++++++ 1 file changed, 96 insertions(+) create mode 100644 test/unit/auth-state-db-error.test.ts diff --git a/test/unit/auth-state-db-error.test.ts b/test/unit/auth-state-db-error.test.ts new file mode 100644 index 0000000000..213c3a0164 --- /dev/null +++ b/test/unit/auth-state-db-error.test.ts @@ -0,0 +1,96 @@ +// The Prisma auth store (creds in the session table) is opened +// on every connect and reconnect. It used to turn a failed session read into +// "no session": keyExists and getAuthKey swallowed the error and returned +// false/null, so the store started a fresh, unlinked session (initAuthCreds) +// and wrote it over the linked one as soon as the database answered again. A +// database blip during a reconnect therefore unlinked the account for good. +// +// A failed read must fail the open (so the connect fails and can be retried) +// and must leave the stored creds exactly as they were. A session that is +// genuinely absent still starts fresh: that is how a new instance links. +import { rmSync } from 'node:fs'; + +import { afterAll, beforeEach, describe, expect, it, vi } from 'vitest'; + +import { prismaRepository } from '../helpers/fake-server-module'; + +// Creds live in the fake Prisma's session table; with Redis off, keys go to files +// under INSTANCE_DIR (a temp dir here, not the repo). +const tmp = await vi.hoisted(async () => { + const { mkdtempSync } = await import('node:fs'); + const { tmpdir } = await import('node:os'); + const { join } = await import('node:path'); + return mkdtempSync(join(tmpdir(), 'evo-auth-dberr-')); +}); +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('@config/path.config', async (importOriginal) => ({ ...(await importOriginal()), INSTANCE_DIR: tmp })); + +const SESSION = 'linked-session'; +const ME = { id: '972500000000:7@s.whatsapp.net', name: 'Linked account' }; + +const open = async (sessionId = SESSION) => { + const { default: useMultiFileAuthStatePrisma } = await import('@utils/use-multi-file-auth-state-prisma'); + return useMultiFileAuthStatePrisma(sessionId, null as any); +}; + +const storedRow = (sessionId = SESSION) => prismaRepository.session.rows.find((r: any) => r.sessionId === sessionId); + +/** Make the session table's next read fail the way a dropped connection does; later reads succeed. */ +function failNextSessionRead() { + const read = prismaRepository.session.findUnique; + let failed = false; + prismaRepository.session.findUnique = async (args: any) => { + if (!failed) { + failed = true; + throw Object.assign(new Error("Can't reach database server at `127.0.0.1:5432`"), { code: 'P1001' }); + } + return read(args); + }; + return () => void (prismaRepository.session.findUnique = read); +} + +beforeEach(() => void prismaRepository.session.rows.splice(0)); +afterAll(() => rmSync(tmp, { recursive: true, force: true })); + +describe('the prisma auth store on a database error', () => { + it('a failed session read fails the open and never replaces a linked session', async () => { + // A linked session: creds that carry the account (`me`) and are registered. + const linked = await open(); + Object.assign(linked.state.creds, { me: ME, registered: true }); + await linked.saveCreds(); + const before = storedRow()!.creds; + + const restore = failNextSessionRead(); + let outcome: { opened: boolean; error?: string; me?: unknown }; + try { + outcome = await open().then( + (s) => ({ opened: true, me: s.state.creds.me ?? null }), + (e) => ({ opened: false, error: e?.message }), + ); + } finally { + restore(); + } + // Give any write the store left behind time to land once the database answers again. + await new Promise((r) => setTimeout(r, 10)); + + // The open failed, and nothing was written over the linked creds. + expect({ + ...outcome, + rows: prismaRepository.session.rows.filter((r: any) => r.sessionId === SESSION).length, + credsUnchanged: storedRow()?.creds === before, + }).toEqual({ opened: false, error: "Can't reach database server at `127.0.0.1:5432`", rows: 1, credsUnchanged: true }); + + // The retried connect opens the linked session. + const retried = await open(); + expect(retried.state.creds.me).toEqual(ME); + expect(retried.state.creds.registered).toBe(true); + expect(retried.state.creds.noiseKey).toEqual(linked.state.creds.noiseKey); + }); + + it('a session that is genuinely absent still starts fresh and is stored (control)', async () => { + const fresh = await open('new-session'); + expect(fresh.state.creds.me).toBeUndefined(); + expect(fresh.state.creds.registered).toBe(false); + expect(storedRow('new-session')).toBeDefined(); + }); +}); From b3e91ff6cf52ec5b8fb1c182f9ffc2060f381a34 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:30:00 +0300 Subject: [PATCH 029/157] fix(auth-state): fail on a database error instead of starting a fresh session keyExists and getAuthKey swallowed every database error and returned false/null, so opening the Prisma auth store during a database blip read as "no session": it started fresh creds (initAuthCreds) and saved them over the linked session once the database answered, unlinking the account. A failed session read now propagates, so the open (and the connect) fails and can be retried with the stored creds intact. A genuinely absent row still starts a fresh session. saveKey keeps its existence check inside its own try, so a failed check skips the save instead of creating a second row. Co-Authored-By: Claude Opus 5.5 --- src/utils/use-multi-file-auth-state-prisma.ts | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/src/utils/use-multi-file-auth-state-prisma.ts b/src/utils/use-multi-file-auth-state-prisma.ts index bd11afa2fe..d9eeb8e046 100644 --- a/src/utils/use-multi-file-auth-state-prisma.ts +++ b/src/utils/use-multi-file-auth-state-prisma.ts @@ -16,18 +16,17 @@ const fixFileName = (file: string): string | undefined => { return replacedColon; }; +// A database error is not an absent session. keyExists and getAuthKey let it +// propagate: turned into "no session", it made the store start a fresh one and +// save it over the linked creds once the database answered again. export async function keyExists(sessionId: string): Promise { - try { - const key = await prismaRepository.session.findUnique({ where: { sessionId: sessionId } }); - return !!key; - } catch { - return false; - } + const key = await prismaRepository.session.findUnique({ where: { sessionId: sessionId } }); + return !!key; } export async function saveKey(sessionId: string, keyJson: any): Promise { - const exists = await keyExists(sessionId); try { + const exists = await keyExists(sessionId); if (!exists) return await prismaRepository.session.create({ data: { @@ -45,11 +44,10 @@ export async function saveKey(sessionId: string, keyJson: any): Promise { } export async function getAuthKey(sessionId: string): Promise { + const auth = await prismaRepository.session.findUnique({ where: { sessionId: sessionId } }); + if (!auth) return null; try { - const register = await keyExists(sessionId); - if (!register) return null; - const auth = await prismaRepository.session.findUnique({ where: { sessionId: sessionId } }); - return JSON.parse(auth?.creds); + return JSON.parse(auth.creds); } catch { return null; } @@ -105,6 +103,8 @@ export default async function useMultiFileAuthStatePrisma( } async function readData(key: string): Promise { + // Outside the try below: a failed creds read must fail the open, never read as "no creds". + const storedCreds = key === 'creds' ? await getAuthKey(sessionId) : undefined; try { let rawData; const cacheConfig = configService.get('CACHE'); @@ -118,7 +118,7 @@ export default async function useMultiFileAuthStatePrisma( return JSON.parse(rawData, BufferJSON.reviver); } } else { - rawData = await getAuthKey(sessionId); + rawData = storedCreds; } const parsedData = JSON.parse(rawData, BufferJSON.reviver); From aead4007a572c8e58c7c1f8fd5730fd64f23ebae Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:31:23 +0300 Subject: [PATCH 030/157] test: a failed settings read does not drop an event batch Co-Authored-By: Claude Opus 5.5 --- test/handlers/settings-read-failure.test.ts | 67 +++++++++++++++++++++ 1 file changed, 67 insertions(+) create mode 100644 test/handlers/settings-read-failure.test.ts diff --git a/test/handlers/settings-read-failure.test.ts b/test/handlers/settings-read-failure.test.ts new file mode 100644 index 0000000000..262d6b542b --- /dev/null +++ b/test/handlers/settings-read-failure.test.ts @@ -0,0 +1,67 @@ +// eventHandler() starts every event batch by reading the instance's Setting row +// (findSettings), inside the one try that covers the whole batch. When that read +// failed, the batch was dropped: messages.upsert, creds.update and +// connection.update alike, with nothing sent to the webhook. A database blip +// therefore lost the messages that arrived during it. +// +// A failed settings read must not cost the batch. The batch runs with the last +// known settings (loaded at connect, kept by /settings/set), and the failure is +// logged without anything the batch carries. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { beforeEach, describe, expect, it } from 'vitest'; + +import { msg } from '../helpers/baileys-fixtures'; +import { deliver, makeService } from '../helpers/baileys-service'; +import { captureOutput } from '../helpers/capture-output'; +import { emitted } from '../helpers/fake-server-module'; +import type { Profile } from '../helpers/profiles'; + +const PHONE = '972509876543'; +const SENDER = `${PHONE}@s.whatsapp.net`; +const GROUP = '120363000000000002@g.us'; +const TEXT = 'Zq9 a message that arrived during a database blip Zq9'; +const DB_ERROR = "Can't reach database server at `127.0.0.1:5432`"; + +beforeEach(() => void emitted.splice(0)); + +/** A connected instance whose stored settings ignore groups, and whose next Setting read fails. */ +async function serviceWithFailingSettingsRead(profile: Profile) { + const { service, prisma, ev } = await makeService({ profile }); + await prisma.setting.create({ + data: { instanceId: 'inst-1', rejectCall: false, msgCall: '', groupsIgnore: true, readMessages: false, readStatus: false }, + }); + await service.loadSettings(); // what connectToWhatsapp does before the socket opens + const read = prisma.setting.findUnique; + let failed = false; + prisma.setting.findUnique = async (args: any) => { + if (!failed) { + failed = true; + throw Object.assign(new Error(DB_ERROR), { code: 'P1001' }); + } + return read(args); + }; + return { service, ev }; +} + +describe.each(['minimal', 'stored'] as Profile[])('a failed settings read does not drop an event batch (profile %s)', (profile) => { + it('the batch messages still reach the webhook, under the last known settings', async () => { + const { service, ev } = await serviceWithFailingSettingsRead(profile); + const direct = msg(SENDER, 'B1', TEXT).message; + // Stored groupsIgnore is true, so this one is skipped; defaults (no settings) would send it. + const group = msg(GROUP, 'B2', TEXT, { key: { remoteJid: GROUP, fromMe: false, id: 'B2', participant: SENDER } }).message; + const out = await captureOutput(() => deliver(service, ev, { 'messages.upsert': { messages: [direct, group], type: 'notify' } })); + + const upserts = emitted + .filter((e) => e.event === 'messages.upsert') + .map((e) => ({ id: e.data?.key?.id, remoteJid: e.data?.key?.remoteJid, text: e.data?.message?.conversation })); + expect(upserts).toEqual([{ id: 'B1', remoteJid: SENDER, text: TEXT }]); + + // The failure is logged, and nothing the batch carries is. + const lines = out.split('\n').map((l) => l.replace(/\x1b\[[0-9;]*m/g, '')); + expect(lines.some((l) => l.includes('Settings read failed') && l.includes(DB_ERROR))).toBe(true); + expect(lines.filter((l) => l.includes('Zq9') || l.includes(PHONE))).toEqual([]); + }); +}); From 323530336b806251e577a069a97447a9799297a1 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:31:41 +0300 Subject: [PATCH 031/157] fix(baileys): keep processing a batch when the settings read fails eventHandler read the Setting row at the top of every batch, inside the try that covers the whole batch, so a failed read dropped everything in it: messages.upsert, creds.update, connection.update. The batch now runs with the last known settings (localSettings, loaded at connect and kept by setSettings) and the failure is logged as a warning with the error only, never the batch. Co-Authored-By: Claude Opus 5.5 --- .../channel/whatsapp/whatsapp.baileys.service.ts | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 4b8f5e3949..621929c0f4 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -1961,7 +1961,12 @@ export class BaileysStartupService extends ChannelStartupService { try { if (!this.endSession) { const database = this.configService.get('DATABASE'); - const settings = await this.findSettings(); + // A failed read must not drop the batch (messages, creds, connection updates with it): + // fall back to the settings loaded at connect and kept by setSettings. + const settings = await this.findSettings().catch((error) => { + this.logger.warn(`Settings read failed, using the last known settings: ${error?.message ?? error}`); + return { ...this.localSettings }; + }); if (events.call) { const call = events.call[0]; From fb23a154e9c1d3ff738cbfa93ac602c7e062f236 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:36:29 +0300 Subject: [PATCH 032/157] test: unlinking an instance whose connection is down still unlinks it A linked instance mid-reconnect (its real Baileys socket dialling a local server that never answers, so the ws stays CONNECTING) is logged out and deleted through the real /instance router and guards, the real InstanceController and WAMonitoringService, and Evolution's own Prisma auth store. Baileys' logout throws Boom('Connection Closed', 428) before it ends the socket, and on 2.3.7 nothing after it runs: logout answers 500, delete answers 400, the session row and key files stay, the row still says open (so the boot auto-connects it) and the socket stays up. The helpers gain what this needs: makeService can share the test's Prisma and event emitter, the fake Prisma has every table deleteInstance clears, and http-app can serve just /instance for a test with a faked server module. Co-Authored-By: Claude Opus 5.5 --- test/helpers/baileys-service.ts | 11 +- test/helpers/fake-prisma.ts | 7 + test/helpers/http-app.ts | 28 ++- test/instance/unlink-socket-down.test.ts | 208 +++++++++++++++++++++++ 4 files changed, 248 insertions(+), 6 deletions(-) create mode 100644 test/instance/unlink-socket-down.test.ts diff --git a/test/helpers/baileys-service.ts b/test/helpers/baileys-service.ts index 597f8581ce..3597f28641 100644 --- a/test/helpers/baileys-service.ts +++ b/test/helpers/baileys-service.ts @@ -10,17 +10,22 @@ import { applyProfile, type Profile } from './profiles'; export const WUID = '972500000000@s.whatsapp.net'; -export async function makeService(opts: { profile?: Profile } = {}) { +/** + * `prisma` and `eventEmitter` default to fresh ones. Pass the ones the rest of a + * test uses (the server module's Prisma, the monitor's emitter) when the service + * must share them, as it does in production. + */ +export async function makeService(opts: { profile?: Profile; prisma?: any; eventEmitter?: EventEmitter2 } = {}) { applyProfile(opts.profile ?? 'minimal'); const { BaileysStartupService } = await import('@api/integrations/channel/whatsapp/whatsapp.baileys.service'); const { CacheService } = await import('@api/services/cache.service'); const { LocalCache } = await import('@cache/localcache'); const { ConfigService } = await import('@config/env.config'); const configService = new ConfigService(); - const prisma = fakePrisma(); + const prisma = opts.prisma ?? fakePrisma(); const cache = new CacheService(new LocalCache(configService, 'instance')); const baileysCache = new CacheService(new LocalCache(configService, 'baileys')); - const service: any = new BaileysStartupService(configService, new EventEmitter2(), prisma, cache, null as any, baileysCache, null as any); + const service: any = new BaileysStartupService(configService, opts.eventEmitter ?? new EventEmitter2(), prisma, cache, null as any, baileysCache, null as any); service.setInstance({ instanceName: 'test', instanceId: 'inst-1', integration: 'WHATSAPP-BAILEYS' }); const ev = makeEventBuffer(P({ level: 'silent' }) as any); service.client = { diff --git a/test/helpers/fake-prisma.ts b/test/helpers/fake-prisma.ts index d35b70e411..155adc3a7c 100644 --- a/test/helpers/fake-prisma.ts +++ b/test/helpers/fake-prisma.ts @@ -79,6 +79,13 @@ export function fakePrisma() { proxy: table('proxy', (r) => r.instanceId), webhook: table('webhook', (r) => r.instanceId), chatwoot: table('chatwoot', (r) => r.instanceId), + // The rest of what deleting an instance clears (monitor.service.ts cleaningStoreData). + rabbitmq: table('rabbitmq', (r) => r.instanceId), + nats: table('nats', (r) => r.instanceId), + sqs: table('sqs', (r) => r.instanceId), + integrationSession: table('integrationSession', () => undefined), + typebot: table('typebot', () => undefined), + websocket: table('websocket', (r) => r.instanceId), }; db.$transaction = async (ops: any) => (typeof ops === 'function' ? ops(db) : Promise.all(ops)); db.$queryRaw = async () => []; diff --git a/test/helpers/http-app.ts b/test/helpers/http-app.ts index 5c96b3df48..a6c17a507e 100644 --- a/test/helpers/http-app.ts +++ b/test/helpers/http-app.ts @@ -4,12 +4,11 @@ // error handler's status mapping (main.ts, "app.use((err, req, res, next) ..."). import type { AddressInfo } from 'node:net'; -export async function startApp() { +async function serve(mount: string, router: any) { const express = (await import('express')).default; - const { router } = await import('@api/routes/index.router'); const app = express(); app.use(express.json({ limit: '10mb' })); - app.use('/', router); + app.use(mount, router); app.use((err: any, _req: any, res: any, _next: any) => res.status(err?.status || 500).json({ status: err?.status || 500, error: err?.error, response: { message: err?.message } }), ); @@ -18,3 +17,26 @@ export async function startApp() { const base = `http://127.0.0.1:${(server.address() as AddressInfo).port}`; return { base, close: () => new Promise((r) => server.close(r)) }; } + +export async function startApp() { + const { router } = await import('@api/routes/index.router'); + return serve('/', router); +} + +/** + * Only /instance, behind the guards index.router puts in front of it + * (instanceExistsGuard, instanceLoggedGuard, the apikey guard), for a test that + * fakes the server module and hands the router its own instanceController. + */ +export async function startInstanceApp() { + // index.router is where the import cycle through @exceptions (which takes HttpStatus + // from it) has to start, as it does in main.ts; entered from a router, RouterBroker + // is still undefined when the channel routers extend it. + await import('@api/routes/index.router'); + const { InstanceRouter } = await import('@api/routes/instance.router'); + const { authGuard } = await import('@api/guards/auth.guard'); + const { instanceExistsGuard, instanceLoggedGuard } = await import('@api/guards/instance.guard'); + const { configService } = await import('@config/env.config'); + const guards = [instanceExistsGuard, instanceLoggedGuard, authGuard['apikey']]; + return serve('/instance', new InstanceRouter(configService, ...guards).router); +} diff --git a/test/instance/unlink-socket-down.test.ts b/test/instance/unlink-socket-down.test.ts new file mode 100644 index 0000000000..ddfe8f1799 --- /dev/null +++ b/test/instance/unlink-socket-down.test.ts @@ -0,0 +1,208 @@ +// Unlinking an instance (DELETE /instance/logout, DELETE /instance/delete) must +// unlink it even when its socket is down. Baileys' logout first sends +// remove-companion-device, and sendRawMessage throws Boom('Connection Closed', +// 428) when the ws is not open (rc14 lib/Socket/socket.js, sendRawMessage), +// before logout ends the socket. 2.3.7's logoutInstance awaits it unguarded, so +// nothing after it runs: the credentials stay stored, the row still says open, +// the socket stays up, and delete answers 400 with the instance still in memory. +// Evolution then connects with the kept credentials (the socket in flight, or the +// boot auto-connect of a row that says open) and keeps receiving the person's +// messages after they unlinked. +// +// The instance here is mid-reconnect, the way it is after a dropped connection: +// Evolution's real connect builds a real Baileys socket, pointed at a local +// "WhatsApp" that accepts the TCP connection and never answers the WebSocket +// upgrade, so the socket stays CONNECTING. The session is a linked one (creds +// with `me` in the session table, signal keys in files under INSTANCE_DIR), +// written through Evolution's own Prisma auth store. +import { vi } from 'vitest'; + +const h = vi.hoisted(() => ({ wsUrl: '', sockets: [] as any[], services: [] as any[], waMonitor: undefined as any, instanceController: undefined as any })); +const tmp = await vi.hoisted(async () => { + const { mkdtempSync } = await import('node:fs'); + const { tmpdir } = await import('node:os'); + const { join } = await import('node:path'); + return mkdtempSync(join(tmpdir(), 'evo-unlink-')); +}); + +vi.mock('@api/server.module', async () => { + const fake = await import('../helpers/fake-server-module'); + // The router, the guards and the controller reach the monitor and the controller through the server module. + return { + ...fake, + get waMonitor() { + return h.waMonitor; + }, + get instanceController() { + return h.instanceController; + }, + }; +}); +vi.mock('@config/path.config', async (importOriginal) => ({ ...(await importOriginal()), INSTANCE_DIR: tmp })); +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); +// The real socket, sent to the local "WhatsApp" instead of web.whatsapp.com. Every socket built is kept, so a reconnect shows. +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + const make = (config: any) => { + const socket = orig.makeWASocket({ ...config, waWebSocketUrl: h.wsUrl }); + h.sockets.push(socket); + return socket; + }; + return { ...orig, default: make, makeWASocket: make }; +}); + +import { existsSync, readdirSync, rmSync } from 'node:fs'; +import net from 'node:net'; +import { join } from 'node:path'; + +import EventEmitter2 from 'eventemitter2'; +import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest'; + +import { makeService, settle, WUID } from '../helpers/baileys-service'; +import { prismaRepository as prisma } from '../helpers/fake-server-module'; +import { startInstanceApp } from '../helpers/http-app'; +import { loopbackOnly } from '../helpers/local-net'; + +const TOKEN = 'instance-token'; + +/** Accepts connections and never answers, so a WebSocket to it stays CONNECTING. */ +function silentWhatsapp() { + const open = new Set(); + const server = net.createServer((s) => { + open.add(s); + s.on('close', () => open.delete(s)); + s.on('error', () => undefined); + }); + return { + drop: () => open.forEach((s) => s.destroy()), + listen: () => new Promise((r) => server.listen(0, '127.0.0.1', () => r((server.address() as net.AddressInfo).port))), + close: () => new Promise((r) => (open.forEach((s) => s.destroy()), server.close(() => r()))), + }; +} + +let guard: ReturnType; +let whatsapp: ReturnType; +let app: Awaited>; + +beforeAll(async () => { + guard = loopbackOnly(); + whatsapp = silentWhatsapp(); + h.wsUrl = `ws://127.0.0.1:${await whatsapp.listen()}/ws/chat`; + app = await startInstanceApp(); +}); +afterEach(async () => { + // Tear down without the service answering the close with a reconnect. + for (const service of h.services) service.connectToWhatsapp = async () => undefined; + for (const s of h.sockets) await s.end(undefined).catch(() => undefined); + for (const service of h.services) await settle(service); + whatsapp.drop(); + h.sockets.length = 0; + h.services.length = 0; + for (const t of Object.values(prisma) as any[]) if (Array.isArray(t?.rows)) t.rows.length = 0; + rmSync(join(tmp, 'inst-1'), { recursive: true, force: true }); +}); +afterAll(async () => { + await app.close(); + await whatsapp.close(); + rmSync(tmp, { recursive: true, force: true }); + expect(guard.refused).toEqual([]); + guard.restore(); +}); + +/** A linked instance whose connection dropped: its row says open, and it is reconnecting on a socket that has not opened. */ +async function reconnectingInstance() { + await prisma.instance.create({ + data: { id: 'inst-1', name: 'test', connectionStatus: 'open', token: TOKEN, integration: 'WHATSAPP-BAILEYS' }, + }); + const { default: useMultiFileAuthStatePrisma } = await import('@utils/use-multi-file-auth-state-prisma'); + const auth = await useMultiFileAuthStatePrisma('inst-1', null as any); + auth.state.creds.me = { id: WUID, name: 'Dana' }; + await auth.saveCreds(); + await auth.state.keys.set({ 'pre-key': { '1': { public: Buffer.alloc(32, 1), private: Buffer.alloc(32, 2) } } }); + + const { ConfigService } = await import('@config/env.config'); + const { CacheService } = await import('@api/services/cache.service'); + const { LocalCache } = await import('@cache/localcache'); + const { WAMonitoringService } = await import('@api/services/monitor.service'); + const { InstanceController } = await import('@api/controllers/instance.controller'); + const configService = new ConfigService(); + const cache = new CacheService(new LocalCache(configService, 'instance')); + const emitter = new EventEmitter2(); + const waMonitor = new WAMonitoringService(emitter, configService, prisma, null as any, cache, cache, cache); + const n = null as any; + h.waMonitor = waMonitor; + h.instanceController = new InstanceController(waMonitor, configService, prisma, emitter, n, n, n, cache, cache, cache, n); + + const { service } = await makeService({ prisma, eventEmitter: emitter }); + h.services.push(service); + waMonitor.waInstances.test = service; + await service.connectToWhatsapp(); + await vi.waitFor(() => expect(service.connectionStatus.state).toBe('connecting')); + // The premise: a linked session is stored, and its one socket is dialling WhatsApp, not open. + expect(JSON.parse(JSON.parse(prisma.session.rows[0].creds)).me.id).toBe(WUID); + expect(readdirSync(join(tmp, 'inst-1'))).toEqual(['pre-key-1.json']); + expect(h.sockets.map((s) => s.ws.isConnecting)).toEqual([true]); + return { service, waMonitor }; +} + +async function call(route: 'logout' | 'delete') { + const res = await fetch(`${app.base}/instance/${route}/test`, { method: 'DELETE', headers: { apikey: TOKEN } }); + return { status: res.status, body: await res.json() }; +} + +/** Nothing of the session is left, and nothing is connected or connecting to WhatsApp with it: its one socket is closed, and no other was built. */ +async function expectUnlinked(service: any) { + await settle(service); + expect(prisma.session.rows).toEqual([]); + expect(existsSync(join(tmp, 'inst-1'))).toBe(false); + expect(h.sockets.map((s) => s.ws.isClosed)).toEqual([true]); +} + +describe('unlinking an instance whose connection is down', () => { + it('logout wipes the session, ends the socket, and the instance stays down', async () => { + const { service } = await reconnectingInstance(); + + expect(await call('logout')).toEqual({ + status: 200, + body: { status: 'SUCCESS', error: false, response: { message: 'Instance logged out' } }, + }); + + await expectUnlinked(service); + // The boot auto-connects only a row that says open or connecting (monitor.service.ts setInstance). + expect(prisma.instance.rows.map((r: any) => r.connectionStatus)).toEqual(['close']); + expect(service.connectionStatus.state).toBe('close'); + }); + + it('delete wipes the session and removes the instance', async () => { + const { service, waMonitor } = await reconnectingInstance(); + + expect(await call('delete')).toEqual({ + status: 200, + body: { status: 'SUCCESS', error: false, response: { message: 'Instance deleted' } }, + }); + + await expectUnlinked(service); + expect(waMonitor.waInstances.test).toBeUndefined(); + expect(prisma.instance.rows).toEqual([]); + }); + + it('delete removes the instance even when its logout fails for another reason', async () => { + const { service, waMonitor } = await reconnectingInstance(); + const findFirst = prisma.session.findFirst; + prisma.session.findFirst = async () => { + prisma.session.findFirst = findFirst; + throw new Error("Can't reach database server"); + }; + + expect(await call('delete')).toEqual({ + status: 200, + body: { status: 'SUCCESS', error: false, response: { message: 'Instance deleted' } }, + }); + + await expectUnlinked(service); + expect(waMonitor.waInstances.test).toBeUndefined(); + expect(prisma.instance.rows).toEqual([]); + }); +}); From 3355775fbfc2a91fa96633cb80b72476d451e2cb Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:38:14 +0300 Subject: [PATCH 033/157] fix(instance): wipe the session on logout and delete even when the socket is down Baileys' logout sends remove-companion-device before it ends the socket, and throws Boom('Connection Closed', 428) when the ws is not open. 2.3.7's logoutInstance awaited it unguarded, so a logout during a reconnect left the credentials stored, the socket dialling and the row saying open, and Evolution went on to connect with them after the person unlinked. logoutInstance now tries the socket logout, and when it cannot reach WhatsApp it still wipes the stored session (moved as is into removeSession) and then ends the socket with a loggedOut close, which is final (no reconnect) and runs the same cleanup as a logout that reached WhatsApp: row to close, key files and session removed. The socket is ended after the wipe, and in a finally, so its cleanup does not race the store being reopened, and it is ended even if the wipe throws. A down socket ended with a plain Error, as upstream 933a28de does, reconnects on 2.3.7, which has no isDeleting guard. deleteInstance takes upstream c29bcc50 (#2520): a failed logout no longer stops the delete, so remove.instance always purges the instance from memory and runs the store cleanup. Its comment is reworded for this fork. The device stays on the person's Linked Devices list when WhatsApp could not be told; only a logout that reaches WhatsApp removes it. Co-Authored-By: Claude Opus 5.5 --- src/api/controllers/instance.controller.ts | 15 ++++++++++- .../whatsapp/whatsapp.baileys.service.ts | 25 +++++++++++++++++-- 2 files changed, 37 insertions(+), 3 deletions(-) diff --git a/src/api/controllers/instance.controller.ts b/src/api/controllers/instance.controller.ts index 6a69106881..f982379d22 100644 --- a/src/api/controllers/instance.controller.ts +++ b/src/api/controllers/instance.controller.ts @@ -456,7 +456,20 @@ export class InstanceController { if (this.configService.get('CHATWOOT').ENABLED) waInstances?.clearCacheChatwoot(); if (instance.state === 'connecting' || instance.state === 'open') { - await this.logout({ instanceName }); + try { + await this.logout({ instanceName }); + } catch (error) { + // A failed logout must not stop the delete. The remove.instance emit + // below is the only path that purges the in-memory entry and runs + // cleaningUp() and cleaningStoreData(), which wipe the session again. + // Without this catch, the stale entry persists until the entire + // process restarts. + this.logger.warn({ + message: 'logout failed during deleteInstance, proceeding with cleanup', + instanceName, + error, + }); + } } try { diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 621929c0f4..36eab10356 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -269,10 +269,31 @@ export class BaileysStartupService extends ChannelStartupService { public async logoutInstance() { this.messageProcessor.onDestroy(); - await this.client?.logout('Log out instance: ' + this.instanceName); - this.client?.ws?.close(); + // Baileys' logout tells WhatsApp to remove this device and then ends the socket. + // With the socket down the first step throws ('Connection Closed') and the socket + // is never ended. Unlink locally anyway: wipe the credentials, then end the socket + // as a logout would. A loggedOut close is final (no reconnect) and runs the same + // cleanup as a logout that reached WhatsApp. The device stays listed on the phone. + let unreachable = false; + try { + await this.client?.logout('Log out instance: ' + this.instanceName); + + this.client?.ws?.close(); + } catch (error) { + unreachable = true; + this.logger.warn(`Logout could not reach WhatsApp (${error?.message}), unlinking locally`); + } + + try { + await this.removeSession(); + } finally { + if (unreachable) + await this.client?.end(new Boom('Intentional Logout', { statusCode: DisconnectReason.loggedOut })); + } + } + private async removeSession() { const db = this.configService.get('DATABASE'); const cache = this.configService.get('CACHE'); const provider = this.configService.get('PROVIDER'); From 7068a1139333b734676d767b6ddd12e4ea191ee5 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:31:34 +0300 Subject: [PATCH 034/157] test: profile pictures are fetched once per contact, not per event Evolution asks WhatsApp for a profile picture on every contacts.upsert, contacts.update and inbound message, and for every group on every fetchAllGroups. rc14 emits contacts.update for each inbound message with a pushName, so a live message costs two IQs, and a history batch of N contacts fires N at once. These tests specify one lookup per jid within an hour, at most four in flight, refresh on WhatsApp's picture notification, and payloads that keep their profilePicUrl. Co-Authored-By: Claude Opus 5.5 --- test/handlers/profile-pictures.test.ts | 213 +++++++++++++++++++++++++ 1 file changed, 213 insertions(+) create mode 100644 test/handlers/profile-pictures.test.ts diff --git a/test/handlers/profile-pictures.test.ts b/test/handlers/profile-pictures.test.ts new file mode 100644 index 0000000000..0f7c0ffbcd --- /dev/null +++ b/test/handlers/profile-pictures.test.ts @@ -0,0 +1,213 @@ +// Evolution asks WhatsApp for a contact's profile picture (one IQ each) on +// every contacts.upsert, every contacts.update and every inbound message, and +// for every group on every fetchAllGroups. Baileys 7.0.0-rc14 emits a +// contacts.update for every inbound message that carries a pushName +// (chats.js upsertMessage), so a live message costs two picture IQs, and a +// history batch of N contacts fires N of them at once (Promise.all). At link +// time that is thousands of IQs in a burst from a device that has just linked. +// +// A lookup is kept per jid for a while, a jid already being looked up is not +// looked up again, and only a few lookups run at once. The webhook payloads +// keep their profilePicUrl, served from what was looked up. WhatsApp says when +// a picture changes (a `picture` notification, which Baileys emits as +// contacts.update with imgUrl 'changed' or 'removed'), and that refreshes it. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; + +import { historyEvent, msg } from '../helpers/baileys-fixtures'; +import { deliver, makeService, settle } from '../helpers/baileys-service'; +import { emitted } from '../helpers/fake-server-module'; + +const INITIAL_BOOTSTRAP = 0; +const PUSH_NAME = 4; +const HOUR = 60 * 60 * 1000; +/** The most picture lookups the service may have in flight at once. */ +const MAX_IN_FLIGHT = 4; + +const jid = (i: number) => `9725${String(i).padStart(8, '0')}@s.whatsapp.net`; +const url = (j: string, v = 1) => `https://pps.whatsapp.test/${j.split('@')[0]}/v${v}.jpg`; + +/** A fake profilePictureUrl IQ that takes a moment, and records how many ran at once. */ +function pictureServer(service: any) { + const state = { inFlight: 0, maxInFlight: 0, version: 1 }; + const fn = vi.fn(async (j: string) => { + state.inFlight++; + state.maxInFlight = Math.max(state.maxInFlight, state.inFlight); + await new Promise((r) => setTimeout(r, 2)); + state.inFlight--; + return url(j, state.version); + }); + service.client.profilePictureUrl = fn; + const perJid = () => { + const counts: Record = {}; + for (const [j] of fn.mock.calls) counts[j] = (counts[j] ?? 0) + 1; + return counts; + }; + return { fn, state, perJid }; +} + +/** Wait until the handlers have stopped asking for pictures. */ +async function quiet(service: any, fn: { mock: { calls: unknown[] } }) { + let last = -1; + while (fn.mock.calls.length !== last) { + last = fn.mock.calls.length; + await settle(service); + await new Promise((r) => setTimeout(r, 20)); + } +} + +const itemsFor = (event: string, j: string) => + emitted + .filter((e) => e.event === event) + .flatMap((e) => [].concat(e.data)) + .filter((c: any) => c?.remoteJid === j); + +/** What the socket emits for an incoming text: the message, and the sender's profile name (chats.js upsertMessage). */ +const incoming = (j: string, id: string, pushName: string) => ({ + 'messages.upsert': { messages: [{ ...msg(j, id, 'hi').message, pushName }], type: 'notify' }, + 'contacts.update': [{ id: j, notify: pushName, verifiedName: undefined }], +}); + +describe('profile pictures are looked up once per contact, not once per event', () => { + beforeEach(() => void emitted.splice(0)); + afterEach(() => void vi.useRealTimers()); + + it('a history batch of many contacts asks for each picture once, a few at a time, and the payload keeps the picture', async () => { + const N = 40; + const { service, ev } = await makeService(); + const server = pictureServer(service); + const contacts = Array.from({ length: N }, (_, i) => jid(i)); + + await deliver(service, ev, { + 'messaging-history.set': historyEvent({ + syncType: INITIAL_BOOTSTRAP, + progress: 100, + conversations: contacts.map((j, i) => ({ id: j, name: `Contact ${i}`, messages: [msg(j, `H${i}`, 'hi')] })), + }), + }); + await quiet(service, server.fn); + // The same people again, as the push-name sync that follows the bootstrap. + await deliver(service, ev, { + 'messaging-history.set': historyEvent({ + syncType: PUSH_NAME, + pushnames: contacts.map((j, i) => ({ id: j, pushname: `Profile ${i}` })), + }), + }); + await quiet(service, server.fn); + + expect(server.fn).toHaveBeenCalledTimes(N); + expect(Object.values(server.perJid()).every((n) => n === 1)).toBe(true); + expect(server.state.maxInFlight).toBeLessThanOrEqual(MAX_IN_FLIGHT); + + for (const j of contacts) { + const updates = itemsFor('contacts.update', j); + expect(updates).toHaveLength(2); + expect(updates.map((u: any) => u.profilePicUrl)).toEqual([url(j), url(j)]); + } + }); + + it('a burst of inbound messages from a few senders asks once per sender, not per message', async () => { + const SENDERS = [jid(101), jid(102), jid(103)]; + const M = 30; + const { service, ev } = await makeService(); + const server = pictureServer(service); + + for (let i = 0; i < M; i++) { + const sender = SENDERS[i % SENDERS.length]; + await deliver(service, ev, incoming(sender, `M${i}`, `Sender ${sender.slice(4, 7)}`)); + } + await quiet(service, server.fn); + + expect(server.perJid()).toEqual(Object.fromEntries(SENDERS.map((j) => [j, 1]))); + for (const j of SENDERS) { + const payloads = [...itemsFor('contacts.upsert', j), ...itemsFor('contacts.update', j)]; + // Each message yields one contact payload from messages.upsert and one from contacts.update. + expect(payloads).toHaveLength((2 * M) / SENDERS.length); + expect(new Set(payloads.map((p: any) => p.profilePicUrl))).toEqual(new Set([url(j)])); + } + }); + + it('a lookup is kept for an hour, then asked again', async () => { + vi.useFakeTimers({ toFake: ['Date'] }); + const A = jid(201); + const { service, ev } = await makeService(); + const server = pictureServer(service); + + await deliver(service, ev, incoming(A, 'T1', 'Alpha')); + await quiet(service, server.fn); + vi.setSystemTime(Date.now() + HOUR - 60_000); + await deliver(service, ev, incoming(A, 'T2', 'Alpha')); + await quiet(service, server.fn); + expect(server.fn).toHaveBeenCalledTimes(1); + + vi.setSystemTime(Date.now() + 2 * 60_000); + server.state.version = 2; + await deliver(service, ev, incoming(A, 'T3', 'Alpha')); + await quiet(service, server.fn); + expect(server.fn).toHaveBeenCalledTimes(2); + expect(itemsFor('contacts.update', A).at(-1).profilePicUrl).toBe(url(A, 2)); + }); + + it('a picture notification refreshes the picture, and a removal clears it without asking', async () => { + const A = jid(301); + const B = jid(302); + const { service, ev } = await makeService(); + const server = pictureServer(service); + + await deliver(service, ev, { ...incoming(A, 'P1', 'Alpha') }); + await deliver(service, ev, { ...incoming(B, 'P2', 'Bravo') }); + await quiet(service, server.fn); + expect(server.fn).toHaveBeenCalledTimes(2); + + // Baileys (messages-recv.js, notification type `picture`) emits no builder-made event: hand-written. + server.state.version = 2; + await deliver(service, ev, { 'contacts.update': [{ id: A, imgUrl: 'changed' }] }); + await deliver(service, ev, { 'contacts.update': [{ id: B, imgUrl: 'removed' }] }); + await quiet(service, server.fn); + + expect(server.perJid()).toEqual({ [A]: 2, [B]: 1 }); + expect(itemsFor('contacts.update', A).at(-1).profilePicUrl).toBe(url(A, 2)); + expect(itemsFor('contacts.update', B).at(-1).profilePicUrl).toBeNull(); + + // And the refreshed picture is what the next message carries, without another lookup. + await deliver(service, ev, incoming(A, 'P3', 'Alpha')); + await quiet(service, server.fn); + expect(server.fn).toHaveBeenCalledTimes(3); + expect(itemsFor('contacts.update', A).at(-1).profilePicUrl).toBe(url(A, 2)); + }); + + it('listing all groups twice asks for each group picture once', async () => { + const GROUPS = Array.from({ length: 12 }, (_, i) => `1203630000000${String(i).padStart(5, '0')}@g.us`); + const { service } = await makeService(); + const server = pictureServer(service); + const meta = (id: string) => ({ id, subject: `Group ${id.slice(-8, -5)}`, participants: [], creation: 1_700_000_000 }); + service.client.groupFetchAllParticipating = async () => Object.fromEntries(GROUPS.map((id) => [id, meta(id)])); + + const first = await service.fetchAllGroups({ getParticipants: 'false' }); + const second = await service.fetchAllGroups({ getParticipants: 'false' }); + + expect(server.fn).toHaveBeenCalledTimes(GROUPS.length); + expect(first.map((g: any) => g.pictureUrl)).toEqual(GROUPS.map((id) => url(id))); + expect(second.map((g: any) => g.pictureUrl)).toEqual(GROUPS.map((id) => url(id))); + }); + + it('an explicit picture request still asks WhatsApp, and later events carry what it found', async () => { + const A = jid(401); + const { service, ev } = await makeService(); + const server = pictureServer(service); + + await deliver(service, ev, incoming(A, 'E1', 'Alpha')); + await quiet(service, server.fn); + server.state.version = 2; + expect((await service.profilePicture(A)).profilePictureUrl).toBe(url(A, 2)); + expect(server.fn).toHaveBeenCalledTimes(2); + + await deliver(service, ev, incoming(A, 'E2', 'Alpha')); + await quiet(service, server.fn); + expect(server.fn).toHaveBeenCalledTimes(2); + expect(itemsFor('contacts.update', A).at(-1).profilePicUrl).toBe(url(A, 2)); + }); +}); From fa79a62359ce2575c4940b0f6b12fc899fd41042 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:32:38 +0300 Subject: [PATCH 035/157] fix(baileys): cache profile picture lookups and bound their concurrency The event handlers (contacts.upsert, contacts.update, messages.upsert) and fetchAllGroups now read a per-instance picture cache: one lookup per jid per hour, a jid already being looked up is not asked again, and at most four lookups are in flight. WhatsApp's picture notification (contacts.update imgUrl 'changed' or 'removed') refreshes or clears the kept picture. Explicit requests (profilePicture: /chat/fetchProfilePictureUrl, findGroup, fetchProfile, Chatwoot, the connection's own picture) still ask WhatsApp and update the cache. Logout clears it. Payloads keep profilePicUrl. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 70 ++++++++++++++++--- 1 file changed, 62 insertions(+), 8 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 36eab10356..7673957288 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -259,6 +259,17 @@ export class BaileysStartupService extends ChannelStartupService { private readonly MESSAGE_CACHE_TTL_SECONDS = 5 * 60; // 5 minutes - avoid duplicate message processing private readonly UPDATE_CACHE_TTL_SECONDS = 30 * 60; // 30 minutes - avoid duplicate status updates + // Profile pictures seen by the event handlers: one IQ per jid per hour, a few + // in flight at once. Every contacts.upsert, contacts.update and inbound message + // used to ask WhatsApp again, which at link time is thousands of IQs in a + // burst. WhatsApp's picture notification (contacts.update imgUrl) refreshes it. + private readonly PICTURE_TTL_MS = 60 * 60 * 1000; + private readonly PICTURE_MAX_IN_FLIGHT = 4; + private readonly pictureCache = new Map(); + private readonly pictureLookups = new Map>(); + private pictureSlots = 0; + private readonly pictureWaiters: (() => void)[] = []; + public stateConnection: wa.StateConnection = { state: 'close' }; public phoneNumber: string; @@ -269,6 +280,7 @@ export class BaileysStartupService extends ChannelStartupService { public async logoutInstance() { this.messageProcessor.onDestroy(); + this.pictureCache.clear(); // Baileys' logout tells WhatsApp to remove this device and then ends the socket. // With the socket down the first step throws ('Connection Closed') and the socket @@ -894,7 +906,7 @@ export class BaileysStartupService extends ChannelStartupService { contacts.map(async (contact) => ({ remoteJid: contact.id, pushName: contact?.name || contact?.verifiedName || contact.id.split('@')[0], - profilePicUrl: (await this.profilePicture(contact.id)).profilePictureUrl, + profilePicUrl: (await this.cachedProfilePicture(contact.id)).profilePictureUrl, instanceId: this.instanceId, })), ); @@ -945,10 +957,13 @@ export class BaileysStartupService extends ChannelStartupService { const contactsRaw: { remoteJid: string; pushName?: string; profilePicUrl?: string; instanceId: string }[] = []; for await (const contact of contacts) { this.logger.debug(`Updating contact: ${JSON.stringify(contact, null, 2)}`); + // imgUrl is set only by WhatsApp's picture notification: 'changed' or 'removed'. + if (contact.imgUrl === 'removed') this.pictureCache.set(createJid(contact.id), { url: null, at: Date.now() }); contactsRaw.push({ remoteJid: contact.id, pushName: contact?.name ?? contact?.verifiedName, - profilePicUrl: (await this.profilePicture(contact.id)).profilePictureUrl, + profilePicUrl: (await this.cachedProfilePicture(contact.id, { fresh: contact.imgUrl === 'changed' })) + .profilePictureUrl, instanceId: this.instanceId, }); } @@ -1600,7 +1615,7 @@ export class BaileysStartupService extends ChannelStartupService { } = { remoteJid: received.key.remoteJid, pushName: received.key.fromMe ? '' : received.key.fromMe == null ? '' : received.pushName, - profilePicUrl: (await this.profilePicture(received.key.remoteJid)).profilePictureUrl, + profilePicUrl: (await this.cachedProfilePicture(received.key.remoteJid)).profilePictureUrl, instanceId: this.instanceId, }; @@ -2155,15 +2170,54 @@ export class BaileysStartupService extends ChannelStartupService { ); } + /** Asks WhatsApp now (an explicit request), and keeps the answer for the event handlers. */ public async profilePicture(number: string) { const jid = createJid(number); + let profilePictureUrl: string | null; try { - const profilePictureUrl = await this.client.profilePictureUrl(jid, 'image'); - - return { wuid: jid, profilePictureUrl }; + profilePictureUrl = await this.client.profilePictureUrl(jid, 'image'); } catch { - return { wuid: jid, profilePictureUrl: null }; + profilePictureUrl = null; + } + + this.pictureCache.set(jid, { url: profilePictureUrl, at: Date.now() }); + return { wuid: jid, profilePictureUrl }; + } + + /** The picture the event handlers report: kept for PICTURE_TTL_MS, one lookup per jid at a time. */ + private async cachedProfilePicture(number: string, opts: { fresh?: boolean } = {}) { + const jid = createJid(number); + const kept = this.pictureCache.get(jid); + + if (!opts.fresh && kept && Date.now() - kept.at < this.PICTURE_TTL_MS) { + return { wuid: jid, profilePictureUrl: kept.url }; + } + + let lookup = opts.fresh ? undefined : this.pictureLookups.get(jid); + if (!lookup) { + const started = this.withPictureSlot(() => this.profilePicture(jid)) + .then((r) => r.profilePictureUrl) + .finally(() => { + if (this.pictureLookups.get(jid) === started) this.pictureLookups.delete(jid); + }); + this.pictureLookups.set(jid, started); + lookup = started; + } + + return { wuid: jid, profilePictureUrl: await lookup }; + } + + private async withPictureSlot(fn: () => Promise): Promise { + while (this.pictureSlots >= this.PICTURE_MAX_IN_FLIGHT) { + await new Promise((resolve) => this.pictureWaiters.push(resolve)); + } + this.pictureSlots++; + try { + return await fn(); + } finally { + this.pictureSlots--; + this.pictureWaiters.shift()?.(); } } @@ -4594,7 +4648,7 @@ export class BaileysStartupService extends ChannelStartupService { let groups = []; for (const group of fetch) { - const picture = await this.profilePicture(group.id); + const picture = await this.cachedProfilePicture(group.id); const result = { id: group.id, From 0050af528fb474d823710f489c4f6a62a736c34f Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:33:58 +0300 Subject: [PATCH 036/157] test: one pairing code per connect attempt, and a fresh QR budget per attempt With a phone number set, Evolution requested a new pairing code on every QR refresh (every 45s), and Baileys makes a fresh code and pushes a "link a device" notification to the phone each time, so the code being typed dies. The QR count was reset only at construction and after the limit, so a later connect attempt inherited the budget earlier ones used. These tests specify one code per socket, a new code for a new socket, a budget that spans Baileys' own reconnects within one attempt, and a fresh budget (and cleared QR state) for a new connect attempt. Co-Authored-By: Claude Opus 5.5 --- test/connect/pairing-code.test.ts | 163 ++++++++++++++++++++++++++++++ 1 file changed, 163 insertions(+) create mode 100644 test/connect/pairing-code.test.ts diff --git a/test/connect/pairing-code.test.ts b/test/connect/pairing-code.test.ts new file mode 100644 index 0000000000..9136bc4ce4 --- /dev/null +++ b/test/connect/pairing-code.test.ts @@ -0,0 +1,163 @@ +// Linking with a phone number: while the socket waits to be paired, Baileys +// rotates the QR every qrTimeout (45s, createClient), emitting connection.update +// { qr } each time, and after the last ref it closes the socket (408) and +// Evolution opens a new one. Evolution asked for a NEW pairing code on every QR +// (connectionUpdate), and Baileys' requestPairingCode (rc14 socket.js) makes a +// fresh code and sends link_code_companion_reg with +// should_show_push_notification 'true': the code the person is typing dies and +// their phone gets another "link a device" push, every 45s. +// +// The QR count (QRCODE_LIMIT) was reset only when the instance is built and +// after the limit is hit, so a later connect attempt on the same instance +// started with the budget earlier attempts had used, and was refused early. +import { vi } from 'vitest'; + +const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + // connectionUpdate waits a second before asking for the code; the wait is not what is tested. + return { ...orig, default: socketSpy, makeWASocket: socketSpy, delay: (ms: number) => orig.delay(Math.min(ms, 5)) }; +}); +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); + +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from 'vitest'; + +import { makeService } from '../helpers/baileys-service'; +import { fakeSocket, stubAuthState } from '../helpers/connect'; +import { emitted } from '../helpers/fake-server-module'; +import { loopbackOnly } from '../helpers/local-net'; + +const PHONE = '972500000009'; + +let codes = 0; +const sockets: any[] = []; +socketSpy.mockImplementation(() => { + const socket: any = { + ...fakeSocket(), + requestPairingCode: vi.fn(async () => `CODE${String(++codes).padStart(4, '0')}`), + logout: vi.fn(async () => undefined), + }; + socket.ws.close = vi.fn(); + sockets.push(socket); + return socket; +}); + +let guard: ReturnType; +beforeAll(() => void (guard = loopbackOnly())); +afterAll(() => { + expect(guard.refused).toEqual([]); + guard.restore(); +}); + +const qrUpdates = () => emitted.filter((e) => e.event === 'qrcode.updated' && e.data?.qrcode); +const refusals = () => emitted.filter((e) => e.event === 'connection.update' && e.data?.state === 'refused'); +const requests = () => sockets.reduce((n, s) => n + s.requestPairingCode.mock.calls.length, 0); + +async function until(done: () => boolean) { + for (let i = 0; i < 400 && !done(); i++) await new Promise((r) => setTimeout(r, 5)); + if (!done()) throw new Error('timed out waiting'); +} + +/** Baileys shows the next QR ref; wait until Evolution has handled it (its webhook, or the refusal). */ +async function nextQr(n: number) { + const socket = sockets.at(-1); + const before = qrUpdates().length + refusals().length; + socket.ev.emit('connection.update', { qr: `2@ref-${n},noise,identity,adv` }); + await until(() => qrUpdates().length + refusals().length > before); + await new Promise((r) => setTimeout(r, 10)); +} + +/** The QR refs ran out: Baileys ends the socket with 408, and Evolution reconnects on its own. */ +async function refsEnded() { + const count = sockets.length; + sockets.at(-1).ev.emit('connection.update', { connection: 'close', lastDisconnect: { error: { output: { statusCode: 408 } } } }); + await until(() => sockets.length > count); +} + +async function service() { + const { service } = await makeService(); + stubAuthState(service); + return service; +} + +describe('one pairing code per socket, and a fresh QR budget per connect attempt', () => { + const limit = process.env.QRCODE_LIMIT; + beforeEach(() => { + emitted.splice(0); + sockets.splice(0); + }); + afterEach(() => { + if (limit === undefined) delete process.env.QRCODE_LIMIT; + else process.env.QRCODE_LIMIT = limit; + }); + + it('QR refreshes on one socket keep the pairing code the person is typing', async () => { + const s = await service(); + await s.connectToWhatsapp(PHONE); + + await nextQr(1); + await nextQr(2); + await nextQr(3); + + expect(sockets).toHaveLength(1); + expect(sockets[0].requestPairingCode.mock.calls).toEqual([[PHONE]]); + const code = qrUpdates()[0].data.qrcode.pairingCode; + expect(code).toMatch(/^CODE\d{4}$/); + expect(qrUpdates().map((e) => e.data.qrcode.pairingCode)).toEqual([code, code, code]); + expect(s.qrCode.pairingCode).toBe(code); + }); + + it('a new socket, after the refs run out, asks for a new code once', async () => { + const s = await service(); + await s.connectToWhatsapp(PHONE); + await nextQr(1); + await nextQr(2); + await refsEnded(); + await nextQr(3); + await nextQr(4); + + expect(sockets).toHaveLength(2); + expect(sockets.map((x) => x.requestPairingCode.mock.calls.length)).toEqual([1, 1]); + const [first, second] = [qrUpdates()[0], qrUpdates()[2]].map((e) => e.data.qrcode.pairingCode); + expect(second).not.toBe(first); + expect(qrUpdates().map((e) => e.data.qrcode.pairingCode)).toEqual([first, first, second, second]); + }); + + it('within one connect attempt the QR budget spans reconnects, and is refused at the limit', async () => { + process.env.QRCODE_LIMIT = '3'; + const s = await service(); + await s.connectToWhatsapp(PHONE); + await nextQr(1); + await nextQr(2); + await refsEnded(); + await nextQr(3); + expect(refusals()).toHaveLength(0); + await nextQr(4); + + expect(refusals()).toHaveLength(1); + expect(requests()).toBe(2); + }); + + it('a new connect attempt (logout, then connect) starts with a fresh QR budget', async () => { + process.env.QRCODE_LIMIT = '3'; + const s = await service(); + await s.connectToWhatsapp(PHONE); + await nextQr(1); + await nextQr(2); + + // What a client does for each new code request: log out, then connect again. + await s.logoutInstance(); + await s.connectToWhatsapp(PHONE); + expect(s.qrCode).toEqual({ pairingCode: undefined, code: undefined, base64: undefined, count: 0 }); + await nextQr(3); + await nextQr(4); + + expect(refusals()).toHaveLength(0); + expect(s.qrCode.count).toBe(2); + expect(sockets.map((x) => x.requestPairingCode.mock.calls.length)).toEqual([1, 1]); + }); +}); From 192605e9d133e0f830189a0ae81d62b48efcd1bd Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:34:34 +0300 Subject: [PATCH 037/157] fix(baileys): request the pairing code once per attempt and reset the QR count connectionUpdate asks for a pairing code only on the first QR of a socket; later QR refreshes on that socket carry the same code, so the code being typed stays valid and the phone gets one push per socket instead of one every 45s. A new socket (Baileys closed it after the QR refs ran out) gets a new code, as it must. connectToWhatsapp is now a new attempt: it resets the QR state (count, code, base64, pairing code) before connecting. Baileys' own reconnect on close goes through a private connect() that keeps the count, so QRCODE_LIMIT still ends an attempt that nobody completes. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 26 ++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 7673957288..64b502c759 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -274,6 +274,11 @@ export class BaileysStartupService extends ChannelStartupService { public phoneNumber: string; + // The socket a pairing code was requested on. Baileys' requestPairingCode makes + // a new code and pushes a notification to the phone, so it is asked once per + // socket, not on every QR refresh (which would kill the code being typed). + private pairingCodeSocket?: WASocket; + public get connectionStatus() { return this.stateConnection; } @@ -409,8 +414,16 @@ export class BaileysStartupService extends ChannelStartupService { }; if (this.phoneNumber) { - await delay(1000); - this.instance.qrcode.pairingCode = await this.client.requestPairingCode(this.phoneNumber); + if (this.pairingCodeSocket !== this.client) { + const socket = (this.pairingCodeSocket = this.client); + try { + await delay(1000); + this.instance.qrcode.pairingCode = await socket.requestPairingCode(this.phoneNumber); + } catch (error) { + if (this.pairingCodeSocket === socket) this.pairingCodeSocket = undefined; + throw error; + } + } } else { this.instance.qrcode.pairingCode = null; } @@ -464,7 +477,8 @@ export class BaileysStartupService extends ChannelStartupService { const codesToNotReconnect = [DisconnectReason.loggedOut, DisconnectReason.forbidden, 402, 406]; const shouldReconnect = !codesToNotReconnect.includes(statusCode); if (shouldReconnect) { - await this.connectToWhatsapp(this.phoneNumber); + // Baileys' own reconnect (QR refs ended, a dropped socket) is the same attempt: the QR budget carries over. + await this.connect(this.phoneNumber); } else { this.sendDataWebhook(Events.STATUS_INSTANCE, { instance: this.instance.name, @@ -766,7 +780,13 @@ export class BaileysStartupService extends ChannelStartupService { return this.client; } + /** A new connect attempt: a fresh QR budget, and no QR or pairing code left from an earlier attempt. */ public async connectToWhatsapp(number?: string): Promise { + this.instance.qrcode = { count: 0 }; + return await this.connect(number); + } + + private async connect(number?: string): Promise { try { this.loadChatwoot(); // The socket takes syncFullHistory, groupsIgnore, readStatus and alwaysOnline as config: read them first. From 053d26d54dc11c0769d236dca389148815431289 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:37:46 +0300 Subject: [PATCH 038/157] test: a group listing does not query every group's metadata again Baileys' groupFetchAllParticipating emits groups.update with the full metadata of every group, and Evolution's groups.update handler refreshed its metadata cache with a groupMetadata query per group, all at once, so each fetchAllGroups cost one extra query per group. These tests specify that a complete item fills the cache as it is, and that partial items refetch each group once, at most four at a time. Co-Authored-By: Claude Opus 5.5 --- test/handlers/group-metadata.test.ts | 123 +++++++++++++++++++++++++++ 1 file changed, 123 insertions(+) create mode 100644 test/handlers/group-metadata.test.ts diff --git a/test/handlers/group-metadata.test.ts b/test/handlers/group-metadata.test.ts new file mode 100644 index 0000000000..34c46e01c4 --- /dev/null +++ b/test/handlers/group-metadata.test.ts @@ -0,0 +1,123 @@ +// Evolution keeps a group metadata cache (updateGroupMetadataCache) that +// Baileys reads before every group send (cachedGroupMetadata). Its +// groups.update handler refreshed that cache with a groupMetadata query per +// group, all at once. Baileys' groupFetchAllParticipating (rc14 groups.js) +// emits groups.update with the FULL metadata of every group, so each +// fetchAllGroups cost one groupMetadata query per group on top of the listing +// itself, although the event already carried the answer: 259 queries every +// listing for one measured account. +// +// An item that carries participants is complete metadata and fills the cache +// as it is. A partial item (a subject or setting change, Baileys' +// process-message) still refetches that group, once, a few at a time. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { beforeEach, describe, expect, it } from 'vitest'; + +import { deliver, makeService, settle } from '../helpers/baileys-service'; +import { emitted } from '../helpers/fake-server-module'; + +/** The most metadata queries the service may have in flight at once. */ +const MAX_IN_FLIGHT = 4; +const MEMBER = (i: number) => `9725${String(i).padStart(8, '0')}@s.whatsapp.net`; + +let extractGroupMetadata: (node: any) => any; + +/** What WhatsApp answers for one group, turned into metadata by the Baileys under test. */ +function groupMeta(id: string, subject: string) { + const node = { + tag: 'group', + attrs: { id: id.split('@')[0], subject, s_t: '1700000000', creation: '1690000000', creator: MEMBER(1) }, + content: [ + { tag: 'participant', attrs: { jid: MEMBER(1), type: 'superadmin' } }, + { tag: 'participant', attrs: { jid: MEMBER(2) } }, + ], + }; + return extractGroupMetadata({ tag: 'result', attrs: {}, content: [node] }); +} + +const groupIds = (prefix: string, n: number) => + Array.from({ length: n }, (_, i) => `120363${prefix}${String(i).padStart(6, '0')}@g.us`); + +/** A fake socket side for groups: the listing emits groups.update as rc14 does; groupMetadata is counted. */ +function groupServer(service: any, listed: string[]) { + const state = { inFlight: 0, maxInFlight: 0 }; + const groupMetadata = vi.fn(async (id: string) => { + state.inFlight++; + state.maxInFlight = Math.max(state.maxInFlight, state.inFlight); + await new Promise((r) => setTimeout(r, 2)); + state.inFlight--; + return groupMeta(id, `Fetched ${id.slice(-8, -5)}`); + }); + service.client.groupMetadata = groupMetadata; + service.client.groupFetchAllParticipating = async () => { + const data = Object.fromEntries(listed.map((id) => [id, groupMeta(id, `Listed ${id.slice(-8, -5)}`)])); + service.client.ev.emit('groups.update', Object.values(data)); + return data; + }; + const perGroup = () => { + const counts: Record = {}; + for (const [id] of groupMetadata.mock.calls) counts[id] = (counts[id] ?? 0) + 1; + return counts; + }; + return { groupMetadata, state, perGroup }; +} + +async function quiet(service: any, fn: { mock: { calls: unknown[] } }) { + let last = -1; + while (fn.mock.calls.length !== last) { + last = fn.mock.calls.length; + await settle(service); + await new Promise((r) => setTimeout(r, 20)); + } +} + +describe('the group metadata cache is filled from groups.update', () => { + beforeEach(async () => { + emitted.splice(0); + ({ extractGroupMetadata } = await import('baileys/lib/Socket/groups.js' as any)); + }); + + it('listing all groups twice queries no group metadata, and the cache holds what the listing carried', async () => { + const GROUPS = groupIds('1000', 12); + const { service, ev } = await makeService(); + const server = groupServer(service, GROUPS); + await deliver(service, ev, {}); // wire the handlers + + await service.fetchAllGroups({ getParticipants: 'false' }); + await quiet(service, server.groupMetadata); + await service.fetchAllGroups({ getParticipants: 'false' }); + await quiet(service, server.groupMetadata); + + expect(server.groupMetadata).not.toHaveBeenCalled(); + for (const id of GROUPS) { + expect(await service.getGroupMetadataCache(id)).toEqual(groupMeta(id, `Listed ${id.slice(-8, -5)}`)); + } + expect(server.groupMetadata).not.toHaveBeenCalled(); + // The webhook is what it was: every listed group, as Baileys emitted it. + const updates = emitted.filter((e) => e.event === 'groups.update'); + expect(updates).toHaveLength(2); + expect(updates[0].data.map((g: any) => g.id)).toEqual(GROUPS); + }); + + it('partial updates refetch each group once, a few at a time', async () => { + const PARTIAL = groupIds('2000', 12); + const { service, ev } = await makeService(); + const server = groupServer(service, []); + + await deliver(service, ev, {}); // wire the handlers + // Hand-written: Baileys emits these from process-message.js (group stubs), which has no builder here. + // Two changes to one group arrive as two batches (in one batch the event buffer merges them). + ev.emit('groups.update', [{ id: PARTIAL[0], subject: 'Renamed' }]); + ev.emit('groups.update', [{ id: PARTIAL[0], announce: true }]); + ev.emit('groups.update', PARTIAL.slice(1).map((id) => ({ id, restrict: true }))); + await quiet(service, server.groupMetadata); + + expect(server.perGroup()).toEqual(Object.fromEntries(PARTIAL.map((id) => [id, 1]))); + expect(server.state.maxInFlight).toBeLessThanOrEqual(MAX_IN_FLIGHT); + expect((await service.getGroupMetadataCache(PARTIAL[0])).subject).toBe(`Fetched ${PARTIAL[0].slice(-8, -5)}`); + expect(server.groupMetadata).toHaveBeenCalledTimes(PARTIAL.length); + }); +}); From 4e3429844e6cbf35122037bebba38d29536778ed Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:38:49 +0300 Subject: [PATCH 039/157] fix(baileys): fill the group metadata cache from groups.update A groups.update item that carries participants is a group's full metadata (Baileys' groupFetchAllParticipating and the community listing emit them), so it goes into the cache as it is, with no groupMetadata query. A partial item (a subject or setting change) and group-participants.update refetch the group in the background: one refresh per group at a time, and at most four background queries in flight per instance, shared with the picture lookups through a small QueryLimiter. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 62 +++++++++++-------- src/utils/queryLimiter.ts | 26 ++++++++ 2 files changed, 62 insertions(+), 26 deletions(-) create mode 100644 src/utils/queryLimiter.ts diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 64b502c759..d4a8dc8941 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -85,6 +85,7 @@ import { fetchLatestWaWebVersion } from '@utils/fetchLatestWaWebVersion'; import { jidKind, makeBaileysLogger } from '@utils/log-privacy'; import { makeProxyAgent, makeProxyAgentUndici } from '@utils/makeProxyAgent'; import { getOnWhatsappCache, saveOnWhatsappCache } from '@utils/onWhatsappCache'; +import { QueryLimiter } from '@utils/queryLimiter'; import { status } from '@utils/renderStatus'; import { sendTelemetry } from '@utils/sendTelemetry'; import useMultiFileAuthStatePrisma from '@utils/use-multi-file-auth-state-prisma'; @@ -264,11 +265,11 @@ export class BaileysStartupService extends ChannelStartupService { // used to ask WhatsApp again, which at link time is thousands of IQs in a // burst. WhatsApp's picture notification (contacts.update imgUrl) refreshes it. private readonly PICTURE_TTL_MS = 60 * 60 * 1000; - private readonly PICTURE_MAX_IN_FLIGHT = 4; private readonly pictureCache = new Map(); private readonly pictureLookups = new Map>(); - private pictureSlots = 0; - private readonly pictureWaiters: (() => void)[] = []; + // Lookups the event handlers start on their own (pictures, group metadata refreshes): four at a time. + private readonly backgroundQueries = new QueryLimiter(4); + private readonly groupRefreshes = new Map>(); public stateConnection: wa.StateConnection = { state: 'close' }; @@ -1876,8 +1877,13 @@ export class BaileysStartupService extends ChannelStartupService { this.sendDataWebhook(Events.GROUPS_UPDATE, groupMetadataUpdate); groupMetadataUpdate.forEach((group) => { - if (isJidGroup(group.id)) { - this.updateGroupMetadataCache(group.id); + if (!isJidGroup(group.id)) return; + // A listing (groupFetchAllParticipating) carries each group's full metadata: keep it as it is. + // A change (subject, settings) carries only what changed: ask for the group once. + if (Array.isArray(group.participants)) { + this.keepGroupMetadata(group.id, group as GroupMetadata); + } else { + this.refreshGroupMetadata(group.id); } }); }, @@ -1943,7 +1949,7 @@ export class BaileysStartupService extends ChannelStartupService { this.sendDataWebhook(Events.GROUP_PARTICIPANTS_UPDATE, participantsUpdate); } - this.updateGroupMetadataCache(participantsUpdate.id); + this.refreshGroupMetadata(participantsUpdate.id); }, }; @@ -2216,7 +2222,8 @@ export class BaileysStartupService extends ChannelStartupService { let lookup = opts.fresh ? undefined : this.pictureLookups.get(jid); if (!lookup) { - const started = this.withPictureSlot(() => this.profilePicture(jid)) + const started = this.backgroundQueries + .run(() => this.profilePicture(jid)) .then((r) => r.profilePictureUrl) .finally(() => { if (this.pictureLookups.get(jid) === started) this.pictureLookups.delete(jid); @@ -2228,19 +2235,6 @@ export class BaileysStartupService extends ChannelStartupService { return { wuid: jid, profilePictureUrl: await lookup }; } - private async withPictureSlot(fn: () => Promise): Promise { - while (this.pictureSlots >= this.PICTURE_MAX_IN_FLIGHT) { - await new Promise((resolve) => this.pictureWaiters.push(resolve)); - } - this.pictureSlots++; - try { - return await fn(); - } finally { - this.pictureSlots--; - this.pictureWaiters.shift()?.(); - } - } - public async getStatus(number: string) { const jid = createJid(number); @@ -4502,16 +4496,32 @@ export class BaileysStartupService extends ChannelStartupService { } // Group + private async keepGroupMetadata(groupJid: string, meta: GroupMetadata) { + const cacheConf = this.configService.get('CACHE'); + + if ((cacheConf?.REDIS?.ENABLED && cacheConf?.REDIS?.URI !== '') || cacheConf?.LOCAL?.ENABLED) { + this.logger.verbose(`Updating cache for group: ${groupJid}`); + await groupMetadataCache.set(groupJid, { timestamp: Date.now(), data: meta }); + } + } + + /** Refetch a group's metadata in the background: one query per group at a time, a few groups at once. */ + private refreshGroupMetadata(groupJid: string) { + let refresh = this.groupRefreshes.get(groupJid); + if (!refresh) { + refresh = this.backgroundQueries + .run(() => this.updateGroupMetadataCache(groupJid)) + .finally(() => this.groupRefreshes.delete(groupJid)); + this.groupRefreshes.set(groupJid, refresh); + } + return refresh; + } + private async updateGroupMetadataCache(groupJid: string) { try { const meta = await this.client.groupMetadata(groupJid); - const cacheConf = this.configService.get('CACHE'); - - if ((cacheConf?.REDIS?.ENABLED && cacheConf?.REDIS?.URI !== '') || cacheConf?.LOCAL?.ENABLED) { - this.logger.verbose(`Updating cache for group: ${groupJid}`); - await groupMetadataCache.set(groupJid, { timestamp: Date.now(), data: meta }); - } + await this.keepGroupMetadata(groupJid, meta); return meta; } catch (error) { diff --git a/src/utils/queryLimiter.ts b/src/utils/queryLimiter.ts new file mode 100644 index 0000000000..a7b812b9fa --- /dev/null +++ b/src/utils/queryLimiter.ts @@ -0,0 +1,26 @@ +/** + * Runs at most `max` WhatsApp queries at once; the rest wait their turn, in order. + * One per instance: a burst of events must not become a burst of IQs. + */ +export class QueryLimiter { + private inFlight = 0; + private readonly waiting: (() => void)[] = []; + + constructor(private readonly max: number) {} + + public async run(query: () => Promise): Promise { + if (this.inFlight >= this.max) { + // The slot is handed over by the query that finishes, so it is ours when we wake. + await new Promise((resolve) => this.waiting.push(resolve)); + } else { + this.inFlight++; + } + try { + return await query(); + } finally { + const next = this.waiting.shift(); + if (next) next(); + else this.inFlight--; + } + } +} From b1a88c01b921e86e042818a7820079ef8de64de9 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:39:46 +0300 Subject: [PATCH 040/157] test: history and live messages never wait on profile picture lookups The messaging-history.set handler awaited contacts.upsert, which awaited a picture lookup for every contact in the batch, and Evolution handles one event batch at a time. With the history contacts' lookups held open, the next history batch and two live messages must still go out, the live sender's picture must not queue behind the history lookups, and the history pictures must follow on contacts.update once answered, one lookup each. Co-Authored-By: Claude Opus 5.5 --- test/handlers/history-pictures.test.ts | 109 +++++++++++++++++++++++++ 1 file changed, 109 insertions(+) create mode 100644 test/handlers/history-pictures.test.ts diff --git a/test/handlers/history-pictures.test.ts b/test/handlers/history-pictures.test.ts new file mode 100644 index 0000000000..1618ed3d2e --- /dev/null +++ b/test/handlers/history-pictures.test.ts @@ -0,0 +1,109 @@ +// Evolution handles events one batch at a time (eventProcessingQueue), and +// its messaging-history.set handler awaited contacts.upsert, which awaited a +// profile picture lookup for every contact in the batch before returning. A +// first link brings hundreds of new contacts, so the next history batch and +// every live message waited behind hundreds of picture IQs. +// +// The history batch goes out as soon as it is read; the pictures follow on +// contacts.update when their lookups finish, and those lookups never take the +// last query slot, so a live sender's lookup does not queue behind them. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; + +import { historyEvent, msg } from '../helpers/baileys-fixtures'; +import { deliver, makeService, settle } from '../helpers/baileys-service'; +import { emitted } from '../helpers/fake-server-module'; + +const INITIAL_BOOTSTRAP = 0; +const RECENT = 3; +const MAX_IN_FLIGHT = 4; + +const jid = (i: number) => `9725${String(i).padStart(8, '0')}@s.whatsapp.net`; +const url = (j: string) => `https://pps.whatsapp.test/${j.split('@')[0]}.jpg`; +const LIVE = jid(900); + +const of = (event: string) => emitted.filter((e) => e.event === event); +const itemsFor = (event: string, j: string) => + of(event) + .flatMap((e) => [].concat(e.data)) + .filter((c: any) => c?.remoteJid === j); + +async function until(done: () => boolean, what: string) { + for (let i = 0; i < 300 && !done(); i++) await new Promise((r) => setTimeout(r, 5)); + if (!done()) throw new Error(`timed out waiting for ${what}`); +} + +const incoming = (j: string, id: string, pushName: string) => ({ + 'messages.upsert': { messages: [{ ...msg(j, id, 'hi').message, pushName }], type: 'notify' }, + 'contacts.update': [{ id: j, notify: pushName, verifiedName: undefined }], +}); + +/** Emit a batch the way the socket does, without waiting for Evolution to handle it. */ +async function emit(ev: any, events: Record) { + ev.buffer(); + for (const [name, payload] of Object.entries(events)) ev.emit(name, payload); + await ev.flush(); +} + +const history = (syncType: number, jids: string[]) => ({ + 'messaging-history.set': historyEvent({ + syncType, + progress: 50, + conversations: jids.map((j, i) => ({ id: j, name: `Contact ${j.slice(8, 12)}`, messages: [msg(j, `H${j.slice(8, 12)}${i}`, 'hi')] })), + }), +}); + +describe('history and live messages never wait on profile picture lookups', () => { + let open = () => undefined as void; + beforeEach(() => void emitted.splice(0)); + // A failing run must not leave lookups hanging on a closed gate. + afterEach(() => open()); + + it('a history batch with slow picture lookups holds up neither the next batch nor a live message', async () => { + const FIRST = Array.from({ length: 30 }, (_, i) => jid(i)); + const SECOND = Array.from({ length: 30 }, (_, i) => jid(100 + i)); + const HISTORY = new Set([...FIRST, ...SECOND]); + const { service, ev } = await makeService(); + await deliver(service, ev, {}); // wire the handlers + + // History contacts' lookups hang until the gate opens; any other lookup answers at once. + const gate = new Promise((r) => (open = r)); + const state = { inFlight: 0, maxInFlight: 0 }; + const lookup = vi.fn(async (j: string) => { + state.inFlight++; + state.maxInFlight = Math.max(state.maxInFlight, state.inFlight); + if (HISTORY.has(j)) await gate; + else await new Promise((r) => setTimeout(r, 1)); + state.inFlight--; + return url(j); + }); + service.client.profilePictureUrl = lookup; + + await emit(ev, history(INITIAL_BOOTSTRAP, FIRST)); + await emit(ev, history(RECENT, SECOND)); + await emit(ev, incoming(LIVE, 'L1', 'Live')); + await emit(ev, incoming(LIVE, 'L2', 'Live')); + + // Everything arrives while not one history picture has been answered. + await until(() => of('messages.set').length === 2, 'both history batches'); + await until(() => of('messages.upsert').length === 2, 'both live messages'); + await until(() => itemsFor('contacts.update', LIVE).length === 2, "the live sender's contact payloads"); + expect(of('messages.upsert').map((e) => e.data.key.id)).toEqual(['L1', 'L2']); + for (const j of HISTORY) expect(itemsFor('contacts.upsert', j)).toHaveLength(1); + // The live sender's picture was looked up beside the stalled history lookups, not behind them. + expect(itemsFor('contacts.update', LIVE).map((c: any) => c.profilePicUrl)).toEqual([url(LIVE), url(LIVE)]); + for (const j of HISTORY) expect(itemsFor('contacts.update', j)).toEqual([]); + expect(state.maxInFlight).toBeLessThanOrEqual(MAX_IN_FLIGHT); + + // The pictures follow on contacts.update once WhatsApp answers, one lookup per contact. + open(); + await until(() => [...HISTORY].every((j) => itemsFor('contacts.update', j).length === 1), 'the history pictures'); + await settle(service); + for (const j of HISTORY) expect(itemsFor('contacts.update', j).map((c: any) => c.profilePicUrl)).toEqual([url(j)]); + expect(lookup.mock.calls.filter(([j]) => HISTORY.has(j))).toHaveLength(HISTORY.size); + expect(state.maxInFlight).toBeLessThanOrEqual(MAX_IN_FLIGHT); + }); +}); From 5b4c3f5ccb6d5b30ff8d6b66aaecb6e10b9a7d30 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:40:55 +0300 Subject: [PATCH 041/157] fix(baileys): look up history contacts' pictures off the event path contacts.upsert (history and address-book sync) sends its webhook and stores the contacts, then returns; the picture lookups run detached and their result goes out on contacts.update as before, when WhatsApp answers. The history handler, which awaits contacts.upsert on the per-instance event queue, no longer holds the next batch or live messages behind them. Those lookups are bulk in the QueryLimiter: they never take the last of the four slots, and waiting live lookups start first, so a live sender's picture waits for at most one query, not the history backlog. A live lookup does not join a queued bulk lookup for the same jid, and a queued lookup rechecks the cache when its turn comes. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 143 ++++++++++-------- src/utils/queryLimiter.ts | 45 ++++-- 2 files changed, 116 insertions(+), 72 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index d4a8dc8941..2a8d0e94cd 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -266,7 +266,7 @@ export class BaileysStartupService extends ChannelStartupService { // burst. WhatsApp's picture notification (contacts.update imgUrl) refreshes it. private readonly PICTURE_TTL_MS = 60 * 60 * 1000; private readonly pictureCache = new Map(); - private readonly pictureLookups = new Map>(); + private readonly pictureLookups = new Map }>(); // Lookups the event handlers start on their own (pictures, group metadata refreshes): four at a time. private readonly backgroundQueries = new QueryLimiter(4); private readonly groupRefreshes = new Map>(); @@ -923,51 +923,8 @@ export class BaileysStartupService extends ChannelStartupService { ); } - const updatedContacts = await Promise.all( - contacts.map(async (contact) => ({ - remoteJid: contact.id, - pushName: contact?.name || contact?.verifiedName || contact.id.split('@')[0], - profilePicUrl: (await this.cachedProfilePicture(contact.id)).profilePictureUrl, - instanceId: this.instanceId, - })), - ); - - if (updatedContacts.length > 0) { - const usersContacts = updatedContacts.filter((c) => c.remoteJid.includes('@s.whatsapp')); - if (usersContacts) { - await saveOnWhatsappCache(usersContacts.map((c) => ({ remoteJid: c.remoteJid }))); - } - - this.sendDataWebhook(Events.CONTACTS_UPDATE, updatedContacts); - await Promise.all( - updatedContacts.map(async (contact) => { - if (this.configService.get('DATABASE').SAVE_DATA.CONTACTS) { - await this.prismaRepository.contact.updateMany({ - where: { remoteJid: contact.remoteJid, instanceId: this.instanceId }, - data: { profilePicUrl: contact.profilePicUrl }, - }); - } - - if (this.configService.get('CHATWOOT').ENABLED && this.localChatwoot?.enabled) { - const instance = { instanceName: this.instance.name, instanceId: this.instance.id }; - - const findParticipant = await this.chatwootService.findContact( - instance, - contact.remoteJid.split('@')[0], - ); - - if (!findParticipant) { - return; - } - - this.chatwootService.updateContact(instance, findParticipant.id, { - name: contact.pushName, - avatar_url: contact.profilePicUrl, - }); - } - }), - ); - } + // Pictures follow on contacts.update when their lookups finish: history waits for none of them. + void this.contactPictures(contacts); } catch (error) { console.error(error); this.logger.error(`Error: ${error.message}`); @@ -2196,6 +2153,56 @@ export class BaileysStartupService extends ChannelStartupService { ); } + /** Look up the pictures of contacts from contacts.upsert (history, address book) and send them on contacts.update. */ + private async contactPictures(contacts: Contact[]) { + try { + const updatedContacts = await Promise.all( + contacts.map(async (contact) => ({ + remoteJid: contact.id, + pushName: contact?.name || contact?.verifiedName || contact.id.split('@')[0], + profilePicUrl: (await this.cachedProfilePicture(contact.id, { bulk: true })).profilePictureUrl, + instanceId: this.instanceId, + })), + ); + + if (updatedContacts.length > 0) { + const usersContacts = updatedContacts.filter((c) => c.remoteJid.includes('@s.whatsapp')); + if (usersContacts) { + await saveOnWhatsappCache(usersContacts.map((c) => ({ remoteJid: c.remoteJid }))); + } + + this.sendDataWebhook(Events.CONTACTS_UPDATE, updatedContacts); + await Promise.all( + updatedContacts.map(async (contact) => { + if (this.configService.get('DATABASE').SAVE_DATA.CONTACTS) { + await this.prismaRepository.contact.updateMany({ + where: { remoteJid: contact.remoteJid, instanceId: this.instanceId }, + data: { profilePicUrl: contact.profilePicUrl }, + }); + } + + if (this.configService.get('CHATWOOT').ENABLED && this.localChatwoot?.enabled) { + const instance = { instanceName: this.instance.name, instanceId: this.instance.id }; + + const findParticipant = await this.chatwootService.findContact(instance, contact.remoteJid.split('@')[0]); + + if (!findParticipant) { + return; + } + + this.chatwootService.updateContact(instance, findParticipant.id, { + name: contact.pushName, + avatar_url: contact.profilePicUrl, + }); + } + }), + ); + } + } catch (error) { + this.logger.error(`Error: ${error.message}`); + } + } + /** Asks WhatsApp now (an explicit request), and keeps the answer for the event handlers. */ public async profilePicture(number: string) { const jid = createJid(number); @@ -2211,26 +2218,36 @@ export class BaileysStartupService extends ChannelStartupService { return { wuid: jid, profilePictureUrl }; } - /** The picture the event handlers report: kept for PICTURE_TTL_MS, one lookup per jid at a time. */ - private async cachedProfilePicture(number: string, opts: { fresh?: boolean } = {}) { + /** + * The picture the event handlers report: kept for PICTURE_TTL_MS, one lookup per jid at a time. + * Bulk lookups (history, address book) queue behind each other; a live event's lookup never joins one. + */ + private async cachedProfilePicture(number: string, opts: { fresh?: boolean; bulk?: boolean } = {}) { const jid = createJid(number); - const kept = this.pictureCache.get(jid); + const bulk = !!opts.bulk; + const kept = () => { + const k = this.pictureCache.get(jid); + return !opts.fresh && k && Date.now() - k.at < this.PICTURE_TTL_MS ? k : undefined; + }; - if (!opts.fresh && kept && Date.now() - kept.at < this.PICTURE_TTL_MS) { - return { wuid: jid, profilePictureUrl: kept.url }; - } + if (kept()) return { wuid: jid, profilePictureUrl: kept().url }; - let lookup = opts.fresh ? undefined : this.pictureLookups.get(jid); - if (!lookup) { - const started = this.backgroundQueries - .run(() => this.profilePicture(jid)) - .then((r) => r.profilePictureUrl) - .finally(() => { - if (this.pictureLookups.get(jid) === started) this.pictureLookups.delete(jid); - }); - this.pictureLookups.set(jid, started); - lookup = started; - } + const pending = opts.fresh ? undefined : this.pictureLookups.get(jid); + if (pending && (bulk || !pending.bulk)) return { wuid: jid, profilePictureUrl: await pending.lookup }; + + const lookup = this.backgroundQueries + // A queued lookup may find the picture already kept by the time its turn comes. + .run( + async () => { + const k = kept(); + return k ? k.url : (await this.profilePicture(jid)).profilePictureUrl; + }, + { bulk }, + ) + .finally(() => { + if (this.pictureLookups.get(jid)?.lookup === lookup) this.pictureLookups.delete(jid); + }); + this.pictureLookups.set(jid, { bulk, lookup }); return { wuid: jid, profilePictureUrl: await lookup }; } diff --git a/src/utils/queryLimiter.ts b/src/utils/queryLimiter.ts index a7b812b9fa..87bda98dc4 100644 --- a/src/utils/queryLimiter.ts +++ b/src/utils/queryLimiter.ts @@ -1,26 +1,53 @@ /** * Runs at most `max` WhatsApp queries at once; the rest wait their turn, in order. * One per instance: a burst of events must not become a burst of IQs. + * + * Bulk queries (a history batch's pictures) never take the last slot, so a + * query for a live event waits for at most one query ahead of it, not for the + * whole bulk backlog. Waiting live queries start before waiting bulk ones. */ export class QueryLimiter { private inFlight = 0; - private readonly waiting: (() => void)[] = []; + private bulkInFlight = 0; + private readonly waitingLive: (() => void)[] = []; + private readonly waitingBulk: (() => void)[] = []; constructor(private readonly max: number) {} - public async run(query: () => Promise): Promise { - if (this.inFlight >= this.max) { - // The slot is handed over by the query that finishes, so it is ours when we wake. - await new Promise((resolve) => this.waiting.push(resolve)); + public async run(query: () => Promise, opts: { bulk?: boolean } = {}): Promise { + const bulk = !!opts.bulk; + if (this.canStart(bulk)) { + this.take(bulk); } else { - this.inFlight++; + // The slot is taken for us by the query that finishes, so it is ours when we wake. + await new Promise((resolve) => (bulk ? this.waitingBulk : this.waitingLive).push(resolve)); } try { return await query(); } finally { - const next = this.waiting.shift(); - if (next) next(); - else this.inFlight--; + this.inFlight--; + if (bulk) this.bulkInFlight--; + this.wake(); + } + } + + private canStart(bulk: boolean) { + return this.inFlight < this.max && (!bulk || this.bulkInFlight < this.max - 1); + } + + private take(bulk: boolean) { + this.inFlight++; + if (bulk) this.bulkInFlight++; + } + + private wake() { + while (this.waitingLive.length && this.canStart(false)) { + this.take(false); + this.waitingLive.shift()(); + } + while (this.waitingBulk.length && this.canStart(true)) { + this.take(true); + this.waitingBulk.shift()(); } } } From 018d2009069f22459161c15ce9be3242a902d679 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:34:33 +0300 Subject: [PATCH 042/157] test: reconnects back off instead of spinning A reconnectable close (408, 428, 500, 503, 411...) makes Evolution connect again at once, with no delay and no limit, and each attempt fetches the WhatsApp Web version with no timeout. When the instance's proxy exit dies this spins. These tests close the socket repeatedly and require the next attempt to wait 1s, 2s, 4s... up to one a minute, never stopping, back to 1s after an open; a failed attempt to be retried; a waiting reconnect to be dropped on logout or delete; and the version fetch to give up after 10s on an exit that never answers. The logout codes stay final. Co-Authored-By: Claude Opus 5.5 --- test/connect/reconnect-backoff.test.ts | 154 +++++++++++++++++++++++++ test/helpers/connect.ts | 6 +- test/proxy/version-fetch.test.ts | 59 +++++++++- 3 files changed, 217 insertions(+), 2 deletions(-) create mode 100644 test/connect/reconnect-backoff.test.ts diff --git a/test/connect/reconnect-backoff.test.ts b/test/connect/reconnect-backoff.test.ts new file mode 100644 index 0000000000..fec97e0176 --- /dev/null +++ b/test/connect/reconnect-backoff.test.ts @@ -0,0 +1,154 @@ +// When a socket closes with a code that is not a logout, Evolution connects +// again. Every attempt builds a socket, fetches the WhatsApp Web version and +// opens a connection through the instance's proxy. If the exit is dead, every +// attempt closes again at once, so the attempts must be spaced out: 1s, 2s, 4s +// and so on, doubling up to one a minute, and back to 1s once a connection has +// opened. They never stop: an instance that gave up would sit disconnected with +// valid credentials until someone noticed. A reconnect still waiting when the +// instance is logged out or deleted must not happen. +import { vi } from 'vitest'; + +const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + return { ...orig, default: socketSpy, makeWASocket: socketSpy }; +}); +// This file is about when a reconnect happens; the version fetch has its own tests. +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); + +import { Boom } from '@hapi/boom'; +import { WAMonitoringService } from '@api/services/monitor.service'; +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; + +import { makeService } from '../helpers/baileys-service'; +import { fakeSocket, stubAuthState } from '../helpers/connect'; + +socketSpy.mockImplementation(fakeSocket); + +const LADDER = [1_000, 2_000, 4_000, 8_000, 16_000, 32_000, 60_000]; +// Longer than any delay the ladder may use: past this, no reconnect is coming. +const NEVER = 5 * 60_000; + +beforeEach(() => { + socketSpy.mockClear(); + // setImmediate stays real: flush() uses it to let the event queue and the connect path run. + vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout', 'setInterval', 'clearInterval', 'Date'] }); +}); + +afterEach(() => { + vi.useRealTimers(); +}); + +async function flush() { + for (let i = 0; i < 20; i++) await new Promise((r) => setImmediate(r)); +} + +const sockets = () => socketSpy.mock.results.length; +const current = () => socketSpy.mock.results[socketSpy.mock.results.length - 1].value; + +async function connected() { + const { service, prisma } = await makeService(); + stubAuthState(service); + await service.connectToWhatsapp(); + expect(sockets()).toBe(1); + return { service, prisma }; +} + +function closeWith(statusCode: number) { + current().ev.emit('connection.update', { + connection: 'close', + lastDisconnect: { error: new Boom('closed', { statusCode }), date: new Date() }, + }); +} + +/** Close the live socket and return how long Evolution waited before the next socket (null: none came). */ +async function reconnectDelay(statusCode: number): Promise { + const before = sockets(); + const start = Date.now(); + closeWith(statusCode); + await flush(); + while (sockets() === before) { + if (Date.now() - start > NEVER) return null; + await vi.advanceTimersByTimeAsync(250); + await flush(); + } + return Date.now() - start; +} + +async function open() { + current().ev.emit('connection.update', { connection: 'open' }); + await flush(); +} + +describe('reconnecting after a close', () => { + it('waits 1s, 2s, 4s... up to a minute between attempts, and never stops', async () => { + await connected(); + const delays: (number | null)[] = []; + for (let i = 0; i < 25; i++) delays.push(await reconnectDelay(500)); + expect(delays).toEqual([...LADDER, ...Array(25 - LADDER.length).fill(60_000)]); + }); + + it('backs off the same way whichever reconnectable code closed it', async () => { + await connected(); + const delays: (number | null)[] = []; + for (const code of [408, 428, 500, 503, 411, 408]) delays.push(await reconnectDelay(code)); + expect(delays).toEqual(LADDER.slice(0, 6)); + }); + + it('starts again from 1s once a connection has opened', async () => { + await connected(); + const before = [await reconnectDelay(408), await reconnectDelay(408), await reconnectDelay(408)]; + await open(); + const after = [await reconnectDelay(408), await reconnectDelay(408)]; + expect({ before, after }).toEqual({ before: [1_000, 2_000, 4_000], after: [1_000, 2_000] }); + }); + + it('tries again when a reconnect attempt itself fails', async () => { + const { service } = await connected(); + // The next reconnect cannot read the session's credentials (a database blip); the one after can. + const read = service.defineAuthState; + let failures = 1; + service.defineAuthState = async () => { + if (failures-- > 0) throw new Error('database unavailable'); + return read(); + }; + expect(await reconnectDelay(500)).toBe(1_000 + 2_000); + }); + + it('does not reconnect after a logout code', async () => { + for (const code of [401, 402, 403, 406]) { + socketSpy.mockClear(); + await connected(); + expect(await reconnectDelay(code)).toBeNull(); + } + }); + + it('drops a waiting reconnect when the instance is logged out', async () => { + const { service } = await connected(); + closeWith(500); + await flush(); + await service.logoutInstance(); + await vi.advanceTimersByTimeAsync(NEVER); + await flush(); + expect(sockets()).toBe(1); + }); + + it('drops a waiting reconnect when the instance is deleted', async () => { + const { service, prisma } = await connected(); + const { ConfigService } = await import('@config/env.config'); + const monitor = new WAMonitoringService(service.eventEmitter, new ConfigService(), prisma, null, null, null, null); + monitor.waInstances.test = service; + closeWith(500); + await flush(); + // What DELETE /instance/delete emits when the instance is not connected (instance.controller.ts deleteInstance). + service.eventEmitter.emit('remove.instance', 'test', 'inner'); + await flush(); + await vi.advanceTimersByTimeAsync(NEVER); + await flush(); + expect(sockets()).toBe(1); + }); +}); diff --git a/test/helpers/connect.ts b/test/helpers/connect.ts index b2429397c9..b1df77ad48 100644 --- a/test/helpers/connect.ts +++ b/test/helpers/connect.ts @@ -12,7 +12,11 @@ import { makeService } from './baileys-service'; export function fakeSocket() { return { ev: makeEventBuffer(P({ level: 'silent' }) as any), - ws: new EventEmitter(), + ws: Object.assign(new EventEmitter(), { close: () => undefined }), + // connectionUpdate reads the account on 'open', and logoutInstance logs the socket out. + user: { id: '972500000000:1@s.whatsapp.net' }, + profilePictureUrl: async () => undefined, + logout: async () => undefined, end: () => undefined, }; } diff --git a/test/proxy/version-fetch.test.ts b/test/proxy/version-fetch.test.ts index 343b08cf33..563c2df3d7 100644 --- a/test/proxy/version-fetch.test.ts +++ b/test/proxy/version-fetch.test.ts @@ -5,14 +5,17 @@ // server's own IP. The proxy here maps those two hosts to local HTTPS servers. import { vi } from 'vitest'; -const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); +const { socketSpy, baileysVersion } = vi.hoisted(() => ({ socketSpy: vi.fn(), baileysVersion: { value: undefined } })); vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); vi.mock('baileys', async (importOriginal) => { const orig = await importOriginal(); + baileysVersion.value = orig.DEFAULT_CONNECTION_CONFIG.version; return { ...orig, default: socketSpy, makeWASocket: socketSpy }; }); +import net from 'node:net'; + import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from 'vitest'; import { connectBehind, fakeSocket, type ProxyProtocol } from '../helpers/connect'; @@ -103,3 +106,57 @@ describe('the WhatsApp Web version fetch on connect', () => { expect(config.version).toEqual([2, 3000, 1011111111]); }); }); + +/** An exit that has died: it accepts the TCP connection and never answers. Logs one line per connection. */ +async function startDeadExit(): Promise { + const log: string[] = []; + const held = new Set(); + const server = net.createServer((s) => { + log.push('connection'); + held.add(s); + s.on('error', () => undefined); + s.on('close', () => held.delete(s)); + }); + await new Promise((r) => server.listen(0, '127.0.0.1', r)); + return { + port: (server.address() as net.AddressInfo).port, + log, + close: () => + new Promise((r) => { + held.forEach((s) => s.destroy()); + server.close(() => r()); + }), + }; +} + +// Kept last in the file: before the fix the connect it starts never finishes. +describe('when the exit never answers', () => { + it('gives up on the version fetch after 10s and connects with the version Baileys ships', async () => { + const realSetTimeout = globalThis.setTimeout; + const dead = await startDeadExit(); + proxies.push(dead); + vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout'] }); + try { + const connecting = connectBehind(socketSpy, { protocol: 'http', port: dead.port }); + let settled = false; + connecting.then( + () => (settled = true), + () => (settled = true), + ); + // Not vi.waitFor: under fake timers it advances the clock while it polls. + for (let i = 0; i < 200 && dead.log.length === 0; i++) await new Promise((r) => realSetTimeout(r, 10)); + expect(dead.log).toEqual(['connection']); + await vi.advanceTimersByTimeAsync(9_999); + const early = settled ? 'connected' : 'waiting'; + await vi.advanceTimersByTimeAsync(1); + const outcome = await Promise.race([ + connecting.then(({ config }) => config.version), + new Promise((r) => realSetTimeout(() => r('still waiting on the version fetch'), 2_000)), + ]); + expect({ early, outcome }).toEqual({ early: 'waiting', outcome: baileysVersion.value }); + expect(guard.refused).toEqual([]); + } finally { + vi.useRealTimers(); + } + }); +}); From ae8ad212a94de9e5ca9ed236b142acb52652276c Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:36:36 +0300 Subject: [PATCH 043/157] fix(baileys): back off between reconnects, and time out the version fetch A reconnectable close now schedules the next connect instead of making it at once (through connect(), so the QR budget of the attempt carries over, as before): 1s, then doubling to one attempt a minute, with no limit, back to 1s when a connection opens. The logout codes (401, 402, 403, 406) are unchanged. An attempt that throws before building a socket schedules the next one, where before it left the instance disconnected for good. A waiting reconnect is dropped by logoutInstance and when the monitor removes the instance. The WhatsApp Web version fetch gives up after 10s (the sw.js request is aborted and the fallback is not started) and the connect goes on with the version Baileys ships. Ported from the idea of upstream PR #2732 (backoff with reset on open), not its code: no attempt cap, since exhausting it strands a session with valid credentials, and none of develop's isDeleting or 515 window. Upstream 933a28de's fixed 3s delay bounds the rate but never backs off. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 39 ++++++++++++++++- src/api/services/monitor.service.ts | 2 + src/utils/fetchLatestWaWebVersion.ts | 42 ++++++++++++++++++- 3 files changed, 80 insertions(+), 3 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 2a8d0e94cd..5a8c1facb7 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -251,6 +251,12 @@ export class BaileysStartupService extends ChannelStartupService { private readonly msgRetryCounterCache: CacheStore = new NodeCache(); private readonly userDevicesCache: CacheStore = new NodeCache({ stdTTL: 300000, useClones: false }); private endSession = false; + // A reconnectable close is retried after 1s, 2s, 4s... doubling to one a minute, for as long as + // it takes: an instance that gave up would sit disconnected on valid credentials. An open resets it. + private static readonly RECONNECT_FIRST_DELAY_MS = 1_000; + private static readonly RECONNECT_MAX_DELAY_MS = 60_000; + private reconnectAttempts = 0; + private reconnectTimer: NodeJS.Timeout | null = null; private logBaileys = this.configService.get('LOG').BAILEYS; private eventProcessingQueue: Promise = Promise.resolve(); // The dispatcher media downloads go through: the instance's proxy, the same exit as the socket. @@ -285,6 +291,7 @@ export class BaileysStartupService extends ChannelStartupService { } public async logoutInstance() { + this.stopReconnecting(); this.messageProcessor.onDestroy(); this.pictureCache.clear(); @@ -479,8 +486,9 @@ export class BaileysStartupService extends ChannelStartupService { const shouldReconnect = !codesToNotReconnect.includes(statusCode); if (shouldReconnect) { // Baileys' own reconnect (QR refs ended, a dropped socket) is the same attempt: the QR budget carries over. - await this.connect(this.phoneNumber); + this.scheduleReconnect(statusCode); } else { + this.stopReconnecting(); this.sendDataWebhook(Events.STATUS_INSTANCE, { instance: this.instance.name, status: 'closed', @@ -516,6 +524,7 @@ export class BaileysStartupService extends ChannelStartupService { } if (connection === 'open') { + this.reconnectAttempts = 0; this.instance.wuid = this.client.user.id.replace(/:\d+/, ''); try { const profilePic = await this.profilePicture(this.instance.wuid); @@ -808,6 +817,34 @@ export class BaileysStartupService extends ChannelStartupService { } } + private scheduleReconnect(statusCode?: number) { + this.stopReconnecting(); + const delay = Math.min( + BaileysStartupService.RECONNECT_FIRST_DELAY_MS * 2 ** Math.min(this.reconnectAttempts, 16), + BaileysStartupService.RECONNECT_MAX_DELAY_MS, + ); + this.reconnectAttempts++; + this.logger.info(`Reconnecting in ${delay / 1000}s (attempt ${this.reconnectAttempts}, status code ${statusCode})`); + this.reconnectTimer = setTimeout(async () => { + this.reconnectTimer = null; + try { + await this.connect(this.phoneNumber); + } catch (error) { + // No socket was built, so no close will come to retry it: schedule the next attempt here. + this.logger.error({ message: 'Reconnect attempt failed', error: error?.toString() }); + this.scheduleReconnect(statusCode); + } + }, delay); + } + + /** Drop a reconnect that is still waiting: the instance is being logged out or deleted. */ + public stopReconnecting() { + if (this.reconnectTimer) { + clearTimeout(this.reconnectTimer); + this.reconnectTimer = null; + } + } + public async reloadConnection(): Promise { try { return await this.createClient(this.phoneNumber); diff --git a/src/api/services/monitor.service.ts b/src/api/services/monitor.service.ts index 438530b57e..8cec2affd2 100644 --- a/src/api/services/monitor.service.ts +++ b/src/api/services/monitor.service.ts @@ -392,6 +392,8 @@ export class WAMonitoringService { private removeInstance() { this.eventEmitter.on('remove.instance', async (instanceName: string) => { try { + // A reconnect waiting on its backoff would otherwise bring the removed instance back. + this.waInstances[instanceName]?.stopReconnecting?.(); await this.waInstances[instanceName]?.sendDataWebhook(Events.REMOVE_INSTANCE, null); this.clearDelInstanceTime(instanceName); diff --git a/src/utils/fetchLatestWaWebVersion.ts b/src/utils/fetchLatestWaWebVersion.ts index 6ba2892425..a7f800a7ed 100644 --- a/src/utils/fetchLatestWaWebVersion.ts +++ b/src/utils/fetchLatestWaWebVersion.ts @@ -1,14 +1,50 @@ import axios, { AxiosRequestConfig } from 'axios'; -import { fetchLatestBaileysVersion, WAVersion } from 'baileys'; +import { DEFAULT_CONNECTION_CONFIG, fetchLatestBaileysVersion, WAVersion } from 'baileys'; + +// Every connect waits on this. An exit that accepts the connection and never answers would +// otherwise hold the connect (and every reconnect) forever: axios' default timeout is none. +export const WA_VERSION_FETCH_TIMEOUT_MS = 10_000; + +type VersionResult = { version: WAVersion; isLatest: boolean; error?: unknown }; /** * `options` go to the sw.js request (axios); `fallbackOptions` to Baileys' * fallback fetch, which takes only an undici `dispatcher` for a proxy. + * Past `timeoutMs` the answer is the version this Baileys ships with. */ -export const fetchLatestWaWebVersion = async (options: AxiosRequestConfig<{}>, fallbackOptions: RequestInit = {}) => { +export const fetchLatestWaWebVersion = async ( + options: AxiosRequestConfig<{}>, + fallbackOptions: RequestInit = {}, + timeoutMs = WA_VERSION_FETCH_TIMEOUT_MS, +): Promise => { + const abort = new AbortController(); + let timer: NodeJS.Timeout; + const deadline = new Promise((resolve) => { + timer = setTimeout(() => { + abort.abort(); + resolve({ + version: DEFAULT_CONNECTION_CONFIG.version, + isLatest: false, + error: { message: `WhatsApp Web version fetch timed out after ${timeoutMs} ms` }, + }); + }, timeoutMs); + }); + try { + return await Promise.race([fetchVersion(options, fallbackOptions, abort.signal), deadline]); + } finally { + clearTimeout(timer); + } +}; + +const fetchVersion = async ( + options: AxiosRequestConfig<{}>, + fallbackOptions: RequestInit, + signal: AbortSignal, +): Promise => { try { const { data } = await axios.get('https://web.whatsapp.com/sw.js', { ...options, + signal, responseType: 'json', }); @@ -32,6 +68,8 @@ export const fetchLatestWaWebVersion = async (options: AxiosRequestConfig<{}>, f isLatest: true, }; } catch (error) { + // Timed out: the deadline has already answered, so do not start the fallback. + if (signal.aborted) return { version: DEFAULT_CONNECTION_CONFIG.version, isLatest: false, error }; return { version: (await fetchLatestBaileysVersion(fallbackOptions)).version as WAVersion, isLatest: false, From f9e3cfb6db56f2cccb3aa644423a43b054d1d21d Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:38:23 +0300 Subject: [PATCH 044/157] test: one instance never runs two sockets createClient builds a new socket without ending the previous one or detaching Evolution's handlers from it, and nothing stops two connects from running at once. So /instance/connect polled while Evolution reconnects, or a connect while the backoff waits, leaves two sockets on one session, and the replaced one still drives the instance: its close reconnects and ends the live one. These tests require overlapping connects to build one socket, a connect during a waiting reconnect to replace the old socket once with no reconnect following, the replaced socket to be detached and then ended, and its late events to be ignored. Two guards pass today and hold the fix to its edges: a 440 on the live socket still reconnects, and a connect that fails before building a socket leaves a waiting reconnect in place. The fake socket now ends the way Baileys' end() does: once, announcing the close on its own events. Co-Authored-By: Claude Opus 5.5 --- test/connect/one-socket.test.ts | 168 ++++++++++++++++++++++++++++++++ test/helpers/connect.ts | 34 ++++++- 2 files changed, 199 insertions(+), 3 deletions(-) create mode 100644 test/connect/one-socket.test.ts diff --git a/test/connect/one-socket.test.ts b/test/connect/one-socket.test.ts new file mode 100644 index 0000000000..8451b1a6cc --- /dev/null +++ b/test/connect/one-socket.test.ts @@ -0,0 +1,168 @@ +// An instance owns one WhatsApp socket at a time. Building a new one (a +// reconnect, /instance/connect, a restart) must first let go of the old one: +// stop listening to it, then end it. Otherwise the old socket keeps talking: +// its close starts a reconnect that ends the new one, its QR codes replace the +// new one's, and two logged-in sockets on one session get 440 (replaced) from +// WhatsApp in turn, forever. Two connects that overlap, which is what +// /instance/connect polled every 2s does while Evolution is reconnecting, +// must build one socket between them. +import { vi } from 'vitest'; + +const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + return { ...orig, default: socketSpy, makeWASocket: socketSpy }; +}); +// This file is about which socket is live; the version fetch has its own tests. +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); + +import { Boom } from '@hapi/boom'; +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; + +import { makeService } from '../helpers/baileys-service'; +import { fakeSocket, stubAuthState } from '../helpers/connect'; + +socketSpy.mockImplementation(fakeSocket); + +// Longer than any reconnect backoff: past this, no reconnect is coming. +const NEVER = 5 * 60_000; + +beforeEach(() => { + socketSpy.mockClear(); + vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout', 'setInterval', 'clearInterval', 'Date'] }); +}); + +afterEach(() => { + vi.useRealTimers(); +}); + +async function flush() { + for (let i = 0; i < 20; i++) await new Promise((r) => setImmediate(r)); +} + +type Sock = ReturnType; +const built = (): Sock[] => socketSpy.mock.results.map((r) => r.value); +const live = () => built().filter((s) => !s.ended); +/** Nothing of Evolution's is still listening to this socket. */ +const detached = (s: Sock) => + s.handlers() + s.ws.listenerCount('CB:call') + s.ws.listenerCount('CB:ack,class:call') === 0; + +async function service() { + const { service } = await makeService(); + stubAuthState(service); + return service; +} + +/** WhatsApp closes the socket: Baileys ends it with the reason, which it announces as a close. */ +function dropped(sock: Sock, statusCode: number) { + sock.end(new Boom('closed', { statusCode })); +} + +async function opened(sock: Sock) { + sock.ev.emit('connection.update', { connection: 'open' }); + await flush(); +} + +async function waitOutAnyReconnect() { + await vi.advanceTimersByTimeAsync(NEVER); + await flush(); +} + +describe('one instance, one socket', () => { + it('two overlapping connects build one socket', async () => { + const s = await service(); + const [a, b] = await Promise.all([s.connectToWhatsapp(), s.connectToWhatsapp()]); + await waitOutAnyReconnect(); + expect({ built: built().length, same: a === b, live: live().length }).toEqual({ built: 1, same: true, live: 1 }); + }); + + it('a connect while a reconnect is waiting replaces the old socket once, and the reconnect does not follow', async () => { + const s = await service(); + await s.connectToWhatsapp(); + const [first] = built(); + dropped(first, 408); + await flush(); + // /instance/connect sees state "close" during the backoff and connects. + await s.connectToWhatsapp(); + await waitOutAnyReconnect(); + const [, second] = built(); + expect({ + built: built().length, + live: live().length, + current: s.client === second, + firstEnded: first.ended, + firstDetached: detached(first), + }).toEqual({ built: 2, live: 1, current: true, firstEnded: true, firstDetached: true }); + }); + + it('a connect ends the socket it replaces, after it has stopped listening to it', async () => { + const s = await service(); + await s.connectToWhatsapp(); + const [first] = built(); + await opened(first); + // /instance/restart on Baileys, or any second connect while the first socket is still up. + await s.connectToWhatsapp(); + await waitOutAnyReconnect(); + // Ending the first socket announces its close; had Evolution still been listening, it would reconnect. + expect({ + built: built().length, + live: live().length, + firstEnded: first.ended, + firstDetached: detached(first), + }).toEqual({ built: 2, live: 1, firstEnded: true, firstDetached: true }); + }); + + it('what a replaced socket still emits does not drive the instance', async () => { + const s = await service(); + await s.connectToWhatsapp(); + const [first] = built(); + await s.connectToWhatsapp(); + const [, second] = built(); + await opened(second); + // The replaced socket's buffered close arrives late (a 440 from WhatsApp, say). + first.ev.emit('connection.update', { + connection: 'close', + lastDisconnect: { error: new Boom('replaced', { statusCode: 440 }), date: new Date() }, + }); + await flush(); + await waitOutAnyReconnect(); + expect({ built: built().length, state: s.connectionStatus.state, current: s.client === second }).toEqual({ + built: 2, + state: 'open', + current: true, + }); + }); + + // Guards the fix rather than the bug: letting a connect replace a waiting reconnect must not + // lose that reconnect when the connect fails before it has built a socket. + it('a connect that fails leaves the waiting reconnect in place', async () => { + const s = await service(); + await s.connectToWhatsapp(); + const [first] = built(); + dropped(first, 408); + await flush(); + const read = s.defineAuthState; + s.defineAuthState = async () => { + s.defineAuthState = read; + throw new Error('database unavailable'); + }; + await expect(s.connectToWhatsapp()).rejects.toBeDefined(); + await waitOutAnyReconnect(); + expect({ built: built().length, live: live().length }).toEqual({ built: 2, live: 1 }); + }); + + it('a 440 on the live socket is still reconnected', async () => { + const s = await service(); + await s.connectToWhatsapp(); + const [first] = built(); + await opened(first); + dropped(first, 440); + await flush(); + await waitOutAnyReconnect(); + expect({ built: built().length, live: live().length }).toEqual({ built: 2, live: 1 }); + }); +}); diff --git a/test/helpers/connect.ts b/test/helpers/connect.ts index b1df77ad48..630e6b7e40 100644 --- a/test/helpers/connect.ts +++ b/test/helpers/connect.ts @@ -10,15 +10,43 @@ import P from 'pino'; import { makeService } from './baileys-service'; export function fakeSocket() { - return { - ev: makeEventBuffer(P({ level: 'silent' }) as any), + const ev = makeEventBuffer(P({ level: 'silent' }) as any); + // ev.process subscriptions still attached: a socket Evolution has let go of should have none. + let handlers = 0; + const process = ev.process.bind(ev); + ev.process = (handler: any) => { + const off = process(handler); + handlers++; + let attached = true; + return () => { + if (attached) handlers--; + attached = false; + off(); + }; + }; + let closed = false; + ev.on('connection.update', (update: any) => { + if (update.connection === 'close') closed = true; + }); + const sock = { + ev, ws: Object.assign(new EventEmitter(), { close: () => undefined }), // connectionUpdate reads the account on 'open', and logoutInstance logs the socket out. user: { id: '972500000000:1@s.whatsapp.net' }, profilePictureUrl: async () => undefined, logout: async () => undefined, - end: () => undefined, + /** Whether Evolution called end() on this socket. */ + ended: false, + handlers: () => handlers, + // As Baileys' end() (Socket/socket.js): once only, and it announces the close on the socket's own events. + end: (error?: Error) => { + sock.ended = true; + if (closed) return; + closed = true; + ev.emit('connection.update', { connection: 'close', lastDisconnect: { error, date: new Date() } }); + }, }; + return sock; } /** The auth state is files on disk under the instances directory; nothing about it touches the network. */ From 5434ac4679889109fa6c0e14d3ebc16ac0bc2d17 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:40:23 +0300 Subject: [PATCH 045/157] fix(baileys): end the old socket and serialise connects connect(), which both connectToWhatsapp (a new attempt, fresh QR budget) and the backoff reconnect go through, now runs one connect at a time: a connect that arrives while one is under way with the same number joins it and gets the same socket; one for a different number waits for it and then runs. createClient retires the previous socket before building the next: it first detaches Evolution from it (the ev.process subscription and the CB:call listeners), then ends it. The order matters: ending a live Baileys socket emits a close, and if Evolution were still listening that close would reconnect and end the new socket, the recreate loop upstream PR #2656 has. A new socket also drops any reconnect still waiting on its backoff, since that reconnect was for the socket just replaced; a connect that fails before building one leaves it in place. Events are tied to the socket that emitted them: a batch from a socket that is no longer this.client is dropped, and connectionUpdate ignores a connection.update from a replaced socket even if it was queued before the switch. 440 stays reconnectable (PR #2655 makes it final, which strands a valid session when the 440 came from our own duplicate socket). Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 87 ++++++++++++++++--- 1 file changed, 75 insertions(+), 12 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 5a8c1facb7..7c14b15bad 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -257,6 +257,10 @@ export class BaileysStartupService extends ChannelStartupService { private static readonly RECONNECT_MAX_DELAY_MS = 60_000; private reconnectAttempts = 0; private reconnectTimer: NodeJS.Timeout | null = null; + // The connect under way, so a second one joins it instead of building a second socket. + private connecting: { number: string | null; socket: Promise } | null = null; + // Stops Evolution listening to the current socket; called before that socket is ended. + private detachClient?: () => void; private logBaileys = this.configService.get('LOG').BAILEYS; private eventProcessingQueue: Promise = Promise.resolve(); // The dispatcher media downloads go through: the instance's proxy, the same exit as the socket. @@ -380,7 +384,10 @@ export class BaileysStartupService extends ChannelStartupService { }; } - private async connectionUpdate({ qr, connection, lastDisconnect }: Partial) { + private async connectionUpdate({ qr, connection, lastDisconnect }: Partial, from?: WASocket) { + // A replaced socket speaks for nobody: its close must not reconnect, its QR must not show. + if (from && from !== this.client) return; + if (qr) { if (this.instance.qrcode.count === this.configService.get('QRCODE').LIMIT) { this.sendDataWebhook(Events.QRCODE_UPDATED, { @@ -765,38 +772,89 @@ export class BaileysStartupService extends ChannelStartupService { this.endSession = false; + this.retireClient(); this.client = makeWASocket(socketConfig); + // Any reconnect still waiting was for the socket just replaced. + this.stopReconnecting(); if (this.localSettings.wavoipToken && this.localSettings.wavoipToken.length > 0) { useVoiceCallsBaileys(this.localSettings.wavoipToken, this.client, this.connectionStatus.state as any, true); } - this.eventHandler(); + const client = this.client; + const stopProcessing = this.eventHandler(); - this.client.ws.on('CB:call', (packet) => { + const onCall = (packet) => { this.logger.verbose(`CB:call id=${packet?.attrs?.id ?? ''}`); const payload = { event: 'CB:call', packet: packet }; this.sendDataWebhook(Events.CALL, payload, true, ['websocket']); - }); - - this.client.ws.on('CB:ack,class:call', (packet) => { + }; + const onCallAck = (packet) => { this.logger.verbose(`CB:ack,class:call id=${packet?.attrs?.id ?? ''}`); const payload = { event: 'CB:ack,class:call', packet: packet }; this.sendDataWebhook(Events.CALL, payload, true, ['websocket']); - }); + }; + client.ws.on('CB:call', onCall); + client.ws.on('CB:ack,class:call', onCallAck); + + this.detachClient = () => { + stopProcessing(); + client.ws.off('CB:call', onCall); + client.ws.off('CB:ack,class:call', onCallAck); + }; this.phoneNumber = number; return this.client; } + /** + * Let go of the current socket before another is built: stop listening to it first, because + * ending a live Baileys socket announces a close, which would otherwise reconnect and end the + * new one in turn. + */ + private retireClient() { + const previous = this.client; + this.detachClient?.(); + this.detachClient = undefined; + if (!previous) return; + try { + previous.end(new Error('Replaced by a new connection')); + } catch (error) { + this.logger.warn({ message: 'Could not end the replaced socket', error: error?.toString() }); + } + } + /** A new connect attempt: a fresh QR budget, and no QR or pairing code left from an earlier attempt. */ public async connectToWhatsapp(number?: string): Promise { this.instance.qrcode = { count: 0 }; return await this.connect(number); } + /** + * One connect at a time, for connectToWhatsapp and the reconnect alike. A connect that arrives + * while one is under way (/instance/connect polled during a reconnect) joins it; one for a + * different number runs after it. The socket it builds takes the place of a reconnect still + * waiting on its backoff (createClient drops that); if it fails before building one, the waiting + * reconnect still happens. + */ private async connect(number?: string): Promise { + const inFlight = this.connecting; + if (inFlight && inFlight.number === (number ?? null)) return inFlight.socket; + + const socket = (inFlight ? inFlight.socket.catch(() => undefined) : Promise.resolve()).then(() => + this.openConnection(number), + ); + const entry = { number: number ?? null, socket }; + this.connecting = entry; + try { + return await socket; + } finally { + if (this.connecting === entry) this.connecting = null; + } + } + + private async openConnection(number?: string): Promise { try { this.loadChatwoot(); // The socket takes syncFullHistory, groupsIgnore, readStatus and alwaysOnline as config: read them first. @@ -1031,7 +1089,8 @@ export class BaileysStartupService extends ChannelStartupService { * looked up, so no network query can follow. */ private tapHistoryLidMappings() { - const ev = this.client.ev as any; + const client = this.client; + const ev = client.ev as any; if (ev.__lidPnMappingsTap) return; const emit = ev.emit.bind(ev); ev.emit = (event: string, data: any) => { @@ -1039,7 +1098,7 @@ export class BaileysStartupService extends ChannelStartupService { const mappings = [...data.lidPnMappings]; this.eventProcessingQueue = this.eventProcessingQueue.then(() => { try { - if (!this.endSession) this.lidMappingHandle(mappings); + if (!this.endSession && client === this.client) this.lidMappingHandle(mappings); } catch (error) { this.logger.error(error); } @@ -2009,10 +2068,14 @@ export class BaileysStartupService extends ChannelStartupService { }, }; - private eventHandler() { + /** Returns the function that stops processing this socket's events. */ + private eventHandler(): () => void { + const client = this.client; this.tapHistoryLidMappings(); - this.client.ev.process(async (events) => { + return client.ev.process(async (events) => { + // Events a replaced socket still emits do not drive the instance. + if (client !== this.client) return; this.eventProcessingQueue = this.eventProcessingQueue.then(async () => { try { if (!this.endSession) { @@ -2044,7 +2107,7 @@ export class BaileysStartupService extends ChannelStartupService { } if (events['connection.update']) { - this.connectionUpdate(events['connection.update']); + this.connectionUpdate(events['connection.update'], client); } if (events['creds.update']) { From 420b9bc26dd5d21f953dfe3b7cd857dffa495850 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 14:55:56 +0300 Subject: [PATCH 046/157] test: a logout that cannot reach WhatsApp is delivered when the connection returns Baileys' logout sends remove-companion-device, which is what takes the device off the person's Linked devices, and it can only go out on an open socket with the session's credentials. The fork (790d21e4) wiped the credentials locally when the socket was down, so the device could never be removed. Today a logout while a reconnect waits is refused as "not connected" (400), and one while the socket dials wipes at once. These tests, through the real /instance router, InstanceController, WAMonitoringService, ChannelController and the Prisma auth store, with the fake socket whose logout now behaves as Baileys' does (throws Connection Closed when the ws is not open, then ends with loggedOut), require: - logout with the socket down: 202 PENDING, creds and key files kept, a logout-pending marker in the instance's directory, connectionState saying logoutPending; - while pending, a live message and a history batch produce no webhook and no stored row; - on the reconnect's open, the logout is sent exactly once, then creds, key files and marker are wiped, the row says close, and nothing reconnects; - a restart while pending resumes it (the marker survives); - loggedOut (401) on the reconnect finishes it without a send; - delete while pending: 202, the instance leaves the API (no in-memory instance, no row, no settings) keeping only creds, keys, marker and proxy; connectionState answers logoutPending until done, then 404; the name cannot be created again meanwhile (403); a purge (logout, then delete) resumes after a restart too; - a logout on an open socket still answers 200 SUCCESS and wipes (red today in this harness: the loggedOut close's cleanup removes the instance directory while removeSession recreates it, ENOENT, 500). test/instance/unlink-socket-down.test.ts (from 790d21e4) asserted an immediate wipe for logout and delete on a dialling socket; its first two cases now require the pending answer with the session kept. Its third case (delete still removes the instance when the logout fails for another reason, upstream c29bcc50) is unchanged and holds today. Co-Authored-By: Claude Opus 5.5 --- test/helpers/connect.ts | 17 +- test/instance/logout-pending.test.ts | 425 +++++++++++++++++++++++ test/instance/unlink-socket-down.test.ts | 52 ++- 3 files changed, 478 insertions(+), 16 deletions(-) create mode 100644 test/instance/logout-pending.test.ts diff --git a/test/helpers/connect.ts b/test/helpers/connect.ts index 630e6b7e40..e11fa40ad9 100644 --- a/test/helpers/connect.ts +++ b/test/helpers/connect.ts @@ -4,12 +4,13 @@ // makeWASocket is `socketSpy`, which returns fakeSocket(). import { EventEmitter } from 'node:events'; +import { Boom } from '@hapi/boom'; import { initAuthCreds, makeEventBuffer } from 'baileys'; import P from 'pino'; import { makeService } from './baileys-service'; -export function fakeSocket() { +export function fakeSocket(config?: any) { const ev = makeEventBuffer(P({ level: 'silent' }) as any); // ev.process subscriptions still attached: a socket Evolution has let go of should have none. let handlers = 0; @@ -25,8 +26,10 @@ export function fakeSocket() { }; }; let closed = false; + let open = false; ev.on('connection.update', (update: any) => { if (update.connection === 'close') closed = true; + if (update.connection === 'open') open = true; }); const sock = { ev, @@ -34,7 +37,17 @@ export function fakeSocket() { // connectionUpdate reads the account on 'open', and logoutInstance logs the socket out. user: { id: '972500000000:1@s.whatsapp.net' }, profilePictureUrl: async () => undefined, - logout: async () => undefined, + /** How many times the socket told WhatsApp to remove this device (remove-companion-device). */ + logouts: 0, + // As Baileys' logout(): with a linked device it first tells WhatsApp, which throws when the ws is + // not open (sendNode: 'Connection Closed'); then it ends the socket with loggedOut. + logout: async (msg?: string) => { + if (config?.auth?.creds?.me?.id) { + if (!open || closed) throw new Boom('Connection Closed', { statusCode: 428 }); + sock.logouts++; + } + sock.end(new Boom(msg || 'Intentional Logout', { statusCode: 401 })); + }, /** Whether Evolution called end() on this socket. */ ended: false, handlers: () => handlers, diff --git a/test/instance/logout-pending.test.ts b/test/instance/logout-pending.test.ts new file mode 100644 index 0000000000..4550a85162 --- /dev/null +++ b/test/instance/logout-pending.test.ts @@ -0,0 +1,425 @@ +// A logout must reach WhatsApp. Baileys' logout sends remove-companion-device, +// which is what takes the device off the person's Linked devices; it can only +// be sent on an open socket, with the session's credentials. When the socket is +// down or a reconnect is waiting, wiping the credentials locally (as the fork +// did) means the device can never be removed and stays on the phone. +// +// So a logout that cannot reach WhatsApp is kept pending: the credentials stay, +// a marker in the instance's directory says so (it survives a restart), the +// instance forwards and stores nothing, and it reconnects only to deliver the +// logout. When the connection opens it sends the logout at once; the session +// is wiped once the socket ends with loggedOut. If WhatsApp answers loggedOut +// on the reconnect (the device was already removed), that also finishes it. +// +// Everything runs through the real /instance router and guards, the real +// InstanceController, WAMonitoringService and ChannelController, and +// Evolution's own Prisma auth store (creds in the session table, keys in files +// under INSTANCE_DIR). The socket is the fake one (test/helpers/connect.ts), +// whose logout behaves as Baileys' does. +import { vi } from 'vitest'; + +const { socketSpy, h } = vi.hoisted(() => ({ + socketSpy: vi.fn(), + h: { waMonitor: undefined as any, instanceController: undefined as any, channelController: undefined as any }, +})); +const tmp = await vi.hoisted(async () => { + const { mkdtempSync } = await import('node:fs'); + const { tmpdir } = await import('node:os'); + const { join } = await import('node:path'); + return mkdtempSync(join(tmpdir(), 'evo-logout-')); +}); + +vi.mock('@api/server.module', async () => { + const fake = await import('../helpers/fake-server-module'); + return { + ...fake, + get waMonitor() { + return h.waMonitor; + }, + get instanceController() { + return h.instanceController; + }, + get channelController() { + return h.channelController; + }, + }; +}); +vi.mock('@config/path.config', async (importOriginal) => ({ ...(await importOriginal()), INSTANCE_DIR: tmp })); +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + return { ...orig, default: socketSpy, makeWASocket: socketSpy }; +}); + +import { existsSync, readdirSync, readFileSync, rmSync } from 'node:fs'; +import { join } from 'node:path'; + +import { Boom } from '@hapi/boom'; +import { proto } from 'baileys'; +import EventEmitter2 from 'eventemitter2'; +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from 'vitest'; + +import { settle, WUID } from '../helpers/baileys-service'; +import { fakeSocket } from '../helpers/connect'; +import { emitted, prismaRepository as prisma } from '../helpers/fake-server-module'; +import { startInstanceApp } from '../helpers/http-app'; +import { loopbackOnly } from '../helpers/local-net'; + +socketSpy.mockImplementation(fakeSocket); + +const TOKEN = 'instance-token'; +const DIR = join(tmp, 'inst-1'); +const MARKER = join(DIR, 'logout-pending.json'); +const PENDING = { + status: 202, + body: { + status: 'PENDING', + error: false, + response: { message: 'Logout pending: WhatsApp will be told when the connection returns' }, + }, +}; + +type Sock = ReturnType; +const built = (): Sock[] => socketSpy.mock.results.map((r) => r.value); +const current = () => built()[built().length - 1]; + +let guard: ReturnType; +let app: Awaited>; +let globalKey: string; +const processes: any[] = []; + +beforeAll(async () => { + guard = loopbackOnly(); + app = await startInstanceApp(); + const { configService } = await import('@config/env.config'); + globalKey = configService.get('AUTHENTICATION').API_KEY.KEY; +}); +beforeEach(() => { + socketSpy.mockClear(); + emitted.length = 0; +}); +afterEach(async () => { + // Stop every process's instances without them answering with a reconnect. + for (const monitor of processes) { + for (const s of [ + ...Object.values(monitor.waInstances), + ...Object.values(monitor.finishingLogouts ?? {}), + ] as any[]) { + s.stopReconnecting?.(); + s.connectToWhatsapp = async () => undefined; + s.connect = async () => undefined; + } + } + processes.length = 0; + for (const t of Object.values(prisma) as any[]) if (Array.isArray(t?.rows)) t.rows.length = 0; + rmSync(DIR, { recursive: true, force: true }); +}); +afterAll(async () => { + await app.close(); + rmSync(tmp, { recursive: true, force: true }); + expect(guard.refused).toEqual([]); + guard.restore(); +}); + +/** What a process start builds (main.ts, server.module): the monitor, the controllers, then the boot load. */ +async function startProcess() { + const { ConfigService } = await import('@config/env.config'); + const { CacheService } = await import('@api/services/cache.service'); + const { LocalCache } = await import('@cache/localcache'); + const { WAMonitoringService } = await import('@api/services/monitor.service'); + const { InstanceController } = await import('@api/controllers/instance.controller'); + const { ChannelController } = await import('@api/integrations/channel/channel.controller'); + const configService = new ConfigService(); + const cache = new CacheService(new LocalCache(configService, 'instance')); + const emitter = new EventEmitter2(); + const waMonitor = new WAMonitoringService(emitter, configService, prisma, null as any, cache, cache, cache); + const n = null as any; + h.waMonitor = waMonitor; + h.channelController = new ChannelController(prisma, waMonitor); + h.instanceController = new InstanceController( + waMonitor, + configService, + prisma, + emitter, + n, + n, + n, + cache, + cache, + cache, + n, + ); + processes.push(waMonitor); + await waMonitor.loadInstance(); + return waMonitor; +} + +/** A linked instance, stored the way Evolution stores one: its row, its creds, a signal key file, a proxy. */ +async function linkedInstance() { + const { configService } = await import('@config/env.config'); + await prisma.instance.create({ + data: { + id: 'inst-1', + name: 'test', + connectionStatus: 'open', + token: TOKEN, + integration: 'WHATSAPP-BAILEYS', + clientName: configService.get('DATABASE').CONNECTION.CLIENT_NAME, + }, + }); + await prisma.proxy.create({ + data: { + instanceId: 'inst-1', + enabled: false, + host: '127.0.0.1', + port: '1', + protocol: 'http', + username: '', + password: '', + }, + }); + await prisma.setting.create({ data: { instanceId: 'inst-1', rejectCall: false, msgCall: '', readMessages: false } }); + const { default: useMultiFileAuthStatePrisma } = await import('@utils/use-multi-file-auth-state-prisma'); + const auth = await useMultiFileAuthStatePrisma('inst-1', null as any); + auth.state.creds.me = { id: WUID, name: 'Dana' }; + await auth.saveCreds(); + await auth.state.keys.set({ 'pre-key': { '1': { public: Buffer.alloc(32, 1), private: Buffer.alloc(32, 2) } } }); +} + +const storedMe = () => prisma.session.rows.map((r: any) => JSON.parse(JSON.parse(r.creds)).me?.id); + +/** Booted, connected, then dropped (a dead exit): the socket closed and the first reconnect (1s) is waiting. */ +async function waitingToReconnect() { + await linkedInstance(); + const monitor = await startProcess(); + await vi.waitFor(() => expect(built()).toHaveLength(1)); + const service = monitor.waInstances.test; + await opened(current()); + current().end(new Boom('Connection Terminated', { statusCode: 428 })); + await vi.waitFor(() => expect(service.connectionStatus.state).toBe('close'), { interval: 5 }); + emitted.length = 0; + return { monitor, service }; +} + +async function opened(sock: Sock) { + sock.ev.emit('connection.update', { connection: 'open' }); + await new Promise((r) => setTimeout(r, 20)); +} + +async function call(method: 'GET' | 'DELETE', route: string, key = TOKEN) { + const res = await fetch(`${app.base}/instance/${route}/test`, { method, headers: { apikey: key } }); + return { status: res.status, body: await res.json() }; +} + +/** Wait for the reconnect the pending logout makes, and return its socket. */ +async function reconnected(count: number) { + await vi.waitFor(() => expect(built()).toHaveLength(count), { timeout: 3_000 }); + return current(); +} + +/** A live message and a history batch arriving on a pending instance, the way Baileys emits them (buffered). */ +async function deliverWhilePending(sock: Sock, service: any) { + const message = { + key: { remoteJid: '972500000001@s.whatsapp.net', fromMe: false, id: 'M1' }, + messageTimestamp: 1_790_000_000, + pushName: 'Noa', + message: { conversation: 'hello' }, + }; + sock.ev.buffer(); + sock.ev.emit('messages.upsert', { messages: [message], type: 'notify' }); + sock.ev.emit('messaging-history.set', { + chats: [{ id: '972500000002@s.whatsapp.net', name: 'Tal' }], + contacts: [{ id: '972500000002@s.whatsapp.net', name: 'Tal' }], + messages: [{ ...message, key: { ...message.key, id: 'H1' } }], + lidPnMappings: [{ lid: '111@lid', pn: '972500000002@s.whatsapp.net' }], + syncType: proto.HistorySync.HistorySyncType.RECENT, + isLatest: true, + }); + await sock.ev.flush(); + await settle(service); +} + +const stored = () => ({ + messages: prisma.message.rows.length, + chats: prisma.chat.rows.length, + contacts: prisma.contact.rows.length, +}); + +describe('a logout that cannot reach WhatsApp', () => { + it('is kept pending, forwards nothing, and is delivered once when the connection returns', async () => { + const { service } = await waitingToReconnect(); + + expect(await call('DELETE', 'logout')).toEqual(PENDING); + // The credentials the logout needs are kept, and the marker says it is pending. + expect(storedMe()).toEqual([WUID]); + expect(readdirSync(DIR).sort()).toEqual(['logout-pending.json', 'pre-key-1.json']); + expect(JSON.parse(readFileSync(MARKER, 'utf8'))).toMatchObject({ instanceName: 'test', deleted: false }); + expect(await call('GET', 'connectionState')).toEqual({ + status: 200, + body: { instance: { instanceName: 'test', state: 'close', logoutPending: true } }, + }); + + // /instance/connect (polled by a consumer) answers the same and starts nothing. + expect(await call('GET', 'connect')).toEqual({ + status: 200, + body: { instance: { instanceName: 'test', state: 'close', logoutPending: true } }, + }); + + // It reconnects only to deliver the logout, and forwards and stores nothing meanwhile. + const sock = await reconnected(2); + await deliverWhilePending(sock, service); + expect({ emitted: emitted.map((e) => e.event), stored: stored() }).toEqual({ + emitted: [], + stored: { messages: 0, chats: 0, contacts: 0 }, + }); + + // The connection returns: the logout goes out at once, exactly once, and then the session is wiped. + await opened(sock); + await settle(service); + expect(sock.logouts).toBe(1); + expect({ + me: storedMe(), + dir: existsSync(DIR), + row: prisma.instance.rows.map((r: any) => r.connectionStatus), + }).toEqual({ me: [], dir: false, row: ['close'] }); + expect(await call('GET', 'connectionState')).toEqual({ + status: 200, + body: { instance: { instanceName: 'test', state: 'close' } }, + }); + // Nothing reconnects afterwards. + await new Promise((r) => setTimeout(r, 1_500)); + expect(built()).toHaveLength(2); + }); + + it('is resumed after a restart', async () => { + const { service } = await waitingToReconnect(); + expect(await call('DELETE', 'logout')).toEqual(PENDING); + // The process dies before the connection returns. + service.stopReconnecting(); + service.connect = async () => undefined; + socketSpy.mockClear(); + + const monitor = await startProcess(); + const sock = await reconnected(1); + const resumed = monitor.waInstances.test; + await deliverWhilePending(sock, resumed); + expect({ emitted: emitted.map((e) => e.event), stored: stored() }).toEqual({ + emitted: [], + stored: { messages: 0, chats: 0, contacts: 0 }, + }); + + await opened(sock); + await settle(resumed); + expect({ logouts: sock.logouts, me: storedMe(), dir: existsSync(DIR) }).toEqual({ logouts: 1, me: [], dir: false }); + }); + + it('finishes when WhatsApp answers loggedOut on the reconnect (the device was already removed)', async () => { + const { service } = await waitingToReconnect(); + expect(await call('DELETE', 'logout')).toEqual(PENDING); + + const sock = await reconnected(2); + sock.end(new Boom('Stream Errored (conflict)', { statusCode: 401 })); + await settle(service); + expect({ logouts: sock.logouts, me: storedMe(), dir: existsSync(DIR) }).toEqual({ logouts: 0, me: [], dir: false }); + await new Promise((r) => setTimeout(r, 1_500)); + expect(built()).toHaveLength(2); + }); + + it('on delete, leaves the API at once and keeps only what the logout needs until it is delivered', async () => { + const { monitor, service } = await waitingToReconnect(); + + expect(await call('DELETE', 'delete')).toEqual({ + status: 202, + body: { + status: 'PENDING', + error: false, + response: { message: 'Instance deleted; its logout will reach WhatsApp when the connection returns' }, + }, + }); + // Gone from the API: no instance, no row, no settings. Kept: the creds, the key files and marker, the proxy. + expect(monitor.waInstances.test).toBeUndefined(); + expect({ + instances: prisma.instance.rows.length, + settings: prisma.setting.rows.length, + proxies: prisma.proxy.rows.length, + me: storedMe(), + marker: JSON.parse(readFileSync(MARKER, 'utf8')), + }).toMatchObject({ + instances: 0, + settings: 0, + proxies: 1, + me: [WUID], + marker: { instanceName: 'test', deleted: true }, + }); + // Until it finishes, connectionState (global key) says so; nothing else answers for the name. + expect(await call('GET', 'connectionState', globalKey)).toEqual({ + status: 200, + body: { instance: { instanceName: 'test', state: 'close', logoutPending: true } }, + }); + + // The name stays taken until then, so connectionState cannot mean a new instance. + const create = await fetch(`${app.base}/instance/create`, { + method: 'POST', + headers: { apikey: globalKey, 'content-type': 'application/json' }, + body: JSON.stringify({ instanceName: 'test', integration: 'WHATSAPP-BAILEYS' }), + }); + expect(create.status).toBe(403); + + const sock = await reconnected(2); + await deliverWhilePending(sock, service); + await opened(sock); + await settle(service); + expect({ + logouts: sock.logouts, + emitted: emitted.map((e) => e.event), + me: storedMe(), + dir: existsSync(DIR), + proxies: prisma.proxy.rows.length, + }).toEqual({ logouts: 1, emitted: [], me: [], dir: false, proxies: 0 }); + // Finished: the name is gone from the API. + expect((await call('GET', 'connectionState', globalKey)).status).toBe(404); + }); + + it('on logout then delete (a purge), resumes after a restart', async () => { + const { service } = await waitingToReconnect(); + expect(await call('DELETE', 'logout')).toEqual(PENDING); + expect((await call('DELETE', 'delete')).status).toBe(202); + service.stopReconnecting(); + service.connect = async () => undefined; + socketSpy.mockClear(); + + const monitor = await startProcess(); + const sock = await reconnected(1); + expect(monitor.waInstances.test).toBeUndefined(); + const finishing = monitor.finishingLogouts.test; + await opened(sock); + await settle(finishing); + expect({ logouts: sock.logouts, me: storedMe(), dir: existsSync(DIR), proxies: prisma.proxy.rows.length }).toEqual({ + logouts: 1, + me: [], + dir: false, + proxies: 0, + }); + }); + + it('a logout on an open connection still completes at once', async () => { + await linkedInstance(); + const monitor = await startProcess(); + await vi.waitFor(() => expect(built()).toHaveLength(1)); + const service = monitor.waInstances.test; + await opened(current()); + + expect(await call('DELETE', 'logout')).toEqual({ + status: 200, + body: { status: 'SUCCESS', error: false, response: { message: 'Instance logged out' } }, + }); + await settle(service); + expect({ logouts: current().logouts, me: storedMe(), marker: existsSync(MARKER) }).toEqual({ + logouts: 1, + me: [], + marker: false, + }); + }); +}); diff --git a/test/instance/unlink-socket-down.test.ts b/test/instance/unlink-socket-down.test.ts index ddfe8f1799..d2efeba4dd 100644 --- a/test/instance/unlink-socket-down.test.ts +++ b/test/instance/unlink-socket-down.test.ts @@ -1,5 +1,5 @@ -// Unlinking an instance (DELETE /instance/logout, DELETE /instance/delete) must -// unlink it even when its socket is down. Baileys' logout first sends +// Unlinking an instance (DELETE /instance/logout, DELETE /instance/delete) whose +// socket is down. Baileys' logout first sends // remove-companion-device, and sendRawMessage throws Boom('Connection Closed', // 428) when the ws is not open (rc14 lib/Socket/socket.js, sendRawMessage), // before logout ends the socket. 2.3.7's logoutInstance awaits it unguarded, so @@ -9,6 +9,12 @@ // boot auto-connect of a row that says open) and keeps receiving the person's // messages after they unlinked. // +// Wiping the credentials locally instead (what this file first required) is no +// better: without them the device can never tell WhatsApp to remove it, so it +// stays on the person's Linked devices. So the logout is kept pending until the +// connection returns (test/instance/logout-pending.test.ts has the delivery): +// the credentials stay, a marker says it is pending, and the answer says so. +// // The instance here is mid-reconnect, the way it is after a dropped connection: // Evolution's real connect builds a real Baileys socket, pointed at a local // "WhatsApp" that accepts the TCP connection and never answers the WebSocket @@ -94,9 +100,13 @@ beforeAll(async () => { }); afterEach(async () => { // Tear down without the service answering the close with a reconnect. - for (const service of h.services) service.connectToWhatsapp = async () => undefined; + for (const service of h.services) { + service.connectToWhatsapp = async () => undefined; + service.connect = async () => undefined; + } for (const s of h.sockets) await s.end(undefined).catch(() => undefined); for (const service of h.services) await settle(service); + for (const service of h.services) service.stopReconnecting(); whatsapp.drop(); h.sockets.length = 0; h.services.length = 0; @@ -160,30 +170,44 @@ async function expectUnlinked(service: any) { expect(h.sockets.map((s) => s.ws.isClosed)).toEqual([true]); } +/** The logout is pending: the session is kept, marked pending, and its socket is still dialling to deliver it. */ +async function expectPending() { + expect(JSON.parse(JSON.parse(prisma.session.rows[0].creds)).me.id).toBe(WUID); + expect(readdirSync(join(tmp, 'inst-1')).sort()).toEqual(['logout-pending.json', 'pre-key-1.json']); + expect(h.sockets.map((s) => s.ws.isConnecting)).toEqual([true]); +} + describe('unlinking an instance whose connection is down', () => { - it('logout wipes the session, ends the socket, and the instance stays down', async () => { + it('logout keeps the session and answers that the logout is pending', async () => { const { service } = await reconnectingInstance(); expect(await call('logout')).toEqual({ - status: 200, - body: { status: 'SUCCESS', error: false, response: { message: 'Instance logged out' } }, + status: 202, + body: { + status: 'PENDING', + error: false, + response: { message: 'Logout pending: WhatsApp will be told when the connection returns' }, + }, }); - await expectUnlinked(service); - // The boot auto-connects only a row that says open or connecting (monitor.service.ts setInstance). - expect(prisma.instance.rows.map((r: any) => r.connectionStatus)).toEqual(['close']); - expect(service.connectionStatus.state).toBe('close'); + await settle(service); + await expectPending(); }); - it('delete wipes the session and removes the instance', async () => { + it('delete removes the instance from the API and keeps the session for the logout', async () => { const { service, waMonitor } = await reconnectingInstance(); expect(await call('delete')).toEqual({ - status: 200, - body: { status: 'SUCCESS', error: false, response: { message: 'Instance deleted' } }, + status: 202, + body: { + status: 'PENDING', + error: false, + response: { message: 'Instance deleted; its logout will reach WhatsApp when the connection returns' }, + }, }); - await expectUnlinked(service); + await settle(service); + await expectPending(); expect(waMonitor.waInstances.test).toBeUndefined(); expect(prisma.instance.rows).toEqual([]); }); From 1d1592922d4d83ec75e09a27597e1ce64300570d Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 15:01:04 +0300 Subject: [PATCH 047/157] fix(instance): keep the session until WhatsApp accepts the logout Baileys' logout sends remove-companion-device, the only thing that takes the device off the person's Linked devices, and it needs an open socket and the session's credentials. A logout that cannot send it now keeps them and stays pending until it can, instead of wiping them (which left the device on the phone for good). logoutInstance returns 'done' or 'pending'. A linked session (creds with `me`) whose socket is not open, or whose send throws, is marked pending: the credentials and key files stay, a marker file logout-pending.json is written in the instance's directory under INSTANCE_DIR, and a reconnect is scheduled if none is waiting. A session that is not linked has nothing to tell WhatsApp and is wiped at once, as before. The marker is a file, not a column: no schema change (the fork stays migration-identical to 2.3.7), it sits next to the session's signal key files so it has their durability, and the rm that wipes those keys removes it. The Instance columns that could carry it are an enum (connectionStatus: open/close/connecting) or overwritten by every final close (disconnection*). While a logout is under way the instance forwards and stores nothing: event batches are dropped except connection.update and creds.update, the history lid tap is off, and sendDataWebhook sends nothing. It reconnects with the existing backoff only to deliver the logout: on open it sends the logout at once; the loggedOut close that follows, or a loggedOut (or other final) close from WhatsApp on the reconnect (the device was already removed), or a QR (WhatsApp does not know the device), finishes it: the session is wiped, then the instance directory with its keys and marker, then the usual loggedOut close runs (row to close, STATUS_INSTANCE, LOGOUT_INSTANCE and CONNECTION_UPDATE close webhooks, cleanup). The wipe now comes before that close's cleanup, which removes the race where the cleanup deleted the directory while removeSession recreated it (ENOENT, 500) on a logout over an open socket. On boot, an instance with a marker connects only to resume it; public connectToWhatsapp refuses while one is under way. HTTP: a pending logout answers 202 { status: 'PENDING', error: false, response: { message } }, and so does a second logout; a logout that reached WhatsApp answers 200 SUCCESS as before. connectionState adds logoutPending: true while pending (state keeps its values), and /instance/connect answers with that connectionState instead of connecting. A logout of a "close" instance is refused as before unless it has a reconnect waiting, a connect under way, or a linked session stored. Delete: when its logout is pending (including a purge: logout, then delete) it answers 202 PENDING and the instance leaves the API at once (memory, its row, settings, webhook, chats and every other stored table) keeping only the session, key files, marker and proxy (so the logout goes out through the same exit). It finishes in WAMonitoringService's finishingLogouts, resumed on boot from markers with deleted: true and no row, sends nothing, then removes the session, directory and proxy. Until then GET /instance/connectionState/{name} (global key) answers logoutPending: true and creating that name answers 403; once finished, connectionState answers 404. remove.instance now shuts the instance down (no reconnect, socket detached then ended) rather than only cancelling a waiting reconnect, so a delete whose logout failed for another reason still leaves no socket behind. Co-Authored-By: Claude Opus 5.5 --- src/api/controllers/instance.controller.ts | 46 +++- src/api/guards/instance.guard.ts | 9 +- .../whatsapp/whatsapp.baileys.service.ts | 228 ++++++++++++++++-- src/api/routes/index.router.ts | 1 + src/api/routes/instance.router.ts | 6 +- src/api/services/monitor.service.ts | 75 +++++- src/utils/logout-marker.ts | 32 +++ 7 files changed, 364 insertions(+), 33 deletions(-) create mode 100644 src/utils/logout-marker.ts diff --git a/src/api/controllers/instance.controller.ts b/src/api/controllers/instance.controller.ts index f982379d22..2aefa946d7 100644 --- a/src/api/controllers/instance.controller.ts +++ b/src/api/controllers/instance.controller.ts @@ -17,6 +17,18 @@ import { v4 } from 'uuid'; import { ProxyController } from './proxy.controller'; +// A logout that could not reach WhatsApp yet: answered 202, and connectionState carries logoutPending: true until it has. +const LOGOUT_PENDING = { + status: 'PENDING', + error: false, + response: { message: 'Logout pending: WhatsApp will be told when the connection returns' }, +}; +const DELETE_PENDING = { + status: 'PENDING', + error: false, + response: { message: 'Instance deleted; its logout will reach WhatsApp when the connection returns' }, +}; + export class InstanceController { constructor( private readonly waMonitor: WAMonitoringService, @@ -315,6 +327,11 @@ export class InstanceController { throw new BadRequestException('The "' + instanceName + '" instance does not exist'); } + // A pending logout owns the connection: answer with that instead of starting a new one. + if (instance.logoutPending) { + return await this.connectionState({ instanceName }); + } + if (state == 'open') { return await this.connectionState({ instanceName }); } @@ -391,10 +408,13 @@ export class InstanceController { } public async connectionState({ instanceName }: InstanceDto) { + // A deleted instance whose logout is still on its way to WhatsApp answers here (and only here) until it is. + const instance = this.waMonitor.waInstances[instanceName] ?? this.waMonitor.finishingLogouts?.[instanceName]; return { instance: { instanceName: instanceName, - state: this.waMonitor.waInstances[instanceName]?.connectionStatus?.state, + state: instance?.connectionStatus?.state, + ...(instance?.logoutPending ? { logoutPending: true } : {}), }, }; } @@ -435,13 +455,18 @@ export class InstanceController { public async logout({ instanceName }: InstanceDto) { const { instance } = await this.connectionState({ instanceName }); + const waInstance = this.waMonitor.waInstances[instanceName]; - if (instance.state === 'close') { + if (waInstance?.logoutPending) return LOGOUT_PENDING; + + // "close" is also an instance whose socket dropped and whose linked session is still stored + // (a reconnect waiting): that one is logged out, or its reconnect brings the session back. + if (instance.state === 'close' && !(await waInstance?.hasSessionToLogOut?.())) { throw new BadRequestException('The "' + instanceName + '" instance is not connected'); } try { - await this.waMonitor.waInstances[instanceName]?.logoutInstance(); + if ((await waInstance?.logoutInstance()) === 'pending') return LOGOUT_PENDING; return { status: 'SUCCESS', error: false, response: { message: 'Instance logged out' } }; } catch (error) { @@ -455,9 +480,13 @@ export class InstanceController { const waInstances = this.waMonitor.waInstances[instanceName]; if (this.configService.get('CHATWOOT').ENABLED) waInstances?.clearCacheChatwoot(); - if (instance.state === 'connecting' || instance.state === 'open') { + let pending = !!waInstances?.logoutPending; + if ( + !pending && + (instance.state === 'connecting' || instance.state === 'open' || (await waInstances?.hasSessionToLogOut?.())) + ) { try { - await this.logout({ instanceName }); + pending = (await this.logout({ instanceName }))?.status === 'PENDING'; } catch (error) { // A failed logout must not stop the delete. The remove.instance emit // below is the only path that purges the in-memory entry and runs @@ -472,6 +501,13 @@ export class InstanceController { } } + // The logout has not reached WhatsApp: the instance leaves the API now, keeping only what the + // logout needs, and finishes it in the background (WAMonitoringService.deleteKeepingLogout). + if (pending) { + await this.waMonitor.deleteKeepingLogout(instanceName); + return DELETE_PENDING; + } + try { waInstances?.sendDataWebhook(Events.INSTANCE_DELETE, { instanceName, diff --git a/src/api/guards/instance.guard.ts b/src/api/guards/instance.guard.ts index e692f3622e..1460ffe2b5 100644 --- a/src/api/guards/instance.guard.ts +++ b/src/api/guards/instance.guard.ts @@ -32,7 +32,11 @@ export async function instanceExistsGuard(req: Request, _: Response, next: NextF throw new BadRequestException('"instanceName" not provided.'); } - if (!(await getInstance(param.instanceName))) { + // A deleted instance whose logout has not reached WhatsApp yet answers connectionState, nothing else. + const finishingLogout = + req.originalUrl.includes('/instance/connectionState/') && !!waMonitor.finishingLogouts?.[param.instanceName]; + + if (!finishingLogout && !(await getInstance(param.instanceName))) { throw new NotFoundException(`The "${param.instanceName}" instance does not exist`); } @@ -42,7 +46,8 @@ export async function instanceExistsGuard(req: Request, _: Response, next: NextF export async function instanceLoggedGuard(req: Request, _: Response, next: NextFunction) { if (req.originalUrl.includes('/instance/create')) { const instance = req.body as InstanceDto; - if (await getInstance(instance.instanceName)) { + // The name of a deleted instance stays taken until its logout has reached WhatsApp. + if ((await getInstance(instance.instanceName)) || waMonitor.finishingLogouts?.[instance.instanceName]) { throw new ForbiddenException(`This name "${instance.instanceName}" is already in use.`); } diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 7c14b15bad..c5236e891d 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -75,6 +75,7 @@ import { QrCode, S3, } from '@config/env.config'; +import { INSTANCE_DIR } from '@config/path.config'; import { BadRequestException, InternalServerErrorException, NotFoundException } from '@exceptions'; import ffmpegPath from '@ffmpeg-installer/ffmpeg'; import { Boom } from '@hapi/boom'; @@ -83,6 +84,7 @@ import { Instance, Message } from '@prisma/client'; import { createJid } from '@utils/createJid'; import { fetchLatestWaWebVersion } from '@utils/fetchLatestWaWebVersion'; import { jidKind, makeBaileysLogger } from '@utils/log-privacy'; +import { readLogoutMarker, writeLogoutMarker } from '@utils/logout-marker'; import { makeProxyAgent, makeProxyAgentUndici } from '@utils/makeProxyAgent'; import { getOnWhatsappCache, saveOnWhatsappCache } from '@utils/onWhatsappCache'; import { QueryLimiter } from '@utils/queryLimiter'; @@ -142,6 +144,7 @@ import { createHash } from 'crypto'; import EventEmitter2 from 'eventemitter2'; import ffmpeg from 'fluent-ffmpeg'; import FormData from 'form-data'; +import { rmSync } from 'fs'; import Long from 'long'; import mimeTypes from 'mime-types'; import NodeCache from 'node-cache'; @@ -261,6 +264,10 @@ export class BaileysStartupService extends ChannelStartupService { private connecting: { number: string | null; socket: Promise } | null = null; // Stops Evolution listening to the current socket; called before that socket is ended. private detachClient?: () => void; + // A logout under way. Until it is done the instance forwards and stores nothing, and it connects + // only to tell WhatsApp. `marked`: it could not reach WhatsApp, so it is pending, with a marker + // (utils/logout-marker.ts) that survives a restart. `deleted`: the instance has left the API. + private logout: { marked: boolean; deleted: boolean; settle: (outcome: 'done' | 'pending') => void } | null = null; private logBaileys = this.configService.get('LOG').BAILEYS; private eventProcessingQueue: Promise = Promise.resolve(); // The dispatcher media downloads go through: the instance's proxy, the same exit as the socket. @@ -294,32 +301,212 @@ export class BaileysStartupService extends ChannelStartupService { return this.stateConnection; } - public async logoutInstance() { - this.stopReconnecting(); + /** + * Log the device out on WhatsApp's side (remove-companion-device, which takes it off the person's + * Linked devices), then wipe the session. That needs an open socket and the credentials, so when + * the socket is down or waiting to reconnect the credentials are kept and the logout is pending + * until the connection returns: 'pending'. A session that is not linked has nothing to tell + * WhatsApp and is wiped at once. + */ + public async logoutInstance(): Promise<'done' | 'pending'> { + // A connect under way finishes first, so the socket logged out is the one it builds. + await this.connecting?.socket.catch(() => undefined); + if (this.logout?.marked) return 'pending'; + if (this.logout) return 'done'; + + const linked = await this.hasLinkedSession(); this.messageProcessor.onDestroy(); this.pictureCache.clear(); + const socketClosed = this.stateConnection.state === 'close'; + const outcome = new Promise<'done' | 'pending'>( + (settle) => (this.logout = { marked: false, deleted: false, settle }), + ); + + if (linked) { + try { + if (this.stateConnection.state !== 'open') throw new Error('the connection is not open'); + // Sends remove-companion-device, then ends the socket with loggedOut: that close finishes it. + await this.client.logout('Log out instance: ' + this.instanceName); + } catch (error) { + this.logger.warn(`Logout could not reach WhatsApp (${error?.message}): pending until the connection returns`); + await this.markLogoutPending(); + } + } else { + this.stopReconnecting(); + try { + // Nothing to tell WhatsApp: Baileys only ends the socket, with loggedOut. + await this.client?.logout('Log out instance: ' + this.instanceName); + } catch { + // No socket to end. + } + // A socket that had already closed announces nothing when ended. + if (socketClosed) await this.finishLogout(); + } + + // The loggedOut close normally follows at once; if it never comes, keep the session (pending). + let timer: NodeJS.Timeout; + const late = new Promise<'late'>((r) => (timer = setTimeout(() => r('late'), 10_000))); + const result = await Promise.race([outcome, late]); + clearTimeout(timer); + if (result === 'late') { + await this.markLogoutPending(); + return 'pending'; + } + return result; + } - // Baileys' logout tells WhatsApp to remove this device and then ends the socket. - // With the socket down the first step throws ('Connection Closed') and the socket - // is never ended. Unlink locally anyway: wipe the credentials, then end the socket - // as a logout would. A loggedOut close is final (no reconnect) and runs the same - // cleanup as a logout that reached WhatsApp. The device stays listed on the phone. - let unreachable = false; + /** Whether a logout is pending (it could not reach WhatsApp yet). */ + public get logoutPending() { + return !!this.logout?.marked; + } + + /** The instance was deleted while its logout is pending: it finishes out of the API, then removes the rest. */ + public async markLogoutDeleted() { + if (!this.logout) return; + this.logout.deleted = true; + await this.writeMarker(); + } + + /** On boot: an instance with a logout marker connects only to finish its logout. Returns whether it had one. */ + public async resumePendingLogout(): Promise { + const marker = readLogoutMarker(this.instanceId); + if (!marker) return false; + this.logout = { marked: true, deleted: !!marker.deleted, settle: () => undefined }; + this.logger.info(`Resuming a pending logout for instance "${this.instance.name}"`); try { - await this.client?.logout('Log out instance: ' + this.instanceName); + await this.connect(this.phoneNumber); + } catch (error) { + this.logger.error({ message: 'Connect for a pending logout failed', error: error?.toString() }); + this.scheduleReconnect(); + } + return true; + } - this.client?.ws?.close(); + /** Whether logging out an instance that is not connected has anything to do. */ + public async hasSessionToLogOut(): Promise { + if (this.reconnectTimer || this.connecting) return true; + return this.hasLinkedSession(); + } + + /** The instance is removed from the API: no reconnect, and its socket let go of without its close being handled. */ + public shutdown() { + this.stopReconnecting(); + this.retireClient(); + } + + /** Whether the stored session is a linked device (creds carry `me`), i.e. whether WhatsApp has something to remove. */ + private async hasLinkedSession(): Promise { + const cache = this.configService.get('CACHE'); + const provider = this.configService.get('PROVIDER'); + const db = this.configService.get('DATABASE'); + if (provider?.ENABLED || (cache?.REDIS.ENABLED && cache?.REDIS.SAVE_INSTANCES) || !db.SAVE_DATA.INSTANCE) { + return !!this.instance.authState?.state?.creds?.me?.id; + } + const row = await this.prismaRepository.session.findFirst({ where: { sessionId: this.instanceId } }); + let creds: any = row?.creds; + while (typeof creds === 'string') creds = JSON.parse(creds); + return !!creds?.me?.id; + } + + private async writeMarker() { + try { + await writeLogoutMarker(this.instanceId, { + instanceName: this.instance.name, + deleted: !!this.logout?.deleted, + since: new Date().toISOString(), + }); } catch (error) { - unreachable = true; - this.logger.warn(`Logout could not reach WhatsApp (${error?.message}), unlinking locally`); + // Still pending in this process; only a restart before it is delivered would lose it. + this.logger.error({ message: 'Could not write the logout marker', error: error?.toString() }); + } + } + + /** The logout could not reach WhatsApp: keep the session, mark it pending, and reconnect to deliver it. */ + private async markLogoutPending() { + if (!this.logout || this.logout.marked) return; + this.logout.marked = true; + await this.writeMarker(); + this.logout.settle('pending'); + if (!this.reconnectTimer && !this.connecting && this.stateConnection.state === 'close') this.scheduleReconnect(); + } + + /** A connection.update while a logout is under way: deliver it on open, finish on loggedOut, else reconnect. */ + private async logoutUpdate({ qr, connection, lastDisconnect }: Partial, from?: WASocket) { + if (from && from !== this.client) return; + const statusCode = (lastDisconnect?.error as Boom)?.output?.statusCode; + if (connection) this.stateConnection = { state: connection, statusReason: statusCode ?? 200 }; + + // A QR: WhatsApp does not know this device, so there is nothing left to remove. + if (qr) return this.finishLogout(); + + if (connection === 'open') { + this.reconnectAttempts = 0; + try { + await this.client.logout('Log out instance: ' + this.instanceName); + } catch (error) { + // The socket dropped before the logout went out: its close reconnects. + this.logger.warn(`Pending logout not sent (${error?.message}), trying again on the next connection`); + } + return; } + if (connection === 'close') { + // loggedOut: the logout went out, or WhatsApp had already removed the device. The other + // final codes leave nothing to log out either. + if ([DisconnectReason.loggedOut, DisconnectReason.forbidden, 402, 406].includes(statusCode)) { + return this.finishLogout(); + } + if (!this.logout.marked) return this.markLogoutPending(); + this.scheduleReconnect(statusCode); + } + } + + /** WhatsApp has been told (or has nothing to remove): wipe the session and the marker, then close as a logout does. */ + private async finishLogout() { + const logout = this.logout; + if (!logout) return; + this.stopReconnecting(); try { await this.removeSession(); - } finally { - if (unreachable) - await this.client?.end(new Boom('Intentional Logout', { statusCode: DisconnectReason.loggedOut })); + } catch (error) { + // The device is off WhatsApp; the next connection answers loggedOut, which finishes it again. + this.logger.error({ message: 'Could not wipe the session after the logout', error: error?.toString() }); + if (!logout.marked) await this.markLogoutPending(); + else this.scheduleReconnect(); + return; + } + rmSync(join(INSTANCE_DIR, this.instanceId), { recursive: true, force: true }); + this.logout = null; + + if (logout.deleted) { + // Out of the API already: remove what was kept for the logout, and announce nothing. + await this.prismaRepository.proxy.deleteMany({ where: { instanceId: this.instanceId } }).catch(() => undefined); + this.shutdown(); + this.stateConnection = { state: 'close', statusReason: DisconnectReason.loggedOut }; + this.eventEmitter.emit('logout.finished', this); + } else { + // The same close, webhooks and cleanup as a logout that reached WhatsApp at once. + await this.connectionUpdate({ + connection: 'close', + lastDisconnect: { + error: new Boom('Intentional Logout', { statusCode: DisconnectReason.loggedOut }), + date: new Date(), + }, + }); } + logout.settle('done'); + } + + // While a logout is under way the instance forwards nothing. + public async sendDataWebhook( + event: Events, + data: T, + local = true, + integration?: string[], + extra?: Record, + ) { + if (this.logout) return; + return super.sendDataWebhook(event, data, local, integration, extra); } private async removeSession() { @@ -524,7 +711,7 @@ export class BaileysStartupService extends ChannelStartupService { this.eventEmitter.emit('logout.instance', this.instance.name, 'inner'); this.client?.ws?.close(); - this.client.end(new Error('Close connection')); + this.client?.end(new Error('Close connection')); this.sendDataWebhook(Events.CONNECTION_UPDATE, { instance: this.instance.name, ...this.stateConnection }); } @@ -827,6 +1014,7 @@ export class BaileysStartupService extends ChannelStartupService { /** A new connect attempt: a fresh QR budget, and no QR or pairing code left from an earlier attempt. */ public async connectToWhatsapp(number?: string): Promise { + if (this.logout) throw new BadRequestException('A logout is pending: the instance connects only to deliver it'); this.instance.qrcode = { count: 0 }; return await this.connect(number); } @@ -1098,7 +1286,7 @@ export class BaileysStartupService extends ChannelStartupService { const mappings = [...data.lidPnMappings]; this.eventProcessingQueue = this.eventProcessingQueue.then(() => { try { - if (!this.endSession && client === this.client) this.lidMappingHandle(mappings); + if (!this.endSession && !this.logout && client === this.client) this.lidMappingHandle(mappings); } catch (error) { this.logger.error(error); } @@ -2078,6 +2266,12 @@ export class BaileysStartupService extends ChannelStartupService { if (client !== this.client) return; this.eventProcessingQueue = this.eventProcessingQueue.then(async () => { try { + // A logout under way: nothing is forwarded or stored; the connection only delivers the logout. + if (this.logout) { + if (events['creds.update']) this.instance.authState.saveCreds(); + if (events['connection.update']) await this.logoutUpdate(events['connection.update'], client); + return; + } if (!this.endSession) { const database = this.configService.get('DATABASE'); // A failed read must not drop the batch (messages, creds, connection updates with it): diff --git a/src/api/routes/index.router.ts b/src/api/routes/index.router.ts index 45c43fca5b..f39576fb61 100644 --- a/src/api/routes/index.router.ts +++ b/src/api/routes/index.router.ts @@ -28,6 +28,7 @@ import { ViewsRouter } from './view.router'; enum HttpStatus { OK = 200, CREATED = 201, + ACCEPTED = 202, NOT_FOUND = 404, FORBIDDEN = 403, BAD_REQUEST = 400, diff --git a/src/api/routes/instance.router.ts b/src/api/routes/instance.router.ts index 3559893e68..1cf61f05dc 100644 --- a/src/api/routes/instance.router.ts +++ b/src/api/routes/instance.router.ts @@ -84,7 +84,8 @@ export class InstanceRouter extends RouterBroker { execute: (instance) => instanceController.logout(instance), }); - return res.status(HttpStatus.OK).json(response); + // A logout still on its way to WhatsApp is accepted, not done. + return res.status(response?.status === 'PENDING' ? HttpStatus.ACCEPTED : HttpStatus.OK).json(response); }) .delete(this.routerPath('delete'), ...guards, async (req, res) => { const response = await this.dataValidate({ @@ -94,7 +95,8 @@ export class InstanceRouter extends RouterBroker { execute: (instance) => instanceController.deleteInstance(instance), }); - return res.status(HttpStatus.OK).json(response); + // A logout still on its way to WhatsApp is accepted, not done. + return res.status(response?.status === 'PENDING' ? HttpStatus.ACCEPTED : HttpStatus.OK).json(response); }); } diff --git a/src/api/services/monitor.service.ts b/src/api/services/monitor.service.ts index 8cec2affd2..36b0ebe627 100644 --- a/src/api/services/monitor.service.ts +++ b/src/api/services/monitor.service.ts @@ -7,9 +7,10 @@ import { CacheConf, Chatwoot, ConfigService, Database, DelInstance, ProviderSess import { Logger } from '@config/logger.config'; import { INSTANCE_DIR, STORE_DIR } from '@config/path.config'; import { NotFoundException } from '@exceptions'; +import { readLogoutMarker } from '@utils/logout-marker'; import { execFileSync } from 'child_process'; import EventEmitter2 from 'eventemitter2'; -import { rmSync } from 'fs'; +import { readdirSync, rmSync } from 'fs'; import { join } from 'path'; import { CacheService } from './cache.service'; @@ -26,6 +27,11 @@ export class WAMonitoringService { ) { this.removeInstance(); this.noConnection(); + this.eventEmitter.on('logout.finished', (instance: any) => { + for (const [name, finishing] of Object.entries(this.finishingLogouts)) { + if (finishing === instance) delete this.finishingLogouts[name]; + } + }); Object.assign(this.db, configService.get('DATABASE')); Object.assign(this.redis, configService.get('CACHE')); @@ -38,6 +44,8 @@ export class WAMonitoringService { private readonly logger = new Logger('WAMonitoringService'); public readonly waInstances: Record = {}; + // Deleted instances whose logout has not reached WhatsApp yet, by name: out of the API, finishing it. + public readonly finishingLogouts: Record = {}; private readonly delInstanceTimeouts: Record = {}; private readonly providerSession: ProviderSession; @@ -188,7 +196,8 @@ export class WAMonitoringService { } } - public async cleaningStoreData(instanceName: string) { + /** `keepForLogout`: keep the session (creds, key files, logout marker) and the proxy a pending logout still needs. */ + public async cleaningStoreData(instanceName: string, { keepForLogout = false } = {}) { if (this.configService.get('CHATWOOT').ENABLED) { const instancePath = join(STORE_DIR, 'chatwoot', instanceName); execFileSync('rm', ['-rf', instancePath]); @@ -200,9 +209,11 @@ export class WAMonitoringService { if (!instance) return; - rmSync(join(INSTANCE_DIR, instance.id), { recursive: true, force: true }); + if (!keepForLogout) { + rmSync(join(INSTANCE_DIR, instance.id), { recursive: true, force: true }); - await this.prismaRepository.session.deleteMany({ where: { sessionId: instance.id } }); + await this.prismaRepository.session.deleteMany({ where: { sessionId: instance.id } }); + } await this.prismaRepository.chat.deleteMany({ where: { instanceId: instance.id } }); await this.prismaRepository.contact.deleteMany({ where: { instanceId: instance.id } }); @@ -211,7 +222,7 @@ export class WAMonitoringService { await this.prismaRepository.webhook.deleteMany({ where: { instanceId: instance.id } }); await this.prismaRepository.chatwoot.deleteMany({ where: { instanceId: instance.id } }); - await this.prismaRepository.proxy.deleteMany({ where: { instanceId: instance.id } }); + if (!keepForLogout) await this.prismaRepository.proxy.deleteMany({ where: { instanceId: instance.id } }); await this.prismaRepository.rabbitmq.deleteMany({ where: { instanceId: instance.id } }); await this.prismaRepository.nats.deleteMany({ where: { instanceId: instance.id } }); await this.prismaRepository.sqs.deleteMany({ where: { instanceId: instance.id } }); @@ -233,11 +244,55 @@ export class WAMonitoringService { } else if (this.redis.REDIS.ENABLED && this.redis.REDIS.SAVE_INSTANCES) { await this.loadInstancesFromRedis(); } + await this.resumeDeletedLogouts(); } catch (error) { this.logger.error(error); } } + /** + * Delete while the logout could not reach WhatsApp: the instance leaves the API now (memory, its + * row and everything else stored for it), keeping only its session and proxy, and finishes the + * logout in the background. The service removes those when it has (BaileysStartupService.finishLogout). + */ + public async deleteKeepingLogout(instanceName: string) { + const instance = this.waInstances[instanceName]; + await instance.markLogoutDeleted(); + this.finishingLogouts[instanceName] = instance; + this.clearDelInstanceTime(instanceName); + delete this.waInstances[instanceName]; + await this.cleaningStoreData(instanceName, { keepForLogout: true }); + this.logger.warn(`Instance "${instanceName}" - REMOVED, its logout pending`); + } + + /** On boot: a deleted instance (no row) whose logout marker is still there finishes its logout. */ + private async resumeDeletedLogouts() { + let ids: string[]; + try { + ids = readdirSync(INSTANCE_DIR); + } catch { + return; + } + for (const instanceId of ids) { + const marker = readLogoutMarker(instanceId); + if (!marker?.deleted || !marker.instanceName) continue; + if (await this.prismaRepository.instance.findUnique({ where: { id: instanceId } })) continue; + const instanceData = { instanceId, instanceName: marker.instanceName, integration: Integration.WHATSAPP_BAILEYS }; + const instance = channelController.init(instanceData, { + configService: this.configService, + eventEmitter: this.eventEmitter, + prismaRepository: this.prismaRepository, + cache: this.cache, + chatwootCache: this.chatwootCache, + baileysCache: this.baileysCache, + providerFiles: this.providerFiles, + }); + instance.setInstance(instanceData); + this.finishingLogouts[marker.instanceName] = instance; + await (instance as any).resumePendingLogout(); + } + } + public async saveInstance(data: any) { try { const clientName = await this.configService.get('DATABASE').CONNECTION.CLIENT_NAME; @@ -293,6 +348,12 @@ export class WAMonitoringService { ownerJid: instanceData.ownerJid, }); + // A logout that had not reached WhatsApp before the restart: connect only to deliver it. + if (await (instance as any).resumePendingLogout?.()) { + this.waInstances[instanceData.instanceName] = instance; + return; + } + if (instanceData.connectionStatus === 'open' || instanceData.connectionStatus === 'connecting') { this.logger.info( `Auto-connecting instance "${instanceData.instanceName}" (status: ${instanceData.connectionStatus})`, @@ -392,8 +453,8 @@ export class WAMonitoringService { private removeInstance() { this.eventEmitter.on('remove.instance', async (instanceName: string) => { try { - // A reconnect waiting on its backoff would otherwise bring the removed instance back. - this.waInstances[instanceName]?.stopReconnecting?.(); + // No reconnect waiting on its backoff may bring the removed instance back, and its socket goes too. + this.waInstances[instanceName]?.shutdown?.(); await this.waInstances[instanceName]?.sendDataWebhook(Events.REMOVE_INSTANCE, null); this.clearDelInstanceTime(instanceName); diff --git a/src/utils/logout-marker.ts b/src/utils/logout-marker.ts new file mode 100644 index 0000000000..5b00869640 --- /dev/null +++ b/src/utils/logout-marker.ts @@ -0,0 +1,32 @@ +import { INSTANCE_DIR } from '@config/path.config'; +import { existsSync, readFileSync } from 'fs'; +import { mkdir, writeFile } from 'fs/promises'; +import { join } from 'path'; + +/** + * A logout that could not reach WhatsApp is kept pending until it does, across restarts. The + * marker is a file in the instance's directory under INSTANCE_DIR, next to the session's signal + * key files: no schema change (the fork stays migration-identical to 2.3.7), the same durability + * as the session it has to log out, and removed by the same rm that wipes those keys. + */ +export type LogoutMarker = { instanceName: string; deleted: boolean; since: string }; + +export const LOGOUT_MARKER_FILE = 'logout-pending.json'; + +export const logoutMarkerPath = (instanceId: string) => join(INSTANCE_DIR, instanceId, LOGOUT_MARKER_FILE); + +export function readLogoutMarker(instanceId: string): LogoutMarker | undefined { + const path = logoutMarkerPath(instanceId); + if (!existsSync(path)) return undefined; + try { + return JSON.parse(readFileSync(path, 'utf8')); + } catch { + // Unreadable is still pending: keeping a session too long is recoverable, dropping a logout is not. + return { instanceName: undefined, deleted: false, since: undefined }; + } +} + +export async function writeLogoutMarker(instanceId: string, marker: LogoutMarker) { + await mkdir(join(INSTANCE_DIR, instanceId), { recursive: true }); + await writeFile(logoutMarkerPath(instanceId), JSON.stringify(marker)); +} From 0f2a0c48402ac2a1a8695d6d6e98005e11986a33 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 15:04:24 +0300 Subject: [PATCH 048/157] test: a pending logout is never undone by a lost marker file A pending logout is recorded only in the marker file next to the session's key files under INSTANCE_DIR. The instance's row keeps the status it had before the drop (open), so if the marker is lost the boot auto-connects the instance as a normal one and it goes back to forwarding and storing an account the person logged out. The test logs out while a reconnect waits, deletes the marker, restarts, and requires the boot not to connect it. Co-Authored-By: Claude Opus 5.5 --- test/instance/logout-pending.test.ts | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/test/instance/logout-pending.test.ts b/test/instance/logout-pending.test.ts index 4550a85162..61b6434965 100644 --- a/test/instance/logout-pending.test.ts +++ b/test/instance/logout-pending.test.ts @@ -315,6 +315,26 @@ describe('a logout that cannot reach WhatsApp', () => { expect({ logouts: sock.logouts, me: storedMe(), dir: existsSync(DIR) }).toEqual({ logouts: 1, me: [], dir: false }); }); + // The marker lives with the key files (INSTANCE_DIR); the row is in the database. If the files + // are lost, the row alone must keep the boot from bringing the instance back as a normal one. + it('is never undone by a lost marker file: the boot does not connect the instance', async () => { + const { service } = await waitingToReconnect(); + expect(await call('DELETE', 'logout')).toEqual(PENDING); + service.stopReconnecting(); + service.connect = async () => undefined; + socketSpy.mockClear(); + rmSync(MARKER); + + const monitor = await startProcess(); + await vi.waitFor(() => expect(monitor.waInstances.test).toBeDefined()); + await new Promise((r) => setTimeout(r, 1_500)); + expect({ + socketsBuilt: built().length, + emitted: emitted.map((e) => e.event), + state: monitor.waInstances.test.connectionStatus.state, + }).toEqual({ socketsBuilt: 0, emitted: [], state: 'close' }); + }); + it('finishes when WhatsApp answers loggedOut on the reconnect (the device was already removed)', async () => { const { service } = await waitingToReconnect(); expect(await call('DELETE', 'logout')).toEqual(PENDING); From df633ca9a3e2ca6918bcbee5d9938665ad1135a6 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 15:04:58 +0300 Subject: [PATCH 049/157] fix(instance): record a pending logout on the instance row too When a logout goes pending, the Instance row is set to connectionStatus 'close' as well as the marker file being written. The boot auto-connects only rows that are open or connecting (WAMonitoringService.setInstance), and checks the marker before that, so: - marker present: the boot resumes the pending logout, as before; - marker lost: the instance loads down and is not connected, so it never goes back to forwarding and storing. Its credentials stay, so the logout can still be delivered: DELETE /instance/logout on it again (the controller lets a "close" instance with a linked session through) marks it pending and reconnects to deliver it. A failure to write the row is logged and does not stop the pending logout; the marker still covers a restart. Co-Authored-By: Claude Opus 5.5 --- .../channel/whatsapp/whatsapp.baileys.service.ts | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index c5236e891d..d7b09a886b 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -426,6 +426,13 @@ export class BaileysStartupService extends ChannelStartupService { if (!this.logout || this.logout.marked) return; this.logout.marked = true; await this.writeMarker(); + // The row too: the boot auto-connects only an open or connecting row, so even if the marker + // file is lost the instance does not come back as a normal one (it stays down, creds kept). + await this.prismaRepository.instance + .update({ where: { id: this.instanceId }, data: { connectionStatus: 'close' } }) + .catch((error) => + this.logger.error({ message: 'Could not record the pending logout on the row', error: error?.toString() }), + ); this.logout.settle('pending'); if (!this.reconnectTimer && !this.connecting && this.stateConnection.state === 'close') this.scheduleReconnect(); } From 0baf1d4d74a07754d2a38c98e8ca675171633c59 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 15:06:36 +0300 Subject: [PATCH 050/157] docs(fork): list what the fork changes Co-Authored-By: Claude Opus 5.5 --- FORK.md | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/FORK.md b/FORK.md index 578a7023e4..d1bbdcb4dc 100644 --- a/FORK.md +++ b/FORK.md @@ -35,8 +35,32 @@ A fix without a failing test first is not merged here. See `AGENTS.md`. ## Changes from 2.3.7 -The list grows with each red and green pair. `git log 2.3.7..` is the source of -truth, and `git diff 2.3.7 --stat` lists every file modified from the original. +Each line is a pair of commits: the test that failed on 2.3.7, then the fix. +`git log 2.3.7..` is the source of truth, and `git diff 2.3.7 --stat` lists +every file modified from the original. Upstream issues and pull requests are +named where one exists. + +**Contacts, names and groups** +- App-state sync keys are reloaded with `fromObject` in all three auth stores, so saved names, labels, mutes and archives keep syncing after a restart (#2576, #2384; fixes also offered in #2685 and #2610). +- Every @lid to phone mapping Baileys learns is forwarded on `contacts.upsert`, captured before Baileys' event buffer drops it. +- Every `contacts.upsert` item says whether its name is the one the owner saved (`saved`), and only when that is certain. +- Group updates reach subscriptions stored as `GROUP_UPDATE`, in all seven transports and in the global configurations (#2652). +- Group metadata is filled from `groups.update` instead of queried again for every group on every listing. + +**Messages and privacy** +- A `getMessage` miss answers nothing, so Baileys no longer relays an empty message on a retry (#2705, #2550). +- No message text, phone number, JID or push name reaches the logs at `LOG_LEVEL=ERROR,WARN`, including Baileys' own error logs. + +**Proxy** +- Media downloads, media uploads and the WhatsApp Web version fetch leave through the instance's proxy (uploads failed outright on a proxied instance). +- A connect waits for the instance's proxy and stored settings, so it never starts from the server's own address or with default settings. + +**Sessions and connections** +- A database error never replaces a linked session's credentials with fresh ones, and a failed settings read no longer drops an event batch. +- A logout or delete while the socket is down keeps the session until WhatsApp is told, so the device leaves the phone's Linked devices; it answers `202 PENDING` meanwhile and forwards nothing (#2520, #2508 in part). +- Reconnects back off from 1s to 60s instead of spinning, the version fetch times out after 10s, and an instance never runs two sockets (#2134, #2184, #2430; ideas from #2732). +- Profile pictures are looked up once per contact per hour, at most four at a time, and history never waits on them (#1883). +- One pairing code per connect attempt, and a fresh QR budget per attempt (#2100, #2696). ## Licence From 2d9aeea4bd927810cee46d8c2e66dbc1467421cd Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 15:14:51 +0300 Subject: [PATCH 051/157] test: pin that sends to @lid chats go through (guard, not a bug) This test passes on 2.3.7, so it is a regression guard, not the first half of a red and green pair. A client may key a person's chat by their phone when the mapping is known and by the @lid otherwise, so it relies on sends to an @lid chat working. They do: createJid keeps a lowercase @lid unchanged (src/utils/createJid.ts:38) and whatsappNumber answers exists for it without asking WhatsApp (whatsapp.baileys.service.ts ~4061, ~4075, ~4085), so the refusal that upstream #2544 exempts on develop is never reached here. Co-Authored-By: Claude Opus 5.5 --- test/handlers/lid-send.test.ts | 70 ++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) create mode 100644 test/handlers/lid-send.test.ts diff --git a/test/handlers/lid-send.test.ts b/test/handlers/lid-send.test.ts new file mode 100644 index 0000000000..d548f634b2 --- /dev/null +++ b/test/handlers/lid-send.test.ts @@ -0,0 +1,70 @@ +// A private chat can be keyed by an @lid (WhatsApp's linked identity) instead +// of a phone number. Baileys' onWhatsApp answers exists:false for an @lid, so +// a send that trusts that answer refuses a chat the person is actually in. +// Upstream fixed it in evolution-api #2544. A phone number that is genuinely +// not on WhatsApp must still be refused. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { describe, expect, it } from 'vitest'; + +import { makeService } from '../helpers/baileys-service'; +import type { Profile } from '../helpers/profiles'; + +const LID = '123456789012345@lid'; +const ABSENT = '972501112233'; + +function fakeSocket(service: any) { + const sent: { jid: string; content: any }[] = []; + const presence: { type: string; jid: string }[] = []; + const asked: string[] = []; + Object.assign(service.client, { + // Baileys' own answer: nothing but a phone number it can resolve exists. + onWhatsApp: async (...jids: string[]) => { + asked.push(...jids); + return jids.map((jid) => ({ exists: false, jid })); + }, + sendMessage: async (jid: string, content: any) => { + sent.push({ jid, content }); + return { key: { remoteJid: jid, fromMe: true, id: 'OUT1' }, message: { conversation: content.text }, messageTimestamp: 1_700_000_000, status: 1 }; + }, + presenceSubscribe: async () => undefined, + sendPresenceUpdate: async (type: string, jid: string) => void presence.push({ type, jid }), + }); + return { sent, presence, asked }; +} + +describe.each(['minimal', 'stored'])('sending to an @lid chat (%s)', (profile) => { + it('sendText to an @lid chat reaches the socket, addressed to that @lid', async () => { + const { service } = await makeService({ profile }); + const { sent } = fakeSocket(service); + + const res = await service.textMessage({ number: LID, text: 'hello' }); + + expect(sent.map(({ jid, content }) => ({ jid, text: content.text }))).toEqual([{ jid: LID, text: 'hello' }]); + expect(res.key).toEqual({ remoteJid: LID, fromMe: true, id: 'OUT1' }); + }); + + it('a typing presence to an @lid chat reaches the socket, addressed to that @lid', async () => { + const { service } = await makeService({ profile }); + const { presence } = fakeSocket(service); + + await service.sendPresence({ number: LID, presence: 'composing', delay: 1 }); + + expect(presence).toEqual([ + { type: 'composing', jid: LID }, + { type: 'paused', jid: LID }, + ]); + }); + + it('control: a phone number not on WhatsApp is still refused, and nothing is sent', async () => { + const { service } = await makeService({ profile }); + const { sent, asked } = fakeSocket(service); + + await expect(service.textMessage({ number: ABSENT, text: 'hello' })).rejects.toMatchObject({ status: 400 }); + + expect(asked).toEqual([`${ABSENT}@s.whatsapp.net`]); + expect(sent).toEqual([]); + }); +}); From 2493e111de8e53559d58b880e31ee93c11f422f0 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 15:27:41 +0300 Subject: [PATCH 052/157] test: getMessage answers nothing when the lookup fails When the database lookup throws, getMessage takes its catch path and answers { conversation: '' }. Baileys relays any truthy answer to a retry request, so a database blip sends the recipient an empty message and uses up the retry. Pinned under both profiles, for both the plain and the full answer, plus a control: once the database answers again, the stored message comes back as stored. Co-Authored-By: Claude Opus 5.5 --- test/handlers/get-message-miss.test.ts | 30 ++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/test/handlers/get-message-miss.test.ts b/test/handlers/get-message-miss.test.ts index 1e495ec6c8..faaa835d05 100644 --- a/test/handlers/get-message-miss.test.ts +++ b/test/handlers/get-message-miss.test.ts @@ -7,6 +7,8 @@ // Evolution answers a database miss with `{ conversation: '' }`, so the recipient // is sent an empty message. A deployment that stores no messages (for example // DATABASE_SAVE_DATA_NEW_MESSAGE=false, the `minimal` profile) misses every time. +// A lookup that THROWS (a database blip) took the catch path, which answered the +// same `{ conversation: '' }`, so the retry was used up on an empty message too. import { vi } from 'vitest'; vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); @@ -54,3 +56,31 @@ describe('getMessage, as Baileys calls it to answer a retry request', () => { expect(await service.getMessage(key)).toEqual({ conversation: 'the real text' }); }); }); + +describe('getMessage, when the database lookup fails', () => { + it.each(['minimal', 'stored'])('under the %s profile, answers undefined, not an empty message', async (profile) => { + const { service, prisma } = await makeService({ profile }); + prisma.$queryRaw = async () => { + throw new Error('Connection terminated unexpectedly'); + }; + const key = { remoteJid: CHAT, fromMe: true, id: '3EB0BLIP0000000000001' }; + expect(await service.getMessage(key)).toBeUndefined(); + expect(await service.getMessage(key, true)).toBeUndefined(); + }); + + it('under the stored profile, once the database answers again, the stored message is returned as stored', async () => { + const { service, prisma, ev } = await makeService({ profile: 'stored' }); + answerFromRows(prisma); + const key = { remoteJid: CHAT, fromMe: true, id: '3EB0BLIP0000000000002' }; + await deliver(service, ev, { + 'messages.upsert': { messages: [{ key, message: { conversation: 'the real text' }, messageTimestamp: 1_700_000_000 }], type: 'notify' }, + }); + const working = prisma.$queryRaw; + prisma.$queryRaw = async () => { + throw new Error('Connection terminated unexpectedly'); + }; + expect(await service.getMessage(key)).toBeUndefined(); + prisma.$queryRaw = working; + expect(await service.getMessage(key)).toEqual({ conversation: 'the real text' }); + }); +}); From 078d6c96a2f7223e81826c8777bfb4cdb225388a Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 15:28:13 +0300 Subject: [PATCH 053/157] fix(baileys): answer nothing when the getMessage lookup fails The catch path answered { conversation: '' }, which Baileys relays to a retry request as the message itself: a database blip sent the recipient an empty message and used up the retry (upstream #2706 describes the group case). A failed lookup now answers undefined, like a miss, and is logged without message content. The same change is offered upstream in #2728 and #2623. #2623 also skips a stored payload that carries only messageContextInfo; that part is not taken here. Evolution stores such a row only when the message it records was itself context-only (prepareMessage keeps the content of every other type), so answering it resends the original faithfully rather than an empty stand-in, and no test shows an empty relay from it. Co-Authored-By: Claude Opus 5.5 --- .../integrations/channel/whatsapp/whatsapp.baileys.service.ts | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index d7b09a886b..bf02bd14f3 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -816,7 +816,9 @@ export class BaileysStartupService extends ChannelStartupService { return webMessageInfo[0].message; } catch { - return { conversation: '' }; + // A failed lookup is a miss too: any truthy answer here is relayed as the message. + this.logger.warn('getMessage: the message lookup failed, answering nothing'); + return undefined; } } From c559cb1fb3e119c9cbd10c00c9923b5807a9862c Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 15:28:25 +0300 Subject: [PATCH 054/157] docs(fork): correct the issue references #2550 is the getBase64FromMediaMessage ephemeralMessage crash (fixed upstream by #2552), not the getMessage bug, so it no longer sits next to the getMessage fix. That line now names #2705, #2706 (the group case) and the upstream fixes #2728 and #2623, and covers the failed lookup as well as the miss. Co-Authored-By: Claude Opus 5.5 --- FORK.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/FORK.md b/FORK.md index d1bbdcb4dc..a25899531f 100644 --- a/FORK.md +++ b/FORK.md @@ -48,7 +48,7 @@ named where one exists. - Group metadata is filled from `groups.update` instead of queried again for every group on every listing. **Messages and privacy** -- A `getMessage` miss answers nothing, so Baileys no longer relays an empty message on a retry (#2705, #2550). +- A `getMessage` miss or failed lookup answers nothing, so Baileys no longer relays an empty message on a retry and uses it up (#2705, and #2706 for groups; fixes also offered in #2728 and #2623). - No message text, phone number, JID or push name reaches the logs at `LOG_LEVEL=ERROR,WARN`, including Baileys' own error logs. **Proxy** From a559dc4df6ede4a479201f1357e7e48ed0432eaf Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:17:40 +0300 Subject: [PATCH 055/157] test: group participant updates carry each participant's jid and phone number Baileys 7 emits group-participants.update with participants as GroupParticipant objects; participantsData must read each object's id and phone number (from the event, the group metadata or the LID mapping store), and keep participants as Baileys emitted it. Red: every item is { jid: , phoneNumber: "[object Object]" }. Co-Authored-By: Claude Opus 5.5 --- test/handlers/group-participants.test.ts | 222 +++++++++++++++++++++++ 1 file changed, 222 insertions(+) create mode 100644 test/handlers/group-participants.test.ts diff --git a/test/handlers/group-participants.test.ts b/test/handlers/group-participants.test.ts new file mode 100644 index 0000000000..b17f720651 --- /dev/null +++ b/test/handlers/group-participants.test.ts @@ -0,0 +1,222 @@ +// GROUP_PARTICIPANTS_UPDATE carries `participantsData` (CHANGELOG 2.3.5), one +// item per participant of the event, so a consumer can read the phone number of +// a participant WhatsApp addresses by a private @lid: +// { jid, phoneNumber?, name?, imgUrl? } +// +// Baileys 7 (rc9+) emits group-participants.update with `participants` as +// GroupParticipant objects ({ id, phoneNumber?, lid?, admin?, ... }), built in +// Socket/messages-recv.js from the group notification and emitted by +// Utils/process-message.js from the GROUP_PARTICIPANT_* stub. The handler read +// them as strings, so every item came out as { jid: , +// phoneNumber: "[object Object]" } and never found its name or picture. +// +// Inputs run through Baileys' own processMessage from a stub message and reach +// Evolution through the real event buffer; group metadata is Baileys' own +// extractGroupMetadata over a synthetic node. `participants` stays +// exactly what Baileys emitted, for consumers that already read it. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { WAMessageStubType } from 'baileys'; +import P from 'pino'; +import { beforeEach, describe, expect, it } from 'vitest'; + +import { deliver, makeService, settle, WUID } from '../helpers/baileys-service'; +import { captureOutput } from '../helpers/capture-output'; +import { emitted } from '../helpers/fake-server-module'; +import { realSignalRepository } from '../helpers/socket-history'; + +const GROUP = '120363000000000077@g.us'; +const lid = (n: number) => `1000000000077${String(n).padStart(2, '0')}@lid`; +const pn = (n: number) => `97250007770${n}@s.whatsapp.net`; +const ADMIN = { lid: lid(1), pn: pn(1) }; + +/** A participant exactly as Baileys rc14 builds it from a group notification (messages-recv.js). */ +const lidMember = (n: number, withPhone: boolean, admin: string | null = null) => ({ + id: lid(n), + phoneNumber: withPhone ? pn(n) : undefined, + lid: undefined, + username: undefined, + admin, +}); +const pnMember = (n: number, admin: string | null = null) => ({ + id: pn(n), + phoneNumber: undefined, + lid: lid(n), + username: undefined, + admin, +}); + +let extractGroupMetadata: (node: any) => any; +let processMessage: (message: any, ctx: any) => Promise; + +/** The group as WhatsApp describes it after the change: members addressed by @lid, with their phone. */ +function groupMeta(members: number[]) { + const node = { + tag: 'group', + attrs: { + id: GROUP.split('@')[0], + subject: 'Synthetic group', + s_t: '1700000000', + creation: '1690000000', + addressing_mode: 'lid', + }, + content: members.map((n) => ({ + tag: 'participant', + attrs: { jid: lid(n), phone_number: pn(n), ...(n === 1 ? { type: 'superadmin' } : {}) }, + })), + }; + return extractGroupMetadata({ tag: 'result', attrs: {}, content: [node] }); +} + +async function groupService(members: number[]) { + const { service, prisma, ev } = await makeService(); + const { repo } = await realSignalRepository(); + service.client.signalRepository = repo; + service.client.groupMetadata = async () => groupMeta(members); + await deliver(service, ev, {}); // wire the handlers + return { service, prisma, ev, repo }; +} + +/** What the socket does with a GROUP_PARTICIPANT_* notification: processMessage over the stub, buffered. */ +async function stub(service: any, ev: any, stubType: number, participants: Record[]) { + const message = { + key: { remoteJid: GROUP, fromMe: false, id: `STUB${stubType}`, participant: ADMIN.lid, participantAlt: ADMIN.pn }, + messageStubType: stubType, + messageStubParameters: participants.map((p) => JSON.stringify(p)), + messageTimestamp: 1_700_000_000, + }; + const ctx = { + shouldProcessHistoryMsg: false, + ev, + logger: P({ level: 'silent' }), + options: {}, + placeholderResendCache: undefined, + getMessage: async () => undefined, + creds: { me: { id: WUID, lid: '999999999999999@lid', name: 'Me' }, processedHistoryMessages: [] }, + keyStore: undefined, + signalRepository: service.client.signalRepository, + }; + ev.buffer(); + await processMessage(message, ctx); + await ev.flush(); + await settle(service); +} + +/** Every GROUP_PARTICIPANTS_UPDATE payload, as a webhook consumer receives it (JSON). */ +const updates = () => + emitted.filter((e) => e.event === 'group-participants.update').map((e) => JSON.parse(JSON.stringify(e.data))); + +/** A participant as the webhook carries it inside `participants`: Baileys' object, as JSON. */ +const asJson = (p: any) => JSON.parse(JSON.stringify(p)); + +describe('group participant updates carry each participant jid and phone number', () => { + beforeEach(async () => { + emitted.splice(0); + ({ extractGroupMetadata } = await import('baileys/lib/Socket/groups.js' as any)); + ({ default: processMessage } = await import('baileys/lib/Utils/process-message.js' as any)); + }); + + it('add: the phone comes from the event, else from the group metadata; name and picture from the contact', async () => { + const { service, prisma, ev } = await groupService([1, 2, 3]); + prisma.contact.rows.push({ + remoteJid: lid(2), + pushName: 'Member Two', + profilePicUrl: 'https://pps.example/2', + instanceId: 'inst-1', + }); + const added = [lidMember(2, true), lidMember(3, false)]; + + await stub(service, ev, WAMessageStubType.GROUP_PARTICIPANT_ADD, added); + + expect(updates()).toEqual([ + { + id: GROUP, + author: ADMIN.lid, + authorPn: ADMIN.pn, + action: 'add', + participants: added.map(asJson), + participantsData: [ + { jid: lid(2), phoneNumber: pn(2), name: 'Member Two', imgUrl: 'https://pps.example/2' }, + { jid: lid(3), phoneNumber: pn(3) }, + ], + }, + ]); + }); + + it('remove: a participant no longer in the group gets its phone from the LID mapping store, or none', async () => { + const { service, ev, repo } = await groupService([1]); + await repo.lidMapping.storeLIDPNMappings([{ lid: lid(4), pn: pn(4) }]); + const removed = [lidMember(4, false), lidMember(5, false)]; + + await stub(service, ev, WAMessageStubType.GROUP_PARTICIPANT_REMOVE, removed); + + expect(updates()).toEqual([ + { + id: GROUP, + author: ADMIN.lid, + authorPn: ADMIN.pn, + action: 'remove', + participants: removed.map(asJson), + participantsData: [{ jid: lid(4), phoneNumber: pn(4) }, { jid: lid(5) }], + }, + ]); + }); + + it('promote: a participant addressed by phone number is its own phone number', async () => { + const { service, ev } = await groupService([1, 6]); + const promoted = [pnMember(6, 'admin')]; + + await stub(service, ev, WAMessageStubType.GROUP_PARTICIPANT_PROMOTE, promoted); + + expect(updates()).toEqual([ + { + id: GROUP, + author: ADMIN.lid, + authorPn: ADMIN.pn, + action: 'promote', + participants: promoted.map(asJson), + participantsData: [{ jid: pn(6), phoneNumber: pn(6) }], + }, + ]); + }); + + it('legacy string participants (Baileys 6) still resolve, and stay strings', async () => { + const { service, prisma, ev } = await groupService([1, 7]); + prisma.contact.rows.push({ + remoteJid: lid(7), + pushName: 'Member Seven', + profilePicUrl: null, + instanceId: 'inst-1', + }); + // Hand-written: no Baileys version the fork runs emits string participants. + const legacy = { id: GROUP, author: ADMIN.lid, participants: [lid(7), pn(8)], action: 'add' }; + + await deliver(service, ev, { 'group-participants.update': legacy }); + + expect(updates()).toEqual([ + { + ...legacy, + participantsData: [ + { jid: lid(7), phoneNumber: pn(7), name: 'Member Seven', imgUrl: null }, + { jid: pn(8), phoneNumber: pn(8) }, + ], + }, + ]); + }); + + it('prints no participant number or name', async () => { + const { service, prisma, ev, repo } = await groupService([1, 2]); + prisma.contact.rows.push({ remoteJid: lid(2), pushName: 'Member Two', profilePicUrl: null, instanceId: 'inst-1' }); + await repo.lidMapping.storeLIDPNMappings([{ lid: lid(4), pn: pn(4) }]); + + const out = await captureOutput(async () => { + await stub(service, ev, WAMessageStubType.GROUP_PARTICIPANT_ADD, [lidMember(2, true)]); + await stub(service, ev, WAMessageStubType.GROUP_PARTICIPANT_REMOVE, [lidMember(4, false)]); + }); + + expect(updates().flatMap((u) => u.participantsData.map((d: any) => d.phoneNumber))).toEqual([pn(2), pn(4)]); + for (const secret of ['0007770', '1000000000077', 'Member Two']) expect(out).not.toContain(secret); + }); +}); From 8fd900409bc3dd86becbd2121634d964264c2371 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:18:23 +0300 Subject: [PATCH 056/157] fix(baileys): read Baileys 7 participant objects in group participant updates Baileys 7 (rc9+) emits group-participants.update with participants as GroupParticipant objects. The handler read them as strings, so every participantsData item carried the whole object as jid and "[object Object]" as phoneNumber, and never found the participant's name or picture. Each item is now { jid, phoneNumber?, name?, imgUrl? }: jid is the participant's id; phoneNumber is a phone jid taken from the event, else the group metadata, else the participant's own id when it is a phone jid, else the LID mapping store, and is left out when none knows it (a @lid's own digits are no longer passed off as a phone number). participants is sent exactly as Baileys emitted it, and Baileys 6 string participants resolve the same way. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 58 ++++++++++--------- 1 file changed, 32 insertions(+), 26 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index bf02bd14f3..4beac9d206 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -114,6 +114,7 @@ import makeWASocket, { getContentType, getDevice, GroupMetadata, + GroupParticipant, isJidBroadcast, isJidGroup, isJidNewsletter, @@ -2140,19 +2141,18 @@ export class BaileysStartupService extends ChannelStartupService { 'group-participants.update': async (participantsUpdate: { id: string; - participants: string[]; + participants: (GroupParticipant | string)[]; action: ParticipantAction; }) => { // ENHANCEMENT: Adds participantsData field while maintaining backward compatibility - // MAINTAINS: participants: string[] (original JID strings) - // ADDS: participantsData: { jid: string, phoneNumber: string, name?: string, imgUrl?: string }[] + // MAINTAINS: participants, exactly as Baileys emitted it + // ADDS: participantsData: { jid: string, phoneNumber?: string, name?: string, imgUrl?: string }[] // This enables LID to phoneNumber conversion without breaking existing webhook consumers - - // Helper to normalize participantId as phone number - const normalizePhoneNumber = (id: string | null | undefined): string => { - // Remove @lid, @s.whatsapp.net suffixes and extract just the number part - return String(id || '').split('@')[0]; - }; + // + // Baileys 7 emits each participant as a GroupParticipant object ({ id, phoneNumber?, ... }); + // Baileys 6 emitted jid strings. phoneNumber is a phone jid (@s.whatsapp.net): the + // participant's own, else the group metadata's, else the LID mapping store's, else none. + const phoneJid = (jid: string | null | undefined) => (isPnUser(jid) ? jidNormalizedUser(jid) : undefined); try { // Usa o mesmo método que o endpoint /group/participants @@ -2164,28 +2164,34 @@ export class BaileysStartupService extends ChannelStartupService { } // Filtra apenas os participantes que estão no evento - const resolvedParticipants = participantsUpdate.participants.map((participantId) => { - const participantData = groupParticipants.participants.find((p) => p.id === participantId); - - let phoneNumber: string; - if (participantData?.phoneNumber) { - phoneNumber = participantData.phoneNumber; - } else { - phoneNumber = normalizePhoneNumber(participantId); - } + const resolvedParticipants = await Promise.all( + participantsUpdate.participants.map(async (participant) => { + const jid = typeof participant === 'string' ? participant : participant?.id; + const participantData = groupParticipants.participants.find((p) => p.id === jid); + + let phoneNumber = + phoneJid(typeof participant === 'string' ? undefined : participant?.phoneNumber) ?? + phoneJid(participantData?.phoneNumber) ?? + phoneJid(jid); + if (!phoneNumber && isLidUser(jid)) { + phoneNumber = phoneJid( + await this.client.signalRepository?.lidMapping?.getPNForLID(jid).catch((): undefined => undefined), + ); + } - return { - jid: participantId, - phoneNumber, - name: participantData?.name, - imgUrl: participantData?.imgUrl, - }; - }); + return { + jid, + phoneNumber, + name: participantData?.name, + imgUrl: participantData?.imgUrl, + }; + }), + ); // Mantém formato original + adiciona dados resolvidos const enhancedParticipantsUpdate = { ...participantsUpdate, - participants: participantsUpdate.participants, // Mantém array original de strings + participants: participantsUpdate.participants, // Mantém o array original, como o Baileys emitiu // Adiciona dados resolvidos em campo separado participantsData: resolvedParticipants, }; From 94d76b277a4eb4ee5802fcda5ab55d5c521aa9bc Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:05:44 +0300 Subject: [PATCH 057/157] test: chat updates and the history chat list carry the archive state Baileys emits chats.update with `archived` for an archive or unarchive app-state action, `pinned` for a pin action and `muteEndTime` for a mute action, and history conversations carry the same three fields. Evolution reduces every chats.update item to { remoteJid, instanceId } and every chats.set item to its name, so a consumer cannot tell an archived chat from any other. The same holds for a chats.upsert that the event buffer merged an archive action into. The test builds each input with Baileys' own processSyncAction and processHistoryMessage, and also checks that an update without archive information does not claim archived: false, and that stored chat rows keep the Chat table's shape. Co-Authored-By: Claude Opus 5.5 --- test/handlers/chat-state.test.ts | 116 +++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+) create mode 100644 test/handlers/chat-state.test.ts diff --git a/test/handlers/chat-state.test.ts b/test/handlers/chat-state.test.ts new file mode 100644 index 0000000000..106912b783 --- /dev/null +++ b/test/handlers/chat-state.test.ts @@ -0,0 +1,116 @@ +// Archiving, pinning and muting a chat reach Evolution from Baileys, and were +// dropped on the way out: every chats.update item was reduced to +// { remoteJid, instanceId }, and the history chat list to its name. A consumer +// that hides archived chats could not tell which ones were. +// +// Where the state comes from in Baileys 7.0.0-rc14: +// - app-state actions (lib/Utils/chat-utils.js processSyncAction): +// archiveChatAction emits chats.update { id, archived }, pinAction +// { id, pinned: | null }, muteAction { id, muteEndTime: | null }. +// Each action carries its own field only; an update that carries none (a +// read marker, say) says nothing about the archive state. +// - history (lib/Utils/history.js) passes each HistorySync Conversation through +// as a chat, with archived, pinned and muteEndTime set when WhatsApp sent them. +// - the event buffer (lib/Utils/event-buffer.js) folds a chats.update into a +// chats.upsert of the same batch, so an upsert can carry the state too. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { beforeEach, describe, expect, it } from 'vitest'; + +import { historyEvent, msg, syncActionEvents } from '../helpers/baileys-fixtures'; +import { deliver, makeService } from '../helpers/baileys-service'; +import { emitted } from '../helpers/fake-server-module'; + +const ALPHA = '972500000001@s.whatsapp.net'; +const BRAVO = '972500000002@s.whatsapp.net'; +const CHARLIE = '972500000003@s.whatsapp.net'; +const GROUP = '120363000000000001@g.us'; +const INITIAL_BOOTSTRAP = 0; + +const items = (event: string) => emitted.filter((e) => e.event === event).flatMap((e) => [].concat(e.data)); + +describe('chat updates and the history chat list carry the archive state', () => { + beforeEach(() => void emitted.splice(0)); + + it('an archive action says archived: true, an unarchive action archived: false', async () => { + const { service, ev } = await makeService(); + await deliver(service, ev, syncActionEvents(['archive', ALPHA], { archiveChatAction: { archived: true } })); + await deliver(service, ev, syncActionEvents(['archive', ALPHA], { archiveChatAction: { archived: false } })); + + expect(items('chats.update')).toEqual([ + { remoteJid: ALPHA, instanceId: 'inst-1', archived: true }, + { remoteJid: ALPHA, instanceId: 'inst-1', archived: false }, + ]); + }); + + it('pin and mute actions say pinned and muteEndTime, and null when cleared', async () => { + const { service, ev } = await makeService(); + await deliver(service, ev, syncActionEvents(['pin_v1', ALPHA], { pinAction: { pinned: true } })); + await deliver(service, ev, syncActionEvents(['pin_v1', ALPHA], { pinAction: { pinned: false } })); + await deliver(service, ev, syncActionEvents(['mute', BRAVO], { muteAction: { muted: true, muteEndTimestamp: 1_800_000_000_000 } })); + await deliver(service, ev, syncActionEvents(['mute', BRAVO], { muteAction: { muted: false } })); + + expect(items('chats.update')).toEqual([ + { remoteJid: ALPHA, instanceId: 'inst-1', pinned: 1_700_000_000 }, + { remoteJid: ALPHA, instanceId: 'inst-1', pinned: null }, + { remoteJid: BRAVO, instanceId: 'inst-1', muteEndTime: 1_800_000_000_000 }, + { remoteJid: BRAVO, instanceId: 'inst-1', muteEndTime: null }, + ]); + }); + + it('an update without archive information does not claim the chat is unarchived', async () => { + const { service, ev } = await makeService(); + await deliver(service, ev, syncActionEvents(['markChatAsRead', ALPHA], { markChatAsReadAction: { read: true } })); + + expect(items('chats.update')).toEqual([{ remoteJid: ALPHA, instanceId: 'inst-1' }]); + }); + + for (const profile of ['minimal', 'stored'] as const) { + it(`history: an archived conversation is archived on its chats.set item (${profile})`, async () => { + const { service, ev, prisma } = await makeService({ profile }); + await deliver(service, ev, { + 'messaging-history.set': historyEvent({ + syncType: INITIAL_BOOTSTRAP, + progress: 100, + conversations: [ + { id: ALPHA, name: 'Alpha', archived: true, messages: [msg(ALPHA, 'A1', 'hi')] }, + { id: BRAVO, name: 'Bravo', pinned: 1_700_000_100, muteEndTime: 1_800_000_000_000, messages: [msg(BRAVO, 'B1', 'hi')] }, + { id: CHARLIE, name: 'Charlie', messages: [msg(CHARLIE, 'C1', 'hi')] }, + ], + }), + }); + + expect(items('chats.set')).toEqual([ + { remoteJid: ALPHA, instanceId: 'inst-1', name: 'Alpha', archived: true }, + { remoteJid: BRAVO, instanceId: 'inst-1', name: 'Bravo', pinned: 1_700_000_100, muteEndTime: 1_800_000_000_000 }, + { remoteJid: CHARLIE, instanceId: 'inst-1', name: 'Charlie' }, + ]); + // The Chat table has no column for any of it: a stored row keeps its own shape. + const stored = prisma.chat.rows.map(({ id: _id, ...row }: any) => row); + expect(stored).toEqual( + profile === 'stored' + ? [ + { remoteJid: ALPHA, instanceId: 'inst-1', name: 'Alpha' }, + { remoteJid: BRAVO, instanceId: 'inst-1', name: 'Bravo' }, + { remoteJid: CHARLIE, instanceId: 'inst-1', name: 'Charlie' }, + ] + : [], + ); + }); + } + + it('an archive action folded into a chat upsert of the same batch reaches chats.upsert', async () => { + const { service, ev } = await makeService(); + // What the socket emits for a group it learns was created (lib/Socket/messages-recv.js). + await deliver(service, ev, { + 'chats.upsert': [{ id: GROUP, name: 'Group', conversationTimestamp: 1_700_000_000 }], + ...syncActionEvents(['archive', GROUP], { archiveChatAction: { archived: true } }), + }); + + expect(items('chats.upsert')).toEqual([ + { remoteJid: GROUP, instanceId: 'inst-1', name: 'Group', unreadMessages: 0, archived: true }, + ]); + }); +}); From 1e79cf1a15f3546abbdc7c9bd778382759d493e3 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:06:39 +0300 Subject: [PATCH 058/157] feat(baileys): forward archive state on chat updates and the history chat list chats.update, chats.set and chats.upsert items now carry `archived`, `pinned` and `muteEndTime` whenever the Baileys chat they come from has the field: an app-state action sets only its own field (null when a pin or a mute is cleared), and a history Conversation sets a field only when WhatsApp sent it. A chat without the field gives an item without it, so a read marker never reads as an unarchive. Long values from history are converted to numbers. Only the webhook items change. The rows written to the Chat table keep their shape, since the table has no column for any of the three. Co-Authored-By: Claude Opus 5.5 --- FORK.md | 1 + .../whatsapp/whatsapp.baileys.service.ts | 13 ++++++--- src/utils/chat-state.ts | 27 +++++++++++++++++++ 3 files changed, 38 insertions(+), 3 deletions(-) create mode 100644 src/utils/chat-state.ts diff --git a/FORK.md b/FORK.md index a25899531f..2ff55213ab 100644 --- a/FORK.md +++ b/FORK.md @@ -46,6 +46,7 @@ named where one exists. - Every `contacts.upsert` item says whether its name is the one the owner saved (`saved`), and only when that is certain. - Group updates reach subscriptions stored as `GROUP_UPDATE`, in all seven transports and in the global configurations (#2652). - Group metadata is filled from `groups.update` instead of queried again for every group on every listing. +- `chats.update`, `chats.set` and `chats.upsert` items carry the chat's archive, pin and mute state (`archived`, `pinned`, `muteEndTime`) when Baileys has it, and omit a field it does not have rather than guess. **Messages and privacy** - A `getMessage` miss or failed lookup answers nothing, so Baileys no longer relays an empty message on a retry and uses it up (#2705, and #2706 for groups; fixes also offered in #2728 and #2623). diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 4beac9d206..9a2a502350 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -81,6 +81,7 @@ import ffmpegPath from '@ffmpeg-installer/ffmpeg'; import { Boom } from '@hapi/boom'; import { createId as cuid } from '@paralleldrive/cuid2'; import { Instance, Message } from '@prisma/client'; +import { chatState } from '@utils/chat-state'; import { createJid } from '@utils/createJid'; import { fetchLatestWaWebVersion } from '@utils/fetchLatestWaWebVersion'; import { jidKind, makeBaileysLogger } from '@utils/log-privacy'; @@ -1128,7 +1129,11 @@ export class BaileysStartupService extends ChannelStartupService { unreadMessages: chat.unreadCount !== undefined ? chat.unreadCount : 0, })); - this.sendDataWebhook(Events.CHATS_UPSERT, chatsToInsert); + const stateOf = new Map(chats.map((chat) => [chat.id, chatState(chat)])); + this.sendDataWebhook( + Events.CHATS_UPSERT, + chatsToInsert.map((chat) => ({ ...chat, ...stateOf.get(chat.remoteJid) })), + ); if (chatsToInsert.length > 0) { if (this.configService.get('DATABASE').SAVE_DATA.CHATS) @@ -1144,7 +1149,7 @@ export class BaileysStartupService extends ChannelStartupService { >[], ) => { const chatsRaw = chats.map((chat) => { - return { remoteJid: chat.id, instanceId: this.instanceId }; + return { remoteJid: chat.id, instanceId: this.instanceId, ...chatState(chat) }; }); this.sendDataWebhook(Events.CHATS_UPDATE, chatsRaw); @@ -1359,6 +1364,7 @@ export class BaileysStartupService extends ChannelStartupService { } const chatsRaw: { remoteJid: string; instanceId: string; name?: string }[] = []; + const chatItems: Record[] = []; const chatsRepository = new Set( (await this.prismaRepository.chat.findMany({ where: { instanceId: this.instanceId } })).map( (chat) => chat.remoteJid, @@ -1371,9 +1377,10 @@ export class BaileysStartupService extends ChannelStartupService { } chatsRaw.push({ remoteJid: chat.id, instanceId: this.instanceId, name: chat.name }); + chatItems.push({ ...chatsRaw[chatsRaw.length - 1], ...chatState(chat) }); } - this.sendDataWebhook(Events.CHATS_SET, chatsRaw); + this.sendDataWebhook(Events.CHATS_SET, chatItems); if (this.configService.get('DATABASE').SAVE_DATA.HISTORIC) { await this.prismaRepository.chat.createMany({ data: chatsRaw, skipDuplicates: true }); diff --git a/src/utils/chat-state.ts b/src/utils/chat-state.ts new file mode 100644 index 0000000000..5373ac179a --- /dev/null +++ b/src/utils/chat-state.ts @@ -0,0 +1,27 @@ +import { toNumber } from 'baileys'; + +export type ChatState = { archived?: boolean; pinned?: number | null; muteEndTime?: number | null }; + +const FIELDS = ['archived', 'pinned', 'muteEndTime'] as const; + +/** + * The archive, pin and mute state a Baileys chat carries, for a webhook item. + * + * A field is included only when the chat itself carries it: an app-state action + * sets its own field (null when it clears a pin or a mute), and a history + * Conversation sets a field only when WhatsApp sent it (protobuf defaults sit on + * the prototype). A chat without the field says nothing about it, so the item + * omits it rather than claiming archived: false. + */ +export function chatState(chat: Record | undefined | null): ChatState { + const state: Record = {}; + if (!chat) return state; + for (const field of FIELDS) { + if (!Object.prototype.hasOwnProperty.call(chat, field)) continue; + const value = chat[field]; + if (value === undefined) continue; + if (value === null) state[field] = null; + else state[field] = field === 'archived' ? !!value : toNumber(value); + } + return state; +} From c029b7621a4931aaae0da33599dc667d4ce7ad5b Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:10:56 +0300 Subject: [PATCH 059/157] test: a media download says whether it asked the phone to re-upload getBase64FromMediaMessage hands Baileys' downloadMediaMessage a reuploadRequest for a file that has expired on the CDN, and records nothing about it: a failed download cannot say whether the phone was asked and did not answer, or was never asked. The test downloads an image whose CDN copy is gone (a local CDN answering 404) through the real Baileys, with a socket whose updateMediaMessage either re-uploads or fails the way Baileys reports a phone that no longer has the file. It asserts the API error and a bounded log line, and that neither the sender's number nor the caption reaches the output. Baileys 7.0.0-rc14 never asks for the re-upload on a CDN 404 or 410: its downloadMediaMessage checks a numeric `status` on the error, and its CDN fetch throws a Boom that carries the status only in `output.statusCode`. The file probes the Baileys under test for that and runs the case that applies, so with rc14 the "not requested" case runs and the two re-upload cases are skipped; under a Baileys that asks (BAILEYS_DIR), the reverse. Co-Authored-By: Claude Opus 5.5 --- test/proxy/media-reupload.test.ts | 171 ++++++++++++++++++++++++++++++ 1 file changed, 171 insertions(+) create mode 100644 test/proxy/media-reupload.test.ts diff --git a/test/proxy/media-reupload.test.ts b/test/proxy/media-reupload.test.ts new file mode 100644 index 0000000000..c1e22e98a1 --- /dev/null +++ b/test/proxy/media-reupload.test.ts @@ -0,0 +1,171 @@ +// When a media file has expired on WhatsApp's CDN, the only copy left is on +// the sender's phone. Evolution hands Baileys' downloadMediaMessage a +// reuploadRequest (the socket's updateMediaMessage) for that case, and nothing +// recorded whether it was used or how it ended, so a failed download could not +// say whether the phone was asked and did not answer, or was never asked. +// +// Baileys 7.0.0-rc14 asks only when the download error has a numeric `status` +// of 404 or 410 (lib/Utils/messages.js downloadMediaMessage), but its CDN fetch +// throws a Boom that carries the HTTP status in `output.statusCode` and has no +// `status` (lib/Utils/messages-media.js getHttpStream). So rc14 never asks: an +// expired file fails without a re-upload request. BAILEYS_ASKS probes the build +// under test with a real 404, and the cases below run for the build that +// behaves each way (BAILEYS_DIR points the suite at another build). +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { readFile, rm } from 'node:fs/promises'; + +import { downloadMediaMessage, encryptedStream } from 'baileys'; +import { getGlobalDispatcher, MockAgent, setGlobalDispatcher } from 'undici'; +import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; + +import { makeService } from '../helpers/baileys-service'; +import { captureOutput } from '../helpers/capture-output'; +import { type Listening, startCdn } from '../helpers/local-net'; + +const PHONE = '972509876543'; +const CAPTION = 'Zq7 a private caption Zq7'; +const ID = '3EB0CCCCCCCCCCCCCCC1'; +const PLAIN = Buffer.from('a photo, as the person sent it '.repeat(200)); +const LIVE = '/v/t62.7118-24/reuploaded.enc'; +const GONE = '/v/t62.7118-24/expired.enc'; + +/** Whether the Baileys under test asks for a re-upload when the CDN answers 404. */ +const BAILEYS_ASKS = await (async () => { + const cdn = await startCdn({}); + let asked = false; + const expired = { key: { id: 'PROBE' }, message: { imageMessage: { url: `http://127.0.0.1:${cdn.port}${GONE}`, mediaKey: new Uint8Array(32) } } }; + const silent: any = { info: () => undefined, debug: () => undefined, trace: () => undefined, warn: () => undefined, error: () => undefined }; + await (downloadMediaMessage as any)(expired, 'buffer', {}, { logger: silent, reuploadRequest: async () => ((asked = true), Promise.reject(new Error('probe'))) }).catch(() => undefined); + await cdn.close(); + return asked; +})(); + +let cdn: Listening; +let mediaKey: Uint8Array; + +// Refuse any connection that is not to 127.0.0.1: Evolution's own fallback +// retries the download at mmg.whatsapp.net, which must fail here, not leave. +const previousDispatcher = getGlobalDispatcher(); +const guard = new MockAgent(); +guard.disableNetConnect(); +guard.enableNetConnect((host: string) => host.startsWith('127.0.0.1:')); + +beforeAll(async () => { + setGlobalDispatcher(guard); + const enc = await encryptedStream(PLAIN, 'image', {}); + mediaKey = enc.mediaKey; + const body = await readFile(enc.encFilePath); + await rm(enc.encFilePath, { force: true }); + cdn = await startCdn({ [LIVE]: body }); + // Evolution waits 5s before its own fallback download; nothing here depends on the wait. + const realSetTimeout = globalThis.setTimeout; + vi.spyOn(globalThis, 'setTimeout').mockImplementation(((fn: any, ms?: number, ...args: any[]) => + realSetTimeout(fn, ms === 5000 ? 0 : ms, ...args)) as any); +}); + +afterAll(async () => { + vi.restoreAllMocks(); + await cdn.close(); + setGlobalDispatcher(previousDispatcher); + await guard.close(); +}); + +beforeEach(() => void cdn.log.splice(0)); + +const expiredImage = () => ({ + key: { remoteJid: `${PHONE}@s.whatsapp.net`, fromMe: false, id: ID }, + message: { + imageMessage: { + url: `http://127.0.0.1:${cdn.port}${GONE}`, + mediaKey, + mimetype: 'image/jpeg', + caption: CAPTION, + fileLength: PLAIN.length, + }, + }, + messageTimestamp: 1_700_000_000, + pushName: 'Sender', +}); + +/** A service whose socket answers a re-upload request the way `answer` says, counting the requests. */ +async function serviceWithPhone(answer: 'reuploads' | 'fails') { + const made = await makeService(); + const asked: string[] = []; + made.service.client.updateMediaMessage = async (message: any) => { + asked.push(message.key.id); + if (answer === 'fails') { + // What Baileys raises when the phone reports the file is gone (messages-send.js updateMediaMessage). + const { Boom } = await import('@hapi/boom'); + throw new Boom('Media re-upload failed by device (NOT_FOUND)', { statusCode: 404 }); + } + // What Baileys does on success: point the message at the new copy (by url + // alone here, since a directPath is fetched over https from the url's host). + message.message.imageMessage.url = `http://127.0.0.1:${cdn.port}${LIVE}`; + return message; + }; + return { ...made, asked }; +} + +/** Run a download, returning what it answered or threw and everything printed meanwhile. */ +async function download(service: any) { + let result: any; + let thrown: any; + const out = await captureOutput(async () => { + try { + result = await service.getBase64FromMediaMessage({ message: expiredImage() }); + } catch (e) { + thrown = e; + } + }); + const plain = out.replace(/\x1b\[[0-9;]*m/g, ''); + return { result, thrown, out: plain, lines: plain.split('\n').filter((l) => l.includes('media download:')) }; +} + +const line = (fields: string) => expect.stringContaining(`media download: message=${ID}, chat=user, ${fields}`); + +function expectNothingPrivate(out: string) { + expect(out).not.toContain(PHONE); + expect(out).not.toContain('Zq7'); +} + +describe('a media download says whether it asked the phone to re-upload', () => { + it.runIf(!BAILEYS_ASKS)('Baileys does not ask: the error and the log say no re-upload was requested', async () => { + const { service, asked } = await serviceWithPhone('reuploads'); + const { thrown, out, lines } = await download(service); + + expect(asked).toEqual([]); + expect(cdn.log).toEqual([`GET ${GONE}`]); + expect(thrown).toEqual({ status: 400, error: 'Bad Request', message: [expect.any(String)], reupload: 'not_requested' }); + expect(lines).toEqual([line('outcome=download_failed, status=404, reupload=not_requested')]); + expectNothingPrivate(out); + }); + + it.runIf(BAILEYS_ASKS)('the phone re-uploads: the download succeeds and the log says so', async () => { + const { service, asked } = await serviceWithPhone('reuploads'); + const { result, thrown, out, lines } = await download(service); + + expect(thrown).toBeUndefined(); + expect(Buffer.from(result.base64, 'base64').equals(PLAIN)).toBe(true); + expect(asked).toEqual([ID]); + expect(cdn.log).toEqual([`GET ${GONE}`, `GET ${LIVE}`]); + expect(lines).toEqual([line('outcome=reupload_requested'), line('outcome=reupload_ok')]); + expectNothingPrivate(out); + }); + + it.runIf(BAILEYS_ASKS)('the phone cannot re-upload: the error and the log say the re-upload failed', async () => { + const { service, asked } = await serviceWithPhone('fails'); + const { thrown, out, lines } = await download(service); + + expect(asked).toEqual([ID]); + expect(thrown).toEqual({ status: 400, error: 'Bad Request', message: [expect.any(String)], reupload: 'failed' }); + expect(lines).toEqual([ + line('outcome=reupload_requested'), + line('outcome=reupload_failed, error=Error, status=404'), + line('outcome=download_failed, status=404, reupload=failed'), + ]); + expectNothingPrivate(out); + }); +}); From 873d901850fa954acfc48bad8255b1bf1b7d72e6 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:12:06 +0300 Subject: [PATCH 060/157] feat(baileys): record whether a media download asked for a re-upload getBase64FromMediaMessage now wraps the reuploadRequest it hands Baileys, and logs one bounded line per step, with the message id and the kind of chat only (no JID, number or content): media download: message=, chat=, outcome=reupload_requested media download: message=, chat=, outcome=reupload_ok media download: message=, chat=, outcome=reupload_failed, error=, status= media download: message=, chat=, outcome=download_failed, status=, reupload= The 400 it throws once a download was attempted keeps its shape and adds `reupload`: not_requested, ok or failed. With Baileys 7.0.0-rc14 an expired file (CDN 404 or 410) is recorded as download_failed with reupload=not_requested, because rc14 never asks the phone (see the test). The two re-upload outcomes were checked against a Baileys build whose CDN error carries a numeric status. Co-Authored-By: Claude Opus 5.5 --- FORK.md | 1 + .../whatsapp/whatsapp.baileys.service.ts | 40 +++++++++++++++++-- 2 files changed, 38 insertions(+), 3 deletions(-) diff --git a/FORK.md b/FORK.md index 2ff55213ab..6c440c841c 100644 --- a/FORK.md +++ b/FORK.md @@ -51,6 +51,7 @@ named where one exists. **Messages and privacy** - A `getMessage` miss or failed lookup answers nothing, so Baileys no longer relays an empty message on a retry and uses it up (#2705, and #2706 for groups; fixes also offered in #2728 and #2623). - No message text, phone number, JID or push name reaches the logs at `LOG_LEVEL=ERROR,WARN`, including Baileys' own error logs. +- A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410, and the record says so. **Proxy** - Media downloads, media uploads and the WhatsApp Web version fetch leave through the instance's proxy (uploads failed outright on a proxied instance). diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 9a2a502350..d0239202b3 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -231,6 +231,12 @@ async function getVideoDuration(input: Buffer | string | Readable): Promise error?.output?.statusCode ?? error?.status ?? 'none'; + export class BaileysStartupService extends ChannelStartupService { private messageProcessor = new BaileysMessageProcessor(); @@ -4380,6 +4386,8 @@ export class BaileysStartupService extends ChannelStartupService { } public async getBase64FromMediaMessage(data: getBase64FromMediaMessageDto, getBuffer = false) { + // Set once a download is attempted: whether the phone was asked to re-upload an expired file. + let reupload: MediaReupload | undefined; try { const m = data?.message; const convertToMp4 = data?.convertToMp4 ?? false; @@ -4439,15 +4447,35 @@ export class BaileysStartupService extends ChannelStartupService { } let buffer: Buffer; + const media = `message=${msg?.key?.id}, chat=${jidKind(msg?.key?.remoteJid)}`; + reupload = 'not_requested'; + const reuploadRequest = async (message: WAMessage) => { + this.logger.warn(`media download: ${media}, outcome=reupload_requested`); + try { + const updated = await this.client.updateMediaMessage(message); + reupload = 'ok'; + this.logger.warn(`media download: ${media}, outcome=reupload_ok`); + return updated; + } catch (error) { + reupload = 'failed'; + this.logger.warn( + `media download: ${media}, outcome=reupload_failed, error=${error?.name ?? 'unknown'}, status=${httpStatus(error)}`, + ); + throw error; + } + }; try { buffer = await downloadMediaMessage( { key: msg?.key, message: msg?.message }, 'buffer', this.mediaDownloadOptions(), - { logger: makeBaileysLogger('error') as any, reuploadRequest: this.client.updateMediaMessage }, + { logger: makeBaileysLogger('error') as any, reuploadRequest }, + ); + } catch (error) { + this.logger.error( + `media download: ${media}, outcome=download_failed, status=${httpStatus(error)}, reupload=${reupload}`, ); - } catch { this.logger.error('Download Media failed, trying to retry in 5 seconds...'); await new Promise((resolve) => setTimeout(resolve, 5000)); const mediaType = Object.keys(msg.message).find((key) => key.endsWith('Message')); @@ -4519,7 +4547,13 @@ export class BaileysStartupService extends ChannelStartupService { } catch (error) { this.logger.error('Error processing media message:'); this.logger.error(error); - throw new BadRequestException(error.toString()); + if (reupload === undefined) throw new BadRequestException(error.toString()); + // The same 400, plus whether the phone was asked to re-upload the file. + try { + new BadRequestException(error.toString()); + } catch (badRequest) { + throw { ...badRequest, reupload }; + } } } From c7e7e3b1c46fe9baca0f2461b995c17cb35f5d77 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:14:07 +0300 Subject: [PATCH 061/157] test: a media download can skip asking the phone to re-upload POST /chat/getBase64FromMediaMessage has no way to download only what is still on WhatsApp's servers: every call hands Baileys a reuploadRequest, so under a Baileys that asks, an expired file makes Evolution ask the phone, and the answer never says whether it did. The test posts to the real chat router (its guards and validation, a new startChatApp helper) with an image whose CDN copy is gone, and specifies an optional boolean `reupload`: with reupload: false the phone is never asked, nothing falls back, and the 400 says the file is gone and no re-upload was attempted; a non-boolean is refused; a file still on the CDN downloads as before; and an omitted field behaves as today, which with Baileys 7.0.0-rc14 is Evolution's own fallback with `reupload: not_requested` in the answer (rc14 never asks the phone), and under a Baileys that asks is the phone being asked. Co-Authored-By: Claude Opus 5.5 --- test/chat/media-skip-reupload.test.ts | 183 ++++++++++++++++++++++++++ test/helpers/http-app.ts | 12 ++ 2 files changed, 195 insertions(+) create mode 100644 test/chat/media-skip-reupload.test.ts diff --git a/test/chat/media-skip-reupload.test.ts b/test/chat/media-skip-reupload.test.ts new file mode 100644 index 0000000000..d20435b910 --- /dev/null +++ b/test/chat/media-skip-reupload.test.ts @@ -0,0 +1,183 @@ +// POST /chat/getBase64FromMediaMessage takes an optional `reupload` (boolean, +// default true). A consumer that only wants what is still on WhatsApp's +// servers sends reupload: false: Evolution does not hand Baileys a +// reuploadRequest, so the phone is never asked, and a file that has expired on +// the CDN (404 or 410) fails at once, without Evolution's own 5s fallback, with +// an error that says the file is gone and no re-upload was attempted. +// Omitting the field keeps today's behaviour. +// +// Baileys 7.0.0-rc14 never asks for a re-upload on a CDN 404 or 410 (see +// test/proxy/media-reupload.test.ts), so with rc14 "today's behaviour" is the +// fallback and reupload: not_requested; under a Baileys that asks +// (BAILEYS_DIR), it is the phone being asked. BAILEYS_ASKS picks the case. +import { vi } from 'vitest'; + +const h = vi.hoisted(() => ({ waMonitor: undefined as any, chatController: undefined as any })); + +vi.mock('@api/server.module', async () => { + const fake = await import('../helpers/fake-server-module'); + return { + ...fake, + get waMonitor() { + return h.waMonitor; + }, + get chatController() { + return h.chatController; + }, + }; +}); + +import { readFile, rm } from 'node:fs/promises'; + +import { downloadMediaMessage, encryptedStream } from 'baileys'; +import { getGlobalDispatcher, MockAgent, setGlobalDispatcher } from 'undici'; +import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; + +import { makeService } from '../helpers/baileys-service'; +import { prismaRepository as prisma } from '../helpers/fake-server-module'; +import { startChatApp } from '../helpers/http-app'; +import { type Listening, startCdn } from '../helpers/local-net'; + +const TOKEN = 'instance-token'; +const ID = '3EB0DDDDDDDDDDDDDDD1'; +const PLAIN = Buffer.from('a photo, as the person sent it '.repeat(200)); +const LIVE = '/v/t62.7118-24/reuploaded.enc'; +const GONE = '/v/t62.7118-24/expired.enc'; +const GONE_MESSAGE = + "The media is no longer on WhatsApp's servers (HTTP 404), and no re-upload from the phone was attempted (reupload: false)"; + +/** Whether the Baileys under test asks for a re-upload when the CDN answers 404. */ +const BAILEYS_ASKS = await (async () => { + const cdn = await startCdn({}); + let asked = false; + const expired = { key: { id: 'PROBE' }, message: { imageMessage: { url: `http://127.0.0.1:${cdn.port}${GONE}`, mediaKey: new Uint8Array(32) } } }; + const silent: any = { info: () => undefined, debug: () => undefined, trace: () => undefined, warn: () => undefined, error: () => undefined }; + await (downloadMediaMessage as any)(expired, 'buffer', {}, { logger: silent, reuploadRequest: async () => ((asked = true), Promise.reject(new Error('probe'))) }).catch(() => undefined); + await cdn.close(); + return asked; +})(); + +let cdn: Listening; +let mediaKey: Uint8Array; +let app: Awaited>; +let asked: string[]; + +// Refuse any connection that is not to 127.0.0.1, before any lookup: Evolution's +// fallback retries the download at mmg.whatsapp.net, which must fail here, not leave. +const previousDispatcher = getGlobalDispatcher(); +const guard = new MockAgent(); +guard.disableNetConnect(); +guard.enableNetConnect((host: string) => host.startsWith('127.0.0.1:')); + +beforeAll(async () => { + setGlobalDispatcher(guard); + const enc = await encryptedStream(PLAIN, 'image', {}); + mediaKey = enc.mediaKey; + const body = await readFile(enc.encFilePath); + await rm(enc.encFilePath, { force: true }); + cdn = await startCdn({ [LIVE]: body }); + // Evolution waits 5s before its own fallback download; shortened, so a test that + // takes it is not slow, and one that skips it is told apart by what it answers. + const realSetTimeout = globalThis.setTimeout; + vi.spyOn(globalThis, 'setTimeout').mockImplementation(((fn: any, ms?: number, ...args: any[]) => + realSetTimeout(fn, ms === 5000 ? 0 : ms, ...args)) as any); + + await prisma.instance.create({ data: { id: 'inst-1', name: 'test', connectionStatus: 'open', token: TOKEN, integration: 'WHATSAPP-BAILEYS' } }); + const { service } = await makeService({ prisma }); + asked = []; + service.client.updateMediaMessage = async (message: any) => { + asked.push(message.key.id); + // What Baileys does when the phone re-uploads: point the message at the new copy. + message.message.imageMessage.url = `http://127.0.0.1:${cdn.port}${LIVE}`; + return message; + }; + h.waMonitor = { waInstances: { test: service } }; + const { ChatController } = await import('@api/controllers/chat.controller'); + h.chatController = new ChatController(h.waMonitor); + app = await startChatApp(); +}); + +afterAll(async () => { + vi.restoreAllMocks(); + await app.close(); + await cdn.close(); + setGlobalDispatcher(previousDispatcher); + await guard.close(); +}); + +beforeEach(() => { + cdn.log.splice(0); + asked.length = 0; +}); + +/** A stored image message as a consumer holds it: JSON, with the media key (a Uint8Array, as Baileys decodes it) an object of bytes. */ +const expiredImage = () => + JSON.parse( + JSON.stringify({ + key: { remoteJid: '972500000001@s.whatsapp.net', fromMe: false, id: ID }, + message: { imageMessage: { url: `http://127.0.0.1:${cdn.port}${GONE}`, mediaKey: Uint8Array.from(mediaKey), mimetype: 'image/jpeg', fileLength: PLAIN.length } }, + messageTimestamp: 1_700_000_000, + }), + ); + +async function post(body: Record) { + const res = await fetch(`${app.base}/chat/getBase64FromMediaMessage/test`, { + method: 'POST', + headers: { 'content-type': 'application/json', apikey: TOKEN }, + body: JSON.stringify(body), + }); + return { status: res.status, body: await res.json() }; +} + +describe('a media download can skip asking the phone to re-upload', () => { + it('reupload: false and an expired file: the phone is not asked, and the error says the file is gone', async () => { + const answer = await post({ message: expiredImage(), reupload: false }); + + expect(asked).toEqual([]); + expect(cdn.log).toEqual([`GET ${GONE}`]); + expect(answer).toEqual({ + status: 400, + body: { status: 400, error: 'Bad Request', response: { message: [GONE_MESSAGE], reupload: 'not_requested' } }, + }); + }); + + it('reupload must be a boolean', async () => { + const answer = await post({ message: expiredImage(), reupload: 'no' }); + + expect(asked).toEqual([]); + expect(cdn.log).toEqual([]); + expect(answer).toEqual({ + status: 400, + // Evolution's validation error: the router wraps the list of schema errors in another list. + body: { status: 400, error: 'Bad Request', response: { message: [['reupload is not of a type(s) boolean']] } }, + }); + }); + + it('reupload: false does not stop a download of a file that is still there', async () => { + const message = expiredImage(); + message.message.imageMessage.url = `http://127.0.0.1:${cdn.port}${LIVE}`; + const answer = await post({ message, reupload: false }); + + expect(answer.status).toBe(201); + expect(Buffer.from(answer.body.base64, 'base64').equals(PLAIN)).toBe(true); + }); + + it.runIf(!BAILEYS_ASKS)('reupload omitted: as before, Evolution falls back to its own retry (rc14 never asks the phone)', async () => { + const answer = await post({ message: expiredImage() }); + + expect(asked).toEqual([]); + expect(answer).toEqual({ + status: 400, + body: { status: 400, error: 'Bad Request', response: { message: ['TypeError: fetch failed'], reupload: 'not_requested' } }, + }); + }); + + it.runIf(BAILEYS_ASKS)('reupload omitted: as before, the phone is asked and the download succeeds', async () => { + const answer = await post({ message: expiredImage() }); + + expect(asked).toEqual([ID]); + expect(cdn.log).toEqual([`GET ${GONE}`, `GET ${LIVE}`]); + expect(answer.status).toBe(201); + expect(Buffer.from(answer.body.base64, 'base64').equals(PLAIN)).toBe(true); + }); +}); diff --git a/test/helpers/http-app.ts b/test/helpers/http-app.ts index a6c17a507e..605ada27cc 100644 --- a/test/helpers/http-app.ts +++ b/test/helpers/http-app.ts @@ -40,3 +40,15 @@ export async function startInstanceApp() { const guards = [instanceExistsGuard, instanceLoggedGuard, authGuard['apikey']]; return serve('/instance', new InstanceRouter(configService, ...guards).router); } + +/** + * Only /chat, behind the same guards, for a test that fakes the server module and + * hands the router its own chatController (whose monitor holds the service). + */ +export async function startChatApp() { + await import('@api/routes/index.router'); + const { ChatRouter } = await import('@api/routes/chat.router'); + const { authGuard } = await import('@api/guards/auth.guard'); + const { instanceExistsGuard, instanceLoggedGuard } = await import('@api/guards/instance.guard'); + return serve('/chat', new ChatRouter(instanceExistsGuard, instanceLoggedGuard, authGuard['apikey']).router); +} From c6860faa1efce0c885ef7b79d2880bd01cc773e7 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:15:38 +0300 Subject: [PATCH 062/157] feat(chat): let a media download skip the phone re-upload POST /chat/getBase64FromMediaMessage takes an optional boolean `reupload` (DTO, a JSON schema that validates only that field, so existing bodies pass as before). With reupload: false the service calls Baileys' downloadMediaMessage without a reuploadRequest, so the phone is never asked, and a CDN 404 or 410 ends the download at once, without Evolution's 5s fallback, as a 400 whose message says the file is no longer on WhatsApp's servers and no re-upload was attempted, with reupload: not_requested. Any other failure keeps the fallback. Omitting the field changes nothing. The HTTP error handler in main.ts now passes `reupload` through to `response` when the error carries it (pair 2 added it to the thrown 400, and the handler kept only `message`); the test helper's copy of that handler follows. The controller already hands the whole body to the service. Co-Authored-By: Claude Opus 5.5 --- FORK.md | 1 + src/api/dto/chat.dto.ts | 2 ++ .../channel/whatsapp/whatsapp.baileys.service.ts | 12 ++++++++---- src/api/routes/chat.router.ts | 3 ++- src/main.ts | 2 ++ src/validate/chat.schema.ts | 8 ++++++++ test/helpers/http-app.ts | 6 +++++- 7 files changed, 28 insertions(+), 6 deletions(-) diff --git a/FORK.md b/FORK.md index 6c440c841c..f7f32a1717 100644 --- a/FORK.md +++ b/FORK.md @@ -52,6 +52,7 @@ named where one exists. - A `getMessage` miss or failed lookup answers nothing, so Baileys no longer relays an empty message on a retry and uses it up (#2705, and #2706 for groups; fixes also offered in #2728 and #2623). - No message text, phone number, JID or push name reaches the logs at `LOG_LEVEL=ERROR,WARN`, including Baileys' own error logs. - A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410, and the record says so. +- `POST /chat/getBase64FromMediaMessage` takes an optional `reupload` (boolean, default true). With `false` the phone is never asked to re-upload, and a file gone from the CDN fails at once with a 400 that says so; the HTTP error carries `reupload` for any failed download. **Proxy** - Media downloads, media uploads and the WhatsApp Web version fetch leave through the instance's proxy (uploads failed outright on a proxied instance). diff --git a/src/api/dto/chat.dto.ts b/src/api/dto/chat.dto.ts index b11f32b054..bc5558523c 100644 --- a/src/api/dto/chat.dto.ts +++ b/src/api/dto/chat.dto.ts @@ -20,6 +20,8 @@ export class OnWhatsAppDto { export class getBase64FromMediaMessageDto { message: proto.WebMessageInfo; convertToMp4?: boolean; + /** false: never ask the phone to re-upload a file that has expired on the CDN (default true). */ + reupload?: boolean; } export class WhatsAppNumberDto { diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index d0239202b3..1a969b8589 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -4448,6 +4448,8 @@ export class BaileysStartupService extends ChannelStartupService { let buffer: Buffer; const media = `message=${msg?.key?.id}, chat=${jidKind(msg?.key?.remoteJid)}`; + // reupload: false downloads only what is still on WhatsApp's servers. + const askPhone = data?.reupload !== false; reupload = 'not_requested'; const reuploadRequest = async (message: WAMessage) => { this.logger.warn(`media download: ${media}, outcome=reupload_requested`); @@ -4470,12 +4472,14 @@ export class BaileysStartupService extends ChannelStartupService { { key: msg?.key, message: msg?.message }, 'buffer', this.mediaDownloadOptions(), - { logger: makeBaileysLogger('error') as any, reuploadRequest }, + askPhone ? { logger: makeBaileysLogger('error') as any, reuploadRequest } : undefined, ); } catch (error) { - this.logger.error( - `media download: ${media}, outcome=download_failed, status=${httpStatus(error)}, reupload=${reupload}`, - ); + const status = httpStatus(error); + this.logger.error(`media download: ${media}, outcome=download_failed, status=${status}, reupload=${reupload}`); + if (!askPhone && (status === 404 || status === 410)) { + throw `The media is no longer on WhatsApp's servers (HTTP ${status}), and no re-upload from the phone was attempted (reupload: false)`; + } this.logger.error('Download Media failed, trying to retry in 5 seconds...'); await new Promise((resolve) => setTimeout(resolve, 5000)); const mediaType = Object.keys(msg.message).find((key) => key.endsWith('Message')); diff --git a/src/api/routes/chat.router.ts b/src/api/routes/chat.router.ts index 158947ed22..da13dff7fc 100644 --- a/src/api/routes/chat.router.ts +++ b/src/api/routes/chat.router.ts @@ -24,6 +24,7 @@ import { blockUserSchema, contactValidateSchema, deleteMessageSchema, + getBase64FromMediaMessageSchema, markChatUnreadSchema, messageUpSchema, messageValidateSchema, @@ -113,7 +114,7 @@ export class ChatRouter extends RouterBroker { .post(this.routerPath('getBase64FromMediaMessage'), ...guards, async (req, res) => { const response = await this.dataValidate({ request: req, - schema: null, + schema: getBase64FromMediaMessageSchema, ClassRef: getBase64FromMediaMessageDto, execute: (instance, data) => chatController.getBase64FromMediaMessage(instance, data), }); diff --git a/src/main.ts b/src/main.ts index f1f00ba9ae..4942a458e5 100644 --- a/src/main.ts +++ b/src/main.ts @@ -112,6 +112,8 @@ async function bootstrap() { error: err['error'] || 'Internal Server Error', response: { message: err['message'] || 'Internal Server Error', + // A failed media download says whether the phone was asked to re-upload the file. + ...(err['reupload'] !== undefined && { reupload: err['reupload'] }), }, }); } diff --git a/src/validate/chat.schema.ts b/src/validate/chat.schema.ts index 7dae44539b..89d708d7d8 100644 --- a/src/validate/chat.schema.ts +++ b/src/validate/chat.schema.ts @@ -131,6 +131,14 @@ export const deleteMessageSchema: JSONSchema7 = { ...isNotEmpty('id', 'remoteJid', 'participant'), }; +export const getBase64FromMediaMessageSchema: JSONSchema7 = { + $id: v4(), + type: 'object', + properties: { + reupload: { type: 'boolean' }, + }, +}; + export const profilePictureSchema: JSONSchema7 = { $id: v4(), type: 'object', diff --git a/test/helpers/http-app.ts b/test/helpers/http-app.ts index 605ada27cc..7dbcd47f9b 100644 --- a/test/helpers/http-app.ts +++ b/test/helpers/http-app.ts @@ -10,7 +10,11 @@ async function serve(mount: string, router: any) { app.use(express.json({ limit: '10mb' })); app.use(mount, router); app.use((err: any, _req: any, res: any, _next: any) => - res.status(err?.status || 500).json({ status: err?.status || 500, error: err?.error, response: { message: err?.message } }), + res.status(err?.status || 500).json({ + status: err?.status || 500, + error: err?.error, + response: { message: err?.message, ...(err?.reupload !== undefined && { reupload: err.reupload }) }, + }), ); const server = app.listen(0, '127.0.0.1'); await new Promise((r) => server.once('listening', r)); From 8c95840a4be92a031d694e4ee7c6b54754a99a67 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:17:57 +0300 Subject: [PATCH 063/157] test: an expired media download asks the phone to re-upload, once Baileys 7.0.0-rc14 never asks the phone to re-upload a file that has expired on the CDN: downloadMediaMessage checks a numeric `status` on the error (lib/Utils/messages.js:836), and its CDN fetch throws a Boom that carries the HTTP status only in `output.statusCode` (lib/Utils/messages-media.js:304). So on rc14 no download ever reaches the phone. The test specifies that a CDN 404 or 410 makes Evolution ask the phone once and retry the download once: a re-upload that succeeds downloads the new copy (for 404 and 410); a phone that reports the file gone, or does not answer within MEDIA_REUPLOAD_TIMEOUT_MS (60s, Baileys' own default query timeout, since updateMediaMessage itself waits forever), gives reupload: 'failed'; a re-uploaded copy that is gone as well is not re-uploaded again (reupload: 'ok', one request); and a failure that is not an expired file (a CDN 500) does not ask. The two pair-2 cases that only ran under a Baileys that asks now run on rc14, so the probe that chose between builds is gone. Its rc14-only case ("Baileys does not ask: no re-upload was requested") is replaced by the CDN 500 case, because an expired file is now re-uploaded by design; the same goes for the rc14-only "reupload omitted" case in test/chat/media-skip-reupload.test.ts, where omitting the field now asks the phone. startCdn can answer a path with a status. Co-Authored-By: Claude Opus 5.5 --- test/chat/media-skip-reupload.test.ts | 32 +------ test/helpers/local-net.ts | 4 +- test/proxy/media-reupload.test.ts | 133 +++++++++++++++++--------- 3 files changed, 93 insertions(+), 76 deletions(-) diff --git a/test/chat/media-skip-reupload.test.ts b/test/chat/media-skip-reupload.test.ts index d20435b910..0d0477d705 100644 --- a/test/chat/media-skip-reupload.test.ts +++ b/test/chat/media-skip-reupload.test.ts @@ -4,12 +4,7 @@ // reuploadRequest, so the phone is never asked, and a file that has expired on // the CDN (404 or 410) fails at once, without Evolution's own 5s fallback, with // an error that says the file is gone and no re-upload was attempted. -// Omitting the field keeps today's behaviour. -// -// Baileys 7.0.0-rc14 never asks for a re-upload on a CDN 404 or 410 (see -// test/proxy/media-reupload.test.ts), so with rc14 "today's behaviour" is the -// fallback and reupload: not_requested; under a Baileys that asks -// (BAILEYS_DIR), it is the phone being asked. BAILEYS_ASKS picks the case. +// Omitting the field keeps today's behaviour: the phone is asked. import { vi } from 'vitest'; const h = vi.hoisted(() => ({ waMonitor: undefined as any, chatController: undefined as any })); @@ -29,7 +24,7 @@ vi.mock('@api/server.module', async () => { import { readFile, rm } from 'node:fs/promises'; -import { downloadMediaMessage, encryptedStream } from 'baileys'; +import { encryptedStream } from 'baileys'; import { getGlobalDispatcher, MockAgent, setGlobalDispatcher } from 'undici'; import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; @@ -46,17 +41,6 @@ const GONE = '/v/t62.7118-24/expired.enc'; const GONE_MESSAGE = "The media is no longer on WhatsApp's servers (HTTP 404), and no re-upload from the phone was attempted (reupload: false)"; -/** Whether the Baileys under test asks for a re-upload when the CDN answers 404. */ -const BAILEYS_ASKS = await (async () => { - const cdn = await startCdn({}); - let asked = false; - const expired = { key: { id: 'PROBE' }, message: { imageMessage: { url: `http://127.0.0.1:${cdn.port}${GONE}`, mediaKey: new Uint8Array(32) } } }; - const silent: any = { info: () => undefined, debug: () => undefined, trace: () => undefined, warn: () => undefined, error: () => undefined }; - await (downloadMediaMessage as any)(expired, 'buffer', {}, { logger: silent, reuploadRequest: async () => ((asked = true), Promise.reject(new Error('probe'))) }).catch(() => undefined); - await cdn.close(); - return asked; -})(); - let cdn: Listening; let mediaKey: Uint8Array; let app: Awaited>; @@ -162,17 +146,7 @@ describe('a media download can skip asking the phone to re-upload', () => { expect(Buffer.from(answer.body.base64, 'base64').equals(PLAIN)).toBe(true); }); - it.runIf(!BAILEYS_ASKS)('reupload omitted: as before, Evolution falls back to its own retry (rc14 never asks the phone)', async () => { - const answer = await post({ message: expiredImage() }); - - expect(asked).toEqual([]); - expect(answer).toEqual({ - status: 400, - body: { status: 400, error: 'Bad Request', response: { message: ['TypeError: fetch failed'], reupload: 'not_requested' } }, - }); - }); - - it.runIf(BAILEYS_ASKS)('reupload omitted: as before, the phone is asked and the download succeeds', async () => { + it('reupload omitted: as before, the phone is asked and the download succeeds', async () => { const answer = await post({ message: expiredImage() }); expect(asked).toEqual([ID]); diff --git a/test/helpers/local-net.ts b/test/helpers/local-net.ts index 96345a26f5..4c6f1ae927 100644 --- a/test/helpers/local-net.ts +++ b/test/helpers/local-net.ts @@ -31,12 +31,14 @@ async function listen(server: net.Server): Promise { } /** Serves `files` (path -> bytes) and logs `METHOD path` for every request. */ -export async function startCdn(files: Record): Promise { +/** Serves `files` by path; a number instead of a body answers that status. Anything else is 404. */ +export async function startCdn(files: Record): Promise { const log: string[] = []; const server = http.createServer((req, res) => { log.push(`${req.method} ${req.url}`); const body = files[req.url ?? '']; if (!body) return void res.writeHead(404).end(); + if (typeof body === 'number') return void res.writeHead(body).end(); res.writeHead(200, { 'content-type': 'application/octet-stream', 'content-length': body.length }).end(body); }); const close = track(server); diff --git a/test/proxy/media-reupload.test.ts b/test/proxy/media-reupload.test.ts index c1e22e98a1..5537226b11 100644 --- a/test/proxy/media-reupload.test.ts +++ b/test/proxy/media-reupload.test.ts @@ -1,23 +1,22 @@ // When a media file has expired on WhatsApp's CDN, the only copy left is on -// the sender's phone. Evolution hands Baileys' downloadMediaMessage a -// reuploadRequest (the socket's updateMediaMessage) for that case, and nothing -// recorded whether it was used or how it ended, so a failed download could not -// say whether the phone was asked and did not answer, or was never asked. +// the sender's phone. Evolution asks the phone to re-upload it (the socket's +// updateMediaMessage), once per download, and records whether it asked and +// how that ended: a bounded log line, and `reupload` on the download's error. // -// Baileys 7.0.0-rc14 asks only when the download error has a numeric `status` -// of 404 or 410 (lib/Utils/messages.js downloadMediaMessage), but its CDN fetch -// throws a Boom that carries the HTTP status in `output.statusCode` and has no -// `status` (lib/Utils/messages-media.js getHttpStream). So rc14 never asks: an -// expired file fails without a re-upload request. BAILEYS_ASKS probes the build -// under test with a real 404, and the cases below run for the build that -// behaves each way (BAILEYS_DIR points the suite at another build). +// Baileys 7.0.0-rc14 is meant to ask by itself: downloadMediaMessage calls its +// reuploadRequest when the error has a numeric `status` of 404 or 410 +// (lib/Utils/messages.js:836). Its CDN fetch throws a Boom that carries the +// HTTP status only in `output.statusCode` (lib/Utils/messages-media.js:304), so +// that check never matches and rc14 never asks. Evolution asks itself when +// Baileys did not. import { vi } from 'vitest'; vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); import { readFile, rm } from 'node:fs/promises'; -import { downloadMediaMessage, encryptedStream } from 'baileys'; +import { MEDIA_REUPLOAD_TIMEOUT_MS } from '@api/integrations/channel/whatsapp/whatsapp.baileys.service'; +import { encryptedStream } from 'baileys'; import { getGlobalDispatcher, MockAgent, setGlobalDispatcher } from 'undici'; import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; @@ -31,17 +30,9 @@ const ID = '3EB0CCCCCCCCCCCCCCC1'; const PLAIN = Buffer.from('a photo, as the person sent it '.repeat(200)); const LIVE = '/v/t62.7118-24/reuploaded.enc'; const GONE = '/v/t62.7118-24/expired.enc'; - -/** Whether the Baileys under test asks for a re-upload when the CDN answers 404. */ -const BAILEYS_ASKS = await (async () => { - const cdn = await startCdn({}); - let asked = false; - const expired = { key: { id: 'PROBE' }, message: { imageMessage: { url: `http://127.0.0.1:${cdn.port}${GONE}`, mediaKey: new Uint8Array(32) } } }; - const silent: any = { info: () => undefined, debug: () => undefined, trace: () => undefined, warn: () => undefined, error: () => undefined }; - await (downloadMediaMessage as any)(expired, 'buffer', {}, { logger: silent, reuploadRequest: async () => ((asked = true), Promise.reject(new Error('probe'))) }).catch(() => undefined); - await cdn.close(); - return asked; -})(); +const GONE_AGAIN = '/v/t62.7118-24/expired-again.enc'; +const GONE_410 = '/v/t62.7118-24/expired-410.enc'; +const BROKEN = '/v/t62.7118-24/broken.enc'; let cdn: Listening; let mediaKey: Uint8Array; @@ -59,11 +50,12 @@ beforeAll(async () => { mediaKey = enc.mediaKey; const body = await readFile(enc.encFilePath); await rm(enc.encFilePath, { force: true }); - cdn = await startCdn({ [LIVE]: body }); - // Evolution waits 5s before its own fallback download; nothing here depends on the wait. + cdn = await startCdn({ [LIVE]: body, [GONE_410]: 410, [BROKEN]: 500 }); + // Evolution waits 5s before its own fallback download, and gives the phone a + // bounded time to answer a re-upload request; both are shortened here. const realSetTimeout = globalThis.setTimeout; vi.spyOn(globalThis, 'setTimeout').mockImplementation(((fn: any, ms?: number, ...args: any[]) => - realSetTimeout(fn, ms === 5000 ? 0 : ms, ...args)) as any); + realSetTimeout(fn, ms === 5000 ? 0 : MEDIA_REUPLOAD_TIMEOUT_MS && ms === MEDIA_REUPLOAD_TIMEOUT_MS ? 20 : ms, ...args)) as any); }); afterAll(async () => { @@ -75,11 +67,11 @@ afterAll(async () => { beforeEach(() => void cdn.log.splice(0)); -const expiredImage = () => ({ +const expiredImage = (path = GONE) => ({ key: { remoteJid: `${PHONE}@s.whatsapp.net`, fromMe: false, id: ID }, message: { imageMessage: { - url: `http://127.0.0.1:${cdn.port}${GONE}`, + url: `http://127.0.0.1:${cdn.port}${path}`, mediaKey, mimetype: 'image/jpeg', caption: CAPTION, @@ -90,32 +82,36 @@ const expiredImage = () => ({ pushName: 'Sender', }); -/** A service whose socket answers a re-upload request the way `answer` says, counting the requests. */ -async function serviceWithPhone(answer: 'reuploads' | 'fails') { +type Phone = 'reuploads' | 'fails' | 'reuploads-expired' | 'silent'; + +/** A service whose socket answers a re-upload request the way `phone` says, counting the requests. */ +async function serviceWithPhone(phone: Phone) { const made = await makeService(); const asked: string[] = []; made.service.client.updateMediaMessage = async (message: any) => { asked.push(message.key.id); - if (answer === 'fails') { + if (phone === 'silent') return new Promise(() => undefined); + if (phone === 'fails') { // What Baileys raises when the phone reports the file is gone (messages-send.js updateMediaMessage). const { Boom } = await import('@hapi/boom'); throw new Boom('Media re-upload failed by device (NOT_FOUND)', { statusCode: 404 }); } // What Baileys does on success: point the message at the new copy (by url // alone here, since a directPath is fetched over https from the url's host). - message.message.imageMessage.url = `http://127.0.0.1:${cdn.port}${LIVE}`; + const path = phone === 'reuploads' ? LIVE : GONE_AGAIN; + message.message.imageMessage.url = `http://127.0.0.1:${cdn.port}${path}`; return message; }; return { ...made, asked }; } /** Run a download, returning what it answered or threw and everything printed meanwhile. */ -async function download(service: any) { +async function download(service: any, path = GONE) { let result: any; let thrown: any; const out = await captureOutput(async () => { try { - result = await service.getBase64FromMediaMessage({ message: expiredImage() }); + result = await service.getBase64FromMediaMessage({ message: expiredImage(path) }); } catch (e) { thrown = e; } @@ -125,6 +121,7 @@ async function download(service: any) { } const line = (fields: string) => expect.stringContaining(`media download: message=${ID}, chat=user, ${fields}`); +const badRequest = (reupload: string) => ({ status: 400, error: 'Bad Request', message: [expect.any(String)], reupload }); function expectNothingPrivate(out: string) { expect(out).not.toContain(PHONE); @@ -132,40 +129,84 @@ function expectNothingPrivate(out: string) { } describe('a media download says whether it asked the phone to re-upload', () => { - it.runIf(!BAILEYS_ASKS)('Baileys does not ask: the error and the log say no re-upload was requested', async () => { + it('the phone re-uploads: the download succeeds and the log says so', async () => { const { service, asked } = await serviceWithPhone('reuploads'); + const { result, thrown, out, lines } = await download(service); + + expect(thrown).toBeUndefined(); + expect(Buffer.from(result.base64, 'base64').equals(PLAIN)).toBe(true); + expect(asked).toEqual([ID]); + expect(cdn.log).toEqual([`GET ${GONE}`, `GET ${LIVE}`]); + expect(lines).toEqual([line('outcome=reupload_requested'), line('outcome=reupload_ok')]); + expectNothingPrivate(out); + }); + + it('the phone cannot re-upload: the error and the log say the re-upload failed', async () => { + const { service, asked } = await serviceWithPhone('fails'); const { thrown, out, lines } = await download(service); - expect(asked).toEqual([]); - expect(cdn.log).toEqual([`GET ${GONE}`]); - expect(thrown).toEqual({ status: 400, error: 'Bad Request', message: [expect.any(String)], reupload: 'not_requested' }); - expect(lines).toEqual([line('outcome=download_failed, status=404, reupload=not_requested')]); + expect(asked).toEqual([ID]); + expect(thrown).toEqual(badRequest('failed')); + expect(lines).toEqual([ + line('outcome=reupload_requested'), + line('outcome=reupload_failed, error=Error, status=404'), + line('outcome=download_failed, status=404, reupload=failed'), + ]); expectNothingPrivate(out); }); +}); - it.runIf(BAILEYS_ASKS)('the phone re-uploads: the download succeeds and the log says so', async () => { +describe('an expired media download asks the phone to re-upload, once', () => { + it('a CDN 410 asks the phone too', async () => { const { service, asked } = await serviceWithPhone('reuploads'); - const { result, thrown, out, lines } = await download(service); + const { result, thrown, lines } = await download(service, GONE_410); expect(thrown).toBeUndefined(); expect(Buffer.from(result.base64, 'base64').equals(PLAIN)).toBe(true); expect(asked).toEqual([ID]); - expect(cdn.log).toEqual([`GET ${GONE}`, `GET ${LIVE}`]); + expect(cdn.log).toEqual([`GET ${GONE_410}`, `GET ${LIVE}`]); expect(lines).toEqual([line('outcome=reupload_requested'), line('outcome=reupload_ok')]); + }); + + it('a re-uploaded copy that is gone as well is not re-uploaded again', async () => { + const { service, asked } = await serviceWithPhone('reuploads-expired'); + const { thrown, out, lines } = await download(service); + + expect(asked).toEqual([ID]); + expect(cdn.log).toEqual([`GET ${GONE}`, `GET ${GONE_AGAIN}`]); + expect(thrown).toEqual(badRequest('ok')); + expect(lines).toEqual([ + line('outcome=reupload_requested'), + line('outcome=reupload_ok'), + line('outcome=download_failed, status=404, reupload=ok'), + ]); expectNothingPrivate(out); }); - it.runIf(BAILEYS_ASKS)('the phone cannot re-upload: the error and the log say the re-upload failed', async () => { - const { service, asked } = await serviceWithPhone('fails'); + it('a phone that does not answer in time fails the re-upload', async () => { + const { service, asked } = await serviceWithPhone('silent'); const { thrown, out, lines } = await download(service); + expect(MEDIA_REUPLOAD_TIMEOUT_MS).toBe(60_000); expect(asked).toEqual([ID]); - expect(thrown).toEqual({ status: 400, error: 'Bad Request', message: [expect.any(String)], reupload: 'failed' }); + expect(cdn.log).toEqual([`GET ${GONE}`]); + expect(thrown).toEqual(badRequest('failed')); expect(lines).toEqual([ line('outcome=reupload_requested'), - line('outcome=reupload_failed, error=Error, status=404'), + line('outcome=reupload_failed, error=ReuploadTimeoutError, status=none'), line('outcome=download_failed, status=404, reupload=failed'), ]); expectNothingPrivate(out); }); + + it('a failure that is not an expired file (a CDN 500) does not ask the phone', async () => { + const { service, asked } = await serviceWithPhone('reuploads'); + const { thrown, out, lines } = await download(service, BROKEN); + + expect(asked).toEqual([]); + expect(cdn.log).toEqual([`GET ${BROKEN}`]); + expect(thrown).toEqual(badRequest('not_requested')); + expect(lines).toEqual([line('outcome=download_failed, status=500, reupload=not_requested')]); + expectNothingPrivate(out); + }); }); From 48bb6f0eb8c50b66cc4f0b21b3f2dc938f503baa Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:19:42 +0300 Subject: [PATCH 064/157] fix(baileys): re-upload expired media when Baileys' own check misses it Baileys 7.0.0-rc14 never asks the phone to re-upload an expired file. downloadMediaMessage requests the re-upload only when the error has a numeric `status` of 404 or 410 (lib/Utils/messages.js:836), and its CDN fetch throws a Boom that carries the HTTP status only in `output.statusCode` (lib/Utils/messages-media.js:304), so `error.status` is undefined and the check never matches. This fix exists only because of that check; once Baileys reads the status from its own Boom, this code can go and the reuploadRequest can be handed back to downloadMediaMessage. getBase64FromMediaMessage now owns the re-upload. When the download fails with 404 or 410 (read from output.statusCode, then status) and reupload is not false, it calls this.client.updateMediaMessage once and retries the download once with the refreshed message. Baileys is no longer given a reuploadRequest, so a Baileys that does ask cannot add a second request. The phone gets MEDIA_REUPLOAD_TIMEOUT_MS (60s, Baileys' default query timeout) to answer, since updateMediaMessage waits with no timeout of its own; past that the re-upload fails as ReuploadTimeoutError. The outcome goes to the pair-2 log lines and the error's `reupload` (ok or failed). With reupload: false nothing changes: the phone is never asked. When the re-upload fails, the download's own error (and its status) is what is reported. A retried download that fails still takes Evolution's existing fallback. Co-Authored-By: Claude Opus 5.5 --- FORK.md | 2 +- .../whatsapp/whatsapp.baileys.service.ts | 51 ++++++++++++++++--- 2 files changed, 44 insertions(+), 9 deletions(-) diff --git a/FORK.md b/FORK.md index f7f32a1717..a53b99b6c6 100644 --- a/FORK.md +++ b/FORK.md @@ -51,7 +51,7 @@ named where one exists. **Messages and privacy** - A `getMessage` miss or failed lookup answers nothing, so Baileys no longer relays an empty message on a retry and uses it up (#2705, and #2706 for groups; fixes also offered in #2728 and #2623). - No message text, phone number, JID or push name reaches the logs at `LOG_LEVEL=ERROR,WARN`, including Baileys' own error logs. -- A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410, and the record says so. +- A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410 (its check misses the status), so Evolution asks the phone itself, once per download, for at most 60s. - `POST /chat/getBase64FromMediaMessage` takes an optional `reupload` (boolean, default true). With `false` the phone is never asked to re-upload, and a file gone from the CDN fails at once with a 400 that says so; the HTTP error carries `reupload` for any failed download. **Proxy** diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 1a969b8589..e220c7bb8b 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -237,6 +237,16 @@ type MediaReupload = 'not_requested' | 'ok' | 'failed'; /** The HTTP status a Baileys media error carries (a Boom's output.statusCode), or 'none'. */ const httpStatus = (error: any) => error?.output?.statusCode ?? error?.status ?? 'none'; +/** A CDN answer that means the file has expired there, and only the phone still has it. */ +const isExpiredMedia = (error: any) => [404, 410].includes(httpStatus(error)); + +/** + * How long the phone gets to answer a re-upload request. Baileys' updateMediaMessage + * waits for the answer with no timeout of its own, so this is its default query + * timeout (defaultQueryTimeoutMs). + */ +export const MEDIA_REUPLOAD_TIMEOUT_MS = 60_000; + export class BaileysStartupService extends ChannelStartupService { private messageProcessor = new BaileysMessageProcessor(); @@ -4451,10 +4461,18 @@ export class BaileysStartupService extends ChannelStartupService { // reupload: false downloads only what is still on WhatsApp's servers. const askPhone = data?.reupload !== false; reupload = 'not_requested'; - const reuploadRequest = async (message: WAMessage) => { + // Asks the phone for a new copy, for a bounded time. Called at most once per download. + const reuploadRequest = async (message: WAMessage): Promise => { this.logger.warn(`media download: ${media}, outcome=reupload_requested`); + let timer: NodeJS.Timeout; + const timeout = new Promise((_, reject) => { + timer = setTimeout(() => { + const error = new Error(`the phone did not answer the re-upload request in ${MEDIA_REUPLOAD_TIMEOUT_MS}ms`); + reject(Object.assign(error, { name: 'ReuploadTimeoutError' })); + }, MEDIA_REUPLOAD_TIMEOUT_MS); + }); try { - const updated = await this.client.updateMediaMessage(message); + const updated = await Promise.race([this.client.updateMediaMessage(message), timeout]); reupload = 'ok'; this.logger.warn(`media download: ${media}, outcome=reupload_ok`); return updated; @@ -4464,16 +4482,33 @@ export class BaileysStartupService extends ChannelStartupService { `media download: ${media}, outcome=reupload_failed, error=${error?.name ?? 'unknown'}, status=${httpStatus(error)}`, ); throw error; + } finally { + clearTimeout(timer); } }; + const target: WAMessage = { key: msg?.key, message: msg?.message }; + // No reuploadRequest for Baileys: Evolution asks the phone itself, below. Baileys + // means to ask on a 404 or 410, but 7.0.0-rc14 checks error.status + // (lib/Utils/messages.js:836) while its CDN fetch sets only output.statusCode + // (lib/Utils/messages-media.js:304), so it never does. Handing it the request as + // well would let a Baileys that does ask make a second one. + const download = (message: WAMessage) => + downloadMediaMessage(message, 'buffer', this.mediaDownloadOptions()) as Promise; try { - buffer = await downloadMediaMessage( - { key: msg?.key, message: msg?.message }, - 'buffer', - this.mediaDownloadOptions(), - askPhone ? { logger: makeBaileysLogger('error') as any, reuploadRequest } : undefined, - ); + try { + buffer = await download(target); + } catch (error) { + if (!askPhone || !isExpiredMedia(error)) throw error; + let refreshed: WAMessage; + try { + refreshed = await reuploadRequest(target); + } catch { + // The download's own error stands; the log already says how the re-upload ended. + throw error; + } + buffer = await download(refreshed); + } } catch (error) { const status = httpStatus(error); this.logger.error(`media download: ${media}, outcome=download_failed, status=${status}, reupload=${reupload}`); From 8cb00492bbc29a34886f3e28ad469f5b244743cc Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:57:53 +0300 Subject: [PATCH 065/157] test: a message's webhook key keeps its original @lid address For a DM WhatsApp addresses by @lid, Baileys gives remoteJid = the @lid and remoteJidAlt = the phone JID. Evolution 2.3.7 copies remoteJidAlt over remoteJid before the messages.upsert webhook and keeps the alt, so both fields carry the phone and the @lid is gone. The phone keeps the message under the @lid, so a consumer that stores this key cannot name the message the way WhatsApp does (a media re-upload request for it is refused). Red: the DM case, under both profiles (remoteJidAlt is the phone, addressingMode still 'lid'). The group case and the phone-addressed DM are green already and are pinned as guards: Evolution does not rewrite a group participant, and a phone-addressed DM already carries its @lid in the alt. Keys are built by Baileys' own decodeMessageNode from a message stanza. Co-Authored-By: Claude Opus 5.5 --- test/handlers/lid-webhook-key.test.ts | 107 ++++++++++++++++++++++++++ 1 file changed, 107 insertions(+) create mode 100644 test/handlers/lid-webhook-key.test.ts diff --git a/test/handlers/lid-webhook-key.test.ts b/test/handlers/lid-webhook-key.test.ts new file mode 100644 index 0000000000..852c1a302e --- /dev/null +++ b/test/handlers/lid-webhook-key.test.ts @@ -0,0 +1,107 @@ +// WhatsApp can address a private chat by an @lid (a linked identity) instead of +// the person's phone number. Baileys then hands Evolution a key with the @lid in +// remoteJid and the phone JID in remoteJidAlt. Evolution shows the phone JID as +// remoteJid in the messages.upsert webhook (consumers have always keyed chats by +// phone), but the phone keeps the message under the @lid: a consumer that later +// asks for something about that message (a media re-upload) must be able to name +// it the way WhatsApp does. So the webhook key keeps the original @lid in +// remoteJidAlt, the swap upstream develop makes. A group message's participant +// is not rewritten, so it keeps its @lid and phone as Baileys gave them. +// +// The keys are built by Baileys' own decodeMessageNode from a message stanza, +// so they carry exactly the fields this Baileys version produces. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { decodeMessageNode } from 'baileys'; +import { beforeEach, describe, expect, it } from 'vitest'; + +import { deliver, makeService, WUID } from '../helpers/baileys-service'; +import { emitted } from '../helpers/fake-server-module'; +import type { Profile } from '../helpers/profiles'; + +const ME_LID = '100000000000001@lid'; +const LID = '123456789012345@lid'; +const PHONE = '972509876543@s.whatsapp.net'; +const GROUP = '120363000000000001@g.us'; + +/** What Baileys emits in messages.upsert for this stanza, with a text body. */ +function received(attrs: Record) { + const { fullMessage } = decodeMessageNode({ tag: 'message', attrs, content: [] } as any, WUID, ME_LID); + return { ...fullMessage, message: { conversation: 'hello' } }; +} + +const lidDm = () => + received({ id: '3EB0DDDDDDDDDDDDDDD1', from: LID, sender_pn: PHONE, addressing_mode: 'lid', type: 'text', t: '1700000000', notify: 'Sender' }); + +const lidGroup = () => + received({ + id: '3EB0DDDDDDDDDDDDDDD2', + from: GROUP, + participant: LID, + participant_pn: PHONE, + addressing_mode: 'lid', + type: 'text', + t: '1700000000', + notify: 'Sender', + }); + +async function webhookKey(profile: Profile, message: any) { + const { service, ev } = await makeService({ profile }); + await deliver(service, ev, { 'messages.upsert': { messages: [message], type: 'notify' } }); + const upserts = emitted.filter((e) => e.event === 'messages.upsert'); + expect(upserts).toHaveLength(1); + return upserts[0].data.key; +} + +describe.each(['minimal', 'stored'])("a message's webhook key keeps its original @lid address (%s)", (profile) => { + beforeEach(() => void emitted.splice(0)); + + it('a DM WhatsApp addresses by @lid: remoteJid is the phone, remoteJidAlt the original @lid', async () => { + const message = lidDm(); + // What Baileys hands Evolution. + expect(message.key).toEqual({ remoteJid: LID, remoteJidAlt: PHONE, fromMe: false, id: '3EB0DDDDDDDDDDDDDDD1', addressingMode: 'lid' }); + + expect(await webhookKey(profile, message)).toEqual({ + remoteJid: PHONE, + remoteJidAlt: LID, + fromMe: false, + id: '3EB0DDDDDDDDDDDDDDD1', + addressingMode: 'pn', + }); + }); + + it('a group message from an @lid participant keeps the @lid participant and its phone', async () => { + const message = lidGroup(); + expect(message.key).toEqual({ + remoteJid: GROUP, + fromMe: false, + id: '3EB0DDDDDDDDDDDDDDD2', + participant: LID, + participantAlt: PHONE, + addressingMode: 'lid', + }); + + expect(await webhookKey(profile, message)).toEqual({ + remoteJid: GROUP, + fromMe: false, + id: '3EB0DDDDDDDDDDDDDDD2', + participant: LID, + participantAlt: PHONE, + addressingMode: 'lid', + }); + }); + + it('control: a DM addressed by phone is unchanged, its @lid in remoteJidAlt', async () => { + const message = received({ id: '3EB0DDDDDDDDDDDDDDD3', from: PHONE, sender_lid: LID, addressing_mode: 'pn', type: 'text', t: '1700000000', notify: 'Sender' }); + + expect(await webhookKey(profile, message)).toEqual({ + remoteJid: PHONE, + remoteJidAlt: LID, + fromMe: false, + id: '3EB0DDDDDDDDDDDDDDD3', + addressingMode: 'pn', + }); + }); +}); From f3d1052a3c41233e132451375edf1d7f2eb7862e Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:58:25 +0300 Subject: [PATCH 066/157] fix(baileys): keep the original @lid in remoteJidAlt when showing the phone address messages.upsert still shows the phone JID as the webhook key's remoteJid for a DM WhatsApp addresses by @lid (what consumers have always keyed chats by), but now puts the original @lid in remoteJidAlt instead of a second copy of the phone, and sets addressingMode to 'pn' so the mode describes remoteJid again. This is the swap upstream develop makes (b8d6c873). Before: { remoteJid: phone, remoteJidAlt: phone, addressingMode: 'lid' } After: { remoteJid: phone, remoteJidAlt: @lid, addressingMode: 'pn' } The same block feeds the IsOnWhatsapp cache, which now receives the phone with its @lid alt (and lid: null), as on develop, rather than the phone twice. Group keys are untouched: Evolution never rewrote participant, so a group message keeps participant = @lid and participantAlt = phone. No other path (messages.update, sends, history) rewrites a key. Co-Authored-By: Claude Opus 5.5 --- .../channel/whatsapp/whatsapp.baileys.service.ts | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index e220c7bb8b..985119f99a 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -1886,8 +1886,14 @@ export class BaileysStartupService extends ChannelStartupService { this.logger.verbose(messageRaw); sendTelemetry(`received.message.${messageRaw.messageType ?? 'unknown'}`); + // The webhook shows the phone JID as remoteJid, and keeps the @lid WhatsApp stores + // the message under in remoteJidAlt (upstream develop's swap), so a consumer can + // still name the message the way the phone does (a media re-upload request). if (messageRaw.key.remoteJid?.includes('@lid') && messageRaw.key.remoteJidAlt) { + const lid = messageRaw.key.remoteJid; messageRaw.key.remoteJid = messageRaw.key.remoteJidAlt; + messageRaw.key.remoteJidAlt = lid; + messageRaw.key.addressingMode = 'pn'; } this.sendDataWebhook(Events.MESSAGES_UPSERT, messageRaw); From 999d2bb70177d1d24ffc6fe8ada52d9e9c02e459 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:59:10 +0300 Subject: [PATCH 067/157] test: a media re-upload names the chat by the address WhatsApp stores it under A re-upload request names the message by its key: Baileys' encryptMediaRetryRequest puts key.remoteJid in and key.participant in , and the phone looks the message up there. For a chat WhatsApp addresses by @lid, a request naming the phone JID is refused within a second (seen live). getBase64FromMediaMessage passes the key it was given, and the key a consumer holds from the webhook shows the phone as remoteJid. The fake phone records the key updateMediaMessage received and the attributes Baileys' real encryptMediaRetryRequest builds from it. Red: a DM key from the webhook (phone, @lid in remoteJidAlt); a DM key stored from 2.3.7 (the phone twice, addressingMode 'lid'), which Baileys' LID mapping resolves; a group key with the phone as participant and the @lid in participantAlt. Green already, pinned as controls: a key already in its @lid form, a group key as the webhook gives it, and a phone-addressed chat with no @lid known. Co-Authored-By: Claude Opus 5.5 --- test/proxy/media-reupload-address.test.ts | 150 ++++++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 test/proxy/media-reupload-address.test.ts diff --git a/test/proxy/media-reupload-address.test.ts b/test/proxy/media-reupload-address.test.ts new file mode 100644 index 0000000000..bf6c24565b --- /dev/null +++ b/test/proxy/media-reupload-address.test.ts @@ -0,0 +1,150 @@ +// A media re-upload request names the message by its key: Baileys' +// encryptMediaRetryRequest puts key.remoteJid in the attribute and +// key.participant in . The phone looks the message up under +// the address WhatsApp stores it by. For a chat WhatsApp addresses by @lid +// that is the @lid, and a request naming the phone JID is refused within a +// second (seen live, 2026-09-27). +// +// A consumer holds the key the messages.upsert webhook gave it, which shows +// the phone JID as remoteJid: with the @lid in remoteJidAlt since the webhook +// keeps it, or (a key stored from Evolution 2.3.7) with the phone twice and +// addressingMode 'lid'. Evolution asks for the re-upload with the message's +// original key either way. A group key names its sender the same way, in +// participant / participantAlt. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { readFile, rm } from 'node:fs/promises'; + +import { encryptedStream, encryptMediaRetryRequest, getBinaryNodeChild } from 'baileys'; +import { getGlobalDispatcher, MockAgent, setGlobalDispatcher } from 'undici'; +import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; + +import { makeService, WUID } from '../helpers/baileys-service'; +import { type Listening, startCdn } from '../helpers/local-net'; + +const LID = '123456789012345@lid'; +const PHONE = '972509876543@s.whatsapp.net'; +const GROUP = '120363000000000001@g.us'; +const ID = '3EB0EEEEEEEEEEEEEEE1'; +const PLAIN = Buffer.from('a photo, as the person sent it '.repeat(200)); +const LIVE = '/v/t62.7118-24/reuploaded.enc'; +const GONE = '/v/t62.7118-24/expired.enc'; + +let cdn: Listening; +let mediaKey: Uint8Array; + +const previousDispatcher = getGlobalDispatcher(); +const guard = new MockAgent(); +guard.disableNetConnect(); +guard.enableNetConnect((host: string) => host.startsWith('127.0.0.1:')); + +beforeAll(async () => { + setGlobalDispatcher(guard); + const enc = await encryptedStream(PLAIN, 'image', {}); + mediaKey = enc.mediaKey; + const body = await readFile(enc.encFilePath); + await rm(enc.encFilePath, { force: true }); + cdn = await startCdn({ [LIVE]: body }); +}); + +afterAll(async () => { + await cdn.close(); + setGlobalDispatcher(previousDispatcher); + await guard.close(); +}); + +beforeEach(() => void cdn.log.splice(0)); + +/** + * A service whose phone re-uploads the file, recording the key each request + * named and the attributes Baileys' real encryptMediaRetryRequest builds + * from it. `lidForPhone` is what Baileys' LID mapping store knows. + */ +async function serviceWithPhone(lidForPhone: Record = {}) { + const made = await makeService(); + const keys: any[] = []; + const rmr: any[] = []; + made.service.client.signalRepository.lidMapping.getLIDForPN = async (pn: string) => lidForPhone[pn] ?? null; + made.service.client.updateMediaMessage = async (message: any) => { + keys.push({ ...message.key }); + const node: any = encryptMediaRetryRequest(message.key, mediaKey, WUID); + rmr.push({ ...getBinaryNodeChild(node, 'rmr').attrs }); + message.message.imageMessage.url = `http://127.0.0.1:${cdn.port}${LIVE}`; + return message; + }; + return { ...made, keys, rmr }; +} + +async function downloadWithKey(service: any, key: Record) { + const result = await service.getBase64FromMediaMessage({ + message: { + key, + message: { + imageMessage: { url: `http://127.0.0.1:${cdn.port}${GONE}`, mediaKey, mimetype: 'image/jpeg', fileLength: PLAIN.length }, + }, + }, + }); + expect(Buffer.from(result.base64, 'base64').equals(PLAIN)).toBe(true); + expect(cdn.log).toEqual([`GET ${GONE}`, `GET ${LIVE}`]); +} + +describe('a media re-upload names the chat by the address WhatsApp stores it under', () => { + it('a DM key from the webhook (phone, @lid in remoteJidAlt) asks under the @lid', async () => { + const { service, keys, rmr } = await serviceWithPhone(); + + await downloadWithKey(service, { remoteJid: PHONE, remoteJidAlt: LID, fromMe: false, id: ID, addressingMode: 'pn' }); + + expect(keys).toEqual([{ remoteJid: LID, remoteJidAlt: PHONE, fromMe: false, id: ID, addressingMode: 'lid' }]); + expect(rmr).toEqual([{ jid: LID, from_me: 'false', participant: undefined }]); + }); + + it("a DM key stored from Evolution 2.3.7 (the phone twice, addressingMode 'lid') asks under the @lid Baileys maps it to", async () => { + const { service, keys, rmr } = await serviceWithPhone({ [PHONE]: LID }); + + await downloadWithKey(service, { remoteJid: PHONE, remoteJidAlt: PHONE, fromMe: false, id: ID, addressingMode: 'lid' }); + + expect(keys).toEqual([{ remoteJid: LID, remoteJidAlt: PHONE, fromMe: false, id: ID, addressingMode: 'lid' }]); + expect(rmr).toEqual([{ jid: LID, from_me: 'false', participant: undefined }]); + }); + + it('a group key with the phone as participant and the @lid in participantAlt asks under the @lid participant', async () => { + const { service, keys, rmr } = await serviceWithPhone(); + + await downloadWithKey(service, { remoteJid: GROUP, fromMe: false, id: ID, participant: PHONE, participantAlt: LID, addressingMode: 'pn' }); + + expect(keys).toEqual([{ remoteJid: GROUP, fromMe: false, id: ID, participant: LID, participantAlt: PHONE, addressingMode: 'lid' }]); + expect(rmr).toEqual([{ jid: GROUP, from_me: 'false', participant: LID }]); + }); + + it('control: a key already in its original @lid form is asked for as it is', async () => { + const { service, keys, rmr } = await serviceWithPhone(); + const key = { remoteJid: LID, remoteJidAlt: PHONE, fromMe: false, id: ID, addressingMode: 'lid' }; + + await downloadWithKey(service, key); + + expect(keys).toEqual([key]); + expect(rmr).toEqual([{ jid: LID, from_me: 'false', participant: undefined }]); + }); + + it('control: a group key from the webhook (the @lid participant, as Baileys gave it) is asked for as it is', async () => { + const { service, keys, rmr } = await serviceWithPhone(); + const key = { remoteJid: GROUP, fromMe: false, id: ID, participant: LID, participantAlt: PHONE, addressingMode: 'lid' }; + + await downloadWithKey(service, key); + + expect(keys).toEqual([key]); + expect(rmr).toEqual([{ jid: GROUP, from_me: 'false', participant: LID }]); + }); + + it('control: a chat addressed by phone, with no @lid known, is asked for under the phone', async () => { + const { service, keys, rmr } = await serviceWithPhone(); + const key = { remoteJid: PHONE, fromMe: false, id: ID }; + + await downloadWithKey(service, key); + + expect(keys).toEqual([key]); + expect(rmr).toEqual([{ jid: PHONE, from_me: 'false', participant: undefined }]); + }); +}); From 257ebde70f942f90ad16126ab0b3dd5d3c798a71 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:59:54 +0300 Subject: [PATCH 068/157] fix(baileys): ask for a re-upload with the message's original address getBase64FromMediaMessage now asks the phone for a re-upload with the key the phone stores the message under, not the key it was handed: - remoteJid a phone JID and remoteJidAlt an @lid (the webhook's DM key): remoteJid = the @lid, remoteJidAlt = the phone, addressingMode = 'lid'. - remoteJid a phone JID, no @lid alt, addressingMode 'lid' (a key stored from 2.3.7, the phone twice): the @lid Baileys' LID mapping has for the phone, if it has one; otherwise the key as given. - participant a phone JID and participantAlt an @lid: participant = the @lid, participantAlt = the phone, addressingMode = 'lid'. - A key already in its @lid form, or with no @lid known, is unchanged. Only the request changes: the download, the log and the error use the key as given. The mapping lookup runs inside the re-upload's time bound. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 38 ++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 985119f99a..4755668951 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -4401,6 +4401,41 @@ export class BaileysStartupService extends ChannelStartupService { return { options: { dispatcher: this.mediaProxy.dispatcher } as RequestInit }; } + /** + * The key the phone stores a message under, which is how a request about the message + * (a media re-upload) must name it: the phone refuses one that names a DM it keeps + * under an @lid by the phone JID. The messages.upsert webhook shows such a DM under + * the phone JID with the @lid in remoteJidAlt; a key from Evolution 2.3.7 has the phone + * twice and addressingMode 'lid', and only Baileys' LID mapping still knows the @lid. + * A group key names its sender the same way, in participant / participantAlt. + */ + private async originalMessageKey(key: WAMessageKey): Promise { + if (!key) return key; + const original = { ...key }; + if (!isLidUser(key.remoteJid) && isLidUser(key.remoteJidAlt)) { + original.remoteJid = key.remoteJidAlt; + original.remoteJidAlt = key.remoteJid; + original.addressingMode = 'lid'; + } else if (key.addressingMode === 'lid' && isPnUser(key.remoteJid)) { + let lid: string | null = null; + try { + lid = await this.client.signalRepository?.lidMapping?.getLIDForPN(key.remoteJid); + } catch { + // No mapping: the key is asked for as it was given. + } + if (isLidUser(lid)) { + original.remoteJid = lid; + original.remoteJidAlt = key.remoteJid; + } + } + if (!isLidUser(key.participant) && isLidUser(key.participantAlt)) { + original.participant = key.participantAlt; + original.participantAlt = key.participant; + original.addressingMode = 'lid'; + } + return original; + } + public async getBase64FromMediaMessage(data: getBase64FromMediaMessageDto, getBuffer = false) { // Set once a download is attempted: whether the phone was asked to re-upload an expired file. let reupload: MediaReupload | undefined; @@ -4478,7 +4513,8 @@ export class BaileysStartupService extends ChannelStartupService { }, MEDIA_REUPLOAD_TIMEOUT_MS); }); try { - const updated = await Promise.race([this.client.updateMediaMessage(message), timeout]); + const ask = async () => this.client.updateMediaMessage({ ...message, key: await this.originalMessageKey(message.key) }); + const updated = await Promise.race([ask(), timeout]); reupload = 'ok'; this.logger.warn(`media download: ${media}, outcome=reupload_ok`); return updated; From dbde828fe8dde3e63c0905c5fcbfb705a7eb537e Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:02:18 +0300 Subject: [PATCH 069/157] test: a refused re-upload says why The phone's answer to a re-upload request says why it refused: an , or an encrypted MediaRetryNotification whose result is NOT_FOUND, DECRYPTION_ERROR or GENERAL_ERROR. Baileys turns that into the error updateMediaMessage throws, and Evolution records only error= and status= today, so a refusal cannot be told apart from another. The new tests build the phone's notification node and read it with Baileys' own decodeMediaRetryNode and decryptMediaRetryData (the Boom for a refused result is built inline in updateMediaMessage, with no builder, so it is written out the same way). They specify reason= on the reupload_failed log line and reuploadReason on the download's error and the HTTP answer: the result name, error_, missing_ciphertext, or no_answer for a phone that did not answer in time. No content or JID is logged. Two existing expectations in media-reupload.test.ts change with the spec, and are red here for that reason: the refused and the timed-out re-upload now carry their reason. The refused case's fake now carries the phone's result in the Boom's data, as Baileys does. The test app's error handler mirror (helpers/http-app.ts) passes reuploadReason, as main.ts will. Co-Authored-By: Claude Opus 5.5 --- test/chat/media-reupload-reason.test.ts | 206 ++++++++++++++++++++++++ test/helpers/http-app.ts | 6 +- test/proxy/media-reupload.test.ts | 19 ++- 3 files changed, 224 insertions(+), 7 deletions(-) create mode 100644 test/chat/media-reupload-reason.test.ts diff --git a/test/chat/media-reupload-reason.test.ts b/test/chat/media-reupload-reason.test.ts new file mode 100644 index 0000000000..8033fd46ce --- /dev/null +++ b/test/chat/media-reupload-reason.test.ts @@ -0,0 +1,206 @@ +// When the phone refuses to re-upload an expired file, it says why: its +// answer to the request (a notification) carries +// either an or an encrypted MediaRetryNotification whose result is +// NOT_FOUND, DECRYPTION_ERROR or GENERAL_ERROR. Baileys turns that into the +// error updateMediaMessage throws. Evolution records the reason, and only the +// reason (never content or JIDs), in the re-upload log line (`reason=`) and on +// the download's error (`reuploadReason`, which the HTTP answer carries). +// +// The phone's answer is built as the notification node and read by Baileys' +// own decodeMediaRetryNode and decryptMediaRetryData. The Boom for a refused +// result is the one updateMediaMessage builds inline (messages-send.js), with +// no builder of its own, so it is written out here the same way. +import { vi } from 'vitest'; + +const h = vi.hoisted(() => ({ waMonitor: undefined as any, chatController: undefined as any })); + +vi.mock('@api/server.module', async () => { + const fake = await import('../helpers/fake-server-module'); + return { + ...fake, + get waMonitor() { + return h.waMonitor; + }, + get chatController() { + return h.chatController; + }, + }; +}); + +import { randomBytes } from 'node:crypto'; + +import { Boom } from '@hapi/boom'; +import { MEDIA_REUPLOAD_TIMEOUT_MS } from '@api/integrations/channel/whatsapp/whatsapp.baileys.service'; +import { + aesEncryptGCM, + decodeMediaRetryNode, + decryptMediaRetryData, + encryptedStream, + getStatusCodeForMediaRetry, + hkdf, + proto, +} from 'baileys'; +import { rm } from 'node:fs/promises'; +import { getGlobalDispatcher, MockAgent, setGlobalDispatcher } from 'undici'; +import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; + +import { makeService } from '../helpers/baileys-service'; +import { captureOutput } from '../helpers/capture-output'; +import { prismaRepository as prisma } from '../helpers/fake-server-module'; +import { startChatApp } from '../helpers/http-app'; +import { type Listening, startCdn } from '../helpers/local-net'; + +const TOKEN = 'instance-token'; +const PHONE = '972509876543'; +const ID = '3EB0FFFFFFFFFFFFFFF1'; +const CAPTION = 'Zq7 a private caption Zq7'; +const GONE = '/v/t62.7118-24/expired.enc'; +const { NOT_FOUND, DECRYPTION_ERROR, GENERAL_ERROR } = proto.MediaRetryNotification.ResultType; + +let cdn: Listening; +let mediaKey: Uint8Array; +let app: Awaited>; +let service: any; +/** What the phone answers to the next re-upload request: an error to throw, or nothing at all. */ +let phone: () => Promise; + +const previousDispatcher = getGlobalDispatcher(); +const guard = new MockAgent(); +guard.disableNetConnect(); +guard.enableNetConnect((host: string) => host.startsWith('127.0.0.1:')); + +beforeAll(async () => { + setGlobalDispatcher(guard); + const enc = await encryptedStream(Buffer.from('a photo'), 'image', {}); + mediaKey = enc.mediaKey; + await rm(enc.encFilePath, { force: true }); + cdn = await startCdn({}); + // Evolution waits 5s before its own fallback download, and gives the phone a + // bounded time to answer; both are shortened here. + const realSetTimeout = globalThis.setTimeout; + vi.spyOn(globalThis, 'setTimeout').mockImplementation(((fn: any, ms?: number, ...args: any[]) => + realSetTimeout(fn, ms === 5000 ? 0 : ms === MEDIA_REUPLOAD_TIMEOUT_MS ? 20 : ms, ...args)) as any); + + await prisma.instance.create({ data: { id: 'inst-1', name: 'test', connectionStatus: 'open', token: TOKEN, integration: 'WHATSAPP-BAILEYS' } }); + ({ service } = await makeService({ prisma })); + service.client.updateMediaMessage = () => phone(); + h.waMonitor = { waInstances: { test: service } }; + const { ChatController } = await import('@api/controllers/chat.controller'); + h.chatController = new ChatController(h.waMonitor); + app = await startChatApp(); +}); + +afterAll(async () => { + vi.restoreAllMocks(); + await app.close(); + await cdn.close(); + setGlobalDispatcher(previousDispatcher); + await guard.close(); +}); + +beforeEach(() => void cdn.log.splice(0)); + +/** The phone's notification for a re-upload request: an , or an encrypted result. */ +function answerNode(answer: { code: string } | { result: number }) { + const rmr = { tag: 'rmr', attrs: { jid: `${PHONE}@s.whatsapp.net`, from_me: 'false' } }; + if ('code' in answer) return { tag: 'receipt', attrs: { id: ID }, content: [rmr, { tag: 'error', attrs: { code: answer.code } }] }; + const plain = proto.MediaRetryNotification.encode({ stanzaId: ID, result: answer.result }).finish(); + const iv = randomBytes(12); + const retryKey = hkdf(mediaKey, 32, { info: 'WhatsApp Media Retry Notification' }); + const ciphertext = aesEncryptGCM(plain, retryKey, iv, Buffer.from(ID)); + const encrypt = { tag: 'encrypt', attrs: {}, content: [{ tag: 'enc_p', attrs: {}, content: ciphertext }, { tag: 'enc_iv', attrs: {}, content: iv }] }; + return { tag: 'receipt', attrs: { id: ID }, content: [encrypt, rmr] }; +} + +/** What Baileys' updateMediaMessage throws when the phone answers with this node. */ +function refusal(node: any): Error { + const event: any = decodeMediaRetryNode(node); + if (event.error) return event.error; + const media: any = decryptMediaRetryData(event.media, mediaKey, ID); + const resultStr = proto.MediaRetryNotification.ResultType[media.result]; + return new Boom(`Media re-upload failed by device (${resultStr})`, { + data: media, + statusCode: getStatusCodeForMediaRetry(media.result) || 404, + }); +} + +const refuses = (answer: { code: string } | { result: number }) => () => Promise.reject(refusal(answerNode(answer))); +const expiredImage = () => ({ + key: { remoteJid: `${PHONE}@s.whatsapp.net`, fromMe: false, id: ID }, + message: { imageMessage: { url: `http://127.0.0.1:${cdn.port}${GONE}`, mediaKey, mimetype: 'image/jpeg', caption: CAPTION } }, +}); + +async function download() { + let thrown: any; + const out = await captureOutput(async () => { + try { + await service.getBase64FromMediaMessage({ message: expiredImage() }); + } catch (e) { + thrown = e; + } + }); + const plain = out.replace(/\x1b\[[0-9;]*m/g, ''); + return { thrown, out: plain, lines: plain.split('\n').filter((l) => l.includes('media download:')).map((l) => l.slice(l.indexOf('media download:'))) }; +} + +const at = (fields: string) => `media download: message=${ID}, chat=user, ${fields}`; + +describe('a refused re-upload says why', () => { + it.each([ + ['NOT_FOUND', NOT_FOUND, 404], + ['DECRYPTION_ERROR', DECRYPTION_ERROR, 412], + ['GENERAL_ERROR', GENERAL_ERROR, 418], + ])('the phone answers %s: the log line and the error carry it', async (name, result, status) => { + phone = refuses({ result }); + const { thrown, out, lines } = await download(); + + expect(lines).toEqual([ + at('outcome=reupload_requested'), + at(`outcome=reupload_failed, error=Error, status=${status}, reason=${name}`), + at('outcome=download_failed, status=404, reupload=failed'), + ]); + expect(thrown).toEqual({ status: 400, error: 'Bad Request', message: [expect.any(String)], reupload: 'failed', reuploadReason: name }); + expect(out).not.toContain(PHONE); + expect(out).not.toContain('Zq7'); + }); + + it('the phone answers with an error code: the reason is that code', async () => { + phone = refuses({ code: '2' }); + const { thrown, lines } = await download(); + + expect(lines[1]).toBe(at('outcome=reupload_failed, error=Error, status=404, reason=error_2')); + expect(thrown.reuploadReason).toBe('error_2'); + }); + + it('the phone answers with neither an error nor a result: missing_ciphertext', async () => { + phone = () => Promise.reject(refusal({ tag: 'receipt', attrs: { id: ID }, content: [{ tag: 'rmr', attrs: { jid: `${PHONE}@s.whatsapp.net`, from_me: 'false' } }] })); + const { thrown, lines } = await download(); + + expect(lines[1]).toBe(at('outcome=reupload_failed, error=Error, status=404, reason=missing_ciphertext')); + expect(thrown.reuploadReason).toBe('missing_ciphertext'); + }); + + it('the phone does not answer in time: no_answer', async () => { + phone = () => new Promise(() => undefined); + const { thrown, lines } = await download(); + + expect(lines[1]).toBe(at('outcome=reupload_failed, error=ReuploadTimeoutError, status=none, reason=no_answer')); + expect(thrown.reuploadReason).toBe('no_answer'); + }); + + it('the HTTP answer carries the reason', async () => { + phone = refuses({ result: NOT_FOUND }); + const res = await fetch(`${app.base}/chat/getBase64FromMediaMessage/test`, { + method: 'POST', + headers: { 'content-type': 'application/json', apikey: TOKEN }, + body: JSON.stringify({ message: JSON.parse(JSON.stringify({ ...expiredImage(), message: { imageMessage: { ...expiredImage().message.imageMessage, mediaKey: Uint8Array.from(mediaKey) } } })) }), + }); + + expect(res.status).toBe(400); + expect(await res.json()).toEqual({ + status: 400, + error: 'Bad Request', + response: { message: [expect.any(String)], reupload: 'failed', reuploadReason: 'NOT_FOUND' }, + }); + }); +}); diff --git a/test/helpers/http-app.ts b/test/helpers/http-app.ts index 7dbcd47f9b..2370380f7b 100644 --- a/test/helpers/http-app.ts +++ b/test/helpers/http-app.ts @@ -13,7 +13,11 @@ async function serve(mount: string, router: any) { res.status(err?.status || 500).json({ status: err?.status || 500, error: err?.error, - response: { message: err?.message, ...(err?.reupload !== undefined && { reupload: err.reupload }) }, + response: { + message: err?.message, + ...(err?.reupload !== undefined && { reupload: err.reupload }), + ...(err?.reuploadReason !== undefined && { reuploadReason: err.reuploadReason }), + }, }), ); const server = app.listen(0, '127.0.0.1'); diff --git a/test/proxy/media-reupload.test.ts b/test/proxy/media-reupload.test.ts index 5537226b11..eb055a6d0f 100644 --- a/test/proxy/media-reupload.test.ts +++ b/test/proxy/media-reupload.test.ts @@ -94,7 +94,8 @@ async function serviceWithPhone(phone: Phone) { if (phone === 'fails') { // What Baileys raises when the phone reports the file is gone (messages-send.js updateMediaMessage). const { Boom } = await import('@hapi/boom'); - throw new Boom('Media re-upload failed by device (NOT_FOUND)', { statusCode: 404 }); + // Baileys puts the phone's decoded answer in the Boom's data (its result: 2 is NOT_FOUND). + throw new Boom('Media re-upload failed by device (NOT_FOUND)', { data: { stanzaId: message.key.id, result: 2 }, statusCode: 404 }); } // What Baileys does on success: point the message at the new copy (by url // alone here, since a directPath is fetched over https from the url's host). @@ -121,7 +122,13 @@ async function download(service: any, path = GONE) { } const line = (fields: string) => expect.stringContaining(`media download: message=${ID}, chat=user, ${fields}`); -const badRequest = (reupload: string) => ({ status: 400, error: 'Bad Request', message: [expect.any(String)], reupload }); +const badRequest = (reupload: string, reuploadReason?: string) => ({ + status: 400, + error: 'Bad Request', + message: [expect.any(String)], + reupload, + ...(reuploadReason && { reuploadReason }), +}); function expectNothingPrivate(out: string) { expect(out).not.toContain(PHONE); @@ -146,10 +153,10 @@ describe('a media download says whether it asked the phone to re-upload', () => const { thrown, out, lines } = await download(service); expect(asked).toEqual([ID]); - expect(thrown).toEqual(badRequest('failed')); + expect(thrown).toEqual(badRequest('failed', 'NOT_FOUND')); expect(lines).toEqual([ line('outcome=reupload_requested'), - line('outcome=reupload_failed, error=Error, status=404'), + line('outcome=reupload_failed, error=Error, status=404, reason=NOT_FOUND'), line('outcome=download_failed, status=404, reupload=failed'), ]); expectNothingPrivate(out); @@ -190,10 +197,10 @@ describe('an expired media download asks the phone to re-upload, once', () => { expect(MEDIA_REUPLOAD_TIMEOUT_MS).toBe(60_000); expect(asked).toEqual([ID]); expect(cdn.log).toEqual([`GET ${GONE}`]); - expect(thrown).toEqual(badRequest('failed')); + expect(thrown).toEqual(badRequest('failed', 'no_answer')); expect(lines).toEqual([ line('outcome=reupload_requested'), - line('outcome=reupload_failed, error=ReuploadTimeoutError, status=none'), + line('outcome=reupload_failed, error=ReuploadTimeoutError, status=none, reason=no_answer'), line('outcome=download_failed, status=404, reupload=failed'), ]); expectNothingPrivate(out); From d12de0283e4208be114383762b38750e389ab025 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:02:58 +0300 Subject: [PATCH 070/157] feat(baileys): record why the phone refused a re-upload A failed re-upload now says why the phone refused it, read from the error Baileys' updateMediaMessage throws, never from content or a JID: - the phone's MediaRetryNotification result, from the Boom's data.result: NOT_FOUND, DECRYPTION_ERROR or GENERAL_ERROR (result_ for a value this Baileys does not name); - error_ for an answer (data.code, digits only); - missing_ciphertext for an answer with neither; - no_answer when the phone did not answer within MEDIA_REUPLOAD_TIMEOUT_MS; - unknown otherwise. Log: outcome=reupload_failed, error=, status=, reason= Error: { status: 400, error, message, reupload: 'failed', reuploadReason } HTTP (main.ts): response.reuploadReason next to response.reupload. reuploadReason is present only when the re-upload failed. Co-Authored-By: Claude Opus 5.5 --- FORK.md | 2 +- .../whatsapp/whatsapp.baileys.service.ts | 25 ++++++++++++++++--- src/main.ts | 3 ++- 3 files changed, 25 insertions(+), 5 deletions(-) diff --git a/FORK.md b/FORK.md index a53b99b6c6..92f988651a 100644 --- a/FORK.md +++ b/FORK.md @@ -51,7 +51,7 @@ named where one exists. **Messages and privacy** - A `getMessage` miss or failed lookup answers nothing, so Baileys no longer relays an empty message on a retry and uses it up (#2705, and #2706 for groups; fixes also offered in #2728 and #2623). - No message text, phone number, JID or push name reaches the logs at `LOG_LEVEL=ERROR,WARN`, including Baileys' own error logs. -- A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410 (its check misses the status), so Evolution asks the phone itself, once per download, for at most 60s. +- A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error), and when the phone refused, why (`reason=` on the log line, `reuploadReason` on the error and the HTTP answer: the phone's result such as `NOT_FOUND`, `error_`, `missing_ciphertext`, or `no_answer`). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410 (its check misses the status), so Evolution asks the phone itself, once per download, for at most 60s. - `POST /chat/getBase64FromMediaMessage` takes an optional `reupload` (boolean, default true). With `false` the phone is never asked to re-upload, and a file gone from the CDN fails at once with a 400 that says so; the HTTP error carries `reupload` for any failed download. **Proxy** diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 4755668951..877796ae1f 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -237,6 +237,22 @@ type MediaReupload = 'not_requested' | 'ok' | 'failed'; /** The HTTP status a Baileys media error carries (a Boom's output.statusCode), or 'none'. */ const httpStatus = (error: any) => error?.output?.statusCode ?? error?.status ?? 'none'; +/** + * Why the phone refused a re-upload, from the error Baileys' updateMediaMessage threw: + * the phone's MediaRetryNotification result (NOT_FOUND, DECRYPTION_ERROR, GENERAL_ERROR), + * error_ for an answer, missing_ciphertext for an answer with neither, + * no_answer when it did not answer in time, else unknown. Never content or a JID. + */ +const reuploadRefusal = (error: any): string => { + if (error?.name === 'ReuploadTimeoutError') return 'no_answer'; + const result = error?.data?.result; + if (typeof result === 'number') return proto.MediaRetryNotification.ResultType[result] ?? `result_${result}`; + const code = String(error?.data?.code ?? ''); + if (/^\d{1,6}$/.test(code)) return `error_${code}`; + if (error?.message === 'Failed to re-upload media (missing ciphertext)') return 'missing_ciphertext'; + return 'unknown'; +}; + /** A CDN answer that means the file has expired there, and only the phone still has it. */ const isExpiredMedia = (error: any) => [404, 410].includes(httpStatus(error)); @@ -4439,6 +4455,8 @@ export class BaileysStartupService extends ChannelStartupService { public async getBase64FromMediaMessage(data: getBase64FromMediaMessageDto, getBuffer = false) { // Set once a download is attempted: whether the phone was asked to re-upload an expired file. let reupload: MediaReupload | undefined; + // Set when the re-upload failed: why (reuploadRefusal). + let reuploadReason: string | undefined; try { const m = data?.message; const convertToMp4 = data?.convertToMp4 ?? false; @@ -4520,8 +4538,9 @@ export class BaileysStartupService extends ChannelStartupService { return updated; } catch (error) { reupload = 'failed'; + reuploadReason = reuploadRefusal(error); this.logger.warn( - `media download: ${media}, outcome=reupload_failed, error=${error?.name ?? 'unknown'}, status=${httpStatus(error)}`, + `media download: ${media}, outcome=reupload_failed, error=${error?.name ?? 'unknown'}, status=${httpStatus(error)}, reason=${reuploadReason}`, ); throw error; } finally { @@ -4629,11 +4648,11 @@ export class BaileysStartupService extends ChannelStartupService { this.logger.error('Error processing media message:'); this.logger.error(error); if (reupload === undefined) throw new BadRequestException(error.toString()); - // The same 400, plus whether the phone was asked to re-upload the file. + // The same 400, plus whether the phone was asked to re-upload the file, and why it refused. try { new BadRequestException(error.toString()); } catch (badRequest) { - throw { ...badRequest, reupload }; + throw { ...badRequest, reupload, ...(reuploadReason && { reuploadReason }) }; } } } diff --git a/src/main.ts b/src/main.ts index 4942a458e5..b6d9d733cc 100644 --- a/src/main.ts +++ b/src/main.ts @@ -112,8 +112,9 @@ async function bootstrap() { error: err['error'] || 'Internal Server Error', response: { message: err['message'] || 'Internal Server Error', - // A failed media download says whether the phone was asked to re-upload the file. + // A failed media download says whether the phone was asked to re-upload the file, and why it refused. ...(err['reupload'] !== undefined && { reupload: err['reupload'] }), + ...(err['reuploadReason'] !== undefined && { reuploadReason: err['reuploadReason'] }), }, }); } From 0d4ce582295f596433ffdc6a01b6303c6c3b890d Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:03:36 +0300 Subject: [PATCH 071/157] test: a 403 from the media servers also asks the phone to re-upload Live evidence (2026-09-27, a real linked phone): 8 media messages from the history sync, 30 to 180 days old (7 in groups with @lid participants, 1 in an @lid DM, all with their original keys), answered HTTP 403 from WhatsApp's media CDN, both at the directPath and at the url as received. Evolution asks the phone to re-upload only on 404 or 410, so it never asked for them. Earlier, 24-day-old live messages answered 410. Treating 403 as expired is empirical, and will be confirmed on the live rig right after this pair. Red: a CDN 403 does not ask the phone (the download fails through Evolution's fallback instead); with reupload: false a 403 is not recognised as a file that is gone, so it takes the 5s fallback instead of failing at once with the "no longer on WhatsApp's servers" 400. The 500 case (not expired, never asks) is unchanged and stays green. Co-Authored-By: Claude Opus 5.5 --- test/chat/media-skip-reupload.test.ts | 16 +++++++++++++++- test/proxy/media-reupload.test.ts | 16 +++++++++++++++- 2 files changed, 30 insertions(+), 2 deletions(-) diff --git a/test/chat/media-skip-reupload.test.ts b/test/chat/media-skip-reupload.test.ts index 0d0477d705..447630b5a3 100644 --- a/test/chat/media-skip-reupload.test.ts +++ b/test/chat/media-skip-reupload.test.ts @@ -38,6 +38,7 @@ const ID = '3EB0DDDDDDDDDDDDDDD1'; const PLAIN = Buffer.from('a photo, as the person sent it '.repeat(200)); const LIVE = '/v/t62.7118-24/reuploaded.enc'; const GONE = '/v/t62.7118-24/expired.enc'; +const GONE_403 = '/v/t62.7118-24/expired-403.enc'; const GONE_MESSAGE = "The media is no longer on WhatsApp's servers (HTTP 404), and no re-upload from the phone was attempted (reupload: false)"; @@ -59,7 +60,7 @@ beforeAll(async () => { mediaKey = enc.mediaKey; const body = await readFile(enc.encFilePath); await rm(enc.encFilePath, { force: true }); - cdn = await startCdn({ [LIVE]: body }); + cdn = await startCdn({ [LIVE]: body, [GONE_403]: 403 }); // Evolution waits 5s before its own fallback download; shortened, so a test that // takes it is not slow, and one that skips it is told apart by what it answers. const realSetTimeout = globalThis.setTimeout; @@ -125,6 +126,19 @@ describe('a media download can skip asking the phone to re-upload', () => { }); }); + it('reupload: false and a CDN 403 (an expired file too): the phone is not asked, and the error says the file is gone', async () => { + const message = expiredImage(); + message.message.imageMessage.url = `http://127.0.0.1:${cdn.port}${GONE_403}`; + const answer = await post({ message, reupload: false }); + + expect(asked).toEqual([]); + expect(cdn.log).toEqual([`GET ${GONE_403}`]); + expect(answer).toEqual({ + status: 400, + body: { status: 400, error: 'Bad Request', response: { message: [GONE_MESSAGE.replace('HTTP 404', 'HTTP 403')], reupload: 'not_requested' } }, + }); + }); + it('reupload must be a boolean', async () => { const answer = await post({ message: expiredImage(), reupload: 'no' }); diff --git a/test/proxy/media-reupload.test.ts b/test/proxy/media-reupload.test.ts index eb055a6d0f..f07ba2e773 100644 --- a/test/proxy/media-reupload.test.ts +++ b/test/proxy/media-reupload.test.ts @@ -32,6 +32,7 @@ const LIVE = '/v/t62.7118-24/reuploaded.enc'; const GONE = '/v/t62.7118-24/expired.enc'; const GONE_AGAIN = '/v/t62.7118-24/expired-again.enc'; const GONE_410 = '/v/t62.7118-24/expired-410.enc'; +const GONE_403 = '/v/t62.7118-24/expired-403.enc'; const BROKEN = '/v/t62.7118-24/broken.enc'; let cdn: Listening; @@ -50,7 +51,7 @@ beforeAll(async () => { mediaKey = enc.mediaKey; const body = await readFile(enc.encFilePath); await rm(enc.encFilePath, { force: true }); - cdn = await startCdn({ [LIVE]: body, [GONE_410]: 410, [BROKEN]: 500 }); + cdn = await startCdn({ [LIVE]: body, [GONE_410]: 410, [GONE_403]: 403, [BROKEN]: 500 }); // Evolution waits 5s before its own fallback download, and gives the phone a // bounded time to answer a re-upload request; both are shortened here. const realSetTimeout = globalThis.setTimeout; @@ -175,6 +176,19 @@ describe('an expired media download asks the phone to re-upload, once', () => { expect(lines).toEqual([line('outcome=reupload_requested'), line('outcome=reupload_ok')]); }); + // Empirical (2026-09-27): media from a history sync, 30 to 180 days old, answered + // 403 on a real linked phone where 24-day-old media answered 410. + it('a CDN 403 asks the phone too', async () => { + const { service, asked } = await serviceWithPhone('reuploads'); + const { result, thrown, lines } = await download(service, GONE_403); + + expect(thrown).toBeUndefined(); + expect(Buffer.from(result.base64, 'base64').equals(PLAIN)).toBe(true); + expect(asked).toEqual([ID]); + expect(cdn.log).toEqual([`GET ${GONE_403}`, `GET ${LIVE}`]); + expect(lines).toEqual([line('outcome=reupload_requested'), line('outcome=reupload_ok')]); + }); + it('a re-uploaded copy that is gone as well is not re-uploaded again', async () => { const { service, asked } = await serviceWithPhone('reuploads-expired'); const { thrown, out, lines } = await download(service); From c09c604811526984c2cda31078b2f7c2a42f04cb Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:04:01 +0300 Subject: [PATCH 072/157] fix(baileys): ask the phone to re-upload after a 403 from the media servers A CDN 403 now counts as an expired file, like 404 and 410: with re-upload allowed, Evolution asks the phone once, retries the download once and records the outcome; with reupload: false the phone is never asked and the download fails at once with the "no longer on WhatsApp's servers (HTTP 403)" 400. A 500 or any other status still never asks the phone. This is empirical. On a real linked phone (2026-09-27), 8 history-sync media messages 30 to 180 days old (7 in groups with @lid participants, 1 in an @lid DM, original keys) answered 403 at both the directPath and the url, while 24-day-old live media answered 410. It will be confirmed on the live rig right after this change. Co-Authored-By: Claude Opus 5.5 --- FORK.md | 4 ++-- .../channel/whatsapp/whatsapp.baileys.service.ts | 10 +++++++--- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/FORK.md b/FORK.md index 92f988651a..c9033aa456 100644 --- a/FORK.md +++ b/FORK.md @@ -51,8 +51,8 @@ named where one exists. **Messages and privacy** - A `getMessage` miss or failed lookup answers nothing, so Baileys no longer relays an empty message on a retry and uses it up (#2705, and #2706 for groups; fixes also offered in #2728 and #2623). - No message text, phone number, JID or push name reaches the logs at `LOG_LEVEL=ERROR,WARN`, including Baileys' own error logs. -- A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error), and when the phone refused, why (`reason=` on the log line, `reuploadReason` on the error and the HTTP answer: the phone's result such as `NOT_FOUND`, `error_`, `missing_ciphertext`, or `no_answer`). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410 (its check misses the status), so Evolution asks the phone itself, once per download, for at most 60s. -- `POST /chat/getBase64FromMediaMessage` takes an optional `reupload` (boolean, default true). With `false` the phone is never asked to re-upload, and a file gone from the CDN fails at once with a 400 that says so; the HTTP error carries `reupload` for any failed download. +- A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error), and when the phone refused, why (`reason=` on the log line, `reuploadReason` on the error and the HTTP answer: the phone's result such as `NOT_FOUND`, `error_`, `missing_ciphertext`, or `no_answer`). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410 (its check misses the status), so Evolution asks the phone itself, once per download, for at most 60s. A CDN 403 counts as expired too (seen on history media 30 to 180 days old). +- `POST /chat/getBase64FromMediaMessage` takes an optional `reupload` (boolean, default true). With `false` the phone is never asked to re-upload, and a file gone from the CDN (403, 404 or 410) fails at once with a 400 that says so; the HTTP error carries `reupload` for any failed download. **Proxy** - Media downloads, media uploads and the WhatsApp Web version fetch leave through the instance's proxy (uploads failed outright on a proxied instance). diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 877796ae1f..5465d7d83d 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -253,8 +253,12 @@ const reuploadRefusal = (error: any): string => { return 'unknown'; }; -/** A CDN answer that means the file has expired there, and only the phone still has it. */ -const isExpiredMedia = (error: any) => [404, 410].includes(httpStatus(error)); +/** + * A CDN answer that means the file has expired there, and only the phone still has it. + * 403 is empirical: history-sync media 30 to 180 days old answered 403 on a real linked + * phone (2026-09-27), where 24-day-old media answered 410. + */ +const isExpiredMedia = (error: any) => [403, 404, 410].includes(httpStatus(error)); /** * How long the phone gets to answer a re-upload request. Baileys' updateMediaMessage @@ -4573,7 +4577,7 @@ export class BaileysStartupService extends ChannelStartupService { } catch (error) { const status = httpStatus(error); this.logger.error(`media download: ${media}, outcome=download_failed, status=${status}, reupload=${reupload}`); - if (!askPhone && (status === 404 || status === 410)) { + if (!askPhone && isExpiredMedia(error)) { throw `The media is no longer on WhatsApp's servers (HTTP ${status}), and no re-upload from the phone was attempted (reupload: false)`; } this.logger.error('Download Media failed, trying to retry in 5 seconds...'); From c147db14f227b26b38033194a3d3d2b36a883205 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:04:09 +0300 Subject: [PATCH 073/157] docs(fork): list the @lid webhook key and the re-upload address Co-Authored-By: Claude Opus 5.5 --- FORK.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/FORK.md b/FORK.md index c9033aa456..96ab809490 100644 --- a/FORK.md +++ b/FORK.md @@ -51,6 +51,8 @@ named where one exists. **Messages and privacy** - A `getMessage` miss or failed lookup answers nothing, so Baileys no longer relays an empty message on a retry and uses it up (#2705, and #2706 for groups; fixes also offered in #2728 and #2623). - No message text, phone number, JID or push name reaches the logs at `LOG_LEVEL=ERROR,WARN`, including Baileys' own error logs. +- The `messages.upsert` webhook key of a DM WhatsApp addresses by @lid shows the phone JID as `remoteJid`, as 2.3.7 does, but keeps the original @lid in `remoteJidAlt` (2.3.7 copied the phone there too and lost it), with `addressingMode: 'pn'`: upstream develop's swap. +- A media re-upload request names the message by the address the phone stores it under: the @lid from `remoteJidAlt` or `participantAlt` when the key shows the phone, or, for a key stored from 2.3.7 (the phone twice, `addressingMode: 'lid'`), the @lid Baileys' LID mapping has for the phone. The phone refuses a request that names an @lid chat by its phone JID. - A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error), and when the phone refused, why (`reason=` on the log line, `reuploadReason` on the error and the HTTP answer: the phone's result such as `NOT_FOUND`, `error_`, `missing_ciphertext`, or `no_answer`). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410 (its check misses the status), so Evolution asks the phone itself, once per download, for at most 60s. A CDN 403 counts as expired too (seen on history media 30 to 180 days old). - `POST /chat/getBase64FromMediaMessage` takes an optional `reupload` (boolean, default true). With `false` the phone is never asked to re-upload, and a file gone from the CDN (403, 404 or 410) fails at once with a 400 that says so; the HTTP error carries `reupload` for any failed download. From cc5241fa0d97b91a813c8373f735a96e5b1ae5f5 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:12:18 +0300 Subject: [PATCH 074/157] test: a re-upload works when the media key arrives as text getBase64FromMediaMessage receives the message over HTTP JSON, so its media key is not bytes. Baileys 7.0.0-rc14's download reads a base64 string key itself (getMediaKeys, lib/Utils/messages-media.js:74), but its re-upload does not: updateMediaMessage passes the key as given to encryptMediaRetryRequest and decryptMediaRetryData, whose getMediaRetryKey (messages-media.js:701) runs hkdf over it unconverted, so the phone's answer, encrypted under the true key, fails with "Unsupported state or unable to authenticate data" (aesDecryptGCM, crypto.js:61). Seen live on 2026-09-27 on every re-upload of 8 files. The fake socket runs Baileys' own updateMediaMessage steps on its real encryptMediaRetryRequest, decodeMediaRetryNode and decryptMediaRetryData; the phone answers with a MediaRetryNotification (the new directPath) encrypted under the file's true key, and the retried download fetches that directPath from a local HTTPS server. Red: a base64 string key (the GCM error above, confirmed directly against decryptMediaRetryData) and a {type:'Buffer', data} key, the JSON of a Node Buffer, which Evolution's Object.values conversion turns into two zero bytes. Green already, pinned: bytes, and the index-keyed object JSON makes of a Uint8Array, which that conversion handles. Co-Authored-By: Claude Opus 5.5 --- test/proxy/media-reupload-key-bytes.test.ts | 163 ++++++++++++++++++++ 1 file changed, 163 insertions(+) create mode 100644 test/proxy/media-reupload-key-bytes.test.ts diff --git a/test/proxy/media-reupload-key-bytes.test.ts b/test/proxy/media-reupload-key-bytes.test.ts new file mode 100644 index 0000000000..ce383c5d17 --- /dev/null +++ b/test/proxy/media-reupload-key-bytes.test.ts @@ -0,0 +1,163 @@ +// A consumer hands getBase64FromMediaMessage a message over HTTP JSON, so the +// media key is no longer bytes: a base64 string, the index-keyed object +// JSON.stringify makes of a Uint8Array ({"0":..,"1":..}), or the +// {type:'Buffer',data:[..]} it makes of a Node Buffer. Baileys' download reads +// a base64 string itself (getMediaKeys), but its re-upload does not: +// updateMediaMessage derives the retry key with hkdf(mediaKey) as given +// (getMediaRetryKey, lib/Utils/messages-media.js:701), and decryptMediaRetryData +// then fails the phone's answer with "Unsupported state or unable to +// authenticate data" (seen live, 2026-09-27, on every re-upload asked for over +// HTTP). Evolution turns the key back into bytes before it asks. +// +// The socket's updateMediaMessage below is Baileys' own (lib/Socket/messages-send.js:1011), +// on Baileys' real encryptMediaRetryRequest, decodeMediaRetryNode and +// decryptMediaRetryData. The phone answers the way a phone does: a +// MediaRetryNotification encrypted under the file's true media key. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { randomBytes } from 'node:crypto'; +import { readFile, rm } from 'node:fs/promises'; + +import { Boom } from '@hapi/boom'; +import { + aesEncryptGCM, + assertMediaContent, + decodeMediaRetryNode, + decryptMediaRetryData, + encryptedStream, + encryptMediaRetryRequest, + getUrlFromDirectPath, + hkdf, + proto, +} from 'baileys'; +import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; + +import { makeService, WUID } from '../helpers/baileys-service'; +import { captureOutput } from '../helpers/capture-output'; +import { type Listening, loopbackOnly, startHttpsServer, trustTestCertificate } from '../helpers/local-net'; + +const ID = '3EB0ABABABABABABABA1'; +const PLAIN = Buffer.from('a photo, as the person sent it '.repeat(200)); +const GONE = '/v/t62.7118-24/expired.enc'; +const NEW = '/v/t62.7118-24/reuploaded.enc'; + +let cdn: Listening; +let host: string; +let mediaKey: Buffer; +let untrust: () => void; +let net: ReturnType; + +beforeAll(async () => { + net = loopbackOnly(); + untrust = trustTestCertificate(); + const enc = await encryptedStream(PLAIN, 'image', {}); + mediaKey = Buffer.from(enc.mediaKey); + const body = await readFile(enc.encFilePath); + await rm(enc.encFilePath, { force: true }); + // WhatsApp's media servers: the expired copy is gone, the phone's new one is there. + cdn = await startHttpsServer((req, _body, res) => { + if (req.url === NEW) return void res.writeHead(200, { 'content-length': body.length }).end(body); + res.writeHead(404).end(); + }); + host = `127.0.0.1:${cdn.port}`; + // Evolution waits 5s before its own fallback download. + const realSetTimeout = globalThis.setTimeout; + vi.spyOn(globalThis, 'setTimeout').mockImplementation(((fn: any, ms?: number, ...args: any[]) => + realSetTimeout(fn, ms === 5000 ? 0 : ms, ...args)) as any); +}); + +afterAll(async () => { + vi.restoreAllMocks(); + await cdn.close(); + untrust(); + net.restore(); +}); + +beforeEach(() => { + cdn.log.splice(0); + net.refused.splice(0); +}); + +/** The phone's answer to a re-upload request: the new copy's directPath, encrypted under the file's true media key. */ +function phoneAnswers(request: any) { + const id = request.attrs.id; + const plain = proto.MediaRetryNotification.encode({ stanzaId: id, directPath: NEW, result: proto.MediaRetryNotification.ResultType.SUCCESS }).finish(); + const iv = randomBytes(12); + const retryKey = hkdf(mediaKey, 32, { info: 'WhatsApp Media Retry Notification' }); + const ciphertext = aesEncryptGCM(plain, retryKey, iv, Buffer.from(id)); + return { + tag: 'receipt', + attrs: { id }, + content: [ + { tag: 'encrypt', attrs: {}, content: [{ tag: 'enc_p', attrs: {}, content: ciphertext }, { tag: 'enc_iv', attrs: {}, content: iv }] }, + { tag: 'rmr', attrs: { jid: '972509876543@s.whatsapp.net', from_me: 'false' } }, + ], + }; +} + +async function serviceWithPhone() { + const made = await makeService(); + const asked: string[] = []; + // Baileys' updateMediaMessage, with the node round trip done in process. + made.service.client.updateMediaMessage = async (message: any) => { + asked.push(message.key.id); + const content: any = assertMediaContent(message.message); + const mediaKey = content.mediaKey; + const request = encryptMediaRetryRequest(message.key, mediaKey, WUID); + const result: any = decodeMediaRetryNode(phoneAnswers(request) as any); + if (result.error) throw result.error; + const media = decryptMediaRetryData(result.media, mediaKey, result.key.id); + if (media.result !== proto.MediaRetryNotification.ResultType.SUCCESS) { + throw new Boom(`Media re-upload failed by device (${proto.MediaRetryNotification.ResultType[media.result]})`, { data: media }); + } + content.directPath = media.directPath; + content.url = getUrlFromDirectPath(content.directPath, host); + return message; + }; + return { ...made, asked }; +} + +/** The media key as a consumer holding the message as JSON sends it. */ +const shapes: [string, () => any][] = [ + ['bytes (control)', () => Uint8Array.from(mediaKey)], + ['a base64 string', () => mediaKey.toString('base64')], + ['an index-keyed object (JSON of a Uint8Array)', () => JSON.parse(JSON.stringify(Uint8Array.from(mediaKey)))], + ["a {type:'Buffer', data} object (JSON of a Buffer)", () => JSON.parse(JSON.stringify(mediaKey))], +]; + +describe('a re-upload works when the media key arrives as text', () => { + it.each(shapes)('media key as %s: the phone is asked, and the download gets the new copy', async (_shape, key) => { + const { service, asked } = await serviceWithPhone(); + let result: any; + let thrown: any; + const out = await captureOutput(async () => { + try { + result = await service.getBase64FromMediaMessage({ + message: { + key: { remoteJid: '972509876543@s.whatsapp.net', fromMe: false, id: ID }, + message: { imageMessage: { url: `https://${host}${GONE}`, mediaKey: key(), mimetype: 'image/jpeg', fileLength: PLAIN.length } }, + }, + }); + } catch (e) { + thrown = e; + } + }); + const lines = out + .replace(/\x1b\[[0-9;]*m/g, '') + .split('\n') + .filter((l) => l.includes('media download:')) + .map((l) => l.slice(l.indexOf('media download:'))); + + expect(asked).toEqual([ID]); + expect(lines).toEqual([ + `media download: message=${ID}, chat=user, outcome=reupload_requested`, + `media download: message=${ID}, chat=user, outcome=reupload_ok`, + ]); + expect(thrown).toBeUndefined(); + expect(cdn.log).toEqual([`GET ${GONE}`, `GET ${NEW}`]); + expect(Buffer.from(result.base64, 'base64').equals(PLAIN)).toBe(true); + expect(net.refused).toEqual([]); + }); +}); From d149189cb727ed411026e77deab6d50041e483c7 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:12:51 +0300 Subject: [PATCH 075/157] fix(baileys): turn the media key back into bytes before asking the phone getBase64FromMediaMessage now turns the media key into a Buffer before any download or re-upload, from every shape a message sent as JSON carries: - a base64 string (with or without Baileys' 'data:;base64,' prefix); - {type:'Buffer', data:[..]}, the JSON of a Node Buffer; - the index-keyed object {"0":..,"1":..}, the JSON of a Uint8Array; - bytes (Buffer or Uint8Array), copied as they are. Before, only an object was converted, with Object.values, which is right for the index-keyed shape and turns a {type:'Buffer'} one into junk; a string was left alone. Baileys' download decodes a string key itself, but updateMediaMessage derives the re-upload's retry key from the value as given (getMediaRetryKey, messages-media.js:701), so the phone's answer could not be decrypted ("Unsupported state or unable to authenticate data"), which is how every re-upload asked for over HTTP failed live on 2026-09-27. Upstream Baileys PR #2729 (closed, unreviewed) carried the same conversion inside getMediaRetryKey; once Baileys converts there, this is only belt and braces. Only the media key is converted: fileSha256 and fileEncSha256 are read by neither the download nor the re-upload. Co-Authored-By: Claude Opus 5.5 --- FORK.md | 1 + .../whatsapp/whatsapp.baileys.service.ts | 18 ++++++++++++++++-- 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/FORK.md b/FORK.md index 96ab809490..6a25c8e4a2 100644 --- a/FORK.md +++ b/FORK.md @@ -54,6 +54,7 @@ named where one exists. - The `messages.upsert` webhook key of a DM WhatsApp addresses by @lid shows the phone JID as `remoteJid`, as 2.3.7 does, but keeps the original @lid in `remoteJidAlt` (2.3.7 copied the phone there too and lost it), with `addressingMode: 'pn'`: upstream develop's swap. - A media re-upload request names the message by the address the phone stores it under: the @lid from `remoteJidAlt` or `participantAlt` when the key shows the phone, or, for a key stored from 2.3.7 (the phone twice, `addressingMode: 'lid'`), the @lid Baileys' LID mapping has for the phone. The phone refuses a request that names an @lid chat by its phone JID. - A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error), and when the phone refused, why (`reason=` on the log line, `reuploadReason` on the error and the HTTP answer: the phone's result such as `NOT_FOUND`, `error_`, `missing_ciphertext`, or `no_answer`). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410 (its check misses the status), so Evolution asks the phone itself, once per download, for at most 60s. A CDN 403 counts as expired too (seen on history media 30 to 180 days old). +- A media download turns the media key back into bytes (from a base64 string, or the object JSON makes of a Uint8Array or a Buffer) before it asks the phone to re-upload, since Baileys 7.0.0-rc14 derives the re-upload's key from the value as given and then cannot decrypt the phone's answer (Baileys #2729 proposed the same fix there). - `POST /chat/getBase64FromMediaMessage` takes an optional `reupload` (boolean, default true). With `false` the phone is never asked to re-upload, and a file gone from the CDN (403, 404 or 410) fails at once with a 400 that says so; the HTTP error carries `reupload` for any failed download. **Proxy** diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 5465d7d83d..84e43a232b 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -231,6 +231,20 @@ async function getVideoDuration(input: Buffer | string | Readable): Promise { + if (typeof value === 'string') return Buffer.from(value.replace('data:;base64,', ''), 'base64'); + if (value instanceof Uint8Array) return Buffer.from(value); + if (value?.type === 'Buffer' && Array.isArray(value.data)) return Buffer.from(value.data); + return Buffer.from(Object.values(value) as number[]); +}; + /** Whether a media download asked the phone to re-upload an expired file, and how that ended. */ type MediaReupload = 'not_requested' | 'ok' | 'failed'; @@ -4515,8 +4529,8 @@ export class BaileysStartupService extends ChannelStartupService { } } - if (typeof mediaMessage['mediaKey'] === 'object') { - msg.message[mediaType].mediaKey = Uint8Array.from(Object.values(mediaMessage['mediaKey'])); + if (mediaMessage['mediaKey'] != null) { + msg.message[mediaType].mediaKey = mediaKeyBytes(mediaMessage['mediaKey']); } let buffer: Buffer; From 7b09c475dbae11c51ecd546eb657fbc898bd3efe Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:21:38 +0300 Subject: [PATCH 076/157] test: a 403 asks the phone only when the media link has expired Measured on WhatsApp's media CDN (2026-09-27, a random sample of 84 history-sync attachments from one linked account, downloaded directly with Baileys 7.0.0-rc14, no re-upload): 403 answered 34 of 34 links whose `oe` query parameter (hex unix seconds) had passed and 0 of 50 valid links. A valid link to a file the CDN no longer holds answered 404 or 410. So a 403 means the signed link expired, not that the file is gone. The rule these tests specify: a 403 asks the phone to re-upload only when the link's `oe` has passed (read from the url, else from the directPath). 404 and 410 keep asking regardless. A 403 on a link whose `oe` has not passed, or that has none, does not ask. With reupload: false the phone is never asked, and only an expired-link 403 fails at once as a file that is gone. The two earlier 403 cases (0d4ce582) used a link with no `oe`; they now carry one that has passed, since under this rule a bare 403 is no longer an expiry. New cases: 403 with a future `oe`, 403 with no `oe`, 403 whose `oe` is only in the directPath, and reupload: false with a future `oe`. Red: a 403 with a future `oe` and a 403 with no `oe` still ask the phone, and with reupload: false a 403 on a valid link is still reported as a file that is gone. The expired-link, directPath, 404/410 and reupload: false cases are already green and stay green. Co-Authored-By: Claude Opus 5.5 --- test/chat/media-skip-reupload.test.ts | 33 +++++++++-- test/proxy/media-reupload.test.ts | 85 ++++++++++++++++++++++++--- 2 files changed, 105 insertions(+), 13 deletions(-) diff --git a/test/chat/media-skip-reupload.test.ts b/test/chat/media-skip-reupload.test.ts index 447630b5a3..cb60250bc8 100644 --- a/test/chat/media-skip-reupload.test.ts +++ b/test/chat/media-skip-reupload.test.ts @@ -2,8 +2,9 @@ // default true). A consumer that only wants what is still on WhatsApp's // servers sends reupload: false: Evolution does not hand Baileys a // reuploadRequest, so the phone is never asked, and a file that has expired on -// the CDN (404 or 410) fails at once, without Evolution's own 5s fallback, with -// an error that says the file is gone and no re-upload was attempted. +// the CDN (404 or 410, or 403 on a link whose `oe` has passed) fails at once, +// without Evolution's own 5s fallback, with an error that says the file is gone +// and no re-upload was attempted. // Omitting the field keeps today's behaviour: the phone is asked. import { vi } from 'vitest'; @@ -39,6 +40,11 @@ const PLAIN = Buffer.from('a photo, as the person sent it '.repeat(200)); const LIVE = '/v/t62.7118-24/reuploaded.enc'; const GONE = '/v/t62.7118-24/expired.enc'; const GONE_403 = '/v/t62.7118-24/expired-403.enc'; +/** A media link's `oe` (when it stops working): hex unix seconds, as WhatsApp writes it. */ +const oe = (fromNowS: number) => (Math.floor(Date.now() / 1000) + fromNowS).toString(16).toUpperCase(); +const DAY = 24 * 60 * 60; +const EXPIRED_LINK_403 = `${GONE_403}?ccb=11-4&oh=01_Q5Aa&oe=${oe(-DAY)}&_nc_sid=5e03e0`; +const VALID_LINK_403 = `${GONE_403}?ccb=11-4&oh=01_Q5Aa&oe=${oe(14 * DAY)}&_nc_sid=5e03e0`; const GONE_MESSAGE = "The media is no longer on WhatsApp's servers (HTTP 404), and no re-upload from the phone was attempted (reupload: false)"; @@ -60,7 +66,7 @@ beforeAll(async () => { mediaKey = enc.mediaKey; const body = await readFile(enc.encFilePath); await rm(enc.encFilePath, { force: true }); - cdn = await startCdn({ [LIVE]: body, [GONE_403]: 403 }); + cdn = await startCdn({ [LIVE]: body, [EXPIRED_LINK_403]: 403, [VALID_LINK_403]: 403 }); // Evolution waits 5s before its own fallback download; shortened, so a test that // takes it is not slow, and one that skips it is told apart by what it answers. const realSetTimeout = globalThis.setTimeout; @@ -126,19 +132,34 @@ describe('a media download can skip asking the phone to re-upload', () => { }); }); - it('reupload: false and a CDN 403 (an expired file too): the phone is not asked, and the error says the file is gone', async () => { + it('reupload: false and a CDN 403 on a link whose oe has passed: the phone is not asked, and the error says the file is gone', async () => { const message = expiredImage(); - message.message.imageMessage.url = `http://127.0.0.1:${cdn.port}${GONE_403}`; + message.message.imageMessage.url = `http://127.0.0.1:${cdn.port}${EXPIRED_LINK_403}`; const answer = await post({ message, reupload: false }); expect(asked).toEqual([]); - expect(cdn.log).toEqual([`GET ${GONE_403}`]); + expect(cdn.log).toEqual([`GET ${EXPIRED_LINK_403}`]); expect(answer).toEqual({ status: 400, body: { status: 400, error: 'Bad Request', response: { message: [GONE_MESSAGE.replace('HTTP 404', 'HTTP 403')], reupload: 'not_requested' } }, }); }); + it('reupload: false and a CDN 403 on a link whose oe has not passed: not reported as a file that is gone', async () => { + const message = expiredImage(); + message.message.imageMessage.url = `http://127.0.0.1:${cdn.port}${VALID_LINK_403}`; + const answer = await post({ message, reupload: false }); + + expect(asked).toEqual([]); + // The first GET is the download; Evolution's own fallback then tries mmg.whatsapp.net, refused here. + expect(cdn.log).toEqual([`GET ${VALID_LINK_403}`]); + // Its answer is that fallback's own failure, not the "no longer on WhatsApp's servers" 400. + expect(answer).toEqual({ + status: 400, + body: { status: 400, error: 'Bad Request', response: { message: ['TypeError: fetch failed'], reupload: 'not_requested' } }, + }); + }); + it('reupload must be a boolean', async () => { const answer = await post({ message: expiredImage(), reupload: 'no' }); diff --git a/test/proxy/media-reupload.test.ts b/test/proxy/media-reupload.test.ts index f07ba2e773..684be69dc8 100644 --- a/test/proxy/media-reupload.test.ts +++ b/test/proxy/media-reupload.test.ts @@ -9,6 +9,12 @@ // HTTP status only in `output.statusCode` (lib/Utils/messages-media.js:304), so // that check never matches and rc14 never asks. Evolution asks itself when // Baileys did not. +// +// A 403 asks the phone only when the media link itself has expired: its `oe` +// query parameter (hex unix seconds, read from the url, else the directPath) +// has passed. Measured on WhatsApp's media CDN (2026-09-27, 84 history-sync +// attachments): 403 on 34 of 34 links whose `oe` had passed, on 0 of 50 valid +// ones, and a valid link to a file the CDN dropped answered 404 or 410. import { vi } from 'vitest'; vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); @@ -22,7 +28,7 @@ import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; import { makeService } from '../helpers/baileys-service'; import { captureOutput } from '../helpers/capture-output'; -import { type Listening, startCdn } from '../helpers/local-net'; +import { type Listening, startCdn, startHttpsServer, trustTestCertificate } from '../helpers/local-net'; const PHONE = '972509876543'; const CAPTION = 'Zq7 a private caption Zq7'; @@ -33,10 +39,16 @@ const GONE = '/v/t62.7118-24/expired.enc'; const GONE_AGAIN = '/v/t62.7118-24/expired-again.enc'; const GONE_410 = '/v/t62.7118-24/expired-410.enc'; const GONE_403 = '/v/t62.7118-24/expired-403.enc'; +/** A media link's `oe` (when it stops working): hex unix seconds, as WhatsApp writes it. */ +const oe = (fromNowS: number) => (Math.floor(Date.now() / 1000) + fromNowS).toString(16).toUpperCase(); +const DAY = 24 * 60 * 60; +const EXPIRED_LINK_403 = `${GONE_403}?ccb=11-4&oh=01_Q5Aa&oe=${oe(-DAY)}&_nc_sid=5e03e0`; +const VALID_LINK_403 = `${GONE_403}?ccb=11-4&oh=01_Q5Aa&oe=${oe(14 * DAY)}&_nc_sid=5e03e0`; const BROKEN = '/v/t62.7118-24/broken.enc'; let cdn: Listening; let mediaKey: Uint8Array; +let liveBody: Buffer; // Refuse any connection that is not to 127.0.0.1: Evolution's own fallback // retries the download at mmg.whatsapp.net, which must fail here, not leave. @@ -50,8 +62,16 @@ beforeAll(async () => { const enc = await encryptedStream(PLAIN, 'image', {}); mediaKey = enc.mediaKey; const body = await readFile(enc.encFilePath); + liveBody = body; await rm(enc.encFilePath, { force: true }); - cdn = await startCdn({ [LIVE]: body, [GONE_410]: 410, [GONE_403]: 403, [BROKEN]: 500 }); + cdn = await startCdn({ + [LIVE]: body, + [GONE_410]: 410, + [GONE_403]: 403, + [EXPIRED_LINK_403]: 403, + [VALID_LINK_403]: 403, + [BROKEN]: 500, + }); // Evolution waits 5s before its own fallback download, and gives the phone a // bounded time to answer a re-upload request; both are shortened here. const realSetTimeout = globalThis.setTimeout; @@ -176,19 +196,70 @@ describe('an expired media download asks the phone to re-upload, once', () => { expect(lines).toEqual([line('outcome=reupload_requested'), line('outcome=reupload_ok')]); }); - // Empirical (2026-09-27): media from a history sync, 30 to 180 days old, answered - // 403 on a real linked phone where 24-day-old media answered 410. - it('a CDN 403 asks the phone too', async () => { + it('a CDN 403 on a link whose oe has passed asks the phone', async () => { const { service, asked } = await serviceWithPhone('reuploads'); - const { result, thrown, lines } = await download(service, GONE_403); + const { result, thrown, lines } = await download(service, EXPIRED_LINK_403); expect(thrown).toBeUndefined(); expect(Buffer.from(result.base64, 'base64').equals(PLAIN)).toBe(true); expect(asked).toEqual([ID]); - expect(cdn.log).toEqual([`GET ${GONE_403}`, `GET ${LIVE}`]); + expect(cdn.log).toEqual([`GET ${EXPIRED_LINK_403}`, `GET ${LIVE}`]); expect(lines).toEqual([line('outcome=reupload_requested'), line('outcome=reupload_ok')]); }); + it('a CDN 403 on a link whose oe has not passed does not ask the phone', async () => { + const { service, asked } = await serviceWithPhone('reuploads'); + const { thrown, out, lines } = await download(service, VALID_LINK_403); + + expect(asked).toEqual([]); + expect(cdn.log).toEqual([`GET ${VALID_LINK_403}`]); + expect(thrown).toEqual(badRequest('not_requested')); + expect(lines).toEqual([line('outcome=download_failed, status=403, reupload=not_requested')]); + expectNothingPrivate(out); + }); + + it('a CDN 403 on a link with no oe does not ask the phone', async () => { + const { service, asked } = await serviceWithPhone('reuploads'); + const { thrown, out, lines } = await download(service, GONE_403); + + expect(asked).toEqual([]); + expect(cdn.log).toEqual([`GET ${GONE_403}`]); + expect(thrown).toEqual(badRequest('not_requested')); + expect(lines).toEqual([line('outcome=download_failed, status=403, reupload=not_requested')]); + expectNothingPrivate(out); + }); + + it('a CDN 403 whose url has no oe reads it from the directPath', async () => { + // A directPath is fetched over https from the url's host, so this CDN is https. + const untrust = trustTestCertificate(); + const tlsCdn = await startHttpsServer((req, _body, res) => { + if (req.url === LIVE) return void res.writeHead(200, { 'content-length': liveBody.length }).end(liveBody); + res.writeHead(403).end(); + }); + try { + const { service } = await makeService(); + const asked: string[] = []; + service.client.updateMediaMessage = async (message: any) => { + asked.push(message.key.id); + message.message.imageMessage.directPath = LIVE; + return message; + }; + const directPath = `${GONE_403}?ccb=11-4&oh=01_Q5Aa&oe=${oe(-DAY)}&_nc_sid=5e03e0`; + const message = expiredImage(); + message.message.imageMessage.url = `https://127.0.0.1:${tlsCdn.port}${GONE_403}`; + (message.message.imageMessage as any).directPath = directPath; + + const result = await service.getBase64FromMediaMessage({ message }); + + expect(Buffer.from(result.base64, 'base64').equals(PLAIN)).toBe(true); + expect(asked).toEqual([ID]); + expect(tlsCdn.log).toEqual([`GET ${directPath}`, `GET ${LIVE}`]); + } finally { + await tlsCdn.close(); + untrust(); + } + }); + it('a re-uploaded copy that is gone as well is not re-uploaded again', async () => { const { service, asked } = await serviceWithPhone('reuploads-expired'); const { thrown, out, lines } = await download(service); From 431ebb84b9021f68652c3e0092430f0359328809 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:22:37 +0300 Subject: [PATCH 077/157] fix(baileys): ask the phone after a 403 only when the media link has expired A CDN 403 now counts as an expired file only when the media link's `oe` query parameter (hex unix seconds, read from the url, else from the directPath) has passed. 404 and 410 still count regardless, and a 403 on a link whose `oe` has not passed, or that has none, is treated like any other failure: the phone is not asked, and with reupload: false it is not reported as a file that is gone. Measured on WhatsApp's media CDN (2026-09-27, 84 history-sync attachments): 403 on 34 of 34 links whose `oe` had passed, on 0 of 50 valid ones, and a valid link to a file the CDN dropped answered 404 or 410. Asking the phone after any 403 (c09c6048) would also have asked for 403s that have another cause. Co-Authored-By: Claude Opus 5.5 --- FORK.md | 4 +- .../whatsapp/whatsapp.baileys.service.ts | 39 +++++++++++++++---- 2 files changed, 34 insertions(+), 9 deletions(-) diff --git a/FORK.md b/FORK.md index 6a25c8e4a2..9e3568bebd 100644 --- a/FORK.md +++ b/FORK.md @@ -53,9 +53,9 @@ named where one exists. - No message text, phone number, JID or push name reaches the logs at `LOG_LEVEL=ERROR,WARN`, including Baileys' own error logs. - The `messages.upsert` webhook key of a DM WhatsApp addresses by @lid shows the phone JID as `remoteJid`, as 2.3.7 does, but keeps the original @lid in `remoteJidAlt` (2.3.7 copied the phone there too and lost it), with `addressingMode: 'pn'`: upstream develop's swap. - A media re-upload request names the message by the address the phone stores it under: the @lid from `remoteJidAlt` or `participantAlt` when the key shows the phone, or, for a key stored from 2.3.7 (the phone twice, `addressingMode: 'lid'`), the @lid Baileys' LID mapping has for the phone. The phone refuses a request that names an @lid chat by its phone JID. -- A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error), and when the phone refused, why (`reason=` on the log line, `reuploadReason` on the error and the HTTP answer: the phone's result such as `NOT_FOUND`, `error_`, `missing_ciphertext`, or `no_answer`). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410 (its check misses the status), so Evolution asks the phone itself, once per download, for at most 60s. A CDN 403 counts as expired too (seen on history media 30 to 180 days old). +- A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error), and when the phone refused, why (`reason=` on the log line, `reuploadReason` on the error and the HTTP answer: the phone's result such as `NOT_FOUND`, `error_`, `missing_ciphertext`, or `no_answer`). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410 (its check misses the status), so Evolution asks the phone itself, once per download, for at most 60s. A CDN 403 counts as expired only when the media link's `oe` (hex unix seconds, from the url, else the directPath) has passed: measured on 84 history-sync attachments, 403 answered 34 of 34 expired links and 0 of 50 valid ones, where a valid link to a dropped file answered 404 or 410. - A media download turns the media key back into bytes (from a base64 string, or the object JSON makes of a Uint8Array or a Buffer) before it asks the phone to re-upload, since Baileys 7.0.0-rc14 derives the re-upload's key from the value as given and then cannot decrypt the phone's answer (Baileys #2729 proposed the same fix there). -- `POST /chat/getBase64FromMediaMessage` takes an optional `reupload` (boolean, default true). With `false` the phone is never asked to re-upload, and a file gone from the CDN (403, 404 or 410) fails at once with a 400 that says so; the HTTP error carries `reupload` for any failed download. +- `POST /chat/getBase64FromMediaMessage` takes an optional `reupload` (boolean, default true). With `false` the phone is never asked to re-upload, and a file gone from the CDN (404, 410, or 403 on an expired link) fails at once with a 400 that says so; the HTTP error carries `reupload` for any failed download. **Proxy** - Media downloads, media uploads and the WhatsApp Web version fetch leave through the instance's proxy (uploads failed outright on a proxied instance). diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 84e43a232b..d2149c5872 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -268,11 +268,33 @@ const reuploadRefusal = (error: any): string => { }; /** - * A CDN answer that means the file has expired there, and only the phone still has it. - * 403 is empirical: history-sync media 30 to 180 days old answered 403 on a real linked - * phone (2026-09-27), where 24-day-old media answered 410. + * When a WhatsApp media link stops working, in unix seconds: its `oe` query parameter + * (hex), read from the url, else from the directPath. Undefined when neither has one. */ -const isExpiredMedia = (error: any) => [403, 404, 410].includes(httpStatus(error)); +const mediaLinkExpiry = ( + media: { url?: string | null; directPath?: string | null } | undefined, +): number | undefined => { + for (const link of [media?.url, media?.directPath]) { + const oe = link?.match(/[?&]oe=([0-9a-fA-F]+)(?:&|#|$)/)?.[1]; + if (oe) return parseInt(oe, 16); + } + return undefined; +}; + +/** + * A CDN answer that means the file has expired there, and only the phone still has it: + * 404 or 410, or 403 on a link whose `oe` has passed. Measured on WhatsApp's media CDN + * (2026-09-27, 84 history-sync attachments): 403 on 34 of 34 links whose `oe` had + * passed and on 0 of 50 valid ones, where a valid link to a dropped file answered 404 + * or 410. So a 403 means the signed link expired; on a valid link it is not an expiry. + */ +const isExpiredMedia = (error: any, media: Parameters[0]) => { + const status = httpStatus(error); + if (status === 404 || status === 410) return true; + if (status !== 403) return false; + const expiry = mediaLinkExpiry(media); + return expiry !== undefined && expiry * 1000 <= Date.now(); +}; /** * How long the phone gets to answer a re-upload request. Baileys' updateMediaMessage @@ -4549,7 +4571,8 @@ export class BaileysStartupService extends ChannelStartupService { }, MEDIA_REUPLOAD_TIMEOUT_MS); }); try { - const ask = async () => this.client.updateMediaMessage({ ...message, key: await this.originalMessageKey(message.key) }); + const ask = async () => + this.client.updateMediaMessage({ ...message, key: await this.originalMessageKey(message.key) }); const updated = await Promise.race([ask(), timeout]); reupload = 'ok'; this.logger.warn(`media download: ${media}, outcome=reupload_ok`); @@ -4566,6 +4589,8 @@ export class BaileysStartupService extends ChannelStartupService { } }; const target: WAMessage = { key: msg?.key, message: msg?.message }; + // The link as first downloaded, whose `oe` says whether a 403 is an expired link. + const link = { url: msg.message[mediaType]?.url, directPath: msg.message[mediaType]?.directPath }; // No reuploadRequest for Baileys: Evolution asks the phone itself, below. Baileys // means to ask on a 404 or 410, but 7.0.0-rc14 checks error.status // (lib/Utils/messages.js:836) while its CDN fetch sets only output.statusCode @@ -4578,7 +4603,7 @@ export class BaileysStartupService extends ChannelStartupService { try { buffer = await download(target); } catch (error) { - if (!askPhone || !isExpiredMedia(error)) throw error; + if (!askPhone || !isExpiredMedia(error, link)) throw error; let refreshed: WAMessage; try { refreshed = await reuploadRequest(target); @@ -4591,7 +4616,7 @@ export class BaileysStartupService extends ChannelStartupService { } catch (error) { const status = httpStatus(error); this.logger.error(`media download: ${media}, outcome=download_failed, status=${status}, reupload=${reupload}`); - if (!askPhone && isExpiredMedia(error)) { + if (!askPhone && isExpiredMedia(error, link)) { throw `The media is no longer on WhatsApp's servers (HTTP ${status}), and no re-upload from the phone was attempted (reupload: false)`; } this.logger.error('Download Media failed, trying to retry in 5 seconds...'); From 3167ff7b06b6556718cb35dfcee8fbc6ea6efa2d Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:40:10 +0300 Subject: [PATCH 078/157] test: a failed media download never logs the media URL Co-Authored-By: Claude Opus 5.5 --- test/chat/media-download-url.test.ts | 132 +++++++++++++++++++++++++++ 1 file changed, 132 insertions(+) create mode 100644 test/chat/media-download-url.test.ts diff --git a/test/chat/media-download-url.test.ts b/test/chat/media-download-url.test.ts new file mode 100644 index 0000000000..0da41aab5c --- /dev/null +++ b/test/chat/media-download-url.test.ts @@ -0,0 +1,132 @@ +// A WhatsApp media link is signed per message: its `oh` and `oe` query +// parameters let anyone holding it fetch the (encrypted) file until it +// expires. A deployment running with LOG_LEVEL=ERROR,WARN and LOG_BAILEYS=error +// must not print one. Seen live: when the download, the re-upload and the +// fallback download all failed, Evolution logged the fallback's error object, +// whose message and data both carry https://mmg.whatsapp.net/...&oh=...&oe=... +// +// Baileys downloads from https://, so the first +// download goes to a local HTTPS CDN (the test certificate, trusted for the +// test) that answers 410. The phone refuses the re-upload with an +// answer, read by Baileys' own decodeMediaRetryNode. The fallback, which +// Evolution sends to https://mmg.whatsapp.net + directPath whatever the url +// says, is answered 410 in-process by the undici mock agent, so nothing leaves +// 127.0.0.1. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { rm } from 'node:fs/promises'; + +import { makeBaileysLogger } from '@utils/log-privacy'; +import { decodeMediaRetryNode, encryptedStream, getHttpStream } from 'baileys'; +import { getGlobalDispatcher, MockAgent, setGlobalDispatcher } from 'undici'; +import { afterAll, beforeAll, describe, expect, it } from 'vitest'; + +import { makeService } from '../helpers/baileys-service'; +import { captureOutput } from '../helpers/capture-output'; +import { type Listening, startHttpsServer, trustTestCertificate } from '../helpers/local-net'; + +const PHONE = '972509876543'; +const ID = '3EB0CCCCCCCCCCCCCCC1'; +// The shape WhatsApp's links have: a path, then ccb, oh (the signature), oe (the expiry, hex) and _nc_sid. +const DIRECT_PATH = '/v/t62.7118-24/gone.enc?ccb=11-4&oh=01_Q5Aa1wSecretSignatureXyz&oe=6A0B1C2D&_nc_sid=5e03e0'; +const LEAKS = ['http://', 'https://', 'mmg.whatsapp.net', 'oh=', 'oe=', 'Q5Aa1wSecretSignatureXyz', '_nc_sid', PHONE]; + +let cdn: Listening; +let mediaKey: Uint8Array; +let untrust: () => void; + +const previousDispatcher = getGlobalDispatcher(); +const guard = new MockAgent(); +guard.disableNetConnect(); +guard.enableNetConnect((host: string) => host.startsWith('127.0.0.1:')); + +beforeAll(async () => { + setGlobalDispatcher(guard); + const enc = await encryptedStream(Buffer.from('a photo'), 'image', {}); + mediaKey = enc.mediaKey; + await rm(enc.encFilePath, { force: true }); + untrust = trustTestCertificate(); + cdn = await startHttpsServer((_req, _body, res) => void res.writeHead(410).end()); + // Evolution waits 5s before its fallback download; shortened here. + const realSetTimeout = globalThis.setTimeout; + vi.spyOn(globalThis, 'setTimeout').mockImplementation(((fn: any, ms?: number, ...args: any[]) => + realSetTimeout(fn, ms === 5000 ? 0 : ms, ...args)) as any); +}); + +afterAll(async () => { + vi.restoreAllMocks(); + await cdn.close(); + untrust(); + setGlobalDispatcher(previousDispatcher); + await guard.close(); +}); + +/** Every line of `out` that carries any part of a media link, stripped of colour and truncated. */ +function leaks(out: string) { + return out + .replace(/\x1b\[[0-9;]*m/g, '') + .split('\n') + .filter((line) => LEAKS.some((l) => line.includes(l))) + .map((line) => line.trim().slice(0, 200)); +} + +const cdnGone = () => guard.get('https://mmg.whatsapp.net').intercept({ path: DIRECT_PATH, method: 'GET' }).reply(410); + +describe('a failed media download never logs the media URL', () => { + it('download, re-upload and fallback all fail: no link, signature or expiry in the output', async () => { + const { service } = await makeService(); + // The phone's answer, as updateMediaMessage throws it. + const rmr = { tag: 'rmr', attrs: { jid: `${PHONE}@s.whatsapp.net`, from_me: 'false' } }; + service.client.updateMediaMessage = async () => { + throw (decodeMediaRetryNode({ tag: 'receipt', attrs: { id: ID }, content: [rmr, { tag: 'error', attrs: { code: '2' } }] } as any) as any).error; + }; + cdnGone(); + const message = { + key: { remoteJid: `${PHONE}@s.whatsapp.net`, fromMe: false, id: ID }, + message: { + imageMessage: { url: `https://127.0.0.1:${cdn.port}${DIRECT_PATH}`, directPath: DIRECT_PATH, mediaKey, mimetype: 'image/jpeg' }, + }, + }; + + let thrown: any; + const out = await captureOutput(async () => { + try { + await service.getBase64FromMediaMessage({ message }); + } catch (e) { + thrown = e; + } + }); + + expect(cdn.log).toEqual([`GET ${DIRECT_PATH}`]); + guard.assertNoPendingInterceptors(); // the fallback asked mmg.whatsapp.net, and got its 410 + expect(thrown?.status).toBe(400); + expect(leaks(out)).toEqual([]); + // The download's own lines are unchanged; the fallback and the final failure each get a line of bounded fields. + const plain = out.replace(/\x1b\[[0-9;]*m/g, '').split('\n'); + const from = (prefix: string) => plain.filter((l) => l.includes(prefix)).map((l) => l.slice(l.indexOf(prefix)).trim()); + expect(from('media download:')).toEqual([ + `media download: message=${ID}, chat=user, outcome=reupload_requested`, + `media download: message=${ID}, chat=user, outcome=reupload_failed, error=Error, status=404, reason=error_2`, + `media download: message=${ID}, chat=user, outcome=download_failed, status=410, reupload=failed`, + ]); + expect(from('media fallback:')).toEqual([`media fallback: message=${ID}, chat=user, outcome=failed, error=Error, status=410`]); + expect(from('media processing failed:')).toEqual([`media processing failed: message=${ID}, chat=user, error=Error, status=410`]); + }); + + it('the same error in a Baileys log line (the logger Evolution hands Baileys) is printed without its link', async () => { + cdnGone(); + const error = await getHttpStream(`https://mmg.whatsapp.net${DIRECT_PATH}`).catch((e) => e); + expect(error?.message).toBe(`Failed to fetch stream from https://mmg.whatsapp.net${DIRECT_PATH}`); + + const out = await captureOutput(async () => { + const logger = makeBaileysLogger('error'); + logger.error({ err: error, key: { remoteJid: `${PHONE}@s.whatsapp.net`, id: ID } }, `failed to download ${error.message}`); + await new Promise((r) => setTimeout(r, 20)); // pino's async write + }); + + expect(out).toContain('Failed to fetch stream from'); + expect(leaks(out)).toEqual([]); + }); +}); From db182c869079d9570f01388d356a96bec464513b Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:41:56 +0300 Subject: [PATCH 079/157] fix(baileys): keep media URLs out of the logs The fallback download's error carries the signed media link in its message and data.url, and the final catch printed the whole error object. Both sites now log bounded fields (message id, chat kind, error name, HTTP status), and scrub() masks any URL, so the logger Evolution hands Baileys drops them too. Co-Authored-By: Claude Opus 5.5 --- .../channel/whatsapp/whatsapp.baileys.service.ts | 15 ++++++++++++--- src/utils/log-privacy.ts | 7 ++++++- 2 files changed, 18 insertions(+), 4 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index d2149c5872..b1465c1065 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -251,6 +251,10 @@ type MediaReupload = 'not_requested' | 'ok' | 'failed'; /** The HTTP status a Baileys media error carries (a Boom's output.statusCode), or 'none'. */ const httpStatus = (error: any) => error?.output?.statusCode ?? error?.status ?? 'none'; +/** What kind of error was thrown, never what it says: a thrown string is 'string'. */ +const errorName = (error: any): string => + typeof error === 'string' ? 'string' : typeof error?.name === 'string' ? error.name.slice(0, 40) : 'unknown'; + /** * Why the phone refused a re-upload, from the error Baileys' updateMediaMessage threw: * the phone's MediaRetryNotification result (NOT_FOUND, DECRYPTION_ERROR, GENERAL_ERROR), @@ -4641,7 +4645,10 @@ export class BaileysStartupService extends ChannelStartupService { buffer = Buffer.concat(chunks); this.logger.info('Download Media with downloadContentFromMessage was successful!'); } catch (fallbackErr) { - this.logger.error('Download Media with downloadContentFromMessage also failed!'); + // Its error carries the signed media URL (message and data.url): name and status only. + this.logger.error( + `media fallback: ${media}, outcome=failed, error=${errorName(fallbackErr)}, status=${httpStatus(fallbackErr)}`, + ); throw fallbackErr; } } @@ -4688,8 +4695,10 @@ export class BaileysStartupService extends ChannelStartupService { buffer: getBuffer ? buffer : null, }; } catch (error) { - this.logger.error('Error processing media message:'); - this.logger.error(error); + const key = data?.message?.key; + this.logger.error( + `media processing failed: message=${key?.id}, chat=${jidKind(key?.remoteJid)}, error=${errorName(error)}, status=${httpStatus(error)}`, + ); if (reupload === undefined) throw new BadRequestException(error.toString()); // The same 400, plus whether the phone was asked to re-upload the file, and why it refused. try { diff --git a/src/utils/log-privacy.ts b/src/utils/log-privacy.ts index d47edc5d24..ec209be659 100644 --- a/src/utils/log-privacy.ts +++ b/src/utils/log-privacy.ts @@ -15,9 +15,14 @@ export function jidKind(jid: unknown): string { return 'other'; } -/** Mask anything in a diagnostic string that looks like a JID or a phone number. */ +/** + * Mask anything in a diagnostic string that looks like a URL, a JID or a phone + * number. A URL goes first, and whole: a WhatsApp media link is signed per + * message (`oh`, `oe` in its query), and whoever holds it can fetch the file. + */ export function scrub(value: unknown): string { return String(value ?? '') + .replace(/[a-z][a-z0-9+.-]*:\/\/[^\s'"<>]+/gi, '[url]') .replace(/[^\s'"<>=,;:()[\]{}]+@[^\s'"<>=,;:()[\]{}]+/g, '[jid]') .replace(/\d{6,}/g, '[number]'); } From c8fdc03a79879527b2b0d44e6e5e7fe79750800c Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:43:04 +0300 Subject: [PATCH 080/157] test: a failed reconnect says why Co-Authored-By: Claude Opus 5.5 --- test/connect/reconnect-failure-log.test.ts | 101 +++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 test/connect/reconnect-failure-log.test.ts diff --git a/test/connect/reconnect-failure-log.test.ts b/test/connect/reconnect-failure-log.test.ts new file mode 100644 index 0000000000..36b7bb1c79 --- /dev/null +++ b/test/connect/reconnect-failure-log.test.ts @@ -0,0 +1,101 @@ +// When a reconnect attempt fails before a socket exists (the network is down, +// the credentials cannot be read), Evolution logs it and schedules the next +// one. Seen live during an outage: the line said +// `{ message: 'Reconnect attempt failed', error: '[object Object]' }`, every +// time, so the operator could not tell why. The line must say what failed: +// the error's name, its message (scrubbed: no URL, JID or phone number) and +// its status code when it has one. Two shapes are thrown in practice: a Boom +// (an Error with output.statusCode) and a plain object that is not an Error. +import { vi } from 'vitest'; + +const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + return { ...orig, default: socketSpy, makeWASocket: socketSpy }; +}); +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); + +import { Boom } from '@hapi/boom'; +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; + +import { makeService } from '../helpers/baileys-service'; +import { captureOutput } from '../helpers/capture-output'; +import { fakeSocket, stubAuthState } from '../helpers/connect'; + +socketSpy.mockImplementation(fakeSocket); + +const PHONE = '972509876543'; +const URL_SECRET = 'https://web.whatsapp.com/check?token=Zq7secret'; + +beforeEach(() => { + socketSpy.mockClear(); + vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout', 'setInterval', 'clearInterval', 'Date'] }); +}); + +afterEach(() => { + vi.useRealTimers(); +}); + +async function flush() { + for (let i = 0; i < 20; i++) await new Promise((r) => setImmediate(r)); +} + +/** Connect, close with a reconnectable code, and let the first reconnect attempt throw `thrown`. */ +async function failedReconnect(thrown: unknown) { + const { service } = await makeService(); + stubAuthState(service); + await service.connectToWhatsapp(); + const read = service.defineAuthState; + let failures = 1; + service.defineAuthState = async () => { + if (failures-- > 0) throw thrown; + return read(); + }; + const out = await captureOutput(async () => { + const socket = socketSpy.mock.results[socketSpy.mock.results.length - 1].value; + socket.ev.emit('connection.update', { + connection: 'close', + lastDisconnect: { error: new Boom('closed', { statusCode: 503 }), date: new Date() }, + }); + await flush(); + await vi.advanceTimersByTimeAsync(1_000); + await flush(); + }); + const plain = out.replace(/\x1b\[[0-9;]*m/g, ''); + const lines = plain.split('\n').filter((l) => l.includes('Reconnect attempt failed')); + return { plain, logged: lines.map((l) => JSON.parse(l.slice(l.indexOf('{')))) }; +} + +describe('a failed reconnect says why', () => { + it('a Boom: its name, scrubbed message and status code', async () => { + const { plain, logged } = await failedReconnect( + new Boom(`request to ${URL_SECRET} for ${PHONE}@s.whatsapp.net failed, reason: getaddrinfo ENOTFOUND`, { statusCode: 503 }), + ); + + expect(logged).toEqual([ + { + message: 'Reconnect attempt failed', + error: { name: 'Error', message: 'request to [url] for [jid] failed, reason: getaddrinfo ENOTFOUND', statusCode: 503 }, + }, + ]); + expect(plain).not.toContain(PHONE); + expect(plain).not.toContain('Zq7secret'); + }); + + it('a plain object that is not an Error: its message, scrubbed, and no status code when it has none', async () => { + const { plain, logged } = await failedReconnect({ code: 'ECONNREFUSED', message: `connect ECONNREFUSED to ${URL_SECRET} as ${PHONE}` }); + + expect(logged).toEqual([ + { + message: 'Reconnect attempt failed', + error: { name: 'Object', message: 'connect ECONNREFUSED to [url] as [number]' }, + }, + ]); + expect(plain).not.toContain(PHONE); + expect(plain).not.toContain('Zq7secret'); + }); +}); From 7eb95c9ca5c01b413d33c762226160081954985a Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:44:32 +0300 Subject: [PATCH 081/157] fix(baileys): log why a reconnect attempt failed openConnection wrapped every failure in InternalServerErrorException, which throws a plain object, and the reconnect loop logged its toString(): '[object Object]'. The wrapper now keeps the original error as a non-enumerable cause (the HTTP answer is unchanged), and both the connect and the reconnect lines log errorFields(): name, scrubbed message, status code when there is one. The connect line printed the raw error before, message and all. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 13 ++++++++---- src/utils/log-privacy.ts | 20 +++++++++++++++++++ 2 files changed, 29 insertions(+), 4 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index b1465c1065..d17b183e9d 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -84,7 +84,7 @@ import { Instance, Message } from '@prisma/client'; import { chatState } from '@utils/chat-state'; import { createJid } from '@utils/createJid'; import { fetchLatestWaWebVersion } from '@utils/fetchLatestWaWebVersion'; -import { jidKind, makeBaileysLogger } from '@utils/log-privacy'; +import { errorFields, jidKind, makeBaileysLogger } from '@utils/log-privacy'; import { readLogoutMarker, writeLogoutMarker } from '@utils/logout-marker'; import { makeProxyAgent, makeProxyAgentUndici } from '@utils/makeProxyAgent'; import { getOnWhatsappCache, saveOnWhatsappCache } from '@utils/onWhatsappCache'; @@ -1145,8 +1145,13 @@ export class BaileysStartupService extends ChannelStartupService { return await this.createClient(number); } catch (error) { - this.logger.error(error); - throw new InternalServerErrorException(error?.toString()); + this.logger.error({ message: 'Connect failed', error: errorFields(error) }); + // The same 500, still carrying what failed (not enumerable, so not in an HTTP answer): a reconnect logs it. + try { + new InternalServerErrorException(error?.toString()); + } catch (serverError) { + throw Object.defineProperty(serverError, 'cause', { value: error, enumerable: false }); + } } } @@ -1164,7 +1169,7 @@ export class BaileysStartupService extends ChannelStartupService { await this.connect(this.phoneNumber); } catch (error) { // No socket was built, so no close will come to retry it: schedule the next attempt here. - this.logger.error({ message: 'Reconnect attempt failed', error: error?.toString() }); + this.logger.error({ message: 'Reconnect attempt failed', error: errorFields(error?.cause ?? error) }); this.scheduleReconnect(statusCode); } }, delay); diff --git a/src/utils/log-privacy.ts b/src/utils/log-privacy.ts index ec209be659..d94e796b32 100644 --- a/src/utils/log-privacy.ts +++ b/src/utils/log-privacy.ts @@ -27,6 +27,26 @@ export function scrub(value: unknown): string { .replace(/\d{6,}/g, '[number]'); } +/** + * What an operator needs to know about a thrown value, bounded and scrubbed: + * its name (for a value that is not an Error, what it is: Object, string...), + * its message, and its HTTP status code when it carries one (a Boom's + * output.statusCode, or statusCode / status). + */ +export function errorFields(error: unknown): { name: string; message: string; statusCode?: number } { + const e: any = error; + let name: string = typeof e; + if (typeof e?.name === 'string') name = e.name; + else if (e !== null && typeof e === 'object') name = e.constructor?.name ?? 'Object'; + const message = typeof e?.message === 'string' ? e.message : typeof e === 'string' ? e : ''; + const statusCode = [e?.output?.statusCode, e?.statusCode, e?.status].find((v) => typeof v === 'number'); + return { + name: scrub(name).slice(0, 40), + message: scrub(message).slice(0, 200), + ...(statusCode !== undefined && { statusCode }), + }; +} + const PRIMITIVE_FIELDS = ['messageType', 'isSessionRecordError', 'opName', 'count', 'attempt', 'retryCount']; const ERROR_FIELDS = ['err', 'error', 'ackErr']; From ce65a3fca98af738690d66f476a2614f47068adb Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:46:55 +0300 Subject: [PATCH 082/157] test: a failed incoming media conversion never logs the media URL Co-Authored-By: Claude Opus 5.5 --- test/chat/media-base64-log.test.ts | 109 +++++++++++++++++++++++++++++ 1 file changed, 109 insertions(+) create mode 100644 test/chat/media-base64-log.test.ts diff --git a/test/chat/media-base64-log.test.ts b/test/chat/media-base64-log.test.ts new file mode 100644 index 0000000000..39e1dcde46 --- /dev/null +++ b/test/chat/media-base64-log.test.ts @@ -0,0 +1,109 @@ +// With webhookBase64 on, Evolution downloads a message's media to put it in +// the webhook. When that download fails, the error Baileys throws says +// "Failed to fetch stream from ": its `oh` and `oe` +// let anyone holding it fetch the file until it expires. The log line for the +// failure must say which message and why, never the link. +// +// Baileys downloads from https://, so the file is +// on a local HTTPS CDN (the test certificate, trusted for the test) that +// answers 410. The undici mock agent refuses anything not on 127.0.0.1. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { rm } from 'node:fs/promises'; + +import { encryptedStream } from 'baileys'; +import { getGlobalDispatcher, MockAgent, setGlobalDispatcher } from 'undici'; +import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; + +import { deliver, makeService } from '../helpers/baileys-service'; +import { captureOutput } from '../helpers/capture-output'; +import { emitted } from '../helpers/fake-server-module'; +import { type Listening, startHttpsServer, trustTestCertificate } from '../helpers/local-net'; +import type { Profile } from '../helpers/profiles'; + +const PHONE = '972509876543'; +const DIRECT_PATH = '/v/t62.7118-24/gone.enc?ccb=11-4&oh=01_Q5Aa1wSecretSignatureXyz&oe=6A0B1C2D&_nc_sid=5e03e0'; +const LEAKS = ['http://', 'https://', 'mmg.whatsapp.net', 'oh=', 'oe=', 'Q5Aa1wSecretSignatureXyz', '_nc_sid', PHONE]; + +let cdn: Listening; +let mediaKey: Uint8Array; +let untrust: () => void; + +const previousDispatcher = getGlobalDispatcher(); +const guard = new MockAgent(); +guard.disableNetConnect(); +guard.enableNetConnect((host: string) => host.startsWith('127.0.0.1:')); + +beforeAll(async () => { + setGlobalDispatcher(guard); + const enc = await encryptedStream(Buffer.from('a photo'), 'image', {}); + mediaKey = enc.mediaKey; + await rm(enc.encFilePath, { force: true }); + untrust = trustTestCertificate(); + cdn = await startHttpsServer((_req, _body, res) => void res.writeHead(410).end()); +}); + +afterAll(async () => { + await cdn.close(); + untrust(); + setGlobalDispatcher(previousDispatcher); + await guard.close(); +}); + +beforeEach(() => { + cdn.log.splice(0); + emitted.splice(0); +}); + +/** Every line of `out` that carries any part of a media link, stripped of colour and truncated. */ +function leaks(out: string) { + return out + .replace(/\x1b\[[0-9;]*m/g, '') + .split('\n') + .filter((line) => LEAKS.some((l) => line.includes(l))) + .map((line) => line.trim().slice(0, 200)); +} + +/** The log objects printed for a failed base64 conversion. */ +function conversionLines(out: string) { + return out + .replace(/\x1b\[[0-9;]*m/g, '') + .split('\n') + .filter((l) => l.includes('Error converting media to base64')) + .map((l) => JSON.parse(l.slice(l.indexOf('{')))); +} + +describe.each(['minimal', 'stored'] as Profile[])('a failed media conversion never logs the media URL (profile %s)', (profile) => { + it('an incoming image whose download fails', async () => { + const { service, ev } = await makeService({ profile }); + Object.assign(service.localWebhook, { enabled: true, webhookBase64: true }); + const id = `3EB0DDDDDDDDDDDDDD-${profile}`; + const incoming = { + key: { remoteJid: `${PHONE}@s.whatsapp.net`, fromMe: false, id }, + message: { + imageMessage: { url: `https://127.0.0.1:${cdn.port}${DIRECT_PATH}`, directPath: DIRECT_PATH, mediaKey, mimetype: 'image/jpeg' }, + }, + messageTimestamp: 1_700_000_000, + pushName: 'Sender', + }; + + const out = await captureOutput(() => deliver(service, ev, { 'messages.upsert': { messages: [incoming], type: 'notify' } })); + + expect(cdn.log).toEqual([`GET ${DIRECT_PATH}`]); + // The message still reaches the webhook, without the media. + const upsert = emitted.find((e) => e.event === 'messages.upsert'); + expect(upsert?.data?.key?.id).toBe(id); + expect(upsert?.data?.message?.base64).toBeUndefined(); + expect(leaks(out)).toEqual([]); + expect(conversionLines(out)).toEqual([ + { + message: 'Error converting media to base64', + messageId: id, + chatType: 'user', + error: { name: 'Error', message: 'Failed to fetch stream from [url]', statusCode: 410 }, + }, + ]); + }); +}); From 6780ae8237fa93ebf2bf0bcb779a1b8357709e44 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:47:27 +0300 Subject: [PATCH 083/157] fix(baileys): keep the media URL out of the base64 conversion log With webhookBase64 on, a failed download of an incoming message's media logged the error's message, which is "Failed to fetch stream from" the signed media link. The line now carries the message id, the chat kind and errorFields(): name, scrubbed message, status code. Co-Authored-By: Claude Opus 5.5 --- .../channel/whatsapp/whatsapp.baileys.service.ts | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index d17b183e9d..c172ca6677 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -1943,7 +1943,13 @@ export class BaileysStartupService extends ChannelStartupService { } } } catch (error) { - this.logger.error(['Error converting media to base64', error?.message]); + // A failed download's error names the signed media link: bounded, scrubbed fields only. + this.logger.error({ + message: 'Error converting media to base64', + messageId: received.key?.id, + chatType: jidKind(received.key?.remoteJid), + error: errorFields(error), + }); } } } From 639ab6eaf446991f11de2aaa66017f0168925f1b Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:48:17 +0300 Subject: [PATCH 084/157] test: a failed sent media conversion never logs the media URL Co-Authored-By: Claude Opus 5.5 --- test/chat/media-base64-log.test.ts | 45 ++++++++++++++++++++++++++++-- 1 file changed, 43 insertions(+), 2 deletions(-) diff --git a/test/chat/media-base64-log.test.ts b/test/chat/media-base64-log.test.ts index 39e1dcde46..fc44a15e9d 100644 --- a/test/chat/media-base64-log.test.ts +++ b/test/chat/media-base64-log.test.ts @@ -4,6 +4,9 @@ // let anyone holding it fetch the file until it expires. The log line for the // failure must say which message and why, never the link. // +// Both directions: an incoming message's media, and a sent message's (Evolution +// downloads what it just uploaded, from the link WhatsApp's upload answer gave). +// // Baileys downloads from https://, so the file is // on a local HTTPS CDN (the test certificate, trusted for the test) that // answers 410. The undici mock agent refuses anything not on 127.0.0.1. @@ -13,11 +16,11 @@ vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); import { rm } from 'node:fs/promises'; -import { encryptedStream } from 'baileys'; +import { encryptedStream, generateWAMessage } from 'baileys'; import { getGlobalDispatcher, MockAgent, setGlobalDispatcher } from 'undici'; import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; -import { deliver, makeService } from '../helpers/baileys-service'; +import { deliver, makeService, WUID } from '../helpers/baileys-service'; import { captureOutput } from '../helpers/capture-output'; import { emitted } from '../helpers/fake-server-module'; import { type Listening, startHttpsServer, trustTestCertificate } from '../helpers/local-net'; @@ -106,4 +109,42 @@ describe.each(['minimal', 'stored'] as Profile[])('a failed media conversion nev }, ]); }); + + it('a sent document whose download fails', async () => { + const { service } = await makeService({ profile }); + Object.assign(service.localWebhook, { enabled: true, webhookBase64: true }); + const id = `3EB0EEEEEEEEEEEEEE-${profile}`; + Object.assign(service.client, { + onWhatsApp: async (...jids: string[]) => jids.map((jid) => ({ exists: true, jid })), + // WhatsApp's answer to an upload: where the encrypted file now is. + waUploadToServer: async () => ({ mediaUrl: `https://127.0.0.1:${cdn.port}${DIRECT_PATH}`, directPath: DIRECT_PATH }), + // What the socket sends and returns, built by Baileys (Evolution sends media as a { forward }). + sendMessage: (jid: string, content: any) => generateWAMessage(jid, content, { userJid: WUID, messageId: id } as any), + presenceSubscribe: async () => undefined, + sendPresenceUpdate: async () => undefined, + }); + + let sent: any; + const out = await captureOutput(async () => { + sent = await service.mediaMessage({ + number: PHONE, + mediatype: 'document', + mimetype: 'application/pdf', + fileName: 'a.pdf', + media: Buffer.from('%PDF-1.4 a document').toString('base64'), + }); + }); + + expect(sent?.key?.id).toBe(id); + expect(cdn.log).toEqual([`GET ${DIRECT_PATH}`]); + expect(leaks(out)).toEqual([]); + expect(conversionLines(out)).toEqual([ + { + message: 'Error converting media to base64', + messageId: id, + chatType: 'user', + error: { name: 'Error', message: 'Failed to fetch stream from [url]', statusCode: 410 }, + }, + ]); + }); }); From 6850a9b2f0ee13df7b47c5e28c4448fc868c5e04 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:48:47 +0300 Subject: [PATCH 085/157] fix(baileys): keep the media URL out of the sent media conversion log The same leak as the incoming path, on a sent message: with webhookBase64 on, a failed download of the media just sent logged the signed link. Same bounded fields. Co-Authored-By: Claude Opus 5.5 --- .../channel/whatsapp/whatsapp.baileys.service.ts | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index c172ca6677..11f2c7ceed 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -3142,7 +3142,13 @@ export class BaileysStartupService extends ChannelStartupService { } } } catch (error) { - this.logger.error(['Error converting media to base64', error?.message]); + // A failed download's error names the signed media link: bounded, scrubbed fields only. + this.logger.error({ + message: 'Error converting media to base64', + messageId: messageRaw.key?.id, + chatType: jidKind(messageRaw.key?.remoteJid), + error: errorFields(error), + }); } } } From 63c5170e4915ac2ba11008753c8aad1bb26990df Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:49:16 +0300 Subject: [PATCH 086/157] test: a failed connect for a pending logout says why Co-Authored-By: Claude Opus 5.5 --- .../pending-logout-connect-log.test.ts | 76 +++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 test/connect/pending-logout-connect-log.test.ts diff --git a/test/connect/pending-logout-connect-log.test.ts b/test/connect/pending-logout-connect-log.test.ts new file mode 100644 index 0000000000..f5851b1ae5 --- /dev/null +++ b/test/connect/pending-logout-connect-log.test.ts @@ -0,0 +1,76 @@ +// A logout that could not reach WhatsApp stays pending across a restart (a +// marker in the instance's directory); on boot Evolution connects only to +// deliver it. When that connect fails (the network is still down), the line +// it logs must say why, as the reconnect loop's does: the error's name, its +// message (scrubbed: no URL, JID or phone number) and its status code when it +// has one. It printed the thrown value's toString(), which for the 500 that +// openConnection throws is "[object Object]". +import { vi } from 'vitest'; + +const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); +const tmp = await vi.hoisted(async () => { + const { mkdtempSync } = await import('node:fs'); + const { tmpdir } = await import('node:os'); + const { join } = await import('node:path'); + return mkdtempSync(join(tmpdir(), 'evo-pending-log-')); +}); + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('@config/path.config', async (importOriginal) => ({ ...(await importOriginal()), INSTANCE_DIR: tmp })); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + return { ...orig, default: socketSpy, makeWASocket: socketSpy }; +}); +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); + +import { rmSync } from 'node:fs'; + +import { Boom } from '@hapi/boom'; +import { writeLogoutMarker } from '@utils/logout-marker'; +import { afterAll, describe, expect, it } from 'vitest'; + +import { makeService } from '../helpers/baileys-service'; +import { captureOutput } from '../helpers/capture-output'; +import { fakeSocket } from '../helpers/connect'; + +socketSpy.mockImplementation(fakeSocket); + +const PHONE = '972509876543'; +const URL_SECRET = 'https://web.whatsapp.com/check?token=Zq7secret'; + +afterAll(() => rmSync(tmp, { recursive: true, force: true })); + +describe('a failed connect for a pending logout says why', () => { + it('a Boom: its name, scrubbed message and status code', async () => { + await writeLogoutMarker('inst-1', { instanceName: 'test', deleted: false, since: new Date(0).toISOString() }); + const { service } = await makeService(); + service.defineAuthState = async () => { + throw new Boom(`request to ${URL_SECRET} for ${PHONE}@s.whatsapp.net failed, reason: getaddrinfo ENOTFOUND`, { + statusCode: 503, + }); + }; + + let resumed: boolean | undefined; + const out = await captureOutput(async () => { + resumed = await service.resumePendingLogout(); + }); + service.stopReconnecting(); + + expect(resumed).toBe(true); + const plain = out.replace(/\x1b\[[0-9;]*m/g, ''); + const logged = plain + .split('\n') + .filter((l) => l.includes('Connect for a pending logout failed')) + .map((l) => JSON.parse(l.slice(l.indexOf('{')))); + expect(logged).toEqual([ + { + message: 'Connect for a pending logout failed', + error: { name: 'Error', message: 'request to [url] for [jid] failed, reason: getaddrinfo ENOTFOUND', statusCode: 503 }, + }, + ]); + expect(plain).not.toContain(PHONE); + expect(plain).not.toContain('Zq7secret'); + }); +}); From a24f9283c5ce158307c49061c2b0b102136d1e72 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:49:41 +0300 Subject: [PATCH 087/157] fix(baileys): log why a connect for a pending logout failed Same as the reconnect loop: it logged the toString() of the 500 that openConnection throws, "[object Object]". It now logs errorFields() of the original error. Co-Authored-By: Claude Opus 5.5 --- .../integrations/channel/whatsapp/whatsapp.baileys.service.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 11f2c7ceed..4f142c8610 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -454,7 +454,7 @@ export class BaileysStartupService extends ChannelStartupService { try { await this.connect(this.phoneNumber); } catch (error) { - this.logger.error({ message: 'Connect for a pending logout failed', error: error?.toString() }); + this.logger.error({ message: 'Connect for a pending logout failed', error: errorFields(error?.cause ?? error) }); this.scheduleReconnect(); } return true; From f20c85b2b28fdff2d8212aef16fbfe991c823c5f Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:50:08 +0300 Subject: [PATCH 088/157] test: a failed connection reload says why, scrubbed Co-Authored-By: Claude Opus 5.5 --- test/connect/reconnect-failure-log.test.ts | 33 ++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/test/connect/reconnect-failure-log.test.ts b/test/connect/reconnect-failure-log.test.ts index 36b7bb1c79..1893155308 100644 --- a/test/connect/reconnect-failure-log.test.ts +++ b/test/connect/reconnect-failure-log.test.ts @@ -6,6 +6,8 @@ // the error's name, its message (scrubbed: no URL, JID or phone number) and // its status code when it has one. Two shapes are thrown in practice: a Boom // (an Error with output.statusCode) and a plain object that is not an Error. +// reloadConnection (after a privacy or profile picture change) rebuilds the +// socket the same way and printed the raw error, message and all. import { vi } from 'vitest'; const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); @@ -99,3 +101,34 @@ describe('a failed reconnect says why', () => { expect(plain).not.toContain('Zq7secret'); }); }); + +describe('a failed reload says why, scrubbed', () => { + it('a Boom: its name, scrubbed message and status code, and nothing unscrubbed anywhere', async () => { + const { service } = await makeService(); + service.defineAuthState = async () => { + throw new Boom(`request to ${URL_SECRET} for ${PHONE}@s.whatsapp.net failed, reason: getaddrinfo ENOTFOUND`, { + statusCode: 503, + }); + }; + + let thrown: any; + const out = await captureOutput(async () => { + await service.reloadConnection().catch((e: any) => (thrown = e)); + }); + + expect(thrown?.status).toBe(500); + const plain = out.replace(/\x1b\[[0-9;]*m/g, ''); + expect(plain).not.toContain(PHONE); + expect(plain).not.toContain('Zq7secret'); + const logged = plain + .split('\n') + .filter((l) => l.includes('Reload connection failed')) + .map((l) => JSON.parse(l.slice(l.indexOf('{')))); + expect(logged).toEqual([ + { + message: 'Reload connection failed', + error: { name: 'Error', message: 'request to [url] for [jid] failed, reason: getaddrinfo ENOTFOUND', statusCode: 503 }, + }, + ]); + }); +}); From 59a7b82bafd14da041840c027b32e38b4e240668 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:50:26 +0300 Subject: [PATCH 089/157] fix(baileys): log why a connection reload failed, scrubbed reloadConnection printed the raw error, message and all. It now logs errorFields(), as the connect and reconnect lines do. Co-Authored-By: Claude Opus 5.5 --- .../integrations/channel/whatsapp/whatsapp.baileys.service.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 4f142c8610..0d227eec3a 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -1187,7 +1187,7 @@ export class BaileysStartupService extends ChannelStartupService { try { return await this.createClient(this.phoneNumber); } catch (error) { - this.logger.error(error); + this.logger.error({ message: 'Reload connection failed', error: errorFields(error) }); throw new InternalServerErrorException(error?.toString()); } } From 96e45fc9fd9dae7958662248e338f3aae1d5eb0e Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:35:46 +0300 Subject: [PATCH 090/157] test: the live-record codec round-trips bytes, Longs, undefined, protobuf classes and Boom errors Co-Authored-By: Claude Opus 5.5 --- test/live/codec.test.ts | 80 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) create mode 100644 test/live/codec.test.ts diff --git a/test/live/codec.test.ts b/test/live/codec.test.ts new file mode 100644 index 0000000000..9678494fb3 --- /dev/null +++ b/test/live/codec.test.ts @@ -0,0 +1,80 @@ +// A live-check tape stores Baileys events as JSON lines, and a replay must hand +// Evolution the same values the socket did. JSON alone loses exactly the +// shapes Evolution branches on: a Buffer and a Uint8Array both become base64 +// (or an index object), a Long becomes {low, high, unsigned}, an undefined +// field disappears (Object.assign in the event buffer overwrites with it), and +// a protobuf instance becomes a plain object (the webhook serializer calls +// toJSON on instances only). +import { Boom } from '@hapi/boom'; +import { proto } from 'baileys'; +import Long from 'long'; +import { describe, expect, it } from 'vitest'; + +import { decode, encode } from '@utils/live-record/codec'; + +/** Through a file and back: encode, JSON text, parse, decode. */ +const roundTrip = (value: any) => decode(JSON.parse(JSON.stringify(encode(value)))); + +describe('live-record codec', () => { + it('keeps a Buffer a Buffer and a Uint8Array a Uint8Array, byte for byte', () => { + const back = roundTrip({ buf: Buffer.from([1, 2, 3]), u8: new Uint8Array([250, 0, 7]) }); + expect(Buffer.isBuffer(back.buf)).toBe(true); + expect([...back.buf]).toEqual([1, 2, 3]); + expect(back.u8).toBeInstanceOf(Uint8Array); + expect(Buffer.isBuffer(back.u8)).toBe(false); + expect([...back.u8]).toEqual([250, 0, 7]); + }); + + it('keeps a Long a Long, with its value and signedness', () => { + const back = roundTrip({ ts: Long.fromString('1758873600', true), big: Long.fromString('-9007199254740993') }); + expect(Long.isLong(back.ts)).toBe(true); + expect(back.ts.toString()).toBe('1758873600'); + expect(back.ts.unsigned).toBe(true); + expect(Long.isLong(back.big)).toBe(true); + expect(back.big.toString()).toBe('-9007199254740993'); + expect(back.big.unsigned).toBe(false); + }); + + it('keeps an explicit undefined field, in objects and arrays', () => { + const back = roundTrip({ a: 1, gone: undefined, list: [undefined, 2] }); + expect(Object.keys(back)).toEqual(['a', 'gone', 'list']); + expect(back.gone).toBeUndefined(); + expect(back.list).toEqual([undefined, 2]); + expect(back.list.length).toBe(2); + }); + + it('rebuilds protobuf classes, nested ones included, with their fields as they were', () => { + const message = proto.Message.fromObject({ + imageMessage: { mimetype: 'image/jpeg', mediaKey: new Uint8Array([9, 8, 7]), fileLength: Long.fromNumber(48213, true) }, + }); + const info = proto.WebMessageInfo.fromObject({ + key: { remoteJid: '972500000001@s.whatsapp.net', fromMe: false, id: '3EB0000000000000000001' }, + messageTimestamp: Long.fromNumber(1758873600, true), + status: proto.WebMessageInfo.Status.SERVER_ACK, + }); + info.message = message; + + const back = roundTrip({ info }); + expect(back.info).toBeInstanceOf(proto.WebMessageInfo); + expect(back.info.key).toBeInstanceOf(proto.MessageKey); + expect(back.info.message).toBeInstanceOf(proto.Message); + expect(back.info.message.imageMessage).toBeInstanceOf(proto.Message.ImageMessage); + expect(back.info.status).toBe(proto.WebMessageInfo.Status.SERVER_ACK); + expect(Long.isLong(back.info.messageTimestamp)).toBe(true); + expect(back.info.messageTimestamp.toString()).toBe('1758873600'); + expect(back.info.message.imageMessage.mediaKey).toBeInstanceOf(Uint8Array); + expect([...back.info.message.imageMessage.mediaKey]).toEqual([9, 8, 7]); + expect(back.info.message.imageMessage.fileLength.toString()).toBe('48213'); + // What the webhook serializer sees is the same JSON as the original's. + expect(JSON.stringify(back.info)).toBe(JSON.stringify(info)); + }); + + it('keeps a disconnect error a Boom with its status code', () => { + const back = roundTrip({ lastDisconnect: { error: new Boom('Connection Failure', { statusCode: 401 }), date: new Date(0) } }); + expect(back.lastDisconnect.error).toBeInstanceOf(Boom); + expect(back.lastDisconnect.error.output.statusCode).toBe(401); + expect(back.lastDisconnect.error.message).toBe('Connection Failure'); + expect(back.lastDisconnect.date).toBeInstanceOf(Date); + expect(back.lastDisconnect.date.getTime()).toBe(0); + }); +}); From 88c15dfb9c6892faa8921d47f5b8a6e43cf136ee Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:36:09 +0300 Subject: [PATCH 091/157] feat(live-record): a tagged JSON codec that keeps bytes, Longs, undefined and protobuf classes Co-Authored-By: Claude Opus 5.5 --- src/utils/live-record/codec.ts | 69 ++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) create mode 100644 src/utils/live-record/codec.ts diff --git a/src/utils/live-record/codec.ts b/src/utils/live-record/codec.ts new file mode 100644 index 0000000000..22b040dfb7 --- /dev/null +++ b/src/utils/live-record/codec.ts @@ -0,0 +1,69 @@ +// The tagged JSON a live-check tape is written in. It walks the value itself and +// never lets JSON.stringify see a payload, because JSON loses what Evolution +// branches on: Buffer vs Uint8Array, Long, an explicit undefined, protobuf +// classes (the webhook serializer calls toJSON on instances only), Boom errors. +// +// {"$bytes":"","as":"Buffer"|"Uint8Array"} {"$long":"","u":true} +// {"$u":1} undefined {"$big":""} {"$date":""} +// {"$err":{"message","statusCode","data"}} {"$fn":""} (decodes to undefined) +// {"$proto":"proto.Message.ImageMessage", ...fields} +import { Boom } from '@hapi/boom'; +import { proto } from 'baileys'; +import Long from 'long'; + +const protoName = (value: any): string | undefined => { + const typeUrl = value?.constructor?.getTypeUrl; + if (typeof typeUrl !== 'function') return undefined; + return String(typeUrl.call(value.constructor)).split('/').pop(); +}; + +export function encode(value: any): any { + if (value === undefined) return { $u: 1 }; + if (value === null || typeof value === 'string' || typeof value === 'boolean') return value; + if (typeof value === 'number') return value; + if (typeof value === 'bigint') return { $big: value.toString() }; + if (typeof value === 'function') return { $fn: value.name || 'fn' }; + if (Long.isLong(value)) return { $long: value.toString(), u: !!value.unsigned }; + if (Buffer.isBuffer(value)) return { $bytes: value.toString('base64'), as: 'Buffer' }; + if (value instanceof Uint8Array) return { $bytes: Buffer.from(value).toString('base64'), as: 'Uint8Array' }; + if (value instanceof Date) return { $date: value.toISOString() }; + if (value instanceof Error) { + const statusCode = (value as Boom).output?.statusCode; + return { $err: { message: value.message, statusCode, data: encode((value as Boom).data) } }; + } + if (Array.isArray(value)) return value.map(encode); + const out: Record = {}; + const name = protoName(value); + if (name) out.$proto = name; + for (const key of Object.keys(value)) out[key] = encode(value[key]); + return out; +} + +const protoClass = (name: string) => { + const Ctor = name.split('.').slice(1).reduce((node: any, part) => node?.[part], proto); + if (typeof Ctor !== 'function') throw new Error(`live-record: unknown protobuf class ${name}`); + return Ctor; +}; + +export function decode(value: any): any { + if (value === null || typeof value !== 'object') return value; + if (Array.isArray(value)) return value.map(decode); + if ('$u' in value) return undefined; + if ('$fn' in value) return undefined; + if ('$bytes' in value) { + const bytes = Buffer.from(value.$bytes, 'base64'); + return value.as === 'Buffer' ? bytes : new Uint8Array(bytes); + } + if ('$long' in value) return Long.fromString(value.$long, !!value.u); + if ('$big' in value) return BigInt(value.$big); + if ('$date' in value) return new Date(value.$date); + if ('$err' in value) { + const { message, statusCode, data } = value.$err; + return statusCode ? new Boom(message, { statusCode, data: decode(data) }) : new Error(message); + } + const out: Record = value.$proto ? new (protoClass(value.$proto))() : {}; + for (const key of Object.keys(value)) { + if (key !== '$proto') out[key] = decode(value[key]); + } + return out; +} From 70a3df97ecd2175f662f31d5ed374bd63a92721c Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:40:13 +0300 Subject: [PATCH 092/157] test: a live check records its events, webhooks and a manifest under LIVE_RECORD_DIR Six fail: there is no recorder. The first (nothing written and the same webhooks without LIVE_RECORD_DIR) passes before the feature by construction; it is the property the recorder must keep. Co-Authored-By: Claude Opus 5.5 --- test/helpers/live-session.ts | 86 +++++++++++++++ test/live/recorder.test.ts | 207 +++++++++++++++++++++++++++++++++++ 2 files changed, 293 insertions(+) create mode 100644 test/helpers/live-session.ts create mode 100644 test/live/recorder.test.ts diff --git a/test/helpers/live-session.ts b/test/helpers/live-session.ts new file mode 100644 index 0000000000..57ddd33e87 --- /dev/null +++ b/test/helpers/live-session.ts @@ -0,0 +1,86 @@ +// A short synthetic live session, played on the socket Evolution's real connect +// built (the test file mocks makeWASocket with fakeSocket). The identities look +// real on purpose: the recorder must keep them, and the scrubber must replace them. +import { proto } from 'baileys'; +import Long from 'long'; + +import { settle } from './baileys-service'; + +export const OWNER = { id: '972529998877:14@s.whatsapp.net', lid: '987654321098765:14@lid', name: 'Noa Barak' }; +export const PERSON = { pn: '972541112233@s.whatsapp.net', lid: '123456789012345@lid', saved: 'Dana Levi', push: 'Dana' }; +export const TEXT = 'see you at the cafe on Herzl street at 8'; +export const MESSAGE_ID = '3EB0C431C26A1D6C5A5D'; +export const MESSAGE_SECRET = Buffer.from('0f1e2d3c4b5a69788796a5b4c3d2e1f00112233445566778899aabbccddeeff', 'hex'); +export const CALL_ID = 'A1B2C3D4E5F60718293A4B5C6D7E8F90'; + +/** Every identifying string in the session, for tests that look for leaks. */ +export const ORIGINALS = [ + '972529998877', + '987654321098765', + OWNER.name, + '972541112233', + '123456789012345', + PERSON.saved, + PERSON.push, + TEXT, + MESSAGE_ID.slice(2), + CALL_ID.slice(2), + MESSAGE_SECRET.toString('base64'), +]; + +export const incoming = () => + proto.WebMessageInfo.fromObject({ + key: { remoteJid: PERSON.lid, remoteJidAlt: PERSON.pn, fromMe: false, id: MESSAGE_ID, addressingMode: 'lid' }, + messageTimestamp: Long.fromNumber(1758873600, true), + pushName: PERSON.push, + message: { conversation: TEXT, messageContextInfo: { messageSecret: new Uint8Array(MESSAGE_SECRET) } }, + }); + +/** What the owner's phone answers a call with, when the instance has a call message set. */ +export const callReply = (jid: string, text: string) => + proto.WebMessageInfo.fromObject({ + key: { remoteJid: jid, fromMe: true, id: 'BAE5F00D12345678' }, + messageTimestamp: Long.fromNumber(1758873801, true), + message: { conversation: text }, + }); + +/** The socket as a linked session has it: the account, a LID mapping, a phone that sends. */ +export function linkedSocket(service: any) { + Object.assign(service.client, { + user: { ...OWNER }, + signalRepository: { + lidMapping: { + getPNForLID: async (lid: string) => (lid === PERSON.lid ? PERSON.pn : undefined), + getLIDForPN: async (pn: string) => (pn === PERSON.pn ? PERSON.lid : undefined), + }, + }, + sendMessage: async (jid: string, content: { text: string }) => callReply(jid, content.text), + }); + // WhatsApp reports the phone's platform at pairing; smba is WhatsApp Business on Android. + if (service.instance.authState?.state?.creds) service.instance.authState.state.creds.platform = 'smba'; +} + +/** Open, then a contact and a message in one buffered batch, then a call offer. */ +export async function playSession(service: any) { + linkedSocket(service); + const ev = service.client.ev; + ev.emit('connection.update', { connection: 'open' }); + ev.emit('creds.update', { me: { id: OWNER.id, lid: OWNER.lid, name: OWNER.name } }); + ev.buffer(); + ev.emit('contacts.upsert', [{ id: PERSON.pn, lid: PERSON.lid, name: PERSON.saved, notify: PERSON.push }]); + ev.emit('messages.upsert', { type: 'notify', messages: [incoming()] }); + ev.flush(); + ev.emit('call', [ + { + chatId: PERSON.lid, + from: PERSON.lid, + id: CALL_ID, + date: new Date(1758873800000), + offline: false, + status: 'offer', + isVideo: false, + isGroup: false, + }, + ]); + await settle(service); +} diff --git a/test/live/recorder.test.ts b/test/live/recorder.test.ts new file mode 100644 index 0000000000..fd0ef5849b --- /dev/null +++ b/test/live/recorder.test.ts @@ -0,0 +1,207 @@ +// A live check records the session it runs (LIVE_RECORD_DIR), so what the phone +// did can be replayed in a test later. The recorder sits on the real socket's +// events and on sendDataWebhook, and must be invisible otherwise: without the +// variable it writes nothing, and with it Evolution sends exactly the same. +import { vi } from 'vitest'; + +const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + return { ...orig, default: socketSpy, makeWASocket: socketSpy }; +}); +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); + +import { mkdtempSync, readdirSync, readFileSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { proto } from 'baileys'; +import Long from 'long'; +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; + +import { decode, encode } from '@utils/live-record/codec'; + +import { makeService } from '../helpers/baileys-service'; +import { connectBehind, fakeSocket, stubAuthState } from '../helpers/connect'; +import { emitted } from '../helpers/fake-server-module'; +import { CALL_ID, MESSAGE_ID, OWNER, PERSON, playSession, TEXT } from '../helpers/live-session'; + +socketSpy.mockImplementation(fakeSocket); + +let root: string; +beforeEach(() => { + emitted.splice(0); + root = mkdtempSync(join(tmpdir(), 'live-record-')); +}); +afterEach(() => { + delete process.env.LIVE_RECORD_DIR; + rmSync(root, { recursive: true, force: true }); +}); + +const lines = (file: string) => + readFileSync(file, 'utf8') + .trim() + .split('\n') + .map((l) => JSON.parse(l)); + +/** The one session directory the recorder made: ///. */ +function sessionDir() { + const instances = readdirSync(root); + expect(instances).toEqual(['test']); + const sessions = readdirSync(join(root, 'test')); + expect(sessions).toHaveLength(1); + expect(sessions[0]).toMatch(/^\d{4}-\d{2}-\d{2}T\d{2}-\d{2}-\d{2}\.\d{3}Z$/); + return join(root, 'test', sessions[0]); +} + +/** The webhooks sent so far, each as its encoded JSON, in a stable order (background lookups interleave). */ +const sent = () => + emitted + .splice(0) + .map((e) => JSON.stringify({ event: e.event, data: encode(e.data) })) + .sort(); + +async function connected(opts: { msgCall?: string } = {}) { + const { service, prisma } = await makeService(); + if (opts.msgCall) prisma.setting.rows.push({ instanceId: 'inst-1', msgCall: opts.msgCall }); + stubAuthState(service); + await service.connectToWhatsapp(); + return service; +} + +describe('live-check recorder', () => { + it('is inert without LIVE_RECORD_DIR: nothing written, the same webhooks sent', async () => { + delete process.env.LIVE_RECORD_DIR; + await playSession(await connected()); + const without = sent(); + expect(readdirSync(root)).toEqual([]); + + process.env.LIVE_RECORD_DIR = root; + await playSession(await connected()); + const withRecorder = sent(); + + for (const event of ['connection.update', 'contacts.upsert', 'messages.upsert', 'call']) { + expect(without.map((w) => JSON.parse(w).event)).toContain(event); + } + expect(withRecorder).toEqual(without); + }); + + it('writes the events, each with its sequence and whether the buffer held it, in the codec', async () => { + process.env.LIVE_RECORD_DIR = root; + await playSession(await connected()); + const dir = sessionDir(); + expect(readdirSync(dir).sort()).toEqual(['events.ndjson', 'manifest.json', 'owner.json', 'webhooks.ndjson']); + + const events = lines(join(dir, 'events.ndjson')); + const seqs = events.map((e) => e.seq); + expect(seqs).toEqual([...seqs].sort((a, b) => a - b)); + expect(new Set(seqs).size).toBe(seqs.length); + + const emits = events.filter((e) => e.event); + expect(emits.map((e) => [e.event, e.buffered])).toEqual([ + ['connection.update', false], + ['creds.update', false], + ['contacts.upsert', true], + ['messages.upsert', true], + ['call', false], + ]); + expect(emits.every((e) => e.socket === 1 && e.origin === undefined)).toBe(true); + // The batches the buffer handed Evolution: the two buffered events arrived as one. + expect(events.filter((e) => e.batch).map((e) => e.batch)).toEqual([ + ['connection.update'], + ['creds.update'], + ['contacts.upsert', 'messages.upsert'], + ['call'], + ]); + + // The payload decodes to the values the socket emitted: classes, Longs and bytes included. + const upsert = decode(emits[3].data); + const message = upsert.messages[0]; + expect(message).toBeInstanceOf(proto.WebMessageInfo); + expect(message.key.id).toBe(MESSAGE_ID); + expect(Long.isLong(message.messageTimestamp)).toBe(true); + expect(message.message.conversation).toBe(TEXT); + expect(message.message.messageContextInfo.messageSecret).toBeInstanceOf(Uint8Array); + expect(decode(emits[2].data)).toEqual([{ id: PERSON.pn, lid: PERSON.lid, name: PERSON.saved, notify: PERSON.push }]); + + // The auth creds never reach the tape. + expect(emits[1].data).toEqual({ $redacted: 'creds', keys: ['me'] }); + }); + + it('writes every webhook Evolution sent, as it sent it (the golden output)', async () => { + process.env.LIVE_RECORD_DIR = root; + await playSession(await connected()); + const webhooks = lines(join(sessionDir(), 'webhooks.ndjson')); + expect(webhooks.map((w) => ({ event: w.event, data: w.data }))).toEqual( + emitted.map((e) => ({ event: e.event, data: encode(e.data) })), + ); + const call = webhooks.find((w) => w.event === 'call'); + expect(call.data.id).toBe(CALL_ID); + }); + + it('marks an event Evolution emits itself (the call message), so a replay does not emit it twice', async () => { + process.env.LIVE_RECORD_DIR = root; + await playSession(await connected({ msgCall: 'In a meeting' })); + const emits = lines(join(sessionDir(), 'events.ndjson')).filter((e) => e.event); + const fromApp = emits.filter((e) => e.origin === 'app'); + expect(fromApp.map((e) => e.event)).toEqual(['messages.upsert']); + expect(decode(fromApp[0].data).messages[0].message.conversation).toBe('In a meeting'); + }); + + it('writes a manifest of versions and conditions, with no number, JID or name in it', async () => { + process.env.LIVE_RECORD_DIR = root; + const service = await connected(); + const dir = sessionDir(); + const atStart = JSON.parse(readFileSync(join(dir, 'manifest.json'), 'utf8')); + expect(atStart.openedAt).toBeNull(); + expect(atStart.phonePlatform).toBeNull(); + + await playSession(service); + const manifest = JSON.parse(readFileSync(join(dir, 'manifest.json'), 'utf8')); + const baileys = JSON.parse(readFileSync(join(process.env.BAILEYS_RESOLVED_DIR, 'package.json'), 'utf8')).version; + expect(manifest).toEqual({ + format: 'live-record/1', + forkCommit: expect.stringMatching(/^[0-9a-f]{8}(-dirty)?$/), + baileysVersion: baileys, + nodeVersion: process.version, + waWebVersion: '2.3000.1', + phonePlatform: 'smba', + accountType: 'business', + linkMethod: 'qr', + proxy: { used: false, protocol: null }, + sockets: 1, + startedAt: atStart.startedAt, + openedAt: expect.stringMatching(/^\d{4}-\d{2}-\d{2}T/), + endedAt: null, + }); + const text = JSON.stringify(manifest); + for (const secret of ['972529998877', '987654321098765', '972541112233', '123456789012345', OWNER.name, PERSON.saved, '@']) { + expect(text).not.toContain(secret); + } + // The account is kept apart, for the scrubber only. + expect(JSON.parse(readFileSync(join(dir, 'owner.json'), 'utf8'))).toEqual(OWNER); + }); + + it('records the link method a pairing code asked for, without the number', async () => { + process.env.LIVE_RECORD_DIR = root; + const { service } = await makeService(); + stubAuthState(service); + await service.connectToWhatsapp('972541112233'); + const manifest = readFileSync(join(sessionDir(), 'manifest.json'), 'utf8'); + expect(JSON.parse(manifest).linkMethod).toBe('code'); + expect(manifest).not.toContain('972541112233'); + }); + + it('records that a proxy was used and its protocol, never its address', async () => { + process.env.LIVE_RECORD_DIR = root; + await connectBehind(socketSpy, { protocol: 'socks5', port: 18461 }); + const manifest = readFileSync(join(sessionDir(), 'manifest.json'), 'utf8'); + expect(JSON.parse(manifest).proxy).toEqual({ used: true, protocol: 'socks5' }); + expect(manifest).not.toContain('127.0.0.1'); + expect(manifest).not.toContain('18461'); + }); +}); From ad048e621f46bb010811e5b4b19b70a2bf80e2ce Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:41:03 +0300 Subject: [PATCH 093/157] feat(live-record): record a live check's events, webhooks and manifest when LIVE_RECORD_DIR is set The recorder wraps the socket's ev.emit (as tapHistoryLidMappings does) and writes each event with a global sequence, the socket it came from and whether the buffer was holding it, plus each batch the buffer delivered, in the tagged codec. sendDataWebhook writes the golden output tape. The manifest carries versions and conditions only: fork commit, Baileys, Node, WhatsApp Web version, creds.platform and the account type it implies, link method, proxy protocol, timestamps. The auth creds and the QR payload are redacted at record time. Evolution's own call-message emit is marked origin app. The batch-order expectation in the test is corrected to the buffer's own key order (messages.upsert before contacts.upsert): the test had it backwards. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 18 +- src/utils/live-record/recorder.ts | 215 ++++++++++++++++++ test/live/recorder.test.ts | 4 +- 3 files changed, 234 insertions(+), 3 deletions(-) create mode 100644 src/utils/live-record/recorder.ts diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 0d227eec3a..d3049879d3 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -86,6 +86,7 @@ import { createJid } from '@utils/createJid'; import { fetchLatestWaWebVersion } from '@utils/fetchLatestWaWebVersion'; import { errorFields, jidKind, makeBaileysLogger } from '@utils/log-privacy'; import { readLogoutMarker, writeLogoutMarker } from '@utils/logout-marker'; +import { LiveRecorder } from '@utils/live-record/recorder'; import { makeProxyAgent, makeProxyAgentUndici } from '@utils/makeProxyAgent'; import { getOnWhatsappCache, saveOnWhatsappCache } from '@utils/onWhatsappCache'; import { QueryLimiter } from '@utils/queryLimiter'; @@ -348,6 +349,8 @@ export class BaileysStartupService extends ChannelStartupService { private logout: { marked: boolean; deleted: boolean; settle: (outcome: 'done' | 'pending') => void } | null = null; private logBaileys = this.configService.get('LOG').BAILEYS; private eventProcessingQueue: Promise = Promise.resolve(); + // Records this session's events and webhooks for a live check; undefined unless LIVE_RECORD_DIR is set. + private liveRecorder?: LiveRecorder; // The dispatcher media downloads go through: the instance's proxy, the same exit as the socket. private mediaProxy?: { key: string; dispatcher: ReturnType }; @@ -591,6 +594,7 @@ export class BaileysStartupService extends ChannelStartupService { extra?: Record, ) { if (this.logout) return; + this.liveRecorder?.webhook(event, data, extra); return super.sendDataWebhook(event, data, local, integration, extra); } @@ -1051,6 +1055,16 @@ export class BaileysStartupService extends ChannelStartupService { // Any reconnect still waiting was for the socket just replaced. this.stopReconnecting(); + this.liveRecorder ??= LiveRecorder.start(this.instance.name); + const creds = this.instance.authState.state.creds; + this.liveRecorder?.attach(this.client, { + waWebVersion: version.join('.'), + // creds.account is set once a pairing succeeded. + linkMethod: creds?.account ? 'existing-session' : this.phoneNumber ? 'code' : 'qr', + proxyProtocol: options ? (this.localProxy?.protocol ?? 'http') : null, + creds: () => this.instance.authState?.state?.creds, + }); + if (this.localSettings.wavoipToken && this.localSettings.wavoipToken.length > 0) { useVoiceCallsBaileys(this.localSettings.wavoipToken, this.client, this.connectionStatus.state as any, true); } @@ -2409,7 +2423,9 @@ export class BaileysStartupService extends ChannelStartupService { } const msg = await this.client.sendMessage(call.from, { text: settings.msgCall }); - this.client.ev.emit('messages.upsert', { messages: [msg], type: 'notify' }); + const upsert = () => this.client.ev.emit('messages.upsert', { messages: [msg], type: 'notify' }); + if (this.liveRecorder) this.liveRecorder.fromApp(upsert); + else upsert(); } this.sendDataWebhook(Events.CALL, call); diff --git a/src/utils/live-record/recorder.ts b/src/utils/live-record/recorder.ts new file mode 100644 index 0000000000..0eefb61eda --- /dev/null +++ b/src/utils/live-record/recorder.ts @@ -0,0 +1,215 @@ +// Records a live check, and does nothing unless LIVE_RECORD_DIR is set. +// +// LIVE_RECORD_DIR/// +// events.ndjson every Baileys event the socket emitted (the input tape), and +// every batch the event buffer handed Evolution +// webhooks.ndjson every payload Evolution sent out (the golden output tape) +// manifest.json versions and conditions, never a number, a JID, a name or content +// owner.json the linked account (raw only: the scrubber reads it, never copies it) +// +// One line per record, one sequence across both tapes, values in the tagged +// codec (codec.ts) so a replay rebuilds identical ones. Everything is written +// synchronously, at the moment it happens: Evolution mutates payloads later. +// The raw files hold personal data. They stay on the machine that recorded them +// until scripts/live-scrub.ts turns them into a fixture (docs/LIVE-CHECKS.md). +import { execFileSync } from 'child_process'; +import { appendFileSync, existsSync, mkdirSync, readFileSync, writeFileSync } from 'fs'; +import { createRequire } from 'module'; +import { join } from 'path'; +import { performance } from 'perf_hooks'; + +import { encode } from './codec'; + +export type LinkMethod = 'qr' | 'code' | 'existing-session'; + +/** What Evolution knows about a socket when it builds one. */ +export type SocketFacts = { + waWebVersion: string; + linkMethod: LinkMethod; + /** The proxy's protocol, or null for none. Never its host or credentials. */ + proxyProtocol: string | null; + /** The auth creds, read when the connection opens (creds.platform is set at pairing). */ + creds: () => any; +}; + +export type Manifest = { + format: 'live-record/1'; + forkCommit: string | null; + baileysVersion: string | null; + nodeVersion: string; + waWebVersion: string | null; + /** creds.platform as WhatsApp reported it at pairing (smba, smbi, android, iphone...), when known. */ + phonePlatform: string | null; + /** From the platform: WhatsApp Business apps report smb*. Null when unknown. */ + accountType: 'business' | 'personal' | null; + linkMethod: LinkMethod | null; + proxy: { used: boolean; protocol: string | null }; + sockets: number; + startedAt: string; + openedAt: string | null; + endedAt: string | null; +}; + +const safeName = (name: string) => name.replace(/[^A-Za-z0-9._-]/g, '_') || 'instance'; + +function forkCommit(env: NodeJS.ProcessEnv): string | null { + const file = join(process.cwd(), 'FORK_SHA'); + if (existsSync(file)) return readFileSync(file, 'utf8').trim() || null; + try { + return execFileSync('git', ['describe', '--always', '--dirty', '--abbrev=8', '--match=NONE'], { + stdio: ['ignore', 'pipe', 'ignore'], + }) + .toString() + .trim(); + } catch { + return env.FORK_SHA || null; + } +} + +function baileysVersion(): string | null { + try { + const path = createRequire(join(process.cwd(), 'package.json')).resolve('baileys/package.json'); + return JSON.parse(readFileSync(path, 'utf8')).version ?? null; + } catch { + return null; + } +} + +const accountType = (platform?: string | null): Manifest['accountType'] => { + if (!platform) return null; + return /^smb/i.test(platform) ? 'business' : 'personal'; +}; + +export class LiveRecorder { + private seq = 0; + private sockets = 0; + private readonly t0 = performance.now(); + private appDepth = 0; + private broken = false; + private readonly manifest: Manifest; + + private constructor(readonly dir: string) { + this.manifest = { + format: 'live-record/1', + forkCommit: forkCommit(process.env), + baileysVersion: baileysVersion(), + nodeVersion: process.version, + waWebVersion: null, + phonePlatform: null, + accountType: null, + linkMethod: null, + proxy: { used: false, protocol: null }, + sockets: 0, + startedAt: new Date().toISOString(), + openedAt: null, + endedAt: null, + }; + this.writeManifest(); + } + + /** A recorder for this instance's session, or undefined when LIVE_RECORD_DIR is not set. */ + static start(instanceName: string, env: NodeJS.ProcessEnv = process.env): LiveRecorder | undefined { + const root = env.LIVE_RECORD_DIR; + if (!root) return undefined; + const stamp = new Date().toISOString().replace(/:/g, '-'); + const dir = join(root, safeName(instanceName), stamp); + mkdirSync(dir, { recursive: true, mode: 0o700 }); + return new LiveRecorder(dir); + } + + /** Record every event this socket emits, and every batch its buffer delivers. Call before eventHandler(). */ + attach(client: any, facts: SocketFacts) { + const socket = ++this.sockets; + this.manifest.sockets = socket; + this.manifest.waWebVersion = facts.waWebVersion; + // The method that linked the account is the one asked for before the first open. + if (!this.manifest.openedAt) this.manifest.linkMethod = facts.linkMethod; + this.manifest.proxy = { used: !!facts.proxyProtocol, protocol: facts.proxyProtocol }; + this.writeManifest(); + + const ev = client.ev; + const emit = ev.emit.bind(ev); + ev.emit = (event: string, data: any) => { + this.guard(() => { + const line: Record = { seq: ++this.seq, t: this.elapsed(), socket, event }; + line.buffered = typeof ev.isBuffering === 'function' ? ev.isBuffering() : false; + if (this.appDepth) line.origin = 'app'; + line.data = encode(redact(event, data)); + this.append('events.ndjson', line); + if (event === 'connection.update') this.connectionUpdate(client, facts, data); + }); + return emit(event, data); + }; + ev.process((events: Record) => { + this.guard(() => + this.append('events.ndjson', { seq: ++this.seq, t: this.elapsed(), socket, batch: Object.keys(events) }), + ); + }); + } + + /** Evolution emitting into the socket's events itself: marked, so a replay does not emit it twice. */ + fromApp(fn: () => T): T { + this.appDepth++; + try { + return fn(); + } finally { + this.appDepth--; + } + } + + /** A payload Evolution sends out (sendDataWebhook), as it was at that moment. */ + webhook(event: string, data: any, extra?: Record) { + this.guard(() => { + const line: Record = { seq: ++this.seq, t: this.elapsed(), event, data: encode(data) }; + if (extra !== undefined) line.extra = encode(extra); + this.append('webhooks.ndjson', line); + }); + } + + private connectionUpdate(client: any, facts: SocketFacts, update: any) { + if (update?.connection === 'open') { + const platform = facts.creds()?.platform ?? null; + this.manifest.phonePlatform = platform; + this.manifest.accountType = accountType(platform); + this.manifest.openedAt ??= new Date().toISOString(); + this.manifest.endedAt = null; + const user = client.user ?? {}; + writeFileSync(join(this.dir, 'owner.json'), JSON.stringify({ id: user.id, lid: user.lid, name: user.name })); + this.writeManifest(); + } + if (update?.connection === 'close') { + this.manifest.endedAt = new Date().toISOString(); + this.writeManifest(); + } + } + + private elapsed() { + return Math.round((performance.now() - this.t0) * 10) / 10; + } + + private append(file: string, line: object) { + appendFileSync(join(this.dir, file), JSON.stringify(line) + '\n', { mode: 0o600 }); + } + + private writeManifest() { + writeFileSync(join(this.dir, 'manifest.json'), JSON.stringify(this.manifest, null, 2) + '\n', { mode: 0o600 }); + } + + /** A recorder that fails stops recording; it never breaks the socket or the webhook it was watching. */ + private guard(fn: () => void) { + if (this.broken) return; + try { + fn(); + } catch (error) { + this.broken = true; + console.warn(`[live-record] recording stopped: ${error?.message ?? error}`); + } + } +} + +/** Secrets with no replay value never reach the tape: the auth creds and the QR payload. */ +function redact(event: string, data: any) { + if (event === 'creds.update') return { $redacted: 'creds', keys: Object.keys(data ?? {}) }; + if (event === 'connection.update' && data?.qr) return { ...data, qr: '$qr' }; + return data; +} diff --git a/test/live/recorder.test.ts b/test/live/recorder.test.ts index fd0ef5849b..f15f45d6dc 100644 --- a/test/live/recorder.test.ts +++ b/test/live/recorder.test.ts @@ -110,11 +110,11 @@ describe('live-check recorder', () => { ['call', false], ]); expect(emits.every((e) => e.socket === 1 && e.origin === undefined)).toBe(true); - // The batches the buffer handed Evolution: the two buffered events arrived as one. + // The batches the buffer handed Evolution: the two buffered events arrived as one, in the buffer's key order. expect(events.filter((e) => e.batch).map((e) => e.batch)).toEqual([ ['connection.update'], ['creds.update'], - ['contacts.upsert', 'messages.upsert'], + ['messages.upsert', 'contacts.upsert'], ['call'], ]); From acce1ee57e92b0770130ad2400ded8cc6c7b999d Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:42:52 +0300 Subject: [PATCH 094/157] test: the live-check scrubber replaces identities consistently and its leak gate aborts on a survivor Co-Authored-By: Claude Opus 5.5 --- test/helpers/live-session.ts | 25 ++++- test/live/scrub.test.ts | 172 +++++++++++++++++++++++++++++++++++ 2 files changed, 196 insertions(+), 1 deletion(-) create mode 100644 test/live/scrub.test.ts diff --git a/test/helpers/live-session.ts b/test/helpers/live-session.ts index 57ddd33e87..45255810bd 100644 --- a/test/helpers/live-session.ts +++ b/test/helpers/live-session.ts @@ -1,10 +1,14 @@ // A short synthetic live session, played on the socket Evolution's real connect // built (the test file mocks makeWASocket with fakeSocket). The identities look // real on purpose: the recorder must keep them, and the scrubber must replace them. +import { readdirSync } from 'node:fs'; +import { join } from 'node:path'; + import { proto } from 'baileys'; import Long from 'long'; -import { settle } from './baileys-service'; +import { makeService, settle } from './baileys-service'; +import { stubAuthState } from './connect'; export const OWNER = { id: '972529998877:14@s.whatsapp.net', lid: '987654321098765:14@lid', name: 'Noa Barak' }; export const PERSON = { pn: '972541112233@s.whatsapp.net', lid: '123456789012345@lid', saved: 'Dana Levi', push: 'Dana' }; @@ -84,3 +88,22 @@ export async function playSession(service: any) { ]); await settle(service); } + +/** + * Record the session under `root` the way a live check does (LIVE_RECORD_DIR) and + * return the raw session directory. The calling test mocks makeWASocket. + */ +export async function recordSession(root: string, opts: { msgCall?: string } = {}) { + process.env.LIVE_RECORD_DIR = root; + try { + const { service, prisma } = await makeService(); + if (opts.msgCall) prisma.setting.rows.push({ instanceId: 'inst-1', msgCall: opts.msgCall }); + stubAuthState(service); + await service.connectToWhatsapp(); + await playSession(service); + } finally { + delete process.env.LIVE_RECORD_DIR; + } + const [session] = readdirSync(join(root, 'test')); + return join(root, 'test', session); +} diff --git a/test/live/scrub.test.ts b/test/live/scrub.test.ts new file mode 100644 index 0000000000..f48fb7fd52 --- /dev/null +++ b/test/live/scrub.test.ts @@ -0,0 +1,172 @@ +// A raw recording holds real numbers, names, texts and keys. The scrubber turns +// it into a fixture that can be committed: every identity replaced by a stable +// fake (one person keeps one index across phone JID, @lid and device suffix), +// the shapes Evolution reads kept (lengths, id prefixes, byte types), and then a +// leak gate that searches the output for every original and writes nothing when +// one survives. +import { vi } from 'vitest'; + +const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + return { ...orig, default: socketSpy, makeWASocket: socketSpy }; +}); +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); + +import { appendFileSync, existsSync, mkdtempSync, readdirSync, readFileSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; + +import { decode } from '@utils/live-record/codec'; + +import { fakeSocket } from '../helpers/connect'; +import { emitted } from '../helpers/fake-server-module'; +import { MESSAGE_ID, MESSAGE_SECRET, ORIGINALS, PERSON, recordSession, TEXT } from '../helpers/live-session'; +import { scrubSession } from '../tools/live-scrub'; + +socketSpy.mockImplementation(fakeSocket); + +let root: string; +let raw: string; +let out: string; +beforeEach(async () => { + emitted.splice(0); + root = mkdtempSync(join(tmpdir(), 'live-scrub-')); + raw = await recordSession(join(root, 'raw')); + out = join(root, 'fixtures'); +}); +afterEach(() => rmSync(root, { recursive: true, force: true })); + +const lines = (file: string) => + readFileSync(file, 'utf8') + .trim() + .split('\n') + .map((l) => JSON.parse(l)); + +const scrub = (extra: Partial[1]> = {}) => + scrubSession(raw, { checkId: 'synthetic-session', date: '2026-09-27', outRoot: out, ...extra }); + +describe('live-check scrubber', () => { + it('writes the fixture files under -, and never the raw owner file', () => { + const { dir } = scrub(); + expect(dir).toBe(join(out, '2026-09-27-synthetic-session')); + expect(readdirSync(dir).sort()).toEqual(['events.ndjson', 'manifest.json', 'scrub-report.json', 'webhooks.ndjson']); + }); + + it('leaves no original anywhere in the fixture', () => { + const { dir } = scrub(); + const text = readdirSync(dir) + .map((f) => readFileSync(join(dir, f), 'utf8')) + .join('\n'); + for (const original of ORIGINALS) expect(text.includes(original), `an original of ${original.length} chars`).toBe(false); + }); + + it('gives one person one fake across phone JID, @lid and device suffix, in both tapes', () => { + const { dir } = scrub(); + const emits = lines(join(dir, 'events.ndjson')).filter((e) => e.event); + const [contact] = decode(emits.find((e) => e.event === 'contacts.upsert').data); + expect(contact.id).toMatch(/^972500\d{6}@s\.whatsapp\.net$/); + const index = contact.id.slice(6, 12); + expect(contact.lid).toBe(`100000000${index}@lid`); + expect(index).not.toBe('000000'); // index 0 is the owner + + const upsert = decode(emits.find((e) => e.event === 'messages.upsert').data).messages[0]; + expect(upsert.key.remoteJid).toBe(contact.lid); + expect(upsert.key.remoteJidAlt).toBe(contact.id); + + // The owner is index 0, device suffix kept. + const webhooks = lines(join(dir, 'webhooks.ndjson')); + const open = webhooks.find((w) => w.event === 'connection.update' && w.data.state === 'open'); + expect(open.data.wuid).toBe('972500000000@s.whatsapp.net'); + const manifest = JSON.parse(readFileSync(join(dir, 'manifest.json'), 'utf8')); + expect(manifest.replay.owner.id).toBe('972500000000:14@s.whatsapp.net'); + expect(manifest.replay.owner.lid).toBe('100000000000000:14@lid'); + + // Evolution showed the phone JID as remoteJid and kept the @lid: the same fakes. + const sent = decode(webhooks.find((w) => w.event === 'messages.upsert').data); + expect(sent.key.remoteJid).toBe(contact.id); + expect(sent.key.remoteJidAlt).toBe(contact.lid); + }); + + it('keeps name equality, id shape, byte length and type, and text length', () => { + const { dir } = scrub(); + const emits = lines(join(dir, 'events.ndjson')).filter((e) => e.event); + const [contact] = decode(emits.find((e) => e.event === 'contacts.upsert').data); + const message = decode(emits.find((e) => e.event === 'messages.upsert').data).messages[0]; + + // Saved name and profile name stay different; the same profile name stays the same. + expect(contact.name).not.toBe(contact.notify); + expect(contact.name).not.toBe(PERSON.saved); + expect(message.pushName).toBe(contact.notify); + + expect(message.key.id).toHaveLength(MESSAGE_ID.length); + expect(message.key.id.slice(0, 2)).toBe(MESSAGE_ID.slice(0, 2)); + expect(message.key.id).not.toBe(MESSAGE_ID); + + const secret = message.message.messageContextInfo.messageSecret; + expect(secret).toBeInstanceOf(Uint8Array); + expect(Buffer.isBuffer(secret)).toBe(false); + expect(secret.length).toBe(MESSAGE_SECRET.length); + expect(Buffer.from(secret).equals(MESSAGE_SECRET)).toBe(false); + + expect(message.message.conversation).toHaveLength(TEXT.length); + expect(message.message.conversation).not.toBe(TEXT); + + // The webhook carries the same fakes as the event it came from. + const sent = decode(lines(join(dir, 'webhooks.ndjson')).find((w) => w.event === 'messages.upsert').data); + expect(sent.key.id).toBe(message.key.id); + expect(sent.pushName).toBe(message.pushName); + expect(sent.message.conversation).toBe(message.message.conversation); + expect(Buffer.from(sent.message.messageContextInfo.messageSecret).equals(Buffer.from(secret))).toBe(true); + }); + + it('adds what the operator records, and a report of counts only', () => { + const { dir } = scrub({ + operator: { phoneModel: 'Pixel 8', osVersion: 'Android 15', whatsappAppVersion: '2.25.27.78', countryCode: '972' }, + }); + const manifest = JSON.parse(readFileSync(join(dir, 'manifest.json'), 'utf8')); + expect(manifest).toMatchObject({ + checkId: 'synthetic-session', + date: '2026-09-27', + phoneModel: 'Pixel 8', + osVersion: 'Android 15', + whatsappAppVersion: '2.25.27.78', + countryCode: '972', + phonePlatform: 'smba', + waWebVersion: '2.3000.1', + }); + const report = JSON.parse(readFileSync(join(dir, 'scrub-report.json'), 'utf8')); + expect(report.leakGate).toBe('pass'); + const numbers = Object.entries(report).filter(([k]) => k !== 'leakGate'); + expect(numbers.length).toBeGreaterThan(0); + for (const [, value] of numbers) expect(typeof value).toBe('number'); + expect(report.people).toBe(2); + }); + + it('takes a country code only, never a number', () => { + expect(() => scrub({ operator: { countryCode: '972541112233' } })).toThrow(/country code/); + expect(existsSync(join(out, '2026-09-27-synthetic-session'))).toBe(false); + }); + + it('aborts and writes nothing when an original survives the rewrite', () => { + // An object key is kept as written unless it is an address: plant a saved name there. + const planted = { seq: 999, t: 1, socket: 1, event: 'labels.edit', buffered: false, data: { [PERSON.saved]: 1 } }; + appendFileSync(join(raw, 'events.ndjson'), JSON.stringify(planted) + '\n'); + let message = ''; + try { + scrub(); + } catch (error) { + message = String(error?.message); + } + expect(message).toMatch(/leak gate/i); + expect(message).toContain('events.ndjson'); + for (const original of ORIGINALS) expect(message.includes(original)).toBe(false); + expect(existsSync(out)).toBe(false); + }); +}); From 938ad85527415f1c66415a1482e9a68b1b94fcb0 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:45:11 +0300 Subject: [PATCH 095/157] feat(live-record): a scrubber that turns a raw recording into a fixture, behind a fail-closed leak gate test/tools/live-scrub.ts rewrites every string leaf of both tapes with one identity table (one person, one index, across phone JID, @lid and device suffix; the owner is index 0), keeps the shapes Evolution reads (message id prefix and length, byte length and type, text length, name equality), and then searches the whole output for every original. One hit aborts and nothing is written. scripts/live-scrub.ts is the CLI (npx tsx). The synthetic message now sets remoteJidAlt and addressingMode on the key after fromObject, as Baileys does: fromObject dropped them (not proto fields). Co-Authored-By: Claude Opus 5.5 --- scripts/live-scrub.ts | 40 +++++ test/helpers/live-session.ts | 10 +- test/tools/live-scrub.ts | 329 +++++++++++++++++++++++++++++++++++ 3 files changed, 376 insertions(+), 3 deletions(-) create mode 100644 scripts/live-scrub.ts create mode 100644 test/tools/live-scrub.ts diff --git a/scripts/live-scrub.ts b/scripts/live-scrub.ts new file mode 100644 index 0000000000..cef67a3c50 --- /dev/null +++ b/scripts/live-scrub.ts @@ -0,0 +1,40 @@ +// Scrub a raw live-check recording into a committable fixture (docs/LIVE-CHECKS.md). +// +// npx tsx scripts/live-scrub.ts [--date YYYY-MM-DD] +// [--phone-model "Pixel 8"] [--os-version "Android 15"] [--wa-version 2.25.27.78] +// [--country-code 972] [--out test/fixtures/live] +// +// Exits 1 and writes nothing when the leak gate finds an original in the output. +import { scrubSession } from '../test/tools/live-scrub'; + +const args = process.argv.slice(2); +const flags: Record = {}; +const positional: string[] = []; +for (let i = 0; i < args.length; i++) { + if (args[i].startsWith('--')) flags[args[i].slice(2)] = args[++i]; + else positional.push(args[i]); +} +const [rawDir, checkId] = positional; +if (!rawDir || !checkId) { + console.error('usage: npx tsx scripts/live-scrub.ts [--date YYYY-MM-DD] [--phone-model ...] [--os-version ...] [--wa-version ...] [--country-code ...] [--out dir]'); + process.exit(2); +} + +try { + const { dir, report } = scrubSession(rawDir, { + checkId, + date: flags.date, + outRoot: flags.out, + operator: { + phoneModel: flags['phone-model'], + osVersion: flags['os-version'], + whatsappAppVersion: flags['wa-version'], + countryCode: flags['country-code'], + }, + }); + console.log(`fixture written: ${dir}`); + console.log(JSON.stringify(report, null, 2)); +} catch (error) { + console.error(error?.message ?? error); + process.exit(1); +} diff --git a/test/helpers/live-session.ts b/test/helpers/live-session.ts index 45255810bd..3966930b88 100644 --- a/test/helpers/live-session.ts +++ b/test/helpers/live-session.ts @@ -32,13 +32,17 @@ export const ORIGINALS = [ MESSAGE_SECRET.toString('base64'), ]; -export const incoming = () => - proto.WebMessageInfo.fromObject({ - key: { remoteJid: PERSON.lid, remoteJidAlt: PERSON.pn, fromMe: false, id: MESSAGE_ID, addressingMode: 'lid' }, +export const incoming = () => { + const info = proto.WebMessageInfo.fromObject({ + key: { remoteJid: PERSON.lid, fromMe: false, id: MESSAGE_ID }, messageTimestamp: Long.fromNumber(1758873600, true), pushName: PERSON.push, message: { conversation: TEXT, messageContextInfo: { messageSecret: new Uint8Array(MESSAGE_SECRET) } }, }); + // Not proto fields: Baileys sets them on the decoded key (fromObject would drop them). + Object.assign(info.key, { remoteJidAlt: PERSON.pn, addressingMode: 'lid' }); + return info; +}; /** What the owner's phone answers a call with, when the instance has a call message set. */ export const callReply = (jid: string, text: string) => diff --git a/test/tools/live-scrub.ts b/test/tools/live-scrub.ts new file mode 100644 index 0000000000..c5c33fce28 --- /dev/null +++ b/test/tools/live-scrub.ts @@ -0,0 +1,329 @@ +// Turns a raw live-check recording (LIVE_RECORD_DIR///) into a +// fixture that can be committed: test/fixtures/live/-/. +// +// It works on the tapes as written (the tagged codec, never decoded), in three steps: +// 1. collect: pair each person's phone JID with their @lid wherever one object +// names both, so one person keeps one fake index everywhere; +// 2. rewrite every string leaf, and every object key that is an address; +// 3. the leak gate: search the whole output for every original (numbers, names, +// byte strings, and every raw string of 4+ characters the rewrite replaced). +// One hit aborts, and nothing is written. +// +// What a string becomes: +// phone JID / @lid / group 972500<6> / 100000000<6> / 120363<12>, device suffix kept; index 0 is the owner +// a bare number of 7-15 digits the same person's fake digits (epoch timestamps are kept) +// a name (name, notify, pushName, subject...) "Name ", one per distinct original ("Você" kept) +// a message id same first two characters and length, the rest a counter +// bytes ($bytes) random bytes of the same length and type, tagged fake +// a URL https://example.invalid/ +// structure (event names, enums, mimetypes, dates, 3 characters or fewer) kept +// anything else, text included lorem of the same length +import { randomBytes } from 'node:crypto'; +import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs'; +import { basename, dirname, join } from 'node:path'; + +export type Operator = { phoneModel?: string; osVersion?: string; whatsappAppVersion?: string; countryCode?: string }; +export type ScrubOptions = { checkId: string; date?: string; outRoot?: string; operator?: Operator }; + +export class LeakError extends Error {} + +const FAKE_PN = (i: number) => `972500${String(i).padStart(6, '0')}`; +const FAKE_LID = (i: number) => `100000000${String(i).padStart(6, '0')}`; +const FAKE_GROUP = (i: number) => `120363${String(i).padStart(12, '0')}`; +/** Addresses WhatsApp itself uses, never a person's. */ +const SERVICE_USERS = new Set(['0', '16505361212', '13135550002']); +/** The harness's instance name: a replay runs under it. */ +const REPLAY_INSTANCE = 'test'; + +const JID = /^(\d+(?:-\d+)?)((?:[:_]\d+)*)@(s\.whatsapp\.net|c\.us|hosted|lid|hosted\.lid|g\.us|broadcast|newsletter)$/; +const PN_SERVERS = new Set(['s.whatsapp.net', 'c.us', 'hosted']); +const LID_SERVERS = new Set(['lid', 'hosted.lid']); + +const NAME_KEYS = new Set([ + 'name', 'notify', 'verifiedName', 'verifiedBizName', 'pushName', 'username', 'subject', 'profileName', + 'fullName', 'firstName', 'shortName', 'displayName', 'vname', +]); +const TEXT_KEYS = new Set([ + 'conversation', 'text', 'caption', 'desc', 'description', 'title', 'body', 'matchedText', 'canonicalUrl', + 'fileName', 'address', 'contentText', 'footerText', 'headerText', 'vcard', 'selectedDisplayText', 'optionName', + 'comment', 'message', 'msgCall', +]); +const ID_KEYS = new Set(['id', 'stanzaId', 'keyId', 'messageId', 'callId']); +const STRUCTURAL_KEYS = new Set([ + '$proto', '$redacted', 'as', 'event', 'type', 'messageType', 'mimetype', 'addressingMode', 'action', 'connection', + 'source', 'origin', 'state', 'status', 'platform', 'mediaType', +]); + +const isEpoch = (digits: string) => /^1\d{9}$/.test(digits) || /^1\d{12}$/.test(digits); +const isStructural = (key: string, s: string) => + s.length <= 3 || + (STRUCTURAL_KEYS.has(key) && !/\s/.test(s) && s.length <= 64) || + /^[A-Z][A-Z0-9]*(_[A-Z0-9]+)+$/.test(s) || // enum names: SERVER_ACK + /^[A-Z]{2,12}$/.test(s) || // single-word enums: READ, PLAYED + /^[a-z][a-zA-Z0-9]*([._-][a-zA-Z0-9]+)*$/.test(s) || // identifiers: messages.upsert, imageMessage + /^[a-z]+\/[\w.+-]+(;\s*[\w=.-]+)*$/.test(s) || // mimetypes + /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d+)?(Z|[+-]\d{2}:\d{2})?$/.test(s); // ISO dates + +const LOREM = 'lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod tempor incididunt ut labore '; + +/** A disjoint set over "pn:" / "lid:", so one person's forms share a root. */ +class People { + private parent = new Map(); + private index = new Map(); + find(x: string): string { + if (!this.parent.has(x)) this.parent.set(x, x); + const p = this.parent.get(x); + if (p === x) return x; + const root = this.find(p); + this.parent.set(x, root); + return root; + } + union(a: string, b: string) { + const [ra, rb] = [this.find(a), this.find(b)]; + if (ra === rb) return; + // Keep an indexed root (the owner, seeded first) as the root. + if (this.index.has(rb) && !this.index.has(ra)) this.parent.set(ra, rb); + else this.parent.set(rb, ra); + } + indexOf(x: string): number { + const root = this.find(x); + if (!this.index.has(root)) this.index.set(root, this.index.size); + return this.index.get(root); + } + get count() { + return this.index.size; + } +} + +type Line = Record; + +class Scrubber { + readonly people = new People(); + private groups = new Map(); + private names = new Map(); + private texts = new Map(); + private ids = new Map(); + private bytes = new Map(); + private urls = new Map(); + private digits = new Map(); + /** Raw strings the rewrite replaced, and raw strings it kept as structure. */ + readonly replaced = new Set(); + readonly kept = new Set(); + + constructor(private readonly instanceName: string) {} + + /** Step 1: pair phone and @lid where one object names both. */ + collect(value: any) { + if (Array.isArray(value)) return value.forEach((v) => this.collect(v)); + if (!value || typeof value !== 'object' || '$bytes' in value) return; + const pns = new Set(); + const lids = new Set(); + for (const v of Object.values(value)) { + const m = typeof v === 'string' ? JID.exec(v) : null; + if (m && PN_SERVERS.has(m[3])) pns.add(m[1]); + if (m && LID_SERVERS.has(m[3])) lids.add(m[1]); + } + if (pns.size === 1 && lids.size === 1) this.people.union(`pn:${[...pns][0]}`, `lid:${[...lids][0]}`); + for (const v of Object.values(value)) this.collect(v); + } + + seedOwner(owner: { id?: string; lid?: string; name?: string }) { + const pn = owner.id && JID.exec(owner.id); + const lid = owner.lid && JID.exec(owner.lid); + if (pn) this.people.indexOf(`pn:${pn[1]}`); + if (pn && lid) this.people.union(`pn:${pn[1]}`, `lid:${lid[1]}`); + else if (lid) this.people.indexOf(`lid:${lid[1]}`); + if (owner.name) this.names.set(owner.name, 'Owner'); + } + + /** Step 2. */ + rewrite(value: any, key = ''): any { + if (typeof value === 'string') return this.string(value, key); + if (typeof value === 'number' && Number.isInteger(value) && value >= 1e6) { + const known = this.digits.get(String(value)); + return known ? Number(known) : value; + } + if (Array.isArray(value)) return value.map((v) => this.rewrite(v, key)); + if (!value || typeof value !== 'object') return value; + if ('$bytes' in value) return { $bytes: this.fakeBytes(value.$bytes), as: value.as, fake: 1 }; + if ('$long' in value || '$date' in value || '$big' in value || '$u' in value || '$fn' in value) return value; + const out: Record = {}; + for (const [k, v] of Object.entries(value)) { + const newKey = JID.test(k) || /^\d{7,15}$/.test(k) ? this.string(k, '') : k; + out[newKey] = this.rewrite(v, k); + } + return out; + } + + private string(s: string, key: string): string { + const out = this.stringOf(s, key); + if (out === s) this.kept.add(s); + else this.replaced.add(s); + return out; + } + + private stringOf(s: string, key: string): string { + if (!s) return s; + if (s === this.instanceName && (key === 'instance' || key === 'instanceName')) return REPLAY_INSTANCE; + const jid = JID.exec(s); + if (jid) return this.fakeJid(jid[1], jid[2], jid[3]); + if (/^\+?\d{7,15}$/.test(s)) { + const plus = s.startsWith('+') ? '+' : ''; + const digits = s.slice(plus.length); + if (this.digits.has(digits)) return plus + this.digits.get(digits); + if (isEpoch(digits)) return s; + return plus + this.fakeUser(digits, 's.whatsapp.net'); + } + if (NAME_KEYS.has(key)) return this.fakeName(s); + if (TEXT_KEYS.has(key)) return this.lorem(s); + if (this.bytes.has(s)) return this.bytes.get(s); + if (s.length >= 8 && (ID_KEYS.has(key) || /^(?=.*\d)[0-9A-F]{12,64}$/.test(s))) return this.fakeId(s); + if (/^https?:\/\//i.test(s)) return this.memo(this.urls, s, (n) => `https://example.invalid/${n}`); + if (isStructural(key, s)) return s; + return this.lorem(s); + } + + private fakeJid(user: string, suffix: string, server: string) { + if (SERVICE_USERS.has(user)) return `${user}${suffix}@${server}`; + return `${this.fakeUser(user, server)}${suffix}@${server}`; + } + + private fakeUser(user: string, server: string): string { + if (server === 'g.us' || server === 'broadcast' || server === 'newsletter') { + const [first, ts] = user.split('-'); + if (ts) return `${this.fakeUser(first, 's.whatsapp.net')}-${ts}`; // an old group: creator phone and time + if (!this.groups.has(user)) this.groups.set(user, this.groups.size + 1); + return this.remember(user, FAKE_GROUP(this.groups.get(user))); + } + if (LID_SERVERS.has(server)) return this.remember(user, FAKE_LID(this.people.indexOf(`lid:${user}`))); + return this.remember(user, FAKE_PN(this.people.indexOf(`pn:${user}`))); + } + + private remember(original: string, fake: string) { + this.digits.set(original, fake); + return fake; + } + + private fakeName(s: string) { + if (s === 'Você') return s; + return this.memo(this.names, s, (n) => `Name ${n}`); + } + + private fakeId(s: string) { + return this.memo(this.ids, s, (n) => { + const body = n.toString(16).toUpperCase().padStart(s.length - 2, 'F'); + const id = s.slice(0, 2) + body.slice(-(s.length - 2)); + return s === s.toLowerCase() ? id.toLowerCase() : id; + }); + } + + private fakeBytes(b64: string) { + return this.memo(this.bytes, b64, () => randomBytes(Buffer.from(b64, 'base64').length).toString('base64')); + } + + private lorem(s: string) { + return this.memo(this.texts, s, (n) => { + const start = (n * 7) % LOREM.length; + return LOREM.repeat(Math.ceil((s.length + start) / LOREM.length) + 1).slice(start, start + s.length); + }); + } + + private memo(map: Map, s: string, make: (n: number) => string) { + if (!map.has(s)) map.set(s, make(map.size + 1)); + return map.get(s); + } + + /** Every original the gate searches for: numbers, names, byte strings, and replaced strings of 4+ characters. */ + originals(): string[] { + const all = new Set(); + for (const d of this.digits.keys()) all.add(d); + for (const n of this.names.keys()) if (n.length > 3 && n !== 'Você') all.add(n); + for (const b of this.bytes.keys()) if (b.length > 3) all.add(b); + for (const s of this.replaced) if (s.length > 3 && !this.kept.has(s)) all.add(s); + return [...all]; + } + + counts() { + return { + people: this.people.count, + groups: this.groups.size, + names: this.names.size, + texts: this.texts.size, + messageIds: this.ids.size, + bytes: this.bytes.size, + urls: this.urls.size, + }; + } +} + +const readLines = (file: string): Line[] => + existsSync(file) + ? readFileSync(file, 'utf8') + .split('\n') + .filter((l) => l.trim()) + .map((l) => JSON.parse(l)) + : []; + +const describeOriginal = (s: string) => + /^\d+$/.test(s) ? `a number of ${s.length} digits` : `a string of ${s.length} characters`; + +/** Scrub one raw session. Returns the fixture directory; throws LeakError (writing nothing) on a leak. */ +export function scrubSession(rawDir: string, opts: ScrubOptions) { + if (!/^[a-z0-9]+(-[a-z0-9]+)*$/.test(opts.checkId)) throw new Error(`check id must be kebab-case: ${opts.checkId}`); + const date = opts.date ?? new Date().toISOString().slice(0, 10); + if (!/^\d{4}-\d{2}-\d{2}$/.test(date)) throw new Error('date must be YYYY-MM-DD'); + const operator = opts.operator ?? {}; + if (operator.countryCode !== undefined && !/^\d{1,3}$/.test(operator.countryCode)) { + throw new Error('country code only (1 to 3 digits), never a phone number'); + } + + const instanceName = basename(dirname(rawDir)); + const events = readLines(join(rawDir, 'events.ndjson')); + const webhooks = readLines(join(rawDir, 'webhooks.ndjson')); + const rawManifest = JSON.parse(readFileSync(join(rawDir, 'manifest.json'), 'utf8')); + const ownerFile = join(rawDir, 'owner.json'); + const owner = existsSync(ownerFile) ? JSON.parse(readFileSync(ownerFile, 'utf8')) : {}; + + const scrubber = new Scrubber(instanceName); + scrubber.seedOwner(owner); + for (const line of [...events, ...webhooks]) scrubber.collect(line); + + const files: Record = {}; + const tape = (lines: Line[]) => lines.map((l) => JSON.stringify(scrubber.rewrite(l)) + '\n').join(''); + files['events.ndjson'] = tape(events); + files['webhooks.ndjson'] = tape(webhooks); + const fakeOwner = scrubber.rewrite({ id: owner.id, lid: owner.lid, name: owner.name }); + const manifest = { + ...rawManifest, + checkId: opts.checkId, + date, + phoneModel: operator.phoneModel ?? null, + osVersion: operator.osVersion ?? null, + whatsappAppVersion: operator.whatsappAppVersion ?? null, + countryCode: operator.countryCode ?? null, + replay: { owner: fakeOwner }, + }; + files['manifest.json'] = JSON.stringify(manifest, null, 2) + '\n'; + + // Step 3, the leak gate: fail closed. + const hits: string[] = []; + for (const original of scrubber.originals()) { + const escaped = JSON.stringify(original).slice(1, -1); + for (const [file, text] of Object.entries(files)) { + const at = text.includes(original) ? text.indexOf(original) : text.indexOf(escaped); + if (at >= 0) hits.push(`${file} line ${text.slice(0, at).split('\n').length}: ${describeOriginal(original)}`); + } + } + if (hits.length) { + throw new LeakError(`leak gate: ${hits.length} original(s) survived, nothing written:\n ${hits.join('\n ')}`); + } + + const report = { leakGate: 'pass', events: events.length, webhooks: webhooks.length, ...scrubber.counts() }; + files['scrub-report.json'] = JSON.stringify(report, null, 2) + '\n'; + + const dir = join(opts.outRoot ?? join(process.cwd(), 'test', 'fixtures', 'live'), `${date}-${opts.checkId}`); + if (existsSync(dir)) throw new Error(`${dir} exists: remove it or pick another check id`); + mkdirSync(dir, { recursive: true }); + for (const [file, text] of Object.entries(files)) writeFileSync(join(dir, file), text); + return { dir, report }; +} From 03fc07a69cd988fffc66c1c2d6f6d2da4097937e Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:45:51 +0300 Subject: [PATCH 096/157] test: a scrubbed live-check fixture replays through the real buffer and service and reproduces its webhooks Co-Authored-By: Claude Opus 5.5 --- test/live/replay.test.ts | 70 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) create mode 100644 test/live/replay.test.ts diff --git a/test/live/replay.test.ts b/test/live/replay.test.ts new file mode 100644 index 0000000000..c05c041352 --- /dev/null +++ b/test/live/replay.test.ts @@ -0,0 +1,70 @@ +// A scrubbed live-check fixture replays through the real event buffer into the +// real BaileysStartupService, and what Evolution sends must match the golden +// webhooks the live session recorded (scrubbed with the same identity table). +// This is the end-to-end proof of the chain: record, scrub, replay. +import { vi } from 'vitest'; + +const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + return { ...orig, default: socketSpy, makeWASocket: socketSpy }; +}); +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); + +import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; + +import { fakeSocket } from '../helpers/connect'; +import { emitted } from '../helpers/fake-server-module'; +import { compareGolden, loadFixture, replayFixture } from '../helpers/live-replay'; +import { recordSession } from '../helpers/live-session'; +import { scrubSession } from '../tools/live-scrub'; + +socketSpy.mockImplementation(fakeSocket); + +let root: string; +let fixture: string; +beforeEach(async () => { + root = mkdtempSync(join(tmpdir(), 'live-replay-')); + const raw = await recordSession(join(root, 'raw')); + fixture = scrubSession(raw, { checkId: 'synthetic-session', date: '2026-09-27', outRoot: join(root, 'fixtures') }).dir; + emitted.splice(0); +}); +afterEach(() => rmSync(root, { recursive: true, force: true })); + +describe('live-check replay', () => { + it('replays a scrubbed session through the real buffer and service, and reproduces its webhooks', async () => { + const { webhooks } = await replayFixture(fixture); + expect(webhooks.map((w) => w.event)).toContain('messages.upsert'); + expect(compareGolden(webhooks, loadFixture(fixture).webhooks)).toEqual([]); + }); + + it('delivers the batches the live buffer delivered', async () => { + const { batches } = await replayFixture(fixture); + const recorded = loadFixture(fixture) + .events.filter((e) => e.batch) + .map((e) => e.batch); + expect(batches).toEqual(recorded); + }); + + it('fails the golden comparison when what Evolution sends differs', async () => { + const file = join(fixture, 'webhooks.ndjson'); + const golden = readFileSync(file, 'utf8').trim().split('\n').map((l) => JSON.parse(l)); + const upsert = golden.find((w) => w.event === 'messages.upsert'); + upsert.data.key.remoteJidAlt = upsert.data.key.remoteJid; // what 2.3.7 sent: the phone twice + writeFileSync(file, golden.map((w) => JSON.stringify(w)).join('\n') + '\n'); + + const { webhooks } = await replayFixture(fixture); + const diff = compareGolden(webhooks, loadFixture(fixture).webhooks); + expect(diff).toHaveLength(2); + expect(diff.join('\n')).toMatch(/missing messages\.upsert/); + expect(diff.join('\n')).toMatch(/unexpected messages\.upsert/); + }); +}); From 00735ff22b6bddc4513fdbfd58f9e2c6d49f8102 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:46:35 +0300 Subject: [PATCH 097/157] feat(live-record): replay a scrubbed fixture through the real buffer and service, with a golden comparison Co-Authored-By: Claude Opus 5.5 --- test/helpers/live-replay.ts | 113 ++++++++++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+) create mode 100644 test/helpers/live-replay.ts diff --git a/test/helpers/live-replay.ts b/test/helpers/live-replay.ts new file mode 100644 index 0000000000..fd3055c656 --- /dev/null +++ b/test/helpers/live-replay.ts @@ -0,0 +1,113 @@ +// Replays a scrubbed live-check fixture (test/fixtures/live/-/) +// through Baileys' real event buffer into the real BaileysStartupService, and +// compares what Evolution sends with the webhooks the live session recorded. +// +// The replay follows the tape: an event the buffer held is emitted inside a +// buffer, a batch line flushes it, an event Evolution emitted itself +// (origin: app) is left for the replayed Evolution to emit again, and events +// from a socket Evolution had already replaced are skipped (it ignored them +// live). Socket queries (profile pictures, group metadata, LID lookups) were +// not recorded: they answer as the harness does unless a test passes `client`. +import { readFileSync } from 'node:fs'; +import { join } from 'node:path'; + +import { decode, encode } from '@utils/live-record/codec'; + +import { makeService, settle } from './baileys-service'; +import { emitted } from './fake-server-module'; +import type { Profile } from './profiles'; + +type Line = Record; + +const readLines = (file: string): Line[] => + readFileSync(file, 'utf8') + .split('\n') + .filter((l) => l.trim()) + .map((l) => JSON.parse(l)); + +export function loadFixture(dir: string) { + return { + events: readLines(join(dir, 'events.ndjson')), + webhooks: readLines(join(dir, 'webhooks.ndjson')), + manifest: JSON.parse(readFileSync(join(dir, 'manifest.json'), 'utf8')), + }; +} + +export async function replayFixture(dir: string, opts: { profile?: Profile; client?: Record } = {}) { + const { events, manifest } = loadFixture(dir); + const { service, ev, prisma } = await makeService({ profile: opts.profile }); + if (manifest.replay?.owner?.id) service.client.user = manifest.replay.owner; + Object.assign(service.client, opts.client); + // A close must not build a real socket. + service.connectToWhatsapp = async () => service.client; + service.scheduleReconnect = () => undefined; + + const batches: string[][] = []; + ev.process((batch: Record) => void batches.push(Object.keys(batch))); + service.eventHandler(); + service.__wired = true; + const start = emitted.length; + + // Live, each batch was handled before the next one arrived. + const flush = async () => { + if (ev.isBuffering()) ev.flush(); + await service.eventProcessingQueue; + }; + let socket = 0; + for (const line of events) { + if (line.socket < socket) continue; + socket = line.socket; + if (line.batch) { + await flush(); + continue; + } + if (line.origin === 'app') continue; + if (!line.buffered) await flush(); + if (line.buffered && !ev.isBuffering()) ev.buffer(); + ev.emit(line.event, decode(line.data)); + } + await flush(); + await settle(service); + return { service, prisma, batches, webhooks: emitted.slice(start) }; +} + +/** Fields Evolution fills from the clock, the database or a socket query: not part of the comparison. */ +export const VOLATILE = ['dateTime', 'date_time', 'createdAt', 'updatedAt', 'instanceId', 'profilePicUrl', 'profilePictureUrl']; + +function normalize(value: any, volatile: Set): any { + if (Array.isArray(value)) return value.map((v) => normalize(v, volatile)); + if (!value || typeof value !== 'object') return value; + if ('$bytes' in value) return { $bytes: value.$bytes, as: value.as }; // the scrubber tags fakes + const out: Record = {}; + for (const [k, v] of Object.entries(value)) out[k] = volatile.has(k) ? '' : normalize(v, volatile); + return out; +} + +/** + * The differences between what the replay sent and the golden webhooks, as + * "missing " / "unexpected " lines; empty when they match. The + * comparison is of the two multisets, since background lookups interleave. + * `events` limits it to those event names (webhooks a live API call caused have + * no event on the tape). + */ +export function compareGolden( + actual: { event: string; data: any }[], + golden: Line[], + opts: { events?: string[]; volatile?: string[] } = {}, +) { + const volatile = new Set([...VOLATILE, ...(opts.volatile ?? [])]); + const keep = (event: string) => !opts.events || opts.events.includes(event); + const canon = (event: string, encoded: any) => JSON.stringify({ event, data: normalize(encoded, volatile) }); + const want = golden.filter((w) => keep(w.event)).map((w) => canon(w.event, w.data)); + const got = actual.filter((w) => keep(w.event)).map((w) => canon(w.event, encode(w.data))); + + const diff: string[] = []; + const left = [...got]; + for (const w of want) { + const i = left.indexOf(w); + if (i >= 0) left.splice(i, 1); + else diff.push(`missing ${JSON.parse(w).event}: ${w}`); + } + for (const g of left) diff.push(`unexpected ${JSON.parse(g).event}: ${g}`); + return diff; +} From 8ccda96578170a12c8362f58bbbc5dd382ad454a Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:47:19 +0300 Subject: [PATCH 098/157] test: a guard finds personal data in live fixtures and names only the file, line and kind Co-Authored-By: Claude Opus 5.5 --- test/live/fixture-guard.test.ts | 97 +++++++++++++++++++++++++++++++++ 1 file changed, 97 insertions(+) create mode 100644 test/live/fixture-guard.test.ts diff --git a/test/live/fixture-guard.test.ts b/test/live/fixture-guard.test.ts new file mode 100644 index 0000000000..49f44c03b5 --- /dev/null +++ b/test/live/fixture-guard.test.ts @@ -0,0 +1,97 @@ +// Nothing personal may reach the repository through a live-check fixture. The +// scrubber's leak gate checks its own output against the originals it saw; this +// guard is the second, independent check, run on every commit: it scans every +// file under test/fixtures/live/ for what personal data looks like, knowing only +// the scrubber's fake ranges. A finding names the file, the line, the path and +// the kind of value, never the value. +import { vi } from 'vitest'; + +const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + return { ...orig, default: socketSpy, makeWASocket: socketSpy }; +}); +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); + +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; + +import { fakeSocket } from '../helpers/connect'; +import { recordSession } from '../helpers/live-session'; +import { formatFindings, scanFixtures } from '../tools/fixture-guard'; +import { scrubSession } from '../tools/live-scrub'; + +socketSpy.mockImplementation(fakeSocket); + +let root: string; +beforeEach(() => void (root = mkdtempSync(join(tmpdir(), 'live-guard-')))); +afterEach(() => rmSync(root, { recursive: true, force: true })); + +const PLANTED = { + phone: '972541112233', + pnJid: '972541112233:3@s.whatsapp.net', + lidJid: '123456789012345@lid', + url: 'https://mmg.whatsapp.net/v/t62.7118-24/19_A.enc?ccb=11-4&oh=01_Q5AaIBq&oe=68D1A2B3', + email: 'dana.levi@gmail.com', + bytes: 'q83vEjRWeJq8Dd7wESIzRFVmd4iZqrvM3e7/ABEiM0Q=', +}; + +describe('live fixture guard', () => { + it('finds every kind of leak in a planted fixture, and never prints the value', () => { + const dir = join(root, 'live', '2026-09-27-planted'); + mkdirSync(dir, { recursive: true }); + const clean = { seq: 1, event: 'contacts.upsert', data: [{ id: '972500000001@s.whatsapp.net', name: 'Name 2' }] }; + const lines = [ + clean, + { seq: 2, event: 'messages.upsert', data: { text: `call me on ${PLANTED.phone}` } }, + { seq: 3, event: 'contacts.upsert', data: [{ id: PLANTED.pnJid, lid: PLANTED.lidJid }] }, + { seq: 4, event: 'messages.upsert', data: { imageMessage: { url: PLANTED.url } } }, + { seq: 5, event: 'contacts.update', data: [{ about: `write to ${PLANTED.email}` }] }, + { seq: 6, event: 'messages.upsert', data: { mediaKey: { $bytes: PLANTED.bytes, as: 'Uint8Array' } } }, + { seq: 7, event: 'messages.upsert', data: { jpegThumbnail: PLANTED.bytes } }, + { seq: 8, event: 'presence.update', data: { presences: { [PLANTED.pnJid]: { lastKnownPresence: 'available' } } } }, + ]; + writeFileSync(join(dir, 'events.ndjson'), lines.map((l) => JSON.stringify(l)).join('\n') + '\n'); + writeFileSync(join(dir, 'manifest.json'), JSON.stringify({ note: `owner ${PLANTED.phone}` })); + + const findings = scanFixtures(join(root, 'live')); + const kinds = findings.map((f) => `${f.file}:${f.line} ${f.kind}`); + expect(kinds).toEqual( + expect.arrayContaining([ + expect.stringMatching(/events\.ndjson:2 phone-like digit run/), + expect.stringMatching(/events\.ndjson:3 address not in the fake ranges/), + expect.stringMatching(/events\.ndjson:4 signed media URL/), + expect.stringMatching(/events\.ndjson:5 email address/), + expect.stringMatching(/events\.ndjson:6 bytes not tagged fake/), + expect.stringMatching(/events\.ndjson:7 base64 blob/), + expect.stringMatching(/events\.ndjson:8 address not in the fake ranges/), + expect.stringMatching(/manifest\.json:1 phone-like digit run/), + ]), + ); + // The clean line is clean. + expect(findings.some((f) => f.file.endsWith('events.ndjson') && f.line === 1)).toBe(false); + + const report = formatFindings(findings); + for (const value of [...Object.values(PLANTED), '972541112233', '123456789012345', 'dana.levi']) { + expect(report.includes(value), `the report printed a planted value of ${value.length} chars`).toBe(false); + } + }); + + it('passes the scrubber output of a recorded session', async () => { + const raw = await recordSession(join(root, 'raw')); + scrubSession(raw, { checkId: 'synthetic-session', date: '2026-09-27', outRoot: join(root, 'live') }); + expect(formatFindings(scanFixtures(join(root, 'live')))).toBe(''); + }); + + it('passes every fixture committed under test/fixtures/live/', () => { + const findings = scanFixtures(join(process.cwd(), 'test', 'fixtures', 'live')); + expect(formatFindings(findings)).toBe(''); + }); +}); From 5372d713bdfd18d1ad702100d065340899c0350f Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:48:38 +0300 Subject: [PATCH 099/157] feat(live-record): a guard that scans live fixtures for personal data on every commit test/tools/fixture-guard.ts knows only the scrubber's fake ranges and flags real-looking addresses, 8+ digit runs that are neither fakes nor epoch timestamps, signed media URLs, emails, bytes not tagged fake and other long base64 blobs, naming the file, line, masked path and kind, never the value. The vitest run covers test/fixtures/live/ in CI; lint-staged runs scripts/live-guard.ts on staged fixture files (about half a second). Co-Authored-By: Claude Opus 5.5 --- package.json | 3 + scripts/live-guard.ts | 16 +++++ test/tools/fixture-guard.ts | 130 ++++++++++++++++++++++++++++++++++++ 3 files changed, 149 insertions(+) create mode 100644 scripts/live-guard.ts create mode 100644 test/tools/fixture-guard.ts diff --git a/package.json b/package.json index b34e0134d5..d8cf259a1a 100644 --- a/package.json +++ b/package.json @@ -58,6 +58,9 @@ ], "src/**/*.ts": [ "sh -c 'tsc --noEmit'" + ], + "test/fixtures/live/**": [ + "tsx scripts/live-guard.ts" ] }, "config": { diff --git a/scripts/live-guard.ts b/scripts/live-guard.ts new file mode 100644 index 0000000000..413fcd47b5 --- /dev/null +++ b/scripts/live-guard.ts @@ -0,0 +1,16 @@ +// Scan live-check fixtures for personal data (test/tools/fixture-guard.ts). +// +// npx tsx scripts/live-guard.ts [files or directories...] (default: test/fixtures/live) +// +// Prints the file, line, masked path and kind of each finding, never the value, +// and exits 1 when there is one. lint-staged runs it on staged fixture files. +import { formatFindings, scanFixtures } from '../test/tools/fixture-guard'; + +const paths = process.argv.slice(2); +const findings = scanFixtures(...(paths.length ? paths : ['test/fixtures/live'])); +if (findings.length) { + console.error(`live fixture guard: ${findings.length} finding(s). Fix the scrubber and re-scrub; never edit a fixture by hand.`); + console.error(formatFindings(findings)); + process.exit(1); +} +console.log('live fixture guard: clean'); diff --git a/test/tools/fixture-guard.ts b/test/tools/fixture-guard.ts new file mode 100644 index 0000000000..c49c298e86 --- /dev/null +++ b/test/tools/fixture-guard.ts @@ -0,0 +1,130 @@ +// Scans live-check fixtures for what personal data looks like, independently of +// the scrubber that made them: it knows only the scrubber's fake ranges. +// +// an address (JID) whose user part is not a fake: 972500<6>, 100000000<6>, 120363<12> +// a run of 8+ digits that is neither a fake nor an epoch timestamp (s or ms) +// a signed media URL (mmg/pps/media*.whatsapp.net, oh= / oe= parameters) +// an email address +// bytes ({"$bytes"}) of 16+ bytes not tagged fake by the scrubber, and any other +// base64 blob of 24+ characters that is not one of those fake bytes +// +// A finding names the file, the line, a masked JSON path and the kind of value, +// never the value itself. Run on every commit by test/live/fixture-guard.test.ts, +// and by scripts/live-guard.ts from lint-staged. +import { existsSync, readdirSync, readFileSync, statSync } from 'node:fs'; +import { join, relative } from 'node:path'; + +export type Finding = { file: string; line: number; path: string; kind: string }; + +const FAKE_USER = [/^972500\d{6}$/, /^100000000\d{6}$/, /^120363\d{12}$/, /^(0|16505361212|13135550002)$/]; +const JID = /(\d+(?:-\d+)?)(?:[:_]\d+)*@(s\.whatsapp\.net|c\.us|hosted\.lid|hosted|lid|g\.us|broadcast|newsletter)\b/g; +const WA_DOMAIN = /^(s\.whatsapp\.net|c\.us|g\.us|lid|hosted|hosted\.lid|broadcast|newsletter)$/; +const EMAIL = /[A-Za-z0-9._%+-]+@((?:[A-Za-z0-9-]+\.)+[A-Za-z]{2,})/g; +const SIGNED_URL = /(mmg|pps|media[\w.-]*)\.whatsapp\.net|[?&](oh|oe)=/i; +const BASE64 = /^[A-Za-z0-9+/]{24,}={0,2}$/; +/** Numbers under these keys are sizes, counts and times, not people. */ +const NUMERIC_KEY = /(length|size|seconds|duration|count|progress|timestamp|time|^t$|^seq$|at$|height|width|expiration|ttl)/i; + +const isEpoch = (d: string) => /^1\d{9}$/.test(d) || /^1\d{12}$/.test(d); +const isFakeUser = (user: string) => + user.split('-').every((part, i) => (i === 0 ? FAKE_USER.some((r) => r.test(part)) : isEpoch(part))); +/** A path segment that could itself be the leak (an address or a number used as a key) is masked. */ +const mask = (segment: string) => (/\d{5,}|@/.test(segment) ? '' : segment); + +function scanString(s: string, report: (kind: string) => void, fakeBytes: Set) { + let rest = s; + for (const m of s.matchAll(JID)) { + if (!isFakeUser(m[1])) report('address not in the fake ranges'); + rest = rest.replace(m[0], ' '); + } + for (const m of rest.matchAll(EMAIL)) if (!WA_DOMAIN.test(m[1])) report('email address'); + if (SIGNED_URL.test(s)) report('signed media URL'); + for (const run of rest.match(/\d{8,}/g) ?? []) { + if (!isFakeUser(run) && !isEpoch(run)) report('phone-like digit run'); + } + if (BASE64.test(s) && /[a-z]/.test(s) && /[A-Z]/.test(s) && !fakeBytes.has(s)) report('base64 blob'); +} + +function walk(value: any, path: string[], report: (kind: string, path: string[]) => void, fakeBytes: Set) { + const at = (kind: string) => report(kind, path); + // A commit hash can hold eight digits in a row. + if (typeof value === 'string' && path[path.length - 1] === 'forkCommit' && /^[0-9a-f]{7,40}(-dirty)?$/.test(value)) return; + if (typeof value === 'string') return scanString(value, at, fakeBytes); + if (typeof value === 'number') { + const key = path[path.length - 1] ?? ''; + if (!NUMERIC_KEY.test(key)) scanString(String(value), at, fakeBytes); + return; + } + if (Array.isArray(value)) return value.forEach((v, i) => walk(v, [...path, String(i)], report, fakeBytes)); + if (!value || typeof value !== 'object') return; + if (typeof value.$bytes === 'string') { + if (!value.fake && Buffer.from(value.$bytes, 'base64').length >= 16) at('bytes not tagged fake'); + return; + } + if (typeof value.$long === 'string') { + const key = path[path.length - 1] ?? ''; + if (!NUMERIC_KEY.test(key)) scanString(value.$long.replace('-', ''), at, fakeBytes); + return; + } + for (const [k, v] of Object.entries(value)) { + scanString(k, (kind) => report(kind, [...path, k]), fakeBytes); + walk(v, [...path, k], report, fakeBytes); + } +} + +/** Every fake-tagged byte string in a fixture: a plain copy of one elsewhere is not a leak. */ +function collectFakeBytes(value: any, into: Set) { + if (Array.isArray(value)) return value.forEach((v) => collectFakeBytes(v, into)); + if (!value || typeof value !== 'object') return; + if (typeof value.$bytes === 'string' && value.fake) into.add(value.$bytes); + for (const v of Object.values(value)) collectFakeBytes(v, into); +} + +const filesUnder = (dir: string): string[] => + readdirSync(dir).flatMap((name) => { + const full = join(dir, name); + return statSync(full).isDirectory() ? filesUnder(full) : [full]; + }); + +/** Parse a file into [line number, JSON value or raw text] units. */ +function unitsOf(file: string): [number, any][] { + const text = readFileSync(file, 'utf8'); + const parse = (s: string) => { + try { + return JSON.parse(s); + } catch { + return s; // not JSON: scanned as text + } + }; + if (file.endsWith('.ndjson')) { + return text.split('\n').flatMap((l, i): [number, any][] => (l.trim() ? [[i + 1, parse(l)]] : [])); + } + return [[1, parse(text)]]; +} + +/** Scan files, or every file under a directory. A missing path has nothing to find. */ +export function scanFixtures(...paths: string[]): Finding[] { + const files = paths.flatMap((p) => (!existsSync(p) ? [] : statSync(p).isDirectory() ? filesUnder(p) : [p])); + const units = files.map((file) => ({ file, units: unitsOf(file) })); + const fakeBytes = new Set(); + for (const { units: us } of units) for (const [, v] of us) collectFakeBytes(v, fakeBytes); + + const findings: Finding[] = []; + for (const { file, units: us } of units) { + for (const [line, value] of us) { + walk( + value, + [], + (kind, path) => + findings.push({ file: relative(process.cwd(), file), line, path: '$.' + path.map(mask).join('.'), kind }), + fakeBytes, + ); + } + } + return findings; +} + +/** One line per finding, no values; empty when there are none. */ +export function formatFindings(findings: Finding[]) { + return findings.map((f) => `${f.file}:${f.line} ${f.kind} at ${f.path}`).join('\n'); +} From 797e52ee6bd520926742c1b90c76c97cdd6648aa Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:51:37 +0300 Subject: [PATCH 100/157] style: lint and prettier on the live-record files Co-Authored-By: Claude Opus 5.5 --- scripts/live-scrub.ts | 4 ++- .../whatsapp/whatsapp.baileys.service.ts | 2 +- src/utils/live-record/codec.ts | 5 ++- test/tools/live-scrub.ts | 36 +++++++++++-------- 4 files changed, 29 insertions(+), 18 deletions(-) diff --git a/scripts/live-scrub.ts b/scripts/live-scrub.ts index cef67a3c50..3fc5d7463a 100644 --- a/scripts/live-scrub.ts +++ b/scripts/live-scrub.ts @@ -16,7 +16,9 @@ for (let i = 0; i < args.length; i++) { } const [rawDir, checkId] = positional; if (!rawDir || !checkId) { - console.error('usage: npx tsx scripts/live-scrub.ts [--date YYYY-MM-DD] [--phone-model ...] [--os-version ...] [--wa-version ...] [--country-code ...] [--out dir]'); + console.error( + 'usage: npx tsx scripts/live-scrub.ts [--date YYYY-MM-DD] [--phone-model ...] [--os-version ...] [--wa-version ...] [--country-code ...] [--out dir]', + ); process.exit(2); } diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index d3049879d3..4822c20656 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -84,9 +84,9 @@ import { Instance, Message } from '@prisma/client'; import { chatState } from '@utils/chat-state'; import { createJid } from '@utils/createJid'; import { fetchLatestWaWebVersion } from '@utils/fetchLatestWaWebVersion'; +import { LiveRecorder } from '@utils/live-record/recorder'; import { errorFields, jidKind, makeBaileysLogger } from '@utils/log-privacy'; import { readLogoutMarker, writeLogoutMarker } from '@utils/logout-marker'; -import { LiveRecorder } from '@utils/live-record/recorder'; import { makeProxyAgent, makeProxyAgentUndici } from '@utils/makeProxyAgent'; import { getOnWhatsappCache, saveOnWhatsappCache } from '@utils/onWhatsappCache'; import { QueryLimiter } from '@utils/queryLimiter'; diff --git a/src/utils/live-record/codec.ts b/src/utils/live-record/codec.ts index 22b040dfb7..4c5de1746b 100644 --- a/src/utils/live-record/codec.ts +++ b/src/utils/live-record/codec.ts @@ -40,7 +40,10 @@ export function encode(value: any): any { } const protoClass = (name: string) => { - const Ctor = name.split('.').slice(1).reduce((node: any, part) => node?.[part], proto); + const Ctor = name + .split('.') + .slice(1) + .reduce((node: any, part) => node?.[part], proto); if (typeof Ctor !== 'function') throw new Error(`live-record: unknown protobuf class ${name}`); return Ctor; }; diff --git a/test/tools/live-scrub.ts b/test/tools/live-scrub.ts index c5c33fce28..298e9874e9 100644 --- a/test/tools/live-scrub.ts +++ b/test/tools/live-scrub.ts @@ -39,20 +39,23 @@ const JID = /^(\d+(?:-\d+)?)((?:[:_]\d+)*)@(s\.whatsapp\.net|c\.us|hosted|lid|ho const PN_SERVERS = new Set(['s.whatsapp.net', 'c.us', 'hosted']); const LID_SERVERS = new Set(['lid', 'hosted.lid']); -const NAME_KEYS = new Set([ - 'name', 'notify', 'verifiedName', 'verifiedBizName', 'pushName', 'username', 'subject', 'profileName', - 'fullName', 'firstName', 'shortName', 'displayName', 'vname', -]); -const TEXT_KEYS = new Set([ - 'conversation', 'text', 'caption', 'desc', 'description', 'title', 'body', 'matchedText', 'canonicalUrl', - 'fileName', 'address', 'contentText', 'footerText', 'headerText', 'vcard', 'selectedDisplayText', 'optionName', - 'comment', 'message', 'msgCall', -]); -const ID_KEYS = new Set(['id', 'stanzaId', 'keyId', 'messageId', 'callId']); -const STRUCTURAL_KEYS = new Set([ - '$proto', '$redacted', 'as', 'event', 'type', 'messageType', 'mimetype', 'addressingMode', 'action', 'connection', - 'source', 'origin', 'state', 'status', 'platform', 'mediaType', -]); +/** Key lists, one word per key. */ +const words = (list: string) => new Set(list.trim().split(/\s+/)); +const NAME_KEYS = words(` + name notify verifiedName verifiedBizName pushName username subject profileName fullName + firstName shortName displayName vname +`); +const TEXT_KEYS = words(` + conversation text caption desc description title body matchedText canonicalUrl fileName address + contentText footerText headerText vcard selectedDisplayText optionName comment message msgCall +`); +const ID_KEYS = words(` + id stanzaId keyId messageId callId +`); +const STRUCTURAL_KEYS = words(` + $proto $redacted as event type messageType mimetype addressingMode action connection source + origin state status platform mediaType +`); const isEpoch = (digits: string) => /^1\d{9}$/.test(digits) || /^1\d{12}$/.test(digits); const isStructural = (key: string, s: string) => @@ -211,7 +214,10 @@ class Scrubber { private fakeId(s: string) { return this.memo(this.ids, s, (n) => { - const body = n.toString(16).toUpperCase().padStart(s.length - 2, 'F'); + const body = n + .toString(16) + .toUpperCase() + .padStart(s.length - 2, 'F'); const id = s.slice(0, 2) + body.slice(-(s.length - 2)); return s === s.toLowerCase() ? id.toLowerCase() : id; }); From 71b5ae0ba3211282c02853684fdcf880708f00b6 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:51:37 +0300 Subject: [PATCH 101/157] docs: the live-check protocol, catalogue and results log, and the rules for recordings and fixtures docs/LIVE-CHECKS.md: how to record, scrub and replay a check, what the manifest records by itself and what the operator adds, the pre-commit checklist for fixtures, the check catalogue, and the results log seeded with the 2026-09-27 runs. AGENTS.md: the recordings and fixtures rules every agent follows. live-records/ is gitignored. Co-Authored-By: Claude Opus 5.5 --- .gitignore | 3 + AGENTS.md | 23 +++++ FORK.md | 3 + docs/LIVE-CHECKS.md | 199 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 228 insertions(+) create mode 100644 docs/LIVE-CHECKS.md diff --git a/.gitignore b/.gitignore index 27f2df0d3d..1f86abc52a 100644 --- a/.gitignore +++ b/.gitignore @@ -41,6 +41,9 @@ lerna-debug.log* /temp/* +# Raw live-check recordings (docs/LIVE-CHECKS.md): never committed +/live-records/ + .DS_Store *.DS_Store .tool-versions diff --git a/AGENTS.md b/AGENTS.md index 35a4c0b47d..0750c9147f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -37,6 +37,29 @@ without saying why in the commit. socket call, the HTTP answer. Assert exact fields, not substrings. - No test touches WhatsApp, a real account or the network beyond localhost. +## Recordings and fixtures (live checks) + +A live check records a real session (`LIVE_RECORD_DIR`, `docs/LIVE-CHECKS.md`). +Raw recordings hold real people's numbers, names and messages. Any agent or +person working here follows these rules, with no exception: + +- Never commit, stage or copy anything from `LIVE_RECORD_DIR` (or + `live-records/`). Only the scrubber's output (`scripts/live-scrub.ts`) goes + into `test/fixtures/live/`. +- Before committing a fixture, run `npx tsx scripts/live-guard.ts` and read + `scrub-report.json` (`"leakGate": "pass"`, counts that fit the check). +- Open and skim every new fixture file yourself. The guard cannot recognise a + name or a message text; you can. +- If anything looks like a phone number, a user part of an `@lid` or + `@s.whatsapp.net` address that is not a scrubber fake, a name, a message + text, a signed media URL (`mmg.whatsapp.net`, `oh=` / `oe=` parameters), an + email, a token or a key: stop. Fix the scrubber and scrub again. Never edit + a fixture by hand. +- Never paste a raw recording, or any part of one, into a commit, an issue, a + pull request or a chat. +- A fixture's replay test must fail when the behaviour it covers is broken: + run it once against the code before the fix and say so in the commit. + --- # Evolution API - AI Agent Guidelines diff --git a/FORK.md b/FORK.md index 9e3568bebd..7a2619b92d 100644 --- a/FORK.md +++ b/FORK.md @@ -68,6 +68,9 @@ named where one exists. - Profile pictures are looked up once per contact per hour, at most four at a time, and history never waits on them (#1883). - One pairing code per connect attempt, and a fresh QR budget per attempt (#2100, #2696). +**Live checks** +- A live check against a real phone can be recorded (`LIVE_RECORD_DIR`), scrubbed into a fixture behind a fail-closed leak gate, and replayed through the real event buffer and service in a test. The protocol, the check catalogue and the results log are in `docs/LIVE-CHECKS.md`. + ## Licence Evolution API is licensed under the Apache License 2.0 with additional diff --git a/docs/LIVE-CHECKS.md b/docs/LIVE-CHECKS.md new file mode 100644 index 0000000000..8cf805b120 --- /dev/null +++ b/docs/LIVE-CHECKS.md @@ -0,0 +1,199 @@ +# Live checks + +The unit and harness tests run Evolution's source against the real Baileys, but +the input is ours. A live check runs the fork against a real phone and records +what WhatsApp actually sent, so the behaviour can be replayed in a test from then +on. This file is the protocol, the catalogue of checks, and the log of results. + +## The chain + +1. **Record.** The fork records a session when `LIVE_RECORD_DIR` is set + (`src/utils/live-record/recorder.ts`), and does nothing otherwise. Per + instance and session start it writes: + - `events.ndjson`: every Baileys event (name, payload, a sequence number, + the socket it came from, whether the event buffer held it) and every batch + the buffer delivered. This is the input tape. + - `webhooks.ndjson`: every payload Evolution sent (`sendDataWebhook`). This is + the golden output tape. + - `manifest.json`: versions and conditions (below). No number, JID, name or + content. + - `owner.json`: the linked account, for the scrubber only. It is never copied + into a fixture. + + Values are written in a tagged codec (`src/utils/live-record/codec.ts`) so a + replay rebuilds identical ones: `$bytes` (Buffer or Uint8Array), `$long`, + `$u` (undefined), `$proto` (the protobuf class), `$err`, `$date`. The auth + creds and the QR payload are redacted when recorded. +2. **Scrub.** `scripts/live-scrub.ts` turns a raw session into + `test/fixtures/live/-/`, then runs a leak gate that + searches the output for every original and writes nothing if one survives. +3. **Replay.** `test/helpers/live-replay.ts` feeds the fixture's events through + Baileys' real event buffer (buffered where they were) into the real + `BaileysStartupService`, and `compareGolden` compares what Evolution sends + with `webhooks.ndjson`. + +### What the manifest records + +| Field | Source | +| --- | --- | +| `forkCommit` | `/evolution/FORK_SHA` (the working directory's `FORK_SHA` file), else `git describe`, else the `FORK_SHA` variable | +| `baileysVersion` | the installed `baileys/package.json` | +| `nodeVersion` | the running Node | +| `waWebVersion` | the version the socket was built with | +| `phonePlatform` | `creds.platform`, which WhatsApp reports at pairing (`smba`, `smbi`, `android`, `iphone`...). Known after the connection opens; kept in the creds, so also known after a restart | +| `accountType` | derived from the platform: `smb*` is WhatsApp Business. Null when the platform is unknown | +| `linkMethod` | `qr`, `code` (a pairing code was asked for) or `existing-session` (the creds were already paired) | +| `proxy` | used or not, and the protocol. Never the host or the credentials | +| `sockets`, `startedAt`, `openedAt`, `endedAt` | the session's own bookkeeping | + +The operator adds by hand, when scrubbing: **phone model, OS version, WhatsApp +app version, country code** (the code only, never a number). Nothing on the +socket reveals them. Record them every time: a result without them cannot be +compared with the next one. + +## Protocol + +**Who and what phone.** A maintainer, on a test account on a phone kept for it, +linked to nothing else that matters. Use a real account only when the check needs +a real history, and then only one whose contacts know it is used for testing. +Write down the phone model, the OS version and the WhatsApp app version before +you start. + +**Run the rig with recording on.** Build the fork image and keep the raw +recordings outside the repository (or in `live-records/`, which is gitignored): + +```bash +docker build -t evolution-fork:$(git rev-parse --short=8 HEAD) . +mkdir -p ~/live-records && chmod 700 ~/live-records +docker run --rm -p 127.0.0.1:8080:8080 --env-file .env \ + -e FORK_SHA=$(git rev-parse --short=8 HEAD) \ + -e LIVE_RECORD_DIR=/evolution/live-records \ + -v ~/live-records:/evolution/live-records \ + evolution-fork:$(git rev-parse --short=8 HEAD) +``` + +Or from the source, which reads the commit from git: + +```bash +LIVE_RECORD_DIR=~/live-records npm run dev:server +``` + +Each process start is a new session directory, +`~/live-records///`. Run one check per session where you can: +restart the container between checks. + +**Scrub.** + +```bash +npx tsx scripts/live-scrub.ts ~/live-records// \ + --phone-model "Pixel 8" --os-version "Android 15" --wa-version 2.25.27.78 --country-code 972 +npx tsx scripts/live-guard.ts +``` + +The scrubber prints the fixture directory and a report of counts. It exits 1 +and writes nothing when its leak gate finds an original in the output: fix the +scrubber (a new field it does not know, most often), never the fixture. Delete +the raw session once the fixture is committed. + +What the scrubber does: one person keeps one fake index across their phone JID, +@lid and device suffix (the owner is index 0, `972500000000`); names keep +equality (a saved name and a profile name stay different, the same name stays +the same); message ids keep their first two characters and length (Evolution +reads the device from them); bytes keep their length and type, with random +content, so a replay test must never depend on real crypto; text becomes lorem +of the same length; URLs become `https://example.invalid/`. + +**Add the fixture and its replay test.** Put the test next to the behaviour it +covers, and assert the exact thing the check is about, then the whole output: + +```ts +import { compareGolden, loadFixture, replayFixture } from '../helpers/live-replay'; + +const FIXTURE = 'test/fixtures/live/2026-10-01-archive-toggle'; + +it('archiving on the phone reaches chats.update with archived: true', async () => { + const { webhooks } = await replayFixture(FIXTURE); + const updates = webhooks.filter((w) => w.event === 'chats.update').flatMap((w) => w.data); + expect(updates).toContainEqual(expect.objectContaining({ remoteJid: '972500000001@s.whatsapp.net', archived: true })); + expect(compareGolden(webhooks, loadFixture(FIXTURE).webhooks, { events: ['chats.update'] })).toEqual([]); +}); +``` + +Socket queries (profile pictures, group metadata, LID lookups) were not +recorded: they answer as the harness does, and `replayFixture(dir, { client })` +takes answers a test needs. Fields Evolution fills from the clock, the database +or those queries are left out of the comparison (`VOLATILE` in +`live-replay.ts`). + +**The rule: a fixture test must fail when the behaviour is broken.** A fixture +recorded on fixed code carries the fix in its golden output. Prove the test sees +it once: run it against the code before the fix (check out the parent of the +fix, or revert it locally) and see it fail for the reason the check is about. +Say so in the commit. A fixture test that passes either way proves nothing. + +## Before you commit a recording (checklist) + +Nothing personal may reach the repository. Before any commit or push that +touches `test/fixtures/live/`: + +- [ ] Nothing from `LIVE_RECORD_DIR` (or `live-records/`) is staged. Only the + scrubber's output is committed; `owner.json` never is. +- [ ] `npx tsx scripts/live-guard.ts` says clean, and `scrub-report.json` says + `"leakGate": "pass"` with counts that make sense for the check. +- [ ] You opened every new fixture file and skimmed it yourself. +- [ ] Nothing in it looks like a phone number, the user part of an + `@lid` / `@s.whatsapp.net` address that is not a fake (`972500......`, + `100000000......`, `120363............`), a name, a message text, a signed + media URL (`mmg.whatsapp.net`, `oh=` / `oe=` parameters), an email, a + token or a key. +- [ ] If anything does: stop, fix the scrubber, and scrub again. Never edit a + fixture by hand. +- [ ] No raw recording is pasted into a commit message, an issue, a pull + request or a chat. + +`test/live/fixture-guard.test.ts` runs the same guard over +`test/fixtures/live/` in CI on every commit, and lint-staged runs it on staged +fixture files. The guard knows only what personal data looks like: it cannot +recognise a name or a text, which is why the skim is not optional. + +## Check catalogue + +Each check says what it proves and the steps. "Fixture" means the session is +worth scrubbing into a replay test. + +| Check id | Proves | Steps | Fixture | +| --- | --- | --- | --- | +| `stock-to-fork-switch` | a session linked on the stock image keeps working on the fork | link on the stock 2.3.7 image; stop it; start the fork on the same volume and database; send and receive one message | no | +| `app-state-after-restart` | saved names, labels, mutes and archives keep syncing after a restart | link; wait for history; restart the container; on the phone rename a contact and archive a chat; watch `contacts.upsert` and `chats.update` | yes | +| `saved-vs-profile-names` | `saved` is true only for the name the owner saved | save contact A under a name that differs from A's profile name; leave B unsaved; both send a message | yes | +| `archive-toggle` | archive and unarchive on the phone reach `chats.update` with `archived` | archive a chat on the phone, wait, unarchive it | yes | +| `group-rename-participants` | group updates and participant changes reach their webhooks | create a group with two test numbers; rename it; add, promote, demote and remove a participant | yes | +| `live-lid-message-key` | a DM WhatsApp addresses by @lid shows the phone as `remoteJid` and keeps the @lid in `remoteJidAlt` | from a number that is not a saved contact, send a message to the account | yes | +| `history-lid-keys` | history arrives with the original @lid keys | link a fresh device on an account with @lid chats; wait for history to finish | yes (history is large: scrub a short account) | +| `pairing-code-over-45s` | one pairing code per connect attempt, still valid after the 45s QR window | connect with a number; wait more than 45s before typing the code on the phone | no | +| `logout-reaches-phone` | a logout takes the device off the phone's Linked devices | log out through the API; check Linked devices on the phone | no | +| `logout-while-offline` | a logout with the socket down is delivered when it reconnects | cut the container's network; log out through the API (expect `202 PENDING`); restore the network; check Linked devices | no | +| `reconnect-backoff-one-socket` | reconnects back off 1s to 60s, never give up, and one socket at a time | cut the container's network for 3 minutes; read the reconnect log lines; restore; count sockets in the manifest and the log | yes (the close and reopen) | +| `proxy-per-number` | each instance leaves through its own proxy (socket, version fetch, media) | two instances behind two proxies; check each proxy's log for its instance's traffic and none of the other's | no | +| `clean-logs` | no message text, number, JID or push name in logs at `LOG_LEVEL=ERROR,WARN` | run a session with restarts, reconnects and group listings; count digit runs of 8+, addresses and media URLs in the log | no | +| `bounded-queries` | profile pictures and group metadata are queried within their bounds | on an account in several groups, link and list groups; count the queries per contact and group | yes | +| `media-reupload-expired` | a download asks the phone to re-upload an expired file, and a 403 counts as expired only when the link has expired | download media 30 to 180 days old through `getBase64FromMediaMessage`; compare SHA-256 with the phone's copy | no | + +## Results log + +Outcomes are counts. "Not recorded" means the operator did not write it down; +it is not a guess. + +| Date | Check id | Fork commit | Baileys | WA Web | Phone (platform / model / app) | Outcome | +| --- | --- | --- | --- | --- | --- | --- | +| 2026-09-27 | `stock-to-fork-switch` | not recorded | 7.0.0-rc14 | not recorded | not recorded / not recorded / not recorded | the session survived the switch from the stock image to the fork | +| 2026-09-27 | `logout-reaches-phone` | not recorded | 7.0.0-rc14 | not recorded | not recorded / not recorded / not recorded | 2 of 2 logouts reached WhatsApp; the device left Linked devices both times | +| 2026-09-27 | `history-lid-keys` | not recorded | 7.0.0-rc14 | not recorded | not recorded / not recorded / not recorded | history arrived with the original @lid keys | +| 2026-09-27 | `media-reupload-expired` (403 rule) | bb269777 | 7.0.0-rc14 | not recorded | not recorded / not recorded / not recorded | 84-file sample: a 403 answered 34 of 34 expired links and 0 of 50 valid ones | +| 2026-09-27 | `media-reupload-expired` | 3fc63a62 | 7.0.0-rc14 | not recorded | not recorded / not recorded / not recorded | 7 of 8 expired files recovered (30 to 180 days old), SHA-256 verified; 1 refused `NOT_FOUND` | +| 2026-09-27 | `clean-logs` | 3fc63a62 | 7.0.0-rc14 | not recorded | business, platform not recorded / not recorded / not recorded; linked earlier by QR; no proxy | at `LOG_LEVEL=ERROR,WARN`, `LOG_BAILEYS=error`: 48 lines across a restart, 2 sessions reconnecting and 2 group listings held 0 digit runs of 8+, 0 WhatsApp addresses, 0 media URLs. 1 finding: an earlier ERROR-level media download failure printed the signed media URL (fixed separately). At full verbosity (INFO to WEBHOOKS, Baileys debug) addresses appear by design | +| 2026-09-27 | `reconnect-backoff-one-socket` | 3fc63a62 | 7.0.0-rc14 | not recorded | business, platform not recorded / not recorded / not recorded; linked earlier by QR; no proxy | network cut for 180s: attempts 1, 2, 4, 8, 16, 32, 60s apart (status 408), capped at 60s, never gave up; reopened on the first attempt after the network returned. 1 finding: the failure line logged `[object Object]` (fixed separately). Sockets not counted | +| 2026-09-27 | `bounded-queries` | 3fc63a62 | 7.0.0-rc14 | not recorded | business, platform not recorded / not recorded / not recorded | not measurable: the account is in no groups. Moved to a later session | + +The phone platform and model were not recorded for any of these runs: the +recorder did not exist yet, and the operator did not write them down. From 96ab1aa3186caa633596e7586cea495faf499e3c Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 18:23:06 +0300 Subject: [PATCH 102/157] test: the scrubber keeps a numeric message id a message id, never a person WhatsApp gives group notifications numeric message ids. The scrubber read them as bare phone numbers, so each became a fake phone and counted as a person in the report. Red against the scrubber as it is: the id comes out as a 12-digit fake phone. Co-Authored-By: Claude Opus 5.5 --- test/live/scrub.test.ts | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/test/live/scrub.test.ts b/test/live/scrub.test.ts index f48fb7fd52..ee450a0da2 100644 --- a/test/live/scrub.test.ts +++ b/test/live/scrub.test.ts @@ -149,6 +149,30 @@ describe('live-check scrubber', () => { expect(report.people).toBe(2); }); + it('keeps a numeric message id a message id, never a person', () => { + // WhatsApp gives group notifications (a create, a rename, an add) numeric ids. + const id = '8347261905'; + const stub = { + seq: 999, + t: 1, + socket: 1, + event: 'messages.upsert', + buffered: false, + data: { type: 'append', messages: [{ key: { remoteJid: '120363401234567890@g.us', fromMe: false, id }, messageStubType: 20 }] }, + }; + appendFileSync(join(raw, 'events.ndjson'), JSON.stringify(stub) + '\n'); + const { dir, report } = scrub(); + + const line = lines(join(dir, 'events.ndjson')).find((e) => e.seq === 999); + const fake = line.data.messages[0].key.id; + expect(fake).toMatch(/^\d+$/); + expect(fake).toHaveLength(id.length); + expect(fake.slice(0, 2)).toBe(id.slice(0, 2)); + expect(fake).not.toBe(id); + expect(report.people).toBe(2); + expect(report.messageIds).toBeGreaterThan(0); + }); + it('takes a country code only, never a number', () => { expect(() => scrub({ operator: { countryCode: '972541112233' } })).toThrow(/country code/); expect(existsSync(join(out, '2026-09-27-synthetic-session'))).toBe(false); From bf53c7c4bc043ca9a885ceb5d0572e1103414adb Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 18:23:24 +0300 Subject: [PATCH 103/157] fix(live-record): the scrubber fakes a numeric message id as an id A numeric string under an id key that is not already a known person's number now keeps its length and first two digits, the rest a counter, and counts as a message id rather than a person. Co-Authored-By: Claude Opus 5.5 --- test/tools/live-scrub.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/test/tools/live-scrub.ts b/test/tools/live-scrub.ts index 298e9874e9..3b2eb424af 100644 --- a/test/tools/live-scrub.ts +++ b/test/tools/live-scrub.ts @@ -13,7 +13,7 @@ // phone JID / @lid / group 972500<6> / 100000000<6> / 120363<12>, device suffix kept; index 0 is the owner // a bare number of 7-15 digits the same person's fake digits (epoch timestamps are kept) // a name (name, notify, pushName, subject...) "Name ", one per distinct original ("Você" kept) -// a message id same first two characters and length, the rest a counter +// a message id same first two characters and length, the rest a counter (digits stay digits) // bytes ($bytes) random bytes of the same length and type, tagged fake // a URL https://example.invalid/ // structure (event names, enums, mimetypes, dates, 3 characters or fewer) kept @@ -170,6 +170,8 @@ class Scrubber { if (s === this.instanceName && (key === 'instance' || key === 'instanceName')) return REPLAY_INSTANCE; const jid = JID.exec(s); if (jid) return this.fakeJid(jid[1], jid[2], jid[3]); + // A numeric message id (group notifications have them) is an id, not a person. + if (ID_KEYS.has(key) && /^\d{7,}$/.test(s) && !this.digits.has(s)) return this.fakeId(s); if (/^\+?\d{7,15}$/.test(s)) { const plus = s.startsWith('+') ? '+' : ''; const digits = s.slice(plus.length); @@ -214,6 +216,7 @@ class Scrubber { private fakeId(s: string) { return this.memo(this.ids, s, (n) => { + if (/^\d+$/.test(s)) return s.slice(0, 2) + String(n).padStart(s.length - 2, '9').slice(-(s.length - 2)); const body = n .toString(16) .toUpperCase() From d4d1e2949f98abfab7e18c994f6ee08fb1d313d0 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 18:24:50 +0300 Subject: [PATCH 104/157] test: the fixture guard passes identifiers and the scrubber's numeric message ids A real scrubbed session tripped the guard in two places that are not personal data: long camelCase identifiers (creds key names, proto field names) read as base64 blobs, and a group notification's numeric message id, faked by the scrubber, read as a phone-like digit run. Red against the guard as it is. A letters-only blob that is not identifier-shaped, and a phone number or an arbitrary number under an id, are still flagged. Co-Authored-By: Claude Opus 5.5 --- test/live/fixture-guard.test.ts | 72 ++++++++++++++++++++++++++++++++- 1 file changed, 70 insertions(+), 2 deletions(-) diff --git a/test/live/fixture-guard.test.ts b/test/live/fixture-guard.test.ts index 49f44c03b5..df7b5d2d41 100644 --- a/test/live/fixture-guard.test.ts +++ b/test/live/fixture-guard.test.ts @@ -17,7 +17,7 @@ vi.mock('@utils/fetchLatestWaWebVersion', () => ({ fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), })); -import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { appendFileSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; @@ -56,7 +56,11 @@ describe('live fixture guard', () => { { seq: 5, event: 'contacts.update', data: [{ about: `write to ${PLANTED.email}` }] }, { seq: 6, event: 'messages.upsert', data: { mediaKey: { $bytes: PLANTED.bytes, as: 'Uint8Array' } } }, { seq: 7, event: 'messages.upsert', data: { jpegThumbnail: PLANTED.bytes } }, - { seq: 8, event: 'presence.update', data: { presences: { [PLANTED.pnJid]: { lastKnownPresence: 'available' } } } }, + { + seq: 8, + event: 'presence.update', + data: { presences: { [PLANTED.pnJid]: { lastKnownPresence: 'available' } } }, + }, ]; writeFileSync(join(dir, 'events.ndjson'), lines.map((l) => JSON.stringify(l)).join('\n') + '\n'); writeFileSync(join(dir, 'manifest.json'), JSON.stringify({ note: `owner ${PLANTED.phone}` })); @@ -84,8 +88,72 @@ describe('live fixture guard', () => { } }); + it('reads a long camelCase identifier as a name, not a base64 blob', () => { + const dir = join(root, 'live', '2026-09-27-identifiers'); + mkdirSync(dir, { recursive: true }); + const lines = [ + // Baileys' own key names: creds keys, proto fields. + { seq: 1, event: 'connection.update', data: { receivedPendingNotifications: true } }, + { + seq: 2, + event: 'creds.update', + data: { $redacted: 'creds', keys: ['processedHistoryMessages', 'lastAccountSyncTimestamp'] }, + }, + { seq: 3, event: 'messages.upsert', data: { message: { axolotlSenderKeyDistributionMessage: {} } } }, + // A blob of letters only, not identifier-shaped, is still a blob. + { + seq: 4, + event: 'messages.upsert', + data: { thumb: 'QmFzZVNpeHRyRmxvYkxldHRlcnNPbmxWWFpBQkNE' }, + }, + ]; + writeFileSync(join(dir, 'events.ndjson'), lines.map((l) => JSON.stringify(l)).join('\n') + '\n'); + const findings = scanFixtures(join(root, 'live')); + expect(findings.filter((f) => f.line <= 3)).toEqual([]); + expect(findings).toContainEqual(expect.objectContaining({ line: 4, kind: 'base64 blob' })); + }); + + it("accepts the scrubber's fake numeric message id, and nothing else numeric under an id", () => { + const dir = join(root, 'live', '2026-09-27-numeric-ids'); + mkdirSync(dir, { recursive: true }); + const stub = (seq: number, id: string) => ({ + seq, + event: 'messages.upsert', + data: { messages: [{ key: { remoteJid: '120363000000000001@g.us', id }, messageStubType: 20 }] }, + }); + const lines = [stub(1, '740000002'), stub(2, '4100000013'), stub(3, PLANTED.phone), stub(4, '834726190')]; + writeFileSync(join(dir, 'events.ndjson'), lines.map((l) => JSON.stringify(l)).join('\n') + '\n'); + const flagged = scanFixtures(join(root, 'live')).map((f) => `${f.line} ${f.kind}`); + expect(flagged).toEqual(['3 phone-like digit run', '4 phone-like digit run']); + }); + it('passes the scrubber output of a recorded session', async () => { const raw = await recordSession(join(root, 'raw')); + // Group notifications carry numeric message ids; the creds keys are long identifiers. + const extra = [ + { + seq: 998, + t: 1, + socket: 1, + event: 'creds.update', + buffered: false, + data: { $redacted: 'creds', keys: ['processedHistoryMessages'] }, + }, + { + seq: 999, + t: 1, + socket: 1, + event: 'messages.upsert', + buffered: false, + data: { + type: 'append', + messages: [ + { key: { remoteJid: '120363401234567890@g.us', fromMe: false, id: '834726190' }, messageStubType: 20 }, + ], + }, + }, + ]; + appendFileSync(join(raw, 'events.ndjson'), extra.map((l) => JSON.stringify(l)).join('\n') + '\n'); scrubSession(raw, { checkId: 'synthetic-session', date: '2026-09-27', outRoot: join(root, 'live') }); expect(formatFindings(scanFixtures(join(root, 'live')))).toBe(''); }); From ec02f6f4eae4159105df133118960ffe3977c281 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 18:25:31 +0300 Subject: [PATCH 105/157] fix(live-record): the guard passes identifiers and the scrubber's numeric message id A camelCase identifier is not a base64 blob, and under a message-id key the scrubber's numeric id fake (two digits kept, zeros, a short counter) is not a phone. The scrubber now builds that shape for numeric ids. A phone number or any other number under an id is still flagged. Co-Authored-By: Claude Opus 5.5 --- test/tools/fixture-guard.ts | 15 ++++++++++++--- test/tools/live-scrub.ts | 4 ++-- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/test/tools/fixture-guard.ts b/test/tools/fixture-guard.ts index c49c298e86..0071d3053f 100644 --- a/test/tools/fixture-guard.ts +++ b/test/tools/fixture-guard.ts @@ -2,11 +2,13 @@ // the scrubber that made them: it knows only the scrubber's fake ranges. // // an address (JID) whose user part is not a fake: 972500<6>, 100000000<6>, 120363<12> -// a run of 8+ digits that is neither a fake nor an epoch timestamp (s or ms) +// a run of 8+ digits that is neither a fake nor an epoch timestamp (s or ms); under a +// message-id key, the scrubber's numeric id fake (two digits, zeros, a short counter) is allowed // a signed media URL (mmg/pps/media*.whatsapp.net, oh= / oe= parameters) // an email address // bytes ({"$bytes"}) of 16+ bytes not tagged fake by the scrubber, and any other -// base64 blob of 24+ characters that is not one of those fake bytes +// base64 blob of 24+ characters that is not one of those fake bytes (a camelCase +// identifier is not a blob) // // A finding names the file, the line, a masked JSON path and the kind of value, // never the value itself. Run on every commit by test/live/fixture-guard.test.ts, @@ -22,6 +24,11 @@ const WA_DOMAIN = /^(s\.whatsapp\.net|c\.us|g\.us|lid|hosted|hosted\.lid|broadca const EMAIL = /[A-Za-z0-9._%+-]+@((?:[A-Za-z0-9-]+\.)+[A-Za-z]{2,})/g; const SIGNED_URL = /(mmg|pps|media[\w.-]*)\.whatsapp\.net|[?&](oh|oe)=/i; const BASE64 = /^[A-Za-z0-9+/]{24,}={0,2}$/; +/** A camelCase name (receivedPendingNotifications), which BASE64 alone would match. */ +const IDENTIFIER = /^[a-z]+(?:[A-Z][a-z]+)+$/; +/** The scrubber's fake numeric message id: two digits kept, zeros, a short counter. */ +const FAKE_NUMERIC_ID = /^\d{2}0{3,}[1-9]\d{0,3}$/; +const ID_KEYS = new Set(['id', 'stanzaId', 'keyId', 'messageId']); /** Numbers under these keys are sizes, counts and times, not people. */ const NUMERIC_KEY = /(length|size|seconds|duration|count|progress|timestamp|time|^t$|^seq$|at$|height|width|expiration|ttl)/i; @@ -42,13 +49,15 @@ function scanString(s: string, report: (kind: string) => void, fakeBytes: Set void, fakeBytes: Set) { const at = (kind: string) => report(kind, path); // A commit hash can hold eight digits in a row. if (typeof value === 'string' && path[path.length - 1] === 'forkCommit' && /^[0-9a-f]{7,40}(-dirty)?$/.test(value)) return; + if (typeof value === 'string' && ID_KEYS.has(path[path.length - 1]) && FAKE_NUMERIC_ID.test(value)) return; if (typeof value === 'string') return scanString(value, at, fakeBytes); if (typeof value === 'number') { const key = path[path.length - 1] ?? ''; diff --git a/test/tools/live-scrub.ts b/test/tools/live-scrub.ts index 3b2eb424af..31c0fd6a5a 100644 --- a/test/tools/live-scrub.ts +++ b/test/tools/live-scrub.ts @@ -13,7 +13,7 @@ // phone JID / @lid / group 972500<6> / 100000000<6> / 120363<12>, device suffix kept; index 0 is the owner // a bare number of 7-15 digits the same person's fake digits (epoch timestamps are kept) // a name (name, notify, pushName, subject...) "Name ", one per distinct original ("Você" kept) -// a message id same first two characters and length, the rest a counter (digits stay digits) +// a message id same first two characters and length, the rest a counter (digits: zeros, then it) // bytes ($bytes) random bytes of the same length and type, tagged fake // a URL https://example.invalid/ // structure (event names, enums, mimetypes, dates, 3 characters or fewer) kept @@ -216,7 +216,7 @@ class Scrubber { private fakeId(s: string) { return this.memo(this.ids, s, (n) => { - if (/^\d+$/.test(s)) return s.slice(0, 2) + String(n).padStart(s.length - 2, '9').slice(-(s.length - 2)); + if (/^\d+$/.test(s)) return s.slice(0, 2) + String(n).padStart(s.length - 2, '0').slice(2 - s.length); const body = n .toString(16) .toUpperCase() From 356384e8ed6a830ada1ba3919ef202d236febc73 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 18:27:37 +0300 Subject: [PATCH 106/157] test(live): a contact renamed on the phone after a restart, replayed from a live check The rig-session fixture is the scrubber's output of one live session (2026-09-27, fork 71b5ae0b, Baileys 7.0.0-rc14, WhatsApp Business on an iPhone, platform smbi), guard clean. It covers four checks in order: an app-state rename after a restart, an archive toggle, a group rename with a participant removed and added back, and a text and an image received in a DM addressed by @lid. This test pins live-verified behaviour that already exists, so it goes in with its fixture: the renamed contact reaches contacts.upsert as a mapping item (lid and phoneNumber) and as the saved name with saved: true, then contacts.update, matching the recorded webhooks. Teeth, checked locally and restored: - f7b56062 (saved names) reverted: fails, no item carries saved. - the @lid mapping forward (71ba3e8f) stubbed out, since the revert no longer applies cleanly: fails, no mapping item. Co-Authored-By: Claude Opus 5.5 --- .../live/2026-09-27-rig-session/events.ndjson | 54 +++++++++++++++++++ .../live/2026-09-27-rig-session/manifest.json | 31 +++++++++++ .../2026-09-27-rig-session/scrub-report.json | 12 +++++ .../2026-09-27-rig-session/webhooks.ndjson | 23 ++++++++ test/live/app-state-rename.test.ts | 39 ++++++++++++++ 5 files changed, 159 insertions(+) create mode 100644 test/fixtures/live/2026-09-27-rig-session/events.ndjson create mode 100644 test/fixtures/live/2026-09-27-rig-session/manifest.json create mode 100644 test/fixtures/live/2026-09-27-rig-session/scrub-report.json create mode 100644 test/fixtures/live/2026-09-27-rig-session/webhooks.ndjson create mode 100644 test/live/app-state-rename.test.ts diff --git a/test/fixtures/live/2026-09-27-rig-session/events.ndjson b/test/fixtures/live/2026-09-27-rig-session/events.ndjson new file mode 100644 index 0000000000..9f63a77e34 --- /dev/null +++ b/test/fixtures/live/2026-09-27-rig-session/events.ndjson @@ -0,0 +1,54 @@ +{"seq":1,"t":37.7,"socket":1,"event":"connection.update","buffered":true,"data":{"connection":"connecting","receivedPendingNotifications":false,"qr":{"$u":1}}} +{"seq":2,"t":43.5,"socket":1,"batch":["connection.update"]} +{"seq":4,"t":1181.6,"socket":1,"event":"creds.update","buffered":true,"data":{"$redacted":"creds","keys":["noiseKey","pairingEphemeralKeyPair","signedIdentityKey","signedPreKey","registrationId","advSecretKey","processedHistoryMessages","nextPreKeyId","firstUnuploadedPreKeyId","accountSyncCounter","accountSettings","registered","account","me","signalIdentities","platform","routingInfo","lastAccountSyncTimestamp","myAppStateKeyId","lastPropHash"]}} +{"seq":5,"t":1184.8,"socket":1,"batch":["creds.update"]} +{"seq":6,"t":1726.8,"socket":1,"event":"connection.update","buffered":false,"data":{"receivedPendingNotifications":true}} +{"seq":7,"t":1728.5,"socket":1,"batch":["connection.update"]} +{"seq":8,"t":1978.5,"socket":1,"event":"creds.update","buffered":false,"data":{"$redacted":"creds","keys":["me"]}} +{"seq":9,"t":1979.3,"socket":1,"batch":["creds.update"]} +{"seq":10,"t":1979.5,"socket":1,"event":"connection.update","buffered":false,"data":{"connection":"open"}} +{"seq":11,"t":1985.2,"socket":1,"event":"connection.update","buffered":false,"data":{"isOnline":false}} +{"seq":12,"t":1985.7,"socket":1,"batch":["connection.update"]} +{"seq":13,"t":1990.1,"socket":1,"batch":["connection.update"]} +{"seq":15,"t":100574,"socket":1,"event":"contacts.upsert","buffered":true,"data":[{"id":"972500000001@s.whatsapp.net","name":"Name 2","username":{"$u":1},"lid":"100000000000001@lid","phoneNumber":"972500000001@s.whatsapp.net"}]} +{"seq":16,"t":100577.3,"socket":1,"event":"lid-mapping.update","buffered":true,"data":{"lid":"100000000000001@lid","pn":"972500000001@s.whatsapp.net"}} +{"seq":17,"t":100579.4,"socket":1,"batch":["lid-mapping.update"]} +{"seq":18,"t":100584.5,"socket":1,"batch":["contacts.upsert"]} +{"seq":22,"t":152322.3,"socket":1,"event":"messages.update","buffered":true,"data":[{"key":{"remoteJid":"100000000000002@lid","id":"3AFFFFFFFFFFFFFFFFF1","fromMe":false,"participant":{"$u":1}},"update":{"status":4,"messageTimestamp":1790521569}}]} +{"seq":23,"t":152323.5,"socket":1,"batch":["messages.update"]} +{"seq":25,"t":160776.5,"socket":1,"event":"chats.update","buffered":true,"data":[{"id":"100000000000002@lid","archived":true,"conditional":{"$u":1}}]} +{"seq":26,"t":160777.3,"socket":1,"event":"chats.update","buffered":true,"data":[{"id":"100000000000002@lid","pinned":null,"conditional":{"$u":1}}]} +{"seq":27,"t":160778,"socket":1,"event":"chats.lock","buffered":true,"data":{"id":"100000000000002@lid","locked":false}} +{"seq":28,"t":160778.3,"socket":1,"batch":["chats.lock"]} +{"seq":29,"t":160779.7,"socket":1,"batch":["chats.update"]} +{"seq":31,"t":168490.3,"socket":1,"event":"chats.update","buffered":true,"data":[{"id":"100000000000002@lid","archived":false,"conditional":{"$u":1}}]} +{"seq":32,"t":168491.1,"socket":1,"batch":["chats.update"]} +{"seq":34,"t":237566.9,"socket":1,"event":"chats.upsert","buffered":true,"data":[{"id":"120363000000000001@g.us","name":"Name 3","conversationTimestamp":1790521654}]} +{"seq":35,"t":237571.4,"socket":1,"event":"groups.upsert","buffered":true,"data":[{"id":"120363000000000001@g.us","notify":{"$u":1},"addressingMode":"pn","subject":"Name 3","subjectOwner":"100000000000000@lid","subjectOwnerPn":"972500000000@s.whatsapp.net","subjectOwnerUsername":{"$u":1},"subjectTime":1790521654,"size":2,"creation":1790521654,"owner":"100000000000000@lid","ownerPn":"972500000000@s.whatsapp.net","ownerUsername":{"$u":1},"owner_country_code":"IL","desc":{"$u":1},"descId":{"$u":1},"descOwner":{"$u":1},"descOwnerPn":{"$u":1},"descOwnerUsername":{"$u":1},"descTime":{"$u":1},"linkedParent":{"$u":1},"restrict":false,"announce":false,"isCommunity":false,"isCommunityAnnounce":false,"joinApprovalMode":false,"memberAddMode":true,"participants":[{"id":"100000000000000@lid","phoneNumber":"972500000000@s.whatsapp.net","lid":{"$u":1},"username":{"$u":1},"admin":"superadmin"},{"id":"100000000000002@lid","phoneNumber":"972500000002@s.whatsapp.net","lid":{"$u":1},"username":{"$u":1},"admin":null}],"ephemeralDuration":{"$u":1},"author":"100000000000000@lid","authorPn":"972500000000@s.whatsapp.net","authorUsername":{"$u":1}}]} +{"seq":36,"t":237572.1,"socket":1,"batch":["groups.upsert"]} +{"seq":37,"t":237575.5,"socket":1,"event":"messages.upsert","buffered":true,"data":{"messages":[{"$proto":"proto.WebMessageInfo","messageStubParameters":["psum "],"labels":[],"userReceipt":[],"reactions":[],"pollUpdates":[],"eventResponses":[],"statusMentions":[],"messageAddOns":[],"statusMentionSources":[],"supportAiCitations":[],"key":{"$proto":"proto.MessageKey","remoteJid":"120363000000000001@g.us","fromMe":false,"id":"740000002","participant":"100000000000000@lid"},"messageTimestamp":{"$long":"1790521655","u":true},"participant":"100000000000000@lid","messageStubType":20}],"type":"append"}} +{"seq":38,"t":237579.7,"socket":1,"batch":["chats.upsert","messages.upsert"]} +{"seq":41,"t":238782.4,"socket":1,"event":"messages.upsert","buffered":true,"data":{"messages":[{"key":{"remoteJid":"120363000000000001@g.us","remoteJidAlt":{"$u":1},"remoteJidUsername":{"$u":1},"fromMe":true,"id":"2AFFFFFFFFFFFFFFFFF3","participant":"100000000000000@lid","participantAlt":{"$u":1},"participantUsername":{"$u":1},"addressingMode":"lid"},"category":{"$u":1},"messageTimestamp":1790521656,"pushName":"Owner","broadcast":false,"status":2,"message":{"$proto":"proto.Message","senderKeyDistributionMessage":{"$proto":"proto.Message.SenderKeyDistributionMessage","groupId":"120363000000000001@g.us","axolotlSenderKeyDistributionMessage":{"$bytes":"RMVqn/2oPBvhRQQ+vwdRq4i0599AxnAj8q/Pr9jnNcXjV5tkyRCvLV57OMKvISU752SeJ4v1ClqwM5Ovanxyf9JBA4LO7usOgLnpu0Ki","as":"Uint8Array","fake":1}},"messageContextInfo":{"$proto":"proto.MessageContextInfo","threadId":[],"messageSecret":{"$bytes":"yafjJLN3NVieG57h7IIxT18A3rYpFB+bEswMJ/ttSLA=","as":"Uint8Array","fake":1}},"protocolMessage":{"$proto":"proto.Message.ProtocolMessage","type":30,"memberLabel":{"$proto":"proto.MemberLabel","label":"","labelTimestamp":{"$long":"1790521655","u":false}}}},"verifiedBizName":"Owner"}],"type":"notify"}} +{"seq":42,"t":238783.1,"socket":1,"batch":["messages.upsert"]} +{"seq":44,"t":248606.5,"socket":1,"event":"messages.upsert","buffered":true,"data":{"messages":[{"$proto":"proto.WebMessageInfo","messageStubParameters":["lor sit "],"labels":[],"userReceipt":[],"reactions":[],"pollUpdates":[],"eventResponses":[],"statusMentions":[],"messageAddOns":[],"statusMentionSources":[],"supportAiCitations":[],"key":{"$proto":"proto.MessageKey","remoteJid":"120363000000000001@g.us","fromMe":false,"id":"510000004","participant":"100000000000000@lid"},"messageTimestamp":{"$long":"1790521666","u":true},"participant":"100000000000000@lid","messageStubType":21}],"type":"append"}} +{"seq":45,"t":248608.1,"socket":1,"event":"groups.update","buffered":true,"data":[{"id":"120363000000000001@g.us","subject":"Name 4","author":"100000000000000@lid","authorPn":{"$u":1},"authorUsername":{"$u":1}}]} +{"seq":46,"t":248608.4,"socket":1,"event":"chats.update","buffered":true,"data":[{"id":"120363000000000001@g.us","name":"Name 4"}]} +{"seq":47,"t":248609.6,"socket":1,"batch":["chats.update","messages.upsert","groups.update"]} +{"seq":50,"t":255184.8,"socket":1,"event":"messages.upsert","buffered":true,"data":{"messages":[{"$proto":"proto.WebMessageInfo","messageStubParameters":[" amet consectetur adipiscing elit sed do eiusmod tempor incididunt ut labore lorem ip"],"labels":[],"userReceipt":[],"reactions":[],"pollUpdates":[],"eventResponses":[],"statusMentions":[],"messageAddOns":[],"statusMentionSources":[],"supportAiCitations":[],"key":{"$proto":"proto.MessageKey","remoteJid":"120363000000000001@g.us","fromMe":false,"id":"4100000005","participant":"100000000000000@lid"},"messageTimestamp":{"$long":"1790521672","u":true},"participant":"100000000000000@lid","messageStubType":28}],"type":"append"}} +{"seq":51,"t":255185.3,"socket":1,"event":"group-participants.update","buffered":true,"data":{"id":"120363000000000001@g.us","author":"100000000000000@lid","authorPn":{"$u":1},"authorUsername":{"$u":1},"participants":[{"id":"100000000000002@lid","phoneNumber":"972500000002@s.whatsapp.net","admin":null}],"action":"remove"}} +{"seq":52,"t":255185.5,"socket":1,"batch":["group-participants.update"]} +{"seq":53,"t":255186.5,"socket":1,"batch":["messages.upsert"]} +{"seq":55,"t":262753.8,"socket":1,"event":"messages.upsert","buffered":true,"data":{"messages":[{"$proto":"proto.WebMessageInfo","messageStubParameters":[" amet consectetur adipiscing elit sed do eiusmod tempor incididunt ut labore lorem ip"],"labels":[],"userReceipt":[],"reactions":[],"pollUpdates":[],"eventResponses":[],"statusMentions":[],"messageAddOns":[],"statusMentionSources":[],"supportAiCitations":[],"key":{"$proto":"proto.MessageKey","remoteJid":"120363000000000001@g.us","fromMe":false,"id":"840000006","participant":"100000000000000@lid"},"messageTimestamp":{"$long":"1790521680","u":true},"participant":"100000000000000@lid","messageStubType":27}],"type":"append"}} +{"seq":56,"t":262754.7,"socket":1,"event":"group-participants.update","buffered":true,"data":{"id":"120363000000000001@g.us","author":"100000000000000@lid","authorPn":{"$u":1},"authorUsername":{"$u":1},"participants":[{"id":"100000000000002@lid","phoneNumber":"972500000002@s.whatsapp.net","admin":null}],"action":"add"}} +{"seq":57,"t":262754.8,"socket":1,"batch":["group-participants.update"]} +{"seq":58,"t":262756.7,"socket":1,"batch":["messages.upsert"]} +{"seq":60,"t":601899.7,"socket":1,"event":"creds.update","buffered":false,"data":{"$redacted":"creds","keys":["noiseKey","pairingEphemeralKeyPair","signedIdentityKey","signedPreKey","registrationId","advSecretKey","processedHistoryMessages","nextPreKeyId","firstUnuploadedPreKeyId","accountSyncCounter","accountSettings","registered","account","me","signalIdentities","platform","routingInfo","lastAccountSyncTimestamp","myAppStateKeyId","lastPropHash"]}} +{"seq":61,"t":601900.7,"socket":1,"batch":["creds.update"]} +{"seq":62,"t":975850.7,"socket":1,"event":"messages.upsert","buffered":true,"data":{"messages":[{"key":{"remoteJid":"100000000000002@lid","remoteJidAlt":"972500000002@s.whatsapp.net","remoteJidUsername":{"$u":1},"fromMe":false,"id":"3AFFFFFFFFFFFFFFFFF7","participant":"","participantAlt":{"$u":1},"participantUsername":{"$u":1},"addressingMode":"lid"},"category":{"$u":1},"messageTimestamp":1790522393,"pushName":"Name 5","broadcast":false,"message":{"$proto":"proto.Message","conversation":"ons","messageContextInfo":{"$proto":"proto.MessageContextInfo","threadId":[],"deviceListMetadata":{"$proto":"proto.DeviceListMetadata","senderKeyIndexes":[],"recipientKeyIndexes":[],"senderKeyHash":{"$bytes":"I5Jh4NW30cAwCw==","as":"Uint8Array","fake":1},"senderTimestamp":{"$long":"1790519013","u":true},"recipientKeyHash":{"$bytes":"du0CJVs95FDSlw==","as":"Uint8Array","fake":1},"recipientTimestamp":{"$long":"1790338634","u":true}},"deviceListMetadataVersion":2,"messageSecret":{"$bytes":"xm/1WTK3DwFOXhTt118a9sxR6phRPmPy/gzLmVoCBjI=","as":"Uint8Array","fake":1}}}}],"type":"notify"}} +{"seq":63,"t":975852,"socket":1,"event":"contacts.update","buffered":true,"data":[{"id":"100000000000002@lid","notify":"Name 5","verifiedName":{"$u":1}}]} +{"seq":64,"t":975852.3,"socket":1,"event":"chats.update","buffered":true,"data":[{"id":"100000000000002@lid","messages":[{"message":{"key":{"remoteJid":"100000000000002@lid","remoteJidAlt":"972500000002@s.whatsapp.net","remoteJidUsername":{"$u":1},"fromMe":false,"id":"3AFFFFFFFFFFFFFFFFF7","participant":"","participantAlt":{"$u":1},"participantUsername":{"$u":1},"addressingMode":"lid"},"category":{"$u":1},"messageTimestamp":1790522393,"pushName":"Name 5","broadcast":false,"message":{"$proto":"proto.Message","conversation":"ons","messageContextInfo":{"$proto":"proto.MessageContextInfo","threadId":[],"deviceListMetadata":{"$proto":"proto.DeviceListMetadata","senderKeyIndexes":[],"recipientKeyIndexes":[],"senderKeyHash":{"$bytes":"I5Jh4NW30cAwCw==","as":"Uint8Array","fake":1},"senderTimestamp":{"$long":"1790519013","u":true},"recipientKeyHash":{"$bytes":"du0CJVs95FDSlw==","as":"Uint8Array","fake":1},"recipientTimestamp":{"$long":"1790338634","u":true}},"deviceListMetadataVersion":2,"messageSecret":{"$bytes":"xm/1WTK3DwFOXhTt118a9sxR6phRPmPy/gzLmVoCBjI=","as":"Uint8Array","fake":1}}}}}],"conversationTimestamp":1790522393,"unreadCount":1}]} +{"seq":65,"t":975852.8,"socket":1,"batch":["chats.update","messages.upsert","contacts.update"]} +{"seq":70,"t":982202.4,"socket":1,"event":"messages.upsert","buffered":true,"data":{"messages":[{"key":{"remoteJid":"100000000000002@lid","remoteJidAlt":"972500000002@s.whatsapp.net","remoteJidUsername":{"$u":1},"fromMe":false,"id":"3AFFFFFFFFFFFFFFFFF8","participant":"","participantAlt":{"$u":1},"participantUsername":{"$u":1},"addressingMode":"lid"},"category":{"$u":1},"messageTimestamp":1790522399,"pushName":"Name 5","broadcast":false,"message":{"$proto":"proto.Message","imageMessage":{"$proto":"proto.Message.ImageMessage","interactiveAnnotations":[],"scanLengths":[10736,36644,13660,16018],"annotations":[],"url":"https://example.invalid/1","mimetype":"image/jpeg","fileSha256":{"$bytes":"3jIO+TEbNIv1XkEOFHukY69vOCqotjwsGtFcDv6bT60=","as":"Uint8Array","fake":1},"fileLength":{"$long":"77060","u":true},"height":2048,"width":945,"mediaKey":{"$bytes":"v+lG1QOSwT+2PCe/80+f6XKeKEZeHwCNze0boDGgjh8=","as":"Uint8Array","fake":1},"fileEncSha256":{"$bytes":"v8DEQOf7nNkxXe74QzxyyC1wIdfkLN9IWvAvVzxlELI=","as":"Uint8Array","fake":1},"directPath":"tur adipiscing elit sed do eiusmod tempor incididunt ut labore lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod tempor incididunt ut labore lorem ipsum dolor sit amet consectetur adipiscing elit ","mediaKeyTimestamp":{"$long":"1790522397","u":false},"jpegThumbnail":{"$bytes":"/mSuHpBcYSR+7bQUQoyOURa2t4c8toD4DQpP4X586D8W3Vronhm989lmrL4fMXowBGgT9GF76rBwFYd5oHDCWlox+ZmQqUy4VQ+27NJ2D6w7KTngDveGJXE/gBT6UD1Wp9LOvMNwkPfXMC4j3EVyNild+S9STRQ5bBbMnn1tU/YOzkGmC2JhKRyH+mMpdQcFm4ziNxC+T/ojXxTIfkVLmiOVKhAVbrWxM3HSl3muxZgQt3S8IxBQyiEZ2ZdheJXy5CU4rxH4hBf3R76X+1zerw0VxqI5yMS4VmXk0X/wubJNcRnFQBmiuLk6yMK44qDl17ATNh2zXxLRFqYU30Rha8BO6dRh3HZYAvrM17uPZD+OKDHJm54cTlHk61EzA9BdiVLcSH6pG7xzga1zvQJFB4/VLLPaFk0mif9Ydk1jkWmANn3PrJ1iFymn0XO7PAenUq0pR4roZ41pUYGXYkjBVLuMkMdq5P6kNtNWN/SiFszAIEtCbOIeFm3usNLsnKj7F+HVmhOLe1rT6uTFYJF7EzgAQZ4/MjkijemeRfo6y+ZiTi8KcHekYh5uFEQkljsmhS6LGEqFiSAY73whQd1K8vGxYkPHVsdoga48yiO+emAqpCt5QrJgcMqWibLF079hxZdzi+6IDl79nfKPKik7fSG21KfBWBWEACteUbUmqnb6P0AwDXA4bgacd6oY3s8ngjEGGwvgHdvZDMdkav6/Q7G17FeCVJhrVyhOtT53bXHoncXnLVCYtKJbW5pc9TC6BwxQuOzlxTshYKpuJLuT/G3R3a8GUfREorEh3dZLJU9o6jVmZJZ5jnLEdEmxL2ZIsZX7FxaAxiZ3FQPaZQwVznx4bh8NkBMplZXp9/c/2p668joyymcR3IThB4oVhZPchjtyvNTbZ3Y57KCu3FKTLpDDnL3qdv/z3rKondwSP3ThFnqYZ05RXe2viuO3+Urhwd4WAtj8iYcgact62CL/2SI5rU4zg2KukhvRkYElpIh7g/wjFMLwL5E9/QmzZFCxuVo7CvX0fQSSn1mByDUT4Y0WFWrJl6RnbelZaaYpe3Ty3B0N7LSBhE4r8ukDa2wjrdVWJj86iE8mflilfExv20b1OjAPxhGgLeGCDQ628ZSl4JkzGfseisywrH9g2exLV01UNrDPZsB2uYv7v7Zvp0O1a2VJeOVj/zmPxKSIO81YMbQvHEGgitenrHxcPpsqhPGsLWzwMbaJmLjGgUUsFwgUuKcwN4Lr63oG+OqUStJdqrhdS02w9bVvnp6TFKTx+tV2o3G2bxATpQ7ODUD2H0tpj1jUfBN6zzPqHW0BADv/YwVPzXLL5xiQKSEK/GlgAa7SbhpyN+gUsFl3NPMmkcilw6toGvAeyOu2IiABgzwudLde499cM5vJ4Lb7RPAZfAlJusnNeJOQQ9/aCVFDGWduJjL88SQBLsdki518FgnW+EFXTcHbg6gyNF3x0fvc0I/aW790fU5Y+/V9DntHwqc/U/fxjXCirTIzG7tAGTKu/vHqEhJ6YHSe8Q9QXo4ufjvCMmJawpJtlgkRQbzi2iRzc/ivgAXsLypDhBI+DSlRp8WDhYLo1nDk78xrmzHmOt4enQToVwI=","as":"Uint8Array","fake":1},"contextInfo":{"$proto":"proto.ContextInfo","mentionedJid":[],"groupMentions":[],"statusAttributions":[],"statusSourceType":0},"firstScanSidecar":{"$bytes":"tCO2Lfa2lx3upQ==","as":"Uint8Array","fake":1},"firstScanLength":10736,"scansSidecar":{"$bytes":"VhshEorYGlY74CThpO8TeolkSD9lBL9Iz8clozPyMUXB5J5QHuTFsQ==","as":"Uint8Array","fake":1},"midQualityFileSha256":{"$bytes":"KIp21nO7212crP+1UGlo/BO+QoWgy1JcqC3uoazC5FQ=","as":"Uint8Array","fake":1},"imageSourceType":0},"messageContextInfo":{"$proto":"proto.MessageContextInfo","threadId":[],"deviceListMetadata":{"$proto":"proto.DeviceListMetadata","senderKeyIndexes":[],"recipientKeyIndexes":[],"senderKeyHash":{"$bytes":"I5Jh4NW30cAwCw==","as":"Uint8Array","fake":1},"senderTimestamp":{"$long":"1790519013","u":true},"recipientKeyHash":{"$bytes":"du0CJVs95FDSlw==","as":"Uint8Array","fake":1},"recipientTimestamp":{"$long":"1790338634","u":true}},"deviceListMetadataVersion":2,"messageSecret":{"$bytes":"3ekIumLVue4GSjI+lzREKduDdSlTVHodQbLqttE4Zzw=","as":"Uint8Array","fake":1}}}}],"type":"notify"}} +{"seq":71,"t":982203.6,"socket":1,"event":"contacts.update","buffered":true,"data":[{"id":"100000000000002@lid","notify":"Name 5","verifiedName":{"$u":1}}]} +{"seq":72,"t":982203.9,"socket":1,"event":"chats.update","buffered":true,"data":[{"id":"100000000000002@lid","messages":[{"message":{"key":{"remoteJid":"100000000000002@lid","remoteJidAlt":"972500000002@s.whatsapp.net","remoteJidUsername":{"$u":1},"fromMe":false,"id":"3AFFFFFFFFFFFFFFFFF8","participant":"","participantAlt":{"$u":1},"participantUsername":{"$u":1},"addressingMode":"lid"},"category":{"$u":1},"messageTimestamp":1790522399,"pushName":"Name 5","broadcast":false,"message":{"$proto":"proto.Message","imageMessage":{"$proto":"proto.Message.ImageMessage","interactiveAnnotations":[],"scanLengths":[10736,36644,13660,16018],"annotations":[],"url":"https://example.invalid/1","mimetype":"image/jpeg","fileSha256":{"$bytes":"3jIO+TEbNIv1XkEOFHukY69vOCqotjwsGtFcDv6bT60=","as":"Uint8Array","fake":1},"fileLength":{"$long":"77060","u":true},"height":2048,"width":945,"mediaKey":{"$bytes":"v+lG1QOSwT+2PCe/80+f6XKeKEZeHwCNze0boDGgjh8=","as":"Uint8Array","fake":1},"fileEncSha256":{"$bytes":"v8DEQOf7nNkxXe74QzxyyC1wIdfkLN9IWvAvVzxlELI=","as":"Uint8Array","fake":1},"directPath":"tur adipiscing elit sed do eiusmod tempor incididunt ut labore lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod tempor incididunt ut labore lorem ipsum dolor sit amet consectetur adipiscing elit ","mediaKeyTimestamp":{"$long":"1790522397","u":false},"jpegThumbnail":{"$bytes":"/mSuHpBcYSR+7bQUQoyOURa2t4c8toD4DQpP4X586D8W3Vronhm989lmrL4fMXowBGgT9GF76rBwFYd5oHDCWlox+ZmQqUy4VQ+27NJ2D6w7KTngDveGJXE/gBT6UD1Wp9LOvMNwkPfXMC4j3EVyNild+S9STRQ5bBbMnn1tU/YOzkGmC2JhKRyH+mMpdQcFm4ziNxC+T/ojXxTIfkVLmiOVKhAVbrWxM3HSl3muxZgQt3S8IxBQyiEZ2ZdheJXy5CU4rxH4hBf3R76X+1zerw0VxqI5yMS4VmXk0X/wubJNcRnFQBmiuLk6yMK44qDl17ATNh2zXxLRFqYU30Rha8BO6dRh3HZYAvrM17uPZD+OKDHJm54cTlHk61EzA9BdiVLcSH6pG7xzga1zvQJFB4/VLLPaFk0mif9Ydk1jkWmANn3PrJ1iFymn0XO7PAenUq0pR4roZ41pUYGXYkjBVLuMkMdq5P6kNtNWN/SiFszAIEtCbOIeFm3usNLsnKj7F+HVmhOLe1rT6uTFYJF7EzgAQZ4/MjkijemeRfo6y+ZiTi8KcHekYh5uFEQkljsmhS6LGEqFiSAY73whQd1K8vGxYkPHVsdoga48yiO+emAqpCt5QrJgcMqWibLF079hxZdzi+6IDl79nfKPKik7fSG21KfBWBWEACteUbUmqnb6P0AwDXA4bgacd6oY3s8ngjEGGwvgHdvZDMdkav6/Q7G17FeCVJhrVyhOtT53bXHoncXnLVCYtKJbW5pc9TC6BwxQuOzlxTshYKpuJLuT/G3R3a8GUfREorEh3dZLJU9o6jVmZJZ5jnLEdEmxL2ZIsZX7FxaAxiZ3FQPaZQwVznx4bh8NkBMplZXp9/c/2p668joyymcR3IThB4oVhZPchjtyvNTbZ3Y57KCu3FKTLpDDnL3qdv/z3rKondwSP3ThFnqYZ05RXe2viuO3+Urhwd4WAtj8iYcgact62CL/2SI5rU4zg2KukhvRkYElpIh7g/wjFMLwL5E9/QmzZFCxuVo7CvX0fQSSn1mByDUT4Y0WFWrJl6RnbelZaaYpe3Ty3B0N7LSBhE4r8ukDa2wjrdVWJj86iE8mflilfExv20b1OjAPxhGgLeGCDQ628ZSl4JkzGfseisywrH9g2exLV01UNrDPZsB2uYv7v7Zvp0O1a2VJeOVj/zmPxKSIO81YMbQvHEGgitenrHxcPpsqhPGsLWzwMbaJmLjGgUUsFwgUuKcwN4Lr63oG+OqUStJdqrhdS02w9bVvnp6TFKTx+tV2o3G2bxATpQ7ODUD2H0tpj1jUfBN6zzPqHW0BADv/YwVPzXLL5xiQKSEK/GlgAa7SbhpyN+gUsFl3NPMmkcilw6toGvAeyOu2IiABgzwudLde499cM5vJ4Lb7RPAZfAlJusnNeJOQQ9/aCVFDGWduJjL88SQBLsdki518FgnW+EFXTcHbg6gyNF3x0fvc0I/aW790fU5Y+/V9DntHwqc/U/fxjXCirTIzG7tAGTKu/vHqEhJ6YHSe8Q9QXo4ufjvCMmJawpJtlgkRQbzi2iRzc/ivgAXsLypDhBI+DSlRp8WDhYLo1nDk78xrmzHmOt4enQToVwI=","as":"Uint8Array","fake":1},"contextInfo":{"$proto":"proto.ContextInfo","mentionedJid":[],"groupMentions":[],"statusAttributions":[],"statusSourceType":0},"firstScanSidecar":{"$bytes":"tCO2Lfa2lx3upQ==","as":"Uint8Array","fake":1},"firstScanLength":10736,"scansSidecar":{"$bytes":"VhshEorYGlY74CThpO8TeolkSD9lBL9Iz8clozPyMUXB5J5QHuTFsQ==","as":"Uint8Array","fake":1},"midQualityFileSha256":{"$bytes":"KIp21nO7212crP+1UGlo/BO+QoWgy1JcqC3uoazC5FQ=","as":"Uint8Array","fake":1},"imageSourceType":0},"messageContextInfo":{"$proto":"proto.MessageContextInfo","threadId":[],"deviceListMetadata":{"$proto":"proto.DeviceListMetadata","senderKeyIndexes":[],"recipientKeyIndexes":[],"senderKeyHash":{"$bytes":"I5Jh4NW30cAwCw==","as":"Uint8Array","fake":1},"senderTimestamp":{"$long":"1790519013","u":true},"recipientKeyHash":{"$bytes":"du0CJVs95FDSlw==","as":"Uint8Array","fake":1},"recipientTimestamp":{"$long":"1790338634","u":true}},"deviceListMetadataVersion":2,"messageSecret":{"$bytes":"3ekIumLVue4GSjI+lzREKduDdSlTVHodQbLqttE4Zzw=","as":"Uint8Array","fake":1}}}}}],"conversationTimestamp":1790522399,"unreadCount":1}]} +{"seq":73,"t":982204.4,"socket":1,"batch":["chats.update","messages.upsert","contacts.update"]} diff --git a/test/fixtures/live/2026-09-27-rig-session/manifest.json b/test/fixtures/live/2026-09-27-rig-session/manifest.json new file mode 100644 index 0000000000..eff28446ae --- /dev/null +++ b/test/fixtures/live/2026-09-27-rig-session/manifest.json @@ -0,0 +1,31 @@ +{ + "format": "live-record/1", + "forkCommit": "bc522750", + "baileysVersion": "7.0.0-rc14", + "nodeVersion": "v24.21.0", + "waWebVersion": "2.3000.1048596303", + "phonePlatform": "smbi", + "accountType": "business", + "linkMethod": "existing-session", + "proxy": { + "used": false, + "protocol": null + }, + "sockets": 1, + "startedAt": "2026-09-27T15:03:37.370Z", + "openedAt": "2026-09-27T15:03:39.334Z", + "endedAt": null, + "checkId": "rig-session", + "date": "2026-09-27", + "phoneModel": "iPhone 16", + "osVersion": "iOS 18.6", + "whatsappAppVersion": "WhatsApp Business 25.24", + "countryCode": "972", + "replay": { + "owner": { + "id": "972500000000:6@s.whatsapp.net", + "lid": "100000000000000:6@lid", + "name": "Owner" + } + } +} diff --git a/test/fixtures/live/2026-09-27-rig-session/scrub-report.json b/test/fixtures/live/2026-09-27-rig-session/scrub-report.json new file mode 100644 index 0000000000..d4a7cb5c45 --- /dev/null +++ b/test/fixtures/live/2026-09-27-rig-session/scrub-report.json @@ -0,0 +1,12 @@ +{ + "leakGate": "pass", + "events": 54, + "webhooks": 23, + "people": 3, + "groups": 1, + "names": 5, + "texts": 6, + "messageIds": 8, + "bytes": 13, + "urls": 4 +} diff --git a/test/fixtures/live/2026-09-27-rig-session/webhooks.ndjson b/test/fixtures/live/2026-09-27-rig-session/webhooks.ndjson new file mode 100644 index 0000000000..b3f73ed618 --- /dev/null +++ b/test/fixtures/live/2026-09-27-rig-session/webhooks.ndjson @@ -0,0 +1,23 @@ +{"seq":3,"t":50.6,"event":"connection.update","data":{"instance":"test","state":"connecting","statusReason":200}} +{"seq":14,"t":2386.8,"event":"connection.update","data":{"instance":"test","wuid":"972500000000@s.whatsapp.net","profileName":"Owner","profilePictureUrl":"https://example.invalid/2","state":"open","statusReason":200}} +{"seq":19,"t":100585.4,"event":"contacts.upsert","data":[{"remoteJid":"972500000001@s.whatsapp.net","pushName":null,"lid":"100000000000001@lid","phoneNumber":"972500000001@s.whatsapp.net","instanceId":"piscing elit sed do eiusmod tempor i"}]} +{"seq":20,"t":100593.3,"event":"contacts.upsert","data":[{"remoteJid":"972500000001@s.whatsapp.net","pushName":"Name 2","profilePicUrl":null,"instanceId":"piscing elit sed do eiusmod tempor i","saved":true}]} +{"seq":21,"t":100852.9,"event":"contacts.update","data":[{"remoteJid":"972500000001@s.whatsapp.net","pushName":"Name 2","profilePicUrl":null,"instanceId":"piscing elit sed do eiusmod tempor i"}]} +{"seq":24,"t":152326.8,"event":"messages.update","data":{"keyId":"3AFFFFFFFFFFFFFFFFF1","remoteJid":"100000000000002@lid","fromMe":false,"participant":{"$u":1},"status":"READ","pollUpdates":{"$u":1},"instanceId":"piscing elit sed do eiusmod tempor i"}} +{"seq":30,"t":160782.5,"event":"chats.update","data":[{"remoteJid":"100000000000002@lid","instanceId":"piscing elit sed do eiusmod tempor i","archived":true,"pinned":null}]} +{"seq":33,"t":168496.5,"event":"chats.update","data":[{"remoteJid":"100000000000002@lid","instanceId":"piscing elit sed do eiusmod tempor i","archived":false}]} +{"seq":39,"t":237581.2,"event":"groups.upsert","data":[{"id":"120363000000000001@g.us","notify":{"$u":1},"addressingMode":"pn","subject":"Name 3","subjectOwner":"100000000000000@lid","subjectOwnerPn":"972500000000@s.whatsapp.net","subjectOwnerUsername":{"$u":1},"subjectTime":1790521654,"size":2,"creation":1790521654,"owner":"100000000000000@lid","ownerPn":"972500000000@s.whatsapp.net","ownerUsername":{"$u":1},"owner_country_code":"IL","desc":{"$u":1},"descId":{"$u":1},"descOwner":{"$u":1},"descOwnerPn":{"$u":1},"descOwnerUsername":{"$u":1},"descTime":{"$u":1},"linkedParent":{"$u":1},"restrict":false,"announce":false,"isCommunity":false,"isCommunityAnnounce":false,"joinApprovalMode":false,"memberAddMode":true,"participants":[{"id":"100000000000000@lid","phoneNumber":"972500000000@s.whatsapp.net","lid":{"$u":1},"username":{"$u":1},"admin":"superadmin"},{"id":"100000000000002@lid","phoneNumber":"972500000002@s.whatsapp.net","lid":{"$u":1},"username":{"$u":1},"admin":null}],"ephemeralDuration":{"$u":1},"author":"100000000000000@lid","authorPn":"972500000000@s.whatsapp.net","authorUsername":{"$u":1}}]} +{"seq":40,"t":237594.2,"event":"chats.upsert","data":[{"remoteJid":"120363000000000001@g.us","instanceId":"piscing elit sed do eiusmod tempor i","name":"Name 3","unreadMessages":0}]} +{"seq":43,"t":238786,"event":"messages.edited","data":{"$proto":"proto.Message.ProtocolMessage","type":30,"memberLabel":{"$proto":"proto.MemberLabel","label":"","labelTimestamp":{"$long":"1790521655","u":false}}}} +{"seq":48,"t":248614.8,"event":"groups.update","data":[{"id":"120363000000000001@g.us","subject":"Name 4","author":"100000000000000@lid","authorPn":{"$u":1},"authorUsername":{"$u":1}}]} +{"seq":49,"t":248618.1,"event":"chats.update","data":[{"remoteJid":"120363000000000001@g.us","instanceId":"piscing elit sed do eiusmod tempor i"}]} +{"seq":54,"t":255472.6,"event":"group-participants.update","data":{"id":"120363000000000001@g.us","author":"100000000000000@lid","authorPn":{"$u":1},"authorUsername":{"$u":1},"participants":[{"id":"100000000000002@lid","phoneNumber":"972500000002@s.whatsapp.net","admin":null}],"action":"remove","participantsData":[{"jid":"100000000000002@lid","phoneNumber":"972500000002@s.whatsapp.net","name":{"$u":1},"imgUrl":{"$u":1}}]}} +{"seq":59,"t":263034.1,"event":"group-participants.update","data":{"id":"120363000000000001@g.us","author":"100000000000000@lid","authorPn":{"$u":1},"authorUsername":{"$u":1},"participants":[{"id":"100000000000002@lid","phoneNumber":"972500000002@s.whatsapp.net","admin":null}],"action":"add","participantsData":[{"jid":"100000000000002@lid","phoneNumber":"972500000002@s.whatsapp.net","name":"Name 5","imgUrl":"https://example.invalid/3"}]}} +{"seq":66,"t":975862.8,"event":"messages.upsert","data":{"key":{"remoteJid":"972500000002@s.whatsapp.net","remoteJidAlt":"100000000000002@lid","remoteJidUsername":{"$u":1},"fromMe":false,"id":"3AFFFFFFFFFFFFFFFFF7","participant":"","participantAlt":{"$u":1},"participantUsername":{"$u":1},"addressingMode":"pn"},"pushName":"Name 5","status":"DELIVERY_ACK","message":{"conversation":"ons","messageContextInfo":{"threadId":[],"deviceListMetadata":{"senderKeyIndexes":[],"recipientKeyIndexes":[],"senderKeyHash":{"$bytes":"I5Jh4NW30cAwCw==","as":"Uint8Array","fake":1},"senderTimestamp":{"low":1790519013,"high":0,"unsigned":true},"recipientKeyHash":{"$bytes":"du0CJVs95FDSlw==","as":"Uint8Array","fake":1},"recipientTimestamp":{"low":1790338634,"high":0,"unsigned":true}},"deviceListMetadataVersion":2,"messageSecret":{"$bytes":"xm/1WTK3DwFOXhTt118a9sxR6phRPmPy/gzLmVoCBjI=","as":"Uint8Array","fake":1}}},"contextInfo":{"$u":1},"messageType":"conversation","messageTimestamp":1790522393,"instanceId":"piscing elit sed do eiusmod tempor i","source":"ios"}} +{"seq":67,"t":976146.6,"event":"contacts.update","data":{"remoteJid":"972500000002@s.whatsapp.net","pushName":"Name 5","profilePicUrl":"https://example.invalid/4","instanceId":"piscing elit sed do eiusmod tempor i"}} +{"seq":68,"t":976154.9,"event":"chats.update","data":[{"remoteJid":"100000000000002@lid","instanceId":"piscing elit sed do eiusmod tempor i"}]} +{"seq":69,"t":976342.8,"event":"contacts.update","data":[{"remoteJid":"100000000000002@lid","pushName":{"$u":1},"profilePicUrl":"https://example.invalid/4","instanceId":"piscing elit sed do eiusmod tempor i"}]} +{"seq":74,"t":982212.9,"event":"messages.upsert","data":{"key":{"remoteJid":"972500000002@s.whatsapp.net","remoteJidAlt":"100000000000002@lid","remoteJidUsername":{"$u":1},"fromMe":false,"id":"3AFFFFFFFFFFFFFFFFF8","participant":"","participantAlt":{"$u":1},"participantUsername":{"$u":1},"addressingMode":"pn"},"pushName":"Name 5","status":"DELIVERY_ACK","message":{"imageMessage":{"interactiveAnnotations":[],"scanLengths":[10736,36644,13660,16018],"annotations":[],"url":"https://example.invalid/1","mimetype":"image/jpeg","fileSha256":{"$bytes":"3jIO+TEbNIv1XkEOFHukY69vOCqotjwsGtFcDv6bT60=","as":"Uint8Array","fake":1},"fileLength":{"low":77060,"high":0,"unsigned":true},"height":2048,"width":945,"mediaKey":{"$bytes":"v+lG1QOSwT+2PCe/80+f6XKeKEZeHwCNze0boDGgjh8=","as":"Uint8Array","fake":1},"fileEncSha256":{"$bytes":"v8DEQOf7nNkxXe74QzxyyC1wIdfkLN9IWvAvVzxlELI=","as":"Uint8Array","fake":1},"directPath":"tur adipiscing elit sed do eiusmod tempor incididunt ut labore lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod tempor incididunt ut labore lorem ipsum dolor sit amet consectetur adipiscing elit ","mediaKeyTimestamp":{"low":1790522397,"high":0,"unsigned":false},"jpegThumbnail":{"$bytes":"/mSuHpBcYSR+7bQUQoyOURa2t4c8toD4DQpP4X586D8W3Vronhm989lmrL4fMXowBGgT9GF76rBwFYd5oHDCWlox+ZmQqUy4VQ+27NJ2D6w7KTngDveGJXE/gBT6UD1Wp9LOvMNwkPfXMC4j3EVyNild+S9STRQ5bBbMnn1tU/YOzkGmC2JhKRyH+mMpdQcFm4ziNxC+T/ojXxTIfkVLmiOVKhAVbrWxM3HSl3muxZgQt3S8IxBQyiEZ2ZdheJXy5CU4rxH4hBf3R76X+1zerw0VxqI5yMS4VmXk0X/wubJNcRnFQBmiuLk6yMK44qDl17ATNh2zXxLRFqYU30Rha8BO6dRh3HZYAvrM17uPZD+OKDHJm54cTlHk61EzA9BdiVLcSH6pG7xzga1zvQJFB4/VLLPaFk0mif9Ydk1jkWmANn3PrJ1iFymn0XO7PAenUq0pR4roZ41pUYGXYkjBVLuMkMdq5P6kNtNWN/SiFszAIEtCbOIeFm3usNLsnKj7F+HVmhOLe1rT6uTFYJF7EzgAQZ4/MjkijemeRfo6y+ZiTi8KcHekYh5uFEQkljsmhS6LGEqFiSAY73whQd1K8vGxYkPHVsdoga48yiO+emAqpCt5QrJgcMqWibLF079hxZdzi+6IDl79nfKPKik7fSG21KfBWBWEACteUbUmqnb6P0AwDXA4bgacd6oY3s8ngjEGGwvgHdvZDMdkav6/Q7G17FeCVJhrVyhOtT53bXHoncXnLVCYtKJbW5pc9TC6BwxQuOzlxTshYKpuJLuT/G3R3a8GUfREorEh3dZLJU9o6jVmZJZ5jnLEdEmxL2ZIsZX7FxaAxiZ3FQPaZQwVznx4bh8NkBMplZXp9/c/2p668joyymcR3IThB4oVhZPchjtyvNTbZ3Y57KCu3FKTLpDDnL3qdv/z3rKondwSP3ThFnqYZ05RXe2viuO3+Urhwd4WAtj8iYcgact62CL/2SI5rU4zg2KukhvRkYElpIh7g/wjFMLwL5E9/QmzZFCxuVo7CvX0fQSSn1mByDUT4Y0WFWrJl6RnbelZaaYpe3Ty3B0N7LSBhE4r8ukDa2wjrdVWJj86iE8mflilfExv20b1OjAPxhGgLeGCDQ628ZSl4JkzGfseisywrH9g2exLV01UNrDPZsB2uYv7v7Zvp0O1a2VJeOVj/zmPxKSIO81YMbQvHEGgitenrHxcPpsqhPGsLWzwMbaJmLjGgUUsFwgUuKcwN4Lr63oG+OqUStJdqrhdS02w9bVvnp6TFKTx+tV2o3G2bxATpQ7ODUD2H0tpj1jUfBN6zzPqHW0BADv/YwVPzXLL5xiQKSEK/GlgAa7SbhpyN+gUsFl3NPMmkcilw6toGvAeyOu2IiABgzwudLde499cM5vJ4Lb7RPAZfAlJusnNeJOQQ9/aCVFDGWduJjL88SQBLsdki518FgnW+EFXTcHbg6gyNF3x0fvc0I/aW790fU5Y+/V9DntHwqc/U/fxjXCirTIzG7tAGTKu/vHqEhJ6YHSe8Q9QXo4ufjvCMmJawpJtlgkRQbzi2iRzc/ivgAXsLypDhBI+DSlRp8WDhYLo1nDk78xrmzHmOt4enQToVwI=","as":"Uint8Array","fake":1},"contextInfo":{"mentionedJid":[],"groupMentions":[],"statusAttributions":[],"statusSourceType":0},"firstScanSidecar":{"$bytes":"tCO2Lfa2lx3upQ==","as":"Uint8Array","fake":1},"firstScanLength":10736,"scansSidecar":{"$bytes":"VhshEorYGlY74CThpO8TeolkSD9lBL9Iz8clozPyMUXB5J5QHuTFsQ==","as":"Uint8Array","fake":1},"midQualityFileSha256":{"$bytes":"KIp21nO7212crP+1UGlo/BO+QoWgy1JcqC3uoazC5FQ=","as":"Uint8Array","fake":1},"imageSourceType":0},"messageContextInfo":{"threadId":[],"deviceListMetadata":{"senderKeyIndexes":[],"recipientKeyIndexes":[],"senderKeyHash":{"$bytes":"I5Jh4NW30cAwCw==","as":"Uint8Array","fake":1},"senderTimestamp":{"low":1790519013,"high":0,"unsigned":true},"recipientKeyHash":{"$bytes":"du0CJVs95FDSlw==","as":"Uint8Array","fake":1},"recipientTimestamp":{"low":1790338634,"high":0,"unsigned":true}},"deviceListMetadataVersion":2,"messageSecret":{"$bytes":"3ekIumLVue4GSjI+lzREKduDdSlTVHodQbLqttE4Zzw=","as":"Uint8Array","fake":1}}},"contextInfo":{"mentionedJid":[],"groupMentions":[],"statusAttributions":[],"statusSourceType":0},"messageType":"imageMessage","messageTimestamp":1790522399,"instanceId":"piscing elit sed do eiusmod tempor i","source":"ios"}} +{"seq":75,"t":982216.6,"event":"contacts.update","data":{"remoteJid":"972500000002@s.whatsapp.net","pushName":"Name 5","profilePicUrl":"https://example.invalid/4","instanceId":"piscing elit sed do eiusmod tempor i"}} +{"seq":76,"t":982224.8,"event":"chats.update","data":[{"remoteJid":"100000000000002@lid","instanceId":"piscing elit sed do eiusmod tempor i"}]} +{"seq":77,"t":982225.2,"event":"contacts.update","data":[{"remoteJid":"100000000000002@lid","pushName":{"$u":1},"profilePicUrl":"https://example.invalid/4","instanceId":"piscing elit sed do eiusmod tempor i"}]} diff --git a/test/live/app-state-rename.test.ts b/test/live/app-state-rename.test.ts new file mode 100644 index 0000000000..34389b0c98 --- /dev/null +++ b/test/live/app-state-rename.test.ts @@ -0,0 +1,39 @@ +// Live check app-state-after-restart, recorded 2026-09-27 (docs/LIVE-CHECKS.md): +// after a restart, the owner renamed a saved contact on the phone. WhatsApp sent +// the rename as an app-state contact action with the @lid to phone mapping, and +// Evolution sent the mapping item, the contact marked saved, and the update echo. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { describe, expect, it } from 'vitest'; + +import { encode } from '@utils/live-record/codec'; + +import { compareGolden, loadFixture, replayFixture } from '../helpers/live-replay'; + +const FIXTURE = 'test/fixtures/live/2026-09-27-rig-session'; +const CONTACT = { pn: '972500000001@s.whatsapp.net', lid: '100000000000001@lid' }; + +/** Webhooks about the renamed contact only (the DM later in the session depends on the live database). */ +const aboutContact = (w: { data: any }) => JSON.stringify(encode(w.data)).includes(CONTACT.pn.split('@')[0]); + +describe('live: a contact renamed on the phone after a restart', () => { + it('reaches contacts.upsert as a mapping item and as a saved name, then contacts.update', async () => { + const { webhooks } = await replayFixture(FIXTURE); + const contacts = webhooks.filter((w) => w.event.startsWith('contacts.') && aboutContact(w)); + const upserts = contacts.filter((w) => w.event === 'contacts.upsert').flatMap((w) => w.data); + + // The @lid to phone mapping, forwarded as its own item. + expect(upserts).toContainEqual( + expect.objectContaining({ remoteJid: CONTACT.pn, lid: CONTACT.lid, phoneNumber: CONTACT.pn }), + ); + // The name the owner saved, marked saved. + const saved = upserts.find((c) => c.saved !== undefined); + expect(saved).toMatchObject({ remoteJid: CONTACT.pn, pushName: 'Name 2', saved: true }); + + expect(contacts.map((w) => w.event)).toEqual(['contacts.upsert', 'contacts.upsert', 'contacts.update']); + const golden = loadFixture(FIXTURE).webhooks.filter((w) => w.event.startsWith('contacts.') && aboutContact(w)); + expect(compareGolden(contacts, golden)).toEqual([]); + }); +}); From 584077c941fd83c175b420ba0955d2f5c357f5a3 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 18:28:01 +0300 Subject: [PATCH 107/157] test(live): archive and unarchive on the phone, replayed from a live check Pins live-verified behaviour that already exists (fixture 2026-09-27-rig-session): the archive reaches chats.update with archived: true (and pinned: null from the unpin WhatsApp sent with it), the unarchive with archived: false, in that order, and every chats.update matches the recorded webhooks. Teeth, checked locally and restored: with the chats.update half of 1e79cf1a reverted (the whole revert no longer applies cleanly), no item carries archived and the test fails. Co-Authored-By: Claude Opus 5.5 --- test/live/archive-toggle.test.ts | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) create mode 100644 test/live/archive-toggle.test.ts diff --git a/test/live/archive-toggle.test.ts b/test/live/archive-toggle.test.ts new file mode 100644 index 0000000000..6ed1376aee --- /dev/null +++ b/test/live/archive-toggle.test.ts @@ -0,0 +1,30 @@ +// Live check archive-toggle, recorded 2026-09-27 (docs/LIVE-CHECKS.md): the owner +// archived a chat on the phone, then unarchived it. WhatsApp sent app-state chat +// actions (the archive together with an unpin), and Evolution sent chats.update +// with archived: true, then archived: false. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { describe, expect, it } from 'vitest'; + +import { compareGolden, loadFixture, replayFixture } from '../helpers/live-replay'; + +const FIXTURE = 'test/fixtures/live/2026-09-27-rig-session'; +const CHAT = '100000000000002@lid'; + +describe('live: archive and unarchive on the phone', () => { + it('reaches chats.update with archived: true, then archived: false', async () => { + const { webhooks } = await replayFixture(FIXTURE); + const archive = webhooks + .filter((w) => w.event === 'chats.update') + .flatMap((w) => w.data) + .filter((c) => c.archived !== undefined); + + expect(archive).toEqual([ + expect.objectContaining({ remoteJid: CHAT, archived: true, pinned: null }), + expect.objectContaining({ remoteJid: CHAT, archived: false }), + ]); + expect(compareGolden(webhooks, loadFixture(FIXTURE).webhooks, { events: ['chats.update'] })).toEqual([]); + }); +}); From e38b506e6938b443db4ccb9891c1c0f3c85da75b Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 18:28:33 +0300 Subject: [PATCH 108/157] test(live): a group renamed and a member removed and added back, replayed from a live check Pins live-verified behaviour that already exists (fixture 2026-09-27-rig-session): groups.upsert for the new group, groups.update with the new subject, and group-participants.update for the remove and the add, where Baileys 7 emitted each participant as an object and participantsData carries the @lid as a jid string and the phone JID, never "[object Object]". Group metadata was a socket query and is answered from the group as groups.upsert described it; participantsData's name and picture came from the live database and are left out of the golden comparison. Teeth, checked locally and restored: - 8fd90040 (Baileys 7 participant objects) reverted: the participants test fails, participantsData's jid is an object. - the groups.update webhook stubbed out (stock behaviour, no fork fix to revert): the rename test fails, no groups.update. Co-Authored-By: Claude Opus 5.5 --- test/live/group-rename-participants.test.ts | 55 +++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 test/live/group-rename-participants.test.ts diff --git a/test/live/group-rename-participants.test.ts b/test/live/group-rename-participants.test.ts new file mode 100644 index 0000000000..9d43854a8f --- /dev/null +++ b/test/live/group-rename-participants.test.ts @@ -0,0 +1,55 @@ +// Live check group-rename-participants, recorded 2026-09-27 (docs/LIVE-CHECKS.md): +// the owner created a group, renamed it, removed a member and added them back. +// Baileys 7 emits each participant as an object ({ id: @lid, phoneNumber }), and +// Evolution's participantsData must read it: a jid string and a phone JID, never +// "[object Object]". +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { describe, expect, it } from 'vitest'; + +import { compareGolden, loadFixture, replayFixture } from '../helpers/live-replay'; + +const FIXTURE = 'test/fixtures/live/2026-09-27-rig-session'; +const GROUP = '120363000000000001@g.us'; +const OWNER = { id: '100000000000000@lid', phoneNumber: '972500000000@s.whatsapp.net', admin: 'superadmin' }; +const MEMBER = { id: '100000000000002@lid', phoneNumber: '972500000002@s.whatsapp.net', admin: null }; + +// Group metadata was a socket query, not recorded: answer with the group as groups.upsert described it. +const client = { groupMetadata: async (id: string) => ({ id, subject: 'Name 4', participants: [OWNER, MEMBER] }) }; +// participantsData's name and picture come from the live database and picture queries. +const FROM_QUERIES = ['name', 'imgUrl']; + +describe('live: a group renamed, a member removed and added back', () => { + it('reaches groups.upsert, then groups.update with the new subject', async () => { + const { webhooks } = await replayFixture(FIXTURE, { client }); + const upserts = webhooks.filter((w) => w.event === 'groups.upsert').flatMap((w) => w.data); + const updates = webhooks.filter((w) => w.event === 'groups.update').flatMap((w) => w.data); + + expect(upserts).toEqual([expect.objectContaining({ id: GROUP, subject: 'Name 3', size: 2 })]); + expect(updates).toEqual([expect.objectContaining({ id: GROUP, subject: 'Name 4' })]); + const events = ['groups.upsert', 'groups.update']; + expect(compareGolden(webhooks, loadFixture(FIXTURE).webhooks, { events })).toEqual([]); + }); + + it('reaches group-participants.update with participantsData holding the @lid and the phone JID', async () => { + const { webhooks } = await replayFixture(FIXTURE, { client }); + const updates = webhooks.filter((w) => w.event === 'group-participants.update').map((w) => w.data); + + expect(updates.map((u) => u.action)).toEqual(['remove', 'add']); + for (const update of updates) { + expect(update.id).toBe(GROUP); + expect(update.participants).toEqual([{ id: MEMBER.id, phoneNumber: MEMBER.phoneNumber, admin: null }]); + expect(update.participantsData).toHaveLength(1); + const [data] = update.participantsData; + expect(typeof data.jid).toBe('string'); + expect(data.jid).toBe(MEMBER.id); + expect(data.phoneNumber).toBe(MEMBER.phoneNumber); + expect(JSON.stringify(update)).not.toContain('[object Object]'); + } + const events = ['group-participants.update']; + const golden = loadFixture(FIXTURE).webhooks; + expect(compareGolden(webhooks, golden, { events, volatile: FROM_QUERIES })).toEqual([]); + }); +}); From bc6ee7895aeabe19c91d091d81136d15756bcc43 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 18:28:56 +0300 Subject: [PATCH 109/157] test(live): a DM addressed by @lid keeps its @lid, replayed from a live check Pins live-verified behaviour that already exists (fixture 2026-09-27-rig-session): a text and an image that WhatsApp addressed by @lid reach messages.upsert with the phone JID as remoteJid, the original @lid in remoteJidAlt and addressingMode 'pn', matching the recorded webhooks. Teeth, checked locally and restored: with f3d1052a reverted, remoteJidAlt is the phone JID a second time and addressingMode stays 'lid', and the test fails. Co-Authored-By: Claude Opus 5.5 --- test/live/live-lid-message-key.test.ts | 33 ++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 test/live/live-lid-message-key.test.ts diff --git a/test/live/live-lid-message-key.test.ts b/test/live/live-lid-message-key.test.ts new file mode 100644 index 0000000000..46a2f472a9 --- /dev/null +++ b/test/live/live-lid-message-key.test.ts @@ -0,0 +1,33 @@ +// Live check live-lid-message-key, recorded 2026-09-27 (docs/LIVE-CHECKS.md): a +// text and an image arrived in a DM that WhatsApp addressed by @lid (remoteJid +// @lid, remoteJidAlt the phone, addressingMode 'lid'). Evolution shows the phone +// as remoteJid and keeps the original @lid in remoteJidAlt, with addressingMode 'pn'. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { describe, expect, it } from 'vitest'; + +import { compareGolden, loadFixture, replayFixture } from '../helpers/live-replay'; + +const FIXTURE = 'test/fixtures/live/2026-09-27-rig-session'; +const SENDER = { pn: '972500000002@s.whatsapp.net', lid: '100000000000002@lid' }; + +describe('live: a DM addressed by @lid', () => { + it('reaches messages.upsert with the phone as remoteJid and the @lid kept in remoteJidAlt', async () => { + const { webhooks } = await replayFixture(FIXTURE); + const upserts = webhooks.filter((w) => w.event === 'messages.upsert').map((w) => w.data); + + expect(upserts.map((m) => m.messageType)).toEqual(['conversation', 'imageMessage']); + for (const message of upserts) { + expect(message.key).toMatchObject({ + remoteJid: SENDER.pn, + remoteJidAlt: SENDER.lid, + addressingMode: 'pn', + fromMe: false, + }); + } + const golden = loadFixture(FIXTURE).webhooks; + expect(compareGolden(webhooks, golden, { events: ['messages.upsert'] })).toEqual([]); + }); +}); From c804f38277a2fb6f5769f58420de638130b96c73 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 18:30:35 +0300 Subject: [PATCH 110/157] style(live-record): prettier on the scrubber's numeric id, and a type for the rename test's filter Co-Authored-By: Claude Opus 5.5 --- test/live/app-state-rename.test.ts | 2 +- test/tools/live-scrub.ts | 5 ++++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/test/live/app-state-rename.test.ts b/test/live/app-state-rename.test.ts index 34389b0c98..105476bb42 100644 --- a/test/live/app-state-rename.test.ts +++ b/test/live/app-state-rename.test.ts @@ -16,7 +16,7 @@ const FIXTURE = 'test/fixtures/live/2026-09-27-rig-session'; const CONTACT = { pn: '972500000001@s.whatsapp.net', lid: '100000000000001@lid' }; /** Webhooks about the renamed contact only (the DM later in the session depends on the live database). */ -const aboutContact = (w: { data: any }) => JSON.stringify(encode(w.data)).includes(CONTACT.pn.split('@')[0]); +const aboutContact = (w: Record) => JSON.stringify(encode(w.data)).includes(CONTACT.pn.split('@')[0]); describe('live: a contact renamed on the phone after a restart', () => { it('reaches contacts.upsert as a mapping item and as a saved name, then contacts.update', async () => { diff --git a/test/tools/live-scrub.ts b/test/tools/live-scrub.ts index 31c0fd6a5a..66c0b58ea4 100644 --- a/test/tools/live-scrub.ts +++ b/test/tools/live-scrub.ts @@ -216,7 +216,10 @@ class Scrubber { private fakeId(s: string) { return this.memo(this.ids, s, (n) => { - if (/^\d+$/.test(s)) return s.slice(0, 2) + String(n).padStart(s.length - 2, '0').slice(2 - s.length); + if (/^\d+$/.test(s)) { + const counter = String(n).padStart(s.length - 2, '0'); + return s.slice(0, 2) + counter.slice(2 - s.length); + } const body = n .toString(16) .toUpperCase() From 2f1c5a7c3a5cf4aec0f036ded1bc3d61ab81c88d Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 18:30:43 +0300 Subject: [PATCH 111/157] docs(live-checks): log the 2026-09-27 rig session Five rows at fork 71b5ae0b, Baileys 7.0.0-rc14, WA Web 2.3000.1048596303, on WhatsApp Business (platform smbi) on an iPhone 16, iOS 18.6, app 25.24: the app-state rename after a restart, the archive toggle, the group rename and participants, the live @lid message key, and clean logs, all passing. The first four are replayed from the fixture 2026-09-27-rig-session. Co-Authored-By: Claude Opus 5.5 --- docs/LIVE-CHECKS.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/docs/LIVE-CHECKS.md b/docs/LIVE-CHECKS.md index 8cf805b120..f4bde09a61 100644 --- a/docs/LIVE-CHECKS.md +++ b/docs/LIVE-CHECKS.md @@ -194,6 +194,13 @@ it is not a guess. | 2026-09-27 | `clean-logs` | 3fc63a62 | 7.0.0-rc14 | not recorded | business, platform not recorded / not recorded / not recorded; linked earlier by QR; no proxy | at `LOG_LEVEL=ERROR,WARN`, `LOG_BAILEYS=error`: 48 lines across a restart, 2 sessions reconnecting and 2 group listings held 0 digit runs of 8+, 0 WhatsApp addresses, 0 media URLs. 1 finding: an earlier ERROR-level media download failure printed the signed media URL (fixed separately). At full verbosity (INFO to WEBHOOKS, Baileys debug) addresses appear by design | | 2026-09-27 | `reconnect-backoff-one-socket` | 3fc63a62 | 7.0.0-rc14 | not recorded | business, platform not recorded / not recorded / not recorded; linked earlier by QR; no proxy | network cut for 180s: attempts 1, 2, 4, 8, 16, 32, 60s apart (status 408), capped at 60s, never gave up; reopened on the first attempt after the network returned. 1 finding: the failure line logged `[object Object]` (fixed separately). Sockets not counted | | 2026-09-27 | `bounded-queries` | 3fc63a62 | 7.0.0-rc14 | not recorded | business, platform not recorded / not recorded / not recorded | not measurable: the account is in no groups. Moved to a later session | - -The phone platform and model were not recorded for any of these runs: the -recorder did not exist yet, and the operator did not write them down. +| 2026-09-27 | `app-state-after-restart` | bc522750 | 7.0.0-rc14 | 2.3000.1048596303 | smbi (WhatsApp Business) / iPhone 16, iOS 18.6 / WhatsApp Business 25.24 | PASS: after a restart, a contact renamed on the phone reached `contacts.upsert` with `saved: true` and a mapping item (`lid`, `phoneNumber`), then `contacts.update`. Replayed from `2026-09-27-rig-session` in `test/live/app-state-rename.test.ts` | +| 2026-09-27 | `archive-toggle` | bc522750 | 7.0.0-rc14 | 2.3000.1048596303 | smbi (WhatsApp Business) / iPhone 16, iOS 18.6 / WhatsApp Business 25.24 | PASS: archive, then unarchive, reached `chats.update` with `archived: true` (with `pinned: null`), then `archived: false`. Replayed in `test/live/archive-toggle.test.ts` | +| 2026-09-27 | `group-rename-participants` | bc522750 | 7.0.0-rc14 | 2.3000.1048596303 | smbi (WhatsApp Business) / iPhone 16, iOS 18.6 / WhatsApp Business 25.24 | PASS: the rename reached `groups.update` with the new subject. PASS: a member removed and added back reached `group-participants.update` with `participantsData` holding the @lid as `jid` and the phone JID as `phoneNumber`. Promote and demote not run. Replayed in `test/live/group-rename-participants.test.ts` | +| 2026-09-27 | `live-lid-message-key` | bc522750 | 7.0.0-rc14 | 2.3000.1048596303 | smbi (WhatsApp Business) / iPhone 16, iOS 18.6 / WhatsApp Business 25.24 | PASS: a text and an image in a DM addressed by @lid reached `messages.upsert` with the phone JID as `remoteJid`, the @lid kept in `remoteJidAlt`, `addressingMode: 'pn'`. Replayed in `test/live/live-lid-message-key.test.ts` | +| 2026-09-27 | `clean-logs` | bc522750 | 7.0.0-rc14 | 2.3000.1048596303 | smbi (WhatsApp Business) / iPhone 16, iOS 18.6 / WhatsApp Business 25.24 | PASS: production-level logs over the session, 49 lines: 0 digit runs of 8+, 0 WhatsApp addresses, 0 media URLs | + +The phone platform and model were not recorded for the earlier runs: the +recorder did not exist yet, and the operator did not write them down. The rig +session (the rows at fork bc522750) ran on a different phone, recorded, and one +fixture (`test/fixtures/live/2026-09-27-rig-session`) covers its four checks. From b86b102b3ab230c91436efcdc7fd7e9410e5e380 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 19:47:08 +0300 Subject: [PATCH 112/157] test: a 403 is judged by the link that actually failed The 403 rule (431ebb84) read `oe` from the message's url first and fell back to the directPath, but Baileys 7.0.0-rc14 downloads the directPath (over https, from the url's host) whenever the message has one, and the url only otherwise. So the rule could judge a link that was never requested: an expired url next to a valid directPath asked the phone, a valid url next to an expired directPath did not, and an expired url next to a directPath with no `oe` asked. Its regex also read an `oe` from the fragment, which is never sent, missed a percent-encoded value, and took the first of two. The rule these tests specify: a 403 asks the phone only when the link that actually failed carries an `oe` that has passed by the local clock. That link is the one on Baileys' error (Boom data.url); an error without one falls back to the link Baileys would request (directPath, else url), never to the other. `oe` is a query parameter: case-sensitive name, percent-decoded, never from the fragment, exactly one value, plain hex. It stays a conservative heuristic from the 84-attachment sample (403 on 34 of 34 links whose `oe` had passed, 0 of 50 valid ones), so the clock is fixed in these tests. Cases: `oe` a day past, exactly now, lowercase hex, percent-encoded (ask); one second ahead, 14 days ahead, missing, empty, not hex, 0x-prefixed, too large, given twice, spelled OE, only in the fragment (do not ask); an expired url with a valid directPath, a valid url with an expired directPath, an expired url with a directPath without `oe`; and an error that carries no link. Red on fork/2.3.7: 8 of the 20 fail (percent-encoded, given twice, the fragment, the three url/directPath conflicts, and the two no-link cases where the directPath decides), each on whether the phone was asked. The other 12 already hold and stay green. Co-Authored-By: Claude Opus 5.5 --- test/proxy/media-reupload.test.ts | 183 +++++++++++++++++++++++++++++- 1 file changed, 177 insertions(+), 6 deletions(-) diff --git a/test/proxy/media-reupload.test.ts b/test/proxy/media-reupload.test.ts index 684be69dc8..d8a0a8cb76 100644 --- a/test/proxy/media-reupload.test.ts +++ b/test/proxy/media-reupload.test.ts @@ -10,11 +10,12 @@ // that check never matches and rc14 never asks. Evolution asks itself when // Baileys did not. // -// A 403 asks the phone only when the media link itself has expired: its `oe` -// query parameter (hex unix seconds, read from the url, else the directPath) -// has passed. Measured on WhatsApp's media CDN (2026-09-27, 84 history-sync -// attachments): 403 on 34 of 34 links whose `oe` had passed, on 0 of 50 valid -// ones, and a valid link to a file the CDN dropped answered 404 or 410. +// A 403 asks the phone only when the link that actually failed carries an `oe` +// query parameter (hex unix seconds) that has passed by the local clock. A +// conservative heuristic, not a documented contract: measured on WhatsApp's media +// CDN (2026-09-27, 84 history-sync attachments), 403 on 34 of 34 links whose `oe` +// had passed, on 0 of 50 valid ones, and a valid link to a file the CDN dropped +// answered 404 or 410. import { vi } from 'vitest'; vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); @@ -24,7 +25,7 @@ import { readFile, rm } from 'node:fs/promises'; import { MEDIA_REUPLOAD_TIMEOUT_MS } from '@api/integrations/channel/whatsapp/whatsapp.baileys.service'; import { encryptedStream } from 'baileys'; import { getGlobalDispatcher, MockAgent, setGlobalDispatcher } from 'undici'; -import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from 'vitest'; import { makeService } from '../helpers/baileys-service'; import { captureOutput } from '../helpers/capture-output'; @@ -302,3 +303,173 @@ describe('an expired media download asks the phone to re-upload, once', () => { expectNothingPrivate(out); }); }); + +// Which link a 403 is judged by: the one the download actually requested. Baileys +// downloads the directPath (over https, from the url's host) when the message has one, +// and the url only otherwise; its error carries the link it requested (Boom data.url). +// An error without a link falls back to that same choice, never to the other link. +// `oe` is read as a query parameter: case-sensitive name, percent-decoded, never from +// the fragment, exactly one value, plain hex. The comparison uses the local clock, so +// these tests fix it. +describe('a 403 is judged by the link that actually failed', () => { + const NOW = 1_790_000_000; + const hex = (unixSeconds: number) => unixSeconds.toString(16).toUpperCase(); + const signed = (path: string, expiry: string) => `${path}?ccb=11-4&oh=01_Q5Aa&oe=${expiry}&_nc_sid=5e03e0`; + const EXPIRED = hex(NOW - DAY); + const VALID = hex(NOW + 14 * DAY); + const FORBIDDEN = '/v/t62.7118-24/forbidden.enc'; + const OTHER = '/v/t62.7118-24/other.enc'; + + let tlsCdn: Listening; + let untrust: () => void; + let clock: { mockRestore: () => void }; + let fetchSpy: { mockRestore: () => void } | undefined; + + beforeAll(async () => { + // Both links are served over https here, since a directPath always is. + untrust = trustTestCertificate(); + tlsCdn = await startHttpsServer((req, _body, res) => { + if (req.url === LIVE) return void res.writeHead(200, { 'content-length': liveBody.length }).end(liveBody); + res.writeHead(403).end(); + }); + }); + + afterAll(async () => { + await tlsCdn.close(); + untrust(); + }); + + beforeEach(() => { + tlsCdn.log.splice(0); + clock = vi.spyOn(Date, 'now').mockReturnValue(NOW * 1000); + }); + + afterEach(() => { + clock.mockRestore(); + fetchSpy?.mockRestore(); + fetchSpy = undefined; + }); + + /** An image whose url and directPath (when given) point at the https CDN. */ + const image = (urlPath: string, directPath?: string) => { + const message: any = expiredImage(); + message.message.imageMessage.url = `https://127.0.0.1:${tlsCdn.port}${urlPath}`; + if (directPath !== undefined) message.message.imageMessage.directPath = directPath; + return message; + }; + + /** A service whose phone re-uploads to LIVE, refreshing both links as Baileys does. */ + async function run(message: any) { + const { service } = await makeService(); + const asked: string[] = []; + service.client.updateMediaMessage = async (m: any) => { + asked.push(m.key.id); + m.message.imageMessage.url = `https://127.0.0.1:${tlsCdn.port}${LIVE}`; + if (m.message.imageMessage.directPath) m.message.imageMessage.directPath = LIVE; + return m; + }; + let result: any; + let thrown: any; + await captureOutput(async () => { + try { + result = await service.getBase64FromMediaMessage({ message }); + } catch (e) { + thrown = e; + } + }); + return { result, thrown, asked }; + } + + async function expectReupload(message: any, requested: string[]) { + const { result, thrown, asked } = await run(message); + + expect(thrown).toBeUndefined(); + expect(Buffer.from(result.base64, 'base64').equals(PLAIN)).toBe(true); + expect(asked).toEqual([ID]); + expect(tlsCdn.log).toEqual([...requested, LIVE].map((p) => `GET ${p}`)); + } + + async function expectNoReupload(message: any, requested: string[]) { + const { thrown, asked } = await run(message); + + expect(asked).toEqual([]); + expect(thrown).toEqual(badRequest('not_requested')); + expect(tlsCdn.log).toEqual(requested.map((p) => `GET ${p}`)); + } + + it.each([ + ['a day past', EXPIRED], + ['exactly now', hex(NOW)], + ['a day past, in lowercase hex', EXPIRED.toLowerCase()], + ['a day past, percent-encoded', [...EXPIRED].map((c) => `%${c.charCodeAt(0).toString(16)}`).join('')], + ])('asks the phone when the oe of the url that failed is %s', async (_, expiry) => { + const path = signed(FORBIDDEN, expiry); + + await expectReupload(image(path), [path]); + }); + + it.each([ + ['one second from now', signed(FORBIDDEN, hex(NOW + 1))], + ['14 days from now', signed(FORBIDDEN, VALID)], + ['missing', FORBIDDEN], + ['empty', signed(FORBIDDEN, '')], + ['not hex', signed(FORBIDDEN, `${EXPIRED}Z`)], + ['0x-prefixed', signed(FORBIDDEN, `0x${EXPIRED}`)], + ['too large to be a time', signed(FORBIDDEN, 'F'.repeat(20))], + ['given twice', `${FORBIDDEN}?oe=${EXPIRED}&oe=${EXPIRED}`], + ['spelled OE', `${FORBIDDEN}?OE=${EXPIRED}`], + ])('does not ask the phone when the oe of the url that failed is %s', async (_, path) => { + await expectNoReupload(image(path), [path]); + }); + + it('ignores an oe in the fragment, which is never sent', async () => { + await expectNoReupload(image(`${FORBIDDEN}#top?oe=${EXPIRED}`), [FORBIDDEN]); + }); + + it('does not ask the phone when the url has expired but the directPath that failed has not', async () => { + const directPath = signed(FORBIDDEN, VALID); + + await expectNoReupload(image(signed(OTHER, EXPIRED), directPath), [directPath]); + }); + + it('asks the phone when the directPath that failed has expired though the url has not', async () => { + const directPath = signed(FORBIDDEN, EXPIRED); + + await expectReupload(image(signed(OTHER, VALID), directPath), [directPath]); + }); + + it('does not ask the phone when the url has expired but the directPath that failed has no oe', async () => { + await expectNoReupload(image(signed(OTHER, EXPIRED), FORBIDDEN), [FORBIDDEN]); + }); + + describe('when the error does not say which link failed', () => { + // An error with a numeric status and no link, before any request reaches the CDN. + function failFirstFetch() { + const realFetch = globalThis.fetch; + let failed = false; + fetchSpy = vi.spyOn(globalThis, 'fetch').mockImplementation(async (...args) => { + if (failed) return realFetch(...args); + failed = true; + throw Object.assign(new Error('forbidden'), { status: 403 }); + }); + } + + it('judges by the directPath, which the download requests first', async () => { + failFirstFetch(); + + await expectReupload(image(signed(OTHER, VALID), signed(FORBIDDEN, EXPIRED)), []); + }); + + it('does not fall back to an expired url when there is a directPath', async () => { + failFirstFetch(); + + await expectNoReupload(image(signed(OTHER, EXPIRED), signed(FORBIDDEN, VALID)), []); + }); + + it('judges by the url when there is no directPath', async () => { + failFirstFetch(); + + await expectReupload(image(signed(FORBIDDEN, EXPIRED)), []); + }); + }); +}); From 6e605a43e030bbe0d8e681c8a4814befb7216cba Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 19:48:01 +0300 Subject: [PATCH 113/157] fix(baileys): judge a 403 by the link that actually failed A CDN 403 now counts as an expired file only when the link that actually failed carries an `oe` that has passed by the local clock. The link is the url on Baileys' error (Boom data.url), which is the one it requested: the directPath when the message has one, else the url. An error without a url falls back to that same choice, never to the link that was not requested. `oe` is read with URL and URLSearchParams (case-sensitive name, percent-decoded, never from the fragment) and counts only as a single plain hex value that makes a safe timestamp. Before, `oe` came from the url first, so an expired url next to a valid directPath asked the phone, and a valid url next to an expired directPath did not, though the directPath was what had failed. The rule stays a conservative heuristic from the 84-attachment sample, now described as one in the code and in FORK.md: a 403 without that evidence is not an expiry, as a policy, and a clock that is off moves the line. 404 and 410 are unchanged. Co-Authored-By: Claude Opus 5.5 --- FORK.md | 2 +- .../whatsapp/whatsapp.baileys.service.ts | 50 ++++++++++++------- 2 files changed, 33 insertions(+), 19 deletions(-) diff --git a/FORK.md b/FORK.md index 7a2619b92d..f9f7f92f86 100644 --- a/FORK.md +++ b/FORK.md @@ -53,7 +53,7 @@ named where one exists. - No message text, phone number, JID or push name reaches the logs at `LOG_LEVEL=ERROR,WARN`, including Baileys' own error logs. - The `messages.upsert` webhook key of a DM WhatsApp addresses by @lid shows the phone JID as `remoteJid`, as 2.3.7 does, but keeps the original @lid in `remoteJidAlt` (2.3.7 copied the phone there too and lost it), with `addressingMode: 'pn'`: upstream develop's swap. - A media re-upload request names the message by the address the phone stores it under: the @lid from `remoteJidAlt` or `participantAlt` when the key shows the phone, or, for a key stored from 2.3.7 (the phone twice, `addressingMode: 'lid'`), the @lid Baileys' LID mapping has for the phone. The phone refuses a request that names an @lid chat by its phone JID. -- A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error), and when the phone refused, why (`reason=` on the log line, `reuploadReason` on the error and the HTTP answer: the phone's result such as `NOT_FOUND`, `error_`, `missing_ciphertext`, or `no_answer`). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410 (its check misses the status), so Evolution asks the phone itself, once per download, for at most 60s. A CDN 403 counts as expired only when the media link's `oe` (hex unix seconds, from the url, else the directPath) has passed: measured on 84 history-sync attachments, 403 answered 34 of 34 expired links and 0 of 50 valid ones, where a valid link to a dropped file answered 404 or 410. +- A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error), and when the phone refused, why (`reason=` on the log line, `reuploadReason` on the error and the HTTP answer: the phone's result such as `NOT_FOUND`, `error_`, `missing_ciphertext`, or `no_answer`). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410 (its check misses the status), so Evolution asks the phone itself, once per download, for at most 60s. A CDN 403 counts as expired only when the link that actually failed (the one on Baileys' error, which is the directPath when the message has one, else the url) carries exactly one `oe` query parameter, in hex unix seconds, that has passed by the local clock. This is a conservative heuristic: on 84 history-sync attachments, 403 answered 34 of 34 expired links and 0 of 50 valid ones, where a valid link to a dropped file answered 404 or 410. - A media download turns the media key back into bytes (from a base64 string, or the object JSON makes of a Uint8Array or a Buffer) before it asks the phone to re-upload, since Baileys 7.0.0-rc14 derives the re-upload's key from the value as given and then cannot decrypt the phone's answer (Baileys #2729 proposed the same fix there). - `POST /chat/getBase64FromMediaMessage` takes an optional `reupload` (boolean, default true). With `false` the phone is never asked to re-upload, and a file gone from the CDN (404, 410, or 403 on an expired link) fails at once with a 400 that says so; the HTTP error carries `reupload` for any failed download. diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 4822c20656..6c9d12bb65 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -273,32 +273,46 @@ const reuploadRefusal = (error: any): string => { }; /** - * When a WhatsApp media link stops working, in unix seconds: its `oe` query parameter - * (hex), read from the url, else from the directPath. Undefined when neither has one. + * When a WhatsApp media link stops working, in ms: its `oe` query parameter (hex unix + * seconds). Read with URLSearchParams, so the name is case-sensitive, the value is + * percent-decoded and a fragment never counts. Undefined unless the query carries exactly + * one `oe` and it is plain hex. */ -const mediaLinkExpiry = ( - media: { url?: string | null; directPath?: string | null } | undefined, -): number | undefined => { - for (const link of [media?.url, media?.directPath]) { - const oe = link?.match(/[?&]oe=([0-9a-fA-F]+)(?:&|#|$)/)?.[1]; - if (oe) return parseInt(oe, 16); - } - return undefined; +const mediaLinkExpiry = (link: string): number | undefined => { + let values: string[]; + try { + // A directPath has no host; the base only makes it parseable. + values = new URL(link, 'https://mmg.whatsapp.net').searchParams.getAll('oe'); + } catch { + return undefined; + } + const [oe] = values; + if (values.length !== 1 || !/^[0-9a-f]+$/i.test(oe)) return undefined; + const expiry = parseInt(oe, 16) * 1000; + return Number.isSafeInteger(expiry) ? expiry : undefined; }; /** * A CDN answer that means the file has expired there, and only the phone still has it: - * 404 or 410, or 403 on a link whose `oe` has passed. Measured on WhatsApp's media CDN - * (2026-09-27, 84 history-sync attachments): 403 on 34 of 34 links whose `oe` had - * passed and on 0 of 50 valid ones, where a valid link to a dropped file answered 404 - * or 410. So a 403 means the signed link expired; on a valid link it is not an expiry. + * 404 or 410, or 403 when the link that actually failed carries an `oe` that has passed + * by the local clock. That link is the url on Baileys' error (Boom data.url), else the + * one Baileys downloads: the directPath when there is one, else the url. + * + * The 403 rule is a conservative heuristic, not a documented contract. Measured on + * WhatsApp's media CDN (2026-09-27, 84 history-sync attachments): 403 on 34 of 34 links + * whose `oe` had passed and on 0 of 50 valid ones, where a valid link to a dropped file + * answered 404 or 410. A 403 without that evidence is not treated as an expiry, as a + * policy; a local clock that is off moves the line. */ -const isExpiredMedia = (error: any, media: Parameters[0]) => { +const isExpiredMedia = (error: any, media: { url?: string | null; directPath?: string | null } | undefined) => { const status = httpStatus(error); if (status === 404 || status === 410) return true; if (status !== 403) return false; - const expiry = mediaLinkExpiry(media); - return expiry !== undefined && expiry * 1000 <= Date.now(); + const requested = error?.data?.url; + const link = + typeof requested === 'string' || requested instanceof URL ? requested.toString() : media?.directPath || media?.url; + const expiry = link ? mediaLinkExpiry(link) : undefined; + return expiry !== undefined && expiry <= Date.now(); }; /** @@ -4626,7 +4640,7 @@ export class BaileysStartupService extends ChannelStartupService { } }; const target: WAMessage = { key: msg?.key, message: msg?.message }; - // The link as first downloaded, whose `oe` says whether a 403 is an expired link. + // The links as first downloaded, for a 403 whose error does not name the one that failed. const link = { url: msg.message[mediaType]?.url, directPath: msg.message[mediaType]?.directPath }; // No reuploadRequest for Baileys: Evolution asks the phone itself, below. Baileys // means to ask on a 404 or 410, but 7.0.0-rc14 checks error.status From 9e4780c7d3ce690ceea5d5de0aef50314b3870e7 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 19:47:20 +0300 Subject: [PATCH 114/157] test: deleting an instance with its logout pending keeps the creds and proxy it needs (real Postgres) Instance is referenced ON DELETE CASCADE by Session and Proxy, so the pending delete that removes the Instance row also removes the credentials and the proxy the logout needs. The in-memory Prisma has no foreign keys and hid it: the logout tests accepted a state the database cannot hold (no Instance row, its Session and Proxy still there). - A new test runs the flow on a throwaway Postgres database built from prisma/postgresql-migrations (FORK_TEST_PG_URL, loopback only; CI gets a postgres service). - The fake Prisma now cascades an Instance delete as the schema does. - The two fake-Prisma delete tests expected the Instance row gone while pending; they now expect it kept, since deleting it is the bug. Red on the current code (5 tests): the creds and proxy are gone right after the 202, and after a restart the reconnect logs out a fresh session instead of the linked device (logouts 0). The restart test checks at its end that the name answers 404 rather than creating a real instance, which outlived the test and raced the next file's rows; and the reconnect setup waits for the open's webhook before dropping the socket, since a real database can land it after the close. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/fork-tests.yml | 24 ++ test/helpers/fake-prisma.ts | 23 ++ test/helpers/real-postgres.ts | 58 +++ test/instance/logout-pending-postgres.test.ts | 331 ++++++++++++++++++ test/instance/logout-pending.test.ts | 8 +- test/instance/unlink-socket-down.test.ts | 3 +- 6 files changed, 443 insertions(+), 4 deletions(-) create mode 100644 test/helpers/real-postgres.ts create mode 100644 test/instance/logout-pending-postgres.test.ts diff --git a/.github/workflows/fork-tests.yml b/.github/workflows/fork-tests.yml index e99183f39b..f4be16c016 100644 --- a/.github/workflows/fork-tests.yml +++ b/.github/workflows/fork-tests.yml @@ -19,6 +19,18 @@ jobs: test: name: typecheck and tests (pinned Baileys) runs-on: ubuntu-latest + # A few tests need what only a real database does (foreign keys, cascades): + # they create and drop their own database on this server (test/helpers/real-postgres.ts). + services: + postgres: + image: postgres:16 + env: + POSTGRES_PASSWORD: postgres + ports: ['5432:5432'] + options: >- + --health-cmd pg_isready --health-interval 5s --health-timeout 5s --health-retries 10 + env: + FORK_TEST_PG_URL: postgresql://postgres:postgres@127.0.0.1:5432/postgres steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 @@ -39,6 +51,18 @@ jobs: name: tests against the newest Baileys if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' runs-on: ubuntu-latest + # A few tests need what only a real database does (foreign keys, cascades): + # they create and drop their own database on this server (test/helpers/real-postgres.ts). + services: + postgres: + image: postgres:16 + env: + POSTGRES_PASSWORD: postgres + ports: ['5432:5432'] + options: >- + --health-cmd pg_isready --health-interval 5s --health-timeout 5s --health-retries 10 + env: + FORK_TEST_PG_URL: postgresql://postgres:postgres@127.0.0.1:5432/postgres steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 diff --git a/test/helpers/fake-prisma.ts b/test/helpers/fake-prisma.ts index 155adc3a7c..d6985963ae 100644 --- a/test/helpers/fake-prisma.ts +++ b/test/helpers/fake-prisma.ts @@ -87,6 +87,29 @@ export function fakePrisma() { typebot: table('typebot', () => undefined), websocket: table('websocket', (r) => r.instanceId), }; + // As the real schema: every table that belongs to an instance references Instance ON DELETE + // CASCADE (Session by sessionId), so deleting the row deletes them too. + const cascade = (removed: Row[]) => { + for (const instance of removed) { + for (const [name, t] of Object.entries(db) as [string, any][]) { + if (name === 'instance' || !Array.isArray(t?.rows)) continue; + const key = name === 'session' ? 'sessionId' : 'instanceId'; + for (const r of t.rows.filter((r: Row) => r[key] === instance.id)) t.rows.splice(t.rows.indexOf(r), 1); + } + } + }; + const { delete: del, deleteMany } = db.instance; + db.instance.delete = async (args: Row) => { + const removed = await del(args); + if (removed) cascade([removed]); + return removed; + }; + db.instance.deleteMany = async (args: Row = {}) => { + const removed = db.instance.rows.filter((r: Row) => matches(r, args.where)); + const result = await deleteMany(args); + cascade(removed); + return result; + }; db.$transaction = async (ops: any) => (typeof ops === 'function' ? ops(db) : Promise.all(ops)); db.$queryRaw = async () => []; db.$executeRaw = async () => 0; diff --git a/test/helpers/real-postgres.ts b/test/helpers/real-postgres.ts new file mode 100644 index 0000000000..2da949196f --- /dev/null +++ b/test/helpers/real-postgres.ts @@ -0,0 +1,58 @@ +// A throwaway Postgres database with Evolution's own migrations applied, for a +// test whose subject is what the database itself does (foreign keys, cascades, +// unique constraints), which the in-memory Prisma (fake-prisma.ts) cannot model. +// +// The server is FORK_TEST_PG_URL (default postgresql://127.0.0.1:5432/postgres) +// and must be on the loopback address. Each call creates one database, applies +// prisma/postgresql-migrations in order, and returns a real PrismaClient on it; +// drop() removes the database. +import { readdirSync, readFileSync } from 'node:fs'; +import { userInfo } from 'node:os'; +import { join } from 'node:path'; + +import { PrismaClient } from '@prisma/client'; +import pg from 'pg'; + +const MIGRATIONS = new URL('../../prisma/postgresql-migrations/', import.meta.url).pathname; + +export const pgServerUrl = () => process.env.FORK_TEST_PG_URL ?? 'postgresql://127.0.0.1:5432/postgres'; + +export async function throwawayDatabase(): Promise<{ url: string; prisma: PrismaClient; drop: () => Promise }> { + const server = new URL(pgServerUrl()); + if (!['127.0.0.1', 'localhost', '[::1]'].includes(server.hostname)) { + throw new Error(`FORK_TEST_PG_URL must be a loopback server, not ${server.hostname}`); + } + const name = `evo_fork_test_${process.pid}_${Date.now().toString(36)}_${Math.random().toString(36).slice(2, 8)}`; + const admin = new pg.Client({ connectionString: server.toString() }); + await admin.connect(); + await admin.query(`CREATE DATABASE "${name}"`); + await admin.end(); + + const dbUrl = new URL(server.toString()); + dbUrl.pathname = `/${name}`; + // libpq defaults the user to the OS user; Prisma's engine does not. + if (!dbUrl.username) dbUrl.username = userInfo().username; + const url = dbUrl.toString(); + + const db = new pg.Client({ connectionString: url }); + await db.connect(); + for (const dir of readdirSync(MIGRATIONS) + .filter((d) => /^\d/.test(d)) + .sort()) { + await db.query(readFileSync(join(MIGRATIONS, dir, 'migration.sql'), 'utf8')); + } + await db.end(); + + const prisma = new PrismaClient({ datasourceUrl: url }); + return { + url, + prisma, + drop: async () => { + await prisma.$disconnect(); + const c = new pg.Client({ connectionString: server.toString() }); + await c.connect(); + await c.query(`DROP DATABASE IF EXISTS "${name}" WITH (FORCE)`); + await c.end(); + }, + }; +} diff --git a/test/instance/logout-pending-postgres.test.ts b/test/instance/logout-pending-postgres.test.ts new file mode 100644 index 0000000000..8a033f8525 --- /dev/null +++ b/test/instance/logout-pending-postgres.test.ts @@ -0,0 +1,331 @@ +// The pending logout of a deleted instance, against a real Postgres with +// Evolution's own migrations (test/helpers/real-postgres.ts). +// +// Every table that belongs to an instance references Instance ON DELETE +// CASCADE, Session and Proxy included. So deleting the Instance row deletes the +// credentials and the proxy a pending logout still needs to reach WhatsApp: +// after that, the reconnect cannot authenticate as the old device, and the +// device stays on the phone's Linked devices. The in-memory Prisma +// (fake-prisma.ts) has no foreign keys and cannot show this. +// +// A deleted instance whose logout is pending keeps its Instance row until the +// logout has reached WhatsApp, and stays out of the API meanwhile. +import { vi } from 'vitest'; + +const { socketSpy, h } = vi.hoisted(() => ({ + socketSpy: vi.fn(), + h: { + waMonitor: undefined as any, + instanceController: undefined as any, + channelController: undefined as any, + prisma: undefined as any, + }, +})); +const tmp = await vi.hoisted(async () => { + const { mkdtempSync } = await import('node:fs'); + const { tmpdir } = await import('node:os'); + const { join } = await import('node:path'); + return mkdtempSync(join(tmpdir(), 'evo-logout-pg-')); +}); + +vi.mock('@api/server.module', async () => { + const fake = await import('../helpers/fake-server-module'); + return { + ...fake, + get prismaRepository() { + return h.prisma; + }, + get waMonitor() { + return h.waMonitor; + }, + get instanceController() { + return h.instanceController; + }, + get channelController() { + return h.channelController; + }, + }; +}); +vi.mock('@config/path.config', async (importOriginal) => ({ ...(await importOriginal()), INSTANCE_DIR: tmp })); +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + return { ...orig, default: socketSpy, makeWASocket: socketSpy }; +}); + +import { existsSync, rmSync } from 'node:fs'; +import { join } from 'node:path'; + +import { Boom } from '@hapi/boom'; +import EventEmitter2 from 'eventemitter2'; +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from 'vitest'; + +import { settle, WUID } from '../helpers/baileys-service'; +import { fakeSocket } from '../helpers/connect'; +import { emitted } from '../helpers/fake-server-module'; +import { startInstanceApp } from '../helpers/http-app'; +import { loopbackOnly } from '../helpers/local-net'; +import { throwawayDatabase } from '../helpers/real-postgres'; + +socketSpy.mockImplementation(fakeSocket); + +const TOKEN = 'instance-token'; +const ID = 'inst-pg-1'; +const DIR = join(tmp, ID); + +type Sock = ReturnType; +const built = (): Sock[] => socketSpy.mock.results.map((r) => r.value); +const current = () => built()[built().length - 1]; + +let guard: ReturnType; +let app: Awaited>; +let db: Awaited>; +let globalKey: string; +const processes: any[] = []; + +beforeAll(async () => { + guard = loopbackOnly(); + db = await throwawayDatabase(); + h.prisma = db.prisma; + app = await startInstanceApp(); + const { configService } = await import('@config/env.config'); + globalKey = configService.get('AUTHENTICATION').API_KEY.KEY; +}, 60_000); +beforeEach(() => { + socketSpy.mockClear(); + emitted.length = 0; +}); +afterEach(async () => { + for (const monitor of processes) { + for (const s of [ + ...Object.values(monitor.waInstances), + ...Object.values(monitor.finishingLogouts ?? {}), + ] as any[]) { + s.stopReconnecting?.(); + s.connectToWhatsapp = async () => undefined; + s.connect = async () => undefined; + } + } + processes.length = 0; + await h.prisma.instance.deleteMany({}); + rmSync(DIR, { recursive: true, force: true }); +}); +afterAll(async () => { + await app?.close(); + await db?.drop(); + rmSync(tmp, { recursive: true, force: true }); + expect(guard.refused).toEqual([]); + guard.restore(); +}); + +/** What a process start builds (main.ts, server.module): the monitor, the controllers, then the boot load. */ +async function startProcess() { + const { ConfigService } = await import('@config/env.config'); + const { CacheService } = await import('@api/services/cache.service'); + const { LocalCache } = await import('@cache/localcache'); + const { WAMonitoringService } = await import('@api/services/monitor.service'); + const { InstanceController } = await import('@api/controllers/instance.controller'); + const { ChannelController } = await import('@api/integrations/channel/channel.controller'); + const configService = new ConfigService(); + const cache = new CacheService(new LocalCache(configService, 'instance')); + const emitter = new EventEmitter2(); + const prisma = h.prisma; + const waMonitor = new WAMonitoringService(emitter, configService, prisma, null as any, cache, cache, cache); + const n = null as any; + h.waMonitor = waMonitor; + h.channelController = new ChannelController(prisma, waMonitor); + h.instanceController = new InstanceController( + waMonitor, + configService, + prisma, + emitter, + n, + n, + n, + cache, + cache, + cache, + n, + ); + processes.push(waMonitor); + await waMonitor.loadInstance(); + return waMonitor; +} + +/** A linked instance, stored the way Evolution stores one: its row, its creds, a signal key file, a proxy. */ +async function linkedInstance() { + const { configService } = await import('@config/env.config'); + await h.prisma.instance.create({ + data: { + id: ID, + name: 'test', + connectionStatus: 'open', + token: TOKEN, + integration: 'WHATSAPP-BAILEYS', + clientName: configService.get('DATABASE').CONNECTION.CLIENT_NAME, + }, + }); + await h.prisma.proxy.create({ + data: { + instanceId: ID, + enabled: false, + host: '127.0.0.1', + port: '1', + protocol: 'http', + username: '', + password: '', + }, + }); + await h.prisma.setting.create({ + data: { + instanceId: ID, + rejectCall: false, + msgCall: '', + readMessages: false, + groupsIgnore: false, + alwaysOnline: false, + readStatus: false, + syncFullHistory: false, + }, + }); + const { default: useMultiFileAuthStatePrisma } = await import('@utils/use-multi-file-auth-state-prisma'); + const auth = await useMultiFileAuthStatePrisma(ID, null as any); + auth.state.creds.me = { id: WUID, name: 'Dana' }; + await auth.saveCreds(); + await auth.state.keys.set({ 'pre-key': { '1': { public: Buffer.alloc(32, 1), private: Buffer.alloc(32, 2) } } }); +} + +const storedMe = async () => + (await h.prisma.session.findMany({ where: { sessionId: ID } })).map((r: any) => { + let creds: any = r.creds; + while (typeof creds === 'string') creds = JSON.parse(creds); + return creds?.me?.id; + }); + +const rows = async () => ({ + instances: await h.prisma.instance.count(), + sessions: await h.prisma.session.count(), + proxies: await h.prisma.proxy.count(), + settings: await h.prisma.setting.count(), +}); + +/** Booted, connected, then dropped (a dead exit): the socket closed and the first reconnect (1s) is waiting. */ +async function waitingToReconnect() { + await linkedInstance(); + const monitor = await startProcess(); + await vi.waitFor(() => expect(built()).toHaveLength(1)); + const service = monitor.waInstances.test; + await opened(current()); + // The open's webhook goes out after its database writes, which a real database can make later + // than the close below: wait for it, so it is not read as sent while the logout is pending. + await vi.waitFor(() => + expect(emitted.some((e) => e.event === 'connection.update' && e.data?.state === 'open')).toBe(true), + ); + current().end(new Boom('Connection Terminated', { statusCode: 428 })); + await vi.waitFor(() => expect(service.connectionStatus.state).toBe('close'), { interval: 5 }); + emitted.length = 0; + return { monitor, service }; +} + +async function opened(sock: Sock) { + sock.ev.emit('connection.update', { connection: 'open' }); + await new Promise((r) => setTimeout(r, 20)); +} + +async function call(method: 'GET' | 'DELETE', route: string, key = TOKEN) { + const res = await fetch(`${app.base}/instance/${route}/test`, { method, headers: { apikey: key } }); + return { status: res.status, body: await res.json() }; +} + +async function fetchInstances(key = globalKey) { + const res = await fetch(`${app.base}/instance/fetchInstances`, { headers: { apikey: key } }); + const body = await res.json(); + return { status: res.status, names: Array.isArray(body) ? body.map((i: any) => i.name) : body }; +} + +async function createNamed(name: string) { + const res = await fetch(`${app.base}/instance/create`, { + method: 'POST', + headers: { apikey: globalKey, 'content-type': 'application/json' }, + body: JSON.stringify({ instanceName: name, integration: 'WHATSAPP-BAILEYS' }), + }); + return res.status; +} + +async function reconnected(count: number) { + await vi.waitFor(() => expect(built()).toHaveLength(count), { timeout: 3_000 }); + return current(); +} + +/** The API's view of a deleted instance whose logout is pending: only connectionState, and a taken name. */ +async function hiddenFromTheApi() { + expect({ + state: await call('GET', 'connectionState', globalKey), + other: (await call('GET', 'connect', globalKey)).status, + ownKey: (await call('GET', 'connectionState')).status, + listed: await fetchInstances(), + byOwnKey: (await fetchInstances(TOKEN)).status, + create: await createNamed('test'), + }).toEqual({ + state: { status: 200, body: { instance: { instanceName: 'test', state: 'close', logoutPending: true } } }, + other: 404, + ownKey: 401, + listed: { status: 200, names: [] }, + byOwnKey: 401, + create: 403, + }); +} + +describe('deleting an instance whose logout cannot reach WhatsApp (real Postgres)', () => { + it('keeps the credentials and the proxy the logout needs, hidden from the API, until it is delivered', async () => { + const { service } = await waitingToReconnect(); + + expect((await call('DELETE', 'delete')).status).toBe(202); + expect({ me: await storedMe(), proxies: (await rows()).proxies, settings: (await rows()).settings }).toEqual({ + me: [WUID], + proxies: 1, + settings: 0, + }); + await hiddenFromTheApi(); + + // The connection returns: the logout goes out as the linked device, then everything goes. + const sock = await reconnected(2); + await opened(sock); + await settle(service); + expect({ logouts: sock.logouts, rows: await rows(), dir: existsSync(DIR) }).toEqual({ + logouts: 1, + rows: { instances: 0, sessions: 0, proxies: 0, settings: 0 }, + dir: false, + }); + expect((await call('GET', 'connectionState', globalKey)).status).toBe(404); + expect(emitted.map((e) => e.event)).toEqual([]); + }); + + it('survives a restart mid-pending: the next process logs out the linked device, then removes it', async () => { + const { service } = await waitingToReconnect(); + expect((await call('DELETE', 'delete')).status).toBe(202); + // The process dies before the connection returns. + service.stopReconnecting(); + service.connect = async () => undefined; + socketSpy.mockClear(); + + const monitor = await startProcess(); + const sock = await reconnected(1); + expect(monitor.waInstances.test).toBeUndefined(); + await hiddenFromTheApi(); + + // The reconnect authenticates as the linked device, so the logout it sends is the real one. + expect(sock.logouts).toBe(0); + await opened(sock); + await settle(monitor.finishingLogouts.test ?? service); + expect({ logouts: sock.logouts, rows: await rows(), dir: existsSync(DIR) }).toEqual({ + logouts: 1, + rows: { instances: 0, sessions: 0, proxies: 0, settings: 0 }, + dir: false, + }); + expect(monitor.finishingLogouts.test).toBeUndefined(); + expect((await call('GET', 'connectionState', globalKey)).status).toBe(404); + }); +}); diff --git a/test/instance/logout-pending.test.ts b/test/instance/logout-pending.test.ts index 61b6434965..86650b1a53 100644 --- a/test/instance/logout-pending.test.ts +++ b/test/instance/logout-pending.test.ts @@ -358,7 +358,8 @@ describe('a logout that cannot reach WhatsApp', () => { response: { message: 'Instance deleted; its logout will reach WhatsApp when the connection returns' }, }, }); - // Gone from the API: no instance, no row, no settings. Kept: the creds, the key files and marker, the proxy. + // Gone from the API: no instance, no settings. Kept: the row (deleting it would cascade to the + // session and the proxy), the creds, the key files and marker, the proxy. expect(monitor.waInstances.test).toBeUndefined(); expect({ instances: prisma.instance.rows.length, @@ -367,7 +368,7 @@ describe('a logout that cannot reach WhatsApp', () => { me: storedMe(), marker: JSON.parse(readFileSync(MARKER, 'utf8')), }).toMatchObject({ - instances: 0, + instances: 1, settings: 0, proxies: 1, me: [WUID], @@ -397,7 +398,8 @@ describe('a logout that cannot reach WhatsApp', () => { me: storedMe(), dir: existsSync(DIR), proxies: prisma.proxy.rows.length, - }).toEqual({ logouts: 1, emitted: [], me: [], dir: false, proxies: 0 }); + instances: prisma.instance.rows.length, + }).toEqual({ logouts: 1, emitted: [], me: [], dir: false, proxies: 0, instances: 0 }); // Finished: the name is gone from the API. expect((await call('GET', 'connectionState', globalKey)).status).toBe(404); }); diff --git a/test/instance/unlink-socket-down.test.ts b/test/instance/unlink-socket-down.test.ts index d2efeba4dd..f2343801b2 100644 --- a/test/instance/unlink-socket-down.test.ts +++ b/test/instance/unlink-socket-down.test.ts @@ -209,7 +209,8 @@ describe('unlinking an instance whose connection is down', () => { await settle(service); await expectPending(); expect(waMonitor.waInstances.test).toBeUndefined(); - expect(prisma.instance.rows).toEqual([]); + // Its row stays until the logout is delivered: deleting it would cascade to the session. + expect(prisma.instance.rows.map((r: any) => r.name)).toEqual(['test']); }); it('delete removes the instance even when its logout fails for another reason', async () => { From 136690d8a273707ba34fe8024b5dc699ecb5c68d Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 19:49:49 +0300 Subject: [PATCH 115/157] fix: a deleted instance keeps its row until its pending logout is delivered The pending delete removed the Instance row, and Postgres cascaded that to the Session (the creds) and the Proxy the logout needs. Now the row stays until the logout has reached WhatsApp, and goes only then (finishLogout deletes it, before the marker, so a failure is finished on the next try). No schema change. While it waits the instance stays out of the API as before: - fetchInstances does not list it; the guards treat its name as absent (except connectionState), and the name stays taken for /instance/create. - Its token is cleared, so its key authenticates nothing (as when the row was gone). - On boot a row with a deleted marker goes to finishingLogouts, not waInstances. resumeDeletedLogouts still finishes a marker with no row, left by a delete made before this change (new guard test; that session's creds are already gone, so it can only finish on a QR). Visible to consumers: a pending-deleted instance's row is still in the database (token null, connectionStatus close) until the logout finishes. The HTTP API answers as before. Co-Authored-By: Claude Opus 5.5 --- FORK.md | 2 +- src/api/guards/instance.guard.ts | 2 + .../whatsapp/whatsapp.baileys.service.ts | 9 ++++- src/api/services/monitor.service.ts | 37 ++++++++++++++----- test/helpers/fake-prisma.ts | 1 + test/instance/logout-pending.test.ts | 23 ++++++++++++ 6 files changed, 62 insertions(+), 12 deletions(-) diff --git a/FORK.md b/FORK.md index f9f7f92f86..5a2d6d457c 100644 --- a/FORK.md +++ b/FORK.md @@ -63,7 +63,7 @@ named where one exists. **Sessions and connections** - A database error never replaces a linked session's credentials with fresh ones, and a failed settings read no longer drops an event batch. -- A logout or delete while the socket is down keeps the session until WhatsApp is told, so the device leaves the phone's Linked devices; it answers `202 PENDING` meanwhile and forwards nothing (#2520, #2508 in part). +- A logout or delete while the socket is down keeps the session until WhatsApp is told, so the device leaves the phone's Linked devices; it answers `202 PENDING` meanwhile and forwards nothing (#2520, #2508 in part). A deleted instance keeps its database row until then (Session and Proxy cascade from it), out of the API, its name taken and its token cleared. - Reconnects back off from 1s to 60s instead of spinning, the version fetch times out after 10s, and an instance never runs two sockets (#2134, #2184, #2430; ideas from #2732). - Profile pictures are looked up once per contact per hour, at most four at a time, and history never waits on them (#1883). - One pairing code per connect attempt, and a fresh QR budget per attempt (#2100, #2696). diff --git a/src/api/guards/instance.guard.ts b/src/api/guards/instance.guard.ts index 1460ffe2b5..bfa87ac33c 100644 --- a/src/api/guards/instance.guard.ts +++ b/src/api/guards/instance.guard.ts @@ -5,6 +5,8 @@ import { BadRequestException, ForbiddenException, InternalServerErrorException, import { NextFunction, Request, Response } from 'express'; async function getInstance(instanceName: string) { + // A deleted instance keeps its row while its logout is pending, but is gone from the API. + if (waMonitor.finishingLogouts?.[instanceName]) return false; try { const cacheConf = configService.get('CACHE'); diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 6c9d12bb65..43b34dce90 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -570,6 +570,12 @@ export class BaileysStartupService extends ChannelStartupService { this.stopReconnecting(); try { await this.removeSession(); + if (logout.deleted) { + // Out of the API already: remove what was kept for the logout, the row last (it cascades to + // the rest). Before the marker goes, so a failure here is finished again on the next try. + await this.prismaRepository.proxy.deleteMany({ where: { instanceId: this.instanceId } }); + await this.prismaRepository.instance.deleteMany({ where: { id: this.instanceId } }); + } } catch (error) { // The device is off WhatsApp; the next connection answers loggedOut, which finishes it again. this.logger.error({ message: 'Could not wipe the session after the logout', error: error?.toString() }); @@ -581,8 +587,7 @@ export class BaileysStartupService extends ChannelStartupService { this.logout = null; if (logout.deleted) { - // Out of the API already: remove what was kept for the logout, and announce nothing. - await this.prismaRepository.proxy.deleteMany({ where: { instanceId: this.instanceId } }).catch(() => undefined); + // Announce nothing. this.shutdown(); this.stateConnection = { state: 'close', statusReason: DisconnectReason.loggedOut }; this.eventEmitter.emit('logout.finished', this); diff --git a/src/api/services/monitor.service.ts b/src/api/services/monitor.service.ts index 36b0ebe627..7998bb43f5 100644 --- a/src/api/services/monitor.service.ts +++ b/src/api/services/monitor.service.ts @@ -104,6 +104,8 @@ export class WAMonitoringService { const clientName = this.configService.get('DATABASE').CONNECTION.CLIENT_NAME; + // A deleted instance finishing its logout keeps its row until then, out of the API. + const finishing = Object.keys(this.finishingLogouts); const where = instanceNames && instanceNames.length > 0 ? { @@ -112,7 +114,7 @@ export class WAMonitoringService { }, clientName, } - : { clientName }; + : { clientName, ...(finishing.length ? { name: { notIn: finishing } } : {}) }; const instances = await this.prismaRepository.instance.findMany({ where, @@ -196,7 +198,12 @@ export class WAMonitoringService { } } - /** `keepForLogout`: keep the session (creds, key files, logout marker) and the proxy a pending logout still needs. */ + /** + * `keepForLogout`: keep what a pending logout still needs: the session (creds, key files, logout + * marker), the proxy, and the Instance row itself, since Session and Proxy reference it ON DELETE + * CASCADE. Its token is cleared, so the deleted instance's key authenticates nothing. The row goes + * when the logout has reached WhatsApp (BaileysStartupService.finishLogout). + */ public async cleaningStoreData(instanceName: string, { keepForLogout = false } = {}) { if (this.configService.get('CHATWOOT').ENABLED) { const instancePath = join(STORE_DIR, 'chatwoot', instanceName); @@ -232,6 +239,10 @@ export class WAMonitoringService { await this.prismaRepository.setting.deleteMany({ where: { instanceId: instance.id } }); await this.prismaRepository.label.deleteMany({ where: { instanceId: instance.id } }); + if (keepForLogout) { + await this.prismaRepository.instance.update({ where: { id: instance.id }, data: { token: null } }); + return; + } await this.prismaRepository.instance.delete({ where: { name: instanceName } }); } @@ -251,9 +262,9 @@ export class WAMonitoringService { } /** - * Delete while the logout could not reach WhatsApp: the instance leaves the API now (memory, its - * row and everything else stored for it), keeping only its session and proxy, and finishes the - * logout in the background. The service removes those when it has (BaileysStartupService.finishLogout). + * Delete while the logout could not reach WhatsApp: the instance leaves the API now (memory and + * everything stored for it), keeping only its row, session and proxy, and finishes the logout in + * the background. The service removes those when it has (BaileysStartupService.finishLogout). */ public async deleteKeepingLogout(instanceName: string) { const instance = this.waInstances[instanceName]; @@ -265,7 +276,11 @@ export class WAMonitoringService { this.logger.warn(`Instance "${instanceName}" - REMOVED, its logout pending`); } - /** On boot: a deleted instance (no row) whose logout marker is still there finishes its logout. */ + /** + * On boot: a deleted instance with no row whose logout marker is still there finishes its logout. + * Only a delete made before the row was kept leaves that (its session went with the row); a row + * that is still there is loaded by setInstance. + */ private async resumeDeletedLogouts() { let ids: string[]; try { @@ -348,9 +363,13 @@ export class WAMonitoringService { ownerJid: instanceData.ownerJid, }); - // A logout that had not reached WhatsApp before the restart: connect only to deliver it. - if (await (instance as any).resumePendingLogout?.()) { - this.waInstances[instanceData.instanceName] = instance; + // A logout that had not reached WhatsApp before the restart: connect only to deliver it. A + // deleted instance's row is kept until then, and it stays out of the API (finishingLogouts). + const marker = readLogoutMarker(instanceData.instanceId); + if (marker && (instance as any).resumePendingLogout) { + if (marker.deleted) this.finishingLogouts[instanceData.instanceName] = instance; + else this.waInstances[instanceData.instanceName] = instance; + await (instance as any).resumePendingLogout(); return; } diff --git a/test/helpers/fake-prisma.ts b/test/helpers/fake-prisma.ts index d6985963ae..8ce83a700f 100644 --- a/test/helpers/fake-prisma.ts +++ b/test/helpers/fake-prisma.ts @@ -7,6 +7,7 @@ const matches = (row: Row, where: Row = {}) => if (k === 'remoteJid_instanceId') return row.remoteJid === v.remoteJid && row.instanceId === v.instanceId; if (v && typeof v === 'object' && !Array.isArray(v)) { if ('in' in v) return (v.in as any[]).includes(row[k]); + if ('notIn' in v) return !(v.notIn as any[]).includes(row[k]); if ('path' in v) return true; // JSON path filters: not modelled, treated as match return matches(row[k] ?? {}, v); } diff --git a/test/instance/logout-pending.test.ts b/test/instance/logout-pending.test.ts index 86650b1a53..5512a505ab 100644 --- a/test/instance/logout-pending.test.ts +++ b/test/instance/logout-pending.test.ts @@ -426,6 +426,29 @@ describe('a logout that cannot reach WhatsApp', () => { }); }); + // A delete made before the row was kept for the logout left a marker and no row. The boot still + // finishes it (resumeDeletedLogouts): out of the API, then the marker and key files go. + it('a deleted marker with no row (a delete from before the row was kept) is still finished on boot', async () => { + const { service } = await waitingToReconnect(); + expect((await call('DELETE', 'delete')).status).toBe(202); + service.stopReconnecting(); + service.connect = async () => undefined; + socketSpy.mockClear(); + prisma.instance.rows.length = 0; + + const monitor = await startProcess(); + const sock = await reconnected(1); + expect(monitor.waInstances.test).toBeUndefined(); + const finishing = monitor.finishingLogouts.test; + expect(finishing).toBeDefined(); + await opened(sock); + await settle(finishing); + expect({ dir: existsSync(DIR), finishing: monitor.finishingLogouts.test }).toEqual({ + dir: false, + finishing: undefined, + }); + }); + it('a logout on an open connection still completes at once', async () => { await linkedInstance(); const monitor = await startProcess(); From 837ab361b2ab433a7800c7212337752ea5ced7c5 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 19:55:31 +0300 Subject: [PATCH 116/157] test: the scrubber, its leak gate and the fixture guard let a username, a stub text, an unknown field and a location through The scrubber kept any identifier-shaped string as structure whatever its field, and every decimal number; its leak gate searched only for what the scrubber itself had replaced (and skipped a value it had also kept once). The fixture guard knew only the fake ranges of addresses and numbers. Synthetic values, planted in a recorded session and in a fixture: - remoteJidUsername 'dana.levi88' (also in a message text) - messageStubParameters ['dana_and_friends'] - a field no list knows, someFutureField 'dana.levi' - locationMessage degreesLatitude/degreesLongitude Red on the current code (5 tests): each survives the scrub with the gate passing, the unknown field is written instead of stopping the scrub, and the guard reports none of the four lines. Co-Authored-By: Claude Opus 5.5 --- test/live/fixture-guard.test.ts | 33 ++++++++++++ test/live/scrub.test.ts | 92 +++++++++++++++++++++++++++++++-- 2 files changed, 122 insertions(+), 3 deletions(-) diff --git a/test/live/fixture-guard.test.ts b/test/live/fixture-guard.test.ts index df7b5d2d41..69e85be25f 100644 --- a/test/live/fixture-guard.test.ts +++ b/test/live/fixture-guard.test.ts @@ -162,4 +162,37 @@ describe('live fixture guard', () => { const findings = scanFixtures(join(process.cwd(), 'test', 'fixtures', 'live')); expect(formatFindings(findings)).toBe(''); }); + + // The guard knew the fake ranges of addresses and numbers only: a username, a group name in a stub + // parameter, a value under a field it did not know, or a location passed as clean. + it('flags a value that is neither a scrubber fake nor a known structural value, and never prints it', () => { + const dir = join(root, 'live', '2026-09-27-unknown'); + mkdirSync(dir, { recursive: true }); + const values = { username: 'dana.levi88', group: 'dana_and_friends', unknown: 'dana.levi' }; + const key = { remoteJid: '100000000000001@lid', fromMe: false, id: '3AFFFFFFFFFFFFFFFFF1' }; + const lines = [ + { + seq: 1, + event: 'messages.upsert', + data: { messages: [{ key: { ...key, remoteJidUsername: values.username } }] }, + }, + { seq: 2, event: 'messages.upsert', data: { messages: [{ key, messageStubParameters: [values.group] }] } }, + { seq: 3, event: 'messages.upsert', data: { type: 'notify', someFutureField: values.unknown } }, + { + seq: 4, + event: 'messages.upsert', + data: { + messages: [{ key, message: { locationMessage: { degreesLatitude: 32.0853, degreesLongitude: 34.7818 } } }], + }, + }, + ]; + writeFileSync(join(dir, 'events.ndjson'), lines.map((l) => JSON.stringify(l)).join('\n') + '\n'); + + const findings = scanFixtures(join(root, 'live')); + expect([...new Set(findings.map((f) => f.line))].sort()).toEqual([1, 2, 3, 4]); + const report = formatFindings(findings); + for (const value of [...Object.values(values), '32.0853', '34.7818']) { + expect(report.includes(value), `the report printed a planted value of ${value.length} chars`).toBe(false); + } + }); }); diff --git a/test/live/scrub.test.ts b/test/live/scrub.test.ts index ee450a0da2..235f6104a8 100644 --- a/test/live/scrub.test.ts +++ b/test/live/scrub.test.ts @@ -64,7 +64,8 @@ describe('live-check scrubber', () => { const text = readdirSync(dir) .map((f) => readFileSync(join(dir, f), 'utf8')) .join('\n'); - for (const original of ORIGINALS) expect(text.includes(original), `an original of ${original.length} chars`).toBe(false); + for (const original of ORIGINALS) + expect(text.includes(original), `an original of ${original.length} chars`).toBe(false); }); it('gives one person one fake across phone JID, @lid and device suffix, in both tapes', () => { @@ -128,7 +129,12 @@ describe('live-check scrubber', () => { it('adds what the operator records, and a report of counts only', () => { const { dir } = scrub({ - operator: { phoneModel: 'Pixel 8', osVersion: 'Android 15', whatsappAppVersion: '2.25.27.78', countryCode: '972' }, + operator: { + phoneModel: 'Pixel 8', + osVersion: 'Android 15', + whatsappAppVersion: '2.25.27.78', + countryCode: '972', + }, }); const manifest = JSON.parse(readFileSync(join(dir, 'manifest.json'), 'utf8')); expect(manifest).toMatchObject({ @@ -158,7 +164,10 @@ describe('live-check scrubber', () => { socket: 1, event: 'messages.upsert', buffered: false, - data: { type: 'append', messages: [{ key: { remoteJid: '120363401234567890@g.us', fromMe: false, id }, messageStubType: 20 }] }, + data: { + type: 'append', + messages: [{ key: { remoteJid: '120363401234567890@g.us', fromMe: false, id }, messageStubType: 20 }], + }, }; appendFileSync(join(raw, 'events.ndjson'), JSON.stringify(stub) + '\n'); const { dir, report } = scrub(); @@ -193,4 +202,81 @@ describe('live-check scrubber', () => { for (const original of ORIGINALS) expect(message.includes(original)).toBe(false); expect(existsSync(out)).toBe(false); }); + + // A string was kept as written whenever it looked like an identifier, whatever its field, and a + // decimal number always was: a username, a group name in a stub parameter, a value in a field the + // scrubber had never seen, a location. The leak gate searched only for what the scrubber replaced. + describe('what it cannot tell from structure', () => { + const plant = (data: any) => + appendFileSync( + join(raw, 'events.ndjson'), + JSON.stringify({ seq: 999, t: 1, socket: 1, event: 'messages.upsert', buffered: false, data }) + '\n', + ); + const fixtureText = (dir: string) => + readdirSync(dir) + .map((f) => readFileSync(join(dir, f), 'utf8')) + .join('\n'); + + it('replaces a username, wherever the same value appears', () => { + const username = 'dana.levi88'; + plant({ + type: 'notify', + messages: [ + { + key: { remoteJid: PERSON.lid, remoteJidUsername: username, fromMe: false, id: MESSAGE_ID }, + message: { conversation: `my handle is ${username}` }, + }, + ], + }); + const { dir } = scrub(); + expect(fixtureText(dir).includes(username)).toBe(false); + }); + + it('replaces the text of a stub parameter', () => { + const groupName = 'dana_and_friends'; + plant({ + type: 'append', + messages: [ + { + key: { remoteJid: '120363401234567890@g.us', fromMe: false, id: '8347261905' }, + messageStubType: 21, + messageStubParameters: [groupName], + }, + ], + }); + const { dir } = scrub(); + expect(fixtureText(dir).includes(groupName)).toBe(false); + }); + + it('replaces a location', () => { + plant({ + type: 'notify', + messages: [ + { + key: { remoteJid: PERSON.lid, fromMe: false, id: MESSAGE_ID }, + message: { locationMessage: { degreesLatitude: 32.0853, degreesLongitude: 34.7818 } }, + }, + ], + }); + const { dir } = scrub(); + const text = fixtureText(dir); + expect({ latitude: text.includes('32.0853'), longitude: text.includes('34.7818') }).toEqual({ + latitude: false, + longitude: false, + }); + }); + + it('stops on a field it does not know, names its path, and writes nothing', () => { + plant({ type: 'notify', messages: [], someFutureField: 'dana.levi' }); + let message = ''; + try { + scrub(); + } catch (error) { + message = String(error?.message); + } + expect(message).toMatch(/unknown field .*someFutureField/); + expect(message.includes('dana.levi')).toBe(false); + expect(existsSync(out)).toBe(false); + }); + }); }); From f63ff5c52eaac4b167f178a420e78bb8590dbc2d Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 19:59:30 +0300 Subject: [PATCH 117/157] fix: the scrubber keeps a string only by field and known value, stops on an unknown field, and its gate reads the raw tapes itself Scrubber (test/tools/live-scrub.ts): - A string is kept as written only under a field test/tools/live-fields.ts lists, with a value that field is known to take (event names, Baileys' and Evolution's enums). Being identifier-shaped, or 3 characters or fewer, no longer keeps anything. - A string under a field no list knows stops the scrub, writes nothing, and names the field's path (never the value). - Usernames (username, *Username) become fake names; stub parameters, instanceId, label and directPath are text (lorem), as the committed fixture already has them. - A location, and any decimal but the tape's clock (t), becomes 0.; an integer or $long of 7+ digits that is neither an epoch nor a size or time by its field becomes the person's fake digits. Leak gate (leakGate): no longer searches only for what the scrubber replaced. It reads the raw tapes itself, takes every string of 4+ characters that is not structure, every address user part, every non-epoch digit run and number of 7+ digits, every decimal but t and every byte string, and looks for each in every value and key of the output. Fixture guard: in a tape it accepts only the scrubber's fakes and the listed structural values, and flags any other string and any decimal. The committed fixture 2026-09-27-rig-session still passes the guard unchanged, and a new test scrubs it again as if raw to prove every field in it is known (it found directPath). New direct gate test: a value the scrubber had kept still fails the gate. docs/LIVE-CHECKS.md updated. Co-Authored-By: Claude Opus 5.5 --- docs/LIVE-CHECKS.md | 32 +++-- test/live/scrub.test.ts | 52 +++++++- test/tools/fixture-guard.ts | 72 +++++++++-- test/tools/live-fields.ts | 91 ++++++++++++++ test/tools/live-scrub.ts | 235 +++++++++++++++++++++++++----------- 5 files changed, 394 insertions(+), 88 deletions(-) create mode 100644 test/tools/live-fields.ts diff --git a/docs/LIVE-CHECKS.md b/docs/LIVE-CHECKS.md index f4bde09a61..53db8e21e3 100644 --- a/docs/LIVE-CHECKS.md +++ b/docs/LIVE-CHECKS.md @@ -26,7 +26,8 @@ on. This file is the protocol, the catalogue of checks, and the log of results. creds and the QR payload are redacted when recorded. 2. **Scrub.** `scripts/live-scrub.ts` turns a raw session into `test/fixtures/live/-/`, then runs a leak gate that - searches the output for every original and writes nothing if one survives. + takes every value of the raw tapes that is not structure and searches the + output for it, and writes nothing if one survives. 3. **Replay.** `test/helpers/live-replay.ts` feeds the fixture's events through Baileys' real event buffer (buffered where they were) into the real `BaileysStartupService`, and `compareGolden` compares what Evolution sends @@ -91,17 +92,30 @@ npx tsx scripts/live-guard.ts ``` The scrubber prints the fixture directory and a report of counts. It exits 1 -and writes nothing when its leak gate finds an original in the output: fix the -scrubber (a new field it does not know, most often), never the fixture. Delete -the raw session once the fixture is committed. +and writes nothing when it meets a string under a field it does not know (the +error names the field's path, never the value), or when its leak gate finds an +original in the output: say what the field is in `test/tools/live-scrub.ts` or +`test/tools/live-fields.ts`, never in the fixture, and scrub again. Delete the +raw session once the fixture is committed. + +The leak gate does not ask the scrubber what it replaced. It reads the raw tapes +itself and searches every value and key of the output for each string of 4+ +characters that is not structure, the user part of every address, every run of +7+ digits that is not an epoch, every decimal but the tape's clock, and every +byte string. What the scrubber does: one person keeps one fake index across their phone JID, @lid and device suffix (the owner is index 0, `972500000000`); names keep equality (a saved name and a profile name stay different, the same name stays the same); message ids keep their first two characters and length (Evolution reads the device from them); bytes keep their length and type, with random -content, so a replay test must never depend on real crypto; text becomes lorem -of the same length; URLs become `https://example.invalid/`. +content, so a replay test must never depend on real crypto; text (a stub +parameter included) becomes lorem of the same length; a username becomes a fake +name; URLs become `https://example.invalid/`; a location, and any decimal but +the tape's clock, becomes `0.`. A string is kept as written only under a +field `test/tools/live-fields.ts` lists, and only with a value that field is +known to take (event names, Baileys' and Evolution's enums), never because it +looks like an identifier. **Add the fixture and its replay test.** Put the test next to the behaviour it covers, and assert the exact thing the check is about, then the whole output: @@ -153,8 +167,10 @@ touches `test/fixtures/live/`: `test/live/fixture-guard.test.ts` runs the same guard over `test/fixtures/live/` in CI on every commit, and lint-staged runs it on staged -fixture files. The guard knows only what personal data looks like: it cannot -recognise a name or a text, which is why the skim is not optional. +fixture files. In a tape it accepts only the scrubber's fakes and the values +`live-fields.ts` lists by field, and flags any other string and any decimal +number. It still cannot tell a name the scrubber mistook for structure from +structure, which is why the skim is not optional. ## Check catalogue diff --git a/test/live/scrub.test.ts b/test/live/scrub.test.ts index 235f6104a8..918a0db43b 100644 --- a/test/live/scrub.test.ts +++ b/test/live/scrub.test.ts @@ -17,7 +17,16 @@ vi.mock('@utils/fetchLatestWaWebVersion', () => ({ fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), })); -import { appendFileSync, existsSync, mkdtempSync, readdirSync, readFileSync, rmSync } from 'node:fs'; +import { + appendFileSync, + existsSync, + mkdirSync, + mkdtempSync, + readdirSync, + readFileSync, + rmSync, + writeFileSync, +} from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; @@ -28,7 +37,7 @@ import { decode } from '@utils/live-record/codec'; import { fakeSocket } from '../helpers/connect'; import { emitted } from '../helpers/fake-server-module'; import { MESSAGE_ID, MESSAGE_SECRET, ORIGINALS, PERSON, recordSession, TEXT } from '../helpers/live-session'; -import { scrubSession } from '../tools/live-scrub'; +import { leakGate, scrubSession, UnknownFieldError } from '../tools/live-scrub'; socketSpy.mockImplementation(fakeSocket); @@ -279,4 +288,43 @@ describe('live-check scrubber', () => { expect(existsSync(out)).toBe(false); }); }); + + it('its leak gate reads the raw tapes itself: a value the scrubber had kept still fails it', () => { + const line = { + seq: 1, + t: 1, + socket: 1, + event: 'messages.upsert', + data: { messages: [{ key: { remoteJidUsername: 'dana.levi88' } }] }, + }; + const raw = { events: [line], webhooks: [], instanceName: 'rig' }; + // As if a rewrite had kept every value. + expect(leakGate(raw, { 'events.ndjson': JSON.stringify(line) + '\n' })).toEqual(['events.ndjson line 1']); + // Structure it keeps is not a leak. + const structure = { seq: 1, t: 1.5, socket: 1, event: 'connection.update', data: { connection: 'open' } }; + const clean = { events: [structure], webhooks: [], instanceName: 'rig' }; + expect(leakGate(clean, { 'events.ndjson': JSON.stringify(structure) + '\n' })).toEqual([]); + }); + + it('knows every field of every committed fixture', () => { + // A committed fixture has the raw tapes' shape: scrubbing it again must not meet an unknown field. + const root = join(process.cwd(), 'test', 'fixtures', 'live'); + for (const fixture of readdirSync(root)) { + const again = join(tmpdir(), `live-rescrub-${process.pid}`, 'test', fixture); + mkdirSync(again, { recursive: true }); + for (const file of ['events.ndjson', 'webhooks.ndjson', 'manifest.json']) { + writeFileSync(join(again, file), readFileSync(join(root, fixture, file))); + } + let error: unknown; + try { + scrubSession(again, { checkId: 'rescrub', date: '2026-09-27', outRoot: join(again, 'out') }); + } catch (e) { + error = e; // Its fakes are originals now, so the leak gate may object; an unknown field may not. + } + rmSync(join(tmpdir(), `live-rescrub-${process.pid}`), { recursive: true, force: true }); + expect( + error instanceof UnknownFieldError ? `${fixture}: ${(error as Error).message}` : undefined, + ).toBeUndefined(); + } + }); }); diff --git a/test/tools/fixture-guard.ts b/test/tools/fixture-guard.ts index 0071d3053f..7ee7815a9b 100644 --- a/test/tools/fixture-guard.ts +++ b/test/tools/fixture-guard.ts @@ -9,6 +9,11 @@ // bytes ({"$bytes"}) of 16+ bytes not tagged fake by the scrubber, and any other // base64 blob of 24+ characters that is not one of those fake bytes (a camelCase // identifier is not a blob) +// in a tape (.ndjson), any string that is neither one of the scrubber's fakes (a fake +// address, "Name ", lorem, an example.invalid URL, a fake message id) nor a value +// its field is known to take (live-fields.ts): a username, a group name, a value in +// a field nobody listed. And any decimal number but the tape's own clock (t) and the +// scrubber's fake decimals (0.): a location. // // A finding names the file, the line, a masked JSON path and the kind of value, // never the value itself. Run on every commit by test/live/fixture-guard.test.ts, @@ -16,6 +21,8 @@ import { existsSync, readdirSync, readFileSync, statSync } from 'node:fs'; import { join, relative } from 'node:path'; +import { isCredsKey, isStructural, OWNER_LABEL } from './live-fields'; + export type Finding = { file: string; line: number; path: string; kind: string }; const FAKE_USER = [/^972500\d{6}$/, /^100000000\d{6}$/, /^120363\d{12}$/, /^(0|16505361212|13135550002)$/]; @@ -30,7 +37,31 @@ const IDENTIFIER = /^[a-z]+(?:[A-Z][a-z]+)+$/; const FAKE_NUMERIC_ID = /^\d{2}0{3,}[1-9]\d{0,3}$/; const ID_KEYS = new Set(['id', 'stanzaId', 'keyId', 'messageId']); /** Numbers under these keys are sizes, counts and times, not people. */ -const NUMERIC_KEY = /(length|size|seconds|duration|count|progress|timestamp|time|^t$|^seq$|at$|height|width|expiration|ttl)/i; +const NUMERIC_KEY = + /(length|size|seconds|duration|count|progress|timestamp|time|^t$|^seq$|at$|height|width|expiration|ttl)/i; + +const LOREM = + 'lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod tempor incididunt ut labore '.repeat(64); +const FAKE_NAME = /^Name \d+$/; +const FAKE_URL = /^https:\/\/example\.invalid\/\d+$/; +/** The scrubber's fake id: two characters kept, then a hex counter padded with F (f), or digits: zeros, a counter. */ +const FAKE_ID = /^.{2}(?:F*[0-9A-F]{1,8}|f*[0-9a-f]{1,8})$/; +const WHOLE_JID = + /^\d+(?:-\d+)?(?:[:_]\d+)*@(s\.whatsapp\.net|c\.us|hosted\.lid|hosted|lid|g\.us|broadcast|newsletter)$/; +/** The scrubber's fake decimal: 0.001 to 0.999. */ +const FAKE_DECIMAL = /^0\.\d{1,3}$/; + +/** A tape string the scrubber wrote or kept on purpose: one of its fakes, or a value its field takes. */ +const isKnownValue = (key: string, s: string) => + s === '' || + s === 'Owner' || + s === OWNER_LABEL || + FAKE_NAME.test(s) || + FAKE_URL.test(s) || + WHOLE_JID.test(s) || + LOREM.includes(s) || + ((ID_KEYS.has(key) || s.length >= 12) && (FAKE_ID.test(s) || FAKE_NUMERIC_ID.test(s))) || + isStructural(key, s); const isEpoch = (d: string) => /^1\d{9}$/.test(d) || /^1\d{12}$/.test(d); const isFakeUser = (user: string) => @@ -53,31 +84,55 @@ function scanString(s: string, report: (kind: string) => void, fakeBytes: Set void, fakeBytes: Set) { +/** The field a value belongs to: the last path segment that is not an array index. */ +const fieldOf = (path: string[]) => [...path].reverse().find((p) => !/^\d+$/.test(p)) ?? ''; + +function walk( + value: any, + path: string[], + report: (kind: string, path: string[]) => void, + fakeBytes: Set, + tape: boolean, +) { const at = (kind: string) => report(kind, path); + const key = fieldOf(path); // A commit hash can hold eight digits in a row. - if (typeof value === 'string' && path[path.length - 1] === 'forkCommit' && /^[0-9a-f]{7,40}(-dirty)?$/.test(value)) return; + if (typeof value === 'string' && path[path.length - 1] === 'forkCommit' && /^[0-9a-f]{7,40}(-dirty)?$/.test(value)) + return; if (typeof value === 'string' && ID_KEYS.has(path[path.length - 1]) && FAKE_NUMERIC_ID.test(value)) return; - if (typeof value === 'string') return scanString(value, at, fakeBytes); + if (typeof value === 'string') { + let found = false; + scanString(value, (kind) => ((found = true), at(kind)), fakeBytes); + if (tape && !found && !isKnownValue(key, value)) at('value that is neither a fake nor known structure'); + return; + } if (typeof value === 'number') { - const key = path[path.length - 1] ?? ''; + if (tape && !Number.isInteger(value) && !(key === 't' && path.length === 1) && !FAKE_DECIMAL.test(String(value))) { + return at('decimal number (a location?)'); + } if (!NUMERIC_KEY.test(key)) scanString(String(value), at, fakeBytes); return; } - if (Array.isArray(value)) return value.forEach((v, i) => walk(v, [...path, String(i)], report, fakeBytes)); + if (Array.isArray(value)) return value.forEach((v, i) => walk(v, [...path, String(i)], report, fakeBytes, tape)); if (!value || typeof value !== 'object') return; if (typeof value.$bytes === 'string') { if (!value.fake && Buffer.from(value.$bytes, 'base64').length >= 16) at('bytes not tagged fake'); return; } if (typeof value.$long === 'string') { - const key = path[path.length - 1] ?? ''; if (!NUMERIC_KEY.test(key)) scanString(value.$long.replace('-', ''), at, fakeBytes); return; } + if (typeof value.$redacted === 'string') { + const keys = Array.isArray(value.keys) ? value.keys : []; + if (tape && !keys.every((k: any) => typeof k === 'string' && isCredsKey(k))) { + at('value that is neither a fake nor known structure'); + } + return; + } for (const [k, v] of Object.entries(value)) { scanString(k, (kind) => report(kind, [...path, k]), fakeBytes); - walk(v, [...path, k], report, fakeBytes); + walk(v, [...path, k], report, fakeBytes, tape); } } @@ -127,6 +182,7 @@ export function scanFixtures(...paths: string[]): Finding[] { (kind, path) => findings.push({ file: relative(process.cwd(), file), line, path: '$.' + path.map(mask).join('.'), kind }), fakeBytes, + file.endsWith('.ndjson'), ); } } diff --git a/test/tools/live-fields.ts b/test/tools/live-fields.ts new file mode 100644 index 0000000000..3620c3ba4b --- /dev/null +++ b/test/tools/live-fields.ts @@ -0,0 +1,91 @@ +// What a live-check tape may keep as written, by field. Shared by the scrubber +// (test/tools/live-scrub.ts), its leak gate and the fixture guard +// (test/tools/fixture-guard.ts): a string is kept only under a field named here +// and only when its value is one this field is known to take. Anything else is +// a person's (a name, a username, a text, an address) until the list says +// otherwise, and a field the scrubber does not know makes it stop, writing +// nothing. Add a field or a value here when a new recording needs one, never a +// pattern that a name or a username could match. +const oneOf = + (...values: string[]) => + (s: string) => + values.includes(s); + +/** Every event Baileys emits and every webhook Evolution sends (Events in src/api/types/wa.types.ts). */ +const EVENTS = new Set( + ` + application.startup instance.create instance.delete qrcode.updated connection.update status.instance + messages.set messages.upsert messages.edited messages.update messages.delete messages.media-update + messages.reaction send.message send.message.update contacts.set contacts.upsert contacts.update + presence.update chats.set chats.update chats.upsert chats.delete chats.lock groups.upsert groups.update + group-participants.update group.join-request group.member-tag.update call typebot.start + typebot.change-status labels.edit labels.association creds.update messaging-history.set + messaging-history.status remove.instance logout.instance blocklist.set blocklist.update + lid-mapping.update message-capping.update message-receipt.update newsletter-participants.update + newsletter-settings.update newsletter.reaction newsletter.view settings.update +` + .trim() + .split(/\s+/), +); +const isEvent = (s: string) => EVENTS.has(s); + +/** The harness's instance name: the scrubber names the instance this, and a replay runs under it. */ +export const REPLAY_INSTANCE = 'test'; + +export const STRUCTURAL: Record boolean> = { + // The tape itself (recorder.ts, codec.ts). + instance: oneOf(REPLAY_INSTANCE), + instanceName: oneOf(REPLAY_INSTANCE), + event: isEvent, + batch: isEvent, + origin: oneOf('app'), + $proto: (s) => /^proto(\.[A-Z][A-Za-z0-9]*)+$/.test(s), + as: oneOf('Buffer', 'Uint8Array'), + $redacted: oneOf('creds'), + $date: (s) => /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d+)?Z$/.test(s), + $fn: (s) => /^[A-Za-z_$][\w$]{0,40}$/.test(s), + // Baileys' and Evolution's enums. + connection: oneOf('open', 'connecting', 'close'), + state: oneOf('open', 'connecting', 'close', 'refused'), + addressingMode: oneOf('pn', 'lid'), + type: oneOf('notify', 'append', 'add', 'remove'), + messageType: (s) => s === 'conversation' || /^[a-z][A-Za-z]*Message$/.test(s), + mimetype: (s) => /^[a-z]+\/[a-z0-9.+-]+(;\s*[a-z0-9-]+=[A-Za-z0-9.-]+)*$/.test(s), + source: oneOf('ios', 'android', 'web', 'desktop', 'unknown'), + status: oneOf( + 'ERROR', + 'PENDING', + 'SERVER_ACK', + 'DELIVERY_ACK', + 'READ', + 'PLAYED', + 'DELETED', + 'offer', + 'ringing', + 'timeout', + 'reject', + 'accept', + 'terminate', + ), + action: oneOf('add', 'remove', 'promote', 'demote', 'modify'), + admin: oneOf('admin', 'superadmin'), + owner_country_code: (s) => /^[A-Z]{2}$/.test(s), +}; + +/** Whether `value` under `key` is structure the tape keeps as written. */ +export const isStructural = (key: string, value: string) => !!STRUCTURAL[key]?.(value); + +/** The creds keys a creds.update carries once redacted ({$redacted:'creds', keys}): Baileys' field names. */ +export const isCredsKey = (s: string) => /^[a-z][A-Za-z0-9]{1,40}$/.test(s); + +/** Numbers under these keys are sizes, counts and times, not people. */ +export const NUMERIC_KEY = + /(length|size|seconds|duration|count|progress|timestamp|time|^t$|^seq$|at$|height|width|expiration|ttl)/i; + +/** A place: always replaced, whole degrees included. */ +export const LOCATION_KEYS = new Set(['degreesLatitude', 'degreesLongitude']); + +export const isEpoch = (digits: string) => /^1\d{9}$/.test(digits) || /^1\d{12}$/.test(digits); + +/** WhatsApp's own label for the owner ("You"), which it puts where a name goes. */ +export const OWNER_LABEL = 'Você'; diff --git a/test/tools/live-scrub.ts b/test/tools/live-scrub.ts index 66c0b58ea4..6e9cc1d64c 100644 --- a/test/tools/live-scrub.ts +++ b/test/tools/live-scrub.ts @@ -4,38 +4,54 @@ // It works on the tapes as written (the tagged codec, never decoded), in three steps: // 1. collect: pair each person's phone JID with their @lid wherever one object // names both, so one person keeps one fake index everywhere; -// 2. rewrite every string leaf, and every object key that is an address; -// 3. the leak gate: search the whole output for every original (numbers, names, -// byte strings, and every raw string of 4+ characters the rewrite replaced). -// One hit aborts, and nothing is written. +// 2. rewrite every string leaf, every number that could be a person's, and every +// object key that is an address. A string under a field no list here knows +// stops the scrub: nothing is written, and the error names the field's path; +// 3. the leak gate (leakGate), independent of what step 2 decided: every value of +// the raw tapes that is not structure (live-fields.ts) is searched for in every +// value and key of the output. One hit aborts, and nothing is written. // -// What a string becomes: +// What a value becomes: // phone JID / @lid / group 972500<6> / 100000000<6> / 120363<12>, device suffix kept; index 0 is the owner -// a bare number of 7-15 digits the same person's fake digits (epoch timestamps are kept) -// a name (name, notify, pushName, subject...) "Name ", one per distinct original ("Você" kept) +// a number of 7-15 digits the same person's fake digits (kept: epoch timestamps, and sizes and times by field) +// a name or a username "Name ", one per distinct original ("Você" kept) // a message id same first two characters and length, the rest a counter (digits: zeros, then it) // bytes ($bytes) random bytes of the same length and type, tagged fake // a URL https://example.invalid/ -// structure (event names, enums, mimetypes, dates, 3 characters or fewer) kept -// anything else, text included lorem of the same length +// a text (TEXT_KEYS) lorem of the same length +// a location, any other decimal 0., one per distinct original (the tape's own clock, t, is kept) +// structure (live-fields.ts) kept: event names and enums, by field, never by shape alone +// anything else stops the scrub import { randomBytes } from 'node:crypto'; import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs'; import { basename, dirname, join } from 'node:path'; +import { + isCredsKey, + isEpoch, + isStructural, + LOCATION_KEYS, + NUMERIC_KEY, + OWNER_LABEL, + REPLAY_INSTANCE, +} from './live-fields'; + export type Operator = { phoneModel?: string; osVersion?: string; whatsappAppVersion?: string; countryCode?: string }; export type ScrubOptions = { checkId: string; date?: string; outRoot?: string; operator?: Operator }; export class LeakError extends Error {} +/** The scrub stopped on a field it does not know. Names the path, never the value. */ +export class UnknownFieldError extends Error {} const FAKE_PN = (i: number) => `972500${String(i).padStart(6, '0')}`; const FAKE_LID = (i: number) => `100000000${String(i).padStart(6, '0')}`; const FAKE_GROUP = (i: number) => `120363${String(i).padStart(12, '0')}`; /** Addresses WhatsApp itself uses, never a person's. */ const SERVICE_USERS = new Set(['0', '16505361212', '13135550002']); -/** The harness's instance name: a replay runs under it. */ -const REPLAY_INSTANCE = 'test'; const JID = /^(\d+(?:-\d+)?)((?:[:_]\d+)*)@(s\.whatsapp\.net|c\.us|hosted|lid|hosted\.lid|g\.us|broadcast|newsletter)$/; +const JID_ANYWHERE = + /(\d+(?:-\d+)?)(?:[:_]\d+)*@(?:s\.whatsapp\.net|c\.us|hosted\.lid|hosted|lid|g\.us|broadcast|newsletter)/g; const PN_SERVERS = new Set(['s.whatsapp.net', 'c.us', 'hosted']); const LID_SERVERS = new Set(['lid', 'hosted.lid']); @@ -45,27 +61,22 @@ const NAME_KEYS = words(` name notify verifiedName verifiedBizName pushName username subject profileName fullName firstName shortName displayName vname `); +/** A username (remoteJidUsername, participantUsername...) is a name. */ +const isNameKey = (key: string) => NAME_KEYS.has(key) || /Username$/.test(key); const TEXT_KEYS = words(` conversation text caption desc description title body matchedText canonicalUrl fileName address contentText footerText headerText vcard selectedDisplayText optionName comment message msgCall + messageStubParameters instanceId label directPath `); const ID_KEYS = words(` id stanzaId keyId messageId callId `); -const STRUCTURAL_KEYS = words(` - $proto $redacted as event type messageType mimetype addressingMode action connection source - origin state status platform mediaType -`); -const isEpoch = (digits: string) => /^1\d{9}$/.test(digits) || /^1\d{12}$/.test(digits); -const isStructural = (key: string, s: string) => - s.length <= 3 || - (STRUCTURAL_KEYS.has(key) && !/\s/.test(s) && s.length <= 64) || - /^[A-Z][A-Z0-9]*(_[A-Z0-9]+)+$/.test(s) || // enum names: SERVER_ACK - /^[A-Z]{2,12}$/.test(s) || // single-word enums: READ, PLAYED - /^[a-z][a-zA-Z0-9]*([._-][a-zA-Z0-9]+)*$/.test(s) || // identifiers: messages.upsert, imageMessage - /^[a-z]+\/[\w.+-]+(;\s*[\w=.-]+)*$/.test(s) || // mimetypes - /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d+)?(Z|[+-]\d{2}:\d{2})?$/.test(s); // ISO dates +const isInstanceKey = (key: string) => key === 'instance' || key === 'instanceName'; + +/** A path segment that could itself be the personal part (an address or a number used as a key) is masked. */ +const maskSegment = (segment: string) => (/\d{5,}|@/.test(segment) ? '' : segment); +const pathOf = (path: string[]) => '$.' + path.map(maskSegment).join('.'); const LOREM = 'lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod tempor incididunt ut labore '; @@ -109,9 +120,7 @@ class Scrubber { private bytes = new Map(); private urls = new Map(); private digits = new Map(); - /** Raw strings the rewrite replaced, and raw strings it kept as structure. */ - readonly replaced = new Set(); - readonly kept = new Set(); + private decimals = new Map(); constructor(private readonly instanceName: string) {} @@ -139,35 +148,55 @@ class Scrubber { if (owner.name) this.names.set(owner.name, 'Owner'); } - /** Step 2. */ - rewrite(value: any, key = ''): any { - if (typeof value === 'string') return this.string(value, key); - if (typeof value === 'number' && Number.isInteger(value) && value >= 1e6) { - const known = this.digits.get(String(value)); - return known ? Number(known) : value; - } - if (Array.isArray(value)) return value.map((v) => this.rewrite(v, key)); + /** Step 2. `path` is where the value sits, for the error when its field is unknown. */ + rewrite(value: any, key = '', path: string[] = []): any { + if (typeof value === 'string') return this.string(value, key, path); + if (typeof value === 'number') return this.number(value, key, path); + if (Array.isArray(value)) return value.map((v, i) => this.rewrite(v, key, [...path, String(i)])); if (!value || typeof value !== 'object') return value; if ('$bytes' in value) return { $bytes: this.fakeBytes(value.$bytes), as: value.as, fake: 1 }; - if ('$long' in value || '$date' in value || '$big' in value || '$u' in value || '$fn' in value) return value; + if ('$long' in value) return { ...value, $long: this.digitString(value.$long, key) }; + if ('$big' in value) return { ...value, $big: this.digitString(value.$big, key) }; + if ('$u' in value || '$fn' in value || '$date' in value) return value; + if ('$redacted' in value) { + // A redacted creds.update keeps the names of the creds fields, and nothing else. + const keys = Array.isArray(value.keys) ? value.keys : []; + const known = value.$redacted === 'creds' && keys.every((k: any) => typeof k === 'string' && isCredsKey(k)); + if (!known || Object.keys(value).some((k) => k !== '$redacted' && k !== 'keys')) { + throw new UnknownFieldError(`unknown field at ${pathOf(path)} (a redacted value): nothing written`); + } + return { $redacted: value.$redacted, keys }; + } const out: Record = {}; for (const [k, v] of Object.entries(value)) { - const newKey = JID.test(k) || /^\d{7,15}$/.test(k) ? this.string(k, '') : k; - out[newKey] = this.rewrite(v, k); + const newKey = JID.test(k) || /^\d{7,15}$/.test(k) ? this.string(k, '', path) : k; + out[newKey] = this.rewrite(v, k, [...path, k]); } return out; } - private string(s: string, key: string): string { - const out = this.stringOf(s, key); - if (out === s) this.kept.add(s); - else this.replaced.add(s); - return out; + /** An integer of 7+ digits is a person's unless it is an epoch, or a size or a time by its field; a decimal is a place. */ + private number(n: number, key: string, path: string[]): number { + if (LOCATION_KEYS.has(key) || !Number.isInteger(n)) { + // The tape's own clock: milliseconds since the recording started. + if (key === 't' && path.length === 1) return n; + return this.fakeDecimal(n); + } + if (Math.abs(n) < 1e6) return n; + return Number(this.digitString(String(n), key)); } - private stringOf(s: string, key: string): string { + private digitString(value: string, key: string): string { + const digits = value.replace(/^-/, ''); + if (digits.length < 7 || !/^\d+$/.test(digits)) return value; + if (this.digits.has(digits)) return value.replace(digits, this.digits.get(digits)); + if (isEpoch(digits) || NUMERIC_KEY.test(key)) return value; + return value.replace(digits, this.fakeUser(digits, 's.whatsapp.net')); + } + + private string(s: string, key: string, path: string[]): string { if (!s) return s; - if (s === this.instanceName && (key === 'instance' || key === 'instanceName')) return REPLAY_INSTANCE; + if (s === this.instanceName && isInstanceKey(key)) return REPLAY_INSTANCE; const jid = JID.exec(s); if (jid) return this.fakeJid(jid[1], jid[2], jid[3]); // A numeric message id (group notifications have them) is an id, not a person. @@ -179,13 +208,16 @@ class Scrubber { if (isEpoch(digits)) return s; return plus + this.fakeUser(digits, 's.whatsapp.net'); } - if (NAME_KEYS.has(key)) return this.fakeName(s); + if (isNameKey(key)) return this.fakeName(s); if (TEXT_KEYS.has(key)) return this.lorem(s); if (this.bytes.has(s)) return this.bytes.get(s); if (s.length >= 8 && (ID_KEYS.has(key) || /^(?=.*\d)[0-9A-F]{12,64}$/.test(s))) return this.fakeId(s); if (/^https?:\/\//i.test(s)) return this.memo(this.urls, s, (n) => `https://example.invalid/${n}`); if (isStructural(key, s)) return s; - return this.lorem(s); + throw new UnknownFieldError( + `unknown field at ${pathOf(path)} (a string of ${s.length} characters): nothing written. ` + + 'Say what the field is in live-scrub.ts or live-fields.ts, then scrub again.', + ); } private fakeJid(user: string, suffix: string, server: string) { @@ -210,7 +242,7 @@ class Scrubber { } private fakeName(s: string) { - if (s === 'Você') return s; + if (s === OWNER_LABEL) return s; return this.memo(this.names, s, (n) => `Name ${n}`); } @@ -229,6 +261,10 @@ class Scrubber { }); } + private fakeDecimal(n: number) { + return Number(this.memo(this.decimals, String(n), (i) => `0.${String(((i - 1) % 999) + 1).padStart(3, '0')}`)); + } + private fakeBytes(b64: string) { return this.memo(this.bytes, b64, () => randomBytes(Buffer.from(b64, 'base64').length).toString('base64')); } @@ -245,16 +281,6 @@ class Scrubber { return map.get(s); } - /** Every original the gate searches for: numbers, names, byte strings, and replaced strings of 4+ characters. */ - originals(): string[] { - const all = new Set(); - for (const d of this.digits.keys()) all.add(d); - for (const n of this.names.keys()) if (n.length > 3 && n !== 'Você') all.add(n); - for (const b of this.bytes.keys()) if (b.length > 3) all.add(b); - for (const s of this.replaced) if (s.length > 3 && !this.kept.has(s)) all.add(s); - return [...all]; - } - counts() { return { people: this.people.count, @@ -264,10 +290,87 @@ class Scrubber { messageIds: this.ids.size, bytes: this.bytes.size, urls: this.urls.size, + decimals: this.decimals.size, }; } } +/** + * Step 3, the leak gate. It reads the raw tapes itself and decides on its own what in them could be + * a person's: every string of 4+ characters that is not structure by its field (live-fields.ts), + * the user part of every address, every run of 7+ digits that is not an epoch, every integer of + * 7+ digits that is not an epoch or a size or a time by its field, every decimal but the tape's + * clock, every byte string. Then it looks for each of them in every string, number and key of the + * output. It never asks the scrubber what it replaced. Returns the hits, as file and line only. + */ +export function leakGate( + raw: { events: Line[]; webhooks: Line[]; owner?: Record; instanceName: string }, + files: Record, +): string[] { + const originals = new Set(); + const addDigits = (digits: string, key = '') => { + if (digits.length >= 7 && !isEpoch(digits) && !SERVICE_USERS.has(digits) && !NUMERIC_KEY.test(key)) { + originals.add(digits); + } + }; + const addString = (s: string, key: string) => { + if (!s || s === OWNER_LABEL || isStructural(key, s)) return; + if (s === raw.instanceName && isInstanceKey(key)) return; + for (const m of s.matchAll(JID_ANYWHERE)) for (const part of m[1].split('-')) addDigits(part); + for (const run of s.match(/\d{7,}/g) ?? []) addDigits(run); + if (s.length >= 4) originals.add(s); + }; + const walk = (value: any, key: string, depth: number) => { + if (typeof value === 'string') return addString(value, key); + if (typeof value === 'number') { + if (Number.isInteger(value) && !LOCATION_KEYS.has(key)) return addDigits(String(Math.abs(value)), key); + if (key === 't' && depth === 1) return; + return void originals.add(String(value)); + } + if (Array.isArray(value)) return value.forEach((v) => walk(v, key, depth + 1)); + if (!value || typeof value !== 'object') return; + if (typeof value.$bytes === 'string') return void (value.$bytes.length >= 4 && originals.add(value.$bytes)); + if (typeof value.$long === 'string') return addDigits(value.$long.replace(/^-/, ''), key); + if (typeof value.$big === 'string') return addDigits(value.$big.replace(/^-/, ''), key); + if ('$u' in value || '$fn' in value || '$date' in value || '$redacted' in value) return; + for (const [k, v] of Object.entries(value)) { + if (JID.test(k) || /^\d{7,15}$/.test(k)) addString(k, ''); + walk(v, k, depth + 1); + } + }; + for (const line of [...raw.events, ...raw.webhooks]) walk(line, '', 0); + const owner = raw.owner ?? {}; + for (const key of ['id', 'lid', 'name']) if (typeof owner[key] === 'string') addString(owner[key], key); + const searched = [...originals]; + + const hits: string[] = []; + for (const [file, text] of Object.entries(files)) { + const units = file.endsWith('.ndjson') + ? text.split('\n').flatMap((l, i): [number, any][] => (l.trim() ? [[i + 1, JSON.parse(l)]] : [])) + : [[1, JSON.parse(text)] as [number, any]]; + for (const [line, unit] of units) { + let hit = false; + const check = (s: string) => { + if (!hit && searched.some((o) => s.includes(o))) hit = true; + }; + const scan = (value: any, key: string) => { + if (hit) return; + if (typeof value === 'string') return isStructural(key, value) ? undefined : check(value); + if (typeof value === 'number') return check(String(value)); + if (Array.isArray(value)) return value.forEach((v) => scan(v, key)); + if (!value || typeof value !== 'object') return; + for (const [k, v] of Object.entries(value)) { + check(k); + scan(v, k); + } + }; + scan(unit, ''); + if (hit) hits.push(`${file} line ${line}`); + } + } + return hits; +} + const readLines = (file: string): Line[] => existsSync(file) ? readFileSync(file, 'utf8') @@ -276,9 +379,6 @@ const readLines = (file: string): Line[] => .map((l) => JSON.parse(l)) : []; -const describeOriginal = (s: string) => - /^\d+$/.test(s) ? `a number of ${s.length} digits` : `a string of ${s.length} characters`; - /** Scrub one raw session. Returns the fixture directory; throws LeakError (writing nothing) on a leak. */ export function scrubSession(rawDir: string, opts: ScrubOptions) { if (!/^[a-z0-9]+(-[a-z0-9]+)*$/.test(opts.checkId)) throw new Error(`check id must be kebab-case: ${opts.checkId}`); @@ -318,16 +418,11 @@ export function scrubSession(rawDir: string, opts: ScrubOptions) { files['manifest.json'] = JSON.stringify(manifest, null, 2) + '\n'; // Step 3, the leak gate: fail closed. - const hits: string[] = []; - for (const original of scrubber.originals()) { - const escaped = JSON.stringify(original).slice(1, -1); - for (const [file, text] of Object.entries(files)) { - const at = text.includes(original) ? text.indexOf(original) : text.indexOf(escaped); - if (at >= 0) hits.push(`${file} line ${text.slice(0, at).split('\n').length}: ${describeOriginal(original)}`); - } - } + const hits = leakGate({ events, webhooks, owner, instanceName }, files); if (hits.length) { - throw new LeakError(`leak gate: ${hits.length} original(s) survived, nothing written:\n ${hits.join('\n ')}`); + throw new LeakError( + `leak gate: an original survived in ${hits.length} place(s), nothing written:\n ${hits.join('\n ')}`, + ); } const report = { leakGate: 'pass', events: events.length, webhooks: webhooks.length, ...scrubber.counts() }; From d82761e4b2437ad449b134d437dc5163564cdde0 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:00:14 +0300 Subject: [PATCH 118/157] test: a recorded link writes the QR payload, its image and the pairing code into the webhook tape The recorder redacted the QR on the events tape only; webhook() encoded the payload as sent, and qrcode.updated carries code, base64 and pairingCode. Anyone holding one of them can link a device to the account. Red on the current code (2 tests): with a QR shown, and with a pairing code asked for, the webhook tape holds the QR payload, the image and the pairing code. Co-Authored-By: Claude Opus 5.5 --- test/live/recorder.test.ts | 58 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 56 insertions(+), 2 deletions(-) diff --git a/test/live/recorder.test.ts b/test/live/recorder.test.ts index f15f45d6dc..7b21975470 100644 --- a/test/live/recorder.test.ts +++ b/test/live/recorder.test.ts @@ -126,7 +126,9 @@ describe('live-check recorder', () => { expect(Long.isLong(message.messageTimestamp)).toBe(true); expect(message.message.conversation).toBe(TEXT); expect(message.message.messageContextInfo.messageSecret).toBeInstanceOf(Uint8Array); - expect(decode(emits[2].data)).toEqual([{ id: PERSON.pn, lid: PERSON.lid, name: PERSON.saved, notify: PERSON.push }]); + expect(decode(emits[2].data)).toEqual([ + { id: PERSON.pn, lid: PERSON.lid, name: PERSON.saved, notify: PERSON.push }, + ]); // The auth creds never reach the tape. expect(emits[1].data).toEqual({ $redacted: 'creds', keys: ['me'] }); @@ -179,7 +181,15 @@ describe('live-check recorder', () => { endedAt: null, }); const text = JSON.stringify(manifest); - for (const secret of ['972529998877', '987654321098765', '972541112233', '123456789012345', OWNER.name, PERSON.saved, '@']) { + for (const secret of [ + '972529998877', + '987654321098765', + '972541112233', + '123456789012345', + OWNER.name, + PERSON.saved, + '@', + ]) { expect(text).not.toContain(secret); } // The account is kept apart, for the scrubber only. @@ -204,4 +214,48 @@ describe('live-check recorder', () => { expect(manifest).not.toContain('127.0.0.1'); expect(manifest).not.toContain('18461'); }); + + // The QR payload and the pairing code link a device to the account: whoever has one can pair it. + // The events tape redacted the QR; the webhook tape wrote qrcode.updated as Evolution sent it, + // code, image and pairing code included. + describe('a link in progress', () => { + const QR = '2@Q1R2S3T4U5V6W7X8Y9Z0qrsecret,keypart,otherpart,lastpart'; + const PAIRING = 'WXYZ4321'; + + async function showQr(number?: string) { + process.env.LIVE_RECORD_DIR = root; + socketSpy.mockImplementationOnce((config: any) => ({ + ...fakeSocket(config), + requestPairingCode: async () => PAIRING, + })); + const { service } = await makeService(); + stubAuthState(service); + await service.connectToWhatsapp(number); + service.client.ev.emit('connection.update', { qr: QR }); + await vi.waitFor(() => expect(emitted.some((e) => e.event === 'qrcode.updated')).toBe(true), { timeout: 5_000 }); + const sentQr = emitted.find((e) => e.event === 'qrcode.updated').data.qrcode; + const dir = sessionDir(); + const tapes = ['events.ndjson', 'webhooks.ndjson'].map((f) => readFileSync(join(dir, f), 'utf8')).join('\n'); + return { sentQr, tapes }; + } + + it('never writes the QR payload or its image, in either tape', async () => { + const { sentQr, tapes } = await showQr(); + // Evolution still sends them: only the recording leaves them out. + expect(sentQr.code).toBe(QR); + expect({ code: tapes.includes('qrsecret'), image: tapes.includes(sentQr.base64.slice(22, 80)) }).toEqual({ + code: false, + image: false, + }); + }); + + it('never writes the pairing code, in either tape', async () => { + const { sentQr, tapes } = await showQr('972541112233'); + expect(sentQr.pairingCode).toBe(PAIRING); + expect({ pairingCode: tapes.includes(PAIRING), code: tapes.includes('qrsecret') }).toEqual({ + pairingCode: false, + code: false, + }); + }); + }); }); From 4d5b98a9e588ccdebc2c1ced4f1d330f7fca725f Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:00:56 +0300 Subject: [PATCH 119/157] fix: the recorder writes a marker, not the QR, its image or the pairing code, on the webhook tape webhook() now replaces qrcode.updated's code, base64 and pairingCode with $qr / $pairingCode before encoding, as the events tape already did for connection.update's qr. Evolution still sends them unchanged: only the recording differs. The scrubber keeps those markers (live-fields.ts lists them, and the events tape's $qr, which it used to turn into lorem) and stops on any other value there, so a real pairing code an older recording kept can never reach a fixture (new test). docs/LIVE-CHECKS.md says what each tape redacts. Co-Authored-By: Claude Opus 5.5 --- docs/LIVE-CHECKS.md | 4 +++- src/utils/live-record/recorder.ts | 28 ++++++++++++++++++++++++++-- test/live/scrub.test.ts | 17 +++++++++++++++++ test/tools/live-fields.ts | 5 +++++ 4 files changed, 51 insertions(+), 3 deletions(-) diff --git a/docs/LIVE-CHECKS.md b/docs/LIVE-CHECKS.md index 53db8e21e3..3f5f4c7596 100644 --- a/docs/LIVE-CHECKS.md +++ b/docs/LIVE-CHECKS.md @@ -23,7 +23,9 @@ on. This file is the protocol, the catalogue of checks, and the log of results. Values are written in a tagged codec (`src/utils/live-record/codec.ts`) so a replay rebuilds identical ones: `$bytes` (Buffer or Uint8Array), `$long`, `$u` (undefined), `$proto` (the protobuf class), `$err`, `$date`. The auth - creds and the QR payload are redacted when recorded. + creds are redacted when recorded, and so is whatever links a device, on both + tapes: the QR payload (`$qr`) in `connection.update`, and the QR payload, its + image and the pairing code (`$qr`, `$pairingCode`) in `qrcode.updated`. 2. **Scrub.** `scripts/live-scrub.ts` turns a raw session into `test/fixtures/live/-/`, then runs a leak gate that takes every value of the raw tapes that is not structure and searches the diff --git a/src/utils/live-record/recorder.ts b/src/utils/live-record/recorder.ts index 0eefb61eda..3d7ee87ac6 100644 --- a/src/utils/live-record/recorder.ts +++ b/src/utils/live-record/recorder.ts @@ -160,7 +160,12 @@ export class LiveRecorder { /** A payload Evolution sends out (sendDataWebhook), as it was at that moment. */ webhook(event: string, data: any, extra?: Record) { this.guard(() => { - const line: Record = { seq: ++this.seq, t: this.elapsed(), event, data: encode(data) }; + const line: Record = { + seq: ++this.seq, + t: this.elapsed(), + event, + data: encode(redactWebhook(data)), + }; if (extra !== undefined) line.extra = encode(extra); this.append('webhooks.ndjson', line); }); @@ -207,9 +212,28 @@ export class LiveRecorder { } } -/** Secrets with no replay value never reach the tape: the auth creds and the QR payload. */ +/** + * Secrets with no replay value never reach the tape: the auth creds, and whatever links a device + * (the QR payload, its image, a pairing code). Replaced before encoding, by a marker that says + * one was there. + */ function redact(event: string, data: any) { if (event === 'creds.update') return { $redacted: 'creds', keys: Object.keys(data ?? {}) }; if (event === 'connection.update' && data?.qr) return { ...data, qr: '$qr' }; return data; } + +/** qrcode.updated carries the QR payload, its image and the pairing code (connectionUpdate). */ +function redactWebhook(data: any) { + const qrcode = data?.qrcode; + if (!qrcode || typeof qrcode !== 'object') return data; + return { + ...data, + qrcode: { + ...qrcode, + ...(qrcode.code ? { code: '$qr' } : {}), + ...(qrcode.base64 ? { base64: '$qr' } : {}), + ...(qrcode.pairingCode ? { pairingCode: '$pairingCode' } : {}), + }, + }; +} diff --git a/test/live/scrub.test.ts b/test/live/scrub.test.ts index 918a0db43b..d724b7a1b7 100644 --- a/test/live/scrub.test.ts +++ b/test/live/scrub.test.ts @@ -327,4 +327,21 @@ describe('live-check scrubber', () => { ).toBeUndefined(); } }); + + it('keeps the markers where a QR or a pairing code was, and stops on a real one an older recording kept', () => { + const qrLine = (seq: number, qrcode: object) => + JSON.stringify({ seq, t: 1, event: 'qrcode.updated', data: { qrcode: { instance: 'test', ...qrcode } } }) + '\n'; + appendFileSync( + join(raw, 'webhooks.ndjson'), + qrLine(990, { pairingCode: '$pairingCode', code: '$qr', base64: '$qr' }), + ); + const { dir } = scrub(); + const [marked] = lines(join(dir, 'webhooks.ndjson')).filter((w) => w.seq === 990); + expect(marked.data.qrcode).toEqual({ instance: 'test', pairingCode: '$pairingCode', code: '$qr', base64: '$qr' }); + + rmSync(out, { recursive: true, force: true }); + appendFileSync(join(raw, 'webhooks.ndjson'), qrLine(991, { pairingCode: 'WXYZ4321', code: '$qr', base64: '$qr' })); + expect(() => scrub()).toThrow(UnknownFieldError); + expect(existsSync(out)).toBe(false); + }); }); diff --git a/test/tools/live-fields.ts b/test/tools/live-fields.ts index 3620c3ba4b..24c2fd7a75 100644 --- a/test/tools/live-fields.ts +++ b/test/tools/live-fields.ts @@ -42,6 +42,11 @@ export const STRUCTURAL: Record boolean> = { $proto: (s) => /^proto(\.[A-Z][A-Za-z0-9]*)+$/.test(s), as: oneOf('Buffer', 'Uint8Array'), $redacted: oneOf('creds'), + // What the recorder puts where a QR, its image or a pairing code was (recorder.ts). + qr: oneOf('$qr'), + code: oneOf('$qr'), + base64: oneOf('$qr'), + pairingCode: oneOf('$pairingCode'), $date: (s) => /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d+)?Z$/.test(s), $fn: (s) => /^[A-Za-z_$][\w$]{0,40}$/.test(s), // Baileys' and Evolution's enums. From 529c4b9a7ecc7b411d6e7fbe94e45482dd4c6407 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:03:03 +0300 Subject: [PATCH 120/157] test: a 202 PENDING logout is undone by losing the instances volume, and answered 202 even when it was not recorded The pending state lived only in the marker file under INSTANCE_DIR; the row got connectionStatus 'close', which reads like any disconnected instance, and a failure to write either was logged and ignored. - Logout, then lose the instances volume and restart: the instance loads as a normal one, connectionState drops logoutPending, and /instance/connect opens a normal session on the linked creds. - Delete, then lose the volume and restart: the kept row loads as a normal instance, back in the API, and nothing delivers the logout (also on a real Postgres). - A logout whose pending state cannot be written answers 202; a delete in that state answers 400 with the instance already out of the API. The old test 'is never undone by a lost marker file' asserted only that the boot does not connect; it is replaced by the stronger expectation that the logout stays pending and is delivered. Red on the current code: 5 tests. Co-Authored-By: Claude Opus 5.5 --- test/instance/logout-pending-postgres.test.ts | 20 +++++ test/instance/logout-pending.test.ts | 86 +++++++++++++++++-- 2 files changed, 97 insertions(+), 9 deletions(-) diff --git a/test/instance/logout-pending-postgres.test.ts b/test/instance/logout-pending-postgres.test.ts index 8a033f8525..cbbe041fda 100644 --- a/test/instance/logout-pending-postgres.test.ts +++ b/test/instance/logout-pending-postgres.test.ts @@ -328,4 +328,24 @@ describe('deleting an instance whose logout cannot reach WhatsApp (real Postgres expect(monitor.finishingLogouts.test).toBeUndefined(); expect((await call('GET', 'connectionState', globalKey)).status).toBe(404); }); + + it('survives a restart that lost the instances volume: the row alone keeps it pending and hidden', async () => { + const { service } = await waitingToReconnect(); + expect((await call('DELETE', 'delete')).status).toBe(202); + service.stopReconnecting(); + service.connect = async () => undefined; + socketSpy.mockClear(); + rmSync(DIR, { recursive: true, force: true }); + + const monitor = await startProcess(); + const sock = await reconnected(1); + expect(monitor.waInstances.test).toBeUndefined(); + await hiddenFromTheApi(); + await opened(sock); + await settle(monitor.finishingLogouts.test); + expect({ logouts: sock.logouts, rows: await rows() }).toEqual({ + logouts: 1, + rows: { instances: 0, sessions: 0, proxies: 0, settings: 0 }, + }); + }); }); diff --git a/test/instance/logout-pending.test.ts b/test/instance/logout-pending.test.ts index 5512a505ab..bf706ce547 100644 --- a/test/instance/logout-pending.test.ts +++ b/test/instance/logout-pending.test.ts @@ -315,24 +315,92 @@ describe('a logout that cannot reach WhatsApp', () => { expect({ logouts: sock.logouts, me: storedMe(), dir: existsSync(DIR) }).toEqual({ logouts: 1, me: [], dir: false }); }); - // The marker lives with the key files (INSTANCE_DIR); the row is in the database. If the files - // are lost, the row alone must keep the boot from bringing the instance back as a normal one. - it('is never undone by a lost marker file: the boot does not connect the instance', async () => { + // The marker lives with the key files (INSTANCE_DIR); the row and the creds are in the database. + // A 202 promised the logout will be delivered: losing the instances volume must not undo that. The + // boot used to load the instance as a normal one (not connected), and /instance/connect then + // opened a normal session on the linked creds. + it('survives the loss of the instances volume: after a restart it is still pending, and delivered', async () => { const { service } = await waitingToReconnect(); expect(await call('DELETE', 'logout')).toEqual(PENDING); service.stopReconnecting(); service.connect = async () => undefined; socketSpy.mockClear(); - rmSync(MARKER); + rmSync(DIR, { recursive: true, force: true }); const monitor = await startProcess(); await vi.waitFor(() => expect(monitor.waInstances.test).toBeDefined()); - await new Promise((r) => setTimeout(r, 1_500)); + const pending = { status: 200, body: { instance: { instanceName: 'test', state: 'close', logoutPending: true } } }; + expect({ state: await call('GET', 'connectionState'), connect: await call('GET', 'connect') }).toEqual({ + state: pending, + connect: pending, + }); + + const sock = await reconnected(1); + await deliverWhilePending(sock, monitor.waInstances.test); + await opened(sock); + await settle(monitor.waInstances.test); + expect({ logouts: sock.logouts, me: storedMe(), emitted: emitted.map((e) => e.event) }).toEqual({ + logouts: 1, + me: [], + emitted: [], + }); + }); + + it('a deleted instance survives the loss of the instances volume: still out of the API, its name taken', async () => { + const { service } = await waitingToReconnect(); + expect((await call('DELETE', 'delete')).status).toBe(202); + service.stopReconnecting(); + service.connect = async () => undefined; + socketSpy.mockClear(); + rmSync(DIR, { recursive: true, force: true }); + + const monitor = await startProcess(); + const sock = await reconnected(1); + const create = await fetch(`${app.base}/instance/create`, { + method: 'POST', + headers: { apikey: globalKey, 'content-type': 'application/json' }, + body: JSON.stringify({ instanceName: 'test', integration: 'WHATSAPP-BAILEYS' }), + }); + expect({ listed: !!monitor.waInstances.test, create: create.status }).toEqual({ listed: false, create: 403 }); + await opened(sock); + await settle(monitor.finishingLogouts.test); + expect({ logouts: sock.logouts, me: storedMe(), instances: prisma.instance.rows.length }).toEqual({ + logouts: 1, + me: [], + instances: 0, + }); + }); + + // A 202 is a promise the logout survives a restart. When it cannot be recorded, the caller is told. + it('answers an error, not 202, when the pending logout cannot be recorded, and 202 once it can', async () => { + const { service } = await waitingToReconnect(); + const update = prisma.instance.update; + prisma.instance.update = async () => { + throw new Error("Can't reach database server"); + }; + const failed = await call('DELETE', 'logout'); + prisma.instance.update = update; + expect(failed.status).toBe(500); + // Still pending in this process meanwhile: the creds stay. + expect({ me: storedMe(), pending: service.logoutPending }).toEqual({ me: [WUID], pending: true }); + + expect(await call('DELETE', 'logout')).toEqual(PENDING); + }); + + it('a delete whose pending logout cannot be recorded fails, and leaves the instance in the API', async () => { + const { monitor } = await waitingToReconnect(); + const update = prisma.instance.update; + prisma.instance.update = async () => { + throw new Error("Can't reach database server"); + }; + const failed = await call('DELETE', 'delete'); + prisma.instance.update = update; expect({ - socketsBuilt: built().length, - emitted: emitted.map((e) => e.event), - state: monitor.waInstances.test.connectionStatus.state, - }).toEqual({ socketsBuilt: 0, emitted: [], state: 'close' }); + status: failed.status, + listed: !!monitor.waInstances.test, + me: storedMe(), + instances: prisma.instance.rows.length, + }).toEqual({ status: 500, listed: true, me: [WUID], instances: 1 }); }); it('finishes when WhatsApp answers loggedOut on the reconnect (the device was already removed)', async () => { From 2d5c296792fae9977b52b552ce2456edd9f56aa4 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:08:11 +0300 Subject: [PATCH 121/157] fix: a pending logout is recorded on the instance's row before 202, survives a lost instances volume, and a failure to record it is answered The pending state lived only in the marker file under INSTANCE_DIR, and a failed write was logged and ignored. Now: - recordPending writes it on the row (connectionStatus close, disconnectionAt, disconnectionObject: { logoutPending: { deleted, since } }), where the creds it needs also are. No schema change. - The boot reads a pending logout from the marker, else from the row (pendingLogout), so after losing the volume the instance is still pending: /instance/connect answers with that instead of opening a normal session, and a deleted one stays out of the API with its name taken. - logoutInstance records it before answering 'pending', and throws when it cannot: the logout answers 500 (asking again retries the record and answers 202 once it lands). A delete in that state answers 500 and leaves the instance, its creds and its pending logout in place; a delete whose deleted flag cannot be recorded changes nothing. - A delivered logout clears the flag before the marker goes, so a later boot never logs out a device linked again since (new test). Tests: two assertions in the red commit's tests were wrong about what the fix should do and are corrected here: a delivered non-deleted logout announces itself (logout.instance) as any logout does, and settle() needs the finishing service captured before it leaves finishingLogouts. Visible to consumers: while a logout is pending, fetchInstances shows disconnectionObject { logoutPending: { deleted, since } } on its row; a logout or delete whose pending state cannot be recorded answers 500 instead of 202 (a delete in that case used to answer 400). Co-Authored-By: Claude Opus 5.5 --- FORK.md | 2 +- src/api/controllers/instance.controller.ts | 17 ++- .../whatsapp/whatsapp.baileys.service.ts | 114 +++++++++++++++--- src/api/services/monitor.service.ts | 8 +- src/utils/logout-marker.ts | 6 +- test/instance/logout-pending-postgres.test.ts | 3 +- test/instance/logout-pending.test.ts | 32 ++++- 7 files changed, 151 insertions(+), 31 deletions(-) diff --git a/FORK.md b/FORK.md index 5a2d6d457c..c4c7f7004b 100644 --- a/FORK.md +++ b/FORK.md @@ -63,7 +63,7 @@ named where one exists. **Sessions and connections** - A database error never replaces a linked session's credentials with fresh ones, and a failed settings read no longer drops an event batch. -- A logout or delete while the socket is down keeps the session until WhatsApp is told, so the device leaves the phone's Linked devices; it answers `202 PENDING` meanwhile and forwards nothing (#2520, #2508 in part). A deleted instance keeps its database row until then (Session and Proxy cascade from it), out of the API, its name taken and its token cleared. +- A logout or delete while the socket is down keeps the session until WhatsApp is told, so the device leaves the phone's Linked devices; it answers `202 PENDING` meanwhile and forwards nothing (#2520, #2508 in part). The pending logout is recorded on the instance's row (`disconnectionObject.logoutPending`) as well as in a file next to the session keys, so a restart that lost the instances volume still finishes it; when it cannot be recorded, the logout or delete answers 500 instead of 202. A deleted instance keeps its database row until then (Session and Proxy cascade from it), out of the API, its name taken and its token cleared. - Reconnects back off from 1s to 60s instead of spinning, the version fetch times out after 10s, and an instance never runs two sockets (#2134, #2184, #2430; ideas from #2732). - Profile pictures are looked up once per contact per hour, at most four at a time, and history never waits on them (#1883). - One pairing code per connect attempt, and a fresh QR budget per attempt (#2100, #2696). diff --git a/src/api/controllers/instance.controller.ts b/src/api/controllers/instance.controller.ts index 2aefa946d7..70252d2ce5 100644 --- a/src/api/controllers/instance.controller.ts +++ b/src/api/controllers/instance.controller.ts @@ -457,7 +457,15 @@ export class InstanceController { const { instance } = await this.connectionState({ instanceName }); const waInstance = this.waMonitor.waInstances[instanceName]; - if (waInstance?.logoutPending) return LOGOUT_PENDING; + // Already pending: answered 202 only once it is recorded (logoutInstance tries again if it was not). + if (waInstance?.logoutPending) { + try { + await waInstance.logoutInstance(); + } catch (error) { + throw new InternalServerErrorException(error.toString()); + } + return LOGOUT_PENDING; + } // "close" is also an instance whose socket dropped and whose linked session is still stored // (a reconnect waiting): that one is logged out, or its reconnect brings the session back. @@ -488,6 +496,8 @@ export class InstanceController { try { pending = (await this.logout({ instanceName }))?.status === 'PENDING'; } catch (error) { + // Pending but not recorded: deleting now would wipe the creds the logout needs. + if (waInstances?.logoutPending) throw error; // A failed logout must not stop the delete. The remove.instance emit // below is the only path that purges the in-memory entry and runs // cleaningUp() and cleaningStoreData(), which wipe the session again. @@ -520,6 +530,11 @@ export class InstanceController { this.eventEmitter.emit('remove.instance', instanceName, 'inner'); return { status: 'SUCCESS', error: false, response: { message: 'Instance deleted' } }; } catch (error) { + if (error instanceof InternalServerErrorException) throw error; + // A pending logout that could not be recorded as deleted: the instance stays, and so does its logout. + if (this.waMonitor.waInstances[instanceName]?.logoutPending) { + throw new InternalServerErrorException(error.toString()); + } throw new BadRequestException(error.toString()); } } diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 43b34dce90..481cdde3b6 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -80,7 +80,7 @@ import { BadRequestException, InternalServerErrorException, NotFoundException } import ffmpegPath from '@ffmpeg-installer/ffmpeg'; import { Boom } from '@hapi/boom'; import { createId as cuid } from '@paralleldrive/cuid2'; -import { Instance, Message } from '@prisma/client'; +import { Instance, Message, Prisma } from '@prisma/client'; import { chatState } from '@utils/chat-state'; import { createJid } from '@utils/createJid'; import { fetchLatestWaWebVersion } from '@utils/fetchLatestWaWebVersion'; @@ -322,6 +322,20 @@ const isExpiredMedia = (error: any, media: { url?: string | null; directPath?: s */ export const MEDIA_REUPLOAD_TIMEOUT_MS = 60_000; +/** The pending logout recordPending wrote on an instance's row (disconnectionObject), if there is one. */ +const pendingOnRow = (value: unknown): { deleted: boolean } | undefined => { + let object: any = value; + if (typeof object === 'string') { + try { + object = JSON.parse(object); + } catch { + return undefined; + } + } + const pending = object?.logoutPending; + return pending && typeof pending === 'object' ? { deleted: !!pending.deleted } : undefined; +}; + export class BaileysStartupService extends ChannelStartupService { private messageProcessor = new BaileysMessageProcessor(); @@ -358,9 +372,16 @@ export class BaileysStartupService extends ChannelStartupService { // Stops Evolution listening to the current socket; called before that socket is ended. private detachClient?: () => void; // A logout under way. Until it is done the instance forwards and stores nothing, and it connects - // only to tell WhatsApp. `marked`: it could not reach WhatsApp, so it is pending, with a marker - // (utils/logout-marker.ts) that survives a restart. `deleted`: the instance has left the API. - private logout: { marked: boolean; deleted: boolean; settle: (outcome: 'done' | 'pending') => void } | null = null; + // only to tell WhatsApp. `marked`: it could not reach WhatsApp, so it is pending, recorded on the + // instance's row (`recorded`, what a 202 promises: it survives a restart and the loss of the + // instances volume) and in a marker file (utils/logout-marker.ts). `deleted`: the instance has + // left the API. + private logout: { + marked: boolean; + deleted: boolean; + recorded?: boolean; + settle: (outcome: 'done' | 'pending') => void; + } | null = null; private logBaileys = this.configService.get('LOG').BAILEYS; private eventProcessingQueue: Promise = Promise.resolve(); // Records this session's events and webhooks for a live check; undefined unless LIVE_RECORD_DIR is set. @@ -406,7 +427,10 @@ export class BaileysStartupService extends ChannelStartupService { public async logoutInstance(): Promise<'done' | 'pending'> { // A connect under way finishes first, so the socket logged out is the one it builds. await this.connecting?.socket.catch(() => undefined); - if (this.logout?.marked) return 'pending'; + if (this.logout?.marked) { + await this.recordPending(); + return 'pending'; + } if (this.logout) return 'done'; const linked = await this.hasLinkedSession(); @@ -443,8 +467,10 @@ export class BaileysStartupService extends ChannelStartupService { const late = new Promise<'late'>((r) => (timer = setTimeout(() => r('late'), 10_000))); const result = await Promise.race([outcome, late]); clearTimeout(timer); - if (result === 'late') { - await this.markLogoutPending(); + if (result === 'late') await this.markLogoutPending(); + // A 202 promises the logout survives a restart: when that cannot be recorded, the caller hears it. + if (result === 'late' || result === 'pending') { + await this.recordPending(); return 'pending'; } return result; @@ -455,18 +481,45 @@ export class BaileysStartupService extends ChannelStartupService { return !!this.logout?.marked; } - /** The instance was deleted while its logout is pending: it finishes out of the API, then removes the rest. */ + /** + * The instance was deleted while its logout is pending: it finishes out of the API, then removes + * the rest. Throws, changing nothing, when that cannot be recorded on its row. + */ public async markLogoutDeleted() { if (!this.logout) return; this.logout.deleted = true; + this.logout.recorded = false; + try { + await this.recordPending(); + } catch (error) { + this.logout.deleted = false; + this.logout.recorded = false; + throw error; + } await this.writeMarker(); } - /** On boot: an instance with a logout marker connects only to finish its logout. Returns whether it had one. */ - public async resumePendingLogout(): Promise { + /** + * A logout that was pending when the process stopped: from the marker file, else from the row + * (the instances volume can be lost; the row and the creds are in the database). + */ + public async pendingLogout(): Promise<{ deleted: boolean } | undefined> { const marker = readLogoutMarker(this.instanceId); - if (!marker) return false; - this.logout = { marked: true, deleted: !!marker.deleted, settle: () => undefined }; + if (marker) return { deleted: !!marker.deleted }; + try { + const row = await this.prismaRepository.instance.findUnique({ where: { id: this.instanceId } }); + return pendingOnRow(row?.disconnectionObject); + } catch (error) { + this.logger.error({ message: 'Could not read whether a logout is pending', error: errorFields(error) }); + return undefined; + } + } + + /** On boot: an instance with a pending logout connects only to finish it. Returns whether it had one. */ + public async resumePendingLogout(pending?: { deleted: boolean }): Promise { + pending ??= await this.pendingLogout(); + if (!pending) return false; + this.logout = { marked: true, deleted: pending.deleted, recorded: true, settle: () => undefined }; this.logger.info(`Resuming a pending logout for instance "${this.instance.name}"`); try { await this.connect(this.phoneNumber); @@ -516,18 +569,34 @@ export class BaileysStartupService extends ChannelStartupService { } } + /** + * Record the pending logout on the instance's row, where the boot finds it even without the + * marker file, and where the creds it needs are. Throws when it cannot: a 202 must not promise + * what a restart would forget. + */ + private async recordPending() { + const logout = this.logout; + if (!logout?.marked || logout.recorded) return; + await this.prismaRepository.instance.update({ + where: { id: this.instanceId }, + data: { + connectionStatus: 'close', + disconnectionAt: new Date(), + disconnectionObject: { logoutPending: { deleted: logout.deleted, since: new Date().toISOString() } }, + }, + }); + logout.recorded = true; + } + /** The logout could not reach WhatsApp: keep the session, mark it pending, and reconnect to deliver it. */ private async markLogoutPending() { if (!this.logout || this.logout.marked) return; this.logout.marked = true; await this.writeMarker(); - // The row too: the boot auto-connects only an open or connecting row, so even if the marker - // file is lost the instance does not come back as a normal one (it stays down, creds kept). - await this.prismaRepository.instance - .update({ where: { id: this.instanceId }, data: { connectionStatus: 'close' } }) - .catch((error) => - this.logger.error({ message: 'Could not record the pending logout on the row', error: error?.toString() }), - ); + // Still pending in this process if it fails; logoutInstance tries again and tells the caller. + await this.recordPending().catch((error) => + this.logger.error({ message: 'Could not record the pending logout on the row', error: errorFields(error) }), + ); this.logout.settle('pending'); if (!this.reconnectTimer && !this.connecting && this.stateConnection.state === 'close') this.scheduleReconnect(); } @@ -570,6 +639,13 @@ export class BaileysStartupService extends ChannelStartupService { this.stopReconnecting(); try { await this.removeSession(); + // Not pending any more: the boot must never resume a logout on a device linked again later. + if (!logout.deleted) { + await this.prismaRepository.instance.updateMany({ + where: { id: this.instanceId }, + data: { disconnectionObject: Prisma.DbNull }, + }); + } if (logout.deleted) { // Out of the API already: remove what was kept for the logout, the row last (it cascades to // the rest). Before the marker goes, so a failure here is finished again on the next try. diff --git a/src/api/services/monitor.service.ts b/src/api/services/monitor.service.ts index 7998bb43f5..a9a6a71454 100644 --- a/src/api/services/monitor.service.ts +++ b/src/api/services/monitor.service.ts @@ -365,11 +365,11 @@ export class WAMonitoringService { // A logout that had not reached WhatsApp before the restart: connect only to deliver it. A // deleted instance's row is kept until then, and it stays out of the API (finishingLogouts). - const marker = readLogoutMarker(instanceData.instanceId); - if (marker && (instance as any).resumePendingLogout) { - if (marker.deleted) this.finishingLogouts[instanceData.instanceName] = instance; + const pending = await (instance as any).pendingLogout?.(); + if (pending) { + if (pending.deleted) this.finishingLogouts[instanceData.instanceName] = instance; else this.waInstances[instanceData.instanceName] = instance; - await (instance as any).resumePendingLogout(); + await (instance as any).resumePendingLogout(pending); return; } diff --git a/src/utils/logout-marker.ts b/src/utils/logout-marker.ts index 5b00869640..e947d35899 100644 --- a/src/utils/logout-marker.ts +++ b/src/utils/logout-marker.ts @@ -6,8 +6,10 @@ import { join } from 'path'; /** * A logout that could not reach WhatsApp is kept pending until it does, across restarts. The * marker is a file in the instance's directory under INSTANCE_DIR, next to the session's signal - * key files: no schema change (the fork stays migration-identical to 2.3.7), the same durability - * as the session it has to log out, and removed by the same rm that wipes those keys. + * key files, and removed by the same rm that wipes those keys. It is the second record: the first + * is on the instance's row (disconnectionObject.logoutPending, BaileysStartupService.recordPending), + * which survives the loss of that directory. No schema change: the fork stays migration-identical + * to 2.3.7. */ export type LogoutMarker = { instanceName: string; deleted: boolean; since: string }; diff --git a/test/instance/logout-pending-postgres.test.ts b/test/instance/logout-pending-postgres.test.ts index cbbe041fda..877cebb0f9 100644 --- a/test/instance/logout-pending-postgres.test.ts +++ b/test/instance/logout-pending-postgres.test.ts @@ -341,8 +341,9 @@ describe('deleting an instance whose logout cannot reach WhatsApp (real Postgres const sock = await reconnected(1); expect(monitor.waInstances.test).toBeUndefined(); await hiddenFromTheApi(); + const finishing = monitor.finishingLogouts.test; await opened(sock); - await settle(monitor.finishingLogouts.test); + await settle(finishing); expect({ logouts: sock.logouts, rows: await rows() }).toEqual({ logouts: 1, rows: { instances: 0, sessions: 0, proxies: 0, settings: 0 }, diff --git a/test/instance/logout-pending.test.ts b/test/instance/logout-pending.test.ts index bf706ce547..6453a518e9 100644 --- a/test/instance/logout-pending.test.ts +++ b/test/instance/logout-pending.test.ts @@ -337,12 +337,37 @@ describe('a logout that cannot reach WhatsApp', () => { const sock = await reconnected(1); await deliverWhilePending(sock, monitor.waInstances.test); + expect(emitted.map((e) => e.event)).toEqual([]); await opened(sock); await settle(monitor.waInstances.test); - expect({ logouts: sock.logouts, me: storedMe(), emitted: emitted.map((e) => e.event) }).toEqual({ + expect({ + logouts: sock.logouts, + me: storedMe(), + loggedOut: emitted.some((e) => e.event === 'logout.instance'), + }).toEqual({ logouts: 1, me: [], - emitted: [], + loggedOut: true, + }); + }); + + // What the row records must go when the logout is delivered, or a later boot would log out a + // device linked again since. + it('once delivered, is not resumed by the next boot', async () => { + const { service } = await waitingToReconnect(); + expect(await call('DELETE', 'logout')).toEqual(PENDING); + const sock = await reconnected(2); + await opened(sock); + await settle(service); + expect(sock.logouts).toBe(1); + socketSpy.mockClear(); + + const monitor = await startProcess(); + await vi.waitFor(() => expect(monitor.waInstances.test).toBeDefined()); + await new Promise((r) => setTimeout(r, 1_200)); + expect({ sockets: built().length, pending: monitor.waInstances.test.logoutPending }).toEqual({ + sockets: 0, + pending: false, }); }); @@ -362,8 +387,9 @@ describe('a logout that cannot reach WhatsApp', () => { body: JSON.stringify({ instanceName: 'test', integration: 'WHATSAPP-BAILEYS' }), }); expect({ listed: !!monitor.waInstances.test, create: create.status }).toEqual({ listed: false, create: 403 }); + const finishing = monitor.finishingLogouts.test; await opened(sock); - await settle(monitor.finishingLogouts.test); + await settle(finishing); expect({ logouts: sock.logouts, me: storedMe(), instances: prisma.instance.rows.length }).toEqual({ logouts: 1, me: [], From 54a389208557e8390cd476149aaaca6917dec471 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:12:40 +0300 Subject: [PATCH 122/157] test: a second logout answers 'logged out' while the first is still in flight logoutInstance returned 'done' whenever it saw a logout under way, so a second logout (or a delete) answered 200 while the first's remove-companion-device had not been sent yet and could still fail. Red on the current code: with the first logout held before it reaches WhatsApp, the second request answers at once. Co-Authored-By: Claude Opus 5.5 --- test/instance/logout-pending.test.ts | 32 ++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/test/instance/logout-pending.test.ts b/test/instance/logout-pending.test.ts index 6453a518e9..effa3c2ffe 100644 --- a/test/instance/logout-pending.test.ts +++ b/test/instance/logout-pending.test.ts @@ -561,4 +561,36 @@ describe('a logout that cannot reach WhatsApp', () => { marker: false, }); }); + + // Two logouts at once (two clients, or a logout and a delete): the second answered 'done' as + // soon as it saw the first under way, while the first's remove-companion-device was still in + // flight and could still fail and leave the logout pending. + it('a second logout while the first is under way answers with the first, not before it', async () => { + await linkedInstance(); + const monitor = await startProcess(); + await vi.waitFor(() => expect(built()).toHaveLength(1)); + const service = monitor.waInstances.test; + const sock = current(); + await opened(sock); + let release: () => void; + const gate = new Promise((r) => (release = r)); + const logout = sock.logout; + sock.logout = async (msg?: string) => { + await gate; + return logout(msg); + }; + + const answered: string[] = []; + const first = call('DELETE', 'logout').then((r) => (answered.push('first'), r)); + await new Promise((r) => setTimeout(r, 50)); + const second = call('DELETE', 'logout').then((r) => (answered.push('second'), r)); + await new Promise((r) => setTimeout(r, 200)); + expect(answered).toEqual([]); + + release(); + const ok = { status: 200, body: { status: 'SUCCESS', error: false, response: { message: 'Instance logged out' } } }; + expect({ first: await first, second: await second }).toEqual({ first: ok, second: ok }); + await settle(service); + expect({ logouts: sock.logouts, me: storedMe() }).toEqual({ logouts: 1, me: [] }); + }); }); From dba9bc9d7e8adba8bc73652233372c2682aaac5b Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:13:34 +0300 Subject: [PATCH 123/157] fix: concurrent logouts share one run and answer with its outcome logoutInstance installs one run (runLogout) before its first wait, and every call awaits that run: a second logout or a delete no longer answers 'done' while the first is in flight, and two calls can no longer both pass hasLinkedSession() and replace each other's state and settle. A pending outcome is recorded (recordPending) on every call, so each caller hears when it cannot be. Visible to consumers: a second DELETE /instance/logout during one in flight answers when the first finishes, with the same outcome (200, or 202 if it turned out pending), instead of 200 at once. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 25 ++++++++++++------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 481cdde3b6..8d67aaa9ad 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -425,13 +425,22 @@ export class BaileysStartupService extends ChannelStartupService { * WhatsApp and is wiped at once. */ public async logoutInstance(): Promise<'done' | 'pending'> { + // One logout at a time: a second call (another client, a delete) waits for the one under way and + // answers with its outcome. Installed before any wait, so two calls can never both start one. + this.logoutRun ??= this.runLogout().finally(() => (this.logoutRun = null)); + const result = await this.logoutRun; + // A 202 promises the logout survives a restart: when that cannot be recorded, the caller hears + // it, and every call tries again. + if (result === 'pending') await this.recordPending(); + return result; + } + + private logoutRun: Promise<'done' | 'pending'> | null = null; + + private async runLogout(): Promise<'done' | 'pending'> { // A connect under way finishes first, so the socket logged out is the one it builds. await this.connecting?.socket.catch(() => undefined); - if (this.logout?.marked) { - await this.recordPending(); - return 'pending'; - } - if (this.logout) return 'done'; + if (this.logout?.marked) return 'pending'; const linked = await this.hasLinkedSession(); this.messageProcessor.onDestroy(); @@ -467,10 +476,8 @@ export class BaileysStartupService extends ChannelStartupService { const late = new Promise<'late'>((r) => (timer = setTimeout(() => r('late'), 10_000))); const result = await Promise.race([outcome, late]); clearTimeout(timer); - if (result === 'late') await this.markLogoutPending(); - // A 202 promises the logout survives a restart: when that cannot be recorded, the caller hears it. - if (result === 'late' || result === 'pending') { - await this.recordPending(); + if (result === 'late') { + await this.markLogoutPending(); return 'pending'; } return result; From 37f49ba23dc90ca9fb4cd6f2e3cd715971ad35e0 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:15:17 +0300 Subject: [PATCH 124/157] test: a logout WhatsApp never confirmed wipes the session, on the socket's own close Baileys' logout() writes remove-companion-device with sendNode, waits for no answer, and ends the socket itself with loggedOut (401). Evolution read that local close as WhatsApp's confirmation and wiped the creds. If the connection fails after the write and before WhatsApp processed it, the device stays linked and nothing can remove it any more. The fake socket now models Baileys more closely: logout() as above, and query() (write, then wait for the answer, failing when the socket closes), with confirmRemove/afterRemove to make WhatsApp silent and drop the connection after the write. Red on the current code: the request is written, WhatsApp never answers, the connection drops, and the logout answers 200 with the creds wiped. Co-Authored-By: Claude Opus 5.5 --- test/helpers/connect.ts | 27 ++++++++++++++++++++++++--- test/instance/logout-pending.test.ts | 25 +++++++++++++++++++++++++ 2 files changed, 49 insertions(+), 3 deletions(-) diff --git a/test/helpers/connect.ts b/test/helpers/connect.ts index e11fa40ad9..f479a1a2ea 100644 --- a/test/helpers/connect.ts +++ b/test/helpers/connect.ts @@ -27,6 +27,7 @@ export function fakeSocket(config?: any) { }; let closed = false; let open = false; + const onClose: (() => void)[] = []; ev.on('connection.update', (update: any) => { if (update.connection === 'close') closed = true; if (update.connection === 'open') open = true; @@ -39,15 +40,31 @@ export function fakeSocket(config?: any) { profilePictureUrl: async () => undefined, /** How many times the socket told WhatsApp to remove this device (remove-companion-device). */ logouts: 0, - // As Baileys' logout(): with a linked device it first tells WhatsApp, which throws when the ws is - // not open (sendNode: 'Connection Closed'); then it ends the socket with loggedOut. + /** Whether WhatsApp answers the remove-companion-device IQ (with type result). */ + confirmRemove: true, + /** Called once a remove-companion-device has been written, e.g. to drop the connection then. */ + afterRemove: undefined as undefined | (() => void), + // As Baileys' logout(): with a linked device it first writes remove-companion-device (sendNode, + // which throws when the ws is not open: 'Connection Closed') without waiting for any answer; then + // it ends the socket itself, locally, with loggedOut. logout: async (msg?: string) => { if (config?.auth?.creds?.me?.id) { if (!open || closed) throw new Boom('Connection Closed', { statusCode: 428 }); sock.logouts++; + sock.afterRemove?.(); } sock.end(new Boom(msg || 'Intentional Logout', { statusCode: 401 })); }, + // As Baileys' query(): write the node, then wait for WhatsApp's answer; the wait fails when the + // socket closes first. Only remove-companion-device is modelled. + query: async (node: any) => { + if (node?.content?.[0]?.tag !== 'remove-companion-device') throw new Error('fake socket: query not modelled'); + if (!open || closed) throw new Boom('Connection Closed', { statusCode: 428 }); + sock.logouts++; + sock.afterRemove?.(); + if (sock.confirmRemove) return { tag: 'iq', attrs: { type: 'result', id: node.attrs.id } }; + return new Promise((_, reject) => onClose.push(() => reject(new Boom('Connection Closed', { statusCode: 428 })))); + }, /** Whether Evolution called end() on this socket. */ ended: false, handlers: () => handlers, @@ -57,6 +74,7 @@ export function fakeSocket(config?: any) { if (closed) return; closed = true; ev.emit('connection.update', { connection: 'close', lastDisconnect: { error, date: new Date() } }); + onClose.splice(0).forEach((f) => f()); }, }; return sock; @@ -74,7 +92,10 @@ export function stubAuthState(service: any) { export type ProxyProtocol = 'http' | 'socks5'; /** Set the proxy the way /proxy/set does, then connect the way Evolution does. Returns the socket config. */ -export async function connectBehind(socketSpy: { mock: { calls: any[][] } }, proxy?: { protocol: ProxyProtocol; port: number }) { +export async function connectBehind( + socketSpy: { mock: { calls: any[][] } }, + proxy?: { protocol: ProxyProtocol; port: number }, +) { const { service } = await makeService(); if (proxy) { await service.setProxy({ diff --git a/test/instance/logout-pending.test.ts b/test/instance/logout-pending.test.ts index effa3c2ffe..76da5669a7 100644 --- a/test/instance/logout-pending.test.ts +++ b/test/instance/logout-pending.test.ts @@ -593,4 +593,29 @@ describe('a logout that cannot reach WhatsApp', () => { await settle(service); expect({ logouts: sock.logouts, me: storedMe() }).toEqual({ logouts: 1, me: [] }); }); + + // Baileys' logout() writes remove-companion-device and then ends the socket itself with loggedOut; + // it never waits for WhatsApp. So that close is not WhatsApp's word: if the connection fails before + // WhatsApp processed the request, the device stays linked, and wiping the creds leaves it on the + // phone for good. + it('is not finished by the socket closing itself: without WhatsApp confirming, it stays pending', async () => { + await linkedInstance(); + const monitor = await startProcess(); + await vi.waitFor(() => expect(built()).toHaveLength(1)); + const service = monitor.waInstances.test; + const sock = current(); + await opened(sock); + // The request is written, then the connection fails before WhatsApp answers. + sock.confirmRemove = false; + sock.afterRemove = () => setTimeout(() => sock.end(new Boom('Connection Terminated', { statusCode: 428 })), 20); + + expect(await call('DELETE', 'logout')).toEqual(PENDING); + expect({ me: storedMe(), pending: service.logoutPending }).toEqual({ me: [WUID], pending: true }); + + // The reconnect: WhatsApp refuses the device, which is its word that it is gone. + const next = await reconnected(2); + next.end(new Boom('Connection Failure', { statusCode: 401 })); + await settle(service); + expect({ me: storedMe(), pending: service.logoutPending }).toEqual({ me: [], pending: false }); + }); }); From a1c69d25044948293ba654d357b7710ef3cae4b1 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:17:14 +0300 Subject: [PATCH 125/157] fix: a logout finishes on WhatsApp's answer, not on the socket closing itself The logout now sends remove-companion-device with query() and waits for WhatsApp's answer (8s), instead of Baileys' logout(), which writes the request, waits for nothing, and ends the socket itself with loggedOut. - WhatsApp answers (IQ result): the socket is ended with loggedOut and the logout finishes, as before. - No answer in time, an error answer, or the connection failing: the socket is ended with connectionClosed (428), the logout is pending, and the next connection says: WhatsApp refusing the device (401) finishes it, an open asks again. The concurrency test held logout() to keep the first request in flight; it now holds query() too, since that is how the request goes out. Not verified against WhatsApp: whether it answers this IQ. If it does not, both paths still end safely (the next connection's 401 finishes it), but an open-connection logout then answers 202 after about 8s instead of 200. Worth a live check before relying on the fast path. Visible to consumers: a logout on an open connection answers 200 only once WhatsApp confirmed it; unconfirmed, it answers 202 PENDING and finishes on the next connection. Co-Authored-By: Claude Opus 5.5 --- FORK.md | 2 +- .../whatsapp/whatsapp.baileys.service.ts | 50 +++++++++++++++++-- test/instance/logout-pending.test.ts | 7 ++- 3 files changed, 52 insertions(+), 7 deletions(-) diff --git a/FORK.md b/FORK.md index c4c7f7004b..759facdbde 100644 --- a/FORK.md +++ b/FORK.md @@ -63,7 +63,7 @@ named where one exists. **Sessions and connections** - A database error never replaces a linked session's credentials with fresh ones, and a failed settings read no longer drops an event batch. -- A logout or delete while the socket is down keeps the session until WhatsApp is told, so the device leaves the phone's Linked devices; it answers `202 PENDING` meanwhile and forwards nothing (#2520, #2508 in part). The pending logout is recorded on the instance's row (`disconnectionObject.logoutPending`) as well as in a file next to the session keys, so a restart that lost the instances volume still finishes it; when it cannot be recorded, the logout or delete answers 500 instead of 202. A deleted instance keeps its database row until then (Session and Proxy cascade from it), out of the API, its name taken and its token cleared. +- A logout or delete while the socket is down keeps the session until WhatsApp is told, so the device leaves the phone's Linked devices; it answers `202 PENDING` meanwhile and forwards nothing (#2520, #2508 in part). The pending logout is recorded on the instance's row (`disconnectionObject.logoutPending`) as well as in a file next to the session keys, so a restart that lost the instances volume still finishes it; when it cannot be recorded, the logout or delete answers 500 instead of 202. A logout is finished only on WhatsApp's word: its answer to remove-companion-device, or its refusal of the device on the next connection; the socket's own close after sending the request (all Baileys' `logout()` waits for) is not one. Concurrent logouts and deletes share one run and answer with its outcome. A deleted instance keeps its database row until then (Session and Proxy cascade from it), out of the API, its name taken and its token cleared. - Reconnects back off from 1s to 60s instead of spinning, the version fetch times out after 10s, and an instance never runs two sockets (#2134, #2184, #2430; ideas from #2732). - Profile pictures are looked up once per contact per hour, at most four at a time, and history never waits on them (#1883). - One pairing code per connect attempt, and a fresh QR budget per attempt (#2100, #2696). diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 8d67aaa9ad..6a86682146 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -131,6 +131,7 @@ import makeWASocket, { prepareWAMessageMedia, Product, proto, + S_WHATSAPP_NET, UserFacingSocketConfig, WABrowserDescription, WAMediaUpload, @@ -365,6 +366,9 @@ export class BaileysStartupService extends ChannelStartupService { // it takes: an instance that gave up would sit disconnected on valid credentials. An open resets it. private static readonly RECONNECT_FIRST_DELAY_MS = 1_000; private static readonly RECONNECT_MAX_DELAY_MS = 60_000; + // How long WhatsApp gets to answer a remove-companion-device before the next connection is asked + // instead; under logoutInstance's own 10s, so a logout that goes unanswered answers 202. + private static readonly REMOVE_ANSWER_TIMEOUT_MS = 8_000; private reconnectAttempts = 0; private reconnectTimer: NodeJS.Timeout | null = null; // The connect under way, so a second one joins it instead of building a second socket. @@ -453,8 +457,9 @@ export class BaileysStartupService extends ChannelStartupService { if (linked) { try { if (this.stateConnection.state !== 'open') throw new Error('the connection is not open'); - // Sends remove-companion-device, then ends the socket with loggedOut: that close finishes it. - await this.client.logout('Log out instance: ' + this.instanceName); + // Confirmed: the socket ends with loggedOut, which finishes it. Not: it ends with a code that + // reconnects, and the logout is pending until the next connection says (logoutUpdate). + await this.removeFromWhatsApp(); } catch (error) { this.logger.warn(`Logout could not reach WhatsApp (${error?.message}): pending until the connection returns`); await this.markLogoutPending(); @@ -620,7 +625,7 @@ export class BaileysStartupService extends ChannelStartupService { if (connection === 'open') { this.reconnectAttempts = 0; try { - await this.client.logout('Log out instance: ' + this.instanceName); + await this.removeFromWhatsApp(); } catch (error) { // The socket dropped before the logout went out: its close reconnects. this.logger.warn(`Pending logout not sent (${error?.message}), trying again on the next connection`); @@ -629,8 +634,9 @@ export class BaileysStartupService extends ChannelStartupService { } if (connection === 'close') { - // loggedOut: the logout went out, or WhatsApp had already removed the device. The other - // final codes leave nothing to log out either. + // loggedOut: WhatsApp confirmed the removal (removeFromWhatsApp then ends the socket so), or + // refused the device on connecting (it is already removed). The other final codes leave + // nothing to log out either. if ([DisconnectReason.loggedOut, DisconnectReason.forbidden, 402, 406].includes(statusCode)) { return this.finishLogout(); } @@ -639,6 +645,40 @@ export class BaileysStartupService extends ChannelStartupService { } } + /** + * Ask WhatsApp to remove this device (remove-companion-device) and wait for its answer. Baileys' + * logout() writes the same request but waits for nothing and then ends the socket itself with + * loggedOut, a close that says nothing about whether WhatsApp got it. Here the socket ends with + * loggedOut only once WhatsApp answered; otherwise (no answer in time, an error answer, the + * connection failing) it ends with a code that reconnects, and the next connection says: WhatsApp + * refusing the device (401) finishes the logout, an open asks again. Throws when the request could + * not be written at all. + */ + private async removeFromWhatsApp() { + const client = this.client; + const jid = this.instance.authState?.state?.creds?.me?.id ?? client.user?.id; + let confirmed = false; + try { + const answer: any = await client.query( + { + tag: 'iq', + attrs: { to: S_WHATSAPP_NET, type: 'set', xmlns: 'md' }, + content: [{ tag: 'remove-companion-device', attrs: { jid, reason: 'user_initiated' } }], + }, + BaileysStartupService.REMOVE_ANSWER_TIMEOUT_MS, + ); + confirmed = answer?.attrs?.type === 'result'; + } catch (error) { + this.logger.warn(`WhatsApp did not confirm the logout (${errorName(error)}): checking on the next connection`); + } + if (client !== this.client) return; + if (confirmed) { + client.end(new Boom('Intentional Logout', { statusCode: DisconnectReason.loggedOut })); + } else { + client.end(new Boom('Logout not confirmed', { statusCode: DisconnectReason.connectionClosed })); + } + } + /** WhatsApp has been told (or has nothing to remove): wipe the session and the marker, then close as a logout does. */ private async finishLogout() { const logout = this.logout; diff --git a/test/instance/logout-pending.test.ts b/test/instance/logout-pending.test.ts index 76da5669a7..eddda5cb2f 100644 --- a/test/instance/logout-pending.test.ts +++ b/test/instance/logout-pending.test.ts @@ -574,11 +574,16 @@ describe('a logout that cannot reach WhatsApp', () => { await opened(sock); let release: () => void; const gate = new Promise((r) => (release = r)); - const logout = sock.logout; + // Hold the request to remove the device, however it is sent. + const { logout, query } = sock; sock.logout = async (msg?: string) => { await gate; return logout(msg); }; + sock.query = async (node: any) => { + await gate; + return query(node); + }; const answered: string[] = []; const first = call('DELETE', 'logout').then((r) => (answered.push('first'), r)); From a20d2d96414a14a6723dcc4cada7713292d30fe3 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:17:55 +0300 Subject: [PATCH 126/157] test: a reload, and a socket built across a shutdown, escape the one-socket guard reloadConnection (after a profile or privacy update) called createClient directly, beside connect()'s serialisation, and shutdown (the instance removed) did not stop a socket already being built. Red on the current code (3 tests): a reload during a connect builds a second socket; a connect held in its build when the instance is shut down goes live afterwards; a reload after shutdown builds a live socket. Co-Authored-By: Claude Opus 5.5 --- test/connect/one-socket.test.ts | 51 +++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/test/connect/one-socket.test.ts b/test/connect/one-socket.test.ts index 8451b1a6cc..d2d116ac9d 100644 --- a/test/connect/one-socket.test.ts +++ b/test/connect/one-socket.test.ts @@ -165,4 +165,55 @@ describe('one instance, one socket', () => { await waitOutAnyReconnect(); expect({ built: built().length, live: live().length }).toEqual({ built: 2, live: 1 }); }); + + // Two more ways a socket was built outside that guard: reloadConnection (after a profile or privacy + // update) called createClient directly, and shutdown (the instance is removed) did not stop a + // socket already being built, which then went live for an instance that no longer exists. + describe('every socket goes through the same guard', () => { + /** Hold the next socket build at its first wait (reading the auth state) until released. */ + function holdNextBuild(s: any) { + let release: () => void; + const gate = new Promise((r) => (release = r)); + const read = s.defineAuthState; + s.defineAuthState = async () => { + await gate; + return read(); + }; + return () => release(); + } + + it('a reload while a connect is being built joins it: one socket', async () => { + const s = await service(); + const release = holdNextBuild(s); + const connect = s.connectToWhatsapp(); + await flush(); + const reload = s.reloadConnection(); + await flush(); + release(); + await Promise.all([connect, reload]); + await waitOutAnyReconnect(); + expect({ built: built().length, live: live().length }).toEqual({ built: 1, live: 1 }); + }); + + it('a socket being built when the instance is shut down never goes live', async () => { + const s = await service(); + const release = holdNextBuild(s); + const connect = s.connectToWhatsapp().catch(() => undefined); + await flush(); + s.shutdown(); + release(); + await connect; + await waitOutAnyReconnect(); + expect({ built: built().length, live: live().length }).toEqual({ built: 0, live: 0 }); + }); + + it('a reload after the instance is shut down builds nothing', async () => { + const s = await service(); + await s.connectToWhatsapp(); + s.shutdown(); + await s.reloadConnection().catch(() => undefined); + await waitOutAnyReconnect(); + expect({ built: built().length, live: live().length }).toEqual({ built: 1, live: 0 }); + }); + }); }); From f147f023b00639a59b156fb7e93d93162d800d8b Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:18:56 +0300 Subject: [PATCH 127/157] fix: a reload goes through the one-at-a-time connect, and nothing is built after a shutdown - reloadConnection now calls connect() instead of createClient(), so it joins a connect under way instead of building a second socket, and it does nothing for an instance shut down or one whose pending logout owns the connection. - shutdown() marks the instance shut down. A socket build checks that after each of its waits (auth state, version fetch) and right before makeWASocket, and stops with ConnectAborted (not logged as a failure, not retried); connect() and scheduleReconnect() refuse too. Visible to consumers: a reload after a profile or privacy update re-reads the instance's settings and proxy like any connect; a profile or privacy update during a pending logout no longer opens a normal socket. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 43 +++++++++++++++++-- 1 file changed, 39 insertions(+), 4 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 6a86682146..b635774fb2 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -323,6 +323,14 @@ const isExpiredMedia = (error: any, media: { url?: string | null; directPath?: s */ export const MEDIA_REUPLOAD_TIMEOUT_MS = 60_000; +/** A socket build stopped because the instance was shut down meanwhile. */ +class ConnectAborted extends Error { + constructor() { + super('The instance was shut down while its connection was being built'); + this.name = 'ConnectAborted'; + } +} + /** The pending logout recordPending wrote on an instance's row (disconnectionObject), if there is one. */ const pendingOnRow = (value: unknown): { deleted: boolean } | undefined => { let object: any = value; @@ -548,12 +556,23 @@ export class BaileysStartupService extends ChannelStartupService { return this.hasLinkedSession(); } - /** The instance is removed from the API: no reconnect, and its socket let go of without its close being handled. */ + /** + * The instance is removed from the API: no reconnect, its socket let go of without its close being + * handled, and no socket built after this, including one being built now (createClient checks). + */ public shutdown() { + this.shutDown = true; this.stopReconnecting(); this.retireClient(); } + private shutDown = false; + + /** A socket build stops at its next step once the instance is shut down. */ + private stillWanted() { + if (this.shutDown) throw new ConnectAborted(); + } + /** Whether the stored session is a linked device (creds carry `me`), i.e. whether WhatsApp has something to remove. */ private async hasLinkedSession(): Promise { const cache = this.configService.get('CACHE'); @@ -1067,7 +1086,10 @@ export class BaileysStartupService extends ChannelStartupService { } private async createClient(number?: string): Promise { - this.instance.authState = await this.defineAuthState(); + this.stillWanted(); + const authState = await this.defineAuthState(); + this.stillWanted(); + this.instance.authState = authState; const session = this.configService.get('CONFIG_SESSION_PHONE'); @@ -1125,6 +1147,7 @@ export class BaileysStartupService extends ChannelStartupService { options ? { httpsAgent: options.fetchAgent, proxy: false } : {}, options ? ({ dispatcher: this.mediaProxy.dispatcher } as RequestInit) : {}, ); + this.stillWanted(); const version = baileysVersion.version; const log = `Baileys version: ${version.join('.')}`; @@ -1192,6 +1215,7 @@ export class BaileysStartupService extends ChannelStartupService { this.endSession = false; + this.stillWanted(); this.retireClient(); this.client = makeWASocket(socketConfig); // Any reconnect still waiting was for the socket just replaced. @@ -1270,6 +1294,7 @@ export class BaileysStartupService extends ChannelStartupService { * reconnect still happens. */ private async connect(number?: string): Promise { + this.stillWanted(); const inFlight = this.connecting; if (inFlight && inFlight.number === (number ?? null)) return inFlight.socket; @@ -1301,6 +1326,8 @@ export class BaileysStartupService extends ChannelStartupService { return await this.createClient(number); } catch (error) { + // Shut down while it was being built: nothing failed, nothing to retry. + if (error instanceof ConnectAborted) throw error; this.logger.error({ message: 'Connect failed', error: errorFields(error) }); // The same 500, still carrying what failed (not enumerable, so not in an HTTP answer): a reconnect logs it. try { @@ -1313,6 +1340,7 @@ export class BaileysStartupService extends ChannelStartupService { private scheduleReconnect(statusCode?: number) { this.stopReconnecting(); + if (this.shutDown) return; const delay = Math.min( BaileysStartupService.RECONNECT_FIRST_DELAY_MS * 2 ** Math.min(this.reconnectAttempts, 16), BaileysStartupService.RECONNECT_MAX_DELAY_MS, @@ -1339,11 +1367,18 @@ export class BaileysStartupService extends ChannelStartupService { } } + /** + * A new socket after a profile or privacy change, through the same one-at-a-time connect as any + * other: it joins a connect under way. Nothing for an instance shut down, or one whose pending + * logout owns the connection. + */ public async reloadConnection(): Promise { + if (this.shutDown || this.logout) return this.client; try { - return await this.createClient(this.phoneNumber); + return await this.connect(this.phoneNumber); } catch (error) { - this.logger.error({ message: 'Reload connection failed', error: errorFields(error) }); + if (error instanceof ConnectAborted) return this.client; + this.logger.error({ message: 'Reload connection failed', error: errorFields(error?.cause ?? error) }); throw new InternalServerErrorException(error?.toString()); } } From 166ff8d62ea7b13e21f943ab6b86e525821edff4 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:20:13 +0300 Subject: [PATCH 128/157] test: batches queued before an instance is shut down still run after it Evolution checks that a batch's socket is the instance's when it queues the batch, not when the batch runs. A batch waiting behind a slow one when the instance is removed still ran afterwards, and a profile picture lookup finishing late still forwarded its update. Red on the current code (2 tests): a queued contacts.upsert is forwarded and stored after shutdown; a picture lookup answered after shutdown sends contacts.update. Co-Authored-By: Claude Opus 5.5 --- test/connect/queued-after-shutdown.test.ts | 54 ++++++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 test/connect/queued-after-shutdown.test.ts diff --git a/test/connect/queued-after-shutdown.test.ts b/test/connect/queued-after-shutdown.test.ts new file mode 100644 index 0000000000..609ae57160 --- /dev/null +++ b/test/connect/queued-after-shutdown.test.ts @@ -0,0 +1,54 @@ +// Evolution handles a socket's batches one at a time, on a queue (eventProcessingQueue): a batch +// can wait there behind a slow one (a large history sync) for a long time. The check that the +// batch's socket is still the instance's ran when the batch was queued, not when it ran. So a +// batch queued before the instance was removed (shut down: DELETE, DEL_INSTANCE) still ran after +// it: webhooks for an instance that no longer exists, and rows written again under it after its +// data was deleted. A lookup finishing late (a profile picture) did the same. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { describe, expect, it } from 'vitest'; + +import { makeService, settle } from '../helpers/baileys-service'; +import { emitted } from '../helpers/fake-server-module'; + +const CONTACT = '972500000001@s.whatsapp.net'; + +describe('an instance shut down while batches wait on its queue', () => { + it('runs none of them afterwards: no webhook, no row', async () => { + emitted.length = 0; + const { service, ev, prisma } = await makeService({ profile: 'stored' }); + service.eventHandler(); + let release: () => void; + service.eventProcessingQueue = new Promise((r) => (release = r)); + + ev.emit('contacts.upsert', [{ id: CONTACT, notify: 'Tal' }]); + service.shutdown(); + release(); + await settle(service); + + expect({ webhooks: emitted.map((e) => e.event), contacts: prisma.contact.rows.length }).toEqual({ + webhooks: [], + contacts: 0, + }); + }); + + it('forwards nothing a lookup finishes after it', async () => { + emitted.length = 0; + const { service, ev } = await makeService(); + let answer: (url: string) => void; + service.client.profilePictureUrl = () => new Promise((r) => (answer = r)); + service.eventHandler(); + ev.emit('contacts.upsert', [{ id: CONTACT, notify: 'Tal' }]); + await settle(service); + await vi.waitFor(() => expect(answer).toBeDefined()); + emitted.length = 0; + + service.shutdown(); + answer('https://pps.whatsapp.net/v/t61/picture.jpg'); + await settle(service); + await new Promise((r) => setTimeout(r, 20)); + expect(emitted.map((e) => e.event)).toEqual([]); + }); +}); From 4e0c39e208c75a2a4a4ee6477de95a0b69bab241 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:21:17 +0300 Subject: [PATCH 129/157] fix: an instance shut down runs no queued batch and forwards nothing that finishes late - A queued batch checks, when it runs, whether the instance was shut down meanwhile, and does nothing if so. - sendDataWebhook forwards nothing once the instance is shut down (a picture lookup, an open's handler or any other late work), except the monitor's own remove.instance announcement, sent after the shutdown (new test guards it). Not applied, on purpose: dropping a batch whose socket a reconnect of the same session replaced while it waited. Its messages were decrypted, receipted and acknowledged to WhatsApp, which will not send them again, so dropping them would lose them. A logout already drops batches at run time (the logout branch), and the queue keeps a logout's close behind the batches queued before it. Visible to consumers: no webhook (and no stored row) arrives from an instance after it was removed. Co-Authored-By: Claude Opus 5.5 --- .../channel/whatsapp/whatsapp.baileys.service.ts | 11 +++++++++-- test/connect/queued-after-shutdown.test.ts | 8 ++++++++ 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index b635774fb2..46a623782d 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -746,7 +746,9 @@ export class BaileysStartupService extends ChannelStartupService { logout.settle('done'); } - // While a logout is under way the instance forwards nothing. + // While a logout is under way, and once the instance is shut down (removed), it forwards nothing: + // work that finishes late (a queued batch, a picture lookup, an open's handler) included. The + // one exception is the removal's own announcement, which the monitor sends after the shutdown. public async sendDataWebhook( event: Events, data: T, @@ -754,7 +756,7 @@ export class BaileysStartupService extends ChannelStartupService { integration?: string[], extra?: Record, ) { - if (this.logout) return; + if (this.logout || (this.shutDown && event !== Events.REMOVE_INSTANCE)) return; this.liveRecorder?.webhook(event, data, extra); return super.sendDataWebhook(event, data, local, integration, extra); } @@ -2571,6 +2573,11 @@ export class BaileysStartupService extends ChannelStartupService { // Events a replaced socket still emits do not drive the instance. if (client !== this.client) return; this.eventProcessingQueue = this.eventProcessingQueue.then(async () => { + // Checked again when the batch runs, which can be long after it was queued: an instance shut + // down meanwhile (removed) handles nothing more. A batch of a socket replaced meanwhile by a + // reconnect of the same session still runs: its messages were delivered and acknowledged to + // WhatsApp, which will not send them again. + if (this.shutDown) return; try { // A logout under way: nothing is forwarded or stored; the connection only delivers the logout. if (this.logout) { diff --git a/test/connect/queued-after-shutdown.test.ts b/test/connect/queued-after-shutdown.test.ts index 609ae57160..913b7b632f 100644 --- a/test/connect/queued-after-shutdown.test.ts +++ b/test/connect/queued-after-shutdown.test.ts @@ -51,4 +51,12 @@ describe('an instance shut down while batches wait on its queue', () => { await new Promise((r) => setTimeout(r, 20)); expect(emitted.map((e) => e.event)).toEqual([]); }); + + it('still announces its own removal, which the monitor sends after shutting it down', async () => { + emitted.length = 0; + const { service } = await makeService(); + service.shutdown(); + await service.sendDataWebhook('remove.instance', null); + expect(emitted.map((e) => e.event)).toEqual(['remove.instance']); + }); }); From 465d8748873ae3189f6ae58b93d43d59346afc75 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:22:05 +0300 Subject: [PATCH 130/157] test: a late picture lookup brings back an old name, and a removed picture contactPictures sends a history batch's pictures on contacts.update once all its lookups finish, with the name each contact had in the batch; and profilePicture writes its answer to the cache whatever happened while it waited. Red on the current code (2 tests): a contact renamed while its batch's lookups were running ends, for a consumer applying the webhooks in order, with the old name; a lookup answered after WhatsApp's 'removed' picture notification puts the old picture back on the consumer and in the cache. Co-Authored-By: Claude Opus 5.5 --- test/handlers/late-pictures.test.ts | 89 +++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) create mode 100644 test/handlers/late-pictures.test.ts diff --git a/test/handlers/late-pictures.test.ts b/test/handlers/late-pictures.test.ts new file mode 100644 index 0000000000..784408551b --- /dev/null +++ b/test/handlers/late-pictures.test.ts @@ -0,0 +1,89 @@ +// A contact's picture is looked up after its contacts.upsert, in the background, and sent on +// contacts.update when the lookups of the whole batch finish. That update carried the name the +// contact had in the batch, so a rename that arrived meanwhile was overwritten for the consumer +// by the old name. And a lookup answered after WhatsApp said the picture was removed wrote the +// old picture back, in the cache and on the update. Each test applies the webhooks in the order +// they were sent, as a consumer does, and checks where the consumer ends. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { beforeEach, describe, expect, it } from 'vitest'; + +import { deliver, makeService, settle } from '../helpers/baileys-service'; +import { emitted } from '../helpers/fake-server-module'; + +const A = '972500000001@s.whatsapp.net'; +const B = '972500000002@s.whatsapp.net'; +const OLD_PICTURE = 'https://pps.whatsapp.test/a/old.jpg'; + +/** A profilePictureUrl whose answers the test gives, one call at a time. */ +function heldPictures(service: any) { + const calls: { jid: string; answer: (url: string | null) => void }[] = []; + service.client.profilePictureUrl = (jid: string) => + new Promise((answer) => calls.push({ jid, answer })); + const next = async (jid: string) => { + await vi.waitFor(() => expect(calls.some((c) => c.jid === jid)).toBe(true)); + const i = calls.findIndex((c) => c.jid === jid); + return calls.splice(i, 1)[0].answer; + }; + return { next }; +} + +/** What a consumer holds for a contact after applying every contacts webhook in order. */ +function consumerView(jid: string) { + const view: Record = {}; + for (const e of emitted.filter((e) => e.event === 'contacts.upsert' || e.event === 'contacts.update')) { + for (const item of [].concat(e.data) as any[]) { + if (item?.remoteJid !== jid) continue; + for (const [k, v] of Object.entries(item)) if (v !== undefined) view[k] = v; + } + } + return view; +} + +describe('a picture lookup that finishes late', () => { + beforeEach(() => void emitted.splice(0)); + + it('does not bring back a name the contact has changed since', async () => { + const { service, ev } = await makeService(); + const pictures = heldPictures(service); + await deliver(service, ev, { + 'contacts.upsert': [ + { id: A, name: 'Old' }, + { id: B, name: 'Bea' }, + ], + }); + const answerA = await pictures.next(A); + const answerB = await pictures.next(B); + + // A is renamed while B's picture is still being looked up. + answerA('https://pps.whatsapp.test/a/1.jpg'); + const rename = deliver(service, ev, { 'contacts.update': [{ id: A, name: 'New' }] }, { buffered: false }); + await rename; + answerB('https://pps.whatsapp.test/b/1.jpg'); + await settle(service); + await new Promise((r) => setTimeout(r, 20)); + + expect(consumerView(A).pushName).toBe('New'); + }); + + it('does not bring back a picture WhatsApp said was removed', async () => { + const { service, ev } = await makeService(); + const pictures = heldPictures(service); + await deliver(service, ev, { 'contacts.upsert': [{ id: A, name: 'Ann' }] }); + const answerA = await pictures.next(A); + + // WhatsApp's picture notification: removed. The lookup started before it answers after it. + await deliver(service, ev, { 'contacts.update': [{ id: A, imgUrl: 'removed' }] }, { buffered: false }); + answerA(OLD_PICTURE); + await settle(service); + await new Promise((r) => setTimeout(r, 20)); + + const next = await service.cachedProfilePicture(A); + expect({ consumer: consumerView(A).profilePicUrl, kept: next.profilePictureUrl }).toEqual({ + consumer: null, + kept: null, + }); + }); +}); From caf5fa47c7cd0c8dfecc58f3629ff8b38cc094a4 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:23:35 +0300 Subject: [PATCH 131/157] fix: a late picture lookup carries the contact's newest name, and never outlives a picture notification - contactPictures sends each contact's newest name: a rename that arrives (contacts.update) while the batch's lookups run replaces the batch's name for that update and for Chatwoot. The payload keeps its shape (remoteJid, pushName, profilePicUrl, instanceId), which the recorded live fixture holds it to. - WhatsApp's picture notification (imgUrl 'changed' or 'removed') bumps a per-contact version and drops the kept picture and the lookup under way. A lookup started before it neither writes the cache nor is sent on the batch's update. Visible to consumers: a history batch's picture update no longer carries a stale name or a picture WhatsApp said was removed; otherwise the same. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 68 ++++++++++++++++--- 1 file changed, 59 insertions(+), 9 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 46a623782d..40d04c71e3 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -1508,6 +1508,10 @@ export class BaileysStartupService extends ChannelStartupService { for await (const contact of contacts) { this.logger.debug(`Updating contact: ${JSON.stringify(contact, null, 2)}`); // imgUrl is set only by WhatsApp's picture notification: 'changed' or 'removed'. + if (contact.imgUrl === 'changed' || contact.imgUrl === 'removed') this.invalidatePicture(createJid(contact.id)); + const renamed = contact?.name ?? contact?.verifiedName; + const awaiting = this.namesAwaitingPicture.get(contact.id); + if (renamed && awaiting) awaiting.name = renamed; if (contact.imgUrl === 'removed') this.pictureCache.set(createJid(contact.id), { url: null, at: Date.now() }); contactsRaw.push({ remoteJid: contact.id, @@ -2762,17 +2766,41 @@ export class BaileysStartupService extends ChannelStartupService { ); } - /** Look up the pictures of contacts from contacts.upsert (history, address book) and send them on contacts.update. */ + // The newest name of each contact whose picture contactPictures is still looking up: a rename + // that arrives meanwhile (contacts.update) is what that update must carry, not the batch's name. + private readonly namesAwaitingPicture = new Map(); + + /** + * Look up the pictures of contacts from contacts.upsert (history, address book) and send them on + * contacts.update, with each contact's newest name (a rename that arrived while the lookups ran + * wins over the batch's). A contact whose picture WhatsApp said changed or was removed while its + * lookup ran is left out: that notification's own update is the newer one. + */ private async contactPictures(contacts: Contact[]) { + const names = contacts.map((contact) => { + const name = contact?.name || contact?.verifiedName || contact.id.split('@')[0]; + const entry = this.namesAwaitingPicture.get(contact.id) ?? { name, refs: 0 }; + entry.name = name; + entry.refs++; + this.namesAwaitingPicture.set(contact.id, entry); + return entry; + }); try { - const updatedContacts = await Promise.all( - contacts.map(async (contact) => ({ + const looked = await Promise.all( + contacts.map(async (contact) => { + const version = this.pictureVersion(createJid(contact.id)); + const { profilePictureUrl } = await this.cachedProfilePicture(contact.id, { bulk: true }); + return { contact, version, profilePictureUrl }; + }), + ); + const updatedContacts = looked + .filter(({ contact, version }) => this.pictureVersion(createJid(contact.id)) === version) + .map(({ contact, profilePictureUrl }) => ({ remoteJid: contact.id, - pushName: contact?.name || contact?.verifiedName || contact.id.split('@')[0], - profilePicUrl: (await this.cachedProfilePicture(contact.id, { bulk: true })).profilePictureUrl, + pushName: this.namesAwaitingPicture.get(contact.id)?.name, + profilePicUrl: profilePictureUrl, instanceId: this.instanceId, - })), - ); + })); if (updatedContacts.length > 0) { const usersContacts = updatedContacts.filter((c) => c.remoteJid.includes('@s.whatsapp')); @@ -2809,12 +2837,23 @@ export class BaileysStartupService extends ChannelStartupService { } } catch (error) { this.logger.error(`Error: ${error.message}`); + } finally { + for (const [i, contact] of contacts.entries()) { + const entry = names[i]; + if (--entry.refs <= 0 && this.namesAwaitingPicture.get(contact.id) === entry) { + this.namesAwaitingPicture.delete(contact.id); + } + } } } - /** Asks WhatsApp now (an explicit request), and keeps the answer for the event handlers. */ + /** + * Asks WhatsApp now (an explicit request), and keeps the answer for the event handlers, unless + * WhatsApp said the picture changed or was removed while it was asking: that is newer. + */ public async profilePicture(number: string) { const jid = createJid(number); + const version = this.pictureVersion(jid); let profilePictureUrl: string | null; try { @@ -2823,10 +2862,21 @@ export class BaileysStartupService extends ChannelStartupService { profilePictureUrl = null; } - this.pictureCache.set(jid, { url: profilePictureUrl, at: Date.now() }); + if (this.pictureVersion(jid) === version) this.pictureCache.set(jid, { url: profilePictureUrl, at: Date.now() }); return { wuid: jid, profilePictureUrl }; } + /** Bumped by WhatsApp's picture notification, so a lookup started before it cannot overwrite it. */ + private readonly pictureVersions = new Map(); + private pictureVersion(jid: string) { + return this.pictureVersions.get(jid) ?? 0; + } + private invalidatePicture(jid: string) { + this.pictureVersions.set(jid, this.pictureVersion(jid) + 1); + this.pictureCache.delete(jid); + this.pictureLookups.delete(jid); + } + /** * The picture the event handlers report: kept for PICTURE_TTL_MS, one lookup per jid at a time. * Bulk lookups (history, address book) queue behind each other; a live event's lookup never joins one. From d2e163c8722545ce4f685cc9f66f25c234af1287 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:24:28 +0300 Subject: [PATCH 132/157] test: a proxied instance can still reach WhatsApp from the server address loadProxy switched the proxy off before reading the Proxy row; a failed proxyscrape list fetch switched it off before the socket was built; and a media download for a proxyscrape proxy with no socket exit to share returned no dispatcher. Red on the current code (4 tests): a download during a proxy reload goes directly; after a failed Proxy read every download goes directly; a connect whose proxyscrape list is down builds a socket with no agent; a proxyscrape download with no socket exit goes directly. Co-Authored-By: Claude Opus 5.5 --- test/proxy/proxy-never-falls-back.test.ts | 153 ++++++++++++++++++++++ 1 file changed, 153 insertions(+) create mode 100644 test/proxy/proxy-never-falls-back.test.ts diff --git a/test/proxy/proxy-never-falls-back.test.ts b/test/proxy/proxy-never-falls-back.test.ts new file mode 100644 index 0000000000..3a74bb0f75 --- /dev/null +++ b/test/proxy/proxy-never-falls-back.test.ts @@ -0,0 +1,153 @@ +// An instance with a proxy must never reach WhatsApp from the server's own address, not for a +// moment. Three ways it still could: +// - loadProxy (every connect) switched the proxy off, then read the Proxy row: a media download +// in that window (a reconnect while messages arrive) went out directly, and a failed read left +// the proxy off for good. +// - A proxyscrape list that could not be fetched switched the proxy off, and the socket connected +// directly. +// - A media download for a proxyscrape proxy with no socket exit to share went out directly. +import { vi } from 'vitest'; + +const { socketSpy } = vi.hoisted(() => ({ socketSpy: vi.fn() })); + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +vi.mock('baileys', async (importOriginal) => { + const orig = await importOriginal(); + return { ...orig, default: socketSpy, makeWASocket: socketSpy }; +}); +vi.mock('@utils/fetchLatestWaWebVersion', () => ({ + fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), +})); + +import http from 'node:http'; +import type { AddressInfo } from 'node:net'; +import { readFile, rm } from 'node:fs/promises'; + +import { encryptedStream } from 'baileys'; +import { getGlobalDispatcher, MockAgent, setGlobalDispatcher } from 'undici'; +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from 'vitest'; + +import { makeService } from '../helpers/baileys-service'; +import { fakeSocket, stubAuthState } from '../helpers/connect'; +import { type Listening, loopbackOnly, startCdn, startHttpProxy } from '../helpers/local-net'; + +socketSpy.mockImplementation(fakeSocket); + +const PLAIN = Buffer.from('a photo, as the person sent it '.repeat(200)); +const PATH = '/v/t62.7118-24/media.enc'; + +let cdn: Listening; +let proxy: Listening; +let mediaKey: Uint8Array; +let netGuard: ReturnType; +const previousDispatcher = getGlobalDispatcher(); +const guard = new MockAgent(); +guard.disableNetConnect(); +guard.enableNetConnect((host: string) => host.startsWith('127.0.0.1:')); + +// A proxyscrape-style list server that is down: every request answers 503. +let listServer: http.Server; +let listUrl: string; + +beforeAll(async () => { + netGuard = loopbackOnly(); + setGlobalDispatcher(guard); + const enc = await encryptedStream(PLAIN, 'image', {}); + mediaKey = enc.mediaKey; + const body = await readFile(enc.encFilePath); + await rm(enc.encFilePath, { force: true }); + cdn = await startCdn({ [PATH]: body }); + proxy = await startHttpProxy(); + listServer = http.createServer((_req, res) => res.writeHead(503).end()); + await new Promise((r) => listServer.listen(0, '127.0.0.1', r)); + listUrl = `http://127.0.0.1:${(listServer.address() as AddressInfo).port}/proxyscrape/v2/list`; +}); + +afterAll(async () => { + await cdn.close(); + await proxy.close(); + await new Promise((r) => listServer.close(r)); + setGlobalDispatcher(previousDispatcher); + await guard.close(); + expect(netGuard.refused).toEqual([]); + netGuard.restore(); +}); + +beforeEach(() => { + cdn.log.splice(0); + proxy.log.splice(0); + socketSpy.mockClear(); +}); +afterEach(() => vi.restoreAllMocks()); + +const imageMessage = () => ({ + key: { remoteJid: '972500000001@s.whatsapp.net', fromMe: false, id: '3EB0BBBBBBBBBBBBBBBB' }, + message: { + imageMessage: { + url: `http://127.0.0.1:${cdn.port}${PATH}`, + mediaKey, + mimetype: 'image/jpeg', + fileLength: PLAIN.length, + }, + }, + messageTimestamp: 1_700_000_000, +}); + +async function serviceBehind(host = '127.0.0.1', port = String(proxy.port)) { + const made = await makeService(); + await made.service.setProxy({ enabled: true, host, port, protocol: 'http', username: '', password: '' }); + await made.service.loadProxy(); + return made; +} + +/** A download, and where it went: through the proxy, directly, or nowhere (it failed). */ +async function download(service: any) { + const result = await service + .getBase64FromMediaMessage({ message: imageMessage() }) + .then(() => 'ok') + .catch(() => 'failed'); + return { result, viaProxy: proxy.log.length > 0, reachedCdn: cdn.log.length > 0 }; +} + +describe('a proxied instance never leaves from the server address', () => { + it('a media download while the proxy is being reloaded still goes through it', async () => { + const { service, prisma } = await serviceBehind(); + let release: () => void; + const held = new Promise((r) => (release = r)); + const read = prisma.proxy.findUnique; + prisma.proxy.findUnique = async (args: any) => { + await held; + return read(args); + }; + const reload = service.loadProxy(); + const during = await download(service); + release(); + await reload; + expect(during).toEqual({ result: 'ok', viaProxy: true, reachedCdn: true }); + }); + + it('a proxy row that cannot be read leaves the proxy in force', async () => { + const { service, prisma } = await serviceBehind(); + prisma.proxy.findUnique = async () => { + throw new Error("Can't reach database server"); + }; + await service.loadProxy().catch(() => undefined); + expect(await download(service)).toEqual({ result: 'ok', viaProxy: true, reachedCdn: true }); + }); + + it('a proxyscrape list that cannot be fetched fails the connect instead of connecting directly', async () => { + const { service } = await serviceBehind(listUrl, '80'); + stubAuthState(service); + const outcome = await service.connectToWhatsapp().then( + () => 'connected', + () => 'failed', + ); + const direct = socketSpy.mock.calls.filter(([config]) => !config?.agent).length; + expect({ outcome, direct }).toEqual({ outcome: 'failed', direct: 0 }); + }); + + it('a media download for a proxyscrape proxy with no exit to share fails instead of going directly', async () => { + const { service } = await serviceBehind(listUrl, '80'); + expect(await download(service)).toEqual({ result: 'failed', viaProxy: false, reachedCdn: false }); + }); +}); From d65032f142cb205a70db753ab4892a09657944bf Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:25:11 +0300 Subject: [PATCH 133/157] fix: a proxied instance fails a connect or a download rather than leave from the server address - loadProxy builds the new configuration apart and puts it in force in one step after reading the Proxy row; until then the one in force stays, and a failed read changes nothing. - A proxyscrape list that cannot be fetched, or is empty, fails the connect (retried by the reconnect) instead of disabling the proxy. - A media download for a proxyscrape proxy uses the socket's own exit even after the configuration was reloaded, and fails (400) when there is none yet, instead of going directly. Visible to consumers: with a proxyscrape proxy whose list is down, the instance stays disconnected (and keeps retrying) instead of connecting from the server's address; a download for such an instance before it has connected answers 400. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 25 +++++++++----- src/api/services/channel.service.ts | 33 +++++++++++-------- 2 files changed, 37 insertions(+), 21 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 40d04c71e3..aaad9306dd 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -1116,15 +1116,20 @@ export class BaileysStartupService extends ChannelStartupService { let proxy: Parameters[0]; if (this.localProxy?.host?.includes('proxyscrape')) { + // No list, no exit: the connect fails (and is retried) rather than leave from the server's address. + let proxyUrls: string[]; try { const response = await axios.get(this.localProxy?.host); - const text = response.data; - const proxyUrls = text.split('\r\n'); - const rand = Math.floor(Math.random() * Math.floor(proxyUrls.length)); - proxy = 'http://' + proxyUrls[rand]; - } catch { - this.localProxy.enabled = false; + proxyUrls = String(response.data ?? '') + .split('\r\n') + .filter((line) => line.trim()); + } catch (error) { + throw new Error( + `The proxy list could not be fetched (${errorName(error)}): not connecting without the proxy`, + ); } + if (!proxyUrls.length) throw new Error('The proxy list is empty: not connecting without the proxy'); + proxy = 'http://' + proxyUrls[Math.floor(Math.random() * proxyUrls.length)]; } else { proxy = { host: this.localProxy.host, @@ -4712,8 +4717,12 @@ export class BaileysStartupService extends ChannelStartupService { if (!this.localProxy?.enabled || !this.localProxy.host) return {}; const key = this.proxyKey(); if (this.mediaProxy?.key !== key) { - // A proxyscrape host is a list the socket picked one exit from; only the socket's own dispatcher is that exit. - if (this.localProxy.host.includes('proxyscrape')) return {}; + // A proxyscrape host is a list the socket picked one exit from; only the socket's own dispatcher + // is that exit. Without one, the download fails rather than leave from the server's address. + if (this.localProxy.host.includes('proxyscrape')) { + if (this.mediaProxy) return { options: { dispatcher: this.mediaProxy.dispatcher } as RequestInit }; + throw new BadRequestException('No proxy exit for this download yet: connect the instance first'); + } this.mediaProxy = { key, dispatcher: makeProxyAgentUndici({ diff --git a/src/api/services/channel.service.ts b/src/api/services/channel.service.ts index 56bec08021..2cca82660d 100644 --- a/src/api/services/channel.service.ts +++ b/src/api/services/channel.service.ts @@ -362,17 +362,22 @@ export class ChannelStartupService { } } + /** + * Read the instance's proxy (the global PROXY_* configuration, then its Proxy row) and put it in + * force at once. The one in force stays until the new one is read: a media download while this + * waits for the database must not leave directly, and a failed read (thrown) changes nothing. + */ public async loadProxy() { - this.localProxy.enabled = false; + const next: Partial = { enabled: false }; const proxyConfig = this.configService.get('PROXY'); if (proxyConfig.HOST) { - this.localProxy.enabled = true; - this.localProxy.host = proxyConfig.HOST; - this.localProxy.port = proxyConfig.PORT || '80'; - this.localProxy.protocol = proxyConfig.PROTOCOL || 'http'; - this.localProxy.username = proxyConfig.USERNAME; - this.localProxy.password = proxyConfig.PASSWORD; + next.enabled = true; + next.host = proxyConfig.HOST; + next.port = proxyConfig.PORT || '80'; + next.protocol = proxyConfig.PROTOCOL || 'http'; + next.username = proxyConfig.USERNAME; + next.password = proxyConfig.PASSWORD; } const data = await this.prismaRepository.proxy.findUnique({ @@ -382,13 +387,15 @@ export class ChannelStartupService { }); if (data?.enabled) { - this.localProxy.enabled = true; - this.localProxy.host = data?.host; - this.localProxy.port = data?.port; - this.localProxy.protocol = data?.protocol; - this.localProxy.username = data?.username; - this.localProxy.password = data?.password; + next.enabled = true; + next.host = data?.host; + next.port = data?.port; + next.protocol = data?.protocol; + next.username = data?.username; + next.password = data?.password; } + + Object.assign(this.localProxy, next); } public async setProxy(data: ProxyDto) { From 88102c74367721a2bfdd41513733ede75222a788 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:26:46 +0300 Subject: [PATCH 134/157] test: errorFields keeps a quoted text and a formatted phone number from an error message An exception's message is arbitrary text: JSON.parse quotes the input it failed on, and a library error can carry a number as a person typed it, with spaces or dashes. errorFields masked URLs, JIDs and runs of 6+ digits only. Red on the current code: a JSON.parse error keeps the quoted text, and +972 54-111-2233 / 054-111-2233 survive. Co-Authored-By: Claude Opus 5.5 --- test/handlers/log-privacy.test.ts | 365 ++++++++++++++++++------------ 1 file changed, 224 insertions(+), 141 deletions(-) diff --git a/test/handlers/log-privacy.test.ts b/test/handlers/log-privacy.test.ts index 62dc873485..aa66caa031 100644 --- a/test/handlers/log-privacy.test.ts +++ b/test/handlers/log-privacy.test.ts @@ -44,12 +44,22 @@ function leaks(out: string) { const found: string[] = []; for (const [what, secret] of Object.entries(SECRETS)) { const line = out.split('\n').find((l) => l.includes(secret)); - if (line !== undefined) found.push(`${what}: ${line.replace(/\x1b\[[0-9;]*m/g, '').trim().slice(0, 160)}`); + if (line !== undefined) + found.push( + `${what}: ${line + .replace(/\x1b\[[0-9;]*m/g, '') + .trim() + .slice(0, 160)}`, + ); } return found; } -const incoming = (id: string, extra: Record = {}) => ({ ...msg(SENDER, id, TEXT).message, pushName: NAME, ...extra }); +const incoming = (id: string, extra: Record = {}) => ({ + ...msg(SENDER, id, TEXT).message, + pushName: NAME, + ...extra, +}); /** Run createClient() against the fake socket, so the ws handlers and the Baileys logger are Evolution's own. */ async function connect(service: any, ev: any) { @@ -63,164 +73,237 @@ async function connect(service: any, ev: any) { return { socket: service.client, config }; } -describe.each(['minimal', 'stored'] as Profile[])('nothing a message carries reaches the logs (profile %s)', (profile) => { - it('an ordinary incoming message', async () => { - const { service, ev } = await makeService({ profile }); - const out = await captureOutput(() => deliver(service, ev, { 'messages.upsert': { messages: [incoming('M1')], type: 'notify' } })); - expect(leaks(out)).toEqual([]); - }); +describe.each(['minimal', 'stored'] as Profile[])( + 'nothing a message carries reaches the logs (profile %s)', + (profile) => { + it('an ordinary incoming message', async () => { + const { service, ev } = await makeService({ profile }); + const out = await captureOutput(() => + deliver(service, ev, { 'messages.upsert': { messages: [incoming('M1')], type: 'notify' } }), + ); + expect(leaks(out)).toEqual([]); + }); - it('an incoming message whose push name renames a known chat, when the chat update fails', async () => { - const { service, prisma, ev } = await makeService({ profile }); - prisma.chat.rows.push({ id: 'chat-1', remoteJid: SENDER, instanceId: 'inst-1', name: 'Old name' }); - prisma.chat.update = async () => { - throw new Error('database unavailable'); - }; - const out = await captureOutput(() => deliver(service, ev, { 'messages.upsert': { messages: [incoming('M2')], type: 'notify' } })); - expect(leaks(out)).toEqual([]); - }); + it('an incoming message whose push name renames a known chat, when the chat update fails', async () => { + const { service, prisma, ev } = await makeService({ profile }); + prisma.chat.rows.push({ id: 'chat-1', remoteJid: SENDER, instanceId: 'inst-1', name: 'Old name' }); + prisma.chat.update = async () => { + throw new Error('database unavailable'); + }; + const out = await captureOutput(() => + deliver(service, ev, { 'messages.upsert': { messages: [incoming('M2')], type: 'notify' } }), + ); + expect(leaks(out)).toEqual([]); + }); - it('a message the phone re-sent on request (placeholder resend)', async () => { - const { service, ev } = await makeService({ profile }); - const out = await captureOutput(() => - deliver(service, ev, { 'messages.upsert': { messages: [incoming('M3')], type: 'notify', requestId: 'R1' } }), - ); - expect(leaks(out)).toEqual([]); - }); + it('a message the phone re-sent on request (placeholder resend)', async () => { + const { service, ev } = await makeService({ profile }); + const out = await captureOutput(() => + deliver(service, ev, { 'messages.upsert': { messages: [incoming('M3')], type: 'notify', requestId: 'R1' } }), + ); + expect(leaks(out)).toEqual([]); + }); - it('a history batch', async () => { - const { service, ev } = await makeService({ profile }); - const event = historyEvent({ - syncType: 3, // RECENT - conversations: [{ id: SENDER, name: NAME, messages: [msg(SENDER, 'H1', TEXT, { pushName: NAME })] }], - pushnames: [{ id: SENDER, pushname: NAME }], + it('a history batch', async () => { + const { service, ev } = await makeService({ profile }); + const event = historyEvent({ + syncType: 3, // RECENT + conversations: [{ id: SENDER, name: NAME, messages: [msg(SENDER, 'H1', TEXT, { pushName: NAME })] }], + pushnames: [{ id: SENDER, pushname: NAME }], + }); + const out = await captureOutput(() => deliver(service, ev, { 'messaging-history.set': event })); + expect(leaks(out)).toEqual([]); }); - const out = await captureOutput(() => deliver(service, ev, { 'messaging-history.set': event })); - expect(leaks(out)).toEqual([]); - }); - it('an on-demand history sync', async () => { - const { service, ev } = await makeService({ profile }); - const event = historyEvent({ - syncType: 6, // ON_DEMAND - conversations: [{ id: SENDER, messages: [msg(SENDER, 'H2', TEXT, { pushName: NAME })] }], + it('an on-demand history sync', async () => { + const { service, ev } = await makeService({ profile }); + const event = historyEvent({ + syncType: 6, // ON_DEMAND + conversations: [{ id: SENDER, messages: [msg(SENDER, 'H2', TEXT, { pushName: NAME })] }], + }); + const out = await captureOutput(() => deliver(service, ev, { 'messaging-history.set': event })); + expect(leaks(out)).toEqual([]); }); - const out = await captureOutput(() => deliver(service, ev, { 'messaging-history.set': event })); - expect(leaks(out)).toEqual([]); - }); - it('a message that failed to decrypt (No session record), through Baileys and then Evolution', async () => { - const { service, ev } = await makeService({ profile }); - const { socket, config } = await connect(service, ev); - const stanza = { - tag: 'message', - attrs: { from: SENDER, id: 'D1', t: '1700000000', type: 'text', notify: NAME }, - content: [{ tag: 'enc', attrs: { v: '2', type: 'msg' }, content: new Uint8Array([1, 2, 3]) }], - }; - const repository = { - lidMapping: { getLIDForPN: async () => null, storeLIDPNMappings: async () => undefined }, - decryptMessage: async () => { - throw new Error('No session record'); - }, - }; - const out = await captureOutput(async () => { - const { fullMessage, decrypt } = decryptMessageNode(stanza as any, socket.user.id, undefined as any, repository as any, config.logger); - await decrypt(); - expect(fullMessage.messageStubParameters).toEqual(['No session record']); - await deliver(service, ev, { 'messages.upsert': { messages: [fullMessage], type: 'notify' } }); - await new Promise((r) => setTimeout(r, 20)); // pino's async write + it('a message that failed to decrypt (No session record), through Baileys and then Evolution', async () => { + const { service, ev } = await makeService({ profile }); + const { socket, config } = await connect(service, ev); + const stanza = { + tag: 'message', + attrs: { from: SENDER, id: 'D1', t: '1700000000', type: 'text', notify: NAME }, + content: [{ tag: 'enc', attrs: { v: '2', type: 'msg' }, content: new Uint8Array([1, 2, 3]) }], + }; + const repository = { + lidMapping: { getLIDForPN: async () => null, storeLIDPNMappings: async () => undefined }, + decryptMessage: async () => { + throw new Error('No session record'); + }, + }; + const out = await captureOutput(async () => { + const { fullMessage, decrypt } = decryptMessageNode( + stanza as any, + socket.user.id, + undefined as any, + repository as any, + config.logger, + ); + await decrypt(); + expect(fullMessage.messageStubParameters).toEqual(['No session record']); + await deliver(service, ev, { 'messages.upsert': { messages: [fullMessage], type: 'notify' } }); + await new Promise((r) => setTimeout(r, 20)); // pino's async write + }); + expect(leaks(out)).toEqual([]); }); - expect(leaks(out)).toEqual([]); - }); - it('a message Baileys fails to handle (its "error in handling message" log)', async () => { - const { service, ev } = await makeService({ profile }); - const { config } = await connect(service, ev); - const node = { tag: 'message', attrs: { from: SENDER, id: 'E1', notify: NAME }, content: [{ tag: 'body', attrs: {}, content: TEXT }] }; - // The call Baileys makes at messages-recv.js:1436 (rc14). - const out = await captureOutput(async () => { - config.logger.error({ error: new Error('boom'), node: binaryNodeToString(node as any) }, 'error in handling message'); - await new Promise((r) => setTimeout(r, 20)); + it('a message Baileys fails to handle (its "error in handling message" log)', async () => { + const { service, ev } = await makeService({ profile }); + const { config } = await connect(service, ev); + const node = { + tag: 'message', + attrs: { from: SENDER, id: 'E1', notify: NAME }, + content: [{ tag: 'body', attrs: {}, content: TEXT }], + }; + // The call Baileys makes at messages-recv.js:1436 (rc14). + const out = await captureOutput(async () => { + config.logger.error( + { error: new Error('boom'), node: binaryNodeToString(node as any) }, + 'error in handling message', + ); + await new Promise((r) => setTimeout(r, 20)); + }); + expect(leaks(out)).toEqual([]); }); - expect(leaks(out)).toEqual([]); - }); - it('a message status update', async () => { - const { service, ev } = await makeService({ profile }); - const out = await captureOutput(() => - deliver(service, ev, { 'messages.update': [{ key: { remoteJid: SENDER, fromMe: true, id: `S-${profile}` }, update: { status: 4 } }] }), - ); - expect(leaks(out)).toEqual([]); - // Nothing about an ordinary status update is an error or a warning. - if (profile === 'minimal') expect(out).toBe(''); - }); + it('a message status update', async () => { + const { service, ev } = await makeService({ profile }); + const out = await captureOutput(() => + deliver(service, ev, { + 'messages.update': [{ key: { remoteJid: SENDER, fromMe: true, id: `S-${profile}` }, update: { status: 4 } }], + }), + ); + expect(leaks(out)).toEqual([]); + // Nothing about an ordinary status update is an error or a warning. + if (profile === 'minimal') expect(out).toBe(''); + }); - it('an incoming call (the raw call stanzas and the call event)', async () => { - const { service, ev } = await makeService({ profile }); - const { socket } = await connect(service, ev); - const offer = { - tag: 'call', - attrs: { from: SENDER, id: 'C1', t: '1700000000', notify: NAME }, - content: [{ tag: 'offer', attrs: { 'call-id': 'CALL1', 'call-creator': SENDER }, content: undefined }], - }; - const ack = { tag: 'ack', attrs: { from: SENDER, id: 'C1', class: 'call', type: 'offer' } }; - const call = [{ chatId: SENDER, from: SENDER, id: 'CALL1', date: new Date(1_700_000_000_000), offline: false, status: 'offer', isVideo: false, isGroup: false }]; - const out = await captureOutput(async () => { - socket.ws.emit('CB:call', offer); - socket.ws.emit('CB:ack,class:call', ack); - await deliver(service, ev, { call }); + it('an incoming call (the raw call stanzas and the call event)', async () => { + const { service, ev } = await makeService({ profile }); + const { socket } = await connect(service, ev); + const offer = { + tag: 'call', + attrs: { from: SENDER, id: 'C1', t: '1700000000', notify: NAME }, + content: [{ tag: 'offer', attrs: { 'call-id': 'CALL1', 'call-creator': SENDER }, content: undefined }], + }; + const ack = { tag: 'ack', attrs: { from: SENDER, id: 'C1', class: 'call', type: 'offer' } }; + const call = [ + { + chatId: SENDER, + from: SENDER, + id: 'CALL1', + date: new Date(1_700_000_000_000), + offline: false, + status: 'offer', + isVideo: false, + isGroup: false, + }, + ]; + const out = await captureOutput(async () => { + socket.ws.emit('CB:call', offer); + socket.ws.emit('CB:ack,class:call', ack); + await deliver(service, ev, { call }); + }); + expect(leaks(out)).toEqual([]); }); - expect(leaks(out)).toEqual([]); - }); - it('a group metadata cache lookup (miss, then hit)', async () => { - const { service } = await makeService({ profile }); - service.client.groupMetadata = async (id: string) => ({ id, subject: TEXT, participants: [{ id: SENDER, admin: null }] }); - const out = await captureOutput(async () => { - await service.getGroupMetadataCache(GROUP); - await service.getGroupMetadataCache(GROUP); + it('a group metadata cache lookup (miss, then hit)', async () => { + const { service } = await makeService({ profile }); + service.client.groupMetadata = async (id: string) => ({ + id, + subject: TEXT, + participants: [{ id: SENDER, admin: null }], + }); + const out = await captureOutput(async () => { + await service.getGroupMetadataCache(GROUP); + await service.getGroupMetadataCache(GROUP); + }); + expect(leaks(out)).toEqual([]); }); - expect(leaks(out)).toEqual([]); - }); - it('a group participants update whose participant lookup fails', async () => { - const { service, ev } = await makeService({ profile }); - service.client.groupMetadata = async () => { - throw new Error('item-not-found'); - }; - const out = await captureOutput(() => - deliver(service, ev, { 'group-participants.update': { id: `${PHONE}-1600000001@g.us`, author: SENDER, participants: [SENDER], action: 'add' } }), - ); - expect(leaks(out)).toEqual([]); - }); + it('a group participants update whose participant lookup fails', async () => { + const { service, ev } = await makeService({ profile }); + service.client.groupMetadata = async () => { + throw new Error('item-not-found'); + }; + const out = await captureOutput(() => + deliver(service, ev, { + 'group-participants.update': { + id: `${PHONE}-1600000001@g.us`, + author: SENDER, + participants: [SENDER], + action: 'add', + }, + }), + ); + expect(leaks(out)).toEqual([]); + }); - it('an outgoing text message', async () => { - const { service } = await makeService({ profile }); - Object.assign(service.client, { - onWhatsApp: async (...jids: string[]) => jids.map((jid) => ({ exists: true, jid })), - sendMessage: async (jid: string, content: any) => ({ - key: { remoteJid: jid, fromMe: true, id: 'OUT1' }, - message: { conversation: content.text ?? TEXT }, - messageTimestamp: 1_700_000_000, - status: 1, - }), - presenceSubscribe: async () => undefined, - sendPresenceUpdate: async () => undefined, + it('an outgoing text message', async () => { + const { service } = await makeService({ profile }); + Object.assign(service.client, { + onWhatsApp: async (...jids: string[]) => jids.map((jid) => ({ exists: true, jid })), + sendMessage: async (jid: string, content: any) => ({ + key: { remoteJid: jid, fromMe: true, id: 'OUT1' }, + message: { conversation: content.text ?? TEXT }, + messageTimestamp: 1_700_000_000, + status: 1, + }), + presenceSubscribe: async () => undefined, + sendPresenceUpdate: async () => undefined, + }); + let sent: any; + const out = await captureOutput(async () => { + sent = await service.textMessage({ number: PHONE, text: TEXT }); + await settle(service); + }); + expect(sent?.key?.id).toBe('OUT1'); + expect(leaks(out)).toEqual([]); }); - let sent: any; - const out = await captureOutput(async () => { - sent = await service.textMessage({ number: PHONE, text: TEXT }); - await settle(service); + + it('a raw node sent through baileysSendNode', async () => { + const { service } = await makeService({ profile }); + service.client.sendNode = async () => undefined; + const stanza = { + tag: 'message', + attrs: { to: SENDER, id: 'N1' }, + content: [{ tag: 'body', attrs: {}, content: TEXT }], + }; + const out = await captureOutput(() => service.baileysSendNode(stanza)); + expect(leaks(out)).toEqual([]); }); - expect(sent?.key?.id).toBe('OUT1'); - expect(leaks(out)).toEqual([]); - }); + }, +); - it('a raw node sent through baileysSendNode', async () => { - const { service } = await makeService({ profile }); - service.client.sendNode = async () => undefined; - const stanza = { tag: 'message', attrs: { to: SENDER, id: 'N1' }, content: [{ tag: 'body', attrs: {}, content: TEXT }] }; - const out = await captureOutput(() => service.baileysSendNode(stanza)); - expect(leaks(out)).toEqual([]); +// An exception's message is arbitrary text: JSON.parse quotes the input it choked on, and a +// library error can carry a number as a person typed it. errorFields scrubbed URLs, JIDs and runs +// of 6+ digits, so a quoted text and a number written with spaces or dashes went through. +describe('errorFields', () => { + it('keeps neither a quoted text nor a formatted phone number from an error message', async () => { + const { errorFields } = await import('@utils/log-privacy'); + const quoted = (() => { + try { + JSON.parse('see you at the cafe on Herzl street'); + } catch (error) { + return error; + } + })(); + const phone = new Error('Could not deliver: recipient +972 54-111-2233 (054-111-2233) is not reachable'); + const fields = [errorFields(quoted), errorFields(phone)]; + const text = JSON.stringify(fields); + expect({ + text: text.includes('see you') || text.includes('Herzl'), + phone: /111.?2233|54.?111/.test(text), + }).toEqual({ text: false, phone: false }); + // What went wrong is still said. + expect(fields.map((f) => f.name)).toEqual(['SyntaxError', 'Error']); }); }); From 90041f3676295120f9869aac0f8d6e18ce82ac56 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:27:21 +0300 Subject: [PATCH 135/157] fix: log scrubbing masks quoted text and phone numbers written with separators scrub() (used by errorFields and the Baileys logger) now also masks any quoted part of a message (JSON.parse repeats the input it failed on) and a run of 7+ digits written with spaces, dashes, dots or brackets, as a person types a phone number. Not done here, and why: a name in the unquoted words of an exception message is not recognisable by pattern, and many inherited catch sites still log raw errors; the FORK.md claim about logs is narrowed instead (docs commit). The review's S3 example did not reproduce: a failed download reaching the S3 upload catch is already the bounded error getBase64FromMediaMessage throws (checked with a CDN answering 410; the line logged holds no URL). Visible to consumers: log lines only (quoted parts and formatted numbers become [quoted] / [number]). Co-Authored-By: Claude Opus 5.5 --- src/utils/log-privacy.ts | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/src/utils/log-privacy.ts b/src/utils/log-privacy.ts index d94e796b32..89646a827d 100644 --- a/src/utils/log-privacy.ts +++ b/src/utils/log-privacy.ts @@ -17,13 +17,18 @@ export function jidKind(jid: unknown): string { /** * Mask anything in a diagnostic string that looks like a URL, a JID or a phone - * number. A URL goes first, and whole: a WhatsApp media link is signed per - * message (`oh`, `oe` in its query), and whoever holds it can fetch the file. + * number, and anything quoted. A URL goes first, and whole: a WhatsApp media + * link is signed per message (`oh`, `oe` in its query), and whoever holds it can + * fetch the file. A quoted part is input an error repeats (JSON.parse quotes + * what it failed on); a phone number is masked also as a person writes it, with + * spaces, dashes, dots or brackets. */ export function scrub(value: unknown): string { return String(value ?? '') .replace(/[a-z][a-z0-9+.-]*:\/\/[^\s'"<>]+/gi, '[url]') .replace(/[^\s'"<>=,;:()[\]{}]+@[^\s'"<>=,;:()[\]{}]+/g, '[jid]') + .replace(/"[^"\n]{2,}"|'[^'\n]{2,}'|`[^`\n]{2,}`/g, '[quoted]') + .replace(/\+?\(?\d[\d\s().-]{5,}\d/g, (run) => (run.replace(/\D/g, '').length >= 7 ? '[number]' : run)) .replace(/\d{6,}/g, '[number]'); } From 3d2907ad2cc89a285c3e9ee776a0730a2ca4686e Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:28:25 +0300 Subject: [PATCH 136/157] test: a DM addressed by phone whose key carries its @lid is asked for only under the @lid Baileys gives a DM WhatsApp addresses by phone the @lid in remoteJidAlt (sender_lid) with addressingMode 'pn': the same key the messages.upsert webhook shows for an @lid-addressed DM after its swap. originalMessageKey turns every such key into the @lid form, so the phone, which keeps this message under the phone JID, refuses the re-upload. Red on the current code: the request names the @lid only, the phone answers NOT_FOUND, and the download fails. Co-Authored-By: Claude Opus 5.5 --- test/proxy/media-reupload-address.test.ts | 80 ++++++++++++++++++++--- 1 file changed, 72 insertions(+), 8 deletions(-) diff --git a/test/proxy/media-reupload-address.test.ts b/test/proxy/media-reupload-address.test.ts index bf6c24565b..4b10e39811 100644 --- a/test/proxy/media-reupload-address.test.ts +++ b/test/proxy/media-reupload-address.test.ts @@ -17,7 +17,8 @@ vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); import { readFile, rm } from 'node:fs/promises'; -import { encryptedStream, encryptMediaRetryRequest, getBinaryNodeChild } from 'baileys'; +import { Boom } from '@hapi/boom'; +import { encryptedStream, encryptMediaRetryRequest, getBinaryNodeChild, proto } from 'baileys'; import { getGlobalDispatcher, MockAgent, setGlobalDispatcher } from 'undici'; import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; @@ -82,7 +83,12 @@ async function downloadWithKey(service: any, key: Record) { message: { key, message: { - imageMessage: { url: `http://127.0.0.1:${cdn.port}${GONE}`, mediaKey, mimetype: 'image/jpeg', fileLength: PLAIN.length }, + imageMessage: { + url: `http://127.0.0.1:${cdn.port}${GONE}`, + mediaKey, + mimetype: 'image/jpeg', + fileLength: PLAIN.length, + }, }, }, }); @@ -94,7 +100,13 @@ describe('a media re-upload names the chat by the address WhatsApp stores it und it('a DM key from the webhook (phone, @lid in remoteJidAlt) asks under the @lid', async () => { const { service, keys, rmr } = await serviceWithPhone(); - await downloadWithKey(service, { remoteJid: PHONE, remoteJidAlt: LID, fromMe: false, id: ID, addressingMode: 'pn' }); + await downloadWithKey(service, { + remoteJid: PHONE, + remoteJidAlt: LID, + fromMe: false, + id: ID, + addressingMode: 'pn', + }); expect(keys).toEqual([{ remoteJid: LID, remoteJidAlt: PHONE, fromMe: false, id: ID, addressingMode: 'lid' }]); expect(rmr).toEqual([{ jid: LID, from_me: 'false', participant: undefined }]); @@ -103,7 +115,13 @@ describe('a media re-upload names the chat by the address WhatsApp stores it und it("a DM key stored from Evolution 2.3.7 (the phone twice, addressingMode 'lid') asks under the @lid Baileys maps it to", async () => { const { service, keys, rmr } = await serviceWithPhone({ [PHONE]: LID }); - await downloadWithKey(service, { remoteJid: PHONE, remoteJidAlt: PHONE, fromMe: false, id: ID, addressingMode: 'lid' }); + await downloadWithKey(service, { + remoteJid: PHONE, + remoteJidAlt: PHONE, + fromMe: false, + id: ID, + addressingMode: 'lid', + }); expect(keys).toEqual([{ remoteJid: LID, remoteJidAlt: PHONE, fromMe: false, id: ID, addressingMode: 'lid' }]); expect(rmr).toEqual([{ jid: LID, from_me: 'false', participant: undefined }]); @@ -112,9 +130,18 @@ describe('a media re-upload names the chat by the address WhatsApp stores it und it('a group key with the phone as participant and the @lid in participantAlt asks under the @lid participant', async () => { const { service, keys, rmr } = await serviceWithPhone(); - await downloadWithKey(service, { remoteJid: GROUP, fromMe: false, id: ID, participant: PHONE, participantAlt: LID, addressingMode: 'pn' }); - - expect(keys).toEqual([{ remoteJid: GROUP, fromMe: false, id: ID, participant: LID, participantAlt: PHONE, addressingMode: 'lid' }]); + await downloadWithKey(service, { + remoteJid: GROUP, + fromMe: false, + id: ID, + participant: PHONE, + participantAlt: LID, + addressingMode: 'pn', + }); + + expect(keys).toEqual([ + { remoteJid: GROUP, fromMe: false, id: ID, participant: LID, participantAlt: PHONE, addressingMode: 'lid' }, + ]); expect(rmr).toEqual([{ jid: GROUP, from_me: 'false', participant: LID }]); }); @@ -130,7 +157,14 @@ describe('a media re-upload names the chat by the address WhatsApp stores it und it('control: a group key from the webhook (the @lid participant, as Baileys gave it) is asked for as it is', async () => { const { service, keys, rmr } = await serviceWithPhone(); - const key = { remoteJid: GROUP, fromMe: false, id: ID, participant: LID, participantAlt: PHONE, addressingMode: 'lid' }; + const key = { + remoteJid: GROUP, + fromMe: false, + id: ID, + participant: LID, + participantAlt: PHONE, + addressingMode: 'lid', + }; await downloadWithKey(service, key); @@ -147,4 +181,34 @@ describe('a media re-upload names the chat by the address WhatsApp stores it und expect(keys).toEqual([key]); expect(rmr).toEqual([{ jid: PHONE, from_me: 'false', participant: undefined }]); }); + + // A DM WhatsApp addresses by phone also carries the @lid, in remoteJidAlt (Baileys' + // extractAddressingContext: sender_lid), with addressingMode 'pn'. That is exactly the key the + // webhook shows for an @lid-addressed DM after its swap, so the key alone cannot say which address + // the phone keeps the message under. Asked only under the @lid, the phone refuses a message it + // keeps under the phone JID. + it('a DM WhatsApp addresses by phone, whose key also carries its @lid, is still re-uploaded', async () => { + const { service, keys } = await serviceWithPhone(); + const upload = service.client.updateMediaMessage; + service.client.updateMediaMessage = async (message: any) => { + if (message.key.remoteJid !== PHONE) { + keys.push({ ...message.key }); + throw new Boom('Media re-upload failed by device', { + statusCode: 404, + data: { result: proto.MediaRetryNotification.ResultType.NOT_FOUND }, + }); + } + return upload(message); + }; + + await downloadWithKey(service, { + remoteJid: PHONE, + remoteJidAlt: LID, + fromMe: false, + id: ID, + addressingMode: 'pn', + }); + + expect(keys.map((k) => k.remoteJid)).toEqual([LID, PHONE]); + }); }); From e4354118235ba75bb2fef97f365cf2a02fa70ba8 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:28:54 +0300 Subject: [PATCH 137/157] fix: a re-upload the phone refuses under the @lid is asked again under the key as given A key with the phone JID and its @lid beside it (addressingMode 'pn') means either an @lid-addressed DM as the webhook shows it, or a DM WhatsApp addresses by phone as Baileys gave it; nothing in the key tells them apart. The re-upload asks under the original (@lid) form first, as before, and when the phone refuses it (a MediaRetryNotification result, NOT_FOUND and the like, not a timeout) asks once more under the key as given, within the same 60s budget. Not done as the review proposed (carrying the original key in the webhook payload): the payload's key shape is upstream develop's swap, which consumers already rely on; the retry needs no new field. Visible to consumers: /chat/getBase64FromMediaMessage now recovers the media of a phone-addressed DM that failed with reuploadReason NOT_FOUND. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 20 +++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index aaad9306dd..89816adf5a 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -4851,8 +4851,24 @@ export class BaileysStartupService extends ChannelStartupService { }, MEDIA_REUPLOAD_TIMEOUT_MS); }); try { - const ask = async () => - this.client.updateMediaMessage({ ...message, key: await this.originalMessageKey(message.key) }); + // The key's original form first; when the key cannot say which address the phone keeps the + // message under (the phone with its @lid beside it), the key as given if the phone refuses. + const ask = async () => { + const original = await this.originalMessageKey(message.key); + const candidates = [original]; + if (JSON.stringify(original) !== JSON.stringify(message.key)) candidates.push(message.key); + for (const [i, key] of candidates.entries()) { + try { + return await this.client.updateMediaMessage({ ...message, key }); + } catch (error) { + const refused = typeof error?.data?.result === 'number'; + if (!refused || i === candidates.length - 1) throw error; + this.logger.warn( + `media download: ${media}, outcome=reupload_refused, reason=${reuploadRefusal(error)}, trying the other address`, + ); + } + } + }; const updated = await Promise.race([ask(), timeout]); reupload = 'ok'; this.logger.warn(`media download: ${media}, outcome=reupload_ok`); From 9ef82e66f665298bab1b37d294f103cc939b249e Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:30:12 +0300 Subject: [PATCH 138/157] test: an unanswered re-upload leaves Baileys' waiters behind, and a late answer still acts Baileys' updateMediaMessage waits for the phone's answer with no timeout (bindWaitForEvent: a messages.media-update and a connection.update listener). Evolution's 60s race stops waiting but not the wait. Red on the current code: after the download fails with no_answer, one media-update and one connection.update listener remain, and an answer arriving later still completes the request and emits messages.update. Co-Authored-By: Claude Opus 5.5 --- test/chat/media-reupload-cancel.test.ts | 93 +++++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 test/chat/media-reupload-cancel.test.ts diff --git a/test/chat/media-reupload-cancel.test.ts b/test/chat/media-reupload-cancel.test.ts new file mode 100644 index 0000000000..2708d679c3 --- /dev/null +++ b/test/chat/media-reupload-cancel.test.ts @@ -0,0 +1,93 @@ +// A re-upload request waits for the phone's answer for MEDIA_REUPLOAD_TIMEOUT_MS, and then the +// download fails with no_answer. But Baileys' updateMediaMessage waits for that answer with no +// timeout of its own: a listener on messages.media-update and one on connection.update, removed +// only when an answer for the message arrives or the connection closes. Evolution stopped waiting +// and left them there: every unanswered request on a long connection kept two listeners, the +// message and its key, and an answer arriving later still rewrote the message and emitted +// messages.update after the API had answered no_answer. +// +// The socket's updateMediaMessage here is Baileys' own shape: it writes the request and waits with +// Baileys' own bindWaitForEvent on the real event buffer. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { rm } from 'node:fs/promises'; + +import { MEDIA_REUPLOAD_TIMEOUT_MS } from '@api/integrations/channel/whatsapp/whatsapp.baileys.service'; +import { bindWaitForEvent, encryptedStream } from 'baileys'; +import { getGlobalDispatcher, MockAgent, setGlobalDispatcher } from 'undici'; +import { afterAll, beforeAll, describe, expect, it } from 'vitest'; + +import { makeService } from '../helpers/baileys-service'; +import { type Listening, startCdn } from '../helpers/local-net'; + +const PHONE = '972509876543@s.whatsapp.net'; +const ID = '3EB0CCCCCCCCCCCCCCC1'; +const GONE = '/v/t62.7118-24/expired.enc'; + +let cdn: Listening; +let mediaKey: Uint8Array; +const previousDispatcher = getGlobalDispatcher(); +const guard = new MockAgent(); +guard.disableNetConnect(); +guard.enableNetConnect((host: string) => host.startsWith('127.0.0.1:')); + +beforeAll(async () => { + setGlobalDispatcher(guard); + const enc = await encryptedStream(Buffer.from('a photo'), 'image', {}); + mediaKey = enc.mediaKey; + await rm(enc.encFilePath, { force: true }); + cdn = await startCdn({}); + // Evolution's 5s fallback wait and the phone's answer time, shortened. + const realSetTimeout = globalThis.setTimeout; + vi.spyOn(globalThis, 'setTimeout').mockImplementation(((fn: any, ms?: number, ...args: any[]) => + realSetTimeout(fn, ms === 5000 ? 0 : ms === MEDIA_REUPLOAD_TIMEOUT_MS ? 20 : ms, ...args)) as any); +}); + +afterAll(async () => { + vi.restoreAllMocks(); + await cdn.close(); + setGlobalDispatcher(previousDispatcher); + await guard.close(); +}); + +describe('a re-upload the phone does not answer in time', () => { + it('leaves no waiter behind, and an answer arriving later changes nothing', async () => { + const { service, ev } = await makeService(); + const listening: Record = {}; + const on = ev.on.bind(ev); + const off = ev.off.bind(ev); + ev.on = (event: string, listener: any) => ((listening[event] = (listening[event] ?? 0) + 1), on(event, listener)); + ev.off = (event: string, listener: any) => ((listening[event] = (listening[event] ?? 0) - 1), off(event, listener)); + const waitForMediaUpdate = bindWaitForEvent(ev, 'messages.media-update'); + const lateUpdates: any[] = []; + service.client.updateMediaMessage = async (message: any) => { + // As Baileys: write the request (nothing to write here), then wait for the answer, untimed. + await waitForMediaUpdate(async (update: any[]) => !!update.find((u) => u.key.id === message.key.id)); + lateUpdates.push(message.key.id); + ev.emit('messages.update', [{ key: message.key, update: { message: message.message } }]); + return message; + }; + + const failed = await service + .getBase64FromMediaMessage({ + message: { + key: { remoteJid: PHONE, fromMe: false, id: ID }, + message: { imageMessage: { url: `http://127.0.0.1:${cdn.port}${GONE}`, mediaKey, mimetype: 'image/jpeg' } }, + }, + }) + .catch((e: any) => e); + expect(failed?.reuploadReason).toBe('no_answer'); + await new Promise((r) => setTimeout(r, 10)); + const waiters = { + media: listening['messages.media-update'] ?? 0, + close: listening['connection.update'] ?? 0, + }; + + // The phone answers after all. + ev.emit('messages.media-update', [{ key: { remoteJid: PHONE, fromMe: false, id: ID }, media: {} }]); + await new Promise((r) => setTimeout(r, 10)); + expect({ waiters, lateUpdates }).toEqual({ waiters: { media: 0, close: 0 }, lateUpdates: [] }); + }); +}); From 2a86f764ff7eb1c56a10e69895fab5378e81f840 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:31:20 +0300 Subject: [PATCH 139/157] fix: an unanswered re-upload ends Baileys' wait for the answer When the phone does not answer within MEDIA_REUPLOAD_TIMEOUT_MS, Evolution now answers Baileys' own wait itself: a messages.media-update carrying an error for that message, on the socket that asked (marked as Evolution's event in a live recording). Baileys' waiter removes its listeners and throws, so nothing is left per unanswered request, and an answer arriving later finds no waiter: the message is not rewritten and no messages.update follows. A second address (the refusal retry) is not tried once the request is abandoned. The test's socket now throws an answer that carries an error, as Baileys' updateMediaMessage does (it only recorded the answer before); checked red against the code before this fix with that change. Visible to consumers: no late messages.update for a media re-upload the API already answered no_answer. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 34 +++++++++++++++++-- test/chat/media-reupload-cancel.test.ts | 11 ++++-- 2 files changed, 41 insertions(+), 4 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 89816adf5a..7d0c5285c6 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -4737,6 +4737,24 @@ export class BaileysStartupService extends ChannelStartupService { return { options: { dispatcher: this.mediaProxy.dispatcher } as RequestInit }; } + /** + * End Baileys' wait for the phone's answer to a re-upload request (updateMediaMessage waits on + * messages.media-update with no timeout): answer it with an error for that message, on the socket + * that asked. Marked as Evolution's own event for a live recording. + */ + private abandonReupload(client: WASocket, key: WAMessageKey) { + const emit = () => + client?.ev?.emit('messages.media-update', [ + { key, error: new Boom('Media re-upload abandoned: no answer in time', { statusCode: 408 }) }, + ]); + try { + if (this.liveRecorder) this.liveRecorder.fromApp(emit); + else emit(); + } catch (error) { + this.logger.warn({ message: 'Could not end the re-upload wait', error: errorFields(error) }); + } + } + /** * The key the phone stores a message under, which is how a request about the message * (a media re-upload) must name it: the phone refuses one that names a DM it keeps @@ -4850,6 +4868,9 @@ export class BaileysStartupService extends ChannelStartupService { reject(Object.assign(error, { name: 'ReuploadTimeoutError' })); }, MEDIA_REUPLOAD_TIMEOUT_MS); }); + const client = this.client; + let abandoned = false; + let asking: Promise; try { // The key's original form first; when the key cannot say which address the phone keeps the // message under (the phone with its @lid beside it), the key as given if the phone refuses. @@ -4858,8 +4879,9 @@ export class BaileysStartupService extends ChannelStartupService { const candidates = [original]; if (JSON.stringify(original) !== JSON.stringify(message.key)) candidates.push(message.key); for (const [i, key] of candidates.entries()) { + if (abandoned) throw new Error('re-upload abandoned'); try { - return await this.client.updateMediaMessage({ ...message, key }); + return await client.updateMediaMessage({ ...message, key }); } catch (error) { const refused = typeof error?.data?.result === 'number'; if (!refused || i === candidates.length - 1) throw error; @@ -4869,11 +4891,19 @@ export class BaileysStartupService extends ChannelStartupService { } } }; - const updated = await Promise.race([ask(), timeout]); + asking = ask(); + const updated = await Promise.race([asking, timeout]); reupload = 'ok'; this.logger.warn(`media download: ${media}, outcome=reupload_ok`); return updated; } catch (error) { + if (error?.name === 'ReuploadTimeoutError') { + // Baileys waits for the answer with no timeout of its own (bindWaitForEvent): end that + // wait, so its listeners go and an answer arriving later changes nothing. + abandoned = true; + asking?.catch(() => undefined); + this.abandonReupload(client, message.key); + } reupload = 'failed'; reuploadReason = reuploadRefusal(error); this.logger.warn( diff --git a/test/chat/media-reupload-cancel.test.ts b/test/chat/media-reupload-cancel.test.ts index 2708d679c3..ee0d712b73 100644 --- a/test/chat/media-reupload-cancel.test.ts +++ b/test/chat/media-reupload-cancel.test.ts @@ -63,8 +63,15 @@ describe('a re-upload the phone does not answer in time', () => { const waitForMediaUpdate = bindWaitForEvent(ev, 'messages.media-update'); const lateUpdates: any[] = []; service.client.updateMediaMessage = async (message: any) => { - // As Baileys: write the request (nothing to write here), then wait for the answer, untimed. - await waitForMediaUpdate(async (update: any[]) => !!update.find((u) => u.key.id === message.key.id)); + // As Baileys: write the request (nothing to write here), then wait for the answer, untimed; an + // answer carrying an error is thrown. + let error: any; + await waitForMediaUpdate(async (update: any[]) => { + const result = update.find((u) => u.key.id === message.key.id); + if (result?.error) error = result.error; + return !!result; + }); + if (error) throw error; lateUpdates.push(message.key.id); ev.emit('messages.update', [{ key: message.key, update: { message: message.message } }]); return message; From 3fc6f59524462798dbb31f791c0df4aa5b65a59b Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:31:46 +0300 Subject: [PATCH 140/157] test: a boot connect that fails on a database read leaves the instance out of the API for good The monitor's loaders call setInstance without awaiting it, and setInstance registers the instance only after its connect. A connect that failed on a read (loadSettings here) rejected unhandled, and the instance was never registered and never retried. Red on the current code: the instance is not in waInstances, no socket is ever built, and the rejection is unhandled. Co-Authored-By: Claude Opus 5.5 --- test/instance/logout-pending.test.ts | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/test/instance/logout-pending.test.ts b/test/instance/logout-pending.test.ts index eddda5cb2f..95948e5278 100644 --- a/test/instance/logout-pending.test.ts +++ b/test/instance/logout-pending.test.ts @@ -624,3 +624,24 @@ describe('a logout that cannot reach WhatsApp', () => { expect({ me: storedMe(), pending: service.logoutPending }).toEqual({ me: [], pending: false }); }); }); + +// On boot the monitor lists the instances and connects each one that was open, then registers it. +// A connect that failed there (the database answering the listing but failing the next read) +// left the instance unregistered, with no socket and no reconnect: out of the API until the +// process restarted, although the database came back a second later. +describe('a boot whose first connect fails on a database read', () => { + it('keeps the instance in the API and connects it once the database answers', async () => { + await linkedInstance(); + const read = prisma.setting.findUnique; + let failures = 1; + prisma.setting.findUnique = async (args: any) => { + if (failures-- > 0) throw new Error("Can't reach database server"); + return read(args); + }; + + const monitor = await startProcess(); + await vi.waitFor(() => expect(monitor.waInstances.test).toBeDefined(), { timeout: 1_000 }); + await vi.waitFor(() => expect(built()).toHaveLength(1), { timeout: 3_000 }); + prisma.setting.findUnique = read; + }); +}); From 26622dc6af036434ae669b02137fe61ae11f00ab Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:32:28 +0300 Subject: [PATCH 141/157] fix: a boot connect that fails keeps the instance in the API and retries it - setInstance registers the instance in waInstances before its connect, and a failed auto-connect (a database read, the network) is logged with bounded fields and handed to the service's reconnect backoff (retryConnect: 1s, 2s, 4s... up to 60s), instead of rejecting. - The loaders await each setInstance through loadOne, which logs one instance's failure without stopping the others; no rejection is left unhandled. (loadInstance runs after the server is listening, so only resumeDeletedLogouts now waits for the connects.) Visible to consumers: an instance whose connect failed at boot answers in the API (state close, then connecting) and connects when the database or network returns, instead of 404 until a restart. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 5 +++ src/api/services/monitor.service.ts | 33 ++++++++++++++++--- 2 files changed, 33 insertions(+), 5 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 7d0c5285c6..5ab0d4b26c 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -1366,6 +1366,11 @@ export class BaileysStartupService extends ChannelStartupService { }, delay); } + /** A connect failed before it built a socket (so no close will retry it): try again after the backoff. */ + public retryConnect() { + if (!this.reconnectTimer && !this.connecting) this.scheduleReconnect(); + } + /** Drop a reconnect that is still waiting: the instance is being logged out or deleted. */ public stopReconnecting() { if (this.reconnectTimer) { diff --git a/src/api/services/monitor.service.ts b/src/api/services/monitor.service.ts index a9a6a71454..71193174f7 100644 --- a/src/api/services/monitor.service.ts +++ b/src/api/services/monitor.service.ts @@ -7,6 +7,7 @@ import { CacheConf, Chatwoot, ConfigService, Database, DelInstance, ProviderSess import { Logger } from '@config/logger.config'; import { INSTANCE_DIR, STORE_DIR } from '@config/path.config'; import { NotFoundException } from '@exceptions'; +import { errorFields } from '@utils/log-privacy'; import { readLogoutMarker } from '@utils/logout-marker'; import { execFileSync } from 'child_process'; import EventEmitter2 from 'eventemitter2'; @@ -373,18 +374,40 @@ export class WAMonitoringService { return; } + // In the API before it connects: a connect that fails (a database read, the network) must not + // leave the instance out of it until the process restarts. + this.waInstances[instanceData.instanceName] = instance; + if (instanceData.connectionStatus === 'open' || instanceData.connectionStatus === 'connecting') { this.logger.info( `Auto-connecting instance "${instanceData.instanceName}" (status: ${instanceData.connectionStatus})`, ); - await instance.connectToWhatsapp(); + try { + await instance.connectToWhatsapp(); + } catch (error) { + this.logger.error({ + message: `Auto-connect of instance "${instanceData.instanceName}" failed, retrying`, + error: errorFields(error?.cause ?? error), + }); + (instance as any).retryConnect?.(); + } } else { this.logger.info( `Skipping auto-connect for instance "${instanceData.instanceName}" (status: ${instanceData.connectionStatus || 'close'})`, ); } + } - this.waInstances[instanceData.instanceName] = instance; + /** setInstance for a loader: awaited, and one instance's failure never stops the others. */ + private async loadOne(instanceData: InstanceDto) { + try { + await this.setInstance(instanceData); + } catch (error) { + this.logger.error({ + message: `Loading instance "${instanceData.instanceName}" failed`, + error: errorFields(error), + }); + } } private async loadInstancesFromRedis() { @@ -411,7 +434,7 @@ export class WAMonitoringService { connectionStatus: instanceData.connectionStatus as any, // Pass connection status }; - this.setInstance(instance); + await this.loadOne(instance); }), ); } @@ -430,7 +453,7 @@ export class WAMonitoringService { await Promise.all( instances.map(async (instance) => { - this.setInstance({ + await this.loadOne({ instanceId: instance.id, instanceName: instance.name, integration: instance.integration, @@ -457,7 +480,7 @@ export class WAMonitoringService { where: { id: instanceId }, }); - this.setInstance({ + await this.loadOne({ instanceId: instance.id, instanceName: instance.name, integration: instance.integration, From a69530d7158b728f6696471d4b642d1233a4535f Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:33:07 +0300 Subject: [PATCH 142/157] test: a failed creds write reads as saved, unreadable creds are overwritten, and a failed creds.update save is lost - saveKey swallowed any error, so saveCreds resolved with the creds only in memory. - A creds row that cannot be parsed read as no session: the store started fresh and wrote the fresh creds over it. - eventHandler's saveCreds on creds.update was neither awaited nor retried, and its rejection went unhandled. Red on the current code (3 tests). Co-Authored-By: Claude Opus 5.5 --- test/handlers/creds-save-retry.test.ts | 33 ++++++++++++++++++++ test/unit/auth-state-db-error.test.ts | 43 +++++++++++++++++++++++++- 2 files changed, 75 insertions(+), 1 deletion(-) create mode 100644 test/handlers/creds-save-retry.test.ts diff --git a/test/handlers/creds-save-retry.test.ts b/test/handlers/creds-save-retry.test.ts new file mode 100644 index 0000000000..5e64bb10a2 --- /dev/null +++ b/test/handlers/creds-save-retry.test.ts @@ -0,0 +1,33 @@ +// Baileys emits creds.update whenever the account's credentials change (pairing, pre-key uploads, +// app-state keys), and Evolution saves them. The save was not awaited and its failure not handled: +// a database blip lost the update (the process ran on creds only in memory, and a restart opened +// the old ones), and the rejection went unhandled. +import { vi } from 'vitest'; + +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); + +import { describe, expect, it } from 'vitest'; + +import { deliver, makeService, settle } from '../helpers/baileys-service'; + +describe('a creds.update whose save fails', () => { + it('is saved again until it lands', async () => { + const { service, ev } = await makeService(); + let attempts = 0; + let saved = false; + service.instance.authState.saveCreds = async () => { + attempts++; + if (attempts === 1) throw new Error("Can't reach database server"); + saved = true; + }; + await deliver( + service, + ev, + { 'creds.update': { me: { id: '972500000000:1@s.whatsapp.net' } } }, + { buffered: false }, + ); + await vi.waitFor(() => expect(saved).toBe(true), { timeout: 3_000 }); + await settle(service); + expect(attempts).toBe(2); + }); +}); diff --git a/test/unit/auth-state-db-error.test.ts b/test/unit/auth-state-db-error.test.ts index 213c3a0164..55b6ca4901 100644 --- a/test/unit/auth-state-db-error.test.ts +++ b/test/unit/auth-state-db-error.test.ts @@ -78,7 +78,12 @@ describe('the prisma auth store on a database error', () => { ...outcome, rows: prismaRepository.session.rows.filter((r: any) => r.sessionId === SESSION).length, credsUnchanged: storedRow()?.creds === before, - }).toEqual({ opened: false, error: "Can't reach database server at `127.0.0.1:5432`", rows: 1, credsUnchanged: true }); + }).toEqual({ + opened: false, + error: "Can't reach database server at `127.0.0.1:5432`", + rows: 1, + credsUnchanged: true, + }); // The retried connect opens the linked session. const retried = await open(); @@ -94,3 +99,39 @@ describe('the prisma auth store on a database error', () => { expect(storedRow('new-session')).toBeDefined(); }); }); + +// Writing had the same flaw as reading: saveKey swallowed any error and returned null, so saveCreds +// resolved while the creds were only in memory, and a restart then opened the old ones. And a +// stored creds row that cannot be parsed read as no session at all: the store started fresh and +// wrote the fresh creds over it. +describe('the prisma auth store on a failed write, and on unreadable creds', () => { + it('saveCreds fails when the creds cannot be written', async () => { + const linked = await open(); + Object.assign(linked.state.creds, { me: ME, registered: true }); + const update = prismaRepository.session.update; + prismaRepository.session.update = async () => { + throw Object.assign(new Error("Can't reach database server"), { code: 'P1001' }); + }; + let outcome: string; + try { + outcome = await linked.saveCreds().then( + () => 'saved', + () => 'failed', + ); + } finally { + prismaRepository.session.update = update; + } + expect(outcome).toBe('failed'); + }); + + it('a stored creds row that cannot be parsed fails the open and is left as it is', async () => { + await open(); + const corrupt = '{"noiseKey": {"private": {"type": "Buffer", "data": "tru'; + storedRow()!.creds = corrupt; + const outcome = await open().then( + () => 'opened', + () => 'failed', + ); + expect({ outcome, unchanged: storedRow()!.creds === corrupt }).toEqual({ outcome: 'failed', unchanged: true }); + }); +}); From b8df08ae4fa07ce968974155c38c35cacd10108c Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:33:51 +0300 Subject: [PATCH 143/157] fix: a creds write that fails is reported and retried, and unreadable creds are never replaced - saveKey lets a database error propagate, so saveCreds rejects instead of resolving with the creds only in memory. - Stored creds that cannot be parsed fail the open (UnreadableCredsError) instead of reading as no session, which started fresh creds and wrote them over the stored ones. - A creds.update's save (both the normal and the pending-logout paths) is handled: a failure is logged with bounded fields and the save is tried again, 1s, 2s, 4s... up to 30s, with the creds as they are by then, until one lands. No rejection is left unhandled. Not done: key files are still written in place (fs.writeFile), not atomically; no failing test was written for a torn write. Visible to consumers: an instance whose stored creds are corrupt stays disconnected (and keeps retrying, logging why) instead of silently showing a new QR, and so losing the linked device. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 31 +++++++++++- src/utils/use-multi-file-auth-state-prisma.ts | 49 +++++++++++-------- 2 files changed, 58 insertions(+), 22 deletions(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index 5ab0d4b26c..a2ccc9640c 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -1366,6 +1366,33 @@ export class BaileysStartupService extends ChannelStartupService { }, delay); } + private credsRetry: NodeJS.Timeout | null = null; + private credsRetryAttempts = 0; + + /** + * Save the creds a creds.update changed. A failed save is tried again (1s, 2s, 4s... up to 30s) + * with the creds as they are by then, until one lands: the creds live in memory meanwhile, and a + * restart before it would open the old ones. + */ + private saveCreds() { + const authState = this.instance.authState; + if (!authState?.saveCreds) return; + Promise.resolve() + .then(() => authState.saveCreds()) + .then(() => { + this.credsRetryAttempts = 0; + }) + .catch((error) => { + this.logger.error({ message: 'Could not save the creds, trying again', error: errorFields(error) }); + if (this.credsRetry || this.shutDown || authState !== this.instance.authState) return; + const delay = Math.min(1_000 * 2 ** this.credsRetryAttempts++, 30_000); + this.credsRetry = setTimeout(() => { + this.credsRetry = null; + this.saveCreds(); + }, delay); + }); + } + /** A connect failed before it built a socket (so no close will retry it): try again after the backoff. */ public retryConnect() { if (!this.reconnectTimer && !this.connecting) this.scheduleReconnect(); @@ -2595,7 +2622,7 @@ export class BaileysStartupService extends ChannelStartupService { try { // A logout under way: nothing is forwarded or stored; the connection only delivers the logout. if (this.logout) { - if (events['creds.update']) this.instance.authState.saveCreds(); + if (events['creds.update']) this.saveCreds(); if (events['connection.update']) await this.logoutUpdate(events['connection.update'], client); return; } @@ -2634,7 +2661,7 @@ export class BaileysStartupService extends ChannelStartupService { } if (events['creds.update']) { - this.instance.authState.saveCreds(); + this.saveCreds(); } if (events['lid-mapping.update']) { diff --git a/src/utils/use-multi-file-auth-state-prisma.ts b/src/utils/use-multi-file-auth-state-prisma.ts index d9eeb8e046..db174d78b5 100644 --- a/src/utils/use-multi-file-auth-state-prisma.ts +++ b/src/utils/use-multi-file-auth-state-prisma.ts @@ -24,22 +24,28 @@ export async function keyExists(sessionId: string): Promise { return !!key; } +// A failed write is not a saved one: saveKey lets it propagate, so saveCreds rejects and its +// caller can try again, instead of running on creds that exist only in memory. export async function saveKey(sessionId: string, keyJson: any): Promise { - try { - const exists = await keyExists(sessionId); - if (!exists) - return await prismaRepository.session.create({ - data: { - sessionId: sessionId, - creds: JSON.stringify(keyJson), - }, - }); - await prismaRepository.session.update({ - where: { sessionId: sessionId }, - data: { creds: JSON.stringify(keyJson) }, + const exists = await keyExists(sessionId); + if (!exists) + return await prismaRepository.session.create({ + data: { + sessionId: sessionId, + creds: JSON.stringify(keyJson), + }, }); - } catch { - return null; + await prismaRepository.session.update({ + where: { sessionId: sessionId }, + data: { creds: JSON.stringify(keyJson) }, + }); +} + +/** Stored creds that cannot be read are not an absent session: starting fresh would overwrite them. */ +export class UnreadableCredsError extends Error { + constructor(sessionId: string) { + super(`The stored creds of session ${sessionId} cannot be read: not replacing them`); + this.name = 'UnreadableCredsError'; } } @@ -49,7 +55,7 @@ export async function getAuthKey(sessionId: string): Promise { try { return JSON.parse(auth.creds); } catch { - return null; + throw new UnreadableCredsError(sessionId); } } @@ -118,12 +124,15 @@ export default async function useMultiFileAuthStatePrisma( return JSON.parse(rawData, BufferJSON.reviver); } } else { - rawData = storedCreds; + if (storedCreds === null || storedCreds === undefined) return null; + try { + return JSON.parse(storedCreds, BufferJSON.reviver); + } catch { + throw new UnreadableCredsError(sessionId); + } } - - const parsedData = JSON.parse(rawData, BufferJSON.reviver); - return parsedData; - } catch { + } catch (error) { + if (error instanceof UnreadableCredsError) throw error; return null; } } From 219e79ce46f28f58a551f2ac4710476df0c65461 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:34:16 +0300 Subject: [PATCH 144/157] test: a recording directory that cannot be created fails the connect, with the socket already built LiveRecorder.start creates its directory and writes the first manifest outside its guard, and createClient calls it after makeWASocket and before eventHandler. Red on the current code: with LIVE_RECORD_DIR under a file, the connect fails (ENOTDIR) and nothing listens to the socket it built. Co-Authored-By: Claude Opus 5.5 --- test/live/recorder.test.ts | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/test/live/recorder.test.ts b/test/live/recorder.test.ts index 7b21975470..22f57d62a2 100644 --- a/test/live/recorder.test.ts +++ b/test/live/recorder.test.ts @@ -15,7 +15,7 @@ vi.mock('@utils/fetchLatestWaWebVersion', () => ({ fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), })); -import { mkdtempSync, readdirSync, readFileSync, rmSync } from 'node:fs'; +import { mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; @@ -258,4 +258,23 @@ describe('live-check recorder', () => { }); }); }); + + // The recorder is prepared right after the socket is built and before Evolution listens to it. + // Creating its directory and first manifest was outside its guard, so a LIVE_RECORD_DIR that + // cannot be written (no permission, a full volume, a file in the way) failed the connect with a + // socket already built and nobody listening to it. + it('a recording directory that cannot be created costs the recording only, never the connection', async () => { + const blocked = join(root, 'not-a-directory'); + writeFileSync(blocked, 'a file where the directory should go'); + process.env.LIVE_RECORD_DIR = join(blocked, 'records'); + socketSpy.mockClear(); + const { service } = await makeService(); + stubAuthState(service); + const outcome = await service.connectToWhatsapp().then( + () => 'connected', + (e: any) => `failed: ${e?.message}`, + ); + const sock = socketSpy.mock.results[0]?.value; + expect({ outcome, listened: sock?.handlers?.() > 0 }).toEqual({ outcome: 'connected', listened: true }); + }); }); From 2ed4afe1e6e79e8d54ca0ef397bd867fd9f0b379 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:35:17 +0300 Subject: [PATCH 145/157] fix: a recording that cannot start or write its manifest costs the recording, never the connection - LiveRecorder.start catches a failure to create its directory or write the first manifest, warns with the error code only, and returns no recorder: createClient goes on to install its handlers. - attach writes the manifest inside the recorder's guard, so a manifest that can no longer be written stops the recording (as any failed write already did) instead of throwing out of createClient. The second test (manifest read-only at attach) was written with this fix and checked red against the code before it. Visible to consumers: none unless LIVE_RECORD_DIR is set. Co-Authored-By: Claude Opus 5.5 --- src/utils/live-record/recorder.ts | 21 +++++++++++++++------ test/live/recorder.test.ts | 22 +++++++++++++++++++++- 2 files changed, 36 insertions(+), 7 deletions(-) diff --git a/src/utils/live-record/recorder.ts b/src/utils/live-record/recorder.ts index 3d7ee87ac6..9071795f0c 100644 --- a/src/utils/live-record/recorder.ts +++ b/src/utils/live-record/recorder.ts @@ -107,14 +107,23 @@ export class LiveRecorder { this.writeManifest(); } - /** A recorder for this instance's session, or undefined when LIVE_RECORD_DIR is not set. */ + /** + * A recorder for this instance's session, or undefined when LIVE_RECORD_DIR is not set, or when + * its directory or first manifest cannot be written: a recording that cannot start costs the + * recording, never the connection it was to watch. + */ static start(instanceName: string, env: NodeJS.ProcessEnv = process.env): LiveRecorder | undefined { const root = env.LIVE_RECORD_DIR; if (!root) return undefined; - const stamp = new Date().toISOString().replace(/:/g, '-'); - const dir = join(root, safeName(instanceName), stamp); - mkdirSync(dir, { recursive: true, mode: 0o700 }); - return new LiveRecorder(dir); + try { + const stamp = new Date().toISOString().replace(/:/g, '-'); + const dir = join(root, safeName(instanceName), stamp); + mkdirSync(dir, { recursive: true, mode: 0o700 }); + return new LiveRecorder(dir); + } catch (error) { + console.warn(`[live-record] recording not started: ${error?.code ?? error?.name ?? 'error'}`); + return undefined; + } } /** Record every event this socket emits, and every batch its buffer delivers. Call before eventHandler(). */ @@ -125,7 +134,7 @@ export class LiveRecorder { // The method that linked the account is the one asked for before the first open. if (!this.manifest.openedAt) this.manifest.linkMethod = facts.linkMethod; this.manifest.proxy = { used: !!facts.proxyProtocol, protocol: facts.proxyProtocol }; - this.writeManifest(); + this.guard(() => this.writeManifest()); const ev = client.ev; const emit = ev.emit.bind(ev); diff --git a/test/live/recorder.test.ts b/test/live/recorder.test.ts index 22f57d62a2..4d2a984cf8 100644 --- a/test/live/recorder.test.ts +++ b/test/live/recorder.test.ts @@ -15,7 +15,7 @@ vi.mock('@utils/fetchLatestWaWebVersion', () => ({ fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), })); -import { mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { chmodSync, mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; @@ -277,4 +277,24 @@ describe('live-check recorder', () => { const sock = socketSpy.mock.results[0]?.value; expect({ outcome, listened: sock?.handlers?.() > 0 }).toEqual({ outcome: 'connected', listened: true }); }); + + it('a manifest that can no longer be written when a socket is attached stops the recording, not the socket', async () => { + process.env.LIVE_RECORD_DIR = root; + const { LiveRecorder } = await import('@utils/live-record/recorder'); + const recorder = LiveRecorder.start('test'); + const manifest = join(recorder.dir, 'manifest.json'); + chmodSync(manifest, 0o400); + try { + expect(() => + recorder.attach(fakeSocket(), { + waWebVersion: '2.3000.1', + linkMethod: 'qr', + proxyProtocol: null, + creds: () => ({}), + }), + ).not.toThrow(); + } finally { + chmodSync(manifest, 0o600); + } + }); }); From b3be75380abfd472d952a96549d20f9be350464d Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:36:15 +0300 Subject: [PATCH 146/157] test: a replay splits a batch the live buffer delivered whole The tape records a batch line for every delivery. A non-bufferable event emitted while the buffer holds others is delivered at once, as its own batch, and the buffer keeps holding the rest; the replay flushed the buffer on every batch line. Red on the current code: a contacts.upsert and a contacts.update the live buffer delivered as one batch are replayed as two. Co-Authored-By: Claude Opus 5.5 --- test/live/replay.test.ts | 36 +++++++++++++++++++++++++++++++++--- 1 file changed, 33 insertions(+), 3 deletions(-) diff --git a/test/live/replay.test.ts b/test/live/replay.test.ts index c05c041352..75179e12ac 100644 --- a/test/live/replay.test.ts +++ b/test/live/replay.test.ts @@ -15,7 +15,7 @@ vi.mock('@utils/fetchLatestWaWebVersion', () => ({ fetchLatestWaWebVersion: async () => ({ version: [2, 3000, 1], isLatest: true }), })); -import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; @@ -34,7 +34,11 @@ let fixture: string; beforeEach(async () => { root = mkdtempSync(join(tmpdir(), 'live-replay-')); const raw = await recordSession(join(root, 'raw')); - fixture = scrubSession(raw, { checkId: 'synthetic-session', date: '2026-09-27', outRoot: join(root, 'fixtures') }).dir; + fixture = scrubSession(raw, { + checkId: 'synthetic-session', + date: '2026-09-27', + outRoot: join(root, 'fixtures'), + }).dir; emitted.splice(0); }); afterEach(() => rmSync(root, { recursive: true, force: true })); @@ -56,7 +60,10 @@ describe('live-check replay', () => { it('fails the golden comparison when what Evolution sends differs', async () => { const file = join(fixture, 'webhooks.ndjson'); - const golden = readFileSync(file, 'utf8').trim().split('\n').map((l) => JSON.parse(l)); + const golden = readFileSync(file, 'utf8') + .trim() + .split('\n') + .map((l) => JSON.parse(l)); const upsert = golden.find((w) => w.event === 'messages.upsert'); upsert.data.key.remoteJidAlt = upsert.data.key.remoteJid; // what 2.3.7 sent: the phone twice writeFileSync(file, golden.map((w) => JSON.stringify(w)).join('\n') + '\n'); @@ -67,4 +74,27 @@ describe('live-check replay', () => { expect(diff.join('\n')).toMatch(/missing messages\.upsert/); expect(diff.join('\n')).toMatch(/unexpected messages\.upsert/); }); + + // A batch line on the tape is what the buffer delivered live. A non-bufferable event (a + // connection.update) emitted while the buffer holds others is delivered at once, as its own + // batch, and the buffer keeps holding the rest. The replay flushed the buffer on every batch line, + // so it split what the live buffer delivered as one batch. + it('delivers the batches the tape recorded, a non-bufferable event inside a buffer included', async () => { + const dir = join(root, 'interleaved'); + mkdirSync(dir); + const contact = (name: string) => [{ id: '972500000001@s.whatsapp.net', notify: name }]; + const lines = [ + { seq: 1, t: 1, socket: 1, event: 'contacts.upsert', buffered: true, data: contact('Name 1') }, + { seq: 2, t: 2, socket: 1, event: 'connection.update', buffered: true, data: { isOnline: true } }, + { seq: 3, t: 3, socket: 1, batch: ['connection.update'] }, + { seq: 4, t: 4, socket: 1, event: 'contacts.update', buffered: true, data: contact('Name 2') }, + { seq: 5, t: 5, socket: 1, batch: ['contacts.upsert', 'contacts.update'] }, + ]; + writeFileSync(join(dir, 'events.ndjson'), lines.map((l) => JSON.stringify(l)).join('\n') + '\n'); + writeFileSync(join(dir, 'webhooks.ndjson'), ''); + writeFileSync(join(dir, 'manifest.json'), JSON.stringify({ format: 'live-record/1' })); + + const { batches } = await replayFixture(dir); + expect(batches).toEqual(lines.filter((l) => l.batch).map((l) => l.batch)); + }); }); From eb9bc27e76d3e772b1d226ab6d2741df761ac288 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:37:03 +0300 Subject: [PATCH 147/157] fix(test): a replay flushes the buffer only where the live buffer did A batch line with a bufferable event is the buffer's flush; one of non-bufferable events only (a connection.update emitted inside a buffer) was delivered at once with the buffer still held, so the replay now only waits for its handling there. The bufferable list is Baileys' own (BUFFERABLE_EVENT, rc14). The test's two contacts are now two different people: one contact's contacts.update is merged into its contacts.upsert by the buffer, which no live batch line would show as two keys. Checked red against the previous helper with that change. Co-Authored-By: Claude Opus 5.5 --- test/helpers/live-replay.ts | 31 +++++++++++++++++++++++++++++-- test/live/replay.test.ts | 6 +++--- 2 files changed, 32 insertions(+), 5 deletions(-) diff --git a/test/helpers/live-replay.ts b/test/helpers/live-replay.ts index fd3055c656..efbad653f6 100644 --- a/test/helpers/live-replay.ts +++ b/test/helpers/live-replay.ts @@ -33,6 +33,22 @@ export function loadFixture(dir: string) { }; } +/** The events Baileys' buffer holds (BUFFERABLE_EVENT in Baileys lib/Utils/event-buffer.js, rc14). */ +const BUFFERABLE = new Set([ + 'messaging-history.set', + 'chats.upsert', + 'chats.update', + 'chats.delete', + 'contacts.upsert', + 'contacts.update', + 'messages.upsert', + 'messages.update', + 'messages.delete', + 'messages.reaction', + 'message-receipt.update', + 'groups.update', +]); + export async function replayFixture(dir: string, opts: { profile?: Profile; client?: Record } = {}) { const { events, manifest } = loadFixture(dir); const { service, ev, prisma } = await makeService({ profile: opts.profile }); @@ -58,7 +74,10 @@ export async function replayFixture(dir: string, opts: { profile?: Profile; clie if (line.socket < socket) continue; socket = line.socket; if (line.batch) { - await flush(); + // Only a batch with a bufferable event is the buffer's flush; one of non-bufferable events + // only (a connection.update inside a buffer) was delivered at once, with the buffer still held. + if (line.batch.some((event: string) => BUFFERABLE.has(event))) await flush(); + else await service.eventProcessingQueue; continue; } if (line.origin === 'app') continue; @@ -72,7 +91,15 @@ export async function replayFixture(dir: string, opts: { profile?: Profile; clie } /** Fields Evolution fills from the clock, the database or a socket query: not part of the comparison. */ -export const VOLATILE = ['dateTime', 'date_time', 'createdAt', 'updatedAt', 'instanceId', 'profilePicUrl', 'profilePictureUrl']; +export const VOLATILE = [ + 'dateTime', + 'date_time', + 'createdAt', + 'updatedAt', + 'instanceId', + 'profilePicUrl', + 'profilePictureUrl', +]; function normalize(value: any, volatile: Set): any { if (Array.isArray(value)) return value.map((v) => normalize(v, volatile)); diff --git a/test/live/replay.test.ts b/test/live/replay.test.ts index 75179e12ac..ffcc8be6a4 100644 --- a/test/live/replay.test.ts +++ b/test/live/replay.test.ts @@ -82,12 +82,12 @@ describe('live-check replay', () => { it('delivers the batches the tape recorded, a non-bufferable event inside a buffer included', async () => { const dir = join(root, 'interleaved'); mkdirSync(dir); - const contact = (name: string) => [{ id: '972500000001@s.whatsapp.net', notify: name }]; + const contact = (i: number) => [{ id: `97250000000${i}@s.whatsapp.net`, notify: `Name ${i}` }]; const lines = [ - { seq: 1, t: 1, socket: 1, event: 'contacts.upsert', buffered: true, data: contact('Name 1') }, + { seq: 1, t: 1, socket: 1, event: 'contacts.upsert', buffered: true, data: contact(1) }, { seq: 2, t: 2, socket: 1, event: 'connection.update', buffered: true, data: { isOnline: true } }, { seq: 3, t: 3, socket: 1, batch: ['connection.update'] }, - { seq: 4, t: 4, socket: 1, event: 'contacts.update', buffered: true, data: contact('Name 2') }, + { seq: 4, t: 4, socket: 1, event: 'contacts.update', buffered: true, data: contact(2) }, { seq: 5, t: 5, socket: 1, batch: ['contacts.upsert', 'contacts.update'] }, ]; writeFileSync(join(dir, 'events.ndjson'), lines.map((l) => JSON.stringify(l)).join('\n') + '\n'); From 49315198568c214632ae9840762b53b3e0188575 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Sun, 27 Sep 2026 20:37:23 +0300 Subject: [PATCH 148/157] docs(fork): state what the fork's guarantees cover, and what they do not FORK.md said more than the tests show (Codex review, item 18): - the red commits: several fail on the fork's harness and Baileys pin, not on 2.3.7 itself; - saved: the @lid mapping items of contacts.upsert have no saved and a null pushName; - logs: bounded fields and scrubbed exception messages, not a guarantee for every line (inherited raw error logs, untested integrations, names in unquoted words); - proxy, sockets, logout, pictures, creds: what this branch's fixes now hold, and the limits (transports tested); - live checks: what the scrubber, gate and guard each check, that a human read is still required, and that a replay compares content, not order, pictures, decryption or timing. Co-Authored-By: Claude Opus 5.5 --- FORK.md | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/FORK.md b/FORK.md index 759facdbde..7534f0f5d1 100644 --- a/FORK.md +++ b/FORK.md @@ -35,7 +35,10 @@ A fix without a failing test first is not merged here. See `AGENTS.md`. ## Changes from 2.3.7 -Each line is a pair of commits: the test that failed on 2.3.7, then the fix. +Each line is a pair of commits: the test that failed on the code before the +fix, then the fix. For most, that code is 2.3.7 itself; several need what the +fork added first (the test harness, the Baileys 7.0.0-rc14 pin), and a red commit +that only fails to import what its fix adds proves nothing about behaviour. `git log 2.3.7..` is the source of truth, and `git diff 2.3.7 --stat` lists every file modified from the original. Upstream issues and pull requests are named where one exists. @@ -43,33 +46,34 @@ named where one exists. **Contacts, names and groups** - App-state sync keys are reloaded with `fromObject` in all three auth stores, so saved names, labels, mutes and archives keep syncing after a restart (#2576, #2384; fixes also offered in #2685 and #2610). - Every @lid to phone mapping Baileys learns is forwarded on `contacts.upsert`, captured before Baileys' event buffer drops it. -- Every `contacts.upsert` item says whether its name is the one the owner saved (`saved`), and only when that is certain. +- Every `contacts.upsert` item for a contact says whether its name is the one the owner saved (`saved`), and only when that is certain. The @lid mapping items above carry `pushName: null` and no `saved`: a consumer that replaces a whole contact with an item, rather than merging its fields, loses the name. - Group updates reach subscriptions stored as `GROUP_UPDATE`, in all seven transports and in the global configurations (#2652). - Group metadata is filled from `groups.update` instead of queried again for every group on every listing. - `chats.update`, `chats.set` and `chats.upsert` items carry the chat's archive, pin and mute state (`archived`, `pinned`, `muteEndTime`) when Baileys has it, and omit a field it does not have rather than guess. **Messages and privacy** - A `getMessage` miss or failed lookup answers nothing, so Baileys no longer relays an empty message on a retry and uses it up (#2705, and #2706 for groups; fixes also offered in #2728 and #2623). -- No message text, phone number, JID or push name reaches the logs at `LOG_LEVEL=ERROR,WARN`, including Baileys' own error logs. +- The fork's own log lines at `LOG_LEVEL=ERROR,WARN`, and Baileys' error logs, carry bounded fields, not message text, phone numbers, JIDs or push names; an exception's message is kept only scrubbed of URLs, JIDs, quoted parts and phone numbers (also as a person writes them). This is not a guarantee for every line: inherited code still logs some raw errors (integrations such as S3, Chatwoot and the chatbots are not covered by the tests), and a name inside an exception's unquoted words is not recognisable. - The `messages.upsert` webhook key of a DM WhatsApp addresses by @lid shows the phone JID as `remoteJid`, as 2.3.7 does, but keeps the original @lid in `remoteJidAlt` (2.3.7 copied the phone there too and lost it), with `addressingMode: 'pn'`: upstream develop's swap. -- A media re-upload request names the message by the address the phone stores it under: the @lid from `remoteJidAlt` or `participantAlt` when the key shows the phone, or, for a key stored from 2.3.7 (the phone twice, `addressingMode: 'lid'`), the @lid Baileys' LID mapping has for the phone. The phone refuses a request that names an @lid chat by its phone JID. +- A media re-upload request names the message by the address the phone stores it under: the @lid from `remoteJidAlt` or `participantAlt` when the key shows the phone, or, for a key stored from 2.3.7 (the phone twice, `addressingMode: 'lid'`), the @lid Baileys' LID mapping has for the phone. The phone refuses a request that names an @lid chat by its phone JID. A key with the phone and its @lid beside it cannot say which of the two the phone keeps (a DM WhatsApp addresses by phone looks the same), so a refusal under the @lid is asked again once under the key as given. A request the phone does not answer in time ends Baileys' own wait for it, so no listener is left and a late answer changes nothing. - A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error), and when the phone refused, why (`reason=` on the log line, `reuploadReason` on the error and the HTTP answer: the phone's result such as `NOT_FOUND`, `error_`, `missing_ciphertext`, or `no_answer`). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410 (its check misses the status), so Evolution asks the phone itself, once per download, for at most 60s. A CDN 403 counts as expired only when the link that actually failed (the one on Baileys' error, which is the directPath when the message has one, else the url) carries exactly one `oe` query parameter, in hex unix seconds, that has passed by the local clock. This is a conservative heuristic: on 84 history-sync attachments, 403 answered 34 of 34 expired links and 0 of 50 valid ones, where a valid link to a dropped file answered 404 or 410. - A media download turns the media key back into bytes (from a base64 string, or the object JSON makes of a Uint8Array or a Buffer) before it asks the phone to re-upload, since Baileys 7.0.0-rc14 derives the re-upload's key from the value as given and then cannot decrypt the phone's answer (Baileys #2729 proposed the same fix there). - `POST /chat/getBase64FromMediaMessage` takes an optional `reupload` (boolean, default true). With `false` the phone is never asked to re-upload, and a file gone from the CDN (404, 410, or 403 on an expired link) fails at once with a 400 that says so; the HTTP error carries `reupload` for any failed download. **Proxy** - Media downloads, media uploads and the WhatsApp Web version fetch leave through the instance's proxy (uploads failed outright on a proxied instance). -- A connect waits for the instance's proxy and stored settings, so it never starts from the server's own address or with default settings. +- A connect waits for the instance's proxy and stored settings, so it never starts from the server's own address or with default settings. Reloading the proxy keeps the one in force until the new one is read; a proxyscrape list that cannot be fetched fails the connect (and it is retried), and a proxyscrape download with no socket exit to share fails, rather than either going out directly. Tested with a local HTTP and SOCKS5 proxy; other transports a deployment adds are not covered. **Sessions and connections** - A database error never replaces a linked session's credentials with fresh ones, and a failed settings read no longer drops an event batch. - A logout or delete while the socket is down keeps the session until WhatsApp is told, so the device leaves the phone's Linked devices; it answers `202 PENDING` meanwhile and forwards nothing (#2520, #2508 in part). The pending logout is recorded on the instance's row (`disconnectionObject.logoutPending`) as well as in a file next to the session keys, so a restart that lost the instances volume still finishes it; when it cannot be recorded, the logout or delete answers 500 instead of 202. A logout is finished only on WhatsApp's word: its answer to remove-companion-device, or its refusal of the device on the next connection; the socket's own close after sending the request (all Baileys' `logout()` waits for) is not one. Concurrent logouts and deletes share one run and answer with its outcome. A deleted instance keeps its database row until then (Session and Proxy cascade from it), out of the API, its name taken and its token cleared. -- Reconnects back off from 1s to 60s instead of spinning, the version fetch times out after 10s, and an instance never runs two sockets (#2134, #2184, #2430; ideas from #2732). -- Profile pictures are looked up once per contact per hour, at most four at a time, and history never waits on them (#1883). +- Reconnects back off from 1s to 60s instead of spinning, the version fetch times out after 10s, and an instance never runs two sockets: a reload after a profile or privacy change joins a connect under way, and once an instance is removed nothing builds a socket for it, runs a batch it had queued, or forwards anything (#2134, #2184, #2430; ideas from #2732). A connect that fails at boot keeps the instance in the API and is retried on the same backoff. +- A creds write that fails is logged and tried again until it lands, and stored creds that cannot be parsed fail the connect instead of being replaced by fresh ones. +- Profile pictures are looked up once per contact per hour, at most four at a time, and history never waits on them (#1883). The picture update a history batch sends carries each contact's newest name, and a lookup that finishes after WhatsApp said the picture changed or was removed is dropped. - One pairing code per connect attempt, and a fresh QR budget per attempt (#2100, #2696). **Live checks** -- A live check against a real phone can be recorded (`LIVE_RECORD_DIR`), scrubbed into a fixture behind a fail-closed leak gate, and replayed through the real event buffer and service in a test. The protocol, the check catalogue and the results log are in `docs/LIVE-CHECKS.md`. +- A live check against a real phone can be recorded (`LIVE_RECORD_DIR`; the QR, its image and pairing codes are never written), scrubbed into a fixture, and replayed through the real event buffer and service in a test. The scrubber keeps a value as written only under a field it lists with a value that field is known to take, and stops on a field it does not know; its leak gate searches the output for every raw value that is not structure; a guard scans every committed fixture. None of them recognises a name the scrubber mistook for structure, so a person still reads every new fixture. A replay compares the webhooks' content, not their order or the pictures, and runs on events already decoded: it shows what Evolution does with what WhatsApp sent, not the encryption or the timing around it. The protocol, the check catalogue and the results log are in `docs/LIVE-CHECKS.md`. ## Licence From fbd558cd16632595c2dfb5e456f50191b2212604 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Tue, 29 Sep 2026 11:34:36 +0300 Subject: [PATCH 149/157] test: a failed media download answers with the signed media link POST /chat/getBase64FromMediaMessage answers a failed download with the error's text, Baileys' "Failed to fetch stream from ", so the HTTP answer carries WhatsApp's signed CDN link (oh, oe, the _nc_ parameters and the directPath). The same object is what the error handler posts to the errors webhook and what the S3 upload logs at ERROR. The new file drives the real route against WhatsApp's CDN host, answered by the undici mock agent: 404 with the phone refusing the re-upload, 410 with no answer in time, 403 on an expired link, a re-upload whose new link fails too, and a 403 on a valid link with and without reupload: false. It asserts the whole answer, its headers and the output meanwhile, the thrown object, and the S3 upload's error log. All eight fail on the code as it is. media-skip-reupload.test.ts asserted the old text ("TypeError: fetch failed") for a fallback that fails without an HTTP status; it now expects the stable reason text (error kind and network code), and fails until the fix. Co-Authored-By: Claude Opus 5.5 --- test/chat/media-error-no-url.test.ts | 229 ++++++++++++++++++++++++++ test/chat/media-skip-reupload.test.ts | 9 +- 2 files changed, 236 insertions(+), 2 deletions(-) create mode 100644 test/chat/media-error-no-url.test.ts diff --git a/test/chat/media-error-no-url.test.ts b/test/chat/media-error-no-url.test.ts new file mode 100644 index 0000000000..4d9718924c --- /dev/null +++ b/test/chat/media-error-no-url.test.ts @@ -0,0 +1,229 @@ +// A media download that fails fails with Baileys' error, whose message is +// "Failed to fetch stream from " and whose Boom data carries the same +// link (data.url). A WhatsApp media link is signed per message (`oh`, `oe`, the +// `_nc_*` parameters) and whoever holds it can fetch the file until it expires; +// even its directPath alone names the file. The logs are scrubbed already, but +// the error itself used to leave the process as it was: in the HTTP answer of +// POST /chat/getBase64FromMediaMessage (the message was the error's text), in +// what main.ts's error handler posts to the errors webhook (the same object), +// and in the S3 upload's error log line (which prints that object's message). +// +// The answer keeps what a caller can act on: the CDN's status, whether the phone +// was asked to re-upload the file and why it refused, and a stable reason text. +// It never carries a link, with or without its query. +// +// The CDN here is WhatsApp's own host, answered by the undici mock agent (so the +// links are the real shape and nothing leaves the machine): Baileys downloads +// from https://, and Evolution's own fallback from +// https://mmg.whatsapp.net. +import { vi } from 'vitest'; + +const h = vi.hoisted(() => ({ waMonitor: undefined as any, chatController: undefined as any })); + +vi.mock('@api/server.module', async () => { + const fake = await import('../helpers/fake-server-module'); + return { + ...fake, + get waMonitor() { + return h.waMonitor; + }, + get chatController() { + return h.chatController; + }, + }; +}); + +import { rm } from 'node:fs/promises'; + +import { Boom } from '@hapi/boom'; +import { MEDIA_REUPLOAD_TIMEOUT_MS } from '@api/integrations/channel/whatsapp/whatsapp.baileys.service'; +import { encryptedStream, proto } from 'baileys'; +import { getGlobalDispatcher, MockAgent, setGlobalDispatcher } from 'undici'; +import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; + +import { deliver, makeService } from '../helpers/baileys-service'; +import { captureOutput } from '../helpers/capture-output'; +import { emitted, prismaRepository as prisma } from '../helpers/fake-server-module'; +import { startChatApp } from '../helpers/http-app'; + +const TOKEN = 'instance-token'; +const PHONE = '972509876543'; +const ID = '3EB0ABABABABABABABA1'; +const CDN = 'https://mmg.whatsapp.net'; +const SIGNATURE = '01_Q5Aa2wSignedHashXyz7Qp'; +/** A media link's `oe` (when it stops working): hex unix seconds, as WhatsApp writes it. */ +const oe = (fromNowS: number) => (Math.floor(Date.now() / 1000) + fromNowS).toString(16).toUpperCase(); +const DAY = 24 * 60 * 60; +/** A directPath as WhatsApp signs it. */ +const signed = (file: string, fromNowS = 14 * DAY) => + `/v/t62.7118-24/${file}_n.enc?ccb=11-4&oh=${SIGNATURE}&oe=${oe(fromNowS)}&_nc_sid=5e03e0&_nc_ohc=AbCdEfGh&mms3=true`; +const GONE_404 = signed('31415926_27182818284590_1234567890123456789'); +const GONE_410 = signed('31415926_27182818284590_1234567890123456790'); +const EXPIRED_403 = signed('31415926_27182818284590_1234567890123456791', -DAY); +const VALID_403 = signed('31415926_27182818284590_1234567890123456792'); +const REUPLOADED = signed('31415926_27182818284590_1234567890123456793'); +const STATUS: Record = { [GONE_404]: 404, [GONE_410]: 410, [EXPIRED_403]: 403, [VALID_403]: 403, [REUPLOADED]: 404 }; +/** Any part of a media link. */ +const LEAKS = ['whatsapp.net', 'http://', 'https://', 'oh=', 'oe=', '_nc_', 'mms3', '/v/t62', 't62.7118', SIGNATURE, '31415926']; + +let mediaKey: Uint8Array; +let app: Awaited>; +let service: any; +/** Every request the CDN answered: `GET `. */ +const cdnLog: string[] = []; +/** What the phone does with the next re-upload request. */ +let phone: (message: any) => Promise; + +const refuses = () => () => + Promise.reject( + new Boom('Media re-upload failed by device (NOT_FOUND)', { + data: { stanzaId: ID, result: proto.MediaRetryNotification.ResultType.NOT_FOUND }, + statusCode: 404, + }), + ); +const silent = () => () => new Promise(() => undefined); +/** What Baileys does when the phone re-uploads: point the message at the new copy, which fails too. */ +const reuploads = () => async (message: any) => { + Object.assign(message.message.imageMessage, { url: `${CDN}${REUPLOADED}`, directPath: REUPLOADED }); + return message; +}; + +const previousDispatcher = getGlobalDispatcher(); +const guard = new MockAgent(); +guard.disableNetConnect(); +guard.enableNetConnect((host: string) => host.startsWith('127.0.0.1:')); +guard + .get(CDN) + .intercept({ path: () => true, method: 'GET' }) + .reply((req: any) => { + cdnLog.push(`GET ${req.path}`); + return { statusCode: STATUS[req.path] ?? 404, data: '' }; + }) + .persist(); + +beforeAll(async () => { + setGlobalDispatcher(guard); + const enc = await encryptedStream(Buffer.from('a photo'), 'image', {}); + mediaKey = enc.mediaKey; + await rm(enc.encFilePath, { force: true }); + // Evolution waits 5s before its own fallback download, and gives the phone a + // bounded time to answer; both are shortened here. + const realSetTimeout = globalThis.setTimeout; + vi.spyOn(globalThis, 'setTimeout').mockImplementation(((fn: any, ms?: number, ...args: any[]) => + realSetTimeout(fn, ms === 5000 ? 0 : ms === MEDIA_REUPLOAD_TIMEOUT_MS ? 20 : ms, ...args)) as any); + + await prisma.instance.create({ data: { id: 'inst-1', name: 'test', connectionStatus: 'open', token: TOKEN, integration: 'WHATSAPP-BAILEYS' } }); + ({ service } = await makeService({ prisma })); + service.client.updateMediaMessage = (message: any) => phone(message); + h.waMonitor = { waInstances: { test: service } }; + const { ChatController } = await import('@api/controllers/chat.controller'); + h.chatController = new ChatController(h.waMonitor); + app = await startChatApp(); +}); + +afterAll(async () => { + vi.restoreAllMocks(); + await app.close(); + setGlobalDispatcher(previousDispatcher); + await guard.close(); +}); + +beforeEach(() => { + cdnLog.splice(0); + emitted.splice(0); +}); + +/** A stored image message as a consumer holds it: JSON, with the media key an object of bytes. */ +const image = (directPath: string) => + JSON.parse( + JSON.stringify({ + key: { remoteJid: `${PHONE}@s.whatsapp.net`, fromMe: false, id: ID }, + message: { imageMessage: { url: `${CDN}${directPath}`, directPath, mediaKey: Uint8Array.from(mediaKey), mimetype: 'image/jpeg' } }, + messageTimestamp: 1_700_000_000, + }), + ); + +/** Every line of `text` that carries any part of a media link. */ +const leaks = (text: string) => + text + .replace(/\x1b\[[0-9;]*m/g, '') + .split('\n') + .filter((line) => LEAKS.some((l) => line.includes(l))) + .map((line) => line.trim().slice(0, 240)); + +async function post(body: Record) { + let res: Response; + let text = ''; + const out = await captureOutput(async () => { + res = await fetch(`${app.base}/chat/getBase64FromMediaMessage/test`, { + method: 'POST', + headers: { 'content-type': 'application/json', apikey: TOKEN }, + body: JSON.stringify(body), + }); + text = await res.text(); + }); + const headers = JSON.stringify([...res!.headers]); + return { status: res!.status, body: JSON.parse(text), leaks: leaks(`${text}\n${headers}\n${out}`) }; +} + +const failed = (status: number) => `The media could not be downloaded (HTTP ${status})`; + +describe('a failed media download never answers with the media link', () => { + it.each([ + ['a CDN 404, and the phone refuses the re-upload', GONE_404, refuses, {}, 404, { reupload: 'failed', reuploadReason: 'NOT_FOUND' }], + ['a CDN 410, and the phone does not answer in time', GONE_410, silent, {}, 410, { reupload: 'failed', reuploadReason: 'no_answer' }], + ['a CDN 403 on an expired link, and the phone refuses the re-upload', EXPIRED_403, refuses, {}, 403, { reupload: 'failed', reuploadReason: 'NOT_FOUND' }], + ['a CDN 404, the phone re-uploads, and the new link fails too', GONE_404, reuploads, {}, 404, { reupload: 'ok' }], + ['a CDN 403 on a link that has not expired: the phone is not asked', VALID_403, refuses, {}, 403, { reupload: 'not_requested' }], + ['reupload: false, and a CDN 403 on a link that has not expired', VALID_403, refuses, { reupload: false }, 403, { reupload: 'not_requested' }], + ])('%s', async (_name, link, answer, extra, status, facts) => { + phone = answer(); + const answered = await post({ message: image(link), ...extra }); + + // The download really went to the signed link, and Evolution's fallback to the same file. + expect(cdnLog[0]).toBe(`GET ${link}`); + expect(answered.status).toBe(400); + expect(answered.body).toEqual({ status: 400, error: 'Bad Request', response: { message: [failed(status)], ...facts } }); + // Not in the body, the headers, or anything logged meanwhile. + expect(answered.leaks).toEqual([]); + }); + + it('the error the service throws (what the error handler, its errors webhook and callers read) holds no link', async () => { + phone = reuploads(); + let thrown: any; + await captureOutput(async () => { + try { + await service.getBase64FromMediaMessage({ message: image(GONE_404) }); + } catch (e) { + thrown = e; + } + }); + + expect(thrown).toEqual({ status: 400, error: 'Bad Request', message: [failed(404)], reupload: 'ok' }); + expect(leaks(JSON.stringify(thrown))).toEqual([]); + expect(leaks(String(thrown?.stack ?? ''))).toEqual([]); + }); + + it('an incoming image the S3 upload cannot download: its error log does not name the link', async () => { + const { service: stored, ev } = await makeService({ profile: 'stored' }); + const config = stored.configService; + const get = config.get.bind(config); + config.get = (key: string) => (key === 'S3' ? { ...get('S3'), ENABLE: true, SAVE_VIDEO: true } : get(key)); + stored.client.updateMediaMessage = () => refuses()(); + const incoming = { + ...image(GONE_404), + key: { remoteJid: `${PHONE}@s.whatsapp.net`, fromMe: false, id: `${ID}-S3` }, + pushName: 'Sender', + }; + + const out = await captureOutput(() => deliver(stored, ev, { 'messages.upsert': { messages: [incoming], type: 'notify' } })); + + expect(cdnLog[0]).toBe(`GET ${GONE_404}`); + expect(out).toContain('Error on upload file to minio'); + expect(leaks(out)).toEqual([]); + // The message still reaches the webhook (with its own link, as every media message does), without an S3 copy. + const upsert = emitted.find((e) => e.event === 'messages.upsert'); + expect(upsert?.data?.key?.id).toBe(`${ID}-S3`); + expect(upsert?.data?.message?.mediaUrl).toBeUndefined(); + }); +}); diff --git a/test/chat/media-skip-reupload.test.ts b/test/chat/media-skip-reupload.test.ts index cb60250bc8..1dae523b50 100644 --- a/test/chat/media-skip-reupload.test.ts +++ b/test/chat/media-skip-reupload.test.ts @@ -153,10 +153,15 @@ describe('a media download can skip asking the phone to re-upload', () => { expect(asked).toEqual([]); // The first GET is the download; Evolution's own fallback then tries mmg.whatsapp.net, refused here. expect(cdn.log).toEqual([`GET ${VALID_LINK_403}`]); - // Its answer is that fallback's own failure, not the "no longer on WhatsApp's servers" 400. + // Its answer is that fallback's own failure, not the "no longer on WhatsApp's servers" 400: what + // kind of error and its network code, never its text (media-error-no-url.test.ts). expect(answer).toEqual({ status: 400, - body: { status: 400, error: 'Bad Request', response: { message: ['TypeError: fetch failed'], reupload: 'not_requested' } }, + body: { + status: 400, + error: 'Bad Request', + response: { message: ['The media could not be downloaded (TypeError, UND_MOCK_ERR_MOCK_NOT_MATCHED)'], reupload: 'not_requested' }, + }, }); }); From 458226b715997e6264d60ce1b4f8f96446f4844e Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Tue, 29 Sep 2026 11:35:28 +0300 Subject: [PATCH 150/157] fix: a failed media download answers without the media link getBase64FromMediaMessage answered a failed download with the error's own text, which for Baileys' download error is "Failed to fetch stream from" and the signed CDN link. Its 400 now carries a stable reason instead: the CDN's HTTP status ("The media could not be downloaded (HTTP 404)"), or, with no status, the kind of error and its network code. reupload and reuploadReason are unchanged, and a text Evolution threw itself (it names no link) stands. The object is built fresh, so no Boom data (data.url) travels with it: not to the HTTP answer, not to the errors webhook main.ts posts, and not to the S3 upload's error log. Co-Authored-By: Claude Opus 5.5 --- .../whatsapp/whatsapp.baileys.service.ts | 24 ++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts index a2ccc9640c..54329dc753 100644 --- a/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts +++ b/src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts @@ -273,6 +273,27 @@ const reuploadRefusal = (error: any): string => { return 'unknown'; }; +/** + * What a failed media download answers with, instead of the error's own text. Baileys' + * download error says "Failed to fetch stream from " and carries the link (Boom + * data.url); a WhatsApp media link is signed per message (oh, oe, the _nc_ parameters), so + * whoever holds it can fetch the file, and even its directPath alone names the file. So the + * answer says what failed and never where: the CDN's HTTP status, else the kind of error and + * its network code. A text Evolution threw itself (it names no link) stands, as does the + * message of an answer already built here (the MP4 conversion's 400). + */ +const mediaDownloadFailure = (error: any): string => { + if (typeof error === 'string') return error; + if (!(error instanceof Error) && Array.isArray(error?.message) && typeof error.message[0] === 'string') { + return error.message[0]; + } + const status = httpStatus(error); + if (typeof status === 'number') return `The media could not be downloaded (HTTP ${status})`; + const code = error?.cause?.code ?? error?.code; + const network = typeof code === 'string' && /^[A-Z][A-Z0-9_]{1,40}$/.test(code) ? `, ${code}` : ''; + return `The media could not be downloaded (${errorName(error)}${network})`; +}; + /** * When a WhatsApp media link stops working, in ms: its `oe` query parameter (hex unix * seconds). Read with URLSearchParams, so the name is case-sensitive, the value is @@ -5055,8 +5076,9 @@ export class BaileysStartupService extends ChannelStartupService { ); if (reupload === undefined) throw new BadRequestException(error.toString()); // The same 400, plus whether the phone was asked to re-upload the file, and why it refused. + // Never the error's own text, which names the signed media link (mediaDownloadFailure). try { - new BadRequestException(error.toString()); + new BadRequestException(mediaDownloadFailure(error)); } catch (badRequest) { throw { ...badRequest, reupload, ...(reuploadReason && { reuploadReason }) }; } From ac5a424c3b98b17b74e4936fd7ee0395ee20957a Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Tue, 29 Sep 2026 11:42:56 +0300 Subject: [PATCH 151/157] test: a signal key file is left torn by a crash, a failed write or a concurrent one The Prisma auth store writes each signal key over its file in place (fs.writeFile truncates, then writes), and reads a file that does not parse as no key at all. Four cases, on the real store and a real filesystem: - Crash: a child process (the real store, bundled with esbuild) writes one key over and over with 4 MB values that name their write, and is SIGKILLed at random moments, 64 times across 4 folders. After each kill the file must hold one whole value. On the code as it is, one run: 32 of 64 whole, 26 torn, 6 empty. - A write that fails partway: the child runs under a file size limit (ulimit -f), so the second write fails with EFBIG after 1 MiB. The error reaches keys.set, and the previous value must survive; it is torn. - Concurrency: twelve writes of the same key at once must end with the last one, whole; they end torn. - A value written is read back by the store and by a new one after a restart (passes today, kept as the guard for the fix). Co-Authored-By: Claude Opus 5.5 --- test/helpers/auth-writer/payload.ts | 21 +++ test/helpers/auth-writer/server-module.js | 12 ++ test/helpers/auth-writer/writer.ts | 39 ++++ test/unit/auth-key-atomic-write.test.ts | 211 ++++++++++++++++++++++ 4 files changed, 283 insertions(+) create mode 100644 test/helpers/auth-writer/payload.ts create mode 100644 test/helpers/auth-writer/server-module.js create mode 100644 test/helpers/auth-writer/writer.ts create mode 100644 test/unit/auth-key-atomic-write.test.ts diff --git a/test/helpers/auth-writer/payload.ts b/test/helpers/auth-writer/payload.ts new file mode 100644 index 0000000000..c8bebbb255 --- /dev/null +++ b/test/helpers/auth-writer/payload.ts @@ -0,0 +1,21 @@ +// A key value that is large and says which write it is: `fill` is `tag`, in base 36, +// repeated to `size` characters. A file that parses to a value whose fill is the one its +// tag and size give holds one complete write; anything else is torn. +export const payload = (tag: number, size: number) => { + const unit = `${tag.toString(36)}:`; + return { tag, size, fill: unit.repeat(Math.ceil(size / unit.length)).slice(0, size) }; +}; + +/** What a key file holds: one complete payload (its tag), or why not. */ +export function complete(raw: string | undefined): { tag: number } | 'missing' | 'empty' | 'torn' { + if (raw === undefined) return 'missing'; + if (raw === '') return 'empty'; + let value: any; + try { + value = JSON.parse(raw); + } catch { + return 'torn'; + } + const whole = Number.isSafeInteger(value?.tag) && Number.isSafeInteger(value?.size) && value.fill === payload(value.tag, value.size).fill; + return whole ? { tag: value.tag } : 'torn'; +} diff --git a/test/helpers/auth-writer/server-module.js b/test/helpers/auth-writer/server-module.js new file mode 100644 index 0000000000..29ceecd4ee --- /dev/null +++ b/test/helpers/auth-writer/server-module.js @@ -0,0 +1,12 @@ +// What the auth store gets for @api/server.module in the child process: the session table +// only, in memory (creds live there; keys are files). The real module builds the whole +// application at import. +const rows = new Map(); +exports.prismaRepository = { + session: { + findUnique: async ({ where }) => rows.get(where.sessionId) ?? null, + create: async ({ data }) => (rows.set(data.sessionId, { ...data }), rows.get(data.sessionId)), + update: async ({ where, data }) => (rows.set(where.sessionId, { ...rows.get(where.sessionId), ...data }), rows.get(where.sessionId)), + delete: async ({ where }) => rows.delete(where.sessionId), + }, +}; diff --git a/test/helpers/auth-writer/writer.ts b/test/helpers/auth-writer/writer.ts new file mode 100644 index 0000000000..1c24687479 --- /dev/null +++ b/test/helpers/auth-writer/writer.ts @@ -0,0 +1,39 @@ +// A child process around the real Prisma auth store, whose keys are files under +// INSTANCE_DIR (./instances of the process's cwd). test/unit/auth-key-atomic-write.test.ts +// bundles it with esbuild (aliases from tsconfig, @api/server.module replaced by +// server-module.js) and runs it with node. +// +// loop prints "ready" once the store is open, then writes +// the key `session-` over and over, tag from, from+1..., +// until it is killed. +// fault writes tag 1 (small) and prints "wrote 1", then tag 2 +// (`size`), and prints "rejected " if that write +// rejects, else "wrote 2". +import useMultiFileAuthStatePrisma from '@utils/use-multi-file-auth-state-prisma'; + +import { payload } from './payload'; + +async function main() { + const [mode, session, id, ...rest] = process.argv.slice(2); + const auth = await useMultiFileAuthStatePrisma(session, null as any); + const write = (tag: number, size: number) => auth.state.keys.set({ session: { [id]: payload(tag, size) } } as any); + if (mode === 'loop') { + const [from, size] = rest.map(Number); + process.stdout.write('ready\n'); + for (let tag = from; ; tag++) await write(tag, size); + } + if (mode === 'fault') { + await write(1, 1000); + process.stdout.write('wrote 1\n'); + const outcome = await write(2, Number(rest[0])).then( + () => 'wrote 2', + (error) => `rejected ${error?.code ?? error?.name}`, + ); + process.stdout.write(`${outcome}\n`); + } +} + +main().catch((error) => { + process.stderr.write(`${error?.stack ?? error}\n`); + process.exit(1); +}); diff --git a/test/unit/auth-key-atomic-write.test.ts b/test/unit/auth-key-atomic-write.test.ts new file mode 100644 index 0000000000..d6e9515f77 --- /dev/null +++ b/test/unit/auth-key-atomic-write.test.ts @@ -0,0 +1,211 @@ +// The Prisma auth store (the one production runs, DATABASE_SAVE_DATA_INSTANCE=true) +// keeps creds in the database and every signal key (sessions, pre-keys, sender +// keys, app-state sync keys) as a JSON file under INSTANCE_DIR. It wrote a key +// with fs.writeFile over the file in place: the file is truncated first and the +// bytes follow, so a process that dies meanwhile (a crash, an OOM kill, a +// deploy's SIGKILL), a disk that fills up, or two writes of the same key at +// once leave a torn or empty file. The store reads a file that does not parse +// as no key at all, so a torn session key silently becomes a lost session. +// +// A key file on disk must always hold one complete value: the previous one or +// the new one. The crash test proves it the only way it can be proved: a child +// process runs the real store on a real filesystem and writes one key over and +// over with large, distinguishable values (4 MB each, so a write spans many +// syscalls and the window is real), and this process SIGKILLs it at random +// moments, dozens of times, reading the file after each kill. +import { vi } from 'vitest'; + +const tmp = await vi.hoisted(async () => { + const { mkdtempSync, realpathSync } = await import('node:fs'); + const { tmpdir } = await import('node:os'); + const { join } = await import('node:path'); + return realpathSync(mkdtempSync(join(tmpdir(), 'evo-auth-atomic-'))); +}); +vi.mock('@api/server.module', () => import('../helpers/fake-server-module')); +// The child's INSTANCE_DIR is ./instances of its cwd (path.config), and its cwd is `tmp`. +vi.mock('@config/path.config', async (importOriginal) => ({ + ...(await importOriginal()), + INSTANCE_DIR: (await import('node:path')).join(tmp, 'instances'), +})); + +import { type ChildProcess, spawn } from 'node:child_process'; +import { existsSync, readdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { mkdir } from 'node:fs/promises'; +import { join } from 'node:path'; +import { createInterface } from 'node:readline'; +import { fileURLToPath } from 'node:url'; + +import { build } from 'esbuild'; +import { afterAll, beforeAll, describe, expect, it } from 'vitest'; + +import { complete, payload } from '../helpers/auth-writer/payload'; + +const REPO = fileURLToPath(new URL('../../', import.meta.url)); +const WRITER = join(tmp, 'writer.cjs'); +/** A session key id as Baileys names one (`session-.`). */ +const ID = '972500000001.0'; +const KEY_FILE = `session-${ID}.json`; +const SIZE = 4_000_000; +const children = new Set(); + +beforeAll(async () => { + // The real store and everything it imports, bundled once; packages stay external and are + // found through NODE_PATH. The logger reads ./package.json from the cwd. + await build({ + entryPoints: [join(REPO, 'test/helpers/auth-writer/writer.ts')], + bundle: true, + platform: 'node', + format: 'cjs', + target: 'node20', + outfile: WRITER, + packages: 'external', + logLevel: 'error', + plugins: [ + { + name: 'server-module', + setup(b) { + b.onResolve({ filter: /^@api\/server\.module$/ }, () => ({ path: join(REPO, 'test/helpers/auth-writer/server-module.js') })); + }, + }, + ], + }); + writeFileSync(join(tmp, 'package.json'), '{"version":"0.0.0"}'); +}); + +afterAll(() => { + children.forEach((c) => c.kill('SIGKILL')); + rmSync(tmp, { recursive: true, force: true }); +}); + +/** The writer in a child process; `fileSizeBlocks` caps the size of any file it writes (ulimit -f, 512-byte blocks). */ +function writer(args: string[], fileSizeBlocks?: number) { + const env = { ...process.env, NODE_PATH: join(REPO, 'node_modules'), CACHE_REDIS_ENABLED: 'false' }; + const argv = [WRITER, ...args]; + const child = fileSizeBlocks + ? spawn('/bin/sh', ['-c', `ulimit -f ${fileSizeBlocks} && exec "$0" "$@"`, process.execPath, ...argv], { cwd: tmp, env }) + : spawn(process.execPath, argv, { cwd: tmp, env }); + children.add(child); + const lines: string[] = []; + let stderr = ''; + child.stderr.on('data', (d) => (stderr += d)); + const waiting: { prefix: string; resolve: (line: string) => void }[] = []; + createInterface({ input: child.stdout }).on('line', (line) => { + lines.push(line); + for (const w of waiting.filter((w) => line.startsWith(w.prefix))) w.resolve(line); + }); + const exited = new Promise((resolve) => + child.once('exit', (code, signal) => { + children.delete(child); + resolve(signal ?? code); + }), + ); + /** The first line that starts with `prefix`; fails if the child exits first. */ + const line = (prefix: string) => + Promise.race([ + new Promise((resolve) => waiting.push({ prefix, resolve })), + exited.then((how) => Promise.reject(new Error(`writer exited (${how}) before "${prefix}": ${stderr}`))), + ]); + return { child, lines, line, exited, stderr: () => stderr }; +} + +const folder = (session: string) => join(tmp, 'instances', session); +const onDisk = (session: string) => { + const file = join(folder(session), KEY_FILE); + return complete(existsSync(file) ? readFileSync(file, 'utf8') : undefined); +}; +/** Whatever is in the session's folder besides the key file. */ +const strays = (session: string) => readdirSync(folder(session)).filter((f) => f !== KEY_FILE); + +const open = async (session: string) => { + const { default: useMultiFileAuthStatePrisma } = await import('@utils/use-multi-file-auth-state-prisma'); + return useMultiFileAuthStatePrisma(session, null as any); +}; +const read = async (auth: any) => (await auth.state.keys.get('session', [ID]))[ID]; + +describe('a signal key file is never left half-written', () => { + it( + 'killed at random moments while writing, the key file always holds one whole value', + async () => { + const LANES = 4; + const KILLS = 16; + const tally = { kills: 0, whole: 0, torn: 0, empty: 0, missing: 0 }; + let mostStrays = 0; + + await Promise.all( + Array.from({ length: LANES }, async (_, lane) => { + const session = `crash-${lane}`; + await mkdir(folder(session), { recursive: true }); + // The value before the first kill: a complete write of tag 0. + writeFileSync(join(folder(session), KEY_FILE), JSON.stringify(payload(0, SIZE))); + for (let k = 0; k < KILLS; k++) { + // Each child writes its own tags, so a whole value also says who wrote it. + const w = writer(['loop', session, ID, String((lane * KILLS + k + 1) * 1_000_000), String(SIZE)]); + await w.line('ready'); + // The random moment: somewhere in the child's stream of writes. + await new Promise((r) => setTimeout(r, Math.random() * 60)); + w.child.kill('SIGKILL'); + expect(await w.exited).toBe('SIGKILL'); + + const state = onDisk(session); + tally.kills++; + if (typeof state === 'object') tally.whole++; + else tally[state]++; + mostStrays = Math.max(mostStrays, strays(session).length); + } + }), + ); + + expect(tally).toEqual({ kills: LANES * KILLS, whole: LANES * KILLS, torn: 0, empty: 0, missing: 0 }); + // A kill can leave the write it interrupted behind, never more: the next start clears it. + expect(mostStrays).toBeLessThanOrEqual(1); + + // A restart (a new store over the same folder) reads the value on disk, and clears what a kill left. + for (let lane = 0; lane < LANES; lane++) { + const session = `crash-${lane}`; + const state = onDisk(session) as { tag: number }; + const auth = await open(session); + expect(strays(session)).toEqual([]); + expect((await read(auth))?.tag).toBe(state.tag); + } + }, + 120_000, + ); + + it('a write that fails partway (the file size limit) keeps the previous value, reaches the caller, and leaves no temp file', async () => { + const session = 'fault'; + // 1 MiB: tag 1 (1 kB) fits, tag 2 (4 MB) fails with EFBIG after the first MiB is written. + const w = writer(['fault', session, ID, String(SIZE)], 2048); + + expect(await w.exited).toBe(0); + expect(w.stderr()).toBe(''); + expect(w.lines).toEqual(['wrote 1', 'rejected EFBIG']); + expect(onDisk(session)).toEqual({ tag: 1 }); + expect(strays(session)).toEqual([]); + }); + + it('concurrent writes of one key end with the last one, whole', async () => { + const session = 'concurrent'; + const auth = await open(session); + // Each value smaller than the one before, so the last to be asked for is the first to be written. + const size = (tag: number) => 1_000_000 + (12 - tag) * 250_000; + const tags = Array.from({ length: 12 }, (_, i) => i + 1); + + await Promise.all(tags.map((tag) => auth.state.keys.set({ session: { [ID]: payload(tag, size(tag)) } } as any))); + + expect(onDisk(session)).toEqual({ tag: 12 }); + expect(strays(session)).toEqual([]); + expect(await read(auth)).toEqual(payload(12, size(12))); + }); + + it('what was written is read back, by the same store and by a new one after a restart', async () => { + const session = 'restart'; + const auth = await open(session); + await auth.state.keys.set({ session: { [ID]: payload(7, 10_000) } } as any); + expect(await read(auth)).toEqual(payload(7, 10_000)); + + vi.resetModules(); + const restarted = await open(session); + expect(await read(restarted)).toEqual(payload(7, 10_000)); + expect(strays(session)).toEqual([]); + }); +}); From d955eb721a7d776cb602ec5a77bb312d09a7e21b Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Tue, 29 Sep 2026 11:44:40 +0300 Subject: [PATCH 152/157] fix: signal key files are replaced whole, never written in place The Prisma auth store now writes a key through writeFileAtomic (src/utils/atomic-file.ts): the value goes to a temp file in the same directory (opened exclusively), is flushed with fsync, and is renamed over the key file, which POSIX makes atomic; the directory is then flushed, so the rename survives a power loss (renames that land together share one directory flush). A failed write removes its temp file, leaves the previous value and rejects, so keys.set fails as saveCreds does. Writes of the same file run one at a time in call order, so the last one asked for stays. A killed writer can leave its temp file behind; the store removes those when it opens its folder (a temp file of a process that no longer runs, or this process's own that it is not writing), and leaves one of another running process alone. Co-Authored-By: Claude Opus 5.5 --- src/utils/atomic-file.ts | 129 ++++++++++++++++++ src/utils/use-multi-file-auth-state-prisma.ts | 6 +- 2 files changed, 134 insertions(+), 1 deletion(-) create mode 100644 src/utils/atomic-file.ts diff --git a/src/utils/atomic-file.ts b/src/utils/atomic-file.ts new file mode 100644 index 0000000000..6e7575da9b --- /dev/null +++ b/src/utils/atomic-file.ts @@ -0,0 +1,129 @@ +import { randomBytes } from 'crypto'; +import { open, readdir, rename, unlink } from 'fs/promises'; +import { basename, dirname, join } from 'path'; + +// Session key material on disk is replaced whole or not at all. A file written in place +// (fs.writeFile truncates it, then writes) is torn or empty when the process dies, the disk +// fills up or two writes of it overlap, and a key file that does not parse reads as no key. +// So the new value goes to a temp file in the same directory, is flushed (fsync), and is +// renamed over the target, which POSIX makes atomic; the directory is then flushed, so the +// rename itself survives a power loss. + +/** `....tmp`, next to the file it replaces. */ +const TEMP = /^\..+\.(\d+)\.[0-9a-f]{8}\.tmp$/; + +/** Temp files this process is writing now. */ +const writing = new Set(); + +/** Per file, the write under way or queued last: writes of one file run one at a time, in call order. */ +const queues = new Map>(); + +/** + * Replace `file` with `data`, atomically: a reader, a crash or a failure at any moment sees the + * previous content or the new one, never a mix, and never an empty file. Writes of the same + * file run in the order they were asked for, so the last one asked for is the one that stays. + * A failed write rejects, removes its temp file and leaves the previous content. + */ +export function writeFileAtomic(file: string, data: string | Uint8Array): Promise { + const previous = queues.get(file) ?? Promise.resolve(); + const write = previous.catch(() => undefined).then(() => replace(file, data)); + queues.set(file, write); + write + .finally(() => { + if (queues.get(file) === write) queues.delete(file); + }) + .catch(() => undefined); + return write; +} + +async function replace(file: string, data: string | Uint8Array) { + const dir = dirname(file); + const temp = join(dir, `.${basename(file)}.${process.pid}.${randomBytes(4).toString('hex')}.tmp`); + writing.add(temp); + try { + const handle = await open(temp, 'wx'); + try { + await handle.writeFile(data); + await handle.sync(); + } finally { + await handle.close(); + } + await rename(temp, file); + } catch (error) { + await unlink(temp).catch(() => undefined); + throw error; + } finally { + writing.delete(temp); + } + await syncDirectory(dir); +} + +/** Per directory, a flush that has not started yet: every rename before it starts is covered by it. */ +const pendingSyncs = new Map>(); +/** Per directory, the flush running now. */ +const runningSyncs = new Map>(); + +/** Flush a directory after a rename in it. Renames that land together share one flush. */ +function syncDirectory(dir: string): Promise { + // Windows cannot open a directory to flush it. + if (process.platform === 'win32') return Promise.resolve(); + const pending = pendingSyncs.get(dir); + if (pending) return pending; + const next = (runningSyncs.get(dir) ?? Promise.resolve()) + .catch(() => undefined) + .then(() => { + pendingSyncs.delete(dir); + const running = flush(dir); + runningSyncs.set(dir, running); + running + .finally(() => { + if (runningSyncs.get(dir) === running) runningSyncs.delete(dir); + }) + .catch(() => undefined); + return running; + }); + pendingSyncs.set(dir, next); + return next; +} + +async function flush(dir: string) { + const handle = await open(dir, 'r'); + try { + await handle.sync(); + } finally { + await handle.close(); + } +} + +/** Whether a process with this pid runs (EPERM: it does, as another user). */ +function alive(pid: number) { + try { + process.kill(pid, 0); + return true; + } catch (error) { + return error?.code === 'EPERM'; + } +} + +/** + * Remove the temp files a killed writer left in `dir`: those of a process that no longer runs, + * and this process's own that it is not writing now (a pid a restart reused). A temp file of + * another running process is left alone. Run when a store opens its directory. + */ +export async function removeStaleTempFiles(dir: string): Promise { + let entries: string[]; + try { + entries = await readdir(dir); + } catch { + return; + } + await Promise.all( + entries.map(async (entry) => { + const pid = Number(TEMP.exec(entry)?.[1]); + if (!pid) return; + const path = join(dir, entry); + if (pid === process.pid ? writing.has(path) : alive(pid)) return; + await unlink(path).catch(() => undefined); + }), + ); +} diff --git a/src/utils/use-multi-file-auth-state-prisma.ts b/src/utils/use-multi-file-auth-state-prisma.ts index db174d78b5..908b0b30d6 100644 --- a/src/utils/use-multi-file-auth-state-prisma.ts +++ b/src/utils/use-multi-file-auth-state-prisma.ts @@ -3,6 +3,7 @@ import { CacheService } from '@api/services/cache.service'; import { CacheConf, configService } from '@config/env.config'; import { Logger } from '@config/logger.config'; import { INSTANCE_DIR } from '@config/path.config'; +import { removeStaleTempFiles, writeFileAtomic } from '@utils/atomic-file'; import { AuthenticationState, BufferJSON, initAuthCreds, WAProto as proto } from 'baileys'; import fs from 'fs/promises'; import path from 'path'; @@ -91,6 +92,8 @@ export default async function useMultiFileAuthStatePrisma( const localFolder = path.join(INSTANCE_DIR, sessionId); const localFile = (key: string) => path.join(localFolder, fixFileName(key) + '.json'); await fs.mkdir(localFolder, { recursive: true }); + // What a writer killed mid-write left behind; the key files themselves are always whole. + await removeStaleTempFiles(localFolder); async function writeData(data: any, key: string): Promise { const dataString = JSON.stringify(data, BufferJSON.replacer); @@ -100,7 +103,8 @@ export default async function useMultiFileAuthStatePrisma( if (cacheConfig.REDIS.ENABLED) { return await cache.hSet(sessionId, key, data); } else { - await fs.writeFile(localFile(key), dataString); + // Replaced whole, never in place: a torn key file reads as no key (atomic-file.ts). + await writeFileAtomic(localFile(key), dataString); return; } } From 2ad6311ecf6a7a502b69c8baa93de2be7380acc2 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Tue, 29 Sep 2026 11:45:17 +0300 Subject: [PATCH 153/157] test: a pending-logout marker write that fails partway leaves half a marker The marker next to the session's key files is written in place (fs.writeFile). A write that fails partway (injected here as a full disk: half the bytes land, then ENOSPC) rejects as it should, but leaves half a marker over the whole one, which reads as pending with no instance name. The previous marker must survive, with no temp file left. Co-Authored-By: Claude Opus 5.5 --- test/unit/logout-marker-atomic.test.ts | 89 ++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) create mode 100644 test/unit/logout-marker-atomic.test.ts diff --git a/test/unit/logout-marker-atomic.test.ts b/test/unit/logout-marker-atomic.test.ts new file mode 100644 index 0000000000..8c7dd19533 --- /dev/null +++ b/test/unit/logout-marker-atomic.test.ts @@ -0,0 +1,89 @@ +// The pending-logout marker (logout-pending.json, next to the session's key +// files) is what finishes a logout WhatsApp was never told about, after a +// restart that lost the database row's record of it. It was written in place +// (fs.writeFile), so a write that failed partway, a full disk here, left half a +// marker over the whole one: it reads as pending, but with no instance name. +// +// The write fails the way a full disk fails it: the first half of the bytes +// lands, then ENOSPC. It is injected into fs/promises for whichever way the +// marker is written (writeFile on a path, or on an open file handle). +import { vi } from 'vitest'; + +const tmp = await vi.hoisted(async () => { + const { mkdtempSync } = await import('node:fs'); + const { tmpdir } = await import('node:os'); + const { join } = await import('node:path'); + return mkdtempSync(join(tmpdir(), 'evo-marker-atomic-')); +}); +const fault = vi.hoisted(() => ({ armed: false })); + +vi.mock('@config/path.config', async (importOriginal) => ({ ...(await importOriginal()), INSTANCE_DIR: tmp })); +vi.mock('fs/promises', async (importOriginal) => { + const real: any = await importOriginal(); + const enospc = () => Object.assign(new Error('ENOSPC: no space left on device, write'), { code: 'ENOSPC', errno: -28, syscall: 'write' }); + const half = (data: any) => { + const bytes = Buffer.from(data); + return bytes.subarray(0, Math.floor(bytes.length / 2)); + }; + const writeFile = async (path: any, data: any, ...rest: any[]) => { + if (!fault.armed) return real.writeFile(path, data, ...rest); + await real.writeFile(path, half(data)); + throw enospc(); + }; + const open = async (...args: any[]) => { + const handle = await real.open(...args); + if (!fault.armed) return handle; + return new Proxy(handle, { + get(target, prop) { + if (prop === 'writeFile' || prop === 'write') { + return async (data: any) => { + await target.write(half(data)); + throw enospc(); + }; + } + const value = Reflect.get(target, prop); + return typeof value === 'function' ? value.bind(target) : value; + }, + }); + }; + const faulty = { ...real, writeFile, open }; + return { ...faulty, default: faulty }; +}); + +import { readdirSync, rmSync } from 'node:fs'; +import { join } from 'node:path'; + +import { afterAll, describe, expect, it } from 'vitest'; + +import { LOGOUT_MARKER_FILE, readLogoutMarker, writeLogoutMarker } from '@utils/logout-marker'; + +afterAll(() => rmSync(tmp, { recursive: true, force: true })); + +describe('the pending-logout marker is never left half-written', () => { + it('a marker write that fails partway keeps the previous marker, rejects, and leaves no temp file', async () => { + const id = 'inst-marker'; + const first = { instanceName: 'first-name', deleted: false, since: new Date(0).toISOString() }; + const second = { instanceName: 'second-name', deleted: true, since: new Date(1000).toISOString() }; + await writeLogoutMarker(id, first); + + fault.armed = true; + let outcome: string; + try { + outcome = await writeLogoutMarker(id, second).then( + () => 'written', + (error) => error?.code, + ); + } finally { + fault.armed = false; + } + + expect(outcome).toBe('ENOSPC'); + expect(readLogoutMarker(id)).toEqual(first); + expect(readdirSync(join(tmp, id))).toEqual([LOGOUT_MARKER_FILE]); + + // The next write, with room on the disk, replaces it. + await writeLogoutMarker(id, second); + expect(readLogoutMarker(id)).toEqual(second); + expect(readdirSync(join(tmp, id))).toEqual([LOGOUT_MARKER_FILE]); + }); +}); From ddb4e2a4cd6cfbeace5be6e5cb57ef8af1333709 Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Tue, 29 Sep 2026 11:46:13 +0300 Subject: [PATCH 154/157] fix: the pending-logout marker is replaced whole, never written in place writeLogoutMarker writes through writeFileAtomic, as the key files do: a temp file in the same directory, fsync, rename over the marker, fsync of the directory. A write that fails keeps the previous marker, removes its temp file and rejects, so the logout or delete still answers 500 as before. Co-Authored-By: Claude Opus 5.5 --- src/utils/logout-marker.ts | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/utils/logout-marker.ts b/src/utils/logout-marker.ts index e947d35899..62f0166741 100644 --- a/src/utils/logout-marker.ts +++ b/src/utils/logout-marker.ts @@ -1,6 +1,7 @@ import { INSTANCE_DIR } from '@config/path.config'; +import { writeFileAtomic } from '@utils/atomic-file'; import { existsSync, readFileSync } from 'fs'; -import { mkdir, writeFile } from 'fs/promises'; +import { mkdir } from 'fs/promises'; import { join } from 'path'; /** @@ -30,5 +31,6 @@ export function readLogoutMarker(instanceId: string): LogoutMarker | undefined { export async function writeLogoutMarker(instanceId: string, marker: LogoutMarker) { await mkdir(join(INSTANCE_DIR, instanceId), { recursive: true }); - await writeFile(logoutMarkerPath(instanceId), JSON.stringify(marker)); + // Replaced whole: a marker half-written over the last one would lose the instance's name. + await writeFileAtomic(logoutMarkerPath(instanceId), JSON.stringify(marker)); } From 95c844b31f455f090a038f1842c0b69093c0adcf Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Tue, 29 Sep 2026 11:46:27 +0300 Subject: [PATCH 155/157] docs(fork): list media errors without the link, and session files replaced whole Co-Authored-By: Claude Opus 5.5 --- FORK.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/FORK.md b/FORK.md index 7534f0f5d1..b6dc0c5830 100644 --- a/FORK.md +++ b/FORK.md @@ -59,6 +59,7 @@ named where one exists. - A media download records whether it asked the phone to re-upload an expired file and how that ended (a bounded log line, and `reupload` on the download's error), and when the phone refused, why (`reason=` on the log line, `reuploadReason` on the error and the HTTP answer: the phone's result such as `NOT_FOUND`, `error_`, `missing_ciphertext`, or `no_answer`). Baileys 7.0.0-rc14 never asks on a CDN 404 or 410 (its check misses the status), so Evolution asks the phone itself, once per download, for at most 60s. A CDN 403 counts as expired only when the link that actually failed (the one on Baileys' error, which is the directPath when the message has one, else the url) carries exactly one `oe` query parameter, in hex unix seconds, that has passed by the local clock. This is a conservative heuristic: on 84 history-sync attachments, 403 answered 34 of 34 expired links and 0 of 50 valid ones, where a valid link to a dropped file answered 404 or 410. - A media download turns the media key back into bytes (from a base64 string, or the object JSON makes of a Uint8Array or a Buffer) before it asks the phone to re-upload, since Baileys 7.0.0-rc14 derives the re-upload's key from the value as given and then cannot decrypt the phone's answer (Baileys #2729 proposed the same fix there). - `POST /chat/getBase64FromMediaMessage` takes an optional `reupload` (boolean, default true). With `false` the phone is never asked to re-upload, and a file gone from the CDN (404, 410, or 403 on an expired link) fails at once with a 400 that says so; the HTTP error carries `reupload` for any failed download. +- A failed media download never answers with the media link. The error's own text (Baileys' "Failed to fetch stream from" and the signed CDN link, whose `oh`, `oe` and `_nc_*` parameters let anyone holding it fetch the file, and whose directPath alone names it) used to be the message of the HTTP answer, of what the error handler posts to the errors webhook, and of the S3 upload's error log. The answer now says what failed, never where: `The media could not be downloaded (HTTP )`, or the kind of error and its network code when there is no status, with `reupload` and `reuploadReason` as before; a text Evolution threw itself (it names no link) stands. **Proxy** - Media downloads, media uploads and the WhatsApp Web version fetch leave through the instance's proxy (uploads failed outright on a proxied instance). @@ -69,6 +70,7 @@ named where one exists. - A logout or delete while the socket is down keeps the session until WhatsApp is told, so the device leaves the phone's Linked devices; it answers `202 PENDING` meanwhile and forwards nothing (#2520, #2508 in part). The pending logout is recorded on the instance's row (`disconnectionObject.logoutPending`) as well as in a file next to the session keys, so a restart that lost the instances volume still finishes it; when it cannot be recorded, the logout or delete answers 500 instead of 202. A logout is finished only on WhatsApp's word: its answer to remove-companion-device, or its refusal of the device on the next connection; the socket's own close after sending the request (all Baileys' `logout()` waits for) is not one. Concurrent logouts and deletes share one run and answer with its outcome. A deleted instance keeps its database row until then (Session and Proxy cascade from it), out of the API, its name taken and its token cleared. - Reconnects back off from 1s to 60s instead of spinning, the version fetch times out after 10s, and an instance never runs two sockets: a reload after a profile or privacy change joins a connect under way, and once an instance is removed nothing builds a socket for it, runs a batch it had queued, or forwards anything (#2134, #2184, #2430; ideas from #2732). A connect that fails at boot keeps the instance in the API and is retried on the same backoff. - A creds write that fails is logged and tried again until it lands, and stored creds that cannot be parsed fail the connect instead of being replaced by fresh ones. +- Session files on disk are replaced whole, never written in place: each signal key file of the Prisma auth store (keys live under INSTANCE_DIR next to creds in the database) and the pending-logout marker go to a temp file in the same directory, are flushed, renamed over the target and the directory flushed. A process killed mid-write, a full disk or two writes of one key at once used to leave a torn or empty file, and a key file that does not parse reads as no key; tested by SIGKILLing a writer at random moments (half the files torn or empty before, none after). A failed write keeps the previous file and rejects; writes of one file run in call order; a temp file a killed writer left is removed when the store next opens. The provider-files store writes on its own server and Redis holds no files, so neither is covered. - Profile pictures are looked up once per contact per hour, at most four at a time, and history never waits on them (#1883). The picture update a history batch sends carries each contact's newest name, and a lookup that finishes after WhatsApp said the picture changed or was removed is dropped. - One pairing code per connect attempt, and a fresh QR budget per attempt (#2100, #2696). From 5daf1fae8cafe3c3988e7a0e950ce9eaae82449a Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Tue, 29 Sep 2026 12:10:42 +0300 Subject: [PATCH 156/157] docs(fork): the trademark policy lives upstream, not in this tree NOTICE said the policy was included in this repository; 2.3.7 does not carry TRADEMARKS.md, so it now links the upstream file. It also states that the fork does not change the user interface or its brand assets. Co-Authored-By: Claude Opus 5.5 --- NOTICE | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/NOTICE b/NOTICE index 6a6a89ad29..0f3727630b 100644 --- a/NOTICE +++ b/NOTICE @@ -7,8 +7,9 @@ This product includes software developed by Evolution Foundation Trademark notice "Evolution Foundation", "Evolution" and "Evolution API" are trademarks of Evolution Foundation. The Evolution API logo, wordmark, and visual identity -are governed by the Trademark and Brand Assets Policy included in this -repository (TRADEMARKS.md). +are governed by Evolution Foundation's Trademark and Brand Assets Policy +(https://github.com/evolution-foundation/evolution-api/blob/main/TRADEMARKS.md). +This fork does not change the Evolution API user interface or its brand assets. Third-party attributions From 34979cddc629b13999feca473cb2da1b8edb12db Mon Sep 17 00:00:00 2001 From: Almog Cohen <3888998+AlmogCohen@users.noreply.github.com> Date: Tue, 29 Sep 2026 19:41:00 +0300 Subject: [PATCH 157/157] docs(fork): record the 2026-09-29 live results (logout on an open socket, a pairing-code link) Co-Authored-By: Claude Opus 5.5 --- docs/LIVE-CHECKS.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/LIVE-CHECKS.md b/docs/LIVE-CHECKS.md index 3f5f4c7596..4cb37ffc63 100644 --- a/docs/LIVE-CHECKS.md +++ b/docs/LIVE-CHECKS.md @@ -217,6 +217,8 @@ it is not a guess. | 2026-09-27 | `group-rename-participants` | bc522750 | 7.0.0-rc14 | 2.3000.1048596303 | smbi (WhatsApp Business) / iPhone 16, iOS 18.6 / WhatsApp Business 25.24 | PASS: the rename reached `groups.update` with the new subject. PASS: a member removed and added back reached `group-participants.update` with `participantsData` holding the @lid as `jid` and the phone JID as `phoneNumber`. Promote and demote not run. Replayed in `test/live/group-rename-participants.test.ts` | | 2026-09-27 | `live-lid-message-key` | bc522750 | 7.0.0-rc14 | 2.3000.1048596303 | smbi (WhatsApp Business) / iPhone 16, iOS 18.6 / WhatsApp Business 25.24 | PASS: a text and an image in a DM addressed by @lid reached `messages.upsert` with the phone JID as `remoteJid`, the @lid kept in `remoteJidAlt`, `addressingMode: 'pn'`. Replayed in `test/live/live-lid-message-key.test.ts` | | 2026-09-27 | `clean-logs` | bc522750 | 7.0.0-rc14 | 2.3000.1048596303 | smbi (WhatsApp Business) / iPhone 16, iOS 18.6 / WhatsApp Business 25.24 | PASS: production-level logs over the session, 49 lines: 0 digit runs of 8+, 0 WhatsApp addresses, 0 media URLs | +| 2026-09-29 | `logout-reaches-phone` | 95c844b3 | 7.0.0-rc14 | not recorded | smbi (WhatsApp Business) / iPhone 16 / not recorded | PASS in part: a logout through the API with the socket open was not left pending, and WhatsApp closed the session with 401 (logged out) in the same second. Linked devices on the phone was not checked this time | +| 2026-09-29 | `pairing-code-over-45s` | 95c844b3 | 7.0.0-rc14 | not recorded | smbi (WhatsApp Business) / iPhone 16 / not recorded | PARTIAL, not the check itself: linked by pairing code 33s after the code was requested, the code typed within seconds, so the 45s window was not tested. One phone notification per code request, although `qrcode.updated` carried the code five times in an earlier attempt: consistent with one code per connect attempt. That earlier attempt failed on the phone ("Couldn't link device") and its window closed with 401 after 216s; cause not found (logs at WARN, no recorder). The phone's prompt named the device `Chrome (Mac OS)`, Baileys' default, since number mode sends no configured browser (a custom label there is rejected by WhatsApp: WhiskeySockets/Baileys#2560) | The phone platform and model were not recorded for the earlier runs: the recorder did not exist yet, and the operator did not write them down. The rig