Add a fast end-to-end release check

.github/workflows/tests.yml

@@ -25,8 +25,12 @@ jobs:
       - name: Run the test suite (standard library only, no install step)
         working-directory: scan
         run: python -m unittest discover -s tests -v
+      - name: Check source and download with no optional dependencies
+        if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.8'
+        run: python tools/release_check.py --e2e-only
 
   smoke-scan:
+    timeout-minutes: 10
     # Prove the scan itself runs end to end on real hardware for each OS, not
     # just the unit tests. The runners have no real browser data, so the result
     # is an honest "nothing found / with gaps" - the point is that it executes
@@ -61,6 +65,9 @@ jobs:
         run: python -m pip install -r requirements-patterns.txt
       - name: Test optional patterns and default checks using harmless fixtures
         run: python -B -m unittest discover -s scan/tests -v
+      - name: End-to-end source and download release fixtures
+        if: matrix.python-version == '3.12'
+        run: python tools/release_check.py --e2e-only --require-patterns
 
   reproducible-build:
     # The build must be deterministic and the committed artifacts up to date, so

.gitignore

@@ -14,6 +14,7 @@ __pycache__/
 *.pyc
 *.pyo
 .pytest_cache/
+.venv/
 .DS_Store
 
 # The single-file build is a release artifact, regenerated by build/bundle.py.

CONTRIBUTING.md

@@ -13,6 +13,12 @@ cd scan && python -m unittest discover -s tests
 
 Every change must keep the suite green. New behaviour needs a test.
 
+Before publishing, run `python tools/release_check.py --require-patterns` in
+the virtual environment with the pinned optional engine installed. This builds
+and tests the download against controlled fixtures without scanning your own
+browser profiles. See [TESTING.md](TESTING.md) for setup, website checks, scope
+and the short release checklist.
+
 ## Rebuilding the generated files
 
 The STIX/MISP feeds, the blocklists, the indicator checksum and the single-file

README.md

@@ -336,6 +336,11 @@ file shipped with the signed release, not later development-branch updates; see
 
 ## Tests
 
+Maintainers: run `python tools/release_check.py --require-patterns` before
+publishing. It builds the download and exercises source/download fixture scans,
+reports, optional patterns and the webpage when supplied. Setup and the short
+release checklist are in [TESTING.md](TESTING.md).
+
 ```
 cd scan && python -m unittest discover -s tests
 ```

TESTING.md

@@ -0,0 +1,98 @@
+# Before publishing
+
+Run the release check from the repository root. It uses temporary, harmless
+fixtures rather than your browser profiles, registry or real DNS history.
+No VM, malware samples, administrator account or test service is needed.
+
+## One-time optional-engine setup
+
+Use Python 3.9+ for the pinned optional YARA-X engine (Python 3.12 is used in CI):
+
+```text
+python -m venv .venv
+```
+
+On Windows:
+
+```powershell
+.venv\Scripts\python.exe -m pip install -r requirements-patterns.txt
+.venv\Scripts\python.exe tools/release_check.py --require-patterns
+```
+
+On macOS/Linux:
+
+```sh
+.venv/bin/python -m pip install -r requirements-patterns.txt
+.venv/bin/python tools/release_check.py --require-patterns
+```
+
+Use the same interpreter for installation and tests. The runner never installs
+anything. For ordinary standard-library development, `python
+tools/release_check.py` works without YARA-X; its summary explicitly records the
+untested optional positives as a skip. Use `--require-patterns` before release.
+
+## What the command checks
+
+- The existing unit/regression suite.
+- Two byte-identical builds of the downloadable script, plus its help/version.
+- Eleven fixture scenarios, each in fresh source-CLI and single-file processes:
+  clean data, contacted domain, harmless research reference, exact hash hit,
+  corrupt database, a hit alongside a corrupt database, `--quick` coverage gap,
+  a DNS hit past the old 20,000-event limit, unread older DNS events without a
+  hit, invalid indicator input, and COM-referenced optional file patterns.
+- Real browser-shaped SQLite files and harmless rule-matching bytes pass through
+  the real scanner modules, evidence grading, exit codes and text/JSON reports.
+  Source and download must agree. Cookie values must stay out of reports;
+  full finding paths must remain visible. Fixtures must remain byte-identical
+  and temporary snapshots must be removed.
+- During fixture scans, an audit hook rejects socket activity, subprocesses and
+  file opens/listings outside fixture, project-code and Python-runtime roots.
+  The scanner's network-guard recorder is checked too, so caught network
+  attempts cannot pass silently. Each unit-suite run has its own temp directory.
+  Each fixture process has a 45-second timeout; timeouts or assertion failures
+  make the command fail. Results/logs stay under ignored
+  `findings/release-checks/summary.json`. `PASS_WITH_SKIPS` names missing coverage.
+
+The host's registry and PowerShell collectors are replaced by fixed inputs;
+the actual OS enumeration is **not** tested by these fixtures. This is an
+end-to-end check of the application flow, not an OS sandbox or a measurement of
+spyware detection rates. CI retains its real `--quick` smoke scans on disposable
+Windows, macOS and Linux runners. It also runs the fixtures on all three with
+YARA-X, and on Python 3.8 without it. Routine local runs normally take seconds;
+timings are recorded rather than used as a hardware benchmark.
+
+## Include the website and staged download
+
+The Scan website is maintained separately from this public repository. When
+its local source is available, include it explicitly:
+
+```text
+python tools/release_check.py --require-patterns --web-dir PATH_TO_SCAN_DIRECTORY --scanner PATH_TO_STAGED_ZERO_BREACH_PY
+```
+
+`--web-dir` requires Node, `index.html`, `scan.js`, and
+`ops/test_scan_logic.js`. It runs the existing offline page-flow matrix and
+  requires the verification card's code text and copy value to match the candidate script. These
+are DOM/logic fixtures; actual rendering and touch behaviour still need the
+browser smoke check. `--scanner` requires the staged download to match the
+fresh build byte for byte. Omit it to test the freshly built download.
+If `web/scan/` exists in this checkout, it is checked automatically; otherwise
+the website is explicitly listed as not tested. A requested website check must
+pass; it is never silently skipped. Use `--output-dir PATH` to keep separate runs.
+
+## Short release checklist
+
+1. Run the command above on the frozen candidate, including optional patterns,
+   staged scanner and prepared website when those are being published.
+2. Require all CI jobs for that exact commit. Run the existing live-browser
+   smoke check for website changes, including a small mobile viewport. For OS
+   collector changes, also use a disposable Windows VM with controlled data;
+   do not substitute a scan of your everyday laptop.
+3. Review remaining findings; get the requested independent Astra final review.
+   Verify release signatures and checksums following [SIGNING.md](SIGNING.md).
+4. Publish the tested bytes with a saved rollback. Verify the public download,
+   page checksum/links and a basic user flow after deployment.
+
+No soak test, load-test infrastructure or live malware is required for this
+routine gate. A timed-out real scan is an unresolved performance observation,
+even when these bounded fixtures pass.

scan/tests/test_release_check.py

@@ -0,0 +1,60 @@
+"""Regressions for false-green failures in the offline release gate itself."""
+import importlib.util
+from pathlib import Path
+import subprocess
+import sys
+import tempfile
+import unittest
+
+ROOT = Path(__file__).resolve().parents[2]
+spec = importlib.util.spec_from_file_location("release_gate", ROOT / "tools/release_check.py")
+gate = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(gate)
+
+
+class TestWebDownloadChecksum(unittest.TestCase):
+    DIGEST = "a" * 64
+
+    def panel(self, displayed=None, copied=None):
+        return ('<div id="verify-download"><div class="ea-copy"><code>{}</code>'
+                '<button data-copy="{}">Copy SHA-256</button></div></div>').format(
+                    displayed or self.DIGEST, copied or self.DIGEST)
+
+    def test_actual_code_and_copy_value_match(self):
+        gate.check_web_checksum(self.panel(), self.DIGEST)
+
+    def test_correct_hash_in_comment_does_not_mask_wrong_display(self):
+        html = self.panel(displayed="b" * 64) + "<!-- " + self.DIGEST + " -->"
+        with self.assertRaisesRegex(RuntimeError, "displayed/copied"):
+            gate.check_web_checksum(html, self.DIGEST)
+
+    def test_correct_display_does_not_mask_wrong_copy_value(self):
+        with self.assertRaisesRegex(RuntimeError, "displayed/copied"):
+            gate.check_web_checksum(self.panel(copied="b" * 64), self.DIGEST)
+
+
+class TestCaughtNetworkAttempt(unittest.TestCase):
+    def test_scanner_cannot_hide_an_attempt_by_catching_guard_exception(self):
+        with tempfile.TemporaryDirectory(prefix="zb_gate_regression_") as tmp:
+            root = Path(tmp)
+            candidate = root / "candidate.py"
+            subprocess.run([sys.executable, "-B", str(ROOT / "build/bundle.py"),
+                            str(ROOT), str(candidate)], check=True, capture_output=True)
+            source = candidate.read_text(encoding="utf-8")
+            needle = "def _pattern_engine():"
+            self.assertIn(needle, source)
+            # no_network blocks this before any real resolution; the pattern
+            # module catches it and would normally just report UNKNOWN.
+            source = source.replace(needle, needle + '\n    __import__("socket").'
+                                    'gethostbyname("release-fixture.invalid")')
+            candidate.write_text(source, encoding="utf-8")
+            proc = subprocess.run(
+                [sys.executable, "-B", str(ROOT / "tools/release_fixture.py"),
+                 "bundle", "patterns", str(root / "fixture"), str(candidate)],
+                stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=45)
+            self.assertNotEqual(proc.returncode, 0)
+            self.assertIn(b"blocked external activity occurred", proc.stderr)
+
+
+if __name__ == "__main__":
+    unittest.main()