diff --git a/.cursor/skills/principal-frontend-torrserver-architect/SKILL.md b/.cursor/skills/principal-frontend-torrserver-architect/SKILL.md new file mode 100644 index 000000000..ed19b7eab --- /dev/null +++ b/.cursor/skills/principal-frontend-torrserver-architect/SKILL.md @@ -0,0 +1,134 @@ +--- +name: principal-frontend-torrserver-architect +description: Principal Frontend, Web UI & Smart TV (PWA/MSX) Architect for TorrServer (`web/`, `gen_web.go`, `server/web/pages`, `server/web/msx`). Use for all frontend development, React/JS state debugging, API polling optimization, XSS/URL encoding fixes, and embedded asset generation. +--- + +# Role & Persona + +You are a **Principal Frontend & Web Systems Architect** specializing in lightweight, high-performance single-page applications (SPA), media streaming dashboards, Smart TV / Apple TV / Media Station X (MSX) interfaces, and Go-embedded static asset pipelines (`gen_web.go`). + +## Communication & Output Rules + +- **No Fluff or Basic Theory:** Skip introductory explanations. Provide immediate root-cause analysis and production-ready code. +- **Zero-Bloat Philosophy:** TorrServer's web UI runs on everything from desktop browsers to memory-constrained Smart TV WebViews and mobile devices. Avoid heavy dependencies, unnecessary re-renders, and memory leaks. + +--- + +# Repository Web Architecture & Asset Pipeline + +1. **Frontend Source (`web/`):** + - Contains the SPA source code, components, hooks/services for interacting with TorrServer's HTTP API (`/torrents`, `/stream`, `/play`, `/cache`, `/settings`, `/viewed`, `/playlistall/all.m3u`, `/echo`), i18n dictionaries, and static assets. +2. **Go Asset Embedding (`gen_web.go` -> `server/web/pages/template/`):** + - Production builds embed the compiled frontend assets into Go source files via `gen_web.go`. + - **CRITICAL INVARIANT:** Any change to `web/` build output MUST be synchronized with `server/web/pages/template` (via `go run gen_web.go` or the `Makefile` web target) so the Go binary serves the updated UI. +3. **Backend Web Integration (`server/web/`):** + - `server/web/pages`: Serves embedded HTML/JS/CSS/PWA manifests and handles cache-control headers. + - `server/web/msx`: Serves Media Station X (MSX) JSON/start configurations for Smart TVs. + - `server/web/api`: REST endpoints consumed by the frontend. + - `server/web/auth` & `server/web/waf`: Basic Auth and WAF protections that the frontend must handle gracefully (401/403 responses). + +--- + +# Engineering Standards & Frontend Patterns + +## 1. Network Polling, Concurrency & Memory Discipline + +- **Single-Flight Polling with `AbortController`:** + - Status and cache polling (`/torrents`, `/cache`) must NEVER use blind `setInterval` that stacks concurrent requests when the network or VPS is slow. + - Always use recursive `setTimeout` after request completion OR `AbortController` to cancel in-flight requests on component unmount, modal close, or tab hidden (`document.visibilityState === 'hidden'`). +- **Unmount & Listener Cleanup:** + - Every `useEffect` / event listener (`resize`, `keydown`, `visibilitychange`, `popstate`) and timer MUST have a deterministic cleanup function to prevent detached DOM trees and memory leaks during long sessions. + +## 2. Streaming URLs, Encoding & Security (XSS / Injection) + +- **Strict Stream & Playlist URL Construction:** + - Torrent file names frequently contain spaces, brackets `[1080p]`, Cyrillic/Unicode characters, `#`, `?`, and `&`. + - Always encode path segments with `encodeURIComponent(fileName)` and construct query parameters (`link`, `index`, `play`, `save`, `m3u`, `fromlast`) using `URLSearchParams` to prevent broken links in VLC, Infuse, IINA, and PotPlayer. +- **XSS & DOM Sanitization:** + - Torrent titles, tracker messages, and file paths come from untrusted P2P metainfo. Never inject raw torrent names or magnet metadata via `innerHTML` or unescaped `dangerouslySetInnerHTML`. + +## 3. State Management, Settings & Error Handling + +- **Settings Synchronization (`/settings`):** + - Validate numeric inputs (cache size, preload percentage, reader read-ahead, connection limits, ports) on the client before `POST /settings` to prevent sending `NaN`, negative numbers, or string-typed numbers that break Go JSON unmarshaling. + - Provide clear visual feedback (toast/status indicator) on save success, network timeout, or `readonly` mode (`403`/`400` from server). +- **Optimistic UI vs. Ground Truth:** + - When adding, dropping, or wiping torrents, disable action buttons during the in-flight request to prevent duplicate submissions, and reconcile local state with the server response. + +## 4. Smart TV, Mobile & Accessibility UX + +- **Keyboard & D-Pad Navigation:** Ensure modals, file lists, and action buttons support `Escape`, `Enter`, `Tab`, and arrow-key focus for TV browsers and desktop keyboard users. +- **Large Torrent File Trees:** Torrents with 100–1,000+ files (e.g., full TV series packs or audio discographies) must render smoothly without freezing the main thread (use memoization, pagination, or virtualized/lazy rendering). + +--- + +# This repository + +Edit `web/src`. The Go binary serves `server/web/pages/template/pages`, produced by `make webgen` (`go run gen_web.go` from the repo root) or `make webgen-clean`. Do not hand-edit that tree. `web/` is Yarn. Node `>=22`. + +Checks, from `web/`: + +```bash +yarn typecheck +yarn lint +yarn test +yarn i18n:check +yarn test:e2e +``` + +`yarn build` refuses `VITE_SERVER_HOST` that contains `localhost` or `127.0.0.1` when `mode` is production. After a `web/src` change that ships in the binary, run `make webgen` and `cd server && go test -race ./web/...`. + +A new string goes in `web/src/locales/{en,ru,ua,bg,fr,ro,zh}/translation.json` under the same flattened key. `yarn i18n:check` fails if one locale is missing it. + +## Polling + +| Hook | Endpoint | Cadence | +| --- | --- | --- | +| `useTorrentsQuery` | `POST /torrents` `action: list` | 500ms while any row is `GETTING_INFO`, else 1s | +| `useTorrentDetail` | `POST /torrents` `action: get` | 500ms `GETTING_INFO`, 1s `PRELOAD`, else 2s | +| `useUpdateCache` | `POST /cache` `action: get` | 100ms while the snake changes, else 1s. Timer is `setTimeout` after the response. AbortController on unmount. Stops while `document.hidden`. Details dialog polls on the Cache tab, the cache-map modal, and the Stats tab (idle, for the progress strip). The Files tab does not poll. | +| `ServerStatusDialog` | `GET /runtime/status` | 2.5s while the dialog is open | + +`web/src/index.tsx` sets `refetchOnWindowFocus: false`. TanStack Query's `refetchInterval` is a `setInterval`, but `query.fetch` joins an in-flight request unless `cancelRefetch` is set, and `refetchIntervalInBackground` defaults to false so a hidden tab does not send those requests. Pass `signal` into axios (`getTorrents`, `getTorrent`). Do not point `queryFn` at a function whose first argument is an `AbortSignal` — React Query passes a context object. + +`PwaUpdateToast` rechecks `sw.js` hourly. `VideoPlayer` heartbeats GStreamer every 30s and clears the interval on close. + +## Stream and playlist URLs + +Build them in `web/src/shared/lib/streamLinks.ts` (`encodePathSegment`, `streamPlayUrl`, `streamPreloadUrl`, `torrentPlaylistUrl`, `buildExternalPlayerHref`, `buildAndroidIntentHref`). `posterPlay.ts` re-exports the stream URLs. `usePlayLauncher` and `FilesDataGrid` call those. Do not interpolate `link=` by hand. + +- Filename is one path segment: `encodePathSegment` (`encodeURIComponent`, then `%` + uppercase hex for `!'()*`). +- `link` and `index` go through `URLSearchParams` so `#`, `?`, `&`, `%` cannot split the query. +- Play flag stays valueless: `&play`. Preload is `GET /stream?…&preload` (the route without `*fname` exists). Playlist is `&m3u&fn=file.m3u`. The `fn` flag is the ForkPlayer workaround also used by `server/web/api/m3u.go` `nestedPlaylistEntry`. +- `externalLink` (`web/src/shared/lib/mediaBase.ts`) applies `GET /mediabase` (scheme, host, port) and then `withAuthMediaUrl`. +- `withAuthMediaUrl` adds Basic userinfo only when `hostname` matches the page. HTTPS UI + HTTP media on another port of the same host must still get the userinfo. A different host must not. +- Desktop VLC href is `vlc://${url}` unencoded. iOS VLC is `vlc-x-callback://x-callback-url/stream?url=` with the absolute URL encoded. Android VLC is `intent://${url-without-scheme}#Intent;scheme=…;type=video/*;end` (`buildAndroidIntentHref`). Infuse, SenPlayer, and IINA percent-encode the whole URL. Infuse episode lists fall back to one m3u URL past `INFUSE_PLAYLIST_HREF_LIMIT` (8000). Clipboard copy is `copyToClipboard` (`navigator.clipboard`, then a textarea inside the open dialog). + +`CacheSize` in `BTSets` is **bytes** on the wire and **megabytes** inside `SettingsDialog` local state. `setSettings` runs `sanitizeBTSetsForWire`: integers are rounded, NaN uses the server default, `ReaderReadAHead` is clamped 5–100, `PreloadCache` 0–100. A JSON float into a Go `int` fails the entire `POST /settings` body. Read-only `set` and `def` return `403` and do not restart DLNA, Bonjour, or rutor. Nil `sets` returns `400` (`SetBTSets(nil)` would panic). Save stays disabled until the settings GET succeeds, so a failed load cannot overwrite the server with `web/src/shared/settings/defaults.ts`. + +`StoreViewedInJson` is not edited in the form. Storage backend is `StorageSettingsPanel` → `/storage/settings`. Keep the field on the object you POST back. + +GStreamer integer fields are rounded in `setGstSettings`. `GSTVersion` is a float. + +## Rendering and XSS + +Torrent titles and paths are rendered as React text. There is no `dangerouslySetInnerHTML` in `web/src`. Keep it that way. + +File rows and the play-file picker use `content-visibility: auto`. The piece map is one `` (`TorrentCache` / `drawSnake`), sized to the pane (not one cell per piece in a 15k-piece remux). `setupHiDpiCanvas` caps `devicePixelRatio` at 2 and does not reallocate the bitmap when the CSS size is unchanged. `buildFocusModel` intersects reader ranges with that window. Paint runs in `requestAnimationFrame` and skips when `cheapCellsFingerprint` is unchanged. `useUpdateCache` keeps at most 8 snake snapshots (`snakeSession.ts`). + +## Service worker and cache + +`web/vite.config.ts` `navigateFallbackDenylist` lists API prefixes so Workbox does not serve `index.html` for `/stream`, `/torrents`, `/cache`, `/settings`, `/echo`, and the rest. Add a new top-level API prefix there in the same change. Embedded headers live in `server/web/pages/template/route.go`: `no-cache` for `index.html`, `sw.js`, `workbox-*`; `immutable` for hashed `/static/`. + +## MSX + +`server/web/msx/msx.go` `/msx/proxy` fetches the `url` query with `http.DefaultClient` and forwards `header` query pairs. Tests expect it to attempt `127.0.0.1`. It is an open proxy on the same trust boundary as the process. Do not remove it as a drive-by. + +Playlist `Content-Disposition` goes through `safeAttachmentName` in `server/web/api/m3u.go` (strip quotes and control characters). + +# Verification Playbook + +Before completing any frontend task: + +1. **Lint & Type/Build Check:** From `web/`, `yarn typecheck`, `yarn lint`, and `yarn test` with 0 errors. `yarn i18n:check` when strings change. `yarn test:e2e` when the shell, settings, or auth gate changes. +2. **Asset Sync Check:** `make webgen` (or `make webgen-clean`), then `cd server && go test -race ./web/...`. diff --git a/.cursor/skills/principal-go-torrserver-architect/SKILL.md b/.cursor/skills/principal-go-torrserver-architect/SKILL.md new file mode 100644 index 000000000..4b0f42357 --- /dev/null +++ b/.cursor/skills/principal-go-torrserver-architect/SKILL.md @@ -0,0 +1,87 @@ +--- +name: principal-go-torrserver-architect +description: >- + Principal Go systems and BitTorrent streaming architect for the TorrServer + `server/` module (anacrolix/torrent fork, torrstor piece cache, HTTP Range + /stream and /play). Use for all Go work in this repo: concurrency and race + debugging, cache and streaming memory, settings and HTTP handlers, and tests. + Read AGENTS.md before editing. There is no internal/ engine. +--- + +# TorrServer Go architect + +Read [AGENTS.md](../../../AGENTS.md) before editing `server/`. That file is the map of locks, forks, traps, and commands. This skill is the working rule. Do not restate Go basics. Ship thread-safe, bounded, cancellable changes with no compatibility regressions. + +Module path is `server`. Run Go from `server/`. `go 1.25.7` in `server/go.mod`. No root `go.mod`, no `internal/` tree. + +Hot path: `GET/HEAD /stream` and `/play` → `torr.Torrent.Stream` → `torrstor.Reader` → piece cache → `anacrolix/torrent`. DLNA, Bonjour, FUSE, WebDAV, Telegram, MCP, and GStreamer stay off that path. + +## Forks + +Import upstream paths. `server/go.mod` replaces them: + +- `github.com/anacrolix/torrent` → `github.com/lampac-nextgen/torrent` +- `github.com/anacrolix/utp` → `github.com/lampac-nextgen/utp` +- `github.com/anacrolix/upnp` → `github.com/tsynik/upnp` + +`NE` comments are APIs this fork does not have (directory file storage, `DisableWebtorrent`, `HeaderObfuscationPolicy`, storage `Capacity`). Do not call them. `OE` comments are the live behavior. + +Peer id prefix stays `-qB4390-` / `qBittorrent/4.3.9`. `UpnpID` stays `TorrServer/`. `TotalHalfOpenConns` is 500. + +## Do not touch unless the task names it + +- `server/docs` (generated Swagger) and `server/web/pages/template/pages` (generated UI). Edit `web/src`, then rebuild. +- `server/gstreamer` CGo. Files are `//go:build gst`. Default tests compile `bridge/bridge_stub.go`. Do not pass `-tags gst` and do not grep `gstreamer` out of `make test`. +- New JS runtimes, plugin engines, or an FFmpeg pipeline. Remux already exists behind `gst`. +- `rutor` is in the default test set. Do not exclude it. + +## Locks + +`sync.RWMutex` is not reentrant. `RLock` then `Lock` on `BTServer.mu` deadlocks. `server/torr/clientrace_test.go` fails if `NewTorrent` holds the read lock across the write lock. + +Order when both are needed: `settings.dbMigrationLock` then `settings.mu`; `BTServer.mu`; `Torrent.muTorrent`; `torrstor.Storage.mu` released before `Cache.Close`; cache `muRemove` → `muReaders` → `muActive`. `Reader.resume` drops `Reader.mu` around the anacrolix `Seek`. `applyZones` drops `muReaders` before `SetZones`. + +Snapshot `t.Torrent`, `t.cache`, and `bt.client` under the mutex that owns them. `drop` nils the first two at any time. `Cache.pieces` is read-only after `Init`. `MemPiece.ReadAt` uses `Lock` because it writes `Accessed`. + +Do not hold those mutexes across tracker HTTP, DHT, `publicip.Get4/Get6`, or disk flush. Full order and the `NewJsonDB` `:=` shadow are in AGENTS.md. + +`FuseAutoMount` still calls `os.Exit(1)`. New library code returns an error or uses the existing `exitFunc` (`cmd`, `dlna`). `settings.Embedded` must not exit the host process. + +## Streaming and trackers + +- Copy buffer is `streamCopyPool`: `*[64KiB]byte`, not a bare `[]byte`. Read-ahead is `newBufferedStreamReader(..., 1<<20)`. Nil the pooled `ResponseWriter` before `Put`. `buf[:nr]` must not escape. +- Piece slabs are `*pieceSlab` pooled by exact length. `putPieceBuf` clears the full cap. `Release` does not recycle when `cache.isClosed`. Never keep a sub-slice after `Release`. +- Do not evict a piece inside a merged reader range, the begin/end keep window (at least 8MiB), or a full piece whose `Complete` flag is still false. +- Download windows are zone `(capacity/readers)*ReaderReadAHead/100`, readahead `min(16MiB/readers, zone)`, next `min(4MiB/readers, readahead)`. Push them on reader create, close, and the 1s `updateRA` tick. Not on every small `Read`. Do not call `Piece.SetPriority` from the cache: it raises over those tiers. `Piece.Release` only calls `UpdateCompletion`. +- Idle park is `minReaderIdleTimeout` (60s), or `TorrentDisconnectTimeout` if larger. `readerOff` must not `Seek` while `inFlight > 0`. +- `Reader.Close` does not unregister. `Cache.CloseReader` does, and it is safe on a nil `*Cache`. +- A handler that accepted the request must write a status. An empty return becomes `200` with an empty body. +- Tracker client is `trackersHTTPClient`: 5s timeout, 3s dial, 3s TLS, body cap 256KiB, its own `Transport`. Do not use `http.DefaultClient`. A failed refresh keeps the previous list and must not block startup. +- `SanitizeTrackers` does not cap the list. `addExtraTrackers` appends one tier. One tier per URL is an announce storm on this fork. +- DHT bootstrap uses `lookupDHTHost` and a 3s timeout. Do not switch it to `dht.GlobalBootstrapAddrs`. + +## Tests + +Use `server/torr/offlinetest`. The fork has no directory file storage. + +```go +offlinetest.Prepare(t) // temp DB, DHT/PEX/uTP/IPv6 off, restores globals +show := offlinetest.WriteShow(t) +seed := offlinetest.Seed(t, show) // 127.0.0.1:0 memStore, already complete +``` + +`ffprobe`: shell script in `t.TempDir()`, assign `binFile`, `goffprobe.SetFFProbeBinPath`, restore in `t.Cleanup`. Windows skips. Telegram: `httptest` plus `tele.Settings{URL: srv.URL, Offline: true, Client: srv.Client()}`. Leaks: `leakcheck.AssertNoGoroutineLeak` and `leakcheck.Heap`. Replace `lookupDHTHost`, `dhtBootstrapHosts`, and `defaultTrackersListURLs` in tests. `LIVE_TORRENT=1` and `LIVE_INDEXER=1` stay opt-in. + +## Done when + +From the repo root: + +```bash +make fmt-check +make lint +make test +``` + +`make test` is `-race -count=1 -timeout=10m -covermode=atomic` on every package except `server/docs`. Coverage floor is **70% total** (`.testcoverage.yml`). File and package floors are 0. + +`make bench` when the diff touches `server/torr`, `server/torr/storage/torrstor`, or `server/web/api`. It benches only `./subs ./torr/storage/torrstor ./torrshash ./torrfs ./web/api`. The `torr` reader bench blocks on `GotInfo` and is excluded on purpose. A hot-path change needs a `-benchmem` check that the copy buffer or piece slab is still pooled. diff --git a/.env.example b/.env.example new file mode 100644 index 000000000..1f0dee803 --- /dev/null +++ b/.env.example @@ -0,0 +1,40 @@ +# TorrServer Docker Compose — copy to `.env` and adjust +# cp .env.example .env + +# Container image (published releases) +TORRSERVER_IMAGE=ghcr.io/yourok/torrserver:latest +TORRSERVER_CONTAINER_NAME=torrserver + +# Host directory for persistent data (config, torrent cache, logs) +TORRSERVER_DATA_DIR=./data + +# Web UI / API port (must match ports mapping in docker-compose.yml) +TS_PORT=8090 + +# Paths inside the container (defaults match Dockerfile) +TS_CONF_PATH=/opt/ts/config +TS_LOG_PATH=/opt/ts/log/torrserver.log +TS_TORR_DIR=/opt/ts/torrents + +# Optional bind address inside the container +# TS_IP=0.0.0.0 + +# Feature flags — set to 1 to enable +TS_HTTPAUTH=0 +TS_RDB=0 +TS_DONTKILL=0 + +# TLS +# TS_EN_SSL=1 +# TS_SSL_PORT=8443 + +# BitTorrent proxy +# TS_PROXYURL=socks5h://user:password@example.com:2080 +# TS_PROXYMODE=tracker + +# Go runtime (Dockerfile default) +GODEBUG=madvdontneed=1 + +# Local build override (docker compose -f docker-compose.yml -f docker-compose.build.yml up -d --build) +# TORRSERVER_BUILD_CONTEXT=./dist/docker-context +# TORRSERVER_IMAGE=torrserver:local diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 000000000..1f3c5ef12 --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,26 @@ +--- +version: 2 +updates: + - package-ecosystem: "npm" + directory: "/web" + schedule: + interval: "weekly" + # Stay on TypeScript 6 until the toolchain (typescript-eslint, vue-tsc, …) supports 7. + ignore: + - dependency-name: "typescript" + update-types: ["version-update:semver-major"] + groups: + web-minor-patch: + update-types: + - "minor" + - "patch" + commit-message: + prefix: "deps(web)" + + - package-ecosystem: "github-actions" + directory: "/" + schedule: + interval: "weekly" + commit-message: + prefix: "ci" + include: "scope" diff --git a/.github/versions.env b/.github/versions.env new file mode 100644 index 000000000..17edecd59 --- /dev/null +++ b/.github/versions.env @@ -0,0 +1,4 @@ +# Go toolchain versions — single source for CI, GoReleaser, and local release builds. +# server/go.mod "go" directive should stay aligned with GO_ANDROID_VERSION (module minimum). +GO_VERSION=1.27.3 +GO_ANDROID_VERSION=1.25.7 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 000000000..e9f79da7a --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,305 @@ +name: CI + +on: + push: + branches: + - master + pull_request: + branches: + - master + +concurrency: + group: ci-${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +permissions: + contents: read + +jobs: + lint: + name: Lint (Go) + runs-on: ubuntu-26.04 + defaults: + run: + working-directory: server + steps: + - name: Checkout + uses: actions/checkout@v7 + + - name: Load toolchain versions + working-directory: ${{ github.workspace }} + run: grep -v '^#' .github/versions.env >> "$GITHUB_ENV" + + - name: Set up Go + uses: actions/setup-go@v7 + with: + go-version: ${{ env.GO_VERSION }} + cache-dependency-path: server/go.sum + + - name: Install golangci-lint + uses: golangci/golangci-lint-action@v9 + with: + version: latest + working-directory: server + install-only: true + verify: true + + - name: go fmt + run: | + unformatted=$(gofmt -l .) + if [ -n "$unformatted" ]; then + echo "The following files need gofmt:" >&2 + echo "$unformatted" >&2 + exit 1 + fi + echo "OK: gofmt clean" + + - name: golangci-lint fmt + run: | + diff=$(golangci-lint fmt -d ./...) + if [ -n "$diff" ]; then + echo "The following formatting changes are required (golangci-lint fmt):" >&2 + echo "$diff" >&2 + exit 1 + fi + echo "OK: golangci-lint fmt clean" + + - name: golangci-lint run + run: golangci-lint run ./... + + test: + name: Race Tests & Coverage + runs-on: ubuntu-26.04 + timeout-minutes: 20 + permissions: + contents: read + actions: write + pull-requests: write + steps: + - name: Checkout + uses: actions/checkout@v7 + + - name: Load toolchain versions + run: grep -v '^#' .github/versions.env >> "$GITHUB_ENV" + + - name: Set up Go + uses: actions/setup-go@v7 + with: + go-version: ${{ env.GO_VERSION }} + cache-dependency-path: server/go.sum + + - name: Run unit tests with race detector + run: make test + + - name: Enforce coverage gate + id: coverage + uses: vladopajic/go-test-coverage@v2 + with: + config: .testcoverage.yml + + - name: Coverage summary + if: always() && steps.coverage.outputs.total-coverage != '' + run: | + { + echo "### Coverage" + echo + echo "${{ steps.coverage.outputs.total-coverage }}% · floor 70%" + } >> "$GITHUB_STEP_SUMMARY" + + - name: Restore benchmark history + uses: actions/cache/restore@v6 + with: + path: .cache/benchmark + key: go-bench-${{ github.run_id }}-${{ github.run_attempt }} + restore-keys: | + go-bench- + + - name: Run memory and throughput benchmarks + id: bench + run: | + mkdir -p .cache/benchmark + # Cache miss on the first run. The benchmark action warns if this file is absent. + if [ ! -s .cache/benchmark/data.json ]; then + printf '%s\n' '{"lastUpdate":0,"repoUrl":"","entries":{}}' > .cache/benchmark/data.json + fi + make bench + # The action scores every Go unit as smaller-is-better. MB/s is throughput, + # so a faster run looks like a regression and trips the 200% alert. + # ns/op, B/op, and allocs/op stay. + sed -Ei 's/[[:space:]]+[0-9.]+[[:space:]]+MB\/s//g' server/bench.txt + # The benchmark action prints a comparison only when this file already has entries. + if grep -Eq '"entries"[[:space:]]*:[[:space:]]*\{\}' .cache/benchmark/data.json; then + echo "fresh=true" >> "$GITHUB_OUTPUT" + else + echo "fresh=false" >> "$GITHUB_OUTPUT" + fi + + - name: Compare benchmarks + uses: benchmark-action/github-action-benchmark@v1 + with: + name: Go benchmarks + tool: go + output-file-path: server/bench.txt + external-data-json-path: .cache/benchmark/data.json + github-token: ${{ github.token }} + skip-fetch-gh-pages: true + alert-threshold: "200%" + # 200ms benches on a shared runner cross 200% with no code change. + # The comparison still lands in the job summary. A PR comment uses + # pull-requests:write. A push to master has no PR, and the commit + # comment API needs contents:write — that 403 fails the job. + fail-on-alert: false + comment-always: ${{ github.event_name == 'pull_request' }} + comment-on-alert: ${{ github.event_name == 'pull_request' }} + summary-always: true + + - name: Benchmark summary + if: steps.bench.outputs.fresh == 'true' + run: | + { + echo "### Benchmarks" + echo + echo '```' + grep '^Benchmark' server/bench.txt + echo '```' + } >> "$GITHUB_STEP_SUMMARY" + + - name: Save benchmark history + if: success() + uses: actions/cache/save@v6 + with: + path: .cache/benchmark + key: go-bench-${{ github.run_id }}-${{ github.run_attempt }} + + - name: Upload coverage and benchmark reports + if: always() + uses: actions/upload-artifact@v7 + with: + name: test-and-bench-reports + path: | + server/coverage.out + server/bench.txt + retention-days: 14 + if-no-files-found: ignore + + goreleaser-check: + name: GoReleaser config + runs-on: ubuntu-26.04 + steps: + - name: Checkout + uses: actions/checkout@v7 + + - name: Validate .goreleaser.yaml + uses: goreleaser/goreleaser-action@v7 + with: + distribution: goreleaser + version: "~> v2" + args: check --config .goreleaser.yaml + + - name: Validate .goreleaser.local.yaml + uses: goreleaser/goreleaser-action@v7 + with: + distribution: goreleaser + version: "~> v2" + args: check --config .goreleaser.local.yaml + + web: + name: Web (typecheck lint test build) + runs-on: ubuntu-26.04 + defaults: + run: + working-directory: web + steps: + - name: Checkout + uses: actions/checkout@v7 + + - name: Set up Node.js + uses: actions/setup-node@v7 + with: + node-version: "22" + cache: yarn + cache-dependency-path: web/yarn.lock + + - name: Install dependencies + run: yarn install --frozen-lockfile + + - name: Typecheck + run: yarn typecheck + + - name: Lint + run: yarn lint + + - name: Test + run: yarn test + + - name: i18n + run: yarn i18n:check + + - name: Install Playwright Chromium + run: yarn playwright install chromium --with-deps + + - name: E2E (mocked) + run: yarn test:e2e + + - name: Build + run: yarn build + env: + # Ensure release-like embed build (no localhost API baked in) + VITE_SERVER_HOST: "" + + build: + name: Build (linux amd64) + runs-on: ubuntu-22.04 + needs: [lint, test, goreleaser-check, web] + env: + VITE_TMDB_API_KEY: ${{ secrets.TMDB_API_KEY }} + steps: + - name: Checkout + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Load toolchain versions + run: grep -v '^#' .github/versions.env >> "$GITHUB_ENV" + + - name: Set up Node.js + uses: actions/setup-node@v7 + with: + node-version: "22" + cache: yarn + cache-dependency-path: web/yarn.lock + + - name: Set up Go + uses: actions/setup-go@v7 + with: + go-version: ${{ env.GO_VERSION }} + cache-dependency-path: server/go.sum + + - name: Install Go toolchain wrapper (GoReleaser tool) + run: | + go install golang.org/dl/go${{ env.GO_VERSION }}@latest + go${{ env.GO_VERSION }} download + + - name: Snapshot build (single target, no docker) + uses: goreleaser/goreleaser-action@v7 + with: + distribution: goreleaser + version: "~> v2" + args: build --snapshot --clean --single-target --id binary + env: + GO_BINARY: go${{ env.GO_VERSION }} + UPX_ENABLED: false + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + - name: Verify binary artifact + run: test -x dist/TorrServer-linux-amd64 + + - name: Verify static Linux binary (no shared libs) + run: | + bin=dist/TorrServer-linux-amd64 + if readelf -d "$bin" 2>/dev/null | grep -q NEEDED; then + echo "Expected a statically linked binary without DT_NEEDED entries:" >&2 + readelf -d "$bin" | grep NEEDED >&2 || true + exit 1 + fi + echo "OK: no dynamic library dependencies" diff --git a/.github/workflows/docker_image.yml b/.github/workflows/docker_image.yml deleted file mode 100644 index 5b36ba97c..000000000 --- a/.github/workflows/docker_image.yml +++ /dev/null @@ -1,88 +0,0 @@ -name: Release Docker multi arch - -on: - push: - branches: - - master - tags: - - "**" - -concurrency: - group: docker-build-${{ github.ref }} - cancel-in-progress: false - -jobs: - docker: - runs-on: ubuntu-latest - permissions: - contents: read - packages: write - - steps: - - name: Checkout - uses: actions/checkout@v6 - with: - fetch-depth: 0 - - - name: Set up QEMU - uses: docker/setup-qemu-action@v3 - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@v3 - - - name: Login to GitHub Container Registry - uses: docker/login-action@v3 - with: - registry: ghcr.io - username: ${{ github.actor }} - password: ${{ secrets.GITHUB_TOKEN }} - - - name: Set lower case repository name - run: | - echo "REG_REPO=${REPO,,}" >>${GITHUB_ENV} - env: - REPO: "${{ github.repository }}" - - - name: Set version - run: | - SERVER_VERSION=$(sed -n 's/.*Version = "\([^"]*\)".*/\1/p' server/version/version.go) - SERVER_VER="${SERVER_VERSION#MatriX.}" - if [[ "${{ github.ref }}" == refs/tags/* ]]; then - TAG="${GITHUB_REF#refs/tags/}" - if [[ "$TAG" == MatriX.* ]]; then - VERSION="$TAG" - else - VERSION="MatriX.${TAG}" - fi - elif [[ "${{ github.ref }}" == refs/heads/* ]]; then - VERSION="MatriX.${SERVER_VER}.${GITHUB_SHA:0:7}" - else - VERSION="${SERVER_VERSION}" - fi - echo "VERSION=${VERSION}" >> "$GITHUB_ENV" - echo "Version: ${VERSION}" - - - name: Extract metadata - id: meta - uses: docker/metadata-action@v5 - with: - images: | - ghcr.io/${{ env.REG_REPO }} - tags: | - type=raw,value=master,enable=${{ github.ref == format('refs/heads/{0}', 'master') }} - type=ref,event=tag - type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/') }} - - - name: Build and push Docker image - uses: docker/build-push-action@v6 - with: - context: . - platforms: linux/amd64,linux/arm64,linux/arm/v7 - push: true - tags: ${{ steps.meta.outputs.tags }} - labels: ${{ steps.meta.outputs.labels }} - build-args: | - REACT_APP_TMDB_API_KEY=${{ secrets.TMDB_API_KEY }} - VERSION=${{ env.VERSION }} - cache-from: type=gha,scope=${{ github.workflow }} - cache-to: type=gha,mode=max,scope=${{ github.workflow }} diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 000000000..22cc33acc --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,86 @@ +name: Release + +on: + push: + tags: + - "*" + +permissions: + contents: write + packages: write + +jobs: + goreleaser: + runs-on: ubuntu-22.04 + env: + VITE_TMDB_API_KEY: ${{ secrets.TMDB_API_KEY }} + steps: + - name: Checkout + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Load toolchain versions + run: grep -v '^#' .github/versions.env >> "$GITHUB_ENV" + + - name: Set up Node.js + uses: actions/setup-node@v7 + with: + node-version: '22' + cache: yarn + cache-dependency-path: web/yarn.lock + + - name: Set registry image name + run: echo "REGISTRY_IMAGE=${REPO,,}" >> "$GITHUB_ENV" + env: + REPO: "${{ github.repository }}" + + - name: Set NDK toolchain path + run: echo "NDK_TOOLCHAIN=${ANDROID_NDK_LATEST_HOME}/toolchains/llvm/prebuilt/linux-x86_64" >> "$GITHUB_ENV" + + - name: Set up Go (default + before hooks) + uses: actions/setup-go@v7 + with: + go-version: ${{ env.GO_VERSION }} + cache-dependency-path: server/go.sum + + - name: Install Go toolchain wrappers (GoReleaser) + run: | + go install golang.org/dl/go${{ env.GO_VERSION }}@latest + go${{ env.GO_VERSION }} download + go install golang.org/dl/go${{ env.GO_ANDROID_VERSION }}@latest + go${{ env.GO_ANDROID_VERSION }} download + + - name: Set up QEMU + uses: docker/setup-qemu-action@v4 + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v4 + + - name: Install AppImage dependencies + run: | + sudo apt-get update + sudo apt-get install -y wget fuse libfuse2 + + - name: Login to GitHub Container Registry + uses: docker/login-action@v4 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Run GoReleaser + uses: goreleaser/goreleaser-action@v7 + with: + distribution: goreleaser + version: "~> v2" + args: release --clean --skip=validate + env: + GO_BINARY: go${{ env.GO_VERSION }} + GO_ANDROID_BINARY: go${{ env.GO_ANDROID_VERSION }} + # GHCR repository names must be lowercase; `github.repository` may contain + # uppercase letters (e.g. `lampac-nextgen/TorrServer`). + DOCKER_IMAGE_ID: ghcr.io/${{ env.REGISTRY_IMAGE }} + UPX_ENABLED: false + APPIMAGE_EXTRACT_AND_RUN: "1" + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/test-install-script.yml b/.github/workflows/test-install-script.yml index 4ae94b252..4c7e3758c 100644 --- a/.github/workflows/test-install-script.yml +++ b/.github/workflows/test-install-script.yml @@ -27,10 +27,10 @@ env: jobs: test-script-syntax: name: Test Script Syntax - runs-on: ubuntu-latest + runs-on: ubuntu-26.04 steps: - name: Checkout repository - uses: actions/checkout@v5 + uses: actions/checkout@v7 - name: Test script syntax run: | @@ -46,7 +46,7 @@ jobs: test-install-script: name: Test on ${{ matrix.os }} (${{ matrix.test-user }}) - runs-on: ubuntu-latest + runs-on: ubuntu-26.04 strategy: fail-fast: false matrix: @@ -68,10 +68,10 @@ jobs: steps: - name: Checkout repository - uses: actions/checkout@v5 + uses: actions/checkout@v7 - name: Setup Docker Buildx - uses: docker/setup-buildx-action@v3 + uses: docker/setup-buildx-action@v4 - name: Get Docker image for OS id: get-image diff --git a/.github/workflows/ts_release.yml b/.github/workflows/ts_release.yml deleted file mode 100644 index 015046c9d..000000000 --- a/.github/workflows/ts_release.yml +++ /dev/null @@ -1,317 +0,0 @@ -name: Build and Release - -on: - push: - tags: - - "*" - -jobs: - build: - runs-on: ubuntu-22.04 - env: - NODE_OPTIONS: --openssl-legacy-provider - strategy: - matrix: - include: - - go_version: "stable" - group: "standard" - - go_version: "1.25" - group: "android" - - steps: - - name: Checkout code - uses: actions/checkout@v7 - - - name: Set up Go - uses: actions/setup-go@v7 - with: - go-version: ${{ matrix.go_version }} - cache: true - - - name: Set version from tag - run: echo "VERSION=${GITHUB_REF#refs/tags/}" >> $GITHUB_ENV - - - name: Install Swag - run: go install github.com/swaggo/swag/cmd/swag@latest - - - name: Build Web and Docs - if: matrix.group == 'standard' - env: - REACT_APP_TMDB_API_KEY: ${{ secrets.TMDB_API_KEY }} - run: | - go run gen_web.go - cd server && ~/go/bin/swag init -g web/server.go --parseInternal --parseDepth 5 - - - name: Build Standard Platforms - if: matrix.group == 'standard' - run: | - LDFLAGS="-s -w -checklinkname=0 -X server/version.Version=${{ env.VERSION }}" - mkdir -p "${{ github.workspace }}/dist" - cd server - export CGO_ENABLED=0 - - # Linux - GOOS=linux GOARCH=amd64 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-linux-amd64" ./cmd - GOOS=linux GOARCH=arm64 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-linux-arm64" ./cmd - GOOS=linux GOARCH=arm GOARM=7 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-linux-arm7" ./cmd - GOOS=linux GOARCH=arm GOARM=5 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-linux-arm5" ./cmd - GOOS=linux GOARCH=386 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-linux-386" ./cmd - - # Windows - GOOS=windows GOARCH=amd64 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-windows-amd64.exe" ./cmd - GOOS=windows GOARCH=386 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-windows-386.exe" ./cmd - - # Darwin - GOOS=darwin GOARCH=amd64 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-darwin-amd64" ./cmd - GOOS=darwin GOARCH=arm64 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-darwin-arm64" ./cmd - - # FreeBSD - GOOS=freebsd GOARCH=amd64 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-freebsd-amd64" ./cmd - GOOS=freebsd GOARCH=arm GOARM=7 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-freebsd-arm7" ./cmd - - # MIPS - GOOS=linux GOARCH=mips GOMIPS=softfloat go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-linux-mips" ./cmd - GOOS=linux GOARCH=mipsle GOMIPS=softfloat go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-linux-mipsle" ./cmd - GOOS=linux GOARCH=mips64 GOMIPS64=softfloat go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-linux-mips64" ./cmd - GOOS=linux GOARCH=mips64le GOMIPS64=softfloat go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-linux-mips64le" ./cmd - GOOS=linux GOARCH=riscv64 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-linux-riscv64" ./cmd - - # GST Linux - GOOS=linux GOARCH=amd64 go build -ldflags="$LDFLAGS" -tags="nosqlite gst" -trimpath -o "${{ github.workspace }}/dist/TorrServer-gst-linux-amd64" ./cmd - GOOS=linux GOARCH=arm64 go build -ldflags="$LDFLAGS" -tags="nosqlite gst" -trimpath -o "${{ github.workspace }}/dist/TorrServer-gst-linux-arm64" ./cmd - - # GST Darwin / macOS - GOOS=darwin GOARCH=amd64 go build -ldflags="$LDFLAGS" -tags="nosqlite gst" -trimpath -o "${{ github.workspace }}/dist/TorrServer-gst-darwin-amd64" ./cmd - GOOS=darwin GOARCH=arm64 go build -ldflags="$LDFLAGS" -tags="nosqlite gst" -trimpath -o "${{ github.workspace }}/dist/TorrServer-gst-darwin-arm64" ./cmd - - - name: Verify static Linux binary (no shared libs) - if: matrix.group == 'standard' - run: | - bin="${{ github.workspace }}/dist/TorrServer-linux-amd64" - if readelf -d "$bin" 2>/dev/null | grep -q NEEDED; then - echo "Expected a statically linked binary without DT_NEEDED entries:" >&2 - readelf -d "$bin" | grep NEEDED >&2 || true - exit 1 - fi - echo "OK: TorrServer-linux-amd64 has no dynamic library dependencies" - - - name: Build Android (Go 1.25) - if: matrix.group == 'android' - run: | - go run gen_web.go - - LDFLAGS="-s -w -checklinkname=0 -X server/version.Version=${{ env.VERSION }}" - mkdir -p "${{ github.workspace }}/dist" - export NDK_TOOLCHAIN="$ANDROID_NDK_LATEST_HOME/toolchains/llvm/prebuilt/linux-x86_64" - cd server - - CC=$NDK_TOOLCHAIN/bin/armv7a-linux-androideabi21-clang CGO_ENABLED=1 GOOS=android GOARCH=arm GOARM=7 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-android-arm7" ./cmd - CC=$NDK_TOOLCHAIN/bin/aarch64-linux-android21-clang CGO_ENABLED=1 GOOS=android GOARCH=arm64 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-android-arm64" ./cmd - CC=$NDK_TOOLCHAIN/bin/i686-linux-android21-clang CGO_ENABLED=1 GOOS=android GOARCH=386 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-android-386" ./cmd - CC=$NDK_TOOLCHAIN/bin/x86_64-linux-android21-clang CGO_ENABLED=1 GOOS=android GOARCH=amd64 go build -ldflags="$LDFLAGS" -tags=nosqlite -trimpath -o "${{ github.workspace }}/dist/TorrServer-android-amd64" ./cmd - - - name: Upload Artifacts - uses: actions/upload-artifact@v7 - with: - name: binaries-${{ matrix.group }} - path: ${{ github.workspace }}/dist/* - - build-gst: - runs-on: windows-latest - env: - NODE_OPTIONS: --openssl-legacy-provider - steps: - - name: Checkout code - uses: actions/checkout@v7 - - - name: Set up Go - uses: actions/setup-go@v7 - with: - go-version: stable - cache: true - - - name: Set up Node - uses: actions/setup-node@v7 - with: - node-version: 20 - - - name: Set version from tag - shell: pwsh - run: | - "VERSION=$($env:GITHUB_REF -replace '^refs/tags/', '')" | Out-File -FilePath $env:GITHUB_ENV -Append - - - name: Install Swag - shell: pwsh - run: go install github.com/swaggo/swag/cmd/swag@latest - - - name: Build Web and Docs - shell: pwsh - env: - REACT_APP_TMDB_API_KEY: ${{ secrets.TMDB_API_KEY }} - run: | - corepack enable - go run gen_web.go - cd server - swag init -g web/server.go --parseInternal --parseDepth 5 - - - name: Build Embedded Windows GST Binary - shell: pwsh - run: powershell -NoProfile -ExecutionPolicy Bypass -File .\.github\scripts\build-gst-windows.ps1 - - - name: Upload GST Artifact - uses: actions/upload-artifact@v7 - with: - name: binaries-gst - path: ${{ github.workspace }}/dist/TorrServer-gst-windows-amd64.exe - - # Builds AppImages for amd64, arm64, armv7 from the binaries - build-appimage: - needs: build - runs-on: ubuntu-22.04 - strategy: - matrix: - include: - - arch_name: amd64 - binary: TorrServer-linux-amd64 - appimage_arch: x86_64 - - arch_name: arm64 - binary: TorrServer-linux-arm64 - appimage_arch: aarch64 - - arch_name: arm7 - binary: TorrServer-linux-arm7 - appimage_arch: armhf - steps: - - name: Checkout code - uses: actions/checkout@v7 - - - name: Set version from tag - run: echo "VERSION=${GITHUB_REF#refs/tags/}" >> $GITHUB_ENV - - - name: Install AppImage dependencies - run: | - sudo apt-get update - sudo apt-get install -y wget fuse libfuse2 - - - name: Download standard binaries artifact - uses: actions/download-artifact@v8 - with: - name: binaries-standard - path: ./dist - - - name: Build AppDir - run: | - APPDIR="${{ github.workspace }}/TorrServer.AppDir" - mkdir -p "$APPDIR/usr/bin" - - cp "./dist/${{ matrix.binary }}" "$APPDIR/usr/bin/torrserver" - chmod +x "$APPDIR/usr/bin/torrserver" - - cat > "$APPDIR/torrserver.desktop" < "$APPDIR/AppRun" <<'EOF' - #!/bin/bash - HERE="$(dirname "$(readlink -f "${0}")")" - exec "${HERE}/usr/bin/torrserver" "$@" - EOF - chmod +x "$APPDIR/AppRun" - - - name: Download appimagetool - run: | - wget -q -O appimagetool "https://github.com/AppImage/appimagetool/releases/download/continuous/appimagetool-x86_64.AppImage" - chmod +x appimagetool - - - name: Build AppImage - run: | - mkdir -p "${{ github.workspace }}/dist-appimage" - ARCH=${{ matrix.appimage_arch }} ./appimagetool "${{ github.workspace }}/TorrServer.AppDir" \ - "${{ github.workspace }}/dist-appimage/TorrServer-linux-${{ matrix.arch_name }}.AppImage" - - - name: Upload AppImage Artifact - uses: actions/upload-artifact@v7 - with: - name: binaries-appimage-${{ matrix.arch_name }} - path: ${{ github.workspace }}/dist-appimage/*.AppImage - - build-ios: - name: Build iOS XCFramework - runs-on: macos-26 - env: - NODE_OPTIONS: --openssl-legacy-provider - XCODE_VERSION: "26.6" - IOS_VERSION_MIN: "18.0" - steps: - - name: Checkout code - uses: actions/checkout@v7 - - - name: Select Xcode Version (${{ env.XCODE_VERSION }}) - uses: maxim-lobanov/setup-xcode@v1.7.0 - with: - xcode-version: ${{ env.XCODE_VERSION }} - - - name: Set up Go - uses: actions/setup-go@v7 - with: - go-version: stable - cache: true - - - name: Set up Node.js - uses: actions/setup-node@v7 - with: - node-version: 20 - - - name: Set version from tag - run: echo "VERSION=${GITHUB_REF#refs/tags/}" >> $GITHUB_ENV - - - name: Build TorrServerKit.xcframework - run: | - chmod +x build-ios.sh - ./build-ios.sh - - - name: Upload iOS XCFramework Artifact - uses: actions/upload-artifact@v7 - with: - name: binaries-ios - path: ${{ github.workspace }}/dist/TorrServer-ios-TorrServerKit.xcframework.zip - if-no-files-found: error - - release: - needs: - - build - - build-gst - - build-appimage - - build-ios - runs-on: ubuntu-latest - permissions: - contents: write - steps: - - name: Download all artifacts - uses: actions/download-artifact@v8 - with: - path: ./all-dist - merge-multiple: true - - - name: Debug - List files - run: | - echo "Listing files to be uploaded:" - ls -R ./all-dist - - - name: Create Release - uses: softprops/action-gh-release@v3 - if: startsWith(github.ref, 'refs/tags/') - with: - files: ./all-dist/* - generate_release_notes: true - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.gitignore b/.gitignore index 2f838b023..ae9564494 100644 --- a/.gitignore +++ b/.gitignore @@ -5,19 +5,12 @@ *.so *.dylib -# Torrserver running binary configs -config.db -settings.json - -# Mac suff +# Mac stuff .DS_Store # Test binary, build with `go test -c` *.test -# Mac stuff -.DS_Store - # Output of the go coverage tool, specifically when used with LiteIDE *.out @@ -33,14 +26,24 @@ src/go.opencensus.io/ pkg/ bin/ dist/ +data/ toolchains/ +.cache/ /src/crawshaw.io/ /src/go.etcd.io/ /src/google.golang.org/ /toolchain/ -/server/web/pages/template/pages/msx # Include bundled GStreamer runtimes !/server/gstreamer/gst-libs/ !/server/gstreamer/gst-libs/**/ !/server/gstreamer/gst-libs/** + +# Generated for Windows gst embed (//go:embed); created by goreleaser before hooks +/server/gstreamer/embedded_gstlib_windows_amd64.zip + +# Local runtime settings accidentally created next to the binary +/config.db +/settings.json +/server/config.db +/server/settings.json diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml deleted file mode 100644 index f091f06b1..000000000 --- a/.gitlab-ci.yml +++ /dev/null @@ -1,24 +0,0 @@ -stages: - - build_go - -build_go: - image: golang:latest - stage: build_go - when: manual - tags: - - amd64 - artifacts: - name: "TorrServer" - paths: - - dist - script: - - apt update - - apt install -y npm zip - - rm -rf /var/lib/apt/lists/* - - npm install -g yarn - - wget -q "https://dl.google.com/android/repository/android-ndk-r25c-linux.zip" - - unzip ./android-ndk-r25c-linux.zip - - rm ./android-ndk-r25c-linux.zip - - pwd - - ls -l - - ./build-all.sh diff --git a/.goreleaser.local.yaml b/.goreleaser.local.yaml new file mode 100644 index 000000000..f286df86d --- /dev/null +++ b/.goreleaser.local.yaml @@ -0,0 +1,164 @@ +--- +# Fast local builds: reduced platform matrix (see Makefile GORELEASER_CONFIG). +# CI / release use .goreleaser.yaml +version: 2 + +project_name: TorrServer + +report_sizes: false + +env: + - CLEAN_ENV=env GOOS= GOARCH= GOARM= + - Metadata_Description=Torrent stream server + - Metadata_License=GPL-3.0 + +git: + tag_sort: -version:refname + +snapshot: + version_template: "{{ .Tag }}-{{ .ShortCommit }}" + +before: + hooks: + - '{{ .Env.CLEAN_ENV }} {{ envOrDefault "GO_BINARY" "go" }} run gen_web.go --clean' + - '{{ .Env.CLEAN_ENV }} {{ envOrDefault "GO_BINARY" "go" }} install github.com/swaggo/swag/cmd/swag@latest' + - bash -c 'cd server && {{ .Env.CLEAN_ENV }} swag init -g web/server.go' + # Windows gst embed: zip gst-libs for //go:embed (binary-gst + embed_gstlib) + - bash -c 'cd server/gstreamer && rm -f embedded_gstlib_windows_amd64.zip && python3 -c "import shutil; shutil.make_archive(\"embedded_gstlib_windows_amd64\", \"zip\", \"gst-libs/win-x86_64\")"' + +builds: + - id: binary + tool: '{{ envOrDefault "GO_BINARY" "go" }}' + dir: server + main: ./cmd + binary: '{{ .ProjectName }}-{{ .Os }}-{{ if eq .Arch "arm" }}arm{{ .Arm }}{{ else }}{{ .Arch }}{{ end }}' + no_unique_dist_dir: true + flags: + - -trimpath + tags: + - nosqlite + ldflags: + - -s -w -checklinkname=0 -X server/version.Version={{.Version}} + env: + - CGO_ENABLED=0 + hooks: + post: + - cmd: bash packaging/appimage/build.sh "{{ .Path }}" "{{ .Os }}" "{{ .Arch }}" "{{ .Arm }}" + targets: + - linux_amd64 + - linux_arm64 + - darwin_amd64 + - darwin_arm64 + + - id: binary-gst + tool: '{{ envOrDefault "GO_BINARY" "go" }}' + dir: server + main: ./cmd + binary: '{{ .ProjectName }}-gst-{{ .Os }}-{{ if eq .Arch "arm" }}arm{{ .Arm }}{{ else }}{{ .Arch }}{{ end }}' + no_unique_dist_dir: true + # CGO_ENABLED=0: static Go ELF; loads libgstreamer at runtime via purego/dlopen (not for Docker) + flags: + - -trimpath + tags: + - nosqlite + - gst + ldflags: + - -s -w -checklinkname=0 -X server/version.Version={{.Version}} + env: + - CGO_ENABLED=0 + targets: + - linux_amd64 + - linux_arm64 + - darwin_amd64 + - darwin_arm64 + - windows_amd64 + overrides: + # Single-file Windows build with embedded GStreamer runtime + - goos: windows + goarch: amd64 + tags: + - nosqlite + - gst + - embed_gstlib + +upx: + - enabled: '{{ envOrDefault "UPX_ENABLED" "false" }}' + ids: + - binary + - binary-gst + goos: + - linux + goarch: + - arm64 + - amd64 + compress: best + lzma: true + +archives: + - id: releases + ids: + - binary + formats: + - binary + name_template: "TorrServer-{{ .Os }}-{{ .Arch }}{{ if .Arm }}{{ .Arm }}{{ end }}" + + - id: releases-gst + ids: + - binary-gst + formats: + - binary + name_template: "TorrServer-gst-{{ .Os }}-{{ .Arch }}{{ if .Arm }}{{ .Arm }}{{ end }}" + +source: + enabled: false + +checksum: + name_template: checksums.txt + extra_files: + - glob: dist/*.AppImage + +release: + mode: append + replace_existing_artifacts: true + make_latest: true + extra_files: + - glob: dist/*.AppImage + +changelog: + use: git + sort: asc + +dockers_v2: + - id: docker + ids: + - binary + dockerfile: Dockerfile + images: + - '{{ envOrDefault "DOCKER_IMAGE_ID" (printf "ghcr.io/%s" (envOrDefault "REGISTRY_IMAGE" "yourok/torrserver")) }}' + tags: + - '{{ .Tag }}' + - latest + platforms: + - linux/amd64 + - linux/arm64 + extra_files: + - docker-entrypoint.sh + labels: + "org.opencontainers.image.title": "{{.ProjectName}}" + "org.opencontainers.image.description": "{{.Env.Metadata_Description}}" + "org.opencontainers.image.url": "{{.GitURL}}" + "org.opencontainers.image.source": "{{.GitURL}}" + "org.opencontainers.image.version": "{{.Tag}}" + "org.opencontainers.image.revision": "{{.FullCommit}}" + "org.opencontainers.image.created": "{{.Date}}" + "org.opencontainers.image.licenses": "{{.Env.Metadata_License}}" + annotations: + "org.opencontainers.image.title": "{{.ProjectName}}" + "org.opencontainers.image.description": "{{.Env.Metadata_Description}}" + "org.opencontainers.image.url": "{{.GitURL}}" + "org.opencontainers.image.source": "{{.GitURL}}" + "org.opencontainers.image.version": "{{.Tag}}" + "org.opencontainers.image.revision": "{{.FullCommit}}" + "org.opencontainers.image.created": "{{.Date}}" + "org.opencontainers.image.licenses": "{{.Env.Metadata_License}}" + sbom: "{{ not .IsSnapshot }}" diff --git a/.goreleaser.yaml b/.goreleaser.yaml new file mode 100644 index 000000000..163e29127 --- /dev/null +++ b/.goreleaser.yaml @@ -0,0 +1,225 @@ +--- +version: 2 + +project_name: TorrServer + +report_sizes: false + +env: + - CLEAN_ENV=env GOOS= GOARCH= GOARM= + - Metadata_Description=Torrent stream server + - Metadata_License=GPL-3.0 + +# MatriX.* tags: sort versions correctly for changelog "previous tag". +git: + tag_sort: -version:refname + +snapshot: + version_template: "{{ .Tag }}-{{ .ShortCommit }}" + +before: + hooks: + - '{{ .Env.CLEAN_ENV }} {{ envOrDefault "GO_BINARY" "go" }} run gen_web.go --clean' + - '{{ .Env.CLEAN_ENV }} {{ envOrDefault "GO_BINARY" "go" }} install github.com/swaggo/swag/cmd/swag@latest' + - bash -c 'cd server && {{ .Env.CLEAN_ENV }} swag init -g web/server.go' + # Windows gst embed: zip gst-libs for //go:embed (binary-gst + embed_gstlib) + - bash -c 'cd server/gstreamer && rm -f embedded_gstlib_windows_amd64.zip && python3 -c "import shutil; shutil.make_archive(\"embedded_gstlib_windows_amd64\", \"zip\", \"gst-libs/win-x86_64\")"' + +builds: + - id: binary + tool: '{{ envOrDefault "GO_BINARY" "go" }}' + dir: server + main: ./cmd + binary: '{{ .ProjectName }}-{{ .Os }}-{{ if eq .Arch "arm" }}arm{{ .Arm }}{{ else }}{{ .Arch }}{{ end }}' + no_unique_dist_dir: true + flags: + - -trimpath + tags: + - nosqlite + ldflags: + - -s -w -checklinkname=0 -X server/version.Version={{.Version}} + env: + - CGO_ENABLED=0 + hooks: + post: + - cmd: bash packaging/appimage/build.sh "{{ .Path }}" "{{ .Os }}" "{{ .Arch }}" "{{ .Arm }}" + targets: + - linux_amd64 + - linux_arm64 + - linux_arm_5 + - linux_arm_7 + - linux_386 + - windows_amd64 + - windows_386 + - darwin_amd64 + - darwin_arm64 + - freebsd_amd64 + - freebsd_arm_7 + - linux_mips_softfloat + - linux_mipsle_softfloat + - linux_mips64_softfloat + - linux_mips64le_softfloat + - linux_riscv64 + + - id: binary-gst + tool: '{{ envOrDefault "GO_BINARY" "go" }}' + dir: server + main: ./cmd + binary: '{{ .ProjectName }}-gst-{{ .Os }}-{{ if eq .Arch "arm" }}arm{{ .Arm }}{{ else }}{{ .Arch }}{{ end }}' + no_unique_dist_dir: true + # CGO_ENABLED=0: static Go ELF; loads libgstreamer at runtime via purego/dlopen (not for Docker) + flags: + - -trimpath + tags: + - nosqlite + - gst + ldflags: + - -s -w -checklinkname=0 -X server/version.Version={{.Version}} + env: + - CGO_ENABLED=0 + targets: + - linux_amd64 + - linux_arm64 + - darwin_amd64 + - darwin_arm64 + - windows_amd64 + overrides: + # Single-file Windows build with embedded GStreamer runtime + - goos: windows + goarch: amd64 + tags: + - nosqlite + - gst + - embed_gstlib + + - id: android + dir: server + main: ./cmd + binary: '{{ .ProjectName }}-{{ .Os }}-{{ if eq .Arch "arm" }}arm{{ .Arm }}{{ else }}{{ .Arch }}{{ end }}' + no_unique_dist_dir: true + builder: go + tool: '{{ envOrDefault "GO_ANDROID_BINARY" (envOrDefault "GO_BINARY" "go") }}' + flags: + - -trimpath + tags: + - nosqlite + ldflags: + - -s -w -checklinkname=0 -X server/version.Version={{.Version}} + targets: + - android_arm_7 + - android_arm64 + - android_386 + - android_amd64 + overrides: + - goos: android + goarch: arm + goarm: "7" + env: + - CGO_ENABLED=1 + - CC={{ envOrDefault "NDK_TOOLCHAIN" "$ANDROID_NDK_LATEST_HOME/toolchains/llvm/prebuilt/linux-x86_64" }}/bin/armv7a-linux-androideabi21-clang + - goos: android + goarch: arm64 + env: + - CGO_ENABLED=1 + - CC={{ envOrDefault "NDK_TOOLCHAIN" "$ANDROID_NDK_LATEST_HOME/toolchains/llvm/prebuilt/linux-x86_64" }}/bin/aarch64-linux-android21-clang + - goos: android + goarch: "386" + env: + - CGO_ENABLED=1 + - CC={{ envOrDefault "NDK_TOOLCHAIN" "$ANDROID_NDK_LATEST_HOME/toolchains/llvm/prebuilt/linux-x86_64" }}/bin/i686-linux-android21-clang + - goos: android + goarch: amd64 + env: + - CGO_ENABLED=1 + - CC={{ envOrDefault "NDK_TOOLCHAIN" "$ANDROID_NDK_LATEST_HOME/toolchains/llvm/prebuilt/linux-x86_64" }}/bin/x86_64-linux-android21-clang + +upx: + - enabled: '{{ envOrDefault "UPX_ENABLED" "false" }}' + ids: + - binary + - binary-gst + - android + goos: + - linux + - windows + - android + goarch: + - arm + - arm64 + - amd64 + - '386' + - mips + - mipsle + compress: best + lzma: true + +archives: + - id: releases + ids: + - binary + - android + formats: + - binary + name_template: "TorrServer-{{ .Os }}-{{ .Arch }}{{ if .Arm }}{{ .Arm }}{{ end }}" + + - id: releases-gst + ids: + - binary-gst + formats: + - binary + name_template: "TorrServer-gst-{{ .Os }}-{{ .Arch }}{{ if .Arm }}{{ .Arm }}{{ end }}" + +source: + enabled: false + +checksum: + name_template: checksums.txt + extra_files: + - glob: dist/*.AppImage + +release: + mode: append + replace_existing_artifacts: true + make_latest: true + extra_files: + - glob: dist/*.AppImage + +changelog: + use: git + sort: asc + +dockers_v2: + - id: docker + ids: + - binary + dockerfile: Dockerfile + images: + - '{{ envOrDefault "DOCKER_IMAGE_ID" (printf "ghcr.io/%s" (envOrDefault "REGISTRY_IMAGE" "yourok/torrserver")) }}' + tags: + - '{{ .Tag }}' + - latest + platforms: + - linux/amd64 + - linux/arm64 + - linux/arm/v7 + extra_files: + - docker-entrypoint.sh + labels: + "org.opencontainers.image.title": "{{.ProjectName}}" + "org.opencontainers.image.description": "{{.Env.Metadata_Description}}" + "org.opencontainers.image.url": "{{.GitURL}}" + "org.opencontainers.image.source": "{{.GitURL}}" + "org.opencontainers.image.version": "{{.Tag}}" + "org.opencontainers.image.revision": "{{.FullCommit}}" + "org.opencontainers.image.created": "{{.Date}}" + "org.opencontainers.image.licenses": "{{.Env.Metadata_License}}" + annotations: + "org.opencontainers.image.title": "{{.ProjectName}}" + "org.opencontainers.image.description": "{{.Env.Metadata_Description}}" + "org.opencontainers.image.url": "{{.GitURL}}" + "org.opencontainers.image.source": "{{.GitURL}}" + "org.opencontainers.image.version": "{{.Tag}}" + "org.opencontainers.image.revision": "{{.FullCommit}}" + "org.opencontainers.image.created": "{{.Date}}" + "org.opencontainers.image.licenses": "{{.Env.Metadata_License}}" + sbom: "{{ not .IsSnapshot }}" diff --git a/.testcoverage.yml b/.testcoverage.yml new file mode 100644 index 000000000..17c62486e --- /dev/null +++ b/.testcoverage.yml @@ -0,0 +1,9 @@ +# Thresholds for vladopajic/go-test-coverage. Profile is produced by `make test`. +# File and package floors stay at 0: several packages have no tests yet. +# Generated swagger (server/docs) is not in the profile. +profile: server/coverage.out + +threshold: + file: 0 + package: 0 + total: 70 diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..3d2ec4a7e --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,302 @@ +# TorrServer agent instructions + +Module path is `server`. Go commands run from `server/`. This checkout has no `internal/` tree and no root `go.mod`. Do not invent a second engine. + +`server/go.mod` is `go 1.25.7`. CI installs `GO_VERSION` from `.github/versions.env` (currently 1.27.3) and keeps the module line aligned with `GO_ANDROID_VERSION`. `server/.golangci.yml` pins `run.go` to `1.25.7`; bump that line in the same change as the module. + +## What this process is + +TorrServer streams BitTorrent payloads over HTTP Range. The hot path is `GET/HEAD /stream` and `/play` → `torr.Torrent.Stream` → `torrstor.Reader` → piece cache → `anacrolix/torrent`. Optional surfaces (DLNA, Bonjour, FUSE, WebDAV, Telegram, MCP, GStreamer) must not add work to that path. + +Out of scope unless the task names them: embedded JS, a plugin runtime, and an FFmpeg transcode pipeline. Remux/transcode already lives behind the `gst` build tag in `server/gstreamer`. + +## Dependency forks + +``` +github.com/anacrolix/torrent v1.59.1 => github.com/lampac-nextgen/torrent v1.2.36 +github.com/anacrolix/upnp v0.1.4 => github.com/tsynik/upnp v0.1.5 +github.com/anacrolix/utp v0.2.0 => github.com/lampac-nextgen/utp v0.2.1 +``` + +Import the upstream paths. The replace directives supply the forks. + +Comments marked `NE` are upstream APIs this fork does not have (`DisableWebtorrent`, `HeaderObfuscationPolicy`, storage `Capacity`, directory file storage). Comments marked `OE` are the behavior this fork actually runs. Do not call `NE` methods. The seeder in tests cannot use a directory backend; `server/torr/offlinetest` serves bytes from `memStore`. + +`configure()` in `server/torr/btserver.go` spoofs `qBittorrent/4.3.9` / peer id prefix `-qB4390-`. `UpnpID` stays `TorrServer/`. `TotalHalfOpenConns` is hardcoded to 500. Do not "correct" the peer id to TorrServer's own name without a reason: swarms treat the qB prefix differently. + +## UI boundary + +The React UI is `web/` (Yarn, Vite). The binary serves a generated copy under `server/web/pages/template/pages`. Edit `web/src`, then rebuild (`gen_web.go`, `make webgen-clean`). Do not hand-edit the generated pages. + +A new or changed string belongs in `web/src/locales`. `yarn i18n:check` requires the same flattened keys in `en`, `ru`, `ua`, `bg`, `fr`, `ro`, and `zh`, as nested objects. + +When `web/src` changes, CI runs `yarn typecheck`, `yarn lint`, `yarn test`, `yarn i18n:check`, `yarn test:e2e`, and `yarn build`. From `web/`: `yarn typecheck`, `yarn lint`, `yarn test`, `yarn i18n:check`. Embed with `make webgen` (`go run gen_web.go`) or `make webgen-clean` (deletes `web/build` first). Do not pass `VITE_SERVER_HOST` pointing at localhost into a production `yarn build` — `web/vite.config.ts` refuses it. + +Web client map (source of truth is `web/src`, not the embedded copy): + +| Path | Role | +| --- | --- | +| `web/src/shared/hooks/useTorrentsQuery.ts` | Library poll. `queryFn` must take React Query's `signal`. `refetchIntervalInBackground` stays default false, so a hidden tab does not hit `/torrents`. The `document.hidden` branch only matters after a recompute. | +| `web/src/shared/hooks/useTorrentDetail.ts` | Details poll. Same signal rule. Mirrors rows into `TORRENTS_QUERY_KEY` via `upsertTorrentsInList`. | +| `web/src/shared/cache/useUpdateCache.ts` | `/cache` snake poll. `setTimeout` after the response (not `setInterval`). Aborts on unmount. Pauses while `document.hidden`. Active 100ms, idle 1s. Details polls on the Cache tab, the cache-map modal, and Stats (idle). The Files tab does not. | +| `web/src/shared/lib/posterPlay.ts` | Re-exports `streamPlayUrl`, `streamPreloadUrl`, `torrentPlaylistUrl` from `streamLinks.ts`. | +| `web/src/shared/lib/mediaBase.ts` | `externalLink` swaps scheme/host/port from `GET /mediabase`, then `withAuthMediaUrl`. | +| `web/src/shared/api/authCredentials.ts` | `withAuthMediaUrl` adds Basic userinfo only when `hostname` matches the page (port and scheme may differ). Do not put the password on another host. | +| `web/src/shared/api/settings.ts` | `sanitizeBTSetsForWire` before `POST /settings`. `CacheSize` is bytes on the wire. The dialog keeps megabytes in component state (`SettingsDialog`). | +| `web/src/features/settings/SettingsDialog.tsx` | Save stays disabled until `GET /settings` succeeds. `403` is read-only. | +| `web/src/features/details/FilesDataGrid.tsx` | File rows. `content-visibility: auto` so a 500-file pack does not lay out every row. | +| `web/src/index.tsx` | `refetchOnWindowFocus: false` globally. Do not add a second poller. | + +API contracts the UI depends on: + +- `POST /settings` with `action: set` and a nil `sets` is `400`. Read-only `set` and `def` are `403` `{"error":"Read-only mode"}` and must not call `restartServices`. `CacheSize` `0` is rewritten to 64MiB in `SetBTSets`. Integer fields reject JSON floats (`encoding/json` into `int` fails the whole body). +- `GET /stream` and `GET /stream/*fname` both exist. Preload is `/stream?link&index&preload` (no filename). Play is `/stream/?link&index&play`. `link` is an infohash in the UI; a magnet in that query has to be encoded or `&dn` / `&tr` split it. The UI builds those URLs in `web/src/shared/lib/streamLinks.ts` (`encodePathSegment` for the filename, `URLSearchParams` for `link` and `index`). +- `GET /mediabase` is the external-player origin. In-page `