DevCollab Platform is a full-stack, enterprise-grade developer collaboration ecosystem engineered to showcase production backend systems, real-time distributed messaging, custom search engines, resilient caching patterns, and end-to-end observability at scale.
+-----------------------+
| Next.js 16 Client |
| (React 19, Zustand) |
+-----------+-----------+
|
HTTP / REST | WebSockets (Socket.io)
v
+-----------------------------------------------------------------------------------+
| NestJS Application Gateway & API Services |
| |
| +-------------------+ +--------------------+ +------------------------------+ |
| | RateLimitGuard | | Auth (Argon2id + | | Notifications Gateway | |
| | (Sliding Window) | | Token Families) | | (JWT Handshake, Multi-Tab) | |
| +---------+---------+ +---------+----------+ +--------------+---------------+ |
| | | | |
| v v v |
| +-------------------+ +--------------------+ +------------------------------+ |
| | Inverted Indexing | | Cache-Aside & | | BullMQ Distributed Queues | |
| | (TF-IDF + ZSET) | | Stampede Protection| | (Search, Notify, Analytics) | |
| +---------+---------+ +---------+----------+ +--------------+---------------+ |
+------------|----------------------|----------------------------|------------------+
| | |
v v v
+----------------------+-----------------------+ +-----------------------+
| Redis 7 | | PostgreSQL 16 |
| - Inverted Search Index (ZSET) | | - Relational Schema |
| - Sliding Window Counters | | - Token Family Store |
| - Cache-Aside Layer | | - ACID Domain Entities|
| - BullMQ Queue Backing Store | | |
+----------------------------------------------+ +-----------------------+
^
| RED Method Metrics
+----------+----------+
| Prometheus + Grafana|
+---------------------+
| Layer | Technology | Key Use Cases / Engineering Purpose |
|---|---|---|
| API Backend | NestJS 11 + TypeScript | Modular architecture, Dependency Injection, Custom Guards, Interceptors, and Pipeline Filters. |
| Frontend Web | Next.js 16 + React 19 | App Router, Server/Client components, Zustand state, TanStack Query v5, Tailwind CSS, Radix UI. |
| Primary Database | PostgreSQL 16 | Relational integrity, gen_random_uuid(), ENUM types, composite indexes, JSONB payloads. |
| In-Memory Store | Redis 7 (ioredis) | Custom inverted search index, sliding-window rate limiting, cache-aside layer, BullMQ jobs. |
| Real-Time Engine | Socket.io 4 | Handshake JWT authentication, room-based multi-tab WebSocket fanout, hybrid push/pull fallback. |
| Job Queue | BullMQ 5 | Async indexing & notification processing, idempotency keys, exponential backoff, concurrency controls. |
| Authentication | Argon2id + JWT | Memory-hard password hashing, short-lived JWT access tokens, HTTP-Only cookies, rotating token families. |
| Observability | Prometheus + Grafana | Custom application metrics, RED method (Rate, Errors, Duration), histogram latency percentiles (p50/p90/p99). |
| Infrastructure | Docker Compose | Containerized service orchestrations for PostgreSQL, Redis, Prometheus, and Grafana. |
Instead of relying on heavy external search clusters for core entity retrieval, DevCollab features a hand-rolled inverted index built natively on Redis Sorted Sets:
-
Tokenization Pipeline: Incoming titles and descriptions undergo lowercasing, punctuation stripping, stop-word filtering, and Porter stemming (
tokenizer.ts). - TF-IDF Scoring: Term Frequency (TF) normalized by document length is computed alongside Inverse Document Frequency ($IDF = \log(N / df)$) to prevent long documents from skewing relevance scores.
-
Redis ZSET Indexing: Indexed tokens map to Redis ZSETs storing
(entityId, score). Intersecting multi-term queries leveragesZINTERSTOREfor zero-latency server-side set operations. -
Async Pipeline: Document updates emit events processed by BullMQ background workers (
search-indexqueue) to decouple search indexing from HTTP request paths.
- Argon2id Hashing: Replaces standard bcrypt with OWASP-recommended Argon2id (memory-hard algorithm resistant to GPU/ASIC brute-force side-channel attacks).
- Token Family Tracking: Refresh tokens are stored hashed in PostgreSQL with a
family_idcolumn (001_init_auth.sql). - Replay Attack Mitigation: If a previously revoked or used refresh token is presented (indicating a stolen token replay attack), the system revokes the entire
family_idimmediately, invalidating all active sessions for that user family.
- Pipelined Atomic Executions: Implemented via
@RateLimit()decorator and custom NestJSRateLimitGuard. - Sliding Window Algorithm: Uses Redis Sorted Sets (
ZSET) where scores are millisecond timestamps. - 1 Round-Trip Overhead: Bundles
ZREMRANGEBYSCORE,ZCARD,ZADD, andPEXPIREinto an atomic Redis pipeline, reducing 4 network round trips to a single network call. - Fair Keying & Standard Headers: Differentiates between authenticated users (
user:<userId>) and unauthenticated clients (ip:<ip>). Injects RFC-standard compliance headers (X-RateLimit-Limit,X-RateLimit-Remaining,X-RateLimit-Reset).
- Handshake JWT Auth: Socket.io connections (
/notificationsnamespace) authenticate during connection handshake (notifications.gateway.ts). - Multi-Tab Fanout: Sockets join a dynamic room
user:<userId>. Multi-tab sessions seamlessly receive real-time events without duplicate processing. - Hybrid Push/Pull Fallback: When an event occurs (e.g. PR merged or issue assigned), the system attempts real-time WebSocket push. If the user is offline (
socketsInRoom === 0), the event falls back to persistent database storage for polling retrieval upon next client login.
- Cache-Aside Pattern:
CacheServicelazy-loads cache data with configurable TTLs and automated non-blockingSCAN-based pattern invalidation (deleteByPattern). - In-Process Request Coalescing: Resolves cache stampedes (Thundering Herd problem). If 500 concurrent HTTP requests miss the cache simultaneously for the same key, an in-flight Promise map (
inflightRequests) ensures exactly 1 request queries PostgreSQL while the other 499 await the shared pending Promise. - Zero-Stale Security Boundary: Security-critical membership and permission checks explicitly bypass the cache layer to guarantee zero revocation propagation lag.
- Asynchronous Execution: Heavy tasks (search indexing, notification dispatch, analytics ingestion) are offloaded to Redis-backed BullMQ queues.
-
Idempotency & Concurrency: Jobs use deterministic idempotency keys to prevent duplicate execution during retries. Configured concurrency bounds prevent downstream database saturation (
search: 3,notifications: 5,analytics: 10). -
Exponential Backoff: Automatic retry policies (1s
$\rightarrow$ 2s$\rightarrow$ 4s) ensure resilience against transient infrastructure blips.
- Prometheus Collector: Implemented via NestJS
ObservabilityModuleandprom-client. - RED Method Implementation:
- Rate: Request throughput tracking per path and HTTP method.
- Errors: 4xx and 5xx status code distribution monitoring.
- Duration: Histogram bucket percentiles for API response latency (p50, p90, p99).
- Deep System Telemetry: Tracks database query execution times, Redis operation latencies, and active BullMQ queue depths.
DevCollab utilizes PostgreSQL 16 with raw SQL migrations (infra/migrations/):
+-------------------+ +-------------------+ +--------------------+
| users |1 *| refresh_tokens | | organizations |
+-------------------+-------+-------------------+ +--------------------+
| id (UUID, PK) | | id (UUID, PK) | | id (UUID, PK) |
| email (VARCHAR) | | user_id (FK) | | handle (VARCHAR) |
| password_hash | | token_hash (TEXT) | | owner_id (FK) |
| created_at | | family_id (UUID) | +---------+----------+
+---------+---------+ | revoked (BOOL) | |1
| +-------------------+ |
|1 |*
| +---------+----------+
|* | repositories |
+---------+---------+ +--------------------+
| org_members | | id (UUID, PK) |
+-------------------+ | org_id (FK) |
| org_id (FK) | | name (VARCHAR) |
| user_id (FK) | | default_branch |
| role (ENUM) | +---------+----------+
+-------------------+ |1
|*
+-------------------+ +---------+----------+
| pull_requests | | issues |
+-------------------+ +--------------------+
| id (UUID, PK) | | id (UUID, PK) |
| repo_id (FK) | | repo_id (FK) |
| author_id (FK) | | author_id (FK) |
| status (ENUM) | | assignee_id (FK) |
+-------------------+ | status (ENUM) |
+--------------------+
- Node.js: v20+
- Docker & Docker Compose: v2.20+
- npm: v10+
git clone https://github.com/imtushar01/devcollab-platform.git
cd devcollab-platform
# Copy API environment file
cp apps/api/.env.example apps/api/.envdocker-compose -f infra/docker-compose.yml up -dVerify running containers:
- PostgreSQL 16:
localhost:5432 - Redis 7:
localhost:6379 - Prometheus:
http://localhost:9090 - Grafana:
http://localhost:3001(Credentials:admin/admin)
Execute raw migration scripts sequentially against Postgres:
docker exec -i devcollab-postgres psql -U devcollab -d devcollab < infra/migrations/001_init_auth.sql
docker exec -i devcollab-postgres psql -U devcollab -d devcollab < infra/migrations/002_orgs_and_repos.sql
docker exec -i devcollab-postgres psql -U devcollab -d devcollab < infra/migrations/003_pull_requests.sql
docker exec -i devcollab-postgres psql -U devcollab -d devcollab < infra/migrations/004_issues_and_notifications.sql
docker exec -i devcollab-postgres psql -U devcollab -d devcollab < infra/migrations/005_search_index.sqlcd apps/api
npm install
npm run start:devAPI running at http://localhost:3000 (Health Check: http://localhost:3000/health, Metrics: http://localhost:3000/metrics)
cd apps/web
npm install
npm run devWeb dashboard running at http://localhost:3002 (or available port)
# Run unit tests across backend services
cd apps/api
npm run test
# Run e2e integration tests
npm run test:e2e
# Inspect test coverage report
npm run test:covTested with k6 against local stack — each virtual user runs a complete journey: register → login → browse org → list repos → search → create org → create repo → create issue
| Metric | Value |
|---|---|
| Peak concurrent users | 100 |
| Total requests | 22,482 |
| Throughput | 92 req/s |
| p95 latency | 474ms |
| p99 latency | 1.55s |
| Business logic error rate | 0% |
| Complete user journeys | 2,498 |
| Test duration | 4 minutes |
| Endpoint | p95 |
|---|---|
| Login (Argon2id verify) | 68ms |
| Search (TF-IDF lookup) | 93ms |
| Create issue (DB + queue) | 253ms |
| Org profile (Redis cached) | ~2ms |
Bottleneck identified: Search degrades under high concurrency due to sequential Redis calls per query term. Fix: batch pipeline Redis lookups or migrate to Elasticsearch at scale. Documented in
docs/architecture.md.
When defending this architecture in SDE-2 / Senior System Design interviews, highlight the following trade-offs:
Answer: Elasticsearch adds significant memory overhead (JVM Heap, Lucene segments) and operational complexity for small-to-medium datasets. By leveraging Redis Sorted Sets (
ZSET), we achieve sub-millisecond keyword lookup directly within our existing caching layer, keeping network hops at zero and maintaining a single in-memory infrastructure footprint. For multi-gigabyte document corpora, we can cleanly migrate the indexing job insearch-index.worker.tsto push to Elasticsearch without altering downstream domain services.
Answer: Bcrypt is CPU-bound and susceptible to GPU farm acceleration attacks. Argon2id (winner of the Password Hashing Competition and OWASP primary choice) is both memory-hard and time-hard. It imposes configurable memory cost constraints that defeat parallelized ASIC/GPU attack vectors while preventing side-channel timing attacks.
Answer: When a client requests an Access Token refresh, the server issues a new Refresh Token and invalidates the old one within the same token family (
family_id). If an attacker steals a Refresh Token and tries to reuse it after the legitimate user has already refreshed, the server detects a second reuse of a revoked token. Recognizing an breach, it immediately revokes the entire family, invalidating all sessions and forcing a clean re-authentication.
Answer: Traditional sliding window implementations require multiple network round trips (
ZREMRANGEBYSCORE->ZCARD->ZADD->EXPIRE). By executing all 4 commands in an atomic ioredis pipeline, network latency drops from$4 \times RTT$ to$1 \times RTT$ . Furthermore, sorting set elements by millisecond timestamp eliminates the boundary burst issues common in Fixed Window or Token Bucket algorithms.
Answer: The
inflightRequestsMap deduplicates simultaneous cache misses on a single application instance. In a horizontally scaled multi-node cluster, 5 nodes receiving simultaneous cache misses would still result in 5 DB queries (one per node). To extend this cluster-wide, we would upgradeCacheServiceto acquire a RedisSET NXdistributed lock with a short TTL before hitting the database.
Answer: We implement a Hybrid Push/Pull pattern. Real-time delivery via Socket.io is an optimization layer, not the source of truth. Every notification is first transactionally committed to PostgreSQL (
notificationstable). The WebSocket gateway then attempts an in-memory room push. If the socket is disconnected or drops, the unread notification remains persisted and is fetched immediately when the frontend reconnects or polls.
This project is open-source and available under the MIT License.