diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 01fcb9be..cb80999c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -71,6 +71,18 @@ jobs: PROOVY_TEST_PG_PASSWORD: proovy123 PROOVY_REQUIRE_DB_TESTS: "true" + # 실제 Brain 연동(task 1.87)은 proovy-brain이 private이라 CI에서 실행하지 않는다(사용자 결정). 통과로 보이지 않게 skip을 드러낸다. + - name: Report real-Brain integration tests NOT EXECUTED + if: always() + run: | + shopt -s nullglob + files=(build/test-results/test/TEST-com.proovy.integration.brain.*.xml) + if [ ${#files[@]} -eq 0 ]; then echo "real-Brain integration: no result file"; exit 0; fi + for f in "${files[@]}"; do + summary=$(grep -o 'tests="[0-9]*" skipped="[0-9]*" failures="[0-9]*" errors="[0-9]*"' "$f" | head -1) + echo "::warning title=Real Brain integration NOT EXECUTED::$(basename "$f" .xml) $summary — run scripts/brain-integration/run-local.sh locally (README)" + done + - name: Upload test report if: always() uses: actions/upload-artifact@v4 diff --git a/README.md b/README.md index c5f6338a..ccfb6c2c 100644 --- a/README.md +++ b/README.md @@ -154,7 +154,7 @@ proovy-api/ - Note가 유일한 대화 단위 (`conversationId = noteId`), 메시지는 Note에 직접 연결 - 캔버스 이미지 업로드 - 대화 검색 -- 신규 run 접수·SSE·취소는 S07 후속 작업 (구형 `/stream/v2` 경로는 제거됨) +- 신규 run 접수·SSE·취소는 `POST /api/conversations`·`/api/ai-runs/*`(browser-run-v1). 구형 `/stream/v2`·ChatSession 경로는 없다 ### 8. Credit 도메인 - 크레딧 잔액 조회 @@ -190,6 +190,24 @@ PROOVY_REQUIRE_DB_TESTS=true ./gradlew test **전용 일회용 Postgres 클러스터에서만 실행한다.** role 경계 테스트는 cluster 단위 role(`proovy_server`, `brain_*`)을 만들고 비밀번호를 덮어쓰며, database를 지워도 role은 남는다. 공유·운영 클러스터를 `PROOVY_TEST_PG_URL`로 지정하지 않는다. +### 실제 Brain 연동 테스트 (task 1.87) — **CI에서는 실행되지 않는다** + +`com.proovy.integration.brain.ServerBrainIntegrationTest`는 proovy-brain `testkit/`이 띄운 실제 Python 내부 API·runtime과 +Server 전체 경로를 연결해 spec 07 Verification을 검사한다. proovy-brain이 private이라 Server CI는 Brain을 checkout할 수 없고, +사용자 결정으로 **CI에서는 skip**한다. skip은 통과가 아니다 — 테스트 로그에 `NOT EXECUTED — real Brain integration …` 이유가 +남고 JUnit 보고서에는 skipped로 집계된다. 결과는 아래 로컬 명령으로 만들고 이슈·PR에 Server·Brain commit과 함께 기록한다. + +```bash +# proovy-brain checkout(uv sync --locked 완료)을 원하는 commit에 맞춘 뒤 +BRAIN_DIR=../proovy-brain PG_CONTAINER=proovy-postgres-brain-it scripts/brain-integration/run-local.sh +# JDK가 없으면 GRADLE_IN_DOCKER=1 (eclipse-temurin:21-jdk 임시 컨테이너, --network host) +``` + +스크립트는 격리 Postgres(tmpfs) → Brain testkit supervisor → Gradle(`PROOVY_REQUIRE_BRAIN_IT=true`라 미구성 시 실패) → 정리 순서로 +돌고, Brain fixture sha256이 `src/test/resources/contracts/brain/PIN*.json`과 같은지·Brain commit이 checkout과 같은지도 검사한다. +모델·Daytona는 offline fake seat이고 일부 결과(자료 부재 안내·scope 밖 출처·변조 envelope)는 testkit scripted다 — 수학 품질·실제 +provider·Cloud Run identity는 이 테스트가 인증하지 않는다(S14). + ## API 문서 - **Swagger UI**: https://api.proovy.ai.kr/swagger-ui/index.html#/ diff --git a/docs/contracts/brain-integration.md b/docs/contracts/brain-integration.md new file mode 100644 index 00000000..ad25f49c --- /dev/null +++ b/docs/contracts/brain-integration.md @@ -0,0 +1,70 @@ +# Server ↔ Brain 로컬 실행 연동 인수 (task 1.87, #226) + +S07 Server 구현 전체(1.74~1.86)를 **실제 Python Brain 내부 API·runtime**과 로컬에서 연결해 인수한다. +Brain 쪽 test bootstrap은 proovy-brain `testkit/`(PR #150, squash `7b8608ee345761526aee0c76186ca984b496f41e`)이고, +Server 쪽 검사는 `com.proovy.integration.brain.ServerBrainIntegrationTest`다. + +## 실행 + +```bash +BRAIN_DIR= \ +PG_CONTAINER=proovy-postgres-brain-it GRADLE_IN_DOCKER=1 scripts/brain-integration/run-local.sh +``` + +순서: Gradle `testClasses` → 격리 Postgres(tmpfs, `postgres:15-alpine`) → Brain testkit supervisor(`uv run --locked python -m testkit.integration supervise`) +→ warm-up child → `./gradlew test --tests 'com.proovy.integration.brain.*'`(`PROOVY_REQUIRE_BRAIN_IT=true`) → 컨테이너·프로세스 정리. + +- **Server CI는 실행하지 않는다**(사용자 결정: proovy-brain이 private, checkout token 없음). CI 로그에는 skip 이유(`NOT EXECUTED — real Brain integration …`)와 + `Report real-Brain integration tests NOT EXECUTED` step의 warning이 남는다. skip은 통과가 아니다. +- 한 database에 빈 제품 baseline(`proovy_server`가 Flyway 실행)과 Brain schema(`brain_migration`·`brain_runtime`, Brain bootstrap·migrate)를 두고 + `db/provisioning/role-boundary.sql`을 적용한다(운영 형태). Server datasource는 `proovy_server`다. +- 스크립트는 `test --rerun`으로 항상 실제 실행하고, 끝에 `TEST-com.proovy.integration.brain.*.xml`이 이번 실행에서 생겼고 tests > 0·skipped·failures·errors = 0인지 검사해 아니면 실패한다(UP-TO-DATE·skip 위장 방지). 정리 때 supervisor에 `/brain/stop`을 보내고 프로세스 그룹 전체를 끈다. +- 공용 호스트 부하 대응: supervisor의 child readiness 대기는 60초 고정이라, 스크립트가 child를 미리 한 번 띄워 page cache를 데우고 + Java client는 `did not become ready`일 때만 start·restart를 최대 3번 다시 보낸다(후속 메모: Brain readiness 대기 설정화). + +## S07 Verification 대응 + +| Verification (spec 07) | 테스트 | 실제로 거친 경로 | +| --- | --- | --- | +| 1. 빈 DB·ChatSession/thread 없음, noteId 없는 중복·같은 Note 동시 전송이 하나만 남고 거부는 메시지 없음 | `firstTextQuestion…`, `concurrentDuplicateNotelessRequests…`, `concurrentSameNoteRequests…` (+ 단계 3 `RunPathEndToEndPostgresTest`·`LegacyExecutionPathGuardTest`) | HTTP 접수 → coordinator → Brain graph 1회 → terminal transaction. 409 RUN4091 + activeRunId, 메시지·run·차감 없음 | +| 2. SSE 단절·재접속 seq·canonical 복원, replay/live 무유실 | `firstTextQuestion…`(POST live = GET replay 바이트), `browserDisconnectAndInternalSseLoss…`, `storedTerminalLostInDelivery…` | 브라우저 Last-Event-ID 재접속, Brain 내부 SSE 첫 frame 직전 abort → status executing → 같은 attempt·fence OBSERVE 재부착, terminal 전 abort → status terminal 회수 | +| 3. 완료·취소·deadline·이전 attempt 경합에서 확정과 보상이 동시에 일어나지 않음 | `runningCancelBlocks…`, `pendingCancel…`, `deadlineRace…`, `brainSideDeadline…`(fake clock), `staleFenceTerminal…`, `serverRestart…`, `brainLostBeforeStorage…` | terminal event 1개·보상 ≤ 1·재실행 0을 run마다 단언. deadline 경합은 승자를 고정하지 않는다(실측: Server sweep). Brain fake clock은 Brain timeout terminal 확정을 단언. `serverRestart`는 같은 JVM의 coordinator `stop()/start()`(프로세스 재시작 아님) | +| 4. clarification·unsupported_only·혼합·no-match·최종 실패·PENDING/RUNNING 취소 순사용량 | `outcomeMatrix…`, `explainFollowUp…`, 취소 2종 | 정상 solve·check·explain 1, 미검증 공개 1, clarification 0, unsupported_only 0, 혼합 1, 자료 부재(scripted) 1, verification_failed 0, provider_error 0, PENDING 취소 0, RUNNING 취소 1, deadline 0, 저장 전 소실 0 | +| 5. Java DTO·fingerprint·source 변환이 공용 fixture와 일치, scope/provenance 위반 거부 | `javaAndPythonAgree…`, `outOfScopeSource…`, `forgedRunEnvelope…`, `staleFenceTerminal…` | Brain record fingerprint = `ai_runs.request_fingerprint`, Brain fixture sha256 = Server PIN 전부, scope 밖 asset(scripted) → FAILED internal_error·출처/solution 없음, 다른 run envelope(stream+status, scripted) → FAILED, 이전 fence(stream, scripted) → 버리고 저장 terminal 회수 | + +보안 경계(`securityBoundariesHold`): Brain이 받은 호출은 전부 shared token이고 body에 사용자 JWT·`Bearer`·grant·`read_url`·서명 URL이 없다. +token 없음·사용자 JWT·틀린 token은 Brain이 401/403으로 거부한다. 브라우저 frame·대화 조회에 fence·grant·read URL·서비스 token이 없다. +같은 database에서 `brain_runtime` → `public.ai_runs` 42501, `proovy_server` → `brain_runtime.run_records` 42501. + +## 구분 — fake·scripted·미검증 + +- **fake provider**: Brain 모델·Daytona·page VLM은 offline seat다. 결과 분류(검증/미검증/실패)는 scripted seat 판정이고 수학 품질을 인증하지 않는다. +- **scripted terminal/fault (그래프 미실행 또는 전달 변조)**: `no_match_notice`(자료 부재 안내 — Brain runtime에 문서 근거 no-match 경로가 아직 없다), + `out_of_scope_source`, `rewrite_terminal`(+`rewrite_status`). 저장·전달은 실제 Brain store·SSE(PgRuntimeBackend)를 거친다. +- **미검증(S14)**: 실제 유료 모델 품질·비용, Cloud Run 서비스 계정 ID token·IAM, Supabase/GCS, 실제 자산 업로드·signed URL(자산 경로는 1.81 seeded 계약 자료로만 검수), + 부하·다중 인스턴스, 브라우저 화면. +- **Server CI**: skip(위). 결과는 이슈 #226·PR 본문에 Server head·Brain SHA와 함께 기록한다. + +## S08이 쓸 외부 계약 + +- 브라우저 wire는 browser-run-v1 그대로다(변경 없음): `POST /api/conversations`(fetch 기반 SSE reader, body·Authorization 필요), + `GET /api/ai-runs/{runId}/events`(Last-Event-ID 또는 `lastEventId` query), `GET /api/ai-runs/{runId}`, `POST /api/ai-runs/{runId}/cancel`(body 없음), + `GET /api/conversations/{noteId}/messages`. 재접속 cursor는 마지막 처리 frame `id`. +- 실제 Brain에서 나오는 frame 순서(정상 solve): `run.created` → `activity.started/completed`(입력 분석·풀이·검증) → `response.page.started` → + `response.page.completed` → `run.completed`. 취소는 `run.canceling` → `run.canceled`, 실패는 `run.failed`(보상된 run은 순사용량 0). +- 이월(계약 변경 필요, 이 태스크는 바꾸지 않음): superseded 재표현 표시 필드, 대화 조회 before/limit fixture, U+0000 RUN4001, `viewerFocus.page` 상한. + +## S13이 재사용할 복구 경계 + +- `RunClaimStore.findStale`·`takeOver`(같은 attempt·fence 인수), `RunExecution` RECOVER/FINAL_CHECK(status 먼저 → `settle(StoredTerminal)`·`failByServer`·`expireDeadline`), + `hasBrainEvents`(미시작 vs 실행 소실). 실제 Brain에서 확인한 결과: + - coordinator 정지 중(같은 JVM `stop()`) Brain이 저장한 terminal → `start()` 뒤 같은 epoch로 회수, graph 1회. Cloud Run 프로세스 교체는 S14. + - Brain이 terminal 저장 전 SIGKILL → 재시작한 Brain record는 `executing`(terminal 없음, 재개·재실행 0). Server는 같은 요청 OBSERVE·status 수습 뒤 + `provider_error`(infrastructure, 실행 소실)로 한 번 보상한다(단언). **Brain record는 executing으로 남는다** — Brain lease 만료 sweep·`cleanup` 호출은 S13. +- `runtime_cleanup_at` 예약까지가 Server 범위이고 Brain `cleanup` 호출·retention은 S13이다. + +## 후속 메모 + +- testkit release·clock route가 잘못된 JSON body에 500을 돌려준다(테스트 도구 한정, r187 r2). +- supervisor child readiness 대기(60s)를 설정값으로 — 공용 호스트 부하에서 재시도에 의존하지 않게(Brain 후속). +- `deploy.yml`의 `PROOVY_AI_HOST`·`INTERNAL_API_TOKEN` secret 주입은 읽는 코드가 없다(S14 정리 후보, 파일 미변경). diff --git a/docs/contracts/internal-run-v1.md b/docs/contracts/internal-run-v1.md index 06c9a3d5..d9b20c44 100644 --- a/docs/contracts/internal-run-v1.md +++ b/docs/contracts/internal-run-v1.md @@ -101,7 +101,7 @@ proovy: - `id-token`: ADC(Cloud Run 서비스 계정)로 고정 audience의 Google ID token을 발급·캐시한다. base-url은 https만. Brain은 audience와 서비스 계정 email을 검증한다(`internal_api_auth_mode=oidc`). - `shared-token`: Brain `internal_api_shared_token`과 같은 값. prod profile에서는 기동이 실패한다. -- 기존 `internal.api.token`은 Server로 들어오는 내부 API용이고 이 client와 무관하다. +- 구형 agent → Server 내부 API token(`internal.api.token`, `X-Internal-Token`)은 1.87에서 제거됐다(이 client와 무관). - **미확인(S14)**: 실제 Cloud Run 서비스 계정의 ID token 발급, Brain의 audience·email 검증 통과, IAM invoker, ingress는 배포 환경에서 확인해야 한다. 이 태스크는 fake provider·fake Brain으로 동작만 검사했다. diff --git a/scripts/brain-integration/run-local.sh b/scripts/brain-integration/run-local.sh new file mode 100755 index 00000000..d0159e46 --- /dev/null +++ b/scripts/brain-integration/run-local.sh @@ -0,0 +1,187 @@ +#!/usr/bin/env bash +# Server ↔ 실제 Python Brain 로컬 연동 인수 (task 1.87). +# +# 격리 Postgres 컨테이너 → proovy-brain testkit supervisor → Gradle 연동 테스트 → 정리. +# Server CI는 proovy-brain(private)을 checkout할 수 없어 이 테스트를 skip한다(사용자 결정). 이 스크립트가 유일한 실행 경로다. +# +# 사용: +# BRAIN_DIR= scripts/brain-integration/run-local.sh [gradle --tests 패턴...] +# +# 환경변수 (기본값): +# BRAIN_DIR 필수. uv 환경이 준비된 proovy-brain checkout +# PG_CONTAINER (proovy-postgres-brain-it) PG_PORT (26432) — 고정 compose 자원(15432 등)과 겹치지 않게 고른다 +# SUPERVISOR_PORT (26081) BRAIN_API_PORT (26080) +# GRADLE_IN_DOCKER (auto) 1이면 eclipse-temurin:21-jdk 임시 컨테이너(--network host)에서 Gradle을 돌린다 +# KEEP_RESOURCES (0) 1이면 끝나도 컨테이너·supervisor를 남긴다(디버깅용) +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +: "${BRAIN_DIR:?BRAIN_DIR must point at a proovy-brain checkout}" +PG_CONTAINER="${PG_CONTAINER:-proovy-postgres-brain-it}" +PG_PORT="${PG_PORT:-26432}" +SUPERVISOR_PORT="${SUPERVISOR_PORT:-26081}" +BRAIN_API_PORT="${BRAIN_API_PORT:-26080}" +PG_USER=brain_it_admin +PG_PASSWORD=brain-it-admin-dev-only +CONTROL_TOKEN="brain-it-control-$(od -An -N8 -tx1 /dev/urandom | tr -d ' \n')" +SHARED_TOKEN="brain-it-shared-$(od -An -N8 -tx1 /dev/urandom | tr -d ' \n')" +RUNTIME_PASSWORD=brain-it-runtime-dev-only +MIGRATION_PASSWORD=brain-it-migration-dev-only +SUPERVISOR_PID="" +LOG_DIR="$(mktemp -d)" + +cleanup() { + if [[ "${KEEP_RESOURCES:-0}" == "1" ]]; then + echo "[brain-it] KEEP_RESOURCES=1 — leaving $PG_CONTAINER and supervisor pid ${SUPERVISOR_PID:-none}" + return + fi + if [[ -n "$SUPERVISOR_PID" ]]; then + # Brain API child는 supervisor가 새 session으로 띄운다 — 먼저 supervisor에게 멈추게 하고, supervisor 프로세스 그룹 전체를 정리한다 + curl -s -m 20 -X POST -H "X-Testkit-Control-Token: $CONTROL_TOKEN" -H "Content-Type: application/json" \ + -d '{"signal": "kill"}' "http://127.0.0.1:$SUPERVISOR_PORT/brain/stop" >/dev/null 2>&1 || true + kill -- "-$SUPERVISOR_PID" 2>/dev/null || kill "$SUPERVISOR_PID" 2>/dev/null || true + wait "$SUPERVISOR_PID" 2>/dev/null || true + fi + pkill -f "testkit.integration serve --host 127.0.0.1 --port $BRAIN_API_PORT" 2>/dev/null || true + docker rm -f "$PG_CONTAINER" >/dev/null 2>&1 || true + rm -rf "${LOG_DIR:?}" +} +trap cleanup EXIT + +GRADLE_IN_DOCKER="${GRADLE_IN_DOCKER:-auto}" +if [[ "$GRADLE_IN_DOCKER" == "auto" ]]; then + if command -v javac >/dev/null 2>&1; then GRADLE_IN_DOCKER=0; else GRADLE_IN_DOCKER=1; fi +fi + +# gradle — 호출 시점의 TEST_ENV를 전달한다 +gradle() { + local env_args=() + if [[ "$GRADLE_IN_DOCKER" == "1" ]]; then + for kv in "${TEST_ENV[@]}"; do env_args+=(-e "$kv"); done + flock /tmp/proovy-s07-gradle.lock docker run --rm --network host --memory 1536m --cpus 2 \ + -u "$(id -u):$(id -g)" -e HOME=/home/gradle -e GRADLE_USER_HOME=/home/gradle/.gradle \ + -v "$HOME/.gradle:/home/gradle/.gradle" -v "$ROOT:$ROOT" -w "$ROOT" "${env_args[@]}" \ + eclipse-temurin:21-jdk sh ./gradlew "$@" --no-daemon --max-workers=2 -Dorg.gradle.jvmargs=-Xmx768m + else + (cd "$ROOT" && env "${TEST_ENV[@]}" flock /tmp/proovy-s07-gradle.lock \ + ./gradlew "$@" --no-daemon --max-workers=2 -Dorg.gradle.jvmargs=-Xmx768m) + fi +} +TEST_ENV=() + +SERVER_HEAD="$(git -C "$ROOT" rev-parse HEAD)" +BRAIN_HEAD="$(git -C "$BRAIN_DIR" rev-parse HEAD)" +BRAIN_DIRTY="$(git -C "$BRAIN_DIR" status --porcelain | head -1)" +echo "[brain-it] server=$SERVER_HEAD brain=$BRAIN_HEAD${BRAIN_DIRTY:+ (brain checkout DIRTY)}" + +# 컴파일을 먼저 끝낸다 — Brain child 기동(supervisor readiness 60s)이 컴파일 메모리 피크와 겹치지 않게 +echo "[brain-it] compiling test classes" +gradle testClasses + +echo "[brain-it] postgres $PG_CONTAINER on 127.0.0.1:$PG_PORT (tmpfs)" +docker run -d --name "$PG_CONTAINER" --tmpfs /var/lib/postgresql/data:rw,size=600m --memory 384m \ + -e POSTGRES_USER="$PG_USER" -e POSTGRES_PASSWORD="$PG_PASSWORD" -e POSTGRES_DB=postgres \ + -p "127.0.0.1:$PG_PORT:5432" postgres:15-alpine >/dev/null +wait_for() { # wait_for — 60초 안에 성공하지 못하면 실패 + local what=$1 + shift + for _ in $(seq 1 60); do + "$@" >/dev/null 2>&1 && return 0 + sleep 1 + done + echo "[brain-it] $what did not become ready in 60s" >&2 + return 1 +} +wait_for postgres docker exec "$PG_CONTAINER" pg_isready -U "$PG_USER" -d postgres + +echo "[brain-it] brain testkit supervisor 127.0.0.1:$SUPERVISOR_PORT (api $BRAIN_API_PORT)" +export PG_USER PG_PASSWORD PG_PORT CONTROL_TOKEN SHARED_TOKEN RUNTIME_PASSWORD MIGRATION_PASSWORD \ + SUPERVISOR_PORT BRAIN_API_PORT +setsid bash -c ' + cd "$0" + env -i PATH="$PATH" HOME="$HOME" \ + TESTKIT_PG_ADMIN_URL="postgresql://$PG_USER:$PG_PASSWORD@127.0.0.1:$PG_PORT/postgres" \ + TESTKIT_CONTROL_TOKEN="$CONTROL_TOKEN" TESTKIT_SHARED_TOKEN="$SHARED_TOKEN" \ + TESTKIT_BRAIN_RUNTIME_PASSWORD="$RUNTIME_PASSWORD" TESTKIT_BRAIN_MIGRATION_PASSWORD="$MIGRATION_PASSWORD" \ + uv run --locked python -m testkit.integration supervise \ + --port "$SUPERVISOR_PORT" --api-port "$BRAIN_API_PORT" +' "$BRAIN_DIR" >"$LOG_DIR/supervisor.log" 2>&1 & +SUPERVISOR_PID=$! # setsid → 이 PID가 프로세스 그룹 ID다 +wait_for supervisor curl -sf -H "X-Testkit-Control-Token: $CONTROL_TOKEN" \ + "http://127.0.0.1:$SUPERVISOR_PORT/brain/status" + +# Brain child 기동은 supervisor의 고정 readiness 대기(60s) 안에 끝나야 한다. 부하가 큰 호스트에서도 넘지 않게 +# import·bytecode를 미리 데운다(같은 checkout, 같은 venv). +echo "[brain-it] warming brain imports" +(cd "$BRAIN_DIR" && env -i PATH="$PATH" HOME="$HOME" uv run --locked python -c \ + 'import proovy_brain.apps.api, proovy_brain.features.solve_execution.text_run, testkit.integration.app') + +# 테스트 전에 child를 한 번 띄워 둔다(빈 warm-up database). 테스트의 /brain/start는 이 child를 죽이고 새로 띄우지만 +# page cache가 데워져 있어 테스트 JVM과 겹쳐도 readiness 안에 뜬다. +docker exec "$PG_CONTAINER" psql -q -U "$PG_USER" -d postgres -c "CREATE DATABASE brain_it_warm" >/dev/null +curl -sf -X POST -H "X-Testkit-Control-Token: $CONTROL_TOKEN" -H "Content-Type: application/json" \ + -d '{"database": "brain_it_warm", "lease_ttl_seconds": 3}' \ + "http://127.0.0.1:$SUPERVISOR_PORT/brain/start" >/dev/null && echo "[brain-it] warm child ready" + +TESTS=("$@") +if [[ ${#TESTS[@]} -eq 0 ]]; then + TESTS=("com.proovy.integration.brain.*") +fi +# --rerun: 환경변수(testkit 주소·token)는 test task 입력이 아니라서, 이전 결과(예: testkit 없이 돈 18 skipped)가 남아 있으면 +# UP-TO-DATE로 테스트 0건이 "통과"한다. 항상 실제로 실행하게 강제한다. +GRADLE_ARGS=(test --rerun) +for pattern in "${TESTS[@]}"; do + GRADLE_ARGS+=(--tests "$pattern") +done + +TEST_ENV=( + "PROOVY_TEST_PG_URL=jdbc:postgresql://127.0.0.1:$PG_PORT/postgres" + "PROOVY_TEST_PG_USER=$PG_USER" "PROOVY_TEST_PG_PASSWORD=$PG_PASSWORD" "PROOVY_REQUIRE_DB_TESTS=true" + "PROOVY_BRAIN_TESTKIT_URL=http://127.0.0.1:$SUPERVISOR_PORT" + "PROOVY_BRAIN_TESTKIT_CONTROL_TOKEN=$CONTROL_TOKEN" "PROOVY_BRAIN_TESTKIT_SHARED_TOKEN=$SHARED_TOKEN" + "PROOVY_BRAIN_TESTKIT_RUNTIME_PASSWORD=$RUNTIME_PASSWORD" + "PROOVY_BRAIN_EXPECTED_COMMIT=$BRAIN_HEAD" "PROOVY_REQUIRE_BRAIN_IT=true" +) + +STARTED_AT="$(date +%s)" +set +e +gradle "${GRADLE_ARGS[@]}" +STATUS=$? +set -e + +# 결과 검증: 이번 실행이 만든 연동 테스트 결과가 있고, 실제로 돌았고(tests > 0), skip이 0이어야 한다 +verify_results() { + local found=0 file tests skipped failures errors + shopt -s nullglob + for file in "$ROOT"/build/test-results/test/TEST-com.proovy.integration.brain.*.xml; do + if [[ "$(stat -c %Y "$file")" -lt "$STARTED_AT" ]]; then + echo "[brain-it] stale result (not produced by this run): $file" >&2 + return 1 + fi + tests=$(grep -o 'tests="[0-9]*"' "$file" | head -1 | tr -dc '0-9') + skipped=$(grep -o 'skipped="[0-9]*"' "$file" | head -1 | tr -dc '0-9') + failures=$(grep -o 'failures="[0-9]*"' "$file" | head -1 | tr -dc '0-9') + errors=$(grep -o 'errors="[0-9]*"' "$file" | head -1 | tr -dc '0-9') + echo "[brain-it] $(basename "$file" .xml): tests=$tests skipped=$skipped failures=$failures errors=$errors" + if [[ "${tests:-0}" -eq 0 || "${skipped:-1}" -ne 0 || "${failures:-1}" -ne 0 || "${errors:-1}" -ne 0 ]]; then + return 1 + fi + found=$((found + 1)) + done + if [[ $found -eq 0 ]]; then + echo "[brain-it] no real-Brain integration result was produced" >&2 + return 1 + fi +} +if [[ $STATUS -eq 0 ]] && ! verify_results; then + echo "[brain-it] FAILED: the real-Brain integration did not actually run green (skip or zero tests)" >&2 + STATUS=1 +fi + +echo "[brain-it] server=$SERVER_HEAD brain=$BRAIN_HEAD gradle exit=$STATUS" +echo "[brain-it] reports: $ROOT/build/reports/tests/test/index.html" +if [[ $STATUS -ne 0 ]]; then + echo "[brain-it] supervisor log tail:" + tail -n 40 "$LOG_DIR/supervisor.log" || true +fi +exit $STATUS diff --git a/src/main/java/com/proovy/global/security/InternalApiTokenAuthenticationFilter.java b/src/main/java/com/proovy/global/security/InternalApiTokenAuthenticationFilter.java deleted file mode 100644 index dac40fae..00000000 --- a/src/main/java/com/proovy/global/security/InternalApiTokenAuthenticationFilter.java +++ /dev/null @@ -1,69 +0,0 @@ -package com.proovy.global.security; - -import com.fasterxml.jackson.databind.ObjectMapper; -import com.proovy.global.response.ApiResponse; -import jakarta.servlet.FilterChain; -import jakarta.servlet.ServletException; -import jakarta.servlet.http.HttpServletRequest; -import jakarta.servlet.http.HttpServletResponse; -import lombok.RequiredArgsConstructor; -import org.springframework.beans.factory.annotation.Value; -import org.springframework.http.MediaType; -import org.springframework.security.authentication.UsernamePasswordAuthenticationToken; -import org.springframework.security.core.authority.SimpleGrantedAuthority; -import org.springframework.security.core.context.SecurityContextHolder; -import org.springframework.stereotype.Component; -import org.springframework.web.filter.OncePerRequestFilter; - -import java.io.IOException; -import java.util.List; - -@Component -@RequiredArgsConstructor -public class InternalApiTokenAuthenticationFilter extends OncePerRequestFilter { - - private static final String INTERNAL_PATH = "/internal"; - private static final String INTERNAL_PATH_PREFIX = "/internal/"; - private static final String INTERNAL_TOKEN_HEADER = "X-Internal-Token"; - - private final ObjectMapper objectMapper; - - @Value("${internal.api.token}") - private String internalApiToken; - - @Override - protected boolean shouldNotFilter(HttpServletRequest request) { - String path = request.getServletPath(); - if (path == null) { - return true; - } - return !(INTERNAL_PATH.equals(path) || path.startsWith(INTERNAL_PATH_PREFIX)); - } - - @Override - protected void doFilterInternal( - HttpServletRequest request, - HttpServletResponse response, - FilterChain filterChain - ) throws ServletException, IOException { - String requestToken = request.getHeader(INTERNAL_TOKEN_HEADER); - if (requestToken == null || !internalApiToken.equals(requestToken)) { - SecurityContextHolder.clearContext(); - response.setStatus(HttpServletResponse.SC_FORBIDDEN); - response.setContentType(MediaType.APPLICATION_JSON_VALUE); - response.setCharacterEncoding("UTF-8"); - ApiResponse errorResponse = ApiResponse.failure("INTERNAL4031", "유효하지 않은 내부 API 토큰입니다."); - response.getWriter().write(objectMapper.writeValueAsString(errorResponse)); - return; - } - - UsernamePasswordAuthenticationToken authentication = - new UsernamePasswordAuthenticationToken( - "internal-api", - null, - List.of(new SimpleGrantedAuthority("ROLE_INTERNAL")) - ); - SecurityContextHolder.getContext().setAuthentication(authentication); - filterChain.doFilter(request, response); - } -} diff --git a/src/main/java/com/proovy/global/security/JwtAuthenticationFilter.java b/src/main/java/com/proovy/global/security/JwtAuthenticationFilter.java index 592f8ddc..ae98022b 100644 --- a/src/main/java/com/proovy/global/security/JwtAuthenticationFilter.java +++ b/src/main/java/com/proovy/global/security/JwtAuthenticationFilter.java @@ -35,19 +35,8 @@ public class JwtAuthenticationFilter extends OncePerRequestFilter { private static final String AUTHORIZATION_HEADER = "Authorization"; private static final String BEARER_PREFIX = "Bearer "; - private static final String INTERNAL_PATH = "/internal"; - private static final String INTERNAL_PATH_PREFIX = "/internal/"; public static final String JWT_ERROR_ATTRIBUTE = "jwtError"; - @Override - protected boolean shouldNotFilter(HttpServletRequest request) { - String path = request.getServletPath(); - if (path == null) { - return false; - } - return INTERNAL_PATH.equals(path) || path.startsWith(INTERNAL_PATH_PREFIX); - } - @Override protected void doFilterInternal( HttpServletRequest request, diff --git a/src/main/java/com/proovy/global/security/SecurityConfig.java b/src/main/java/com/proovy/global/security/SecurityConfig.java index 1c3ba9af..848f6085 100644 --- a/src/main/java/com/proovy/global/security/SecurityConfig.java +++ b/src/main/java/com/proovy/global/security/SecurityConfig.java @@ -31,7 +31,6 @@ public class SecurityConfig { private final JwtAuthenticationFilter jwtAuthFilter; private final JwtAuthenticationEntryPoint jwtAuthEntryPoint; - private final InternalApiTokenAuthenticationFilter internalApiTokenAuthenticationFilter; @Bean public SecurityFilterChain filterChain(HttpSecurity http) throws Exception { @@ -47,7 +46,6 @@ public SecurityFilterChain filterChain(HttpSecurity http) throws Exception { .requestMatchers(HttpMethod.OPTIONS, "/**").permitAll() .requestMatchers("/api/auth/logout").authenticated() .requestMatchers("/api/auth/**").permitAll() - .requestMatchers("/internal/**").authenticated() .requestMatchers("/swagger-ui.html", "/swagger-ui/**", "/api-docs/**", "/v3/api-docs/**").permitAll() .requestMatchers("/health", "/actuator/health").permitAll() // 에러 디스패치(/error) 시에는 인증 필터가 개입하지 않도록 허용 @@ -59,7 +57,6 @@ public SecurityFilterChain filterChain(HttpSecurity http) throws Exception { .anyRequest().authenticated() ) .addFilterBefore(jwtAuthFilter, UsernamePasswordAuthenticationFilter.class) - .addFilterBefore(internalApiTokenAuthenticationFilter, JwtAuthenticationFilter.class) .build(); } diff --git a/src/main/resources/application.yaml b/src/main/resources/application.yaml index e089080b..4cca9423 100644 --- a/src/main/resources/application.yaml +++ b/src/main/resources/application.yaml @@ -130,11 +130,6 @@ proovy: instances: ${BRAIN_MAX_INSTANCES:3} per-instance: ${BRAIN_MAX_CONCURRENT_RUNS:4} -internal: - api: - token: ${INTERNAL_API_TOKEN} - port: 8081 - # =============================== # OpenRouter API (Gemini 2.5 Flash 경유) # =============================== @@ -161,10 +156,6 @@ spring: hibernate: format_sql: true -internal: - api: - token: ${INTERNAL_API_TOKEN:local-dev-token} - --- # =============================== # Production 환경 diff --git a/src/test/java/com/proovy/domain/run/coordinator/RunPathEndToEndPostgresTest.java b/src/test/java/com/proovy/domain/run/coordinator/RunPathEndToEndPostgresTest.java new file mode 100644 index 00000000..5655a71c --- /dev/null +++ b/src/test/java/com/proovy/domain/run/coordinator/RunPathEndToEndPostgresTest.java @@ -0,0 +1,212 @@ +package com.proovy.domain.run.coordinator; + +import static org.assertj.core.api.Assertions.assertThat; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.node.JsonNodeFactory; +import com.fasterxml.jackson.databind.node.ObjectNode; +import com.proovy.domain.auth.service.AccessTokenBlacklistService; +import com.proovy.domain.auth.service.JwtTokenProvider; +import com.proovy.support.db.PostgresTestDatabase; +import com.proovy.support.http.BrowserRunHttp; +import com.proovy.support.http.SseClient; +import com.proovy.support.http.SseClient.Block; +import java.time.Duration; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.UUID; +import java.util.stream.Collectors; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Tag; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.web.server.LocalServerPort; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.ActiveProfiles; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; +import org.springframework.test.context.bean.override.mockito.MockitoBean; +import org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping; + +/** + * 빈 제품 DB에서 실제 Java 경로 전체를 한 번 완주한다(task 1.87 단계 3): 실제 HTTP POST /api/conversations(JWT) → 접수 + * transaction → coordinator claim → 실제 {@code BrainRunClient} → loopback {@link FakeBrain} → terminal transaction + * → POST SSE 종료 → GET replay. Brain만 가짜이고 나머지는 production bean이다. 같은 경로를 실제 Python Brain으로 도는 + * 검사는 {@code com.proovy.integration.brain}(단계 4)에 있다. + */ +@Tag("postgres") +@ActiveProfiles("test") +@DirtiesContext +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, properties = { + "gcs.project-id=proovy-test", + "gcs.endpoint=http://127.0.0.1:1", + "jwt.secret=test-jwt-secret-test-jwt-secret-0123456789", + "oauth.kakao.client-id=test", "oauth.kakao.client-secret=test", + "oauth.naver.client-id=test", "oauth.naver.client-secret=test", + "openrouter.api.key=test", + "spring.data.redis.host=127.0.0.1", "spring.data.redis.port=1", + "spring.task.scheduling.pool.size=1", + "proovy.subscription.plan-transition-fixed-delay-ms=3600000", + "proovy.subscription.auto-renew-fixed-delay-ms=3600000", + "proovy.brain.auth.mode=shared-token", + "proovy.brain.auth.shared-token=t187-path-token", + "proovy.brain.stream-idle-timeout=5s", + "proovy.run.coordinator.enabled=true", + "proovy.run.coordinator.instance-id=t187-path", + "proovy.run.coordinator.poll-interval=100ms", + "proovy.run.coordinator.heartbeat-interval=300ms", + "proovy.run.coordinator.lease-ttl=2s", + "proovy.run.coordinator.min-dispatch-remaining=0s", + "proovy.run.sse.heartbeat=500ms" +}) +class RunPathEndToEndPostgresTest { + + private static final Duration WAIT = Duration.ofSeconds(20); + private static final long DAILY_CREDITS = 100; + + private static PostgresTestDatabase db; + private static FakeBrain brain; + + @BeforeAll + static void setUp() { + // 빈 baseline(V1~)만 — seed 없이 시작한다 + db = PostgresTestDatabase.createMigrated(); + brain = new FakeBrain(); + } + + @DynamicPropertySource + static void properties(DynamicPropertyRegistry registry) { + registry.add("spring.datasource.url", () -> db.jdbcUrl()); + registry.add("spring.datasource.username", PostgresTestDatabase::adminUser); + registry.add("spring.datasource.password", PostgresTestDatabase::adminPassword); + registry.add("proovy.brain.base-url", () -> brain.baseUrl()); + } + + @AfterAll + static void tearDown() { + if (brain != null) { + brain.close(); + } + if (db != null) { + db.close(); + } + } + + @LocalServerPort int port; + @MockitoBean AccessTokenBlacklistService blacklist; + @Autowired JwtTokenProvider tokens; + @Autowired JdbcTemplate jdbc; + @Autowired @Qualifier("requestMappingHandlerMapping") RequestMappingHandlerMapping mappings; + + private BrowserRunHttp browser; + + @BeforeEach + void client() { + browser = new BrowserRunHttp(port, WAIT); + } + + @Test + void firstTextOnAnEmptyBaselineRunsTheWholeJavaPathAndReplaysIdentically() throws Exception { + long user = newUser(); + String token = tokens.generateTokens(user).accessToken(); + String requestId = UUID.randomUUID().toString(); + assertThat(count("SELECT count(*) FROM notes WHERE user_id = ?", user)).isZero(); + + List live; + UUID runId; + long noteId; + try (SseClient post = browser.post(token, body(requestId))) { + assertThat(post.status).as(post.errorBody).isEqualTo(200); + live = new ArrayList<>(post.framesUntilEnd()); + } + Block created = live.get(0); + assertThat(created.event()).isEqualTo("run.created"); + runId = UUID.fromString(created.json().get("runId").asText()); + noteId = created.json().at("/payload/noteId").asLong(); + assertThat(live.get(live.size() - 1).event()).isEqualTo("run.completed"); + assertThat(live.stream().map(Block::seq).toList()) + .isEqualTo(java.util.stream.LongStream.rangeClosed(1, live.size()).boxed().toList()); + + // 브라우저와 독립적인 coordinator가 snapshot 그대로 Brain을 한 번 불렀다 + List sent = brain.deliveries(requestId); + assertThat(sent).hasSize(1); + assertThat(sent.get(0).runId()).isEqualTo(runId.toString()); + assertThat(sent.get(0).attempt()).isEqualTo(1); + + // 완료 뒤 GET replay는 같은 바이트 + try (SseClient replay = browser.events(token, runId, null)) { + assertThat(replay.framesUntilEnd().stream().map(Block::raw).toList()) + .containsExactlyElementsOf(live.stream().map(Block::raw).toList()); + } + + // 하나의 자동 생성 Note·run·메시지 2개, clarification은 보상되어 순사용량 0 + assertThat(count("SELECT count(*) FROM notes WHERE user_id = ?", user)).isOne(); + assertThat(count("SELECT count(*) FROM ai_runs WHERE note_id = ?", noteId)).isOne(); + assertThat(count("SELECT count(*) FROM chat_messages WHERE note_id = ?", noteId)).isEqualTo(2); + assertThat(jdbc.queryForObject("SELECT status FROM ai_runs WHERE run_id = ?", String.class, runId)) + .isEqualTo("COMPLETED"); + assertThat(count("SELECT count(*) FROM request_usage_charges WHERE request_id = ?", requestId)).isOne(); + assertThat(count("SELECT count(*) FROM request_usage_compensations WHERE request_id = ?", requestId)) + .isOne(); + assertThat(jdbc.queryForObject("SELECT daily_free_credit FROM credit_balance WHERE user_id = ?", + Long.class, user)).isEqualTo(DAILY_CREDITS); + + JsonNode page = browser.conversation(token, noteId).body(); + assertThat(page.at("/result/messages")).hasSize(2); + } + + @Test + void baselineHasNoLegacyConversationTablesOrColumns() { + // ChatSession·thread ID는 신규 baseline에 없다 + assertThat(jdbc.queryForList(""" + SELECT table_name || '.' || column_name FROM information_schema.columns + WHERE table_schema = 'public' + AND (table_name ILIKE '%session%' OR column_name ILIKE '%session%' OR column_name ILIKE '%thread%') + """, String.class)).isEmpty(); + } + + @Test + void onlyTheNewRunRoutesExistForExecutionAndUsage() { + Map routes = mappings.getHandlerMethods().keySet().stream() + .flatMap(info -> info.getPatternValues().stream() + .map(pattern -> Map.entry(pattern, info.getMethodsCondition().toString()))) + .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue, (a, b) -> a + b)); + assertThat(routes.keySet()).noneMatch(path -> path.startsWith("/internal")) + .noneMatch(path -> path.contains("/stream")) + .noneMatch(path -> path.startsWith("/api/credits/") && !path.equals("/api/credits/balance") + && !path.equals("/api/credits/history")); + assertThat(routes.keySet()).contains("/api/conversations", "/api/ai-runs/{runId}/events", + "/api/ai-runs/{runId}", "/api/ai-runs/{runId}/cancel", "/api/conversations/{noteId}/messages"); + } + + private static String body(String requestId) { + ObjectNode body = JsonNodeFactory.instance.objectNode(); + body.put("requestId", requestId); + body.put("text", "x^2-5x+6=0을 풀어줘"); + return body.toString(); + } + + private long newUser() { + String key = UUID.randomUUID().toString(); + long user = jdbc.queryForObject(""" + INSERT INTO users (nickname, provider, provider_user_id, email, created_at, updated_at) + VALUES ('t187', 'KAKAO', ?, ?, now(), now()) RETURNING user_id + """, Long.class, key, key + "@example.test"); + jdbc.update(""" + INSERT INTO credit_balance (user_id, daily_free_credit, daily_free_limit, daily_expires_at, free_credit, + paid_credit, created_at, updated_at) + VALUES (?, ?, ?, now() + interval '1 day', 0, 0, now(), now()) + """, user, DAILY_CREDITS, DAILY_CREDITS); + return user; + } + + private long count(String sql, Object... args) { + return jdbc.queryForObject(sql, Long.class, args); + } +} diff --git a/src/test/java/com/proovy/global/LegacyExecutionPathGuardTest.java b/src/test/java/com/proovy/global/LegacyExecutionPathGuardTest.java new file mode 100644 index 00000000..61b322b2 --- /dev/null +++ b/src/test/java/com/proovy/global/LegacyExecutionPathGuardTest.java @@ -0,0 +1,126 @@ +package com.proovy.global; + +import static org.assertj.core.api.Assertions.assertThat; + +import com.proovy.domain.credit.entity.CreditEventType; +import java.io.IOException; +import java.io.UncheckedIOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.Map; +import java.util.regex.Pattern; +import java.util.stream.Stream; +import org.junit.jupiter.api.Test; + +/** + * 신규 실행 경로에 구형 실행·과금 경로가 되살아나지 않게 막는 회귀 검사(task 1.87, spec 07). + * + *

ChatSession·thread ID·구형 stream/client·토큰 누적·기능별 사후 차감·브라우저 임의 차감·구형 내부 API token 진입점이 + * {@code src/main}에 다시 나타나면 실패한다. 사용량 차감은 {@code RequestUsageService}의 request당 한 번 차감·보상뿐이다. + */ +class LegacyExecutionPathGuardTest { + + private static final Path MAIN = Path.of("src/main"); + private static final Pattern COMMENT_LINE = Pattern.compile("^\\s*(--|//|\\*|/\\*)"); + + /** 금지 패턴 → 이유. 대소문자 무시. */ + private static final Map BANNED = Map.ofEntries( + Map.entry("chat_?session", "ChatSession FK·호환 adapter는 신규 오픈에서 제거"), + Map.entry("thread_?id", "Brain thread ID 대신 run_id·attempt·fence를 쓴다"), + Map.entry("/stream/v2", "구형 AI stream 경로"), + Map.entry("proovy[._]ai[._]host", "구형 proovy-agent client 설정"), + Map.entry("ProovyAiRequest|ProovyAiClient|StreamEvent\\b", "구형 AI client·stream DTO"), + Map.entry("X-Internal-Token|internal\\.api\\.token", "구형 agent → Server 내부 API token 진입점"), + Map.entry("credits/use|credits/costs|credits/estimate", "브라우저 임의 차감·기능별 비용표"), + Map.entry("totalTokens|promptTokens|completionTokens|total_tokens|prompt_tokens|completion_tokens", + "토큰 누적 과금")); + + @Test + void mainSourcesCarryNoLegacyExecutionOrBillingPath() { + List violations = new ArrayList<>(); + for (Path file : mainFiles()) { + // 제거 사실을 적은 주석(SQL --, Java //·javadoc)은 검사하지 않는다 + String text = read(file).lines() + .filter(line -> !COMMENT_LINE.matcher(line).find()) + .collect(java.util.stream.Collectors.joining("\n")); + BANNED.forEach((regex, reason) -> { + if (Pattern.compile(regex, Pattern.CASE_INSENSITIVE).matcher(text).find()) { + violations.add(file + " ~ /" + regex + "/ (" + reason + ")"); + } + }); + } + assertThat(violations).isEmpty(); + } + + @Test + void securityHasNoInternalPathCarveOut() { + // 구형 agent → Server 내부 API는 없다. /internal 경로를 인증에서 빼는 예외가 남으면 나중에 생긴 endpoint의 JWT가 조용히 무시된다 + // (Server → Brain client의 /internal/v1/runs 호출 경로는 security 밖이라 대상이 아니다) + List violations = files(MAIN.resolve("java/com/proovy/global/security")).stream() + .filter(file -> read(file).contains("\"/internal")) + .map(Path::toString) + .toList(); + assertThat(violations).isEmpty(); + } + + @Test + void onlyRequestUsageDeductsBalance() { + List callers = mainFiles().stream() + .filter(file -> file.toString().endsWith(".java")) + .filter(file -> read(file).contains("deductCredit(")) + .map(file -> file.getFileName().toString()) + .sorted() + .toList(); + // 엔티티 정의와 request당 한 번 차감하는 RequestUsageService만 — 기능별 사후 차감 경로가 없다 + assertThat(callers).containsExactly("CreditBalance.java", "RequestUsageService.java"); + } + + @Test + void creditEventsAreGrantsPlusOneRequestChargeAndItsCompensation() { + assertThat(Arrays.stream(CreditEventType.values()).map(Enum::name)) + .containsExactlyInAnyOrder("DAILY_RESET", "MONTHLY_GRANT", "MONTHLY_EXPIRE", "SIGNUP_BONUS", + "LLM_QUERY", "LLM_QUERY_COMPENSATION"); + } + + @Test + void brainCallPathNeverReadsTheBrowserIdentity() { + // Brain 호출은 서비스 credential만 쓴다 — 사용자 JWT·요청 header·SecurityContext를 읽는 코드가 없다 + List violations = new ArrayList<>(); + for (String dir : List.of("java/com/proovy/global/infra/brain", "java/com/proovy/domain/run/coordinator", + "java/com/proovy/domain/run/brain")) { + for (Path file : files(MAIN.resolve(dir))) { + String text = read(file); + for (String banned : List.of("SecurityContextHolder", "UserPrincipal", "HttpServletRequest", + "JwtTokenProvider", "\"Authorization\"")) { + if (text.contains(banned)) { + violations.add(file + " ~ " + banned); + } + } + } + } + assertThat(violations).isEmpty(); + } + + private static List mainFiles() { + return files(MAIN); + } + + private static List files(Path root) { + try (Stream walk = Files.walk(root)) { + return walk.filter(Files::isRegularFile).sorted().toList(); + } catch (IOException e) { + throw new UncheckedIOException(e); + } + } + + private static String read(Path file) { + try { + return Files.readString(file); + } catch (IOException e) { + throw new UncheckedIOException(e); + } + } +} diff --git a/src/test/java/com/proovy/integration/brain/BrainTestkit.java b/src/test/java/com/proovy/integration/brain/BrainTestkit.java new file mode 100644 index 00000000..e1ef605a --- /dev/null +++ b/src/test/java/com/proovy/integration/brain/BrainTestkit.java @@ -0,0 +1,218 @@ +package com.proovy.integration.brain; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.node.ObjectNode; +import java.net.URI; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.time.Duration; +import java.util.ArrayList; +import java.util.List; +import org.junit.jupiter.api.Assumptions; + +/** + * proovy-brain {@code testkit/}(PR #150, squash 7b8608e)의 supervisor·control API client. Server 연동 테스트가 실제 Python + * 내부 API child를 띄우고(같은 제품 DB에 Brain schema bootstrap), request별 scenario·fault를 걸고, 받은 호출·실행 수를 읽는다. + * + *

환경변수(모두 있어야 실행한다): + *

    + *
  • {@code PROOVY_BRAIN_TESTKIT_URL} — supervisor, 예 {@code http://127.0.0.1:26081}
    • + *
  • {@code PROOVY_BRAIN_TESTKIT_CONTROL_TOKEN}, {@code PROOVY_BRAIN_TESTKIT_SHARED_TOKEN}
    • + *
  • {@code PROOVY_BRAIN_TESTKIT_RUNTIME_PASSWORD} — Brain {@code brain_runtime} role(역할 격리 검사용)
    • + *
  • {@code PROOVY_REQUIRE_BRAIN_IT=true} — 미구성 시 skip이 아니라 실패
    • + *
+ * proovy-brain이 private이라 Server CI는 Brain을 checkout할 수 없다(사용자 결정: CI skip, 로컬 스크립트 + * {@code scripts/brain-integration/run-local.sh}로 실행). 미구성 skip은 "NOT EXECUTED"로 로그에 남아 통과로 보이지 않는다. + */ +final class BrainTestkit { + + static final String SKIP_REASON = "NOT EXECUTED — real Brain integration (task 1.87) needs a running proovy-brain " + + "testkit (PROOVY_BRAIN_TESTKIT_URL/_CONTROL_TOKEN/_SHARED_TOKEN/_RUNTIME_PASSWORD). proovy-brain is private, " + + "so Server CI skips it by decision; run scripts/brain-integration/run-local.sh"; + + private static final ObjectMapper JSON = new ObjectMapper(); + private static final String CONTROL_HEADER = "X-Testkit-Control-Token"; + + private final HttpClient http = HttpClient.newBuilder().version(HttpClient.Version.HTTP_1_1) + .connectTimeout(Duration.ofSeconds(5)).build(); + private final String supervisorUrl; + private final String controlToken; + private final String sharedToken; + private final String runtimePassword; + private String apiUrl; + + private BrainTestkit(String supervisorUrl, String controlToken, String sharedToken, String runtimePassword) { + this.supervisorUrl = supervisorUrl; + this.controlToken = controlToken; + this.sharedToken = sharedToken; + this.runtimePassword = runtimePassword; + } + + /** 환경이 없으면 skip(이유 명시) 또는 실패(PROOVY_REQUIRE_BRAIN_IT=true). */ + static BrainTestkit fromEnvironment() { + String url = System.getenv("PROOVY_BRAIN_TESTKIT_URL"); + String control = System.getenv("PROOVY_BRAIN_TESTKIT_CONTROL_TOKEN"); + String shared = System.getenv("PROOVY_BRAIN_TESTKIT_SHARED_TOKEN"); + String runtime = System.getenv("PROOVY_BRAIN_TESTKIT_RUNTIME_PASSWORD"); + boolean configured = url != null && control != null && shared != null && runtime != null; + if (!configured && Boolean.parseBoolean(System.getenv("PROOVY_REQUIRE_BRAIN_IT"))) { + throw new IllegalStateException("PROOVY_REQUIRE_BRAIN_IT=true but the Brain testkit is not configured"); + } + if (!configured) { + System.err.println("[brain-integration] " + SKIP_REASON); + } + Assumptions.assumeTrue(configured, SKIP_REASON); + return new BrainTestkit(url, control, shared, runtime); + } + + String sharedToken() { + return sharedToken; + } + + String runtimePassword() { + return runtimePassword; + } + + String apiUrl() { + return apiUrl; + } + + // -- supervisor ---------------------------------------------------------- + + /** 그 database에 Brain schema를 bootstrap·migrate하고 API child를 띄운다. */ + JsonNode start(String database, int leaseTtlSeconds) { + JsonNode status = lifecycle("/brain/start", + JSON.createObjectNode().put("database", database).put("lease_ttl_seconds", leaseTtlSeconds)); + apiUrl = status.get("api_url").asText(); + return status; + } + + JsonNode restart(String signal) { + return lifecycle("/brain/restart", JSON.createObjectNode().put("signal", signal)); + } + + /** + * start·restart. supervisor는 child readiness를 고정 60초만 기다리고 실패하면 child를 죽인다 — 부하가 큰 공용 호스트에서 + * 그 시간을 넘긴 경우만 같은 요청을 다시 보낸다(최대 3번). 다른 실패는 바로 던진다. + */ + private JsonNode lifecycle(String path, JsonNode body) { + IllegalStateException last = null; + for (int attempt = 1; attempt <= 3; attempt++) { + try { + return send("POST", supervisorUrl + path, body, 200); + } catch (IllegalStateException e) { + if (!String.valueOf(e.getMessage()).contains("did not become ready")) { + throw e; + } + System.err.println("[brain-integration] " + path + " readiness timeout, retry " + attempt); + last = e; + } + } + throw last; + } + + JsonNode stop() { + return send("POST", supervisorUrl + "/brain/stop", JSON.createObjectNode().put("signal", "kill"), 200); + } + + JsonNode status() { + return send("GET", supervisorUrl + "/brain/status", null, 200); + } + + // -- child control ------------------------------------------------------ + + JsonNode info() { + return send("GET", apiUrl + "/__testkit__/v1/info", null, 200); + } + + void reset() { + send("POST", apiUrl + "/__testkit__/v1/reset", null, 200); + } + + void plan(String requestId, String scenario, ObjectNode faults) { + ObjectNode body = JSON.createObjectNode().put("scenario", scenario); + body.set("faults", faults == null ? JSON.createObjectNode() : faults); + send("PUT", apiUrl + "/__testkit__/v1/plans/" + requestId, body, 200); + } + + void release(String requestId, String point) { + ObjectNode body = point == null ? null : JSON.createObjectNode().put("point", point); + send("POST", apiUrl + "/__testkit__/v1/plans/" + requestId + "/release", body, 200); + } + + void advanceClock(double seconds) { + send("POST", apiUrl + "/__testkit__/v1/clock", JSON.createObjectNode().put("advance_seconds", seconds), 200); + } + + List observations(String requestId) { + return list(send("GET", apiUrl + "/__testkit__/v1/observations?request_id=" + requestId, null, 200)); + } + + List executions(String requestId) { + return list(send("GET", apiUrl + "/__testkit__/v1/executions?request_id=" + requestId, null, 200)); + } + + static ObjectNode faults() { + return JSON.createObjectNode(); + } + + // -- raw internal API (서비스 인증 거부 검사) ------------------------------------ + + int internalStatus(String runId, String authorization) { + HttpRequest.Builder request = HttpRequest.newBuilder( + URI.create(apiUrl + "/internal/v1/runs/" + runId + "?attempt=1&fence_token=probe")) + .timeout(Duration.ofSeconds(10)).GET(); + if (authorization != null) { + request.header("Authorization", authorization); + } + try { + return http.send(request.build(), HttpResponse.BodyHandlers.discarding()).statusCode(); + } catch (Exception e) { + throw new IllegalStateException("internal API probe failed", e); + } + } + + JsonNode internalRunStatus(String runId, int attempt, String fence) { + return send("GET", apiUrl + "/internal/v1/runs/" + runId + "?attempt=" + attempt + "&fence_token=" + fence, + null, 200, "Bearer " + sharedToken); + } + + private JsonNode send(String method, String url, JsonNode body, int expected) { + return send(method, url, body, expected, null); + } + + private JsonNode send(String method, String url, JsonNode body, int expected, String authorization) { + HttpRequest.Builder request = HttpRequest.newBuilder(URI.create(url)).timeout(Duration.ofSeconds(120)); + if (authorization != null) { + request.header("Authorization", authorization); + } else { + request.header(CONTROL_HEADER, controlToken); + } + HttpRequest.BodyPublisher publisher = body == null ? HttpRequest.BodyPublishers.noBody() + : HttpRequest.BodyPublishers.ofString(body.toString()); + if (body != null) { + request.header("Content-Type", "application/json"); + } + request.method(method, publisher); + try { + HttpResponse response = http.send(request.build(), HttpResponse.BodyHandlers.ofString()); + if (response.statusCode() != expected) { + throw new IllegalStateException(method + " " + url + " -> " + response.statusCode() + " " + + response.body()); + } + return response.body().isBlank() ? JSON.nullNode() : JSON.readTree(response.body()); + } catch (IllegalStateException e) { + throw e; + } catch (Exception e) { + throw new IllegalStateException(method + " " + url + " failed", e); + } + } + + private static List list(JsonNode array) { + List items = new ArrayList<>(); + array.forEach(items::add); + return items; + } +} diff --git a/src/test/java/com/proovy/integration/brain/ServerBrainIntegrationTest.java b/src/test/java/com/proovy/integration/brain/ServerBrainIntegrationTest.java new file mode 100644 index 00000000..ad862cdf --- /dev/null +++ b/src/test/java/com/proovy/integration/brain/ServerBrainIntegrationTest.java @@ -0,0 +1,980 @@ +package com.proovy.integration.brain; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.node.ArrayNode; +import com.fasterxml.jackson.databind.node.JsonNodeFactory; +import com.fasterxml.jackson.databind.node.ObjectNode; +import com.proovy.domain.auth.service.AccessTokenBlacklistService; +import com.proovy.domain.auth.service.JwtTokenProvider; +import com.proovy.domain.run.coordinator.RunCoordinator; +import com.proovy.support.db.PostgresTestDatabase; +import com.proovy.support.http.BrowserRunHttp; +import com.proovy.support.http.SseClient; +import com.proovy.support.http.SseClient.Block; +import java.nio.file.Files; +import java.nio.file.Path; +import java.sql.Connection; +import java.sql.SQLException; +import java.sql.Statement; +import java.time.Duration; +import java.util.ArrayList; +import java.util.HashSet; +import java.util.Iterator; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.UUID; +import java.util.concurrent.Callable; +import java.util.concurrent.CyclicBarrier; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.Future; +import java.util.function.BooleanSupplier; +import java.util.stream.LongStream; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Tag; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.web.server.LocalServerPort; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.ActiveProfiles; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; +import org.springframework.test.context.bean.override.mockito.MockitoBean; + +/** + * Server ↔ 실제 Python Brain 로컬 연동 인수 (task 1.87 단계 4, spec 07 Verification 1~5). + * + *

실제 Postgres 하나에 빈 제품 baseline(proovy_server가 Flyway 실행)과 Brain schema(brain_migration·brain_runtime, + * Brain {@code db bootstrap}·{@code migrate})를 같이 두고 운영 role 경계 스크립트를 적용한다. Server는 production bean 그대로 + * (실제 HTTP 브라우저 API·접수·coordinator·{@code BrainRunClient}·terminal transaction·SSE)이고 Brain은 proovy-brain + * {@code testkit}이 띄운 실제 내부 API·run driver·Postgres runtime·그래프다. 모델·Daytona는 offline fake seat이고 + * {@code no_match_notice}·{@code out_of_scope_source}·{@code rewrite_terminal}은 testkit의 scripted 결과다 — 수학 품질은 인증하지 않는다. + * + *

Brain testkit이 없으면 {@link BrainTestkit#SKIP_REASON}으로 skip한다(Server CI는 사용자 결정으로 skip). + */ +@Tag("postgres") +@Tag("brain-integration") +@ActiveProfiles("test") +@DirtiesContext +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, properties = { + "gcs.project-id=proovy-test", + "gcs.endpoint=http://127.0.0.1:1", + "jwt.secret=test-jwt-secret-test-jwt-secret-0123456789", + "oauth.kakao.client-id=test", "oauth.kakao.client-secret=test", + "oauth.naver.client-id=test", "oauth.naver.client-secret=test", + "openrouter.api.key=test", + "spring.data.redis.host=127.0.0.1", "spring.data.redis.port=1", + "spring.task.scheduling.pool.size=1", + "proovy.subscription.plan-transition-fixed-delay-ms=3600000", + "proovy.subscription.auto-renew-fixed-delay-ms=3600000", + "proovy.run.admission.rate-limit.max-requests=10000", + "proovy.run.snapshot.request-timeout=40s", + "proovy.run.sse.heartbeat=1s", + "proovy.brain.auth.mode=shared-token", + // Brain keep-alive(10s)보다 길게 — 정상 hold는 끊지 않는다 + "proovy.brain.stream-idle-timeout=15s", + "proovy.brain.request-timeout=10s", + "proovy.run.coordinator.enabled=true", + "proovy.run.coordinator.instance-id=t187-it", + "proovy.run.coordinator.max-concurrent-runs=3", + "proovy.run.coordinator.poll-interval=100ms", + "proovy.run.coordinator.heartbeat-interval=300ms", + "proovy.run.coordinator.lease-ttl=2s", + "proovy.run.coordinator.deadline-sweep-interval=200ms", + "proovy.run.coordinator.min-dispatch-remaining=0s", + "proovy.run.coordinator.redelivery.max-resends=4", + "proovy.run.coordinator.redelivery.max-status-checks=8", + "proovy.run.coordinator.redelivery.initial-backoff=200ms", + "proovy.run.coordinator.redelivery.max-backoff=1s", + "proovy.run.coordinator.recovery.interval=300ms", + "proovy.run.coordinator.recovery.max-recoveries=3", + "proovy.run.coordinator.shutdown-timeout=3s", + "proovy.run.cancel.sweep-interval=300ms", + "proovy.run.cancel.cleanup-timeout=20s" +}) +class ServerBrainIntegrationTest { + + private static final Duration WAIT = Duration.ofSeconds(60); + private static final long DAILY_CREDITS = 100; + private static final String SERVER_ROLE = "proovy_server"; + private static final String SERVER_PASSWORD = "t187-server-dev-only"; + private static final String PERMISSION_DENIED = "42501"; + private static final ObjectMapper JSON = new ObjectMapper(); + private static final Path PINS = Path.of("src/test/resources/contracts/brain"); + + private static PostgresTestDatabase db; + private static BrainTestkit brain; + + @BeforeAll + static void provision() throws SQLException { + PostgresTestDatabase.requireConfigured(); + brain = BrainTestkit.fromEnvironment(); + PostgresTestDatabase.executeAsAdminOnServer(""" + DO $$ BEGIN + IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = '%1$s') THEN + CREATE ROLE %1$s LOGIN PASSWORD '%2$s'; + ELSE + ALTER ROLE %1$s LOGIN PASSWORD '%2$s'; + END IF; + END $$; + """.formatted(SERVER_ROLE, SERVER_PASSWORD)); + db = PostgresTestDatabase.createEmpty(); + try (Connection admin = db.connect(); Statement st = admin.createStatement()) { + st.execute("CREATE EXTENSION IF NOT EXISTS pg_trgm"); + st.execute("GRANT USAGE, CREATE ON SCHEMA public TO " + SERVER_ROLE); + } + // 빈 제품 baseline을 Server role로, 같은 database에 Brain schema를 Brain bootstrap으로 만든다 + db.migrate(SERVER_ROLE, SERVER_PASSWORD); + brain.start(db.name(), 3); + db.runScript("db/provisioning/role-boundary.sql"); + } + + @DynamicPropertySource + static void properties(DynamicPropertyRegistry registry) { + registry.add("spring.datasource.url", () -> db.jdbcUrl()); + registry.add("spring.datasource.username", () -> SERVER_ROLE); + registry.add("spring.datasource.password", () -> SERVER_PASSWORD); + registry.add("proovy.brain.base-url", () -> brain.apiUrl()); + registry.add("proovy.brain.auth.shared-token", () -> brain.sharedToken()); + } + + @AfterAll + static void tearDown() { + if (brain != null) { + try { + brain.stop(); + } catch (RuntimeException ignored) { + // supervisor가 이미 내려갔으면 정리할 child도 없다 + } + } + if (db != null) { + db.close(); + } + } + + @LocalServerPort int port; + @MockitoBean AccessTokenBlacklistService blacklist; + @Autowired JwtTokenProvider tokens; + @Autowired JdbcTemplate jdbc; + @Autowired RunCoordinator coordinator; + + private BrowserRunHttp browser; + + @BeforeEach + void client() { + browser = new BrowserRunHttp(port, WAIT); + } + + // ================================================================== + // V1 — 빈 DB 접수·중복·같은 Note 충돌 + // ================================================================== + + @Test + @DisplayName("V1·V2: 빈 baseline 첫 텍스트 질문이 실제 Brain을 거쳐 저장되고 POST live와 GET replay가 같다") + void firstTextQuestionCompletesThroughRealBrainAndReplaysIdentically() throws Exception { + User user = newUser(); + String requestId = requestId(); + + Run run = post(user, body(requestId, null, "x^2 - 5x + 6 = 0을 풀어줘")); + + assertThat(run.status).isEqualTo(200); + assertThat(run.types()).startsWith("run.created").endsWith("run.completed"); + assertThat(run.frames.stream().map(Block::seq).toList()) + .isEqualTo(LongStream.rangeClosed(1, run.frames.size()).boxed().toList()); + assertThat(run.types()).contains("activity.started", "response.page.started", "response.page.completed"); + awaitStatus(run.runId, "COMPLETED"); + assertThat(row(run.runId).get("response_disposition")).isEqualTo("substantive"); + assertThat(netUsage(requestId)).isEqualTo(1); + assertThat(count("SELECT count(*) FROM notes WHERE user_id = ?", user.id)).isOne(); + assertThat(count("SELECT count(*) FROM verified_solutions WHERE source_run_id = ?", run.runId)).isOne(); + assertThat(singleTerminal(run.runId)).isEqualTo("run.completed"); + + try (SseClient replay = browser.events(user.token, run.runId, null)) { + assertThat(replay.framesUntilEnd().stream().map(Block::raw).toList()) + .containsExactlyElementsOf(run.frames.stream().map(Block::raw).toList()); + } + JsonNode page = browser.conversation(user.token, run.noteId).body(); + assertThat(page.at("/result/messages")).hasSize(2); + assertThat(page.at("/result/messages/1/status").asText()).isEqualTo("COMPLETED"); + + // Brain은 graph를 한 번 실행했고, 첫 전달 = attempt 1, cleanup이 예약되었다 + assertThat(brain.executions(requestId)).hasSize(1); + assertThat(streams(requestId)).allSatisfy(o -> assertThat(o.get("attempt").asInt()).isEqualTo(1)); + await(() -> row(run.runId).get("runtime_cleanup_at") != null); + } + + @Test + @DisplayName("V1: noteId 없는 같은 requestId 동시 전송은 Note·run·차감·Brain 실행을 하나만 남긴다") + void concurrentDuplicateNotelessRequestsLeaveOneOfEverything() throws Exception { + User user = newUser(); + String requestId = requestId(); + String body = body(requestId, null, "중복 전송 문제를 풀어줘"); + + List runs = concurrently(5, () -> post(user, body)); + + assertThat(runs).allSatisfy(run -> assertThat(run.status).isEqualTo(200)); + assertThat(runs.stream().map(run -> run.runId).distinct()).hasSize(1); + UUID runId = runs.get(0).runId; + awaitStatus(runId, "COMPLETED"); + assertThat(count("SELECT count(*) FROM notes WHERE user_id = ?", user.id)).isOne(); + assertThat(count("SELECT count(*) FROM ai_runs WHERE request_id = ?", requestId)).isOne(); + assertThat(count("SELECT count(*) FROM chat_messages m JOIN notes n ON n.note_id = m.note_id " + + "WHERE n.user_id = ?", user.id)).isEqualTo(2); + assertThat(count("SELECT count(*) FROM request_usage_charges WHERE request_id = ?", requestId)).isOne(); + assertThat(brain.executions(requestId)).hasSize(1); + assertThat(singleTerminal(runId)).isEqualTo("run.completed"); + } + + @Test + @DisplayName("V1: 같은 Note 동시 전송은 하나만 접수되고 나머지는 activeRunId가 있는 409이며 메시지를 만들지 않는다") + void concurrentSameNoteRequestsAdmitOneAndRejectTheRest() throws Exception { + User user = newUser(); + Run first = post(user, body(requestId(), null, "첫 질문")); + awaitStatus(first.runId, "COMPLETED"); + long messagesBefore = count("SELECT count(*) FROM chat_messages WHERE note_id = ?", first.noteId); + + List requestIds = List.of(requestId(), requestId(), requestId(), requestId()); + for (String id : requestIds) { + brain.plan(id, "default", BrainTestkit.faults().put("hold_at", "classification")); + } + List bodies = new ArrayList<>(); + requestIds.forEach(id -> bodies.add(body(id, Long.toString(first.noteId), "같은 Note 동시 질문"))); + List opened = concurrentlyIndexed(bodies.size(), i -> browser.post(user.token, bodies.get(i))); + try { + List admitted = opened.stream().filter(sse -> sse.status == 200).toList(); + List rejected = opened.stream().filter(sse -> sse.status != 200).toList(); + assertThat(admitted).hasSize(1); + assertThat(rejected).hasSize(3).allSatisfy(sse -> { + assertThat(sse.status).isEqualTo(409); + assertThat(sse.errorBody).contains("RUN4091").contains("activeRunId"); + }); + UUID active = UUID.fromString(admitted.get(0).nextFrame().json().get("runId").asText()); + assertThat(rejected).allSatisfy(sse -> assertThat(sse.errorBody).contains(active.toString())); + // 거부된 요청은 메시지·run·차감을 만들지 않았다 + assertThat(count("SELECT count(*) FROM chat_messages WHERE note_id = ?", first.noteId)) + .isEqualTo(messagesBefore + 2); + assertThat(count("SELECT count(*) FROM ai_runs WHERE note_id = ?", first.noteId)).isEqualTo(2); + String winner = jdbc.queryForObject("SELECT request_id FROM ai_runs WHERE run_id = ?", String.class, active); + for (String id : requestIds) { + long charges = count("SELECT count(*) FROM request_usage_charges WHERE request_id = ?", id); + assertThat(charges).isEqualTo(id.equals(winner) ? 1 : 0); + } + requestIds.forEach(id -> brain.release(id, null)); + awaitStatus(active, "COMPLETED"); + } finally { + requestIds.forEach(id -> brain.release(id, null)); + for (SseClient sse : opened) { + sse.close(); + } + } + } + + // ================================================================== + // V4 — 결과 분류별 canonical 결과·순사용량 + // ================================================================== + + record Outcome(String scenario, List tasks, String terminal, String status, String disposition, + String failureCode, long netUsage, String firstBlock) { + } + + @Test + @DisplayName("V4: 정상·check·미검증·clarification·unsupported_only·혼합·자료 부재(scripted)·최종 실패의 결과와 순사용량") + void outcomeMatrixMatchesCanonicalResultsAndNetUsagePolicy() throws Exception { + List matrix = List.of( + new Outcome("default", List.of("solve"), "run.completed", "COMPLETED", "substantive", null, 1, "solution"), + new Outcome("default", List.of("check"), "run.completed", "COMPLETED", "substantive", null, 1, "check"), + new Outcome("inconclusive", List.of(), "run.completed", "COMPLETED", "substantive", null, 1, "solution"), + new Outcome("clarification", List.of(), "run.completed", "COMPLETED", "clarification_only", null, 0, + "rich_text"), + new Outcome("unsupported_only", List.of(), "run.completed", "COMPLETED", "unsupported_only", null, 0, + "rich_text"), + new Outcome("mixed", List.of(), "run.completed", "COMPLETED", "substantive", null, 1, "solution"), + new Outcome("no_match_notice", List.of(), "run.completed", "COMPLETED", "substantive", null, 1, + "rich_text"), + new Outcome("verification_failed", List.of(), "run.failed", "FAILED", null, "verification_failed", 0, + null), + new Outcome("infrastructure_failure", List.of(), "run.failed", "FAILED", null, "provider_error", 0, + null)); + for (Outcome expected : matrix) { + User user = newUser(); + String requestId = requestId(); + brain.plan(requestId, expected.scenario, null); + + Run run = post(user, body(requestId, null, "x^2 - 5x + 6 = 0", expected.tasks.toArray(String[]::new))); + + String label = expected.scenario + expected.tasks; + assertThat(run.types()).as(label).endsWith(expected.terminal); + awaitStatus(run.runId, expected.status); + Map row = row(run.runId); + assertThat(row.get("response_disposition")).as(label).isEqualTo(expected.disposition); + assertThat(row.get("failure_code")).as(label).isEqualTo(expected.failureCode); + assertThat(netUsage(requestId)).as(label).isEqualTo(expected.netUsage); + assertThat(credits(user)).as(label).isEqualTo(DAILY_CREDITS - expected.netUsage * creditsCharged(requestId)); + assertThat(singleTerminal(run.runId)).as(label).isEqualTo(expected.terminal); + assertThat(brain.executions(requestId)).as(label).hasSize(1); + if (expected.firstBlock != null) { + assertThat(responsePages(row)).as(label).contains("\"" + expected.firstBlock + "\""); + } + if (expected.scenario.equals("inconclusive")) { + assertThat(count("SELECT count(*) FROM verified_solutions WHERE source_run_id = ?", run.runId)) + .as("inconclusive publishes no Verified Solution").isZero(); + } + if (expected.scenario.equals("mixed")) { + assertThat(responsePages(row)).contains("\"notice\""); + } + } + } + + @Test + @DisplayName("V4: 같은 Note의 Verified Solution을 참조한 explain 후속이 재표현으로 완료되고 차감된다") + void explainFollowUpReexpressesTheVerifiedSolutionOfTheNote() throws Exception { + User user = newUser(); + Run solved = post(user, body(requestId(), null, "x^2 - 5x + 6 = 0을 풀어줘", "solve")); + awaitStatus(solved.runId, "COMPLETED"); + String requestId = requestId(); + brain.plan(requestId, "explain_existing", null); + + Run explained = post(user, body(requestId, Long.toString(solved.noteId), "방금 풀이를 설명해줘", "explain")); + + awaitStatus(explained.runId, "COMPLETED"); + assertThat(row(explained.runId).get("response_disposition")).isEqualTo("substantive"); + assertThat(responsePages(row(explained.runId))).contains("\"explanation\""); + // Brain은 Server snapshot의 verified_solution_refs를 받았다 + assertThat(streams(requestId).get(0).get("body").asText()).contains("verified_solution_refs") + .contains(String.valueOf(jdbc.queryForObject( + "SELECT public_solution_id FROM verified_solutions WHERE source_run_id = ?", String.class, + solved.runId))); + assertThat(netUsage(requestId)).isEqualTo(1); + } + + // ================================================================== + // V2 — SSE 단절·재접속·내부 SSE 유실 + // ================================================================== + + @Test + @DisplayName("V2: 브라우저 단절 뒤 Last-Event-ID 재접속과 내부 SSE 유실 뒤 재부착이 같은 canonical 결과를 무유실로 복원한다") + void browserDisconnectAndInternalSseLossRestoreTheSameCanonicalStream() throws Exception { + User user = newUser(); + String requestId = requestId(); + // 내부 SSE: 첫 전달은 첫 frame(입력 분석 activity) 직전에 끊긴다. 실행은 solve에서 붙잡아 재부착을 기다린다 + brain.plan(requestId, "default", BrainTestkit.faults().put("hold_at", "solve") + .put("drop_stream_after_frames", 0)); + + List beforeDisconnect = new ArrayList<>(); + UUID runId; + try (SseClient post = browser.post(user.token, body(requestId, null, "재접속 문제를 풀어줘"))) { + Block created = post.nextFrame(); + beforeDisconnect.add(created); + runId = UUID.fromString(created.json().get("runId").asText()); + beforeDisconnect.add(post.nextFrame()); + } + // 내부 stream이 끊긴 뒤 Server가 status(executing) → 같은 요청으로 다시 붙을 때까지 실행을 붙잡아 둔다 + try { + await(() -> streams(requestId).size() >= 2); + } finally { + brain.release(requestId, null); + } + awaitStatus(runId, "COMPLETED"); + + List rest; + try (SseClient resumed = browser.events(user.token, runId, beforeDisconnect.get(1).id())) { + rest = resumed.framesUntilEnd(); + } + List full; + try (SseClient replay = browser.events(user.token, runId, null)) { + full = replay.framesUntilEnd(); + } + List joined = new ArrayList<>(beforeDisconnect.stream().map(Block::raw).toList()); + joined.addAll(rest.stream().map(Block::raw).toList()); + assertThat(joined).containsExactlyElementsOf(full.stream().map(Block::raw).toList()); + assertThat(full.get(full.size() - 1).event()).isEqualTo("run.completed"); + + // Brain: 같은 attempt·fence로 재부착했고 graph는 한 번, Server는 source event를 중복 저장하지 않았다 + List streams = streams(requestId); + assertThat(streams).hasSizeGreaterThanOrEqualTo(2); + assertThat(streams.get(0).get("dropped").asBoolean()).isTrue(); + assertThat(brain.observations(requestId).stream().map(o -> o.get("operation").asText())).contains("status"); + assertThat(streams.stream().map(o -> o.get("attempt").asInt() + "/" + o.get("fence_token").asText()).distinct()) + .hasSize(1); + assertThat(brain.executions(requestId)).hasSize(1); + assertThat(count("SELECT count(*) FROM ai_run_events WHERE run_id = ? AND source_event_id IS NOT NULL", runId)) + .isEqualTo(count("SELECT count(DISTINCT source_event_id) FROM ai_run_events WHERE run_id = ?", runId)); + assertThat(singleTerminal(runId)).isEqualTo("run.completed"); + } + + @Test + @DisplayName("V2·V3: Brain이 terminal을 저장한 뒤 전달이 끊기면 status로 저장 결과를 회수하고 재실행하지 않는다") + void storedTerminalLostInDeliveryIsRecoveredByStatusWithoutRerun() throws Exception { + User user = newUser(); + String requestId = requestId(); + brain.plan(requestId, "default", BrainTestkit.faults().put("drop_before_terminal", true)); + + Run run = post(user, body(requestId, null, "저장 후 유실 문제를 풀어줘")); + + assertThat(run.types()).endsWith("run.completed"); + awaitStatus(run.runId, "COMPLETED"); + assertThat(brain.executions(requestId)).hasSize(1); + assertThat(brain.observations(requestId).stream().map(o -> o.get("operation").asText())) + .contains("stream"); + assertThat(streams(requestId).get(0).get("dropped").asBoolean()).isTrue(); + assertThat(singleTerminal(run.runId)).isEqualTo("run.completed"); + assertThat(netUsage(requestId)).isEqualTo(1); + } + + // ================================================================== + // 재시작·소실 + // ================================================================== + + @Test + @DisplayName("Server coordinator 재시작(같은 JVM stop/start): 멈춘 동안 Brain이 저장한 terminal을 재시작 뒤 같은 attempt·fence로 회수하고 graph를 다시 돌리지 않는다") + void serverRestartRecoversTheStoredTerminalWithoutRerun() throws Exception { + User user = newUser(); + String requestId = requestId(); + brain.plan(requestId, "default", BrainTestkit.faults().put("hold_at", "solve")); + SseClient post = browser.post(user.token, body(requestId, null, "재시작 문제를 풀어줘")); + UUID runId = UUID.fromString(post.nextFrame().json().get("runId").asText()); + post.close(); + Map before; + try { + await(() -> !brain.executions(requestId).isEmpty()); + before = row(runId); + coordinator.stop(); + try { + brain.release(requestId, null); + await(() -> "terminal".equals(brain.internalRunStatus(runId.toString(), + (Integer) before.get("attempt"), (String) before.get("fence_token")).path("state").asText())); + } finally { + coordinator.start(); + } + } finally { + brain.release(requestId, null); + } + + awaitStatus(runId, "COMPLETED"); + Map after = row(runId); + assertThat(after.get("attempt")).isEqualTo(before.get("attempt")); + assertThat(after.get("fence_token")).isEqualTo(before.get("fence_token")); + assertThat((Integer) after.get("recovery_count")).isGreaterThanOrEqualTo(1); + assertThat(brain.executions(requestId)).hasSize(1); + assertThat(singleTerminal(runId)).isEqualTo("run.completed"); + assertThat(netUsage(requestId)).isEqualTo(1); + } + + @Test + @DisplayName("Brain 저장 전 소실: terminal 저장 직전 Brain을 SIGKILL하면 재시작한 Brain은 graph를 다시 돌리지 않고 Server는 한 번 보상한다") + void brainLostBeforeStorageIsSettledAsFailureNotRerun() throws Exception { + User user = newUser(); + String requestId = requestId(); + brain.plan(requestId, "default", BrainTestkit.faults().put("hold_terminal_write", true)); + SseClient post = browser.post(user.token, body(requestId, null, "저장 전 소실 문제를 풀어줘")); + UUID runId; + try { + runId = UUID.fromString(post.nextFrame().json().get("runId").asText()); + await(() -> count("SELECT count(*) FROM ai_run_events WHERE run_id = ? " + + "AND event_type = 'activity.completed'", runId) >= 2); + Map epoch = row(runId); + + brain.restart("kill"); + + // 새 Brain process: plan·실행 기록이 없고 record는 terminal 없이 executing이다(재개·재실행 없음) + JsonNode lost = brain.internalRunStatus(runId.toString(), (Integer) epoch.get("attempt"), + (String) epoch.get("fence_token")); + assertThat(lost.get("state").asText()).isEqualTo("executing"); + assertThat(lost.has("terminal") && !lost.get("terminal").isNull()).isFalse(); + awaitStatus(runId, "FAILED"); + } finally { + post.close(); + } + Map row = row(runId); + // 같은 요청 재부착·status 수습 뒤 실행 소실(Brain 시작 증거 + 진행 없음)로 확정된다 — 1.85 결정표 + assertThat(row.get("failure_class")).isEqualTo("infrastructure"); + assertThat(row.get("failure_code")).isEqualTo("provider_error"); + assertThat(brain.executions(requestId)).isEmpty(); + assertThat(netUsage(requestId)).isZero(); + assertThat(singleTerminal(runId)).isEqualTo("run.failed"); + assertThat(count("SELECT count(*) FROM request_usage_compensations WHERE request_id = ?", requestId)).isOne(); + } + + // ================================================================== + // V3 — 완료·취소·deadline 경합 + // ================================================================== + + @Test + @DisplayName("V3·V4: RUNNING 취소는 늦은 완료를 막고 CANCELED 하나만 남기며 무보상이다") + void runningCancelBlocksTheLateCompletionAndKeepsTheCharge() throws Exception { + User user = newUser(); + String requestId = requestId(); + brain.plan(requestId, "default", BrainTestkit.faults().put("hold_at", "solve")); + SseClient post = browser.post(user.token, body(requestId, null, "취소할 문제")); + UUID runId; + List frames; + try { + runId = UUID.fromString(post.nextFrame().json().get("runId").asText()); + await(() -> !brain.executions(requestId).isEmpty()); + awaitStatus(runId, "RUNNING"); + + BrowserRunHttp.Response canceled = browser.cancel(user.token, runId); + assertThat(canceled.status()).isEqualTo(200); + assertThat(canceled.body().toString()).contains("CANCELING"); + brain.release(requestId, null); + + awaitStatus(runId, "CANCELED"); + frames = post.framesUntilEnd(); + } finally { + brain.release(requestId, null); + post.close(); + } + assertThat(frames.stream().map(Block::event)).contains("run.canceling", "run.canceled") + .doesNotContain("run.completed"); + assertThat(singleTerminal(runId)).isEqualTo("run.canceled"); + assertThat(netUsage(requestId)).isEqualTo(1); + assertThat(brain.observations(requestId)).anySatisfy(o -> { + assertThat(o.get("operation").asText()).isEqualTo("cancel"); + assertThat(o.get("cause").asText()).isEqualTo("user_requested"); + assertThat(o.get("auth").asText()).isEqualTo("shared_token"); + }); + assertThat(count("SELECT count(*) FROM verified_solutions WHERE source_run_id = ?", runId)).isZero(); + } + + @Test + @DisplayName("V3·V4: 슬롯 대기 PENDING 취소는 Brain을 부르지 않고 한 번 보상한다") + void pendingCancelIsCompensatedAndNeverReachesBrain() throws Exception { + List holders = new ArrayList<>(); + List held = new ArrayList<>(); + try { + for (int i = 0; i < 3; i++) { + User owner = newUser(); + String id = requestId(); + holders.add(id); + brain.plan(id, "default", BrainTestkit.faults().put("hold_at", "classification")); + held.add(browser.post(owner.token, body(id, null, "슬롯 점유 " + i))); + } + await(() -> holders.stream().allMatch(id -> !brain.executions(id).isEmpty())); + + User user = newUser(); + String requestId = requestId(); + SseClient post = browser.post(user.token, body(requestId, null, "대기 중 취소")); + UUID runId = UUID.fromString(post.nextFrame().json().get("runId").asText()); + assertThat(row(runId).get("status")).isEqualTo("PENDING"); + + BrowserRunHttp.Response canceled = browser.cancel(user.token, runId); + + assertThat(canceled.status()).isEqualTo(200); + awaitStatus(runId, "CANCELED"); + post.close(); + assertThat(row(runId).get("attempt")).isEqualTo(0); + assertThat(netUsage(requestId)).isZero(); + assertThat(jdbc.queryForObject("SELECT reason FROM request_usage_compensations WHERE request_id = ?", + String.class, requestId)).isEqualTo("PENDING_CANCELED"); + assertThat(brain.observations(requestId)).isEmpty(); + } finally { + holders.forEach(id -> brain.release(id, null)); + for (SseClient sse : held) { + sse.close(); + } + } + } + + @Test + @DisplayName("V3: 요청 deadline 경합(Server sweep·Brain timeout seal)은 timeout 실패 하나와 보상 한 번만 남기고 늦은 완료를 버린다") + void deadlineRaceSettlesOnceAsTimeoutAndDropsTheLateCompletion() throws Exception { + User user = newUser(); + String requestId = requestId(); + brain.plan(requestId, "default", BrainTestkit.faults().put("hold_at", "solve")); + SseClient post = browser.post(user.token, body(requestId, null, "deadline 문제")); + UUID runId; + try { + runId = UUID.fromString(post.nextFrame().json().get("runId").asText()); + // request-timeout 40s: hold를 풀지 않고 deadline을 넘긴다 + awaitStatus(runId, Duration.ofSeconds(70), "FAILED"); + brain.release(requestId, null); + Thread.sleep(3000); + } finally { + brain.release(requestId, null); + post.close(); + } + + Map row = row(runId); + assertThat(row.get("status")).isEqualTo("FAILED"); + assertThat(row.get("failure_code")).isEqualTo("timeout"); + // 어느 쪽(Server sweep·Brain timeout terminal)이 이기는지는 고정하지 않는다 — 결과 하나·보상 한 번만 단언하고 승자는 실측으로 남긴다 + System.err.println("[brain-integration] deadline race settled by " + + (row.get("terminal_source_event_id") == null ? "server deadline sweep" : "brain timeout terminal")); + assertThat(singleTerminal(runId)).isEqualTo("run.failed"); + assertThat(count("SELECT count(*) FROM request_usage_compensations WHERE request_id = ?", requestId)).isOne(); + assertThat(netUsage(requestId)).isZero(); + assertThat(count("SELECT count(*) FROM verified_solutions WHERE source_run_id = ?", runId)).isZero(); + } + + @Test + @DisplayName("V3: Brain fake clock으로 Brain 쪽 deadline이 먼저 지나면 Brain timeout terminal이 하나의 실패로 확정된다") + void brainSideDeadlineFromTheFakeClockSettlesAsTimeout() throws Exception { + User user = newUser(); + String requestId = requestId(); + brain.plan(requestId, "default", BrainTestkit.faults().put("hold_at", "solve")); + SseClient post = browser.post(user.token, body(requestId, null, "Brain deadline 문제")); + UUID runId = UUID.fromString(post.nextFrame().json().get("runId").asText()); + await(() -> !brain.executions(requestId).isEmpty()); + + brain.advanceClock(120); + try { + awaitStatus(runId, "FAILED"); + } finally { + brain.advanceClock(-120); + brain.release(requestId, null); + post.close(); + } + Map row = row(runId); + assertThat(row.get("failure_code")).isEqualTo("timeout"); + // Server deadline(40s) 전에 Brain이 봉인한 timeout terminal을 받아 확정했다(Server sweep이면 source event가 없다) + assertThat(row.get("terminal_source_event_id")).isNotNull(); + assertThat(singleTerminal(runId)).isEqualTo("run.failed"); + assertThat(netUsage(requestId)).isZero(); + } + + // ================================================================== + // V5 — fixture·fingerprint 일치, scope/provenance 위반 거부 + // ================================================================== + + @Test + @DisplayName("V5: Java와 Python이 실제 요청의 fingerprint에 합의하고 Brain이 쓰는 fixture가 Server pin과 바이트 단위로 같다") + void javaAndPythonAgreeOnTheFingerprintAndPinnedFixtures() throws Exception { + User user = newUser(); + String requestId = requestId(); + Run run = post(user, body(requestId, null, "fingerprint 문제를 풀어줘", "solve")); + awaitStatus(run.runId, "COMPLETED"); + Map row = row(run.runId); + + JsonNode status = brain.internalRunStatus(run.runId.toString(), (Integer) row.get("attempt"), + (String) row.get("fence_token")); + assertThat(status.get("request_fingerprint").asText()).isEqualTo(row.get("request_fingerprint")); + // Brain이 받은 요청은 Server snapshot에서 복원한 그대로다 + JsonNode delivered = JSON.readTree(streams(requestId).get(0).get("body").asText()); + assertThat(delivered.get("request_id").asText()).isEqualTo(requestId); + assertThat(delivered.get("run_id").asText()).isEqualTo(run.runId.toString()); + assertThat(delivered.get("fence_token").asText()).isEqualTo(row.get("fence_token")); + + JsonNode brainFiles = brain.info().get("contract_files"); + int compared = 0; + try (var pins = Files.list(PINS)) { + for (Path pin : pins.filter(p -> p.getFileName().toString().startsWith("PIN")).toList()) { + JsonNode doc = JSON.readTree(pin.toFile()); + if (doc.hasNonNull("manifest") && doc.hasNonNull("manifest_sha256")) { + assertThat(brainFiles.path(doc.get("manifest").asText()).asText()).as(pin + " manifest") + .isEqualTo(doc.get("manifest_sha256").asText()); + compared++; + } + for (Iterator> it = doc.path("files").fields(); it.hasNext(); ) { + Map.Entry file = it.next(); + assertThat(brainFiles.path(file.getKey()).asText()).as(pin + " " + file.getKey()) + .isEqualTo(file.getValue().asText()); + compared++; + } + } + } + assertThat(compared).isGreaterThan(50); + String expectedCommit = System.getenv("PROOVY_BRAIN_EXPECTED_COMMIT"); + if (expectedCommit != null) { + JsonNode info = brain.info(); + assertThat(info.get("brain_commit").asText()).isEqualTo(expectedCommit); + assertThat(info.get("brain_dirty").asBoolean()).isFalse(); + } + } + + @Test + @DisplayName("V5: run scope 밖 asset을 인용한 terminal(scripted)은 SOURCE_SCOPE로 거부되고 출처·Verified Solution 없이 한 번 보상된다") + void outOfScopeSourceTerminalIsRejected() throws Exception { + User user = newUser(); + String requestId = requestId(); + brain.plan(requestId, "out_of_scope_source", null); + + Run run = post(user, body(requestId, null, "범위 밖 출처")); + + awaitStatus(run.runId, "FAILED"); + Map row = row(run.runId); + assertThat(row.get("failure_code")).isEqualTo("internal_error"); + assertThat(run.types()).doesNotContain("run.completed"); + assertThat(singleTerminal(run.runId)).isEqualTo("run.failed"); + assertThat(count("SELECT count(*) FROM verified_solutions WHERE source_run_id = ?", run.runId)).isZero(); + assertThat(String.valueOf(jdbc.queryForObject("SELECT coalesce(source_references::text, '') FROM chat_messages " + + "WHERE message_id = ?", String.class, row.get("assistant_message_id")))).doesNotContain("note-alpha"); + assertThat(netUsage(requestId)).isZero(); + assertThat(brain.executions(requestId)).singleElement() + .satisfies(e -> assertThat(e.get("kind").asText()).isEqualTo("scripted")); + } + + @Test + @DisplayName("V5·V3: 다른 run의 envelope로 변조된 terminal(scripted)은 stream·status 어디서 와도 저장되지 않는다") + void forgedRunEnvelopeIsNeverStored() throws Exception { + User user = newUser(); + String requestId = requestId(); + brain.plan(requestId, "default", BrainTestkit.faults() + .put("rewrite_status", true) + .set("rewrite_terminal", JsonNodeFactory.instance.objectNode().put("run_id", "run-someone-else"))); + + Run run = post(user, body(requestId, null, "변조된 run 문제")); + + awaitStatus(run.runId, "FAILED"); + assertThat(run.types()).doesNotContain("run.completed"); + assertThat(singleTerminal(run.runId)).isEqualTo("run.failed"); + assertThat(netUsage(requestId)).isZero(); + assertThat(count("SELECT count(*) FROM verified_solutions WHERE source_run_id = ?", run.runId)).isZero(); + assertThat(brain.executions(requestId)).hasSize(1); + } + + @Test + @DisplayName("V3·V5: 이전 fence로 변조된 stream terminal(scripted)은 버려지고 저장된 현재 fence 결과만 한 번 확정된다") + void staleFenceTerminalIsDroppedAndOnlyTheCurrentEpochSettles() throws Exception { + User user = newUser(); + String requestId = requestId(); + brain.plan(requestId, "default", BrainTestkit.faults() + .set("rewrite_terminal", JsonNodeFactory.instance.objectNode().put("fence_token", "fence-stale"))); + + Run run = post(user, body(requestId, null, "이전 fence 문제")); + + // 변조 frame은 확정되지 않고, 같은 attempt·fence의 status 조회가 Brain에 저장된 현재 fence terminal을 확정한다 + // (2026-10-19 실측) — 결과는 하나, 변조값은 어디에도 저장되지 않는다 + awaitStatus(run.runId, "COMPLETED"); + assertThat(count("SELECT count(*) FROM ai_run_events WHERE run_id = ? AND payload::text LIKE '%fence-stale%'", + run.runId)).isZero(); + assertThat(singleTerminal(run.runId)).isEqualTo("run.completed"); + assertThat(brain.observations(requestId).stream().map(o -> o.get("operation").asText())).contains("status"); + assertThat(row(run.runId).get("fence_token")).isNotEqualTo("fence-stale"); + assertThat(netUsage(requestId)).isEqualTo(1); + assertThat(brain.executions(requestId)).hasSize(1); + } + + // ================================================================== + // 보안 경계 + // ================================================================== + + @Test + @DisplayName("보안: Brain에는 서비스 token만 가고 사용자 JWT·grant·URL·내부 원문이 브라우저와 Brain 어디에도 새지 않으며 role이 격리된다") + void securityBoundariesHold() throws Exception { + User user = newUser(); + String requestId = requestId(); + Run run = post(user, body(requestId, null, "보안 경계 문제를 풀어줘")); + awaitStatus(run.runId, "COMPLETED"); + + // JWT 비전달: Brain이 받은 모든 호출은 shared token이고 body에 사용자 JWT·grant·URL이 없다 + List calls = brain.observations(requestId); + assertThat(calls).isNotEmpty().allSatisfy(o -> { + assertThat(o.get("auth").asText()).isEqualTo("shared_token"); + assertThat(o.get("body").asText()).doesNotContain(user.token).doesNotContain("Bearer"); + }); + JsonNode delivered = JSON.readTree(streams(requestId).get(0).get("body").asText()); + assertThat(delivered.path("asset_access_grant").isNull() || delivered.path("asset_access_grant").isMissingNode()) + .isTrue(); + assertThat(delivered.toString()).doesNotContain("read_url").doesNotContain("X-Goog-Signature"); + + // 서비스 인증 거부: token 없음·사용자 JWT·틀린 token은 Brain이 거부한다 + assertThat(brain.internalStatus(run.runId.toString(), null)).isIn(401, 403); + assertThat(brain.internalStatus(run.runId.toString(), "Bearer " + user.token)).isIn(401, 403); + assertThat(brain.internalStatus(run.runId.toString(), "Bearer not-the-token")).isIn(401, 403); + + // 브라우저 비노출: frame·대화 조회에 fence·grant·URL·Brain 내부 식별 필드가 없다 + String browserView = String.join("\n", run.frames.stream().map(Block::raw).toList()) + + browser.conversation(user.token, run.noteId).body(); + assertThat(browserView).doesNotContain("fence_token").doesNotContain("fenceToken") + .doesNotContain("asset_access_grant").doesNotContain("assetAccessGrant") + .doesNotContain("read_url").doesNotContain("readUrl").doesNotContain(brain.sharedToken()); + + // 역할별 DB 격리(같은 database): brain_runtime → 제품 테이블 거부, proovy_server → Brain schema 거부 + try (Connection runtime = db.connect("brain_runtime", brain.runtimePassword()); + Statement st = runtime.createStatement()) { + assertThatThrownBy(() -> st.executeQuery("SELECT count(*) FROM public.ai_runs")) + .isInstanceOf(SQLException.class) + .satisfies(e -> assertThat(((SQLException) e).getSQLState()).isEqualTo(PERMISSION_DENIED)); + try (var rs = st.executeQuery("SELECT count(*) FROM brain_runtime.run_records")) { + assertThat(rs.next()).isTrue(); + assertThat(rs.getLong(1)).isPositive(); + } + } + try (Connection server = db.connect(SERVER_ROLE, SERVER_PASSWORD); Statement st = server.createStatement()) { + assertThatThrownBy(() -> st.executeQuery("SELECT count(*) FROM brain_runtime.run_records")) + .isInstanceOf(SQLException.class) + .satisfies(e -> assertThat(((SQLException) e).getSQLState()).isEqualTo(PERMISSION_DENIED)); + } + } + + // ================================================================== + // helpers + // ================================================================== + + record User(long id, String token) { + } + + static final class Run { + final int status; + final List frames; + final UUID runId; + final long noteId; + + Run(int status, List frames) { + this.status = status; + this.frames = frames; + JsonNode created = frames.isEmpty() ? null : frames.get(0).json(); + this.runId = created == null ? null : UUID.fromString(created.get("runId").asText()); + this.noteId = created == null ? -1 : created.at("/payload/noteId").asLong(); + } + + List types() { + return frames.stream().map(Block::event).toList(); + } + } + + private Run post(User user, String body) throws Exception { + try (SseClient sse = browser.post(user.token, body)) { + if (sse.status != 200) { + return new Run(sse.status, List.of()); + } + return new Run(sse.status, sse.framesUntilEnd()); + } + } + + private static String body(String requestId, String noteId, String text, String... tasks) { + ObjectNode body = JsonNodeFactory.instance.objectNode().put("requestId", requestId).put("text", text); + if (noteId != null) { + body.put("noteId", noteId); + } + if (tasks.length > 0) { + ArrayNode list = body.putArray("requestedTasks"); + for (String task : tasks) { + list.add(task); + } + } + return body.toString(); + } + + private static String requestId() { + return UUID.randomUUID().toString(); + } + + private User newUser() { + String key = UUID.randomUUID().toString(); + long id = jdbc.queryForObject(""" + INSERT INTO users (nickname, provider, provider_user_id, email, created_at, updated_at) + VALUES ('t187', 'KAKAO', ?, ?, now(), now()) RETURNING user_id + """, Long.class, key, key + "@example.test"); + jdbc.update(""" + INSERT INTO credit_balance (user_id, daily_free_credit, daily_free_limit, daily_expires_at, free_credit, + paid_credit, created_at, updated_at) + VALUES (?, ?, ?, now() + interval '1 day', 0, 0, now(), now()) + """, id, DAILY_CREDITS, DAILY_CREDITS); + return new User(id, tokens.generateTokens(id).accessToken()); + } + + private String responsePages(Map runRow) { + return String.valueOf(jdbc.queryForObject("SELECT response_pages::text FROM chat_messages WHERE message_id = ?", + String.class, runRow.get("assistant_message_id"))); + } + + private Map row(UUID runId) { + return jdbc.queryForMap("SELECT * FROM ai_runs WHERE run_id = ?", runId); + } + + private long count(String sql, Object... args) { + return jdbc.queryForObject(sql, Long.class, args); + } + + /** 차감 단위 수 − 보상 단위 수 (request당 ledger 한 줄씩). */ + private long netUsage(String requestId) { + long charged = count("SELECT count(*) FROM request_usage_charges WHERE request_id = ?", requestId); + long compensated = count("SELECT count(*) FROM request_usage_compensations WHERE request_id = ?", requestId); + return charged - compensated; + } + + /** 그 요청의 차감 credit 양(request당 ledger 한 줄). */ + private long creditsCharged(String requestId) { + return jdbc.queryForObject("SELECT credit_amount FROM request_usage_charges WHERE request_id = ?", Long.class, + requestId); + } + + private long credits(User user) { + return jdbc.queryForObject("SELECT daily_free_credit + free_credit + paid_credit FROM credit_balance " + + "WHERE user_id = ?", Long.class, user.id); + } + + /** terminal event가 정확히 하나이고 그 type. */ + private String singleTerminal(UUID runId) { + List terminals = jdbc.queryForList(""" + SELECT event_type FROM ai_run_events WHERE run_id = ? + AND event_type IN ('run.completed', 'run.failed', 'run.canceled') + """, String.class, runId); + assertThat(terminals).as("terminal events of " + runId).hasSize(1); + return terminals.get(0); + } + + private List streams(String requestId) { + return brain.observations(requestId).stream().filter(o -> o.get("operation").asText().equals("stream")).toList(); + } + + private void awaitStatus(UUID runId, String... statuses) throws InterruptedException { + awaitStatus(runId, WAIT, statuses); + } + + private void awaitStatus(UUID runId, Duration within, String... statuses) throws InterruptedException { + Set wanted = new HashSet<>(List.of(statuses)); + long deadline = System.nanoTime() + within.toNanos(); + String last = null; + while (System.nanoTime() < deadline) { + last = (String) row(runId).get("status"); + if (wanted.contains(last)) { + return; + } + Thread.sleep(100); + } + throw new AssertionError("run " + runId + " stayed " + last + ", wanted " + wanted); + } + + private static void await(BooleanSupplier condition) throws InterruptedException { + long deadline = System.nanoTime() + WAIT.toNanos(); + while (System.nanoTime() < deadline) { + if (condition.getAsBoolean()) { + return; + } + Thread.sleep(100); + } + throw new AssertionError("condition not reached in " + WAIT); + } + + private static List concurrently(int n, Callable task) throws Exception { + return concurrentlyIndexed(n, i -> task.call()); + } + + interface IndexedTask { + T call(int index) throws Exception; + } + + private static List concurrentlyIndexed(int n, IndexedTask task) throws Exception { + ExecutorService pool = Executors.newFixedThreadPool(n); + CyclicBarrier barrier = new CyclicBarrier(n); + try { + List> futures = new ArrayList<>(); + for (int i = 0; i < n; i++) { + int index = i; + futures.add(pool.submit(() -> { + barrier.await(); + return task.call(index); + })); + } + List results = new ArrayList<>(); + for (Future future : futures) { + results.add(future.get()); + } + return results; + } finally { + pool.shutdownNow(); + } + } +} diff --git a/src/test/java/com/proovy/support/http/BrowserRunHttp.java b/src/test/java/com/proovy/support/http/BrowserRunHttp.java new file mode 100644 index 00000000..07800853 --- /dev/null +++ b/src/test/java/com/proovy/support/http/BrowserRunHttp.java @@ -0,0 +1,86 @@ +package com.proovy.support.http; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import java.net.URI; +import java.net.URLEncoder; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.util.UUID; + +/** 브라우저 run API(browser-run-v1)를 실제 HTTP로 부르는 테스트 도구. 사용자 JWT는 호출마다 받는다. */ +public final class BrowserRunHttp { + + private static final ObjectMapper JSON = new ObjectMapper(); + + private final HttpClient http = HttpClient.newBuilder().version(HttpClient.Version.HTTP_1_1) + .connectTimeout(Duration.ofSeconds(5)).build(); + private final int port; + private final Duration wait; + + public BrowserRunHttp(int port, Duration wait) { + this.port = port; + this.wait = wait; + } + + public record Response(int status, JsonNode body) { + } + + private URI uri(String path) { + return URI.create("http://127.0.0.1:" + port + path); + } + + /** POST /api/conversations (접수 + live SSE). */ + public SseClient post(String token, String body) throws Exception { + return SseClient.open(http, HttpRequest.newBuilder(uri("/api/conversations")) + .header("Authorization", "Bearer " + token).header("Accept", "text/event-stream") + .header("Content-Type", "application/json").POST(HttpRequest.BodyPublishers.ofString(body)).build(), + wait); + } + + /** GET /api/ai-runs/{runId}/events (replay/live). {@code lastEventId}는 header로 보낸다. */ + public SseClient events(String token, UUID runId, String lastEventId) throws Exception { + HttpRequest.Builder request = HttpRequest.newBuilder(uri("/api/ai-runs/" + runId + "/events")) + .header("Authorization", "Bearer " + token).header("Accept", "text/event-stream").GET(); + if (lastEventId != null) { + request.header("Last-Event-ID", lastEventId); + } + return SseClient.open(http, request.build(), wait); + } + + public Response get(String token, String path) throws Exception { + HttpResponse response = http.send(HttpRequest.newBuilder(uri(path)) + .header("Authorization", "Bearer " + token).timeout(Duration.ofSeconds(15)).GET().build(), + HttpResponse.BodyHandlers.ofString()); + return new Response(response.statusCode(), parse(response.body())); + } + + public Response cancel(String token, UUID runId) throws Exception { + HttpResponse response = http.send(HttpRequest.newBuilder(uri("/api/ai-runs/" + runId + "/cancel")) + .header("Authorization", "Bearer " + token).timeout(Duration.ofSeconds(15)) + .POST(HttpRequest.BodyPublishers.noBody()).build(), HttpResponse.BodyHandlers.ofString()); + return new Response(response.statusCode(), parse(response.body())); + } + + public Response conversation(String token, long noteId) throws Exception { + return get(token, "/api/conversations/" + noteId + "/messages"); + } + + public static String encode(String value) { + return URLEncoder.encode(value, StandardCharsets.UTF_8); + } + + private static JsonNode parse(String body) { + if (body == null || body.isBlank()) { + return JSON.nullNode(); + } + try { + return JSON.readTree(body); + } catch (Exception e) { + return JSON.getNodeFactory().textNode(body); + } + } +} diff --git a/src/test/java/com/proovy/support/http/SseClient.java b/src/test/java/com/proovy/support/http/SseClient.java new file mode 100644 index 00000000..c3d56ffb --- /dev/null +++ b/src/test/java/com/proovy/support/http/SseClient.java @@ -0,0 +1,167 @@ +package com.proovy.support.http; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.BlockingQueue; +import java.util.concurrent.LinkedBlockingQueue; +import java.util.concurrent.TimeUnit; + +/** + * 실제 HTTP 연결로 브라우저 SSE를 읽는 테스트 client (1.86 {@code BrowserRunSsePostgresTest}와 같은 판독 규칙). + * {@link #close()}는 연결을 끊는다(브라우저 단절). + */ +public final class SseClient implements AutoCloseable { + + private static final ObjectMapper JSON = new ObjectMapper(); + private static final Block END = new Block(null, null, null, null); + + /** SSE 블록 하나. comment 블록은 event·id·data가 null이다. raw는 끝 빈 줄을 뺀 원문이다. */ + public record Block(String event, String id, String data, String raw) { + + public JsonNode json() { + try { + return JSON.readTree(data); + } catch (IOException e) { + throw new AssertionError("frame data is not JSON: " + data, e); + } + } + + public long seq() { + return Long.parseLong(id.substring(id.indexOf(':') + 1)); + } + } + + public final int status; + public final String contentType; + public final String errorBody; + private final InputStream body; + private final Duration wait; + private final BlockingQueue blocks = new LinkedBlockingQueue<>(); + /** reader thread에서 난 판독 오류 — 다음 frame 요청에서 test thread로 다시 던진다. */ + private volatile Throwable readerFailure; + + private SseClient(int status, String contentType, String errorBody, InputStream body, Duration wait) { + this.status = status; + this.contentType = contentType; + this.errorBody = errorBody; + this.body = body; + this.wait = wait; + } + + public static SseClient open(HttpClient client, HttpRequest request, Duration wait) throws Exception { + HttpResponse response = client.send(request, HttpResponse.BodyHandlers.ofInputStream()); + String contentType = response.headers().firstValue("Content-Type").orElse(""); + if (response.statusCode() != 200) { + String error; + try (InputStream in = response.body()) { + error = new String(in.readAllBytes(), StandardCharsets.UTF_8); + } + SseClient failed = new SseClient(response.statusCode(), contentType, error, null, wait); + failed.blocks.add(END); + return failed; + } + SseClient sse = new SseClient(200, contentType, null, response.body(), wait); + Thread reader = new Thread(sse::read, "sse-test-client"); + reader.setDaemon(true); + reader.start(); + return sse; + } + + private void read() { + try (BufferedReader reader = new BufferedReader(new InputStreamReader(body, StandardCharsets.UTF_8))) { + List lines = new ArrayList<>(); + String line; + while ((line = reader.readLine()) != null) { + if (!line.isEmpty()) { + lines.add(line); + continue; + } + if (!lines.isEmpty()) { + blocks.add(parse(lines)); + lines = new ArrayList<>(); + } + } + } catch (IOException closed) { + // close()로 끊음 + } catch (RuntimeException | AssertionError e) { + readerFailure = e; + } finally { + blocks.add(END); + } + } + + private static Block parse(List lines) { + String event = null; + String id = null; + String data = null; + for (String line : lines) { + if (line.startsWith("event: ")) { + event = line.substring(7); + } else if (line.startsWith("id: ")) { + id = line.substring(4); + } else if (line.startsWith("data: ")) { + // 여러 data 줄은 SSE 규칙대로 줄바꿈으로 잇는다 + data = data == null ? line.substring(6) : data + "\n" + line.substring(6); + } else if (!line.startsWith(":")) { + throw new AssertionError("unexpected SSE line: " + line); + } + } + return new Block(event, id, data, String.join("\n", lines)); + } + + /** heartbeat를 건너뛴 다음 frame. 끝났으면 null. */ + public Block nextFrameOrEnd() throws InterruptedException { + while (true) { + Block block = blocks.poll(wait.toMillis(), TimeUnit.MILLISECONDS); + if (block == null) { + throw new AssertionError("no SSE block in " + wait); + } + if (block == END) { + blocks.add(END); + if (readerFailure != null) { + throw new AssertionError("SSE reader failed", readerFailure); + } + return null; + } + if (block.event() != null) { + return block; + } + } + } + + public Block nextFrame() throws InterruptedException { + Block frame = nextFrameOrEnd(); + if (frame == null) { + throw new AssertionError("stream ended before the next frame"); + } + return frame; + } + + /** 서버가 연결을 닫을 때까지의 frame. */ + public List framesUntilEnd() throws InterruptedException { + List frames = new ArrayList<>(); + Block block; + while ((block = nextFrameOrEnd()) != null) { + frames.add(block); + } + return frames; + } + + @Override + public void close() throws IOException { + if (body != null) { + body.close(); + } + } +} diff --git a/src/test/resources/application-test.yaml b/src/test/resources/application-test.yaml index 470d401c..452af7d9 100644 --- a/src/test/resources/application-test.yaml +++ b/src/test/resources/application-test.yaml @@ -12,7 +12,3 @@ spring: properties: hibernate: format_sql: true - -internal: - api: - token: test-internal-token