Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# Local reports, captures and scratch files
.user/

# Test sources are mounted into the Compose test container instead of shipped
# in the production image.
vectorizing/tests/

### Python template
# Byte-compiled / optimized / DLL files
__pycache__/
Expand Down Expand Up @@ -164,7 +168,6 @@ cython_debug/
.idea/

# Other files
heroku.yml
docker-compose.yml
.pre-commit-config.yaml
.vscode/
Expand Down
21 changes: 8 additions & 13 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,17 +1,10 @@
# Use an official Python runtime as a parent image
FROM python:3.11-slim

# Install necessary packages
RUN apt-get update -y && apt-get install -y \
wget \
build-essential \
python3-dev \
libagg-dev \
libpotrace-dev \
pkg-config \
libgl1 \
libcairo2 \
--no-install-recommends && \
# Install system dependencies shared with the development container.
COPY scripts/install_system_dependencies.sh /tmp/install_system_dependencies.sh
RUN bash /tmp/install_system_dependencies.sh && \
rm /tmp/install_system_dependencies.sh && \
rm -rf /var/lib/apt/lists/*

# Create a non-root user and group with specific IDs for consistency
Expand All @@ -21,8 +14,10 @@ RUN addgroup --gid 1001 appuser && \
# Set the working directory
WORKDIR /app

# Copy only requirements to leverage Docker cache
COPY requirements/dev.txt /app/requirements.txt
# Copy only requirements to leverage Docker cache. The test Compose service
# overrides this with the development requirements.
ARG REQUIREMENTS_FILE=prod.txt
COPY requirements/${REQUIREMENTS_FILE} /app/requirements.txt

# Install Python dependencies
RUN pip install --no-cache-dir -r /app/requirements.txt
Expand Down
12 changes: 8 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ The first time the execution can take few minutes, as it is pulling the dev cont

This will compile dependencies and environments, ensuring a consistent development workflow and deployment.

4. If you want to add or remove **system** dependencies, update the script: [`scripts/install_system_dependencies.sh`](scripts/install_system_dependencies.sh). This is used both in CI and at dev container creation, to keep them consistent.
4. If you want to add or remove **system** dependencies, update the shared script: [`scripts/install_system_dependencies.sh`](scripts/install_system_dependencies.sh). It is used by the production Docker image and at dev container creation, to keep them consistent. The Compose test image uses development Python requirements and mounts `vectorizing/tests/` at runtime because test sources are excluded from the production build context.

## Linting and formatting

Expand Down Expand Up @@ -74,12 +74,16 @@ python scripts/benchmark_vectorization.py --output .user/benchmark.json --case a
- Add `--baseline-root /path/to/baseline` to benchmark two trusted checkouts in alternating order, with exact-output checks. `--repeats` defaults to 3 and `--warmups` to 1 per process. Output mismatches exit `1` but still save the report; timings have no pass/fail threshold.
- Reports include runtime/source fingerprints and raw samples. `--label` can record commit IDs. Checkouts share the installed dependencies; this is not a dependency-isolated or production/concurrency benchmark. Use only trusted source roots.

For Docker, build the current scripts and mount only the output directory:
For Docker, build the current scripts with development dependencies and mount the
fixture images and output directory:

```bash
mkdir -p .user
docker build -t vectorizing:tools .
docker run --rm --user "$(id -u):$(id -g)" -v "$PWD/.user:/results" vectorizing:tools \
docker build --build-arg REQUIREMENTS_FILE=dev.txt -t vectorizing:tools .
docker run --rm --user "$(id -u):$(id -g)" \
-v "$PWD/.user:/results" \
-v "$PWD/vectorizing/tests/images:/app/vectorizing/tests/images:ro" \
vectorizing:tools \
python scripts/vectorization_outputs.py capture /results/candidate
```

Expand Down
8 changes: 6 additions & 2 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,12 +48,16 @@ services:

test:
<<: *app
image: vectorizing:test
build:
context: .
args:
REQUIREMENTS_FILE: dev.txt
profiles: [testing]
environment:
<<: *local-aws
volumes:
- ./vectorizing/tests/results:/app/vectorizing/tests/results
- ./vectorizing/tests/diff_output:/app/vectorizing/tests/diff_output
- ./vectorizing/tests:/app/vectorizing/tests
depends_on:
moto-init:
condition: service_completed_successfully
Expand Down
3 changes: 0 additions & 3 deletions envs/dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@ dependencies:
- faiss-cpu==1.7.4
- filelock==3.18.0
- flask==2.3.3
- flask-cors==4.0.2
- gunicorn==21.2.0
- identify==2.6.12
- idna==3.10
Expand All @@ -54,11 +53,9 @@ dependencies:
- jinja2==3.1.6
- jmespath==1.0.1
- lazy-loader==0.4
- llvmlite==0.40.1
- markupsafe==3.0.2
- networkx==3.4.2
- nodeenv==1.9.1
- numba==0.57.1
- numpy==1.24.4
- opencv-python-headless==4.8.0.76
- packaging==25.0
Expand Down
3 changes: 0 additions & 3 deletions envs/prod.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,18 +34,15 @@ dependencies:
- cuid==0.4
- faiss-cpu==1.7.4
- flask==2.3.3
- flask-cors==4.0.2
- gunicorn==21.2.0
- idna==3.10
- imageio==2.37.0
- itsdangerous==2.2.0
- jinja2==3.1.6
- jmespath==1.0.1
- lazy-loader==0.4
- llvmlite==0.40.1
- markupsafe==3.0.2
- networkx==3.4.2
- numba==0.57.1
- numpy==1.24.4
- opencv-python-headless==4.8.0.76
- packaging==25.0
Expand Down
5 changes: 0 additions & 5 deletions heroku.yml

This file was deleted.

11 changes: 0 additions & 11 deletions requirements/dev.txt
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,6 @@ faiss-cpu==1.7.4
filelock==3.18.0
# via virtualenv
flask==2.3.3
# via
# -r /workspaces/vectorizing/requirements/prod.txt
# flask-cors
flask-cors==4.0.2
# via -r /workspaces/vectorizing/requirements/prod.txt
gunicorn==21.2.0
# via -r /workspaces/vectorizing/requirements/prod.txt
Expand Down Expand Up @@ -90,10 +86,6 @@ lazy-loader==0.4
# via
# -r /workspaces/vectorizing/requirements/prod.txt
# scikit-image
llvmlite==0.40.1
# via
# -r /workspaces/vectorizing/requirements/prod.txt
# numba
markupsafe==3.0.2
# via
# -r /workspaces/vectorizing/requirements/prod.txt
Expand All @@ -105,13 +97,10 @@ networkx==3.4.2
# scikit-image
nodeenv==1.9.1
# via pre-commit
numba==0.57.1
# via -r /workspaces/vectorizing/requirements/prod.txt
numpy==1.24.4
# via
# -r /workspaces/vectorizing/requirements/prod.txt
# imageio
# numba
# opencv-python-headless
# pypotrace
# pywavelets
Expand Down
2 changes: 0 additions & 2 deletions requirements/prod.in
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,7 @@
cuid~=0.4
boto3~=1.35.2
flask~=2.3.3
flask-cors~=4.0.0
sentry-sdk~=1.30.0
numba~=0.57.1
pillow~=10.0.0
faiss-cpu~=1.7.4
pypotrace~=0.3
Expand Down
9 changes: 0 additions & 9 deletions requirements/prod.txt
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,6 @@ cuid==0.4
faiss-cpu==1.7.4
# via -r /workspaces/vectorizing/scripts/../requirements/prod.in
flask==2.3.3
# via
# -r /workspaces/vectorizing/scripts/../requirements/prod.in
# flask-cors
flask-cors==4.0.2
# via -r /workspaces/vectorizing/scripts/../requirements/prod.in
gunicorn==21.2.0
# via -r /workspaces/vectorizing/scripts/../requirements/prod.in
Expand All @@ -46,20 +42,15 @@ jmespath==1.0.1
# botocore
lazy-loader==0.4
# via scikit-image
llvmlite==0.40.1
# via numba
markupsafe==3.0.2
# via
# jinja2
# werkzeug
networkx==3.4.2
# via scikit-image
numba==0.57.1
# via -r /workspaces/vectorizing/scripts/../requirements/prod.in
numpy==1.24.4
# via
# imageio
# numba
# opencv-python-headless
# pypotrace
# pywavelets
Expand Down
24 changes: 21 additions & 3 deletions scripts/install_system_dependencies.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,23 @@
#!/bin/bash
# this script install system dependencies for vectorizing
# Install the system dependencies required by the production image and devcontainer.

sudo apt-get update
sudo apt-get install -y build-essential libagg-dev libpotrace-dev pkg-config libffi-dev libcairo2-dev
set -euo pipefail

if [[ "$(id -u)" -eq 0 ]]; then
APT_GET=(apt-get)
else
APT_GET=(sudo apt-get)
fi

"${APT_GET[@]}" update
"${APT_GET[@]}" install -y --no-install-recommends \
wget \
build-essential \
python3-dev \
libagg-dev \
libpotrace-dev \
pkg-config \
libgl1 \
libcairo2 \
libffi-dev \
libcairo2-dev
2 changes: 1 addition & 1 deletion vectorizing/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ def index() -> str | Response | tuple[Response, int]:
timer.end_timer()

if raw:
return markup
return Response(markup, mimetype="image/svg+xml")

timer.start_timer("Markup Upload")
cuid_str = upload_markup(markup, S3_BUCKET)
Expand Down
1 change: 1 addition & 0 deletions vectorizing/tests/test_api_contracts.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ def test_raw_requests_crop_before_vectorizing_and_do_not_upload(
},
)
assert response.status_code == 200
assert response.mimetype == "image/svg+xml"
assert response.data.startswith(b"<svg ")
assert b'width="8" height="8"' in response.data
chosen, unused = (binary, color) if solver == 0 else (color, binary)
Expand Down
Loading