Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Compose builds all images with context: . — this file is the only ignore Docker reads.

# VCS / tooling
.git
.github
.vagrant
.direnv
.pytest_cache
.mypy_cache
**/__pycache__
**/*.pyc
**/*.pyo
**/*.tsbuildinfo

# Secrets / env
.env
.envrc

# Local data and virtualenvs
base
redis_data
.venv
venv

# Frontend install/build artifacts (must not overlay Alpine npm ci)
**/node_modules
frontend/dist
frontend/public/schemas

# Not needed in any image
tests
scripts
Vagrantfile
examples
*.md
LICENSE
3 changes: 2 additions & 1 deletion .flake8
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,5 @@ exclude =
env,
base/,
redis_data/,
.vagrant/
.vagrant/,
frontend/node_modules/
25 changes: 23 additions & 2 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-24.04

env:
PYTHONPATH: ${{ github.workspace }}:${{ github.workspace }}/web
PYTHONPATH: ${{ github.workspace }}:${{ github.workspace }}/backend

steps:
- name: Checkout code
Expand All @@ -27,8 +27,29 @@ jobs:
python -m pip install --upgrade pip
pip install pytest
pip install -r tests/requirements.txt
pip install -r web/requirements.txt
pip install -r backend/requirements.txt

- name: Run tests
run: |
pytest tests/ -v --tb=short

frontend:
runs-on: ubuntu-24.04

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: '20'
cache: npm
cache-dependency-path: frontend/package-lock.json

- name: Install and check frontend
working-directory: frontend
run: |
npm ci
npm run lint
npm run build
9 changes: 9 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -136,3 +136,12 @@ dump.rdb

# base directory
base

# TypeScript incremental build info
*.tsbuildinfo

# Node / frontend build artifacts
node_modules/

# Generated from repo-root schemas/config by Vite
frontend/public/schemas/
51 changes: 26 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ The ArduPilot Custom Firmware Builder is a web-based application designed to gen
## Running Locally Using Docker
To minimize setup overhead and enhance ease of use, running this application in Docker containers is highly recommended. Follow the instructions below to run the application locally using Docker:

1. **Install Docker and Docker Compose:** Make sure Docker and Docker Compose are installed on your machine. For installation instructions, visit the [Docker website](https://docs.docker.com/engine/install).
1. **Install Docker and Docker Compose:** Make sure Docker and Docker Compose are installed on yo r machine. For installation instructions, visit the [Docker website](https://docs.docker.com/engine/install).

2. **Clone the Repository:**
```bash
Expand Down Expand Up @@ -48,10 +48,12 @@ To minimize setup overhead and enhance ease of use, running this application in
sudo docker compose up -d
```

This starts Redis, the backend API, the builder, and the frontend. Only the frontend is published on the host; nginx serves the UI and proxies `/api` to the backend.

**Note:** When starting the application for the first time, it takes some time to initialize the ArduPilot Git repositories at the backend. This process also involves populating the list of available versions and releases using the GitHub API, so please be patient.

5. **Access the Web Interface:**
The application binds to port 11080 on your host machine by default. Open your web browser and go to `http://localhost:11080` to interact with the web interface. To change the port, set the `WEB_PORT` environment variable in the .env file mentioned in the _Configure Environment Variables_ section.
The frontend binds to port 11080 on your host machine by default. Open your web browser and go to `http://localhost:11080` to interact with the web interface. To change the port, set the `WEB_PORT` environment variable in the `.env` file mentioned in the _Configure Environment Variables_ section.

6. **Stopping the Application:**
To stop the application, you can use the following command:
Expand All @@ -61,7 +63,7 @@ To minimize setup overhead and enhance ease of use, running this application in
This will stop and remove the containers, but it will not delete any built images or volumes, preserving your data for future use.

## Running Locally Without Docker on Ubuntu
To run the ArduPilot Custom Firmware Builder locally without Docker, ensure you have an environment capable of building ArduPilot. Refer to the [ArduPilot Environment Setup Guide](https://ardupilot.org/dev/docs/building-setup-linux.html) if necessary.
This setup is intended for **local development** only, not production. Ensure you have an environment capable of building ArduPilot. Refer to the [ArduPilot Environment Setup Guide](https://ardupilot.org/dev/docs/building-setup-linux.html) if necessary.

1. **Clone the Custom-Build Repository:**
```bash
Expand All @@ -86,7 +88,7 @@ To run the ArduPilot Custom Firmware Builder locally without Docker, ensure you

3. **Install Dependencies:**
```bash
pip install -r web/requirements.txt -r builder/requirements.txt
pip install -r backend/requirements.txt -r builder/requirements.txt
```

If pip is not installed, run:
Expand All @@ -104,46 +106,45 @@ To run the ArduPilot Custom Firmware Builder locally without Docker, ensure you
sudo systemctl status redis-server
```

5. **Execute the Application:**
- For a development environment with auto-reload, run:
```bash
python3 web/main.py
```
To change the port, use the `--port` argument:
```bash
python3 web/main.py --port 9000
```
- For a production environment, use:
```bash
uvicorn web.main:app --host 0.0.0.0 --port 8080
```
5. **Start the Backend and Frontend:**
In one terminal, start the API (listens on port 8080 by default):
```bash
PYTHONPATH=. uvicorn backend.main:app --host 0.0.0.0 --port 8080
```
To use a different port, pass `--port`. If you change it, update the Vite proxy target in `frontend/vite.config.ts` to match.

During the coding and testing phases, use the development environment to easily debug and make changes with auto-reload enabled. When deploying the app for end users, use the production environment to ensure better performance, scalability, and security.
In another terminal, start the UI:
```bash
cd frontend
npm install
npm run dev
```

The application will automatically set up the required base directory at `./base` upon first execution. You may customize this path by setting the `CBS_BASEDIR` environment variable.
The application will automatically set up the required base directory at `./base` upon first execution. You may customize this path by setting the `CBS_BASEDIR` environment variable.

6. **Access the Web Interface:**

Once the application is running, you can access the interface in your web browser at http://localhost:8080.

The default port is 8080, or the value of the `WEB_PORT` environment variable if set. You can override this by passing the `--port` argument when running the application directly (e.g., `python3 web/main.py --port 9000`) or when using uvicorn (e.g., `uvicorn web.main:app --port 5000`). Refer to the [uvicorn documentation](https://www.uvicorn.org/) for additional configuration options.
Open `http://localhost:5173` in your browser. Vite proxies `/api` requests to the backend, so you do not need nginx for local development.

## Directory Structure
The default directory structure is established as follows:
```
/home/<username>
└── CustomBuild
├── schemas
│ └── config
│ └── 0.0.1.json (shared CustomBuild YAML schema)
└── base
├── ardupilot (used by the web component)
├── artifacts
├── ardupilot (used by the backend)
├── artifacts (build bundles include custombuild.yaml)
├── configs
| └── remotes.json (optional, see examples/remotes.json.sample)
├── secrets
| └── admin_token (optional)
├── tmp
└── ardupilot (used by the builder component)
```
The build artifacts are organized under the `base/artifacts` subdirectory.
The build artifacts are organized under the `base/artifacts` subdirectory. Each completed build archive (`.tar.gz`) includes firmware binaries, `build.log`, `extra_hwdef.dat`, and a Builder-generated `custombuild.yaml` for rebuilding. Config schemas live under `schemas/config/` at the repo root (consumed by Builder, backend, and frontend).

## Acknowledgements
This project includes many valuable contributions made during the Google Summer of Code 2021. For more information, please see the [GSOC 2021 Blog Post](https://discuss.ardupilot.org/t/gsoc-2021-custom-firmware-builder/74946).
31 changes: 31 additions & 0 deletions backend/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
FROM python:3.10.16-slim-bookworm

RUN apt-get update \
&& apt-get install -y --no-install-recommends git gosu \
&& rm -rf /var/lib/apt/lists/*

RUN groupadd -g 999 ardupilot && \
useradd -u 999 -g 999 -m ardupilot --shell /bin/false && \
mkdir /app && \
chown ardupilot:ardupilot /app

COPY backend/requirements.txt /app/backend/requirements.txt
RUN pip install --no-cache-dir -r /app/backend/requirements.txt

COPY --chown=ardupilot:ardupilot ap_git /app/ap_git
COPY --chown=ardupilot:ardupilot build_config /app/build_config
COPY --chown=ardupilot:ardupilot build_manager /app/build_manager
COPY --chown=ardupilot:ardupilot builder /app/builder
COPY --chown=ardupilot:ardupilot metadata_manager /app/metadata_manager
COPY --chown=ardupilot:ardupilot utils /app/utils
COPY --chown=ardupilot:ardupilot schemas /app/schemas
COPY --chown=ardupilot:ardupilot backend /app/backend

WORKDIR /app/backend

ENV PYTHONPATH=/app

EXPOSE 8080

ENTRYPOINT ["./docker-entrypoint.sh"]
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]
File renamed without changes.
2 changes: 1 addition & 1 deletion web/api/v1/__init__.py → backend/api/v1/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""API v1 module."""
from web.api.v1.router import router
from backend.api.v1.router import router

__all__ = ["router"]
4 changes: 2 additions & 2 deletions web/api/v1/admin.py → backend/api/v1/admin.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
from fastapi import APIRouter, HTTPException, Depends, status
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials

from web.schemas import RefreshVersionsResponse
from web.services.admin import get_admin_service, AdminService
from backend.schemas import RefreshVersionsResponse
from backend.services.admin import get_admin_service, AdminService


router = APIRouter(prefix="/admin", tags=["admin"])
Expand Down
6 changes: 3 additions & 3 deletions web/api/v1/builds.py → backend/api/v1/builds.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,13 @@
)
from fastapi.responses import FileResponse, PlainTextResponse, Response

from web.schemas import (
from backend.schemas import (
BuildRequest,
BuildSubmitResponse,
BuildOut,
)
from web.services.builds import get_builds_service, BuildsService
from web.core.limiter import limiter
from backend.services.builds import get_builds_service, BuildsService
from backend.core.limiter import limiter

router = APIRouter(prefix="/builds", tags=["builds"])

Expand Down
2 changes: 1 addition & 1 deletion web/api/v1/router.py → backend/api/v1/router.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"""
from fastapi import APIRouter

from web.api.v1 import vehicles, builds, admin
from backend.api.v1 import vehicles, builds, admin

# Create the main v1 router
router = APIRouter(prefix="/v1")
Expand Down
4 changes: 2 additions & 2 deletions web/api/v1/vehicles.py → backend/api/v1/vehicles.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
from typing import List, Optional
from fastapi import APIRouter, Depends, HTTPException, Query, Path

from web.schemas import (
from backend.schemas import (
VehicleBase,
VersionOut,
BoardOut,
StandardArtifactOut,
FeatureOut,
)
from web.services.vehicles import get_vehicles_service, VehiclesService
from backend.services.vehicles import get_vehicles_service, VehiclesService

router = APIRouter(prefix="/vehicles", tags=["vehicles"])

Expand Down
4 changes: 2 additions & 2 deletions web/core/__init__.py → backend/core/__init__.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
"""
Core application components.
"""
from web.core.config import get_settings
from web.core.startup import initialize_application
from backend.core.config import get_settings
from backend.core.startup import initialize_application

__all__ = [
"get_settings",
Expand Down
File renamed without changes.
2 changes: 1 addition & 1 deletion web/core/limiter.py → backend/core/limiter.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from slowapi.errors import RateLimitExceeded
from slowapi import Limiter
from slowapi.util import get_remote_address
from web.core.config import get_settings
from backend.core.config import get_settings

logger = logging.getLogger(__name__)

Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
29 changes: 8 additions & 21 deletions web/main.py → backend/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,20 @@
Main FastAPI application entry point.
"""
from contextlib import asynccontextmanager
from pathlib import Path
import threading
import os
import argparse

from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles
from slowapi.errors import RateLimitExceeded
from slowapi.middleware import SlowAPIMiddleware

from web.api.v1 import router as v1_router
from web.ui import router as ui_router
from backend.api.v1 import router as v1_router

from web.core.config import get_settings
from web.core.startup import initialize_application
from web.core.logging_config import setup_logging
from web.core.limiter import limiter, rate_limit_exceeded_handler
from backend.core.config import get_settings
from backend.core.startup import initialize_application
from backend.core.logging_config import setup_logging
from backend.core.limiter import limiter, rate_limit_exceeded_handler

import ap_git
import build_manager
Expand Down Expand Up @@ -134,6 +131,7 @@ async def lifespan(app: FastAPI):
title="CustomBuild API",
description="API for ArduPilot Custom Firmware Builder",
version="1.0.0",
openapi_url="/api/openapi.json",
docs_url="/api/docs",
redoc_url="/api/redoc",
lifespan=lifespan,
Expand All @@ -143,20 +141,9 @@ async def lifespan(app: FastAPI):
app.add_middleware(SlowAPIMiddleware)
app.add_exception_handler(RateLimitExceeded, rate_limit_exceeded_handler)

# Mount static files
WEB_ROOT = Path(__file__).resolve().parent
app.mount(
"/static",
StaticFiles(directory=str(WEB_ROOT / "static")),
name="static"
)

# Include API v1 router
app.include_router(v1_router, prefix="/api")

# Include Web UI router
app.include_router(ui_router)


@app.get("/health")
async def health_check():
Expand All @@ -169,8 +156,8 @@ async def health_check():
parser.add_argument(
"--port",
type=int,
default=int(os.getenv("WEB_PORT", 8080)),
help="Port to run the server on (default: 8080 or WEB_PORT env var)"
default=int(os.getenv("BACKEND_PORT", 8080)),
help="Port to run the server on (default: 8080 or BACKEND_PORT env var)"
)
args = parser.parse_args()

Expand Down
File renamed without changes.
Loading
Loading