Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions .github/workflows/auto_tests.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
name: Auto Tests

on: push
on:
push:
paths:
# Run just when the backend changed
- "server/**"
- "!server/**/*.md"
- ".github/workflows/auto_tests.yml"

jobs:
tests:
Expand Down Expand Up @@ -44,8 +50,8 @@ jobs:
run: |
cd server
sudo apt-get -y install libsqlite3-mod-spatialite
pip install pipenv==2026.0.3
pipenv install --dev --verbose --python 3.12
pip install pipenv==2026.8.0
pipenv install --dev --deploy --verbose --python 3.12

- name: Run tests
run: |
Expand Down
21 changes: 6 additions & 15 deletions .github/workflows/code_style.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
name: Code Layout

on: push
on:
push:
paths:
# Run just when python files changed
- "server/**/*.py"
- ".github/workflows/code_style.yml"

jobs:
code_style_python:
Expand All @@ -14,17 +19,3 @@ jobs:
version: 25.1.0
options: "--check --verbose --diff"
src: "./server"

code_style_js:
name: JavaScript code convention check
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: Check JS
run: |
cd web-app
yarn install
yarn run lint:no-legacy
23 changes: 23 additions & 0 deletions .github/workflows/code_style_js.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: Code Layout JS

on:
push:
paths:
# Run just when the web app changed
- "web-app/**"
- ".github/workflows/code_style_js.yml"

jobs:
code_style_js:
name: JavaScript code convention check
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: Check JS
run: |
cd web-app
yarn install
yarn run lint:no-legacy
69 changes: 42 additions & 27 deletions development.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,47 +4,60 @@
This page contains useful information for those who wish to develop Mergin.

## Running locally (for dev)

Install dependencies and run services:

### Postgres and Redis

```shell
$ docker run -d --rm --name mergin_maps_dev_db -p 5002:5432 -e POSTGRES_PASSWORD=postgres postgres:14
$ docker run -d --rm --name mergin_maps_dev_redis -p 6379:6379 redis
docker run -d --rm --name mergin_maps_dev_db -p 5002:5432 -e POSTGRES_PASSWORD=postgres postgres:14
docker run -d --rm --name mergin_maps_dev_redis -p 6379:6379 redis
```

### Server

The server requires **Python 3.12** (see `python_version` in `server/Pipfile`). If your system Python is a different version, use a separate environment, e.g. conda / miniforge or pyenv:

```shell
$ pip3 install --upgrade pip==24.0
$ pip3 install pipenv==2024.0.1
$ cd server
# conda / miniforge
conda create -n mergin python=3.12
conda activate mergin
# or pyenv
pyenv install 3.12
pyenv local 3.12
```

Pipenv then creates its own virtualenv on top of that interpreter.

```shell
pip install pipenv==2026.8.0
cd server
# Install dependencies with pipenv
# Note: You can append --three flag in older versions of pipenv (< 3.16.8 2023-02-04)
$ pipenv install --dev
$ pipenv install --categories="telemetry
$ pipenv run pre-commit install
$ pipenv run pre-commit run --all-files
$ export FLASK_APP=application; export COLLECT_STATISTICS=0
$ pipenv run flask init-db
pipenv install --dev --deploy
pipenv install --categories="telemetry" --deploy
pipenv run pre-commit install
pipenv run pre-commit run --all-files
export FLASK_APP=application; export COLLECT_STATISTICS=0
pipenv run flask init-db
# create admin user
$ pipenv run flask user create admin topsecret --is-admin --email admin@example.com
pipenv run flask user create admin topsecret --is-admin --email admin@example.com
# create (non admin) user
$ pipenv run flask user create user topsecret --email user@example.com
$ pipenv run celery -A application.celery worker --loglevel=info &
$ pipenv run flask run # run dev server on port 5000
pipenv run flask user create user topsecret --email user@example.com
pipenv run celery -A application.celery worker --loglevel=info &
pipenv run flask run # run dev server on port 5000
```

### Web applications

Before installing the web applications, make sure you have Node.js installed in a supported version. The applications require Node.js version **18 or higher**.

```shell
$ cd web-app
$ yarn install
$ yarn link:dependencies # link dependencies
$ yarn build:libs # bild libraries @mergin/lib @mergin/admin-lib @mergin/lib-vue2
$ yarn dev # development client web application dev server on port 8080 (package @mergin/app)
$ yarn dev:admin # development admin application dev server on port 8081 (package @mergin/admin-app)
cd web-app
yarn install
yarn link:dependencies # link dependencies
yarn build:libs # bild libraries @mergin/lib @mergin/admin-lib @mergin/lib-vue2
yarn dev # development client web application dev server on port 8080 (package @mergin/app)
yarn dev:admin # development admin application dev server on port 8081 (package @mergin/admin-app)
```

If you are developing a library package (named **-lib*), it is useful to watch the library for changes instead of rebuilding it each time.
Expand All @@ -60,7 +73,6 @@ yarn watch:lib:types

Watching the type definitions is also useful to pick up any changes to imports or new components that are added.


## Running locally in a docker composition

If you want to run the whole stack locally, you can use the docker. Docker will build the images from your local files and run the services.
Expand Down Expand Up @@ -97,6 +109,7 @@ docker exec -it merginmaps-server flask server send-check-email --email admin@e
In docker-compose.dev.yml is started maildev/maildev image that can be used to test emails (see [https://github.com/maildev/maildev/](https://github.com/maildev/maildev/)). In localhost:1080 you can see the emails sent by the application in web interface.

### Running with remote debugger

If you want to run the application with remote debugger, you can use debug compose file with attached source code and reload.
It starts a debugpy session on port 5678 you can attach to.

Expand All @@ -105,10 +118,12 @@ docker compose -f docker-compose.yml -f docker-compose.debug.yml up
```

## Running tests

To launch the unit tests run:

```shell
$ docker run -d --rm --name testing_pg -p 5435:5432 -e POSTGRES_PASSWORD=postgres postgres:14
$ cd server
$ pipenv install --dev --sequential --verbose
$ pipenv run pytest -v --cov=mergin mergin/tests
docker run -d --rm --name testing_pg -p 5435:5432 -e POSTGRES_PASSWORD=postgres postgres:14
cd server
pipenv install --dev --deploy --verbose
pipenv run pytest -v --cov=mergin mergin/tests
```
2 changes: 1 addition & 1 deletion server/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ WORKDIR /app

# keep installing to system packages
ENV PIP_BREAK_SYSTEM_PACKAGES=1
RUN pip install pipenv==2026.0.3
RUN pip install pipenv==2026.8.0
# for locale check this http://click.pocoo.org/5/python3/
ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
Expand Down
26 changes: 14 additions & 12 deletions server/Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,17 @@ name = "pypi"

[packages]
connexion = {extras = ["swagger-ui"],version = "==2.15.1"}
flask = "==3.1.2"
flask = "==3.1.3"
python-dateutil = "==2.8.2"
marshmallow = "==3.26.1"
marshmallow = "==3.26.2"
flask-marshmallow = "==0.15.0"
marshmallow-sqlalchemy = "==1.4.1"
psycopg2-binary = "==2.9.9"
itsdangerous = "==2.2.0"
Flask-SQLAlchemy = "==3.1.1"
sqlalchemy = "==2.0.44"
gunicorn = {extras = ["gevent"],version = "==23.0"}
python-dotenv = "==0.20.0"
python-dotenv = "==1.2.2"
flask-login = "==0.6.3"
bcrypt = "==4.2.0"
wtforms = {extras = ["email"],version = "==3.2.1"}
Expand All @@ -35,34 +35,36 @@ jsons = "==1.6.3"
binaryornot = "==0.4.4"
chardet = "<5"
python-decouple = "==3.6"
urllib3 = "==2.2.2"
urllib3 = "==2.7.0"
shapely = "==2.0.6"
psycogreen = "==1.0.2"
importlib-metadata = "==8.4.0" # https://github.com/pallets/flask/issues/4502
typing_extensions = "==4.12.2"
python-magic = "==0.4.27"
click = "==8.2.0"
click = "==8.3.3"
# requirements for development on windows
colorama = "==0.4.5"

[dev-packages]
pytest = "==8.3.2"
pytest = "==9.1.1"
pytest-cov = "==5.0.0"
pylint = "==3.2.6"
responses = "==0.21.0"
pytest-dotenv= "==0.5.2"
pysqlite3-binary= "==0.5.3"
# sqlite with loadable extensions for tests, pysqlite3-binary wheels are available only for linux
pysqlite3-binary = {version = "==0.5.3", markers = "sys_platform == 'linux'"}
pysqlite3 = {version = "==0.6.0", markers = "sys_platform != 'linux'"}
black = "==25.1.0"
pre-commit = "==4.1.0"
# requirements for pytest on windows
atomicwrites = "==1.4.0"

[telemetry]
opentelemetry-distro = {extras = ["otlp"], version = "*"}
opentelemetry-instrumentation-flask = "*"
opentelemetry-instrumentation-celery = "*"
opentelemetry-instrumentation-sqlalchemy = "*"
opentelemetry-instrumentation-logging = "*"
opentelemetry-distro = {extras = ["otlp"], version = "==0.60b1"}
opentelemetry-instrumentation-flask = "==0.60b1"
opentelemetry-instrumentation-celery = "==0.60b1"
opentelemetry-instrumentation-sqlalchemy = "==0.60b1"
opentelemetry-instrumentation-logging = "==0.60b1"

[requires]
python_version = "3.12"
Loading
Loading