Repository navigation
build(site): move generators out of the web root, add a pull-based deploy #1538
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
13aaa64
build(site): move generators out of the web root, add a pull-based de…
frankbria 0b5ccf0
Merge branch 'main' into site-deploy
frankbria 191baed
fix(site): deploy survives a first run and a re-run after a bad deploy
frankbria 2d89720
Merge branch 'main' into site-deploy
frankbria 1288f72
fix(site): an interrupted deploy no longer costs the last-good backup
frankbria 72a168d
fix(site): a failed backup no longer truncates the previous one
frankbria 4d6b91b
Merge branch 'main' into site-deploy
frankbria 48ec7f8
fix(site): record a deploy only after its last fallible step
frankbria 8d473e2
fix(site): set the deploy marker before the web root is created
frankbria cc4e4f3
fix(site): recover from a clone killed part-way
frankbria File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| # Site tooling | ||
|
|
||
| Scripts for the static `codeframe.sh` page in `site/`. They live here rather than | ||
| in `site/` so that everything under `site/` is servable, and a deploy can copy the | ||
| directory as-is with no exclude list to remember. | ||
|
|
||
| | Script | What it does | | ||
| |---|---| | ||
| | `make-og-image.sh` | Regenerates `site/og-image.png` (the 1200x630 social card) and `site/proof9-loop-still.png`. Needs ffmpeg and DejaVu fonts. | | ||
| | `make-banner.py` | Builds the outlined banner SVG. Usage is in its docstring. | | ||
| | `deploy.sh` | Publishes `site/` at `origin/main` to the web root. Runs on the server. | | ||
|
|
||
| ## Deploying | ||
|
|
||
| The trigger is manual. After a change to `site/` merges: | ||
|
|
||
| ```bash | ||
| tailscale ssh root@frank-prod-server deploy-codeframe-site | ||
| ``` | ||
|
|
||
| One-time setup on the server. It needs `git`, `rsync` and the `www-data` user, which | ||
| nginx creates; the script refuses to start without that user rather than failing | ||
| part-way through. | ||
|
|
||
| ```bash | ||
| git clone --depth 1 https://github.com/frankbria/codeframe /opt/codeframe-site | ||
| install -m 755 /opt/codeframe-site/tools/site/deploy.sh /usr/local/bin/deploy-codeframe-site | ||
| ``` | ||
|
|
||
| Re-run the `install` line whenever `deploy.sh` itself changes; the installed copy | ||
| does not update itself. | ||
|
|
||
| What a run does: | ||
|
|
||
| - pulls the public repository, so no credential is needed and none is stored; | ||
| - keeps one rolling backup of the web root it replaces, at `/root/codeframe.sh-predeploy.tgz`. | ||
| It is skipped on a first deploy, when the root already holds the same commit, and after | ||
| an interrupted run, so a re-run after a bad or broken deploy cannot overwrite the last | ||
| good copy; | ||
| - mirrors `site/` into the web root with `rsync --delete`, so an asset removed or | ||
| renamed in the repository disappears on the next deploy; | ||
| - writes the deployed commit to `.deployed-sha` in the web root. | ||
|
|
||
| Paths can be overridden with `REPO_URL`, `SRC`, `DOCROOT` and `BACKUP`, which is how | ||
| the script is tested in a throwaway container. | ||
|
|
||
| Check what is live: | ||
|
|
||
| ```bash | ||
| tailscale ssh root@frank-prod-server cat /var/www/codeframe.sh/.deployed-sha | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| #!/usr/bin/env bash | ||
| # Publish site/ at origin/main to the codeframe.sh web root. | ||
| # | ||
| # Runs ON the web server, not on a workstation. Installed as | ||
| # /usr/local/bin/deploy-codeframe-site; see tools/site/README.md. | ||
| # | ||
| # The repository is public, so this pulls with no credential. rsync --delete | ||
| # makes the web root an exact copy of site/: an asset removed or renamed in the | ||
| # repository disappears here on the next run. | ||
| set -euo pipefail | ||
|
|
||
| REPO_URL=${REPO_URL:-https://github.com/frankbria/codeframe} | ||
| SRC=${SRC:-/opt/codeframe-site} | ||
| DOCROOT=${DOCROOT:-/var/www/codeframe.sh} | ||
| BACKUP=${BACKUP:-/root/codeframe.sh-predeploy.tgz} | ||
| # Present only while a sync is under way. If it survives, a previous run died | ||
| # mid-sync and the web root is a torn mix of two commits. | ||
| INCOMPLETE=${INCOMPLETE:-${BACKUP%.tgz}.incomplete} | ||
|
|
||
| # A clone killed part-way leaves a .git with no commit, and every later fetch | ||
| # would fail. Remove only that: a checkout with a commit is kept even if fetch | ||
| # fails, and a directory that is not a git checkout is never deleted. | ||
| if [ -d "$SRC/.git" ] && ! git -C "$SRC" rev-parse -q --verify HEAD >/dev/null 2>&1; then | ||
| rm -rf "$SRC" | ||
| fi | ||
| if [ ! -d "$SRC/.git" ]; then | ||
| git clone -q --depth 1 "$REPO_URL" "$SRC" | ||
| fi | ||
| git -C "$SRC" fetch -q --depth 1 origin main | ||
| git -C "$SRC" reset -q --hard origin/main | ||
| HEAD_SHA=$(git -C "$SRC" rev-parse HEAD) | ||
|
|
||
| # Fail before touching anything if the web server user is missing, rather than | ||
| # part-way through with a live site and no record of the deploy. | ||
| id -u www-data >/dev/null 2>&1 || { echo "deploy: user www-data does not exist; create it or install nginx first" >&2; exit 1; } | ||
|
|
||
| # One rolling backup of what is being replaced, taken only when the web root is | ||
| # a complete deploy of a different commit. Skipped on a first deploy (no root), | ||
| # on a re-run at the same commit, and after an interrupted run (torn root): in | ||
| # each of those cases a backup would replace the last good copy with something | ||
| # that is not one. The sync below still runs, so a re-run repairs the root. | ||
| if [ -d "$DOCROOT" ] && [ ! -e "$INCOMPLETE" ] \ | ||
| && [ "$(cat "$DOCROOT/.deployed-sha" 2>/dev/null)" != "$HEAD_SHA" ]; then | ||
| # Written aside and renamed: a tar that fails part-way (disk full) must not | ||
| # truncate the backup it was meant to replace. | ||
| tar -C "$(dirname "$DOCROOT")" -czf "$BACKUP.tmp" "$(basename "$DOCROOT")" | ||
| mv "$BACKUP.tmp" "$BACKUP" | ||
| fi | ||
|
|
||
| # The marker goes down before the first change to the web root and comes up | ||
| # after the last, so no interruption can leave a changed root without it. | ||
| touch "$INCOMPLETE" | ||
| mkdir -p "$DOCROOT" | ||
| # .well-known/ is left alone in case certificate renewal ever uses this root. | ||
| rsync -a --delete --exclude '/.well-known/' --exclude '/.deployed-sha' "$SRC/site/" "$DOCROOT/" | ||
| chown -R www-data:www-data "$DOCROOT" | ||
| # Success is recorded only after the last step that can fail, so a failure | ||
| # leaves the previous sha and the marker in place for the next run to repair. | ||
| echo "$HEAD_SHA" > "$DOCROOT/.deployed-sha" | ||
| rm -f "$INCOMPLETE" | ||
|
|
||
| echo "deployed ${HEAD_SHA:0:12} to $DOCROOT" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.