docs!
This commit is contained in:
my
commit
38d4e68b11
214
Readme.md
Normal file
214
Readme.md
Normal file
|
|
@ -0,0 +1,214 @@
|
|||
# Braids: Intro to Git workshop
|
||||
|
||||
This repo contains technical notes from the preparation for the Braids: Intro to git we gave at Het Nieuwe Instituut on Januari 15 2026 on invitation of The Hmm and NNAD as part of the workshop series "How to Archive Better: Preserving Practices".
|
||||
|
||||
The slides for the workshop live in their own repo at: https://git.hackersanddesigners.nl/hrk/Braid-slides and are viewable at https://braids.hackersanddesigners.nl/slides/
|
||||
|
||||
We’ve set up a Forgejo instance (you're probably reading this there) on a Raspberry Pi, accessible over the web via Tinc. During the workshop, user account registration was temporarily open. After a short introduction to the basics of Git, participants were invited to clone the workshop repository, create a personal branch (for example braids/<slug>), make a few small edits to the starter HTML page, commit their changes, and push the branch back to Forgejo. A webhook on the same machine published each pushed branch to a shared gallery site, so everyone could immediately view each other’s pages in the browser and iterate with additional commits.
|
||||
|
||||
The Forgejo repository is configured with a push webhook that calls a small deployment script on the Raspberry Pi. When a branch is pushed, the webhook triggers the script to fetch the latest updates from Forgejo, export the contents of each participant branch, and write them to the web server directory under a per-branch path. It also regenerates the gallery index page so new or updated branches appear immediately. This provides a near real-time “publish on push” loop during the workshop. The main branch was protect to commits in the Forgejo repo settings.
|
||||
|
||||
This involved:
|
||||
|
||||
- [Installing Forgejo and configuring Tinc](#installing-forgejo)
|
||||
- [Setting up a webhook](#setting-up-a-webhook)
|
||||
- [Creating a script to deplay the sites](#deploying-the-sites)
|
||||
|
||||
## Installing Forgejo
|
||||
|
||||
Mostly based on https://pimylifeup.com/raspberry-pi-forgejo/ with some modification because we're storing the repos on an external SSD
|
||||
|
||||
I am going to pretend i did this in the right order. We first setup the SSD, so we can configure the proper paths rightaway.
|
||||
|
||||
### Set up the SSD.
|
||||
Create mount directory:
|
||||
|
||||
> sudo mkdir /media/ssd
|
||||
|
||||
Test mount
|
||||
> sudo mount /dev/sda1 /media/ssd
|
||||
|
||||
If it works add it to fstab
|
||||
|
||||
To get uuid
|
||||
> lsblk -f
|
||||
|
||||
/etc/fstab
|
||||
UUID=<uuid> /media/ssd ext4 defaults,no-fail 0 2
|
||||
|
||||
No-fail so the raspi boot even if the disk unplugged
|
||||
|
||||
### Then install Forgejo
|
||||
|
||||
Following: https://pimylifeup.com/raspberry-pi-forgejo/
|
||||
|
||||
Install docker with:
|
||||
> curl -sSL https://get.docker.com | sh
|
||||
|
||||
Create user (in current user group)
|
||||
> sudo usermod -aG docker $USER
|
||||
|
||||
Logout
|
||||
> logout
|
||||
Check if the group exist
|
||||
> groups
|
||||
|
||||
test:
|
||||
> docker run hello-world
|
||||
|
||||
Docker should now be set up.
|
||||
|
||||
Create a directory for the installation:
|
||||
|
||||
> sudo mkdir -p /opt/stacks/forgejo
|
||||
> cd /opt/stacks/forgejo
|
||||
> nano compose.yaml
|
||||
|
||||
```yaml
|
||||
services:
|
||||
server:
|
||||
image: codeberg.org/forgejo/forgejo:7
|
||||
container_name: forgejo
|
||||
environment:
|
||||
- USER_UID=1000
|
||||
- USER_GID=1000
|
||||
- FORGEJO__database__DB_TYPE=postgres
|
||||
- FORGEJO__database__HOST=db:5432
|
||||
- FORGEJO__database__NAME=forgejo
|
||||
- FORGEJO__database__USER=forgejo
|
||||
- FORGEJO__database__PASSWD=<db_password>
|
||||
- FORGEJO__WEBHOOK__ALLOWED_HOST_LIST=private,loopback
|
||||
restart: always
|
||||
networks:
|
||||
- forgejo
|
||||
volumes:
|
||||
- /media/ssd/forgejo:/data
|
||||
- /etc/timezone:/etc/timezone:ro
|
||||
- /etc/localtime:/etc/localtime:ro
|
||||
ports:
|
||||
- "3000:3000"
|
||||
- "222:22"
|
||||
depends_on:
|
||||
- db
|
||||
|
||||
db:
|
||||
image: postgres:14
|
||||
restart: always
|
||||
environment:
|
||||
- POSTGRES_USER=forgejo
|
||||
- POSTGRES_PASSWORD=<db_password>
|
||||
- POSTGRES_DB=forgejo
|
||||
networks:
|
||||
- forgejo
|
||||
volumes:
|
||||
- ./postgres:/var/lib/postgresql/data
|
||||
|
||||
networks:
|
||||
forgejo:
|
||||
external: false
|
||||
```
|
||||
|
||||
> docker compose up -d
|
||||
|
||||
It should now be accessible at <ip>:3000
|
||||
|
||||
Now we have to set up tinc proxying, but in this case it was already setup for another service. So only nginx config on the tinc edge node and creating a certificate.
|
||||
|
||||
> sudo certbot --nginx -d git.hackersanddesigners.nl
|
||||
|
||||
And configure nginx in /etc/nginx/sites-available/forgejo.conf:
|
||||
|
||||
Set the newly created domain in the Forgejo config with:
|
||||
> nano /media/ssd/forgejo/gitea/conf/app.ini
|
||||
|
||||
And change the domains:
|
||||
ROOT_URL = https://git.hackersanddesigners.nl
|
||||
DOMAIN = git.hackersanddesigners.nl
|
||||
|
||||
And restart the service
|
||||
> docker compose restart
|
||||
|
||||
We shoudl now have a running instance of Forgejo
|
||||
|
||||
## Setting up a webhook
|
||||
|
||||
Setting up the webhook took a bit of time because it is somewhat confusing what IP to use to get from the Docker container to the host. But that is probably my inexperience with Docker.
|
||||
|
||||
Setup web hooks like this: https://www.shawenyao.com/Voice-Controlled-Raspberry-Pi/
|
||||
|
||||
> sudo apt install webhook
|
||||
|
||||
configure
|
||||
> sudo nano /etc/webhook.conf
|
||||
```
|
||||
[
|
||||
{
|
||||
"id": "git-pull-site",
|
||||
"execute-command": "/usr/local/bin/deploy-git-site.sh",
|
||||
"command-working-directory": "/var/www/html/",
|
||||
"pass-arguments-to-command": []
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
Create the shell script
|
||||
> sudo nano /usr/local/bin/deploy-git-site.sh
|
||||
|
||||
```sh
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
cd /var/www/html/testsite
|
||||
git pull --ff-only
|
||||
```
|
||||
|
||||
Make it executable
|
||||
> sudo chmod +x /usr/local/bin/deploy-braids.py
|
||||
|
||||
Edit the web hook:
|
||||
> sudo systemctl edit webhook
|
||||
Add:
|
||||
```
|
||||
[Service]
|
||||
ExecStart=
|
||||
ExecStart=/usr/bin/webhook -hooks /etc/webhook.conf -ip 0.0.0.0 -port 9010 -verbose
|
||||
```
|
||||
|
||||
> sudo systemctl daemon-reload
|
||||
> sudo systemctl enable --now webhook
|
||||
> sudo systemctl restart webhook
|
||||
|
||||
Check for errors:
|
||||
> sudo systemctl status webhook --no-pager
|
||||
|
||||
Allow the webhook to connect to localhost, add to compose.yaml
|
||||
|
||||
> - FORGEJO__WEBHOOK__ALLOWED_HOST_LIST=private,loopback
|
||||
|
||||
|
||||
We have to call the webhook on the IP of the docker host. Get it with:
|
||||
|
||||
> ip -4 addr show docker0
|
||||
|
||||
Gave:
|
||||
```
|
||||
5: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
|
||||
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
|
||||
valid_lft forever preferred_lft forever
|
||||
```
|
||||
(if i understand correctly 172.17.0.1 is kinda standard, but i did not know that)
|
||||
|
||||
Now add the webhook to the repo with that IP:
|
||||
url: http://172.17.0.1:9010/hooks/git-pull-site
|
||||
|
||||
## Deploying the sites
|
||||
|
||||
The script called by the webhook is a python script that was cobbled together from a bunch of resources:
|
||||
|
||||
- https://stackoverflow.com/questions/31261600/how-to-archive-a-remote-git-repository-programmatically-using-python
|
||||
- https://git-scm.com/docs/git-archive
|
||||
- https://www.funkycloudmedina.com/2018/10/deploy-a-hugo-website-from-github-to-s3-using-github-webhooks-api-gateway-and-lambda/
|
||||
- https://www.aicodesnippet.com/python/quality-and-best-practices/basic-git-operations-in-python-using-subprocess.html
|
||||
- https://forgejo.org/docs/next/contributor/static-pages/
|
||||
|
||||
You can find the script in this repo under /code/deploy-braids.py
|
||||
156
code/deploy-braids.py
Executable file
156
code/deploy-braids.py
Executable file
|
|
@ -0,0 +1,156 @@
|
|||
#!/usr/bin/env python3
|
||||
|
||||
# This script gets all the branches of the repo (set belwo) with a certain prefix
|
||||
# and copies them to the webroot as separate directories.
|
||||
# Not very secure ofcourse so only run the webhook when the workshop is ongoing
|
||||
# and remove the files afterwards.
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import fcntl
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path("/srv/git/braids.git") # bare/mirror repo
|
||||
WEBROOT = Path("/var/www/html/") # public root
|
||||
PREFIX = "braids" # this sets both the prefix for the branches <prefix/name> as the directory the sites will be published in.
|
||||
PEOPLE_DIR = WEBROOT / PREFIX
|
||||
# Had to experiment with a few different location for this. /tmp and /run did not work.
|
||||
LOCKFILE = Path("/var/lock/braids-deploy.lock")
|
||||
|
||||
REF_PREFIX = "refs/heads/" + PREFIX + "/"
|
||||
ALLOWED_SLUG_CHARS = set("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789._-")
|
||||
|
||||
|
||||
def run(cmd, cwd=None, input_bytes=None):
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
cwd=str(cwd) if cwd else None,
|
||||
input=input_bytes,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
check=True,
|
||||
)
|
||||
return result.stdout
|
||||
|
||||
|
||||
def is_valid_slug(slug):
|
||||
if not slug:
|
||||
return False
|
||||
for ch in slug:
|
||||
if ch not in ALLOWED_SLUG_CHARS:
|
||||
return False
|
||||
return True
|
||||
|
||||
# Gets all branches for the prefix.
|
||||
def get_people_refs():
|
||||
out = run([
|
||||
"git",
|
||||
"-C",
|
||||
str(REPO),
|
||||
"for-each-ref",
|
||||
"--format=%(refname)",
|
||||
"refs/heads/" + PREFIX +"/*",
|
||||
])
|
||||
refs = []
|
||||
for line in out.decode("utf-8", errors="replace").splitlines():
|
||||
line = line.strip()
|
||||
if not line.startswith(REF_PREFIX):
|
||||
continue
|
||||
slug = line[len(REF_PREFIX):]
|
||||
if is_valid_slug(slug):
|
||||
refs.append((line, slug))
|
||||
refs.sort(key=lambda item: item[1].lower())
|
||||
return refs
|
||||
|
||||
# create the directories if necessary
|
||||
def ensure_dirs():
|
||||
WEBROOT.mkdir(parents=True, exist_ok=True)
|
||||
PEOPLE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# update the local bare repo
|
||||
def fetch_mirror():
|
||||
run(["git", "-C", str(REPO), "fetch", "--prune", "origin", "+refs/heads/*:refs/heads/*"])
|
||||
|
||||
|
||||
def publish_one(ref, slug):
|
||||
tmpdir = Path(tempfile.mkdtemp(prefix="braids-" + slug + "-"))
|
||||
try:
|
||||
# Export branch root
|
||||
archive = run(["git", "-C", str(REPO), "archive", ref])
|
||||
run([
|
||||
"tar",
|
||||
"-x",
|
||||
"-C",
|
||||
str(tmpdir),
|
||||
"--no-same-owner",
|
||||
"--no-same-permissions",
|
||||
], input_bytes=archive)
|
||||
|
||||
dest = PEOPLE_DIR / slug
|
||||
if dest.exists():
|
||||
shutil.rmtree(dest)
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.move(str(tmpdir), str(dest))
|
||||
make_world_readable(dest)
|
||||
except subprocess.CalledProcessError:
|
||||
# If branch doesn't contain that folder, just skip it.
|
||||
pass
|
||||
finally:
|
||||
shutil.rmtree(tmpdir, ignore_errors=True)
|
||||
|
||||
|
||||
def make_world_readable(path):
|
||||
for root, dirs, files in os.walk(path):
|
||||
root_path = Path(root)
|
||||
for d in dirs:
|
||||
os.chmod(root_path / d, 0o755)
|
||||
for f in files:
|
||||
os.chmod(root_path / f, 0o644)
|
||||
os.chmod(path, 0o755)
|
||||
|
||||
# create a simple html file with links to the documents. iframes because it looks nice.
|
||||
def rebuild_index(refs):
|
||||
ensure_dirs()
|
||||
|
||||
parts = [
|
||||
"<!doctype html><meta charset='utf-8'>",
|
||||
"<title>Braids</title>",
|
||||
"<style>",
|
||||
"body{font-family:sans-serif;margin:20px}",
|
||||
".grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(280px,1fr));gap:16px}",
|
||||
".card{border:1px solid #ddd;padding:12px}",
|
||||
"iframe{width:100%;height:260px;border:1px solid #eee}",
|
||||
"</style>",
|
||||
"<h1>Braids</h1>",
|
||||
"<div class='grid'>",
|
||||
]
|
||||
for _, slug in refs:
|
||||
parts.append("<div class='card'>")
|
||||
parts.append(f"<div><a href='{PREFIX}/{slug}/' target='_blank'>{slug}</a></div>")
|
||||
parts.append(f"<iframe src='{PREFIX}/{slug}/'></iframe>")
|
||||
parts.append("</div>")
|
||||
parts.append("</div>")
|
||||
|
||||
(WEBROOT / "index.html").write_text("\n".join(parts), encoding="utf-8")
|
||||
|
||||
|
||||
def main():
|
||||
# Lock (avoid concurrent deploys)
|
||||
LOCKFILE.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(LOCKFILE, "w") as lf:
|
||||
fcntl.flock(lf, fcntl.LOCK_EX)
|
||||
|
||||
ensure_dirs()
|
||||
fetch_mirror()
|
||||
refs = get_people_refs()
|
||||
|
||||
for ref, slug in refs:
|
||||
publish_one(ref, slug)
|
||||
|
||||
rebuild_index(refs)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Loading…
Reference in a new issue