honey-be/docs/GITEA_VPS_DEPLOY.md

# Gitea Actions: Deploy honey-be to VPS on push to main


This guide sets up automatic deployment of **honey-be** to your Honey VPS (188.116.23.7) when you push to the `main` branch. The workflow runs on your Gitea runner, syncs the repo via rsync over SSH, then runs the rolling-update script on the VPS.

## Prerequisites

- Gitea with Actions enabled and at least one runner registered (e.g. `ubuntu-latest`).
- Honey VPS (188.116.23.7) with backend at `/opt/app/backend/honey-be` and `scripts/rolling-update.staged.sh`.
- The **Gitea server (or the machine where the runner runs)** must be able to reach 188.116.23.7 over the network (e.g. outbound SSH). Gitea itself can stay Tailscale-only.

---

## 1. Create the deploy SSH key on your Mac

**Where:** Local Mac (Terminal).

**1.1** Generate a dedicated deploy key (Ed25519, no passphrase):

```bash
ssh-keygen -t ed25519 -C "gitea-deploy-honey-be" -f ~/.ssh/gitea_deploy_honey_be -N ""
```

**1.2** Where the keys are on your Mac:

| File on your Mac | Purpose |
|------------------|--------|
| `~/.ssh/gitea_deploy_honey_be` | **Private key** — you will paste this into Gitea (Step 3). Never put this on the Staged VPS. |
| `~/.ssh/gitea_deploy_honey_be.pub` | **Public key** — you will put this on the Staged VPS (Step 2). |

**1.3** Optional: display the keys so you can copy them later.

To show the **public** key (for Step 2):

```bash
cat ~/.ssh/gitea_deploy_honey_be.pub
```

To show the **private** key (for Step 3 — paste into Gitea):

```bash
cat ~/.ssh/gitea_deploy_honey_be
```

Copy each output as a whole (including `ssh-ed25519 ...` for the public key and `-----BEGIN ... KEY-----` / `-----END ... KEY-----` for the private key). You can run these commands again anytime.

---

## 2. Put the public key on the Staged VPS

The **Staged VPS** is your Honey server: **188.116.23.7**. The Gitea runner will SSH into this machine as `root` (or your deploy user), so the **public** key must be in that user’s `authorized_keys` on the Staged VPS.

**2.1 — On your Local Mac:** Copy the public key to the clipboard (so you can paste it on the VPS):

```bash
cat ~/.ssh/gitea_deploy_honey_be.pub | pbcopy
```

Or just note the single line that looks like:  
`ssh-ed25519 AAAAC3... gitea-deploy-honey-be`

**2.2 — Log in to the Staged VPS** from your Mac:

```bash
ssh root@188.116.23.7
```

(Use the same user you normally use to manage this server, e.g. `root` or a user with sudo. If you use a different user, replace `root` in the next steps with that user.)

**2.3 — On the Staged VPS (188.116.23.7):** Ensure `.ssh` exists and add the public key.

Run these commands **one by one** on the Staged VPS (after you’re logged in via SSH):

```bash
mkdir -p ~/.ssh
chmod 700 ~/.ssh
```

Then add the public key. **Either** paste the line you copied (replace `PASTE_YOUR_PUBLIC_KEY_LINE_HERE` with the actual line):

```bash
echo 'PASTE_YOUR_PUBLIC_KEY_LINE_HERE' >> ~/.ssh/authorized_keys
```

**Or**, if you have the key in your Mac clipboard, on the Staged VPS you can do (from Mac, one line):

```bash
ssh root@188.116.23.7 "mkdir -p ~/.ssh && chmod 700 ~/.ssh && echo '$(cat ~/.ssh/gitea_deploy_honey_be.pub)' >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
```

If you ran the `echo ... >> authorized_keys` manually on the VPS, set permissions:

```bash
chmod 600 ~/.ssh/authorized_keys
```

**2.4 — Verify from your Mac:** Check that the deploy key can log in without a password:

```bash
ssh -i ~/.ssh/gitea_deploy_honey_be root@188.116.23.7 "echo OK"
```

You should see `OK`. If you get "Permission denied (publickey)", the public key was not added correctly to `~/.ssh/authorized_keys` on the Staged VPS.

---

## 3. Put the private key into Gitea (repository secret)

**Where:** Private key stays on your Mac; you only **paste its contents** into Gitea in the browser.

**3.1 — On your Local Mac:** Show the private key so you can copy it:

```bash
cat ~/.ssh/gitea_deploy_honey_be
```

Copy the **entire** output, including:

- `-----BEGIN OPENSSH PRIVATE KEY-----`
- all lines in the middle
- `-----END OPENSSH PRIVATE KEY-----`

**3.2 — In Gitea (browser):** Add it as a repository secret.

1. Open your repo: `http://100.122.146.65:3000/admin/honey-be` (or your Gitea URL over Tailscale).
2. Go to **Settings** → **Secrets and Variables** → **Actions**.
3. Under **Repository Secrets**, click **Add Secret**.
4. **Name:** `DEPLOY_SSH_PRIVATE_KEY`
5. **Value:** Paste the full private key you copied from the previous command. Paste once, with no extra spaces or blank lines before/after; the key must start with `-----BEGIN OPENSSH PRIVATE KEY-----` and end with `-----END OPENSSH PRIVATE KEY-----`. Do not paste the `.pub` (public) file by mistake — that will cause "Error loading key ... invalid format" in the workflow.
6. Save.

Optional secrets (workflow has defaults):

| Name | Value | When to set |
|------|--------|-------------|
| `DEPLOY_VPS_HOST` | `188.116.23.7` | Only if your Staged VPS has a different IP. |
| `DEPLOY_VPS_USER` | `root` | Only if the deploy user is not `root`. |
| `GITEA_HOST_IP` | e.g. `172.20.0.1` | If checkout fails with "Could not resolve host: server", set this to the **default gateway** seen from the job container. Run a debug job that runs `ip route show default \| awk '{print $3}'` and use that value. The workflow defaults to `172.20.0.1` if unset. |

---

## 4. Add the workflow file to the repo and push it

The workflow is a YAML file that tells Gitea **what to do** when you push to `main`. It does not get “installed” anywhere except inside the repository.

**4.1 — What the file is and where it lives**

- **Path in repo:** `.gitea/workflows/deploy-vps.yaml`
- **What it does:** On every push to `main`, Gitea runs a job that: checks out the code → installs SSH/rsync → syncs the repo to the Staged VPS at `/opt/app/backend/honey-be` → runs `sudo ./scripts/rolling-update.staged.sh` on the VPS.
- Gitea only runs workflows that exist **on the branch you push**. So this file must be committed and pushed to `main`.

**4.2 — What you need to do**

**Where:** Local Mac (in your honey-be project directory).

1. Confirm the workflow file exists:

   ```bash
   cd /path/to/your/honey-be
   ls -la .gitea/workflows/deploy-vps.yaml
   ```

2. If it’s there, add it to git, commit, and push to `main`:

   ```bash
   git add .gitea/workflows/deploy-vps.yaml
   git commit -m "Add VPS deploy workflow"
   git push origin main
   ```

3. If the file is not there, create the directory and the file (the file contents are in this repo), then run the same `git add` / `git commit` / `git push` as above.

After the push, Gitea will see the workflow. It will run **only when** a runner is available (Step 5). So you can push the workflow first and set up the runner next, or the other way around.

---

## 5. Set up Gitea Actions and the runner

Gitea needs **Actions enabled** and at least one **runner** registered. The runner is the machine (or container) that actually runs the workflow steps. Your Gitea runs on a VPS with Docker; the runner is usually the **act_runner** container next to Gitea.

**5.1 — Enable Actions (if not already)**

**Where:** Gitea in the browser (admin or repo).

- **Site-wide:** Log in as admin → **Site Administration** → **Actions** → enable **Enable Actions**.
- Or at **repo level:** open **honey-be** → **Settings** → **Actions** → enable if there is an option.

**5.2 — Ensure the runner container is running**

**Where:** Gitea VPS (the server where Gitea’s Docker runs, e.g. 100.122.146.65 over Tailscale).

SSH into the Gitea server and check that both `gitea` and `gitea_runner` (or your runner container name) are up:

```bash
docker ps
```

You should see something like `gitea` and `gitea_runner`. If the runner container is stopped:

```bash
cd ~/gitea   # or wherever your docker-compose.yml is
docker compose up -d
```

**5.3 — Get the runner registration token from Gitea**

**Where:** Gitea in the browser.

1. Open the **honey-be** repository.
2. Go to **Settings** → **Actions** → **Runners**.
3. Click **Create new runner** (or **Add runner** / **Register runner**).
4. Gitea will show a **registration token** and often a command or instructions. Copy the token (you’ll use it in the next step if the runner is not already registered).

**5.4 — Register the runner (if it isn’t already)**

**Where:** Gitea VPS (SSH).

Your `docker-compose` must set **GITEA_INSTANCE_URL** to a URL the runner container can reach. Use the **Docker service name**, not `127.0.0.1`:

- **Use:** `GITEA_INSTANCE_URL=http://server:3000` (so the runner resolves `server` on the Docker network).
- **Do not use:** `GITEA_INSTANCE_URL=http://127.0.0.1:3000` — from inside the runner container, that is the container itself, so the runner stays "Offline" and never connects.

Your `docker-compose` may already set `GITEA_RUNNER_REGISTRATION_TOKEN` and the runner registers on startup. Check in Gitea: **Settings** → **Actions** → **Runners**. If you see a runner with status **Idle** and label **ubuntu-latest**, you can skip to 5.5.

If no runner appears (or it stays "Offline"), register or re-register on the Gitea VPS. If you changed `GITEA_INSTANCE_URL`, clear the runner’s persisted state first so it uses the new URL:

```bash
cd ~/gitea
docker compose stop runner
rm -rf ./data/runner/*
docker compose up -d --force-recreate runner
```

If you still need to register manually:

```bash
docker exec -it gitea_runner sh
```

Inside the container:

```bash
act_runner register --instance http://server:3000 --token PASTE_TOKEN_FROM_STEP_5.3
```

Then exit and restart the runner container so it runs the daemon:

```bash
exit
docker restart gitea_runner
```

**5.5 — Check that the runner has the right label**

**Where:** Gitea in the browser.

1. Go to **honey-be** → **Settings** → **Actions** → **Runners**.
2. You should see at least one runner with:
   - **Status:** Idle (or Running when a job is active)
   - **Labels:** must include **ubuntu-latest**, because the workflow file has `runs-on: ubuntu-latest`.

If your runner has a different label (e.g. `linux`), you have two options:

- **Option A:** In Gitea when registering the runner, add the label **ubuntu-latest** (if the UI lets you choose labels).
- **Option B:** Edit `.gitea/workflows/deploy-vps.yaml` and change the line `runs-on: ubuntu-latest` to your runner’s label (e.g. `runs-on: linux`), then commit and push.

**5.6 — Summary**

- Actions enabled in Gitea.
- Runner container running on the Gitea VPS.
- Runner registered and visible under **Settings** → **Actions** → **Runners** with label **ubuntu-latest** (or workflow updated to match your label).

---

## 6. Test the deployment

Do this **after** the workflow file is on `main` (Step 4) and the runner is set up (Step 5).

**6.1 — Trigger a run**

Push a commit to `main` (e.g. a small change or the workflow/docs you added):

```bash
cd /path/to/your/honey-be
git add .gitea/workflows/deploy-vps.yaml docs/GITEA_VPS_DEPLOY.md
git commit -m "Add VPS deploy workflow and guide"
git push origin main
```

**6.2 — Check the run in Gitea**

**Where:** Gitea in the browser.

1. Open **honey-be** → **Actions** (tab in the repo).
2. You should see a run for the “Deploy to VPS” workflow. Open it.
3. Confirm all steps are green. If something fails, open the failed step to see the log.

**6.3 — Check the Staged VPS**

**Where:** Staged VPS (188.116.23.7) or your Mac (SSH to VPS).

1. SSH in: `ssh root@188.116.23.7`
2. Check that code was updated: `ls -la /opt/app/backend/honey-be`
3. Check that the app restarted: `docker ps` (or your usual way to verify the backend is running).

---

## Troubleshooting

### Runner stays "Offline" / "Last Online Time: Never" / "Cannot ping the Gitea instance server"

- **Cause:** The runner container is trying to reach Gitea at `http://127.0.0.1:3000`. From inside the runner container, `127.0.0.1` is the container itself, not the Gitea server, so the connection is refused.
- **Fix:**
  1. In your **docker-compose** on the Gitea VPS, set `GITEA_INSTANCE_URL=http://server:3000` (use the Docker service name of the Gitea container, e.g. `server`).
  2. Clear the runner's persisted state and recreate the container so it re-registers with the new URL:
     ```bash
     cd ~/gitea
     docker compose stop runner
     rm -rf ./data/runner/*
     docker compose up -d --force-recreate runner
     ```
  3. In Gitea → **Settings** → **Actions** → **Runners**, the runner should show **Idle** within a few seconds. If you see a new runner and the old one still "Offline", you can remove the old one in the UI.

---

### Checkout fails: "Could not resolve host: server" / "fatal: unable to access 'http://server:3000/...'"

- **Cause:** The workflow job runs inside a **separate** Docker container (started by the runner). That job container is not on the same Docker network as Gitea, so the hostname `server` does not resolve there. Changing Gitea's `LOCAL_ROOT_URL` or the host's `/etc/hosts` does not help, because the job container has its own network.
- **Fix:** The deploy workflow in this repo already uses a **manual checkout** that clones via the runner host's IP instead of `server`. If checkout still fails:
  1. Add repository secret **GITEA_HOST_IP** with the default gateway as seen from the job container. To get it: run a one-off debug workflow that runs `ip route show default | awk '{print $3}'` and use the printed value (often `172.20.0.1` or `172.17.0.1`).
  2. If you don't set the secret, the workflow defaults to `172.20.0.1`; if your Docker uses a different gateway, set **GITEA_HOST_IP** to that value.

---

### Setup SSH fails: "Error loading key ... invalid format"

- **Cause:** The **DEPLOY_SSH_PRIVATE_KEY** secret in Gitea is not valid: pasted with wrong line breaks, truncated, or the **public** key (`.pub`) was pasted by mistake.
- **Fix:**
  1. On your Mac, verify the private key file: `ssh-keygen -y -f ~/.ssh/gitea_deploy_honey_be`. If that fails, generate a new key (Step 1) and add the new public key to the VPS and the new private key to Gitea.
  2. Copy the **full** private key again: `cat ~/.ssh/gitea_deploy_honey_be | pbcopy` (or open the file and copy everything).
  3. In Gitea → **Settings** → **Secrets and Variables** → **Actions**, edit **DEPLOY_SSH_PRIVATE_KEY** and paste the entire key. It must start with `-----BEGIN OPENSSH PRIVATE KEY-----` and end with `-----END OPENSSH PRIVATE KEY-----`, with no extra blank lines or spaces at the start/end. Make sure you are pasting the **private** key file, not the `.pub` file.
  4. Save and re-run the workflow.

---

### Other issues

- **Permission denied (publickey)** (during rsync or SSH to Staged VPS)  
  Check: public key in `authorized_keys` on the Staged VPS, private key in `DEPLOY_SSH_PRIVATE_KEY`, no extra spaces/newlines when pasting. From your Mac, test: `ssh -i ~/.ssh/gitea_deploy_honey_be root@188.116.23.7 "echo OK"`.

- **Runner doesn’t pick up the job**  
  In Gitea: **Settings** → **Actions** → **Runners**. Confirm runner is “idle” and has label `ubuntu-latest`.

- **rsync or SSH fails from the job**  
  The job runs on the Gitea VPS (in a container). Ensure the Gitea VPS can reach the Staged VPS (188.116.23.7) on port 22. From the Gitea VPS host, test: `ssh -i /path/to/private_key root@188.116.23.7 "echo OK"`.

- **sudo or script fails on VPS**  
  SSH in as the deploy user and run manually:  
  `cd /opt/app/backend/honey-be && sudo ./scripts/rolling-update.staged.sh`  
  Fix permissions or sudo rules as needed.