netbird-iac/README.md

# NetBird GitOps PoC

Proof-of-concept for managing NetBird VPN configuration via Infrastructure as Code (IaC) with GitOps workflow using Terraform.

## Project Status: POC Complete

**Start date:** 2026-02-15  
**Status:** Full automation implemented including peer auto-naming

### What Works

- [x] NetBird self-hosted instance deployed (`netbird-poc.networkmonitor.cc`)
- [x] Gitea CI/CD server deployed (`gitea-poc.networkmonitor.cc`)
- [x] Gitea Actions runner for CI/CD
- [x] Terraform implementation - creates groups, policies, setup keys
- [x] CI/CD pipeline - PR shows plan, merge-to-main applies changes
- [x] **Watcher service** - automatically renames peers based on setup key names

---

## Architecture

```
+-------------------+     PR/Merge      +-------------------+
|  Engineer         | ----------------> |  Gitea            |
|  (creates setup   |                   |  (CI/CD)          |
|   key: pilot-X)   |                   +-------------------+
+-------------------+                          |
                                               | terraform apply
                                               v
+-------------------+                   +-------------------+
|  Watcher Service  | <---- polls ----> |  NetBird API      |
|  (auto-rename)    |                   +-------------------+
+-------------------+                          ^
                                               | enrolls
+-------------------+                          |
|  Operator         | -------------------------+
|  (uses setup key) |   peer appears as "DESKTOP-XYZ"
+-------------------+   watcher renames to "pilot-X"
```

## Complete Workflow

1. **Ticket arrives:** "Onboard pilot Ivanov"
2. **Engineer adds to Terraform:**
   ```hcl
   resource "netbird_setup_key" "pilot_ivanov" {
     name        = "pilot-ivanov"  # <-- This becomes the peer name
     type        = "one-off"
     auto_groups = [netbird_group.pilots.id]
     usage_limit = 1
   }
   ```
3. **Engineer creates PR** -> CI runs `terraform plan`
4. **PR merged** -> CI runs `terraform apply` -> setup key created
5. **Engineer retrieves key:** `terraform output -raw pilot_ivanov_key`
6. **Engineer sends key to operator** (via secure channel)
7. **Operator enrolls** -> peer appears as `DESKTOP-ABC123`
8. **Watcher detects** consumed key, renames peer to `pilot-ivanov`
9. **Done** - peer is correctly named, no manual intervention

---

## Directory Structure

```
netbird-gitops-poc/
├── ansible/                    # Deployment playbooks
│   ├── caddy/                  # Shared reverse proxy
│   ├── gitea/                  # Standalone Gitea
│   ├── gitea-runner/           # Gitea Actions runner
│   ├── netbird/                # NetBird server
│   └── netbird-watcher/        # Peer renamer service
├── terraform/                  # Terraform configuration
│   ├── .gitea/workflows/       # CI/CD workflow
│   ├── main.tf
│   ├── groups.tf
│   ├── policies.tf
│   ├── setup_keys.tf
│   └── outputs.tf
├── watcher/                    # Watcher service source
│   ├── netbird_watcher.py
│   ├── netbird-watcher.service
│   └── README.md
├── README.md
└── PAIN_POINTS.md
```

---

## Deployment

### Prerequisites

- VPS with Docker
- DNS records pointing to VPS
- Ansible installed locally
- Terraform installed locally

### 1. Deploy Core Infrastructure

```bash
# NetBird
cd ansible/netbird
./generate-vault.sh
ansible-vault encrypt group_vars/vault.yml
ansible-playbook -i poc-inventory.yml playbook-ssl.yml --ask-vault-pass

# Gitea
cd ../gitea
ansible-playbook -i poc-inventory.yml playbook.yml

# Caddy (reverse proxy)
cd ../caddy
ansible-playbook -i poc-inventory.yml playbook.yml

# Gitea Runner
cd ../gitea-runner
ansible-playbook -i poc-inventory.yml playbook.yml -e vault_gitea_runner_token=<TOKEN>
```

### 2. Deploy Watcher Service

```bash
cd ansible/netbird-watcher
ansible-playbook -i poc-inventory.yml playbook.yml -e vault_netbird_token=<TOKEN>
```

### 3. Initialize Terraform

```bash
cd terraform
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars with NetBird PAT
terraform init
terraform apply
```

### 4. Configure Gitea

Push terraform directory to Gitea repo, configure secret `NETBIRD_TOKEN`.

---

## Adding a New Operator

1. Add setup key to `terraform/setup_keys.tf`:
   ```hcl
   resource "netbird_setup_key" "pilot_ivanov" {
     name        = "pilot-ivanov"
     type        = "one-off"
     auto_groups = [netbird_group.pilots.id]
     usage_limit = 1
     ephemeral   = false
   }
   
   output "pilot_ivanov_key" {
     value     = netbird_setup_key.pilot_ivanov.key
     sensitive = true
   }
   ```

2. Commit, push, merge PR

3. Retrieve key:
   ```bash
   terraform output -raw pilot_ivanov_key
   ```

4. Send key to operator

5. Operator enrolls -> watcher auto-renames peer

---

## Monitoring

### Watcher Service

```bash
# Status
systemctl status netbird-watcher

# Logs
journalctl -u netbird-watcher -f

# Processed keys
cat /var/lib/netbird-watcher/state.json
```

---

## Cleanup

```bash
# Destroy Terraform resources
cd terraform && terraform destroy

# Stop services on VPS
ssh root@observability-poc.networkmonitor.cc
systemctl stop netbird-watcher
cd /opt/caddy && docker compose down
cd /opt/gitea && docker compose down
cd /opt/netbird && docker compose down
```