Switch to terraform
This commit is contained in:
Prox
213
README.md
Normal file
213
README.md
Normal file
@@ -0,0 +1,213 @@
|
||||
# NetBird GitOps PoC
|
||||
|
||||
Proof-of-concept for managing NetBird VPN configuration via Infrastructure as Code (IaC) with GitOps workflow using Terraform.
|
||||
|
||||
## Project Status: POC Complete
|
||||
|
||||
**Start date:** 2026-02-15
|
||||
**Status:** Core functionality working, remaining pain points documented
|
||||
|
||||
### What Works
|
||||
|
||||
- [x] NetBird self-hosted instance deployed (`netbird-poc.networkmonitor.cc`)
|
||||
- [x] Gitea CI/CD server deployed (`gitea-poc.networkmonitor.cc`)
|
||||
- [x] Gitea Actions runner for CI/CD
|
||||
- [x] Terraform implementation - creates groups, policies, setup keys
|
||||
- [x] CI/CD pipeline - PR shows plan, merge-to-main applies changes
|
||||
|
||||
### Remaining Pain Points
|
||||
|
||||
See [PAIN_POINTS.md](./PAIN_POINTS.md) for detailed analysis of:
|
||||
- Peer naming automation (no link between setup keys and enrolled peers)
|
||||
- Per-user vs per-role setup keys
|
||||
- Secure key distribution to operators
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
+-------------------+ PR/Merge +-------------------+
|
||||
| Engineer | ----------------> | Gitea |
|
||||
| (edits .tf) | | (gitea-poc.*) |
|
||||
+-------------------+ +-------------------+
|
||||
|
|
||||
| CI/CD
|
||||
v
|
||||
+-------------------+
|
||||
| Terraform |
|
||||
| (in Actions) |
|
||||
+-------------------+
|
||||
|
|
||||
| API calls
|
||||
v
|
||||
+-------------------+ Enroll +-------------------+
|
||||
| Operators | ----------------> | NetBird |
|
||||
| (use setup keys) | | (netbird-poc.*) |
|
||||
+-------------------+ +-------------------+
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
netbird-gitops-poc/
|
||||
├── ansible/ # Deployment playbooks
|
||||
│ ├── caddy/ # Shared reverse proxy
|
||||
│ ├── gitea/ # Standalone Gitea (no OAuth)
|
||||
│ ├── gitea-runner/ # Gitea Actions runner
|
||||
│ └── netbird/ # NetBird with embedded IdP
|
||||
├── terraform/ # Terraform configuration (Gitea repo content)
|
||||
│ ├── .gitea/workflows/ # CI/CD workflow
|
||||
│ │ └── terraform.yml
|
||||
│ ├── main.tf # Provider config
|
||||
│ ├── variables.tf # Input variables
|
||||
│ ├── groups.tf # Group resources
|
||||
│ ├── policies.tf # Policy resources
|
||||
│ ├── setup_keys.tf # Setup key resources
|
||||
│ ├── outputs.tf # Output values
|
||||
│ ├── terraform.tfstate # State (committed for POC)
|
||||
│ ├── terraform.tfvars # Secrets (gitignored)
|
||||
│ └── terraform.tfvars.example
|
||||
├── README.md
|
||||
└── PAIN_POINTS.md
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- VPS with Docker
|
||||
- DNS records pointing to VPS
|
||||
- Ansible installed locally
|
||||
- Terraform installed locally (for initial setup)
|
||||
|
||||
### 1. Deploy Infrastructure
|
||||
|
||||
```bash
|
||||
# 1. NetBird (generates secrets, needs vault password)
|
||||
cd ansible/netbird
|
||||
./generate-vault.sh
|
||||
ansible-vault encrypt group_vars/vault.yml
|
||||
ansible-playbook -i poc-inventory.yml playbook-ssl.yml --ask-vault-pass
|
||||
|
||||
# 2. Gitea
|
||||
cd ../gitea
|
||||
ansible-playbook -i poc-inventory.yml playbook.yml
|
||||
|
||||
# 3. Caddy (reverse proxy for both)
|
||||
cd ../caddy
|
||||
ansible-playbook -i poc-inventory.yml playbook.yml
|
||||
|
||||
# 4. Gitea Runner (get token from Gitea Admin -> Actions -> Runners)
|
||||
cd ../gitea-runner
|
||||
ansible-playbook -i poc-inventory.yml playbook.yml -e vault_gitea_runner_token=<TOKEN>
|
||||
```
|
||||
|
||||
### 2. Initial Terraform Setup (Local)
|
||||
|
||||
```bash
|
||||
cd terraform
|
||||
|
||||
# Create tfvars with your NetBird PAT
|
||||
cp terraform.tfvars.example terraform.tfvars
|
||||
# Edit terraform.tfvars with actual token
|
||||
|
||||
# Initialize and apply
|
||||
terraform init
|
||||
terraform apply
|
||||
```
|
||||
|
||||
### 3. Push to Gitea
|
||||
|
||||
```bash
|
||||
cd terraform
|
||||
git init
|
||||
git add .
|
||||
git commit -m "Initial Terraform config"
|
||||
git remote add origin git@gitea-poc.networkmonitor.cc:admin/netbird-iac.git
|
||||
git push -u origin main
|
||||
```
|
||||
|
||||
### 4. Configure Gitea Secrets
|
||||
|
||||
In Gitea repository Settings -> Actions -> Secrets:
|
||||
- `NETBIRD_TOKEN`: Your NetBird PAT
|
||||
|
||||
### 5. Make Changes via GitOps
|
||||
|
||||
Edit Terraform files locally, push to create PR:
|
||||
|
||||
```hcl
|
||||
# groups.tf - add a new group
|
||||
resource "netbird_group" "new_team" {
|
||||
name = "new-team"
|
||||
}
|
||||
```
|
||||
|
||||
```bash
|
||||
git checkout -b add-new-team
|
||||
git add groups.tf
|
||||
git commit -m "Add new-team group"
|
||||
git push -u origin add-new-team
|
||||
# Create PR in Gitea -> CI runs terraform plan
|
||||
# Merge PR -> CI runs terraform apply
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## CI/CD Workflow
|
||||
|
||||
The `.gitea/workflows/terraform.yml` workflow:
|
||||
|
||||
| Event | Action |
|
||||
|-------|--------|
|
||||
| Pull Request | `terraform plan` (preview changes) |
|
||||
| Push to main | `terraform apply` (apply changes) |
|
||||
| After apply | Commit updated state file |
|
||||
|
||||
**State Management:** State is committed to git (acceptable for single-operator POC). For production, use a remote backend.
|
||||
|
||||
---
|
||||
|
||||
## Key Discoveries
|
||||
|
||||
### NetBird API Behavior
|
||||
|
||||
1. **Peer IDs are not predictable** - Generated server-side at enrollment time
|
||||
2. **No setup key -> peer link** - NetBird doesn't record which setup key enrolled a peer
|
||||
3. **Peers self-enroll** - Cannot create peers via API (WireGuard keypair generated locally)
|
||||
4. **Terraform URL format** - Use `https://domain.com` NOT `https://domain.com/api`
|
||||
|
||||
---
|
||||
|
||||
## Credentials Reference (POC Only)
|
||||
|
||||
| Service | Credential | Location |
|
||||
|---------|------------|----------|
|
||||
| NetBird PAT | `nbp_T3yD...` | Dashboard -> Team -> Service Users |
|
||||
| Gitea | admin user | Created during setup |
|
||||
| VPS | root | `observability-poc.networkmonitor.cc` |
|
||||
|
||||
**Warning:** Rotate all credentials before any production use.
|
||||
|
||||
---
|
||||
|
||||
## Cleanup
|
||||
|
||||
```bash
|
||||
# Destroy Terraform resources
|
||||
cd terraform
|
||||
terraform destroy
|
||||
|
||||
# Stop VPS services
|
||||
ssh root@observability-poc.networkmonitor.cc
|
||||
cd /opt/caddy && docker compose down
|
||||
cd /opt/gitea && docker compose down
|
||||
cd /opt/netbird && docker compose down
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
See [PAIN_POINTS.md](./PAIN_POINTS.md) for remaining challenges to address before production use.
|
||||
Reference in New Issue
Block a user