> For the complete documentation index, see [llms.txt](https://docs.openg2p.org/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.openg2p.org/operations/deployment/single-node-automation.md).

# Sandbox — Single-Node

The entire deployment process for a single-node setup has been automated and available as shell scripts. This is useful for bringing up an OpenG2P sandbox on your own machine — everything (K8s, Istio, Rancher, Nginx, per-environment Keycloak, environments) runs on a single VM.

<figure><img src="/files/5gzAXkeqVQ69ODYWwHlc" alt=""><figcaption><p>Single-node architecture — all services on one VM</p></figcaption></figure>

{% hint style="info" %}
**No staged flow here.** Sandbox collapses what Production splits into five stages — procurement, provisioning, infrastructure, environment, and modules — into a single VM with two scripts. The [staged Production rollout](/operations/deployment/infrastructure-setup.md#the-five-stages) only applies when deploying across role-specialised nodes for production.
{% endhint %}

{% hint style="info" %}
For adding environments to an existing multi-node infrastructure, see [Environment Setup for Multi-Node](/operations/deployment/infrastructure-setup/environment-setup-multi-node.md).
{% endhint %}

{% hint style="success" %}
**Just want to run it?** Jump to [Quick Start](#quick-start). For the deployment model and how single-node compares to production, see [OpenG2P Deployment Architecture](/operations/deployment/openg2p-deployment-model.md#sandbox-single-node).
{% endhint %}

## Overview

Automated single-node deployment of the complete OpenG2P platform — from bare Ubuntu to running modules. Two scripts handle the entire lifecycle:

| Script                   | Purpose                                                                                 | Run when             |
| ------------------------ | --------------------------------------------------------------------------------------- | -------------------- |
| `openg2p-infra.sh`       | Base infrastructure (K8s, Istio, Rancher, monitoring, logging via OpenTelemetry + Loki) | Once per machine     |
| `openg2p-environment.sh` | Environment + modules (namespace, commons, Registry, PBMS, etc.)                        | Once per environment |

{% hint style="info" %}
The source code for all automation scripts lives in the [`openg2p-deployment`](https://github.com/OpenG2P/openg2p-deployment) repository under `automation/single-node/`.
{% endhint %}

{% hint style="info" %}
Cluster logging uses **OpenTelemetry + Grafana Loki** (no Fluentd/OpenSearch). It is installed automatically by `openg2p-infra.sh`, collects logs from all pods, and is viewed in Grafana — including a ready-made **OpenG2P — Logs & Health** dashboard. See [System Monitoring](/platform/platform-services/system-monitoring.md) for architecture and usage.
{% endhint %}

### Authentication Architecture

There is **no Keycloak at the infrastructure level**. Rancher uses its own **local authentication** — administrators create Rancher users directly in the Rancher UI (see [User Access & Roles](#user-access--roles)).

Keycloak is installed **per environment** for the OpenG2P applications:

| Component            | Deployed by                  | Namespace             | Authentication           | Purpose                                                                                  |
| -------------------- | ---------------------------- | --------------------- | ------------------------ | ---------------------------------------------------------------------------------------- |
| **Rancher**          | `openg2p-infra.sh` Phase 2   | `cattle-system`       | Local users (no SSO)     | Cluster management UI — create users directly in Rancher                                 |
| **Per-env Keycloak** | `openg2p-commons-base` chart | environment namespace | `keycloak.<base_domain>` | Auth for all OpenG2P services (Superset, Kafka UI, MinIO, ODK, etc.) in that environment |

The per-environment Keycloak has its own admin account, database, users, realms, and clients. Its admin credentials are auto-generated by the chart and stored in the `commons-keycloak` K8s secret. It is unrelated to Rancher login.

## Domain & TLS

This is a **local-only sandbox** — no public domain names, DNS provider, or Let's Encrypt are involved. The infrastructure script always:

* Installs `dnsmasq` on the VM to resolve `*.<local_domain>` to the VM's IP (default `openg2p.test`)
* Generates a local Certificate Authority and self-signed certificates
* Configures a Wireguard VPN with split tunnel (only cluster traffic routed through the VPN)
* Pushes the VM as a DNS resolver to Wireguard peers, so they resolve hostnames automatically
* Auto-derives all hostnames:
  * Infra: `rancher.openg2p.test`
  * Per-env: `keycloak.<env>.openg2p.test`, `superset.<env>.openg2p.test`, etc.

You only set `local_domain` (optional — defaults to `openg2p.test`). There is no custom/production domain mode in this automation.

## Access Model — Private by Default

The sandbox is **not exposed to the public Internet by default, even if the VM has a public IP.** The host firewall (and the AWS security group) restrict the web ports (80/443) so they are reachable only:

* over **Wireguard** (the recommended path — DNS resolves automatically), or
* from **inside the VPC**.

Administrative/data-plane ports (Kubernetes API, NodePorts, etcd, NFS) are always VPC/Wireguard-only, so `kubectl`/`helm` access requires the VPN.

{% hint style="danger" %}
**Opening the sandbox to the public Internet (`public_access: true`)** exposes the Rancher cluster-admin UI and every environment service to anyone who can reach the public IP, protected only by a self-signed certificate and local passwords. This is a sandbox, not a hardened deployment. Only enable it if you understand and accept the risk — and prefer restricting the source IPs (ufw / security group) to the specific addresses that need access. See [Optional: Public Access](#optional-public-direct-access-without-wireguard).
{% endhint %}

## Prerequisites

| Requirement  | Needed                                                |
| ------------ | ----------------------------------------------------- |
| **VM**       | Ubuntu 24.04 LTS, 16 vCPU, 64 GB RAM, 128 GB SSD      |
| **Access**   | Root/sudo on the VM                                   |
| **Internet** | Required for downloading packages and Helm charts     |
| **DNS**      | Not needed — dnsmasq + your laptop handle it          |
| **TLS**      | Not needed — local CA handles it                      |
| **VPN**      | Wireguard client on your laptop (default access path) |

## Quick Start

### Step 1: Infrastructure Setup

SSH into the VM as root:

```bash
git clone https://github.com/OpenG2P/openg2p-deployment.git
cd openg2p-deployment/automation/single-node
cp infra-config.example.yaml infra-config.yaml
# Edit infra-config.yaml — at minimum set node_ip
sudo chmod +x openg2p-infra.sh
sudo ./openg2p-infra.sh --config infra-config.yaml
```

{% tabs %}
{% tab title="Minimal" %}
Only `node_ip` is required — everything else has defaults:

```yaml
node_ip: "172.16.0.10"       # Your VM's private IP
cluster_name: "openg2p"      # Display name in Rancher UI
node_name: "node1"           # K8s node name
local_domain: "openg2p.test" # Optional — this is the default
public_access: false         # Private by default (see Access Model)
```

{% endtab %}

{% tab title="AWS EC2" %}
{% hint style="success" %}
**Recommended instance type: `m5a.4xlarge`** — 16 vCPU / 64 GB RAM with AMD EPYC processors. Meets the OpenG2P single-node resource requirements at the lowest hourly cost among comparable instance types (cheaper than `m5.4xlarge`, `c5.4xlarge`, or `m6a.4xlarge` for the same vCPU/RAM). Pair it with a 128 GB gp3 EBS volume.
{% endhint %}

For AWS, also set the public IP for Wireguard:

```yaml
node_ip: "172.16.0.10"       # Private IP
wireguard:
  endpoint: "54.x.x.x"       # Public IP for VPN clients
```

Before running the script, create the security group:

```bash
cd automation/single-node/aws
./create-security-group.sh --vpc-id vpc-xxxxxxxxx
```

After, attach it and disable source/dest check:

```bash
aws ec2 modify-instance-attribute --instance-id i-xxx --groups sg-xxx
aws ec2 modify-instance-attribute --instance-id i-xxx --no-source-dest-check
```

{% endtab %}
{% endtabs %}

Takes \~15-25 minutes. Idempotent — re-run on failure.

### Step 2: Environment Setup

After infrastructure is ready, create environments:

```bash
cp env-config.example.yaml env-config.yaml
# Edit env-config.yaml
sudo ./openg2p-environment.sh --config env-config.yaml
```

{% tabs %}
{% tab title="Single Environment" %}
Everything is auto-derived from the infra config:

```yaml
environment: "dev"
infra_config: "infra-config.yaml"
modules:
  commons: true
```

This creates namespace `dev` with domain `dev.openg2p.test`.

{% hint style="info" %}
Each environment gets its own Keycloak at `https://keycloak.<base_domain>` (e.g., `keycloak.dev.openg2p.test`), deployed by the `openg2p-commons-base` chart. There is no separate per-env Keycloak credential configuration — the chart handles it.
{% endhint %}
{% endtab %}

{% tab title="Multiple Environments" %}
Run the script multiple times with different configs:

```bash
sudo ./openg2p-environment.sh --config env-dev.yaml    # dev.openg2p.test
sudo ./openg2p-environment.sh --config env-qa.yaml     # qa.openg2p.test
sudo ./openg2p-environment.sh --config env-pilot.yaml  # pilot.openg2p.test
```

{% endtab %}
{% endtabs %}

Takes \~15-20 minutes per environment.

## Post-Infrastructure Steps (on your laptop)

After the infra script completes, follow these steps to access the cluster.

### 1. Wireguard VPN

```bash
# On the VM:
sudo cp /etc/wireguard/peers/peer1/peer1.conf /tmp/
sudo chmod 644 /tmp/peer1.conf

# On your laptop:
scp -i <your-key.pem> <user>@<public-ip>:/tmp/peer1.conf .
```

Import `peer1.conf` into the [Wireguard client app](https://www.wireguard.com/install/) and activate the tunnel.

{% hint style="info" %}
The default is **split tunnel** — only Wireguard subnet + VPC traffic routes through the VPN. Your internet stays direct and fast.
{% endhint %}

### 2. DNS Resolution (split tunnel only)

{% tabs %}
{% tab title="macOS" %}

```bash
sudo mkdir -p /etc/resolver
echo "nameserver <node_ip>" | sudo tee /etc/resolver/<local_domain>
# e.g.: echo "nameserver 172.29.8.137" | sudo tee /etc/resolver/openg2p.test
```

> **Note:** `dig` bypasses the macOS resolver system. Use `dscacheutil -q host -a name rancher.openg2p.test` or `ping` to verify.
> {% endtab %}

{% tab title="Windows" %}
Run in PowerShell as Administrator:

```powershell
Add-DnsClientNrptRule -Namespace ".<local_domain>" -NameServers "<node_ip>"
# e.g.: Add-DnsClientNrptRule -Namespace ".openg2p.test" -NameServers "172.29.8.137"
```

{% endtab %}

{% tab title="Linux" %}

```bash
sudo resolvectl dns wg0 <node_ip>
sudo resolvectl domain wg0 '~<local_domain>'
```

{% endtab %}
{% endtabs %}

### 3. CA Certificate

Copy `/etc/openg2p/ca/ca.crt` from the VM to your laptop, then install:

{% tabs %}
{% tab title="macOS" %}

```bash
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ca.crt
```

Or double-click `ca.crt` → System Settings → General → Profiles.
{% endtab %}

{% tab title="Windows" %}
Double-click `ca.crt` → Install Certificate → Local Machine → "Trusted Root Certification Authorities"
{% endtab %}

{% tab title="Linux" %}

```bash
sudo cp ca.crt /usr/local/share/ca-certificates/openg2p-ca.crt
sudo update-ca-certificates
```

{% endtab %}
{% endtabs %}

### 4. kubectl / helm Access

```bash
# On the VM:
sudo cp /etc/rancher/rke2/rke2-remote.yaml /tmp/
sudo chmod 644 /tmp/rke2-remote.yaml

# On your laptop:
scp -i <your-key.pem> <user>@<public-ip>:/tmp/rke2-remote.yaml ~/.kube/openg2p-config
export KUBECONFIG=~/.kube/openg2p-config
kubectl get nodes
```

{% hint style="warning" %}
Requires Wireguard VPN to be active — the K8s API is on the private IP.
{% endhint %}

### 5. Login to Rancher

Rancher uses **local authentication** — there is no Keycloak SSO for Rancher.

Open Rancher at `https://rancher.<domain>` and log in as the local admin:

* **Username:** `admin`
* **Password:** retrieve it from the cluster:

```bash
sudo KUBECONFIG=/etc/rancher/rke2/rke2.yaml kubectl -n cattle-system \
  get secret rancher-secret -o jsonpath='{.data.adminPassword}' | base64 -d && echo
```

{% hint style="info" %}
**Additional Rancher users must be created directly in Rancher.** There is no external identity provider — see [User Access & Roles](#user-access--roles) below.
{% endhint %}

## Optional: Public (Direct) Access Without Wireguard

By default the sandbox is reachable only over Wireguard or from inside the VPC. If you want to reach it **directly over a public IP — without the VPN** — you can opt in. The client just needs to map the hostnames to the public IP and accept the self-signed certificate.

{% hint style="danger" %}
**This exposes the Rancher cluster-admin UI and every environment service to the public Internet**, protected only by a self-signed certificate and local passwords. This is a sandbox, not a hardened deployment. Prefer restricting the opened ports to the specific source IPs that need access (see step 1/2 notes), and never enable this for a deployment holding real data.
{% endhint %}

**On the server — open the web ports:**

1. Set `public_access: true` in `infra-config.yaml` and re-run phase 1 (this rewrites the firewall; it is the durable way — a manually-added `ufw` rule would be wiped on the next run):

   ```bash
   sudo ./openg2p-infra.sh --config infra-config.yaml --phase 1
   ```
2. **On AWS, also open 80/443 in the security group.** Either recreate it with `./create-security-group.sh --vpc-id <VPC_ID> --public-web`, or add the rules to the existing group (ideally scoped to your client's IP rather than `0.0.0.0/0`):

   ```bash
   aws ec2 authorize-security-group-ingress --group-id sg-xxx \
     --protocol tcp --port 443 --cidr <your-client-ip>/32
   aws ec2 authorize-security-group-ingress --group-id sg-xxx \
     --protocol tcp --port 80  --cidr <your-client-ip>/32
   ```

**On the client laptop — resolve the hostnames + trust the CA:**

3. Add `/etc/hosts` entries pointing each hostname at the **public** IP (one line per environment hostname — wildcards are not supported in `/etc/hosts`):

   ```
   <public-ip>  rancher.openg2p.test
   <public-ip>  keycloak.dev.openg2p.test  superset.dev.openg2p.test
   ```
4. Install/trust the CA certificate (`/etc/openg2p/ca/ca.crt`, see step 3 above) or click through the browser warning. Always browse by **hostname** — the cert has no IP SAN, so `https://<public-ip>` will fail validation.

No other changes are required: Nginx already listens on all interfaces and routes by hostname, so once the firewall/security-group allow the traffic and the client resolves the names, access works exactly as it does over the VPN.

## User Access & Roles

Rancher uses **local authentication** — all users that need access to the Rancher UI must be **created directly in Rancher**. There is no Keycloak/SSO integration for Rancher.

**To create a Rancher user:**

1. Log in to Rancher as the local `admin`.
2. Go to **☰ → Users & Authentication → Users → Create**.
3. Set a username and password, and assign a global role (e.g. *Standard User*, or *Administrator* for a super admin).

Rancher ships with built-in project roles, but all include full Secrets access. The automation script creates two additional custom roles that exclude secrets:

| Role                               | Source                | Secrets Access | Permissions                                   |
| ---------------------------------- | --------------------- | -------------- | --------------------------------------------- |
| **Project Owner**                  | Rancher built-in      | Full           | Full control of the project                   |
| **Project Member**                 | Rancher built-in      | Full           | CRUD on workloads, services, configs, secrets |
| **Project Member (No Secrets)**    | Created by automation | None           | Same as Project Member, minus secrets         |
| **Project Read-Only (No Secrets)** | Created by automation | None           | View-only, no secrets                         |

**To give a user access to an environment:**

1. Create the user in **Rancher** (☰ → Users & Authentication → Users → Create), if not already created.
2. In **Rancher**, go to Project (environment) → Members → Add Member.
3. Search for the user and assign a role.

{% hint style="info" %}
The Rancher `admin` global role (super admin) has access to everything. The initial admin user configured during setup already has this role.
{% endhint %}

## Environment Setup Details

### Phase 1: Environment Infrastructure

| Step | What                   | Details                                                    |
| ---- | ---------------------- | ---------------------------------------------------------- |
| E1.1 | Validate prerequisites | Infra completed, kubeconfig works, base domain available   |
| E1.2 | TLS certificate        | Wildcard cert for `*.<base_domain>` signed by the local CA |
| E1.3 | Nginx server block     | `*.dev.openg2p.test` → Istio ingress                       |
| E1.4 | K8s namespace          | Creates the namespace                                      |
| E1.5 | Rancher Project        | Creates project and moves namespace into it (RBAC)         |
| E1.6 | Istio Gateway          | Gateway resource for hostname routing                      |

### Phase 2: Module Installation

openg2p-commons is split into two Helm charts installed sequentially:

| Step       | Chart                            | Details                                                                       |
| ---------- | -------------------------------- | ----------------------------------------------------------------------------- |
| E2.1       | **openg2p-commons-base**         | PostgreSQL, Kafka, MinIO, Redis, SoftHSM, **per-env Keycloak**, keycloak-init |
| E2.2       | **openg2p-commons-services**     | eSignet, KeyManager, Superset, ODK, master-data, reporting                    |
| *(future)* | Registry, PBMS, SPAR, G2P Bridge | Will be added as separate Helm installs                                       |

{% hint style="info" %}
The services chart automatically connects to base infrastructure via release name references (`commons-postgresql`, `commons-redis`, etc.).
{% endhint %}

## Command Reference

### Infrastructure

```bash
sudo ./openg2p-infra.sh --config infra-config.yaml              # Full setup
sudo ./openg2p-infra.sh --config infra-config.yaml --phase 1    # Host setup only
sudo ./openg2p-infra.sh --config infra-config.yaml --phase 2    # Helmfile only
sudo ./openg2p-infra.sh --config infra-config.yaml --phase 3    # Rancher config only (local admin, cluster name, roles, catalog repo)
sudo ./openg2p-infra.sh --config infra-config.yaml --force       # Re-run everything
sudo ./openg2p-infra.sh --config infra-config.yaml --dry-run     # Preview
sudo ./openg2p-infra.sh --reset                                   # Clear state markers
```

### Environment

```bash
sudo ./openg2p-environment.sh --config env-config.yaml              # Full setup
sudo ./openg2p-environment.sh --config env-config.yaml --phase 1    # Infrastructure only
sudo ./openg2p-environment.sh --config env-config.yaml --phase 2    # Module install only
sudo ./openg2p-environment.sh --config env-config.yaml --force       # Re-run everything
```

### Uninstalling

**Remove a single environment** (keeps infrastructure intact):

```bash
sudo ./openg2p-environment-uninstall.sh --config env-config.yaml
```

**Remove the entire infrastructure** (destroys everything):

```bash
sudo ./openg2p-infra-uninstall.sh
```

{% hint style="danger" %}
Infrastructure uninstall requires typing `DELETE EVERYTHING` to confirm. Removes: RKE2 cluster, Wireguard VPN, dnsmasq, Nginx, NFS, TLS certificates, and all state. The VM is left clean for a fresh installation.
{% endhint %}

## File Structure

```
automation/single-node/
├── openg2p-infra.sh                  # Script 1: base infrastructure
├── openg2p-infra-uninstall.sh        # Uninstall: tears down entire infra
├── infra-config.example.yaml         # Config for Script 1
├── helmfile-infra.yaml.gotmpl        # Helmfile for platform components
├── openg2p-environment.sh            # Script 2: environment setup
├── openg2p-environment-uninstall.sh  # Uninstall: removes a single environment
├── env-config.example.yaml           # Config for Script 2
├── lib/
│   ├── utils.sh          # Shared: logging, state, config, wait helpers
│   ├── phase1.sh         # Infra Phase 1: tools, firewall (public_access), RKE2, Wireguard, NFS, DNS, TLS, Nginx
│   ├── phase2.sh         # Infra Phase 2: Istio, Helmfile sync
│   ├── phase3.sh         # Infra Phase 3: Rancher config (local admin, cluster name, roles, catalog repo)
│   ├── env-phase1.sh     # Env Phase 1: certs, Nginx, namespace, Rancher project, Istio GW
│   └── env-phase2.sh     # Env Phase 2: commons helm install (future: more modules)
├── aws/
│   ├── create-security-group.sh   # Creates "openg2p-single-node" SG (--public-web to expose 80/443)
│   └── security-group.json        # Reference: exported SG rules
└── charts/
    ├── raw/               # Minimal chart for raw K8s manifests
    └── istio-install/     # Istio operator YAML
```

## Troubleshooting

{% hint style="info" %}
**Script failed?** Re-run it. Completed steps are skipped. Error messages include diagnostic commands.
{% endhint %}

**Local DNS not resolving on your laptop?** Ensure Wireguard VPN is connected. Configure per-domain DNS on your laptop (see Step 2 above). On macOS, `dig` bypasses the resolver system — use `ping` or `dscacheutil` to test.

**Browser shows certificate warning?** The sandbox uses a self-signed CA — install the CA certificate on your laptop (see Step 3 above) to clear it.

**Check cluster status:**

```bash
kubectl get nodes                              # Node health
kubectl get pods -A | grep -v Running          # Problem pods
helm list -A                                    # Helm releases
journalctl -u rke2-server -n 50               # RKE2 logs
```

{% hint style="info" %}
This automation does not replace the Rancher UI. Your existing umbrella Helm charts with `questions.yml` continue to work for manual installs via the Rancher App Catalog.

Infra Phase 3 pre-registers the **OpenG2P Helm repository** (`https://openg2p.github.io/openg2p-helm/rancher`) as a Rancher catalog ClusterRepo named `openg2p`, so the OpenG2P charts are available out of the box under **Rancher UI → Apps → Repositories / Charts**.
{% endhint %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.openg2p.org/operations/deployment/single-node-automation.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.