Introduction

The guide here provides instructions to deploy OpenG2P on Kubernetes (K8s) cluster.

Pre-requisites

• K8s cluster is set up as given here.

Installation of OpenG2P

• This section assumes the OpenG2P docker is already packaged. See Packaging Instructions.

• Clone the https://github.com/OpenG2P/openg2p-packaging and go to charts/openg2p directory

  • Run, (This installs the ref-impl dockers):

    Copy

    ./install.sh \
        --set global.hostname=openg2p.sandbox.net \
        --set global.selfServiceHostname=selfservice.openg2p.sandbox.net

  • If use different docker image or tag use:

    Copy

    ./install.sh \
        --set odoo.image.repository=<docker image name> \
        --set odoo.image.tag=<docker image tag> \
        --set global.hostname=openg2p.sandbox.net \
        --set global.selfServiceHostname=selfservice.openg2p.sandbox.net

Installation of ODK

• From the charts/odk-central directory, run the following to install ODK helm chart.

  Copy

  ./install.sh \
      --set global.hostname=openg2p.sandbox.net

• Note: The above helm chart uses the following docker images built from https://github.com/getodk/central/tree/v2023.1.0, since ODK Central doesn't provide pre-built docker images for these.

  Copy

  openg2p/odk-central-backend:v2023.1.0
  openg2p/odk-central-frontend:v2023.1.0
  openg2p/odk-central-enketo:v2023.1.0

• Post installation:

  • Exec into the service pod, and create user (and promote if required).

    Copy

    kubectl exec -it <service-pod> -- odk-cmd -u <email> user-create
    kubectl exec -it <service-pod> -- odk-cmd -u <email> user-promote

• Uninstallation:

  • To uninstall, just delete the helm installation of odk-central. Example:

    Copy

    helm -n odk delete odk-central

Hardware requirements

For sandbox setups

For staging setups

For production setups

TBD

Networking requirements

• All the machines in the same network.

• Public IP assigned to the Wireguard machine.

DNS requirements

The following domain names and mappings will be required. Examples:

Certificate requirements

One wildcard certificate is required at least, depending on the above domain names used. This can also be generated using letsencrypt.

Introduction

The following guide uses RKE2 to set up the Kubernetes (K8s) cluster.

Prerequisites

• The requirements for setting up the cluster are met as given here.

• The following tools are installed on all the nodes and the client machine.

  • ufw , wget , curl , kubectl , istioctl , helm , jq

Firewall setup

• Set up firewall rules on each node. The following uses ufw to setup firewall.

  • SSH into each node, and change to superuser.

  • Run the following command for each rule in the following table

  Copy

  ufw allow from <from-ip-range-allowed> to any port <port/range> proto <tcp/udp>

  • Example

  Copy

  ufw allow from any to any port 22 proto tcp
  ufw allow from 10.3.4.0/24 to any port 9345 proto tcp

  • Enable ufw.

  Copy

  ufw enable
  ufw default deny incoming

  • Additional Reference: RKE2 Networking Requirements

K8s setup

• The following setup has to be done for each cluster node.

• Choose odd number of server nodes. Example if there are 3 nodes, choose 1 server node and two agent nodes. If there are 7 nodes, choose 3 server nodes and 4 agent nodes.

• Clone the https://github.com/OpenG2P/openg2p-packaging and go to infra directory.

• For the first server node:

  • Configure rke2-server.conf.primary.template,

  • SSH into the node. Place the file to this path: /etc/rancher/rke2/config.yaml. Create the directory if not present already. mkdir -p /etc/rancher/rke2 .

  • Run this to download rke2.

    Copy

    curl -sfL https://get.rke2.io | sh -

  • Run this to start rke2 server:

    Copy

    systemctl enable rke2-server
    systemctl start rke2-server

• For subsequent server and agent nodes:

  • Configure rke2-server.conf.subsequent.template or rke2-agent.conf.template, with relevant ips for each node.

  • SSH into each node place the relevant file to this path: /etc/rancher/rke2/config.yaml, based on whether its a worker node, or control-plane node. (If worker use agent file. If control-plane use server file).

  • Run this to get download rke2.

    Copy

    curl -sfL https://get.rke2.io | sh -

  • To start rke2, use this

    Copy

    systemctl enable rke2-server
    systemctl start rke2-server

    or, based on server or agent.

    Copy

    systemctl enable rke2-agent
    systemctl start rke2-agent

• Execute these commands on a server node.

  • Copy

    echo -e 'export PATH="$PATH:/var/lib/rancher/rke2/bin"\nexport KUBECONFIG="/etc/rancher/rke2/rke2.yaml"' >> ~/.bashrc
    source ~/.bashrc

  • Copy

    kubectl get nodes

• Additional Reference: RKE2 High Availabilty Installation

Cluster import into Rancher.

• This section assumes a Rancher server has already been setup and operational. Rancher Server Setup in case not already done.

• Navigate to Cluster Management section in Rancher.

• Click on Import Existing cluster. And follow the steps to import the newly created cluster.

• After Rancher import, do not use the the kubeconfig from server anymore. Use it only via downloading kubeconfig from rancher.

Longhorn setup

• Use this to install longhorn. Longhorn Install as a Rancher App

Istio setup

• The following setup can be done from the client machine. This install Istio Operator, Istio Service Mesh, Istio Ingressgateway components.

• From infra directory, configure the istio-operator.yaml, and run;

  Copy

  istioctl operator init
  kubectl apply -f istio-operator.yaml

• Gather Wildcard TLS certificate and key and run;

  Copy

  kubectl create secret tls tls-openg2p-ingress -n istio-system \
      --cert=<CERTIFICATE PATH> \
      --key=<KEY PATH>

• Create istio gateway for all hosts using this command:

  Copy

  kubectl apply -f istio-gateway.yaml

Adding new nodes

• From infra directory, take either the rke2-server.conf.subsequent.template or rke2-agent.conf.template based on whether the new node is control plane node or Worker node. Copy this file to /etc/rancher/rke2/config.yaml in the new node.

• Configure the the config.yaml with relevant values.

• Run this to download rke2.

  Copy

  curl -sfL https://get.rke2.io | sh -

• Run this to start rke2 node:

  Copy

  systemctl enable rke2-server
  systemctl start rke2-server

Introduction

NFS-based storage is recommended for backing up DB and other data in developer/UAT/pilot environments.

Introduction

Rancher is used to managing multiple clusters. Being a critical component of cluster administration it is highly recommended that Rancher itself runs on a Kubernetes cluster with sufficient replication for high availability and avoiding a single point of failure.

Kubernetes cluster setup

• Set up a new RKE2 cluster. Refer to the guide.

  • Do not remove the stock ingress controller in the server config.

  • No need to install Istio.

Rancher installation

• To install Rancher use this (hostname to be edited in the below command):

  Copy

  helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
  helm repo update
  helm install rancher rancher-latest/rancher \
    --namespace cattle-system \
    --create-namespace \
    --set hostname=rancher.openg2p.org \
    --set ingress.tls.source=tls-rancher-ingress

  • Configure/Create TLS secret accordingly.

  Copy

  kubectl create secret tls tls-rancher-ingress -n cattle-system \
      --cert=path/to/cert/file \
      --key=path/to/key/file

Longhorn Setup

Keycloak setup

• Copy

  helm repo add bitnami https://charts.bitnami.com/bitnami
  helm repo update
  helm install keycloak bitnami/keycloak \
    -n keycloak \
    --create-namespace \
    --version "7.1.18" \
    --set ingress.hostname=keycloak.openg2p.org \
    --set ingress.extraTls[0].hosts[0]=keycloak.openg2p.org \
    -f rancher-keycloak-values.yaml

Integrate Rancher and Keycloak

Introduction

To deploy OpenG2P for sandbox, staging and production environments refer to the guide.

To install OpenG2P on your work machine for development refer to the guide in the Developer Zone.

Generate certificates

• Install letsencrypt and certbot.

sudo apt install letsencrypt certbot

• Generate Certificate.

sudo certbot certonly --agree-tos --manual --preferred-challenges=dns -d *openg2p.sandbox.net -d openg2p.sandbox.net

• The above command will ask for _acme-challenge, since the chosen challenge is of type DNS. Create the _acme-challenge TXT DNS record accordingly, and continue with the above prompt to certs generation.

• The generated certs should be present in /etc/letsencrypt directory.

Renew certificates

• Run the same generate certs command to renew certs.

sudo certbot certonly --agree-tos --manual --preferred-challenges=dns -d *openg2p.sandbox.net -d openg2p.sandbox.net

• The above command will generate new pair of certificates. The DNS challenge needs to be performed again, as prompted.

• Run the following to upload new certs back to Kubernetes Cluster. Adjust the certs path in the below command.

kubectl delete secret tls-openg2p-ingress -n istio-system
kubectl create secret tls tls-openg2p-ingress -n istio-system \
  --cert=/etc/letsencrypt/live/openg2p.sandbox.net-renewed/fullchain.pem \
  --key=/etc/letsencrypt/live/openg2p.sandbox.net-renewed/privkey.pem

Post-install configuration

• Once the Odoo server is up, log in as admin. And enter debug mode on odoo.

• Go to Settings -> Technical -> System Parameters.

  • Configure web.base.url to your required base URL.

  • Create another system parameter, with the name web.base.url.freeze, and value True.

  • Create another system parameter, with the name auth_oauth.authorization_header, and value True.

• Go to the Apps sections on UI, and click on the Update Apps List action on top.

• Search through and install required G2P Apps & Modules.

• After all, apps are installed, proceed to create users and assign roles.

• Do not use the admin user after this step. Log back in as a regular user.

• Configure ID Types on Registry -> Configuration.

• WIP.

This page contains steps to be performed for packaging different components and addons, of OpenG2P and similar, into a docker image.

Instructions

• Clone the https://github.com/OpenG2P/openg2p-packaging and go to packaging directory

• Create text file, example my-package.txt . This signifies a package. This file should include all openg2p modules (repositories) to be packaged into a new docker. Each line describes one repository to include, and the structure of each line looks like this.

  Copy

  repo_name = git://<github branch name>//<github url to pull from>
  repo2_name = file://<path of the package in local system>

  • Any underscore in the repository name will be converted to hyphen during installation. For example:

    Copy

    repo_name = git://<github branch name>//<github url to pull from>

    This is internally converted to repo-name.

• The above configuration can be made via environment variables also.

  • Any variable with the prefix G2P_PACKAGE_my_package_ will be considered as repository to install i.e., G2P_PACKAGE_<package_name>_<repo_name>. For example;

    Copy

    G2P_PACKAGE_my_package_repo3_name=git://<github branch name>//<github url to pull from>

  • These env variables can be added in .env file (in the same folder). The .env file will automatically be considered.

  • If same package is available in my-package.txt, .env and environment variable, then this will be the preference order in which they are considered (highest to lowest).

    • .env file

    • Environment variable.

    • my-package.txt

  • Use the .env to overload packages from my-package.txt

• Run the following to download all packages:

  Copy

  ./package.sh my-package.txt

• After downloading packages, run the following to build docker image:

  Copy

  docker build . -t <docker image name>

• Then push the image.

  Copy

  docker push <docker image name>

• Notes:

  • The above uses bitnami's odoo image as base.

  • This script also pulls in any OCA dependencies configured, in oca_dependencies.txt inside each package. Use this env variable to change the version of OCA dependencies to be pulled, OCA_DEPENDENCY_VERSION (defaults to 15.0).

  • This also installs any python requirements configured in requirements.txt inside each package.

  • Reference packages can be found in packages directory inside packaging directory.

Introduction

The table below enumerates various admin/user access to the entire deployment. This includes access to machines, Rancher, Kubernetes cluster as well as OpenG2P application.

Access matrix

Wireguard access to users

The guide below provides steps to provide Wireguard access to users' devices (called peers). Note that the access must be provided to each unique device (like a desktop, laptop, mobile phone etc). Multiple logins with same conf file is not possible.

Steps

1. Login to the Wireguard node via SSH.

   Copy

   > ssh -i <SSH key pem file> <user>@<ip>

2. Navigate to Wireguard conf folder

   Copy

   > cd /etc/wireguard_general

3. You will see several pre-created peer config files. You may assign any one of the file (not assigned before) to a new peer/user.

4. Editassigned.txt file to assign a new the peer (client/user). Make sure a conf file is assigned to a unique user, already assigned file is never re-assigned to another user.

   Copy

   > vim assigned.txt

5. Add the peers with name as mentioned below. Example:

   Copy

   > peer1 : <peer name>

6. Share the conf file with the peer/user securely. Example: peer1/peer1.conf

Wireguard client installation

Follow the guide here.