LogoLogo
1.3
1.3
  • Overview
  • Social Registry
    • Features
      • Individuals and Groups
        • 📔User Guides
          • 📔Create an Individual Registrant
          • 📔Create a Group and Add Individual Registrants to the Group
          • 📔Import CSV file to Social Registry
      • Deduplication
        • 📔User Guides
          • 📔Configure ID Deduplication, Deduplicate, and Save Duplicate Groups/Individuals
        • Deduplicator Service
      • Lock and Unlock
      • Enumerator
        • Enumerator ID
      • Dynamic Updates
      • Document Upload
      • ODK Importer
        • 📔User Guide
          • 📔Configure and Import ODK Form
          • 📔Import Specific ODK Forms using ODK Instance ID
      • Registration Portal
        • 📔User Guides
          • 📔Create a New Household
          • 📔Create a New Individual in Registration Portal
          • 📔Create a New Portal User
          • 📔Configure Portal User to Limit Accessing Location
      • Configurations
        • 📔User Guide
          • 📔Configure ID Types
          • 📔Configure Registrant Tags
          • 📔Configure Gender Types
          • 📔Configure Relationships
          • 📔Configure Group Types
          • 📔Configure Group Membership Kind
      • User Management
        • 📔User Guide
          • 📔Create User
          • 📔Assign a Role to a User
      • Geographic
      • Data Share
      • Languages Support
        • 📔User Guides
          • 📔Set Language Preference
      • API
        • Search APIs
        • Individual APIs
        • Group APIs
      • Privacy and Security
      • Interoperability
      • Monitoring and Reporting
      • ID Integration
        • ID Validation and Tokenisation
        • ID Authentication
          • 📔User Guides
            • 📔Configure eSignet Auth Provider for ID Authentication
            • 📔ID Authentication Process
            • 📔eSignet Client Creation
        • Fayda ID Integration
      • Verifiable Credentials Issuance
        • 📔User Guides
          • 📔Configure Inji to download Social Registry VCs
      • Computed fields
      • Record Revision History
      • SPAR Integration for Account Info
      • Self Service Registration Portal
      • Unique Reference ID
      • Logging
        • Audit Logs
        • System Logs
        • Change log
    • Versions
    • Deployment
      • Domain names and Certificates
      • Install Odoo Modules
    • Developer Zone
      • Technology Stack
      • Repositories
      • Developer Install
        • 📘Developer Install of OpenG2P Package on Linux
      • Packaging
        • 📘Docker Packaging Guide
        • 📘Helm Packaging Guide
      • Odoo Modules
        • ODK App User Mapping
  • PBMS
    • Features
      • Program Management
        • Role of a Program Manager
        • Program Life Cycle
        • 📔User Guides
          • 📔Create Program
          • 📔Create Eligibility Manager under Program
          • 📔Create Program Manager for a Program
          • 📔Create Deduplication Manager under Program
          • 📔Create Manager Type
            • 📔Create Payment Manager Types
              • 📔Create Payment Hub EE Payment Manager
              • 📔Create Payment Interoperability Layer Payment Manager
              • 📔Create Default Payment Manager
              • 📔Create Cash Payment Manager
              • 📔Create File Payment Manager
          • 📔Configure Entitlement Manager under Program
          • 📔Configure Payment Manager in Program
          • 📔Configure Default Program Manager
          • 📔Archive, Delete, End, and Re-activate a Program
      • Program Disbursement Cycles
        • 📔User Guides
          • 📔Create Program Fund
          • 📔Create Cycle Manager for a Program
      • Beneficiary Management
        • Beneficiary Registry
          • 📔User Guides
            • 📔Create an Individual Registrant
            • 📔Create a Group and Add Individual Registrants to the Group
            • 📔Assign a Program to a Group
            • 📔Assign a Program to an Individual
        • Beneficiary Registry Configurations
          • 📔User Guides
            • 📔Configure ID Types
            • 📔Configure Registrant Tags
            • 📔Configure Gender Types
            • 📔Configure Relationships
            • 📔Configure Group Types
            • 📔Configure Group Membership Kind
        • Registration
          • 📔User Guides
            • 📔Import CSV File to Registry Module
      • ID Verification
      • Eligibility
        • Proxy Means Test
        • 📔User Guides
          • 📔Create Eligibility Manager Types
            • 📔Configure Default Eligibility Manager
            • 📔Create ID Document Eligibility Manager
            • 📔Create Phone Number Eligibility Manager
          • 📔Configure Proxy Means Test
          • 📔Verify Eligibility of Enrolled Registrants
      • Deduplication
        • 📔User Guides
          • 📔Deduplicate Registrants
          • 📔Create Deduplication Manager Types
            • 📔Configure Default Deduplication Manager
            • 📔Create ID Deduplication Manager
            • 📔Create Phone Number Deduplication
      • Enrolment
        • 📔User Guides
          • 📔Enroll Registrants into Program
          • 📔Auto-Enroll New Registrants into a Program
          • 📔Enroll Eligible Individual Registrants into a Program
      • Entitlement
        • 📔User Guides
          • 📔Multi-Stage Approval
          • 📔Create Entitlement Manager Type
            • 📔Create Default Entitlement Manager
            • 📔Create Voucher Entitlement Manager
            • 📔Configure Cash Entitlement Manager
          • 📔Create Entitlement Voucher Template
          • 📔Configure the Payments File with QR Code
          • 📔Configure Default Cycle Managers
          • 📔Export Beneficiaries Approved Entitlement
      • Disbursement
        • Payment Batches
        • In-Kind Transfer
          • 📔User Guides
            • 📔Create a Product in Inventory
            • 📔Configure In-Kind Entitlement Manager
            • 📔Create and Approve Program Cycle
            • 📔Verify Eligibility of Registrants in a Cycle
        • Digital Cash Transfer
        • e-Voucher
        • 📔User Guides
          • Prepare and Send Payment
      • Self Service Portal
        • 📔User Guides
          • 📔Self Register Online
          • 📔Create Self Service Portal Form
          • 📔Map Self Service Portal Form
      • Document Management
      • Multi-tenancy
      • Notifications
        • 📔User Guides
          • 📔Send Notifications to Individual Registrants
          • 📔Create Notification Manager Types
            • 📔Create SMS Notification Manager
            • 📔Create Email Notification Manager
            • 📔Create Fast2SMS Notification Manager
          • 📔Create Notification Manager under Program
      • Accounting
      • Administration
        • RBAC
          • 📔User Guides
            • 📔Create User and Assign Role
            • 📔Configure Keycloak Authentication Provider for User Log in
        • i18n
      • ODK Importer
        • 📔User Guides
          • 📔Configure and Import ODK Form
          • 📔Import Specific ODK Forms using ODK Instance ID
          • 📔Import Social Registry Data into PBMS
      • MTS Connector
        • 📔User Guides
          • 📔Create MTS Connector
            • 📔Create ODK MTS Connector
            • 📔Create OpenG2P Registry MTS Connector
      • Audit Logs
      • Service Provider Portal
        • 📔User Guides
          • 📔Submit Reimbursement Using the Service Provider Portal
          • 📔Reimburse the Service Provider
      • Interoperability
      • Privacy and Security
      • Periodic Biometric Authentication for Beneficiaries
      • Beneficiary Exit Process
      • Verifiable Credential Issuance
        • 📔User Guides
          • 📔Configure Inji to download Beneficiary VCs
      • Deduplication
      • Manual In-Kind Entitlement
      • Print Disbursement Summary
      • Monitoring & Reporting
        • Logging
    • Versions
    • Developer Zone
      • Odoo Modules
        • G2P Enumerator
        • OpenG2P Registry MTS Connector
        • G2P Documents Store
        • MTS Connector
        • G2P Formio
        • G2P Registry: Rest API Extension Demo
        • G2P Registry: Additional Info REST API
        • G2P Registry: Bank Details Rest API
        • G2P Registry: Additional Info
        • G2P Registry: Membership
        • G2P Registry: Groups
        • G2P Registry: Individual
        • G2P Registry: Base
        • G2P Registry: Rest API
        • G2P Registry: Bank Details
        • OpenG2P Program Payment (Payment Hub EE)
        • OpenG2P Program Payments: In Files
        • G2P Program : Program Registrant Info Rest API
        • OpenG2P Entitlement: Differential
        • OpenG2P Program: Approval
        • OpenG2P Program: Assessment
        • G2P Program: Registrant Info
        • OpenG2P Program Payment: Simple Mpesa Payment Manager
        • OpenG2P Programs: Cycleless
        • OpenG2P Entitlement: In-Kind
        • G2P Notifications: Wiserv SMS Service Provider
        • G2P: Proxy Means Test
        • G2P Programs: REST API
        • G2P Program Payment (Payment Interoperability Layer)
        • OpenG2P Entitlement: Voucher
        • OpenG2P Programs: Reimbursement
        • OpenG2P Program Payment: Cash
        • OpenG2P Program: Documents
        • OpenG2P Program Payment: G2P Connect Payment Manager
        • OpenG2P Programs: Autoenrol
        • G2P ODK Importer
        • G2P Service Provider Beneficiary Management
        • OpenID Connect Authentication
        • G2P Auth: OIDC - Reg ID
        • G2P OpenID VCI: Base
        • G2P OpenID VCI: Programs
        • G2P OpenID VCI: Rest API
      • Developer Install on Linux
      • Repositories
        • openg2p-fastapi-common
          • OpenG2P FastAPI Common
          • OpenG2P FastAPI Auth
          • OpenG2P Common: G2P Connect ID Mapper
        • social-payments-account-registry
        • g2p-bridge
        • openg2p-packaging
        • openg2p-security
        • spar-load-test
        • 4sure
        • G2P SelfServicePortal
      • Technology Stack
      • Testing
        • Test Workflow
        • Automation Framework
    • Deployment
      • i18n
      • Installation of Odoo Modules
      • Domain names and Certificates
      • Helm Charts
  • SPAR
    • Features
      • SPAR Mapper
      • SPAR Self Service
      • Privacy & Security
      • Interoperability
      • Monitoring & Reporting
    • Deployment
      • Domain Names and Certificates
      • Helm Charts
    • 📔User Guides
      • 📔Link FA (Self Service)
      • 📔Link FA (Admin)
    • Development
      • Jira Board
      • Testing
        • Unit Testing
        • Functional Testing
        • Performance Testing
          • Mapper
            • Resolve API
            • Link API
            • Unlink API
            • Update API
      • Developer Install
        • SPAR Mapper API
        • SPAR Self Service API
        • SPAR Self Service UI
      • Repositories
      • API Reference
      • Tech Guides
    • Releases
      • 1.0.0
      • 1.1.0 - WIP
    • Roadmap
  • G2P Bridge
    • Features
      • Privacy & Security
      • Interoperability
      • Monitoring & Reporting
    • Deployment
    • Development
      • Design
        • IN APIs from PBMS
          • create_disbursement_envelope
          • cancel_disbursement_envelope
          • create_disbursements
          • cancel_disbursements
          • get_disbursement_envelope_status
          • get_disbursement_status
        • OUT APIs to Mapper
          • resolve
        • OUT APIs to Bank
          • check_funds_with_bank
          • block_funds_with_bank
          • disburse_funds_from_bank
        • IN APIs from Bank
          • upload_mt940
        • Helper Tables
          • benefit_program_configuration
        • Configuration parameters
        • Interfaces
        • Physical Organization
        • Example Bank
          • example-bank-models
          • example-bank-api
          • example-bank-celery
      • Testing
        • Unit Testing
        • Functional Testing
        • Performance Testing
      • Repositories
    • Tech Guides
    • 📔User Guides
      • 📔Configure G2P Connect Payment Manager
    • Releases
  • Utilities and Tools
    • ODK
      • 📔User Guides
        • 📔Create a Project for a Program
        • 📔Create a Form
        • 📔Upload a Form
        • 📔Upload revised Form
        • 📔Test a Form
        • 📔Publish a Form
        • 📔Provide Form Access to Field Agent
        • 📔Download a Form on ODK Collect
        • 📔Delete a Form
        • 📔Register Offline
    • 4Sure Verifier App
      • Installation Guide for 4Sure Application
      • 📔User Guides
        • 📔Verify Digital Credentials using 4Sure Application
        • 📔Verify and Populate the form in ODK Collect using 4Sure Application
      • 4Sure Test Summary
    • Smartscanner
      • 📔User Guides
    • Registration Tool Kit
  • Monitoring and Reporting
    • Apache Superset
    • Reporting Framework
      • 📔User Guides
        • 📔Connector Creation Guide
        • 📔Dashboards Creation Guide
        • 📔Installation & Troubleshooting
        • Page 1
      • Kafka Connect Transform Reference
    • System Logging
    • System Health
  • Privacy and Security
    • Key Manager
  • Interoperability
  • Integrations
    • eSignet Integration
    • M-Pesa Integration
    • Mojaloop Integration
    • 📔User Guides
  • Deployment
    • Base Infrastructure
      • Wireguard Bastion
        • Install WireGuard Client on Android Device
        • Wireguard Access to Users
        • Install WireGuard Client on Desktop
      • NFS Server
      • Rancher Cluster
      • OpenG2P Cluster
        • Kubernetes
          • Firewall
          • Istio
          • Adding Nodes to Cluster
          • Deleting Nodes from Cluster
        • Prometheus & Grafana
        • Fluentd & OpenSearch
          • DEPRECATED - OpenSearch
      • Load Balancer
        • Nginx
        • AWS
    • Resource Requirements
    • Helm Charts
    • Upgrades
    • Production
    • OpenG2P In a Box
    • Packaging
    • Versioning
    • Additional Guides
      • Automatic Build and Upload of Private Dockers
      • Generate SSL Certificates using Letsencrypt
      • Packaging Odoo based Docker
      • AWS
        • Create ACM Certificate on AWS
        • Create Security Group on AWS
        • Domain mapping on AWS Route53
        • Make Environment Publicly Accessible using AWS LB Configuration
      • Private Access Channel
      • Odoo Post Install Configuration
      • Pulling Docker from Private Repository on Docker Hub
      • Keycloak Client Creation
      • Troubleshooting: "fsnotify watcher" warning
      • Uninstalling Applications from Rancher UI
      • Access a Database from Outside the Cluster
      • Configure External Database to Connect OpenG2P Environment
      • Configure IPSec VPN Gateway to Connect to External Systems using Strongswan
      • Troubleshooting
        • PostgreSQL Database not Starting due to Replication Checkpoint Error
        • No Space Left on the Device Warning
      • Restart Deployment or StatefulSets to Redistribute Pods across Nodes
      • Rerun Jobs in Kubernetes Cluster
      • Finding URLs in the System
      • Transitioning PostgreSQL From Docker on K8s to Standalone PostgreSQL
      • Restore a PVC from an NFS Folder and Attach it to a Pod
      • View System Logs on the OpenSearch Dashboard
    • Persistent Storage
      • Resizing Persistent Volume Claim in Kubernetes Cluster
  • Documentation Guides
    • Documentation Guidelines
      • Embed a Miro diagram
      • Set an Image for a Start View
    • OpenG2P Module Doc Template
  • Use Cases
    • Technology for Inclusion
      • Registration in Low Connectivity Areas
      • Registration using Self Service Portal
    • Digital Cash Transfer Program
    • Create Social Registry
    • Case Studies
      • Immediate Assistance on Demand
      • Service Provider Reimbursement
      • Customise ODK Form - Add Ethiopian Calendar
  • Releases
    • 1.1.0
      • Release Notes
  • License
    • OpenG2P Support Policy
  • Community
    • Contributing
    • Code of Conduct
  • Blogs
    • OpenG2P and SDG Goals
    • OpenG2P - A Building Block for DPI
Powered by GitBook
LogoLogo

Copyright © OpenG2P. This work is licensed under Creative Common Attribution (CC-BY-4.0) International license unless otherwise noted.

On this page
  • Virtual machines provisioning
  • Firewall setup
  • Cluster installation
  • NFS client provisioner
  • Longhorn
  • Istio
  • Cluster import to Rancher
  • Cluster access to users
  • Backups
  1. Deployment
  2. Base Infrastructure
  3. OpenG2P Cluster

Kubernetes

Kubernetes installation guide

PreviousOpenG2P ClusterNextFirewall

Last updated 7 months ago

OpenG2P modules and components are recommended to be run on (K8s), because of ease-of-use, management, and security features that K8s provides.

K8s cluster may be installed on the following infrastructures:

  • Cloud-native (like EKS on AWS, or AKS on Azure)

  • Non-cloud native, or on-prem (resources provisioned on a cloud or local data centre).

Here we provide instructions to set up K8s cluster on-prem.

Broadly, the steps to install are as follows:

  1. using Rancher's tool .

  2. Provide access to users

Virtual machines provisioning

Provision for virtual machines (VMs) as per configuration mentioned in . Make sure you have root privileges to the machines and have secure access to them.

Install the following tools on all machines including the one you are using to connect to the VMs.

  • wget , curl , kubectl , istioctl , helm , jq

If you have SSH access to the VMs, and root privileges, you are the Super Admin. Make sure very limited access is given to the machines.

Firewall setup

Cluster installation

  • The following setup has to be done on each node on the cluster.

    • SSH into the node. Execute all the below commands as root user.

    • Create the rke2 config directory

      mkdir -p /etc/rancher/rke2
    • Create a config.yaml file in the above directory, using one of the following config file templates:

    • Edit the above config.yaml file with the appropriate names, IPs, and tokens

    • export INSTALL_RKE2_VERSION="v1.28.9+rke2r1"
    • Run this to download rke2.

      curl -sfL https://get.rke2.io | sh -
    • Run this to start rke2:

      • On the control-plane node, run:

        systemctl enable rke2-server
        systemctl start rke2-server
      • On the worker node, run:

        systemctl enable rke2-agent
        systemctl start rke2-agent
  • To export KUBECONFIG, run (only on control-plane nodes):

  • echo -e 'export PATH="$PATH:/var/lib/rancher/rke2/bin"\nexport KUBECONFIG="/etc/rancher/rke2/rke2.yaml"' >> ~/.bashrc
    source ~/.bashrc
    kubectl get nodes    
  • Download the Kubeconfig file rke2.yaml and keep it securely shared with only Super Admins. Rename it so that it can be identified with the cluster. This file will be used if cluster control via Rancher is unavailable.

NFS client provisioner

  • NFS_SERVER=<NFS Node Internal IP> \
    NFS_PATH=/srv/nfs/<cluster_name> \
        ./install-nfs-csi-driver.sh

In StorageClass, when the reclaimPolicy is set to Retainit implies that when a PVC is deleted the PV will not get deleted. And even if PV is deleted the relevant folder in NFS is not deleted.

When reclaimPolicy is set to Delete,if a PVC is deleted, both the PV and the relevant folder in the NFS get deleted.

Longhorn

This installation only applies if Longhorn is used as storage. This may be skipped if you are using NFS.

Istio

Cluster import to Rancher

  • Navigate to the Cluster Management section in Rancher

  • Click on Import Existing Cluster. Follow the steps to import the new OpenG2P cluster

  • After importing, download kubeconfig file for the new cluster from rancher (top right on the main page), to access the cluster through kubectl from the user's machine (client), without SSH

Cluster access to users

Rancher provides "Project" feature. This feature is not a standard Kubernetes feature and hence it is recommended to not use it for OpenG2P deployments.

Backups

Your cluster may hold critical data that needs to be backed up. The following minimal backups are highly recommended:

Backup item
Recommendations

NFS data

Persistent Volumes (PVs)

The PVs on NFS are stored as folders that are hard to associate with original application/pod. Download the YAML of critical PVs - like Postgres, Minio etc and keep it safely. This will be required in case a cluster has to be recreated, in which case, the corresponding PVs may be mounted back from the storage provided the name of the PV is known. On Rancher the PV YAMLs are available under Storage -> PersistentVolumes of a cluster. Alternatively, you may download PVs using command line utility kubectl.

Refer guide .

The following section uses to set up the K8s cluster.

Decide on the number of control-planes (RKE2 server). For high availability the minimum of nodes running control-plane should 3. If your cluster is < 3 nodes, run only 1 control-plane (odd number). Refer . The rest of the nodes are Kubernetes workers (RKE2 agent).

For the first control-plane node, use . The token can be any arbitrary string.

For subsequent control-plane nodes, use . (Make sure the token defined in the first node's control plane is used here too.)

For worker nodes, use . (Make sure the token defined in the first node's control plane is used here too.)

Run the following to set the RKE2 version after referring to .

Additional Reference:

This section assumes an has already been set up. Install NFS client provisioner on the cluster as follows:

Clone .

From directory, run: (Make sure to replace the <NFS Node Internal IP> and <cluster name> parameters appropriately below)

Refer guide .

This step assumes that a has already been set up and operational.

Users may be given access to the cluster using .

Refer .

here
RKE2
RKE2 docs
rke2-server.conf.primary.template
rke2-server.conf.subsequent.template
rke2-agent.conf.template
RKE2 Releases
RKE2 High Availability Installation
NFS server
https://github.com/OpenG2P/openg2p-deployment
kubernetes/nfs-client
Longhorn Install as a Rancher App
here
Rancher server
Rancher's RBAC
Kubernetes
RKE2
Hardware Requirements
Provision virtual machines
Set firewall rules
Set up the K8s cluster
Install NFS client on the cluster
Install Istio on cluster
Import cluster into Rancher
here