📔Installation & Troubleshooting

Installation

Reporting framework is installed as part of the modules' installation via the Helm chart that installs the respective module. Note that during installation you need to specify the GitHub Repository URL and branch and directory that contains the Debezium and OpenSearch connectors. For example:

https://github.com/OpenG2P/openg2p-reporting/tree/develop/scripts/social-registry

Follow this guide to creating/updating connectors.

Assigning roles to users

Create Keycloak client roles for the following components and assign them to users:

Component	Role name
OpenSearch Dashboards for Reporting	admin
Kafka UI for Reporting	Admin

Post-installation check

To ensure that all Kafka connectors are working login into Kafka UI (domain name is set during installation) and check the connectors' status.

Update Connectors

This procedure doesn't update the data present in OpenSearch, it only updates the connector configs, so only the new and incoming data is affected. (If you need the data in OpenSearch/Kafka to be refreshed, then follow the Cleanup guide. And then continue with this guide.)

Upgrade base method

Follow this method if you wish to upgrade to the latest version of reporting (and reporting-init scripts) and to upgrade connector configs.

• After making changes to connectors/dashboards in your GitHub Repo, go to the Installed Apps section on Rancher and upgrade your module, SR/PBMS, etc. (without changing any helm values).

• When the upgrade finishes the new reporting connector changes are automatically applied to the connectors. Log in to Kafka UI and check whether the connector config has been updated.

Rerun initialize method

Follow this method if you don't wish to upgrade the reporting version, but only want to update connector configurations, etc. (Useful mainly during development and testing.)

• Copy your OpenG2P module installation name on Kubernetes. (For example, say social-registry).

• Copy the reporting-init Job yaml from the output of this command into a temporary file, say reporting-init.yaml .

  helm -n namespace get hooks social-registry

• Delete the reporting-init job. (Can also be done from Rancher UI)

  kubectl -n namespace delete job social-registry-reporting-init

• Apply the above reporting-init.yaml after deletion.

  kubectl -n namespace apply -f reporting-init.yaml

Cleanup and uninstall

Warning: All data on OpenSearch will be lost. This is especially important if you are capturing the history of the records as you will not get that history back.

This describes steps to clean up the connectors and the data so fresh connectors can be installed again.

• Log in to Kafka UI -> Kafka Connect Section, and delete all the connectors.

• Delete all the topics related to the connectors as well. Also, delete Kafka consumer offsets (IMPORTANT). TODO: give a guide on how to clear Kafka consumer offsets.

• Log in to OpenSearch -> Index Management, and delete all the relevant indices.

• Delete replication slots and publication on Postgres, using the following commands.

  • List all replication slots:

    select * from pg_replication_slots;

  • Delete a replication slot:

    select pg_drop_replication_slot('replication_slot_name_from_above');

  • List all publications:

    select * from pg_publication;

  • Delete a publication:

    drop publication publication_name_from_above;

If you want to install the connectors again, follow the Update guide.