Deduplication

In Social Registry (SR), the data are received and recorded in several ways. SR is dynamically updated via APIs, Service Provider Portal, ODK, Login-based direct data entry, and so on. In order to maintain the data integrity in SR, a process is a must to ensure the uniqueness and accuracy of the recorded registrants' data. SR has a solution/feature named Deduplication.

Deduplication

Deduplication is the process that identifies and stores the duplicate entries of the recorded registrants' data in SR. It can manage duplicate records within separate registry groups and individual registries. The user must have an Administrator or Registrator role to access this functionality.

ID-based deduplication

ID-based deduplication is a method that detects duplicates based on unique identifiers i.e., IDs assigned to each recorded registrant belong to an individual/group. This method ensures each ID is distinct and helps to maintain single entry for each recorded registrant. In this way, it eliminates redundancies and keeps the database more reliable, clean, and efficient.

To know more about Odoo merge and deduplication, click here

Feature and functionality

Deduplication method	Deduplication process
Individual Registry Deduplication	Individual duplication record identification and management ensures accuracy and granularity in the deduplication process.
Kind based deduplication for Group registrant	It detects the groups that have the same Kind and removes the duplicate groups based on ID. It detects the group member with the same kind and removes the duplicate group member based on ID.
Separate Registry Groups Deduplication	Each registry group's deduplication procedure enables precise duplicate identification and management within discrete categories
Deduplicate Button	With the addition of a Deduplicate button to the Social Registry screen, users can initiate the deduplication process easily.
Automated Record Identification	When users click the Deduplicate button, the system identifies the duplicate records and notifies them about the number of duplicate records found. The duplicates are then saved to a separate duplicate registry.

Deduplication method

Deduplication process

Individual Registry Deduplication

Individual duplication record identification and management ensures accuracy and granularity in the deduplication process.

Kind based deduplication for Group registrant

It detects the groups that have the same Kind and removes the duplicate groups based on ID.
It detects the group member with the same kind and removes the duplicate group member based on ID.

Separate Registry Groups Deduplication

Each registry group's deduplication procedure enables precise duplicate identification and management within discrete categories

Deduplicate Button

With the addition of a Deduplicate button to the Social Registry screen, users can initiate the deduplication process easily.

Automated Record Identification

When users click the Deduplicate button, the system identifies the duplicate records and notifies them about the number of duplicate records found. The duplicates are then saved to a separate duplicate registry.

Source code

https://github.com/OpenG2P/openg2p-social-registry/tree/17.0-develop/g2p_registry_id_deduplication

📔Configure ID Deduplication, Deduplicate, and Save Duplicate Groups/Individual

PreviousImport CSV file to Social Registry NextUser Guides

Last updated 1 month ago

Deduplication

ID-based deduplication

Feature and functionality

Source code

Related user guides