LogoLogo
PlatformUse CasesCommunityBlog
1.2
1.2
  • 🏠Home
  • 🍩PLATFORM
    • Architecture
    • Modules
      • Program & Beneficiary Management System
        • Program Management
        • Program Disbursement Cycles
        • Beneficiary Management
        • ID Verification
        • Beneficiary Registry
        • Eligibility
          • Proxy Means Test
        • Deduplication
        • Enrolment
        • Entitlement
        • Disbursement
          • In-kind Transfer
          • Digital Cash Transfer
          • Voucher
        • Self Service Portal
        • Document Management
        • Multi-tenancy
        • Notifications
        • Accounting
        • Administration
          • Multi-tenancy
          • RBAC
          • i18n
      • Social Registry
      • Registration Tool Kit
        • ODK Collection App
      • SPAR
      • G2P Cash Transfer Bridge
        • File-based Payment Backend
      • 4Sure Verifier
    • Monitoring and Reporting
    • Logging
    • Privacy and Security
      • Key Manager
      • Key Manager Architecture
    • Interoperability
    • Integrations
      • OpenG2P eSignet Integration
      • OpenG2P M-Pesa Integration
      • OpenG2P Mojaloop Integration
    • Technology Stack
    • Reference
      • ↔️API
    • Releases
      • 1.1.0
        • Release Notes
    • License
      • OpenG2P Support Policy
    • FAQ
  • ⛎USE CASES
    • Use Cases
      • Immediate Assistance On Demand
      • Registration using Self Service Portal
      • Registration in Low Connectivity Areas
      • Service Provider Reimbursement
  • 🗄️DEPLOYMENT
    • Deployment Architecture
    • Infrastructure Setup
      • Hardware Requirements
      • Wireguard Server Setup
      • Rancher Setup
      • NFS Server Setup
      • OpenG2P K8s Cluster Setup
      • Loadbalancer Setup
    • External Components Setup
      • PostgreSQL Server Deployment
      • Keycloak Deployment
      • Minio Deployment
      • ODK Central Deployment
      • Kafka Deployment
      • Logging & OpenSearch Deployment
      • Keymanager Deployment
      • eSignet Deployment
    • OpenG2P Modules Deployment
      • PBMS Deployment
        • Post Install Configuration
      • Social Registry Deployment
      • GCTB Deployment
      • SPAR Deployment
        • SPAR Post Installation Configuration
      • Reporting Deployment
    • Deployment Guides
      • Giving Access to Users
      • Packaging OpenG2P Docker
      • SSL Certificates using Letsencrypt
      • Install WireGuard Client on Desktop/Laptop
      • Install WireGuard Client on Android Device
      • Make Environment Publicly Accessible using AWS LB Configuration
  • 👨‍💻DEVELOPER ZONE
    • Getting Started
      • Installing OpenG2P On Linux
    • Repositories
      • openg2p-mts
        • MTS Connector
        • OpenG2P Registry MTS Connector
      • openg2p-documents
      • openg2p-formio
        • G2P Formio
      • openg2p-registry
        • G2P Registry: Rest API Extension Demo
        • G2P Registry: Additional Info REST API
        • G2P Registry: Bank Details Rest API
        • G2P Registry: Additional Info
        • G2P Registry:Bank Details
        • G2P Registry:Membership
        • G2P Registry: Group
        • G2P Registry: Individual
        • G2P Registry: Base
        • G2P Registry: Rest API
      • openg2p-program
        • OpenG2P Program Payments: In Files
        • OpenG2P Program: Documents
        • OpenG2P Program Payment (Payment Hub EE)
        • G2P Programs: REST API
        • G2P Program : Program Registrant Info Rest API
        • OpenG2P Entitlement: Differential
        • G2P Program Payment Manager: Payment Interoperability Layer
        • G2P Program Approval
        • OpenG2P Entitlement Voucher
        • OpenG2P Program Assessment
        • OpenG2P Program Reimbursement
        • OpenG2P Program Registrant Info
        • OpenG2P Program Payment Cash
        • OpenG2P Program Payment Simple Mpesa Payment Manager
        • OpenG2P Programs Cycleless
        • OpenG2P Programs Autoenrol
        • OpenG2P Entitlement In-kind
        • G2P SelfServicePortal
        • OpenG2P Program Payment: G2P Connect Payment Manager
        • G2P Notifications: Wiserv SMS Service Provider
        • G2P: Proxy Means Test
      • openg2p-testing
      • openg2p-fastapi-template
      • openg2p-fastapi-common
        • OpenG2P FastAPI Common
        • OpenG2P FastAPI Auth
        • OpenG2P Common: G2P Connect ID Mapper
      • social-payments-account-registry
      • g2p-cash-transfer-bridge
      • openg2p-deployment
      • openg2p-documentation
      • openg2p-helm
      • openg2p-theme
      • openg2p-portal-api
      • openg2p-mosip
      • openg2p-notifications
      • openg2p-packaging
      • openg2p-importers
        • G2P ODK Importer
      • openg2p-documents
      • openg2p-reporting
      • openg2p-self-service-portal
      • openg2p-portal
      • odoo-json-field
      • spar-ui
      • openg2p-auth
      • openg2p-voucher-scanner-app
      • openg2p-security
      • openg2p-mts
      • server-auth
      • openg2p-data
      • openg2p-esignet
      • spar-load-test
      • 4sure
    • Testing
      • Test Workflow
      • Automation Framework
  • 👩‍💻COMMUNITY
    • Contributing
    • Code of Conduct
  • 📔USER GUIDES
    • Platform Guides
      • Registration
        • Self Register Online
        • ODK
          • Create a Project for a Program
          • Create a Form
          • Upload a Form
          • Upload revised Form
          • Test a Form
          • Publish a Form
          • Provide Form Access to Field Agent
          • Download Form on ODK Collect
          • Delete a Form
          • Register Offline
        • ODK Importer
          • Customize the ODK Importer Configuration based on the ODK Form Fields
      • Authentication
        • Integrate with MOSIP e-Signet
      • Deduplication
        • Deduplicate Registrants
      • Eligibility and Program Enrollment
        • Enrol Registrants into Program
        • Program
          • Create Manager Type
            • Create Eligibility Manager Types
              • Create Default Eligibility Manager
              • Create ID Document Eligibility Manager
              • Create Phone Number Eligibility Manager
            • Create Deduplication Manager Types
              • Create ID Deduplication Manager
              • Create Phone Number Deduplication
            • Create Notification Manager Types
              • Create SMS Notification Manager
              • Create Email Notification Manager
              • Create Fast2SMS Notification Manager
            • Create Entitlement Manager Type
              • Create Default Entitlement Manager
              • Create Voucher Entitlement Manager
            • Create Payment Manager Types
              • Create Payment Hub EE Payment Manager
              • Create Payment Interoperability Layer Payment Manager
              • Create Default Payment Manager
              • Create Cash Payment Manager
              • Create File Payment Manager
          • Create Program
          • Map Self-Service Portal Form
          • Create Eligibility Manager under Program
          • Create Deduplication Manager under Program
          • Create Notification Manager under Program
          • Configure Program Manager under Program
          • Create Entitlement Voucher Template
        • Configuration
          • Configure Proxy Means Test
          • Configure ID Types
          • Configure Entitlement Manager under Program
          • Configure Payment Manager in Program
        • Approval
          • Create and Approve Program Cycle
          • Multi-Stage Approval
        • MTS Connector
          • Create MTS Connector
            • Create ODK MTS Connector
            • Create OpenG2P Registry MTS Connector
        • Settings
          • Create User and Assign Role
        • Website
          • Create Self-Service Portal Form
      • Notification
        • Send Notifications to Individual Registrants
        • Prepare and Send Payment
      • Entitlement
        • Install SmartScanner App
      • Cash Transfer
        • Reimbursement
          • Submit Reimbursement Using the Service Provider Portal
          • Reimburse the service provider
      • Accounting and Reporting
      • SPAR
        • Self Update ID with Financial Address information
        • Admin Guide to Link ID with Financial Address information
      • 4Sure
        • Verify Digital Credentials using 4Sure
        • Verify and Populate the form in ODK Collect using 4Sure
    • Documentation Guides
      • Documentation Guidelines
      • OpenG2P Module Doc Template
  • BLOG
    • Articles
      • OpenG2P and SDG Goals
      • OpenG2P - A Building Block for DPI
    • Case Studies
Powered by GitBook
LogoLogo

Copyright © 2024 OpenG2P. This work is licensed under Creative Commons Attribution International LicenseCC-BY-4.0 unless otherwise noted.

On this page
  • Introduction
  • Architecture
  • Concepts
  • DB Tables
  • Payment File Creator
  • File Store
  • File Dispatcher
  • File Fetcher
  • Payment File Reader
  • Performance and capacity
  • Installation Guide
  • Usage Guide
  • API Docs
  1. PLATFORM
  2. Modules
  3. G2P Cash Transfer Bridge

File-based Payment Backend

PreviousG2P Cash Transfer BridgeNext4Sure Verifier

Last updated 1 year ago

Introduction

TBD

Architecture

Concepts

DB Tables

1. File Outgoing

This table will hold the status of files dispatched

Column
Description

file_name

status

NEW/DISPATCHED

2. File Incoming

This table will hold the status of files received from bank.

Column
Description

file_name

status

NEW/PROCESSED

Payment File Creator

This block will be like a cronjob that runs periodically and does the following

  • Read all the payment instructions that have not been processed yet (status: NEW) via direct query of DB

  • Query ID Account Mapper if translation from ID to bank account information is required.

  • Create an output file specific to the format accepted by a bank.

  • Write the file in the File Storage

  • Update the status of all processed instructions as FILED.

This functional block may be implemented as a script that is run as cronjob. The features will be implemented:

  • Unique file names -- could be composed of the batch ID with a few more fields or it could be completely UUID based.

  • Full in-memory processing

  • Process data in a batch size as expected by the bank, provided the data can be accommodated in the RAM. (This should be fine because typically banks will not expect very large files. The number of rows may be in the order of a few thousand).

  • Entire processing and updates in DB should happen atomically. In case there is a failure, the script should be able to discard the incomplete file and start all over again.

  • The batch size should be configurable

  • The file format will be specific to a bank. The script should be designed such that several file formats can be configured without having to make major changes in the code.

File Store

This will typically be large S3-compliant storage (like MinIO or AWS S3) to store all the payment instruction files. It will have two buckets:

  • Outgoing: For outgoing payment files and another for received payment files.

  • Incoming: For files fetched from bank(s)

File Dispatcher

The File Dispatcher is like a cronjob that does the following:

  • Picks an unprocessed (un-sent) file from the bucket and using bank interface transfers the file to the bank. This could be a mechanism like SFTP. This would be very bank-specific.

Features:

  • The implementation may be done via script called periodically.

  • The file dispatch mechanism will be specific to a bank. The script should be designed such that several dispatch mechanisms can be configured without having to make major changes in the code

File Fetcher

The File Fetcher is like a cronjob that does the following:

  • Fetch fresh files form bank's systems and stores them in Incoming bucket

Features:

  • The implementation may be done via script called periodically.

  • The file dispatch mechanism will be specific to a bank. The script should be designed such that several fetch mechanisms can be configured without having to make major changes in the code

Payment File Reader

This block will be like a cronjob that runs periodically and does the following

  • Parse the file to read the status of each transaction. Translate the status to a common standard code if required.

  • Update the status in DB as PAID or FAILED.

  • Update error code and error messages

This functional block may be implemented as a script that is run as cronjob. The features will be implemented:

  • Full in-memory processing

  • Entire processing and updates in DB should happen atomically. In case there is a failure, the script should be able to discard the incomplete process and start all over again.

Performance and capacity

G2P payments are typically not real-time and can happen over several hours or even days. Assuming that SSD disks are used for DB and file storage, and all processing happens in-memory it should be possible to read a batch of 10, 000 to 100,000 from DB, process it, and write into a file in a few seconds (hypothesis needs to be tested). Thus_,_ several million transactions processing should be possible in a day. A barebone calculation: if we assume 10,000 records can be read from DB, processed and written in a file in 10 seconds, then 3.6 million records can be processed in 1 hour.

The response time of ID Account Mapper also needs to be considered. The query to ID Account Mapper should be in Sync mode to keep the design simple.

The Payment File Creator, Payment File Reader, File Dispatcher, and File Fetcher must be designed such that they can run on independent CPUs.

The DB must be suitably indexed for fast reads and updates.

With this capacity, a single instance of each of the above would suffice. There may not be a need to have multiple instances of these blocks running in parallel as it will significantly increase the complexity of the design.

Installation Guide

TBD

Usage Guide

TBD

API Docs

TBD

Update status in the table as NEW.

Configuration for bucket ID, location, and credentials for accessing .

Update status in table in the DB as DISPATCHED.

Update status in table in the DB as NEW.

Read an unprocessed file from File Store -> Incoming bucket. The list of unprocessed files is obtained by reading the table.

Update the status in the table as PROCESSED.

Configuration for bucket ID, location, and credentials for accessing .

🍩
File Outgoing
File Store
File Outgoing
File Incoming
File Incoming
File Incoming
File Store
File-based payment backend