Intake Forms
Intake forms for Register
1. Overview
This document details the architectural changes to decouple sections from tabs, restructure intake form handling, and implement deduplication logic. The changes enable sections to be reusable across tabs and intake forms, while establishing a complete intake form lifecycle separate from change requests.
2. Model Changes
2.1 Register UI Structure Changes
g2p_register_ui_tabs
Tab definition table
Remains unchanged (tab definitions)
No Change
g2p_register_sections
Sections coupled with tabs; Section definition + Tab reference
Sections decoupled from tabs; Section definition only
Major Change
g2p_register_ui_tab_sections
Does not exist
New junction table: Tab ID ↔ Section ID mapping
New Table
2.2 Register Sections Model Details
Table: g2p_register_section
section_id
UUID
PRIMARY KEY
Unique section identifier
register_id
UUID
NON UNIQUE INDEX
Register reference
section_mnemonic
STRING
UNIQUE
Section identifier name
section_label
STRING
-
Display label
section_description
STRING
-
Description
is_list
BOOLEAN
-
Whether section contains multiple records
created_at
TIMESTAMP
-
Creation timestamp
updated_at
TIMESTAMP
-
Last update timestamp
Changes from current model:
Remove
tab_idforeign keyAdd
section_mnemonicas unique identifierSection now exists independently of any tab
2.3 Register UI Tab Sections Model (New)
Table: g2p_register_ui_tab_sections
tab_section_id
UUID
PRIMARY KEY
Junction record ID
tab_id
UUID
NON UNIQUE INDEX
Reference to g2p_register_ui_tabs
section_id
UUID
NON UNIQUE INDEX
Reference to g2p_register_section
section_order
INTEGER
-
Display order within tab
created_at
TIMESTAMP
-
Creation timestamp
updated_at
TIMESTAMP
-
Last update timestamp
Purpose: Explicit many-to-many mapping between tabs and sections, allowing sections to be reused across tabs.
3. Intake Form Models
3.1 Intake Form Metadata Models
Table: g2p_intake_form_definitions
form_id
UUID
PRIMARY KEY
Unique form identifier
register_id
UUID
NON UNIQUE INDEX
Associated register
form_mnemonic
STRING
UNIQUE
Form identifier name
form_description
STRING
-
Form description
number_of_verifications
INTEGER
-
Required verification count
used_only_for_ingestion_pipeline
BOOLEAN
-
TRUE: ingestion only; FALSE: manual submission allowed
created_at
TIMESTAMP
-
Creation timestamp
updated_at
TIMESTAMP
-
Last update timestamp
Table: g2p_intake_form_ui_tabs
tab_id
UUID
PRIMARY KEY
Unique tab identifier
form_id
UUID
NON UNIQUE INDEX
Associated intake form
tab_label
STRING
-
Tab display label
tab_order
INTEGER
-
Display order
created_at
TIMESTAMP
-
Creation timestamp
updated_at
TIMESTAMP
-
Last update timestamp
Table: g2p_intake_form_ui_tab_sections
tab_section_id
UUID
PRIMARY KEY
Junction record ID
tab_id
UUID
NON UNIQUE INDEX
Reference to g2p_intake_form_ui_tabs
section_id
UUID
NON UNIQUE INDEX
Reference to g2p_register_section
section_order
INTEGER
-
Display order within tab
created_at
TIMESTAMP
-
Creation timestamp
updated_at
TIMESTAMP
-
Last update timestamp
Relationship: Intake form tabs use the same sections (g2p_register_section) as register tabs, enabling section reuse.
3.2 Intake Form Submission Model
Table: g2p_intake_form_submission
submission_id
UUID
PRIMARY KEY
Unique submission identifier
application_reference
STRING
UNIQUE, NOT NULL
Human-readable Application Reference generated on insert. Format is configurable
form_id
UUID
NON UNIQUE INDEX
Associated form
register_id
UUID
NON UNIQUE INDEX
Associated register
draft_status
ENUM
-
DRAFT, FINAL
approval_status
ENUM
-
PENDING, APPROVED, REJECTED
approved_by
UUID
-
User who approved
approved_at
TIMESTAMP
-
Approval timestamp
finalized_at
TIMESTAMP
-
Finalization timestamp
first_created_at
TIMESTAMP
-
Initial creation timestamp
last_updated_at
TIMESTAMP
-
Last update timestamp
created_by
UUID
-
User who created
submission_source
ENUM
-
BENE_PORTAL, AGENT_PORTAL, STAFF_PORTAL, PARTNER
partner_id
UUID
NON UNIQUE INDEX
Partner reference (if applicable)
register_ingest_process_status
ENUM
-
PENDING, PROCESSED, NOT_APPLICABLE
register_ingest_processed_timestamp
TIMESTAMP
-
Ingest completion timestamp
register_ingest_process_attempts
INTEGER
-
Ingest attempt count
register_ingest_process_last_error_code
STRING
-
Last ingest error
number_of_verifications_required
INTEGER
-
Required verification count
number_of_verifications_done
INTEGER
-
Completed verifications
deduplication_status_vs_intake_forms
ENUM
-
PENDING, PROCESSED, FAILED
deduplication_intake_forms_process_timestamp
TIMESTAMP
-
Dedup completion
deduplication_intake_forms_attempts
INTEGER
-
Dedup attempt count
deduplication_intake_forms_error
STRING
-
Dedup error message
deduplication_status_vs_register
ENUM
-
PENDING, PROCESSED, FAILED
deduplication_register_process_timestamp
TIMESTAMP
-
Register dedup completion
deduplication_register_attempts
INTEGER
-
Register dedup attempt count
deduplication_register_error
STRING
-
Register dedup error message
Intake section tables that extend G2PIntakeForm carry a null-able indexed application_reference, copied from the submission header on save so display names and search can read it from the row.
3.3 Deduplication Results Models (New)
Table: g2p_dedup_results_intake_forms_vs_intake_forms
dedup_result_id
UUID
PRIMARY KEY
Result identifier
submission_id
UUID
NON UNIQUE INDEX
Submission being deduplicated
duplicate_submission_id
UUID
NON UNIQUE INDEX
Potential duplicate submission
match_score
DECIMAL
-
Match confidence (0.0-1.0)
match_fields
JSONB
-
Fields that matched
created_at
TIMESTAMP
-
Creation timestamp
Table: g2p_dedup_results_intake_forms_vs_register
dedup_result_id
UUID
PRIMARY KEY
Result identifier
submission_id
UUID
NON UNIQUE INDEX
Submission being deduplicated
register_record_id
UUID
NON UNIQUE INDEX
Potential duplicate in register
register_section_name
STRING
-
Register section name
match_score
DECIMAL
-
Match confidence (0.0-1.0)
match_fields
JSONB
-
Fields that matched
created_at
TIMESTAMP
-
Creation timestamp
4. Extension Model Hierarchy
4.1 Domain-Specific Classes
All domain manifestations (e.g., Farmer Registry) follow this pattern:
Base Structure (Core Repository):
Extension Structure (Extensions Repository):
Example - Farmer Domain:
Location:
farmer_extension.register_domain.farmer.pyClasses:
G2PRegisterFarmer(table:g2p_register_farmer)G2PRegisterHistoryFarmer(table:g2p_register_history_farmer)G2PIntakeFormFarmer(extends bothG2PRegisterFarmerandG2PIntakeForm)
Applied to all domain registers and tables.
5. Existing Code Flow
5.1 Register Tabs and Sections Retrieval
Current Flow:
5.2 Intake Form Submission - Current
Current Flow:
5.3 Intake Form Approval - Current
Current Flow:
6. Proposed Code Flow
6.1 Register Tabs and Sections Retrieval
Proposed Flow:
6.2 Intake Form Submission - Proposed
Proposed Flow:
6.3 Intake Form Finalization - Proposed
Proposed Flow:
6.4 Intake Form Approval - Proposed
Proposed Flow:
6.5 Intake Form Deletion - Proposed
Proposed Flow:
6.6 Intake Form Rejection - Proposed
Proposed Flow:
6.7 Intake Form Search - Proposed
Proposed Flow:
7. Ingest Pipeline Changes
7.1 Current Ingest Process
Current State:
7.2 Proposed Ingest Process
Proposed State:
7.3 Ingest Configuration Changes
Table: IncomingModelSemanticPattern (metadata)
section_id
intake_form_id
Rename and repoint to intake form
register_id
register_id
Remains unchanged
semantic_expression
semantic_expression
Unchanged logic
8. Register Ingest Process (Celery)
8.1 Register-Ingest-Beat
Purpose: Trigger register ingest for approved intake form submissions
Schedule: Periodic (configurable interval)
Process:
8.2 Register-Ingest-Worker
Purpose: Insert intake form data into register tables
Process:
9. Deduplication Process (Celery)
9.1 Deduplication Overview
Two deduplication processes run in parallel:
Intake-Forms-vs-Intake-Forms: Check new submission against existing submissions
Intake-Forms-vs-Register: Check new submission against existing register records
9.2 Deduplication-Intake-Forms-Beat
Purpose: Trigger deduplication of intake form submissions against other intake form submissions
Schedule: Periodic (configurable interval)
Process:
9.3 Deduplication-Intake-Forms-Worker
Purpose: Find potential duplicates in other approved intake form submissions
Process:
9.4 Deduplication-Register-Beat
Purpose: Trigger deduplication of intake form submissions against existing register records
Schedule: Periodic (configurable interval)
Process:
9.5 Deduplication-Register-Worker
Purpose: Find potential duplicates in existing register records
Process:
10. Controller Changes
10.1 G2PIntakeFormMetadataController
Endpoint Base: /intake-form-metadata/
POST
/create_intake_form
form fields (excluding form_id)
form_id
Insert into g2p_intake_form_definitions
PUT
/update_intake_form
form_id, update fields
form_id
Update specific fields (form_mnemonic, form_description)
DELETE
/delete_intake_form
form_id
Empty
Delete intake form definition
GET
/get_all_intake_forms
register_id (optional), pagination
list[form_definitions]
Retrieve forms for register or all forms
GET
/get_intake_form
form_id
form_definition
Retrieve single form
POST
/create_tab
tab fields (excluding tab_id)
tab_id
Insert into g2p_intake_form_ui_tabs
DELETE
/delete_tab
tab_id
Empty
Delete tab from form
POST
/toggle_tab_status
tab_id
tab_id
Toggle active/inactive status
GET
/get_tab
tab_id
tab_definition
Retrieve tab definition
GET
/get_all_tabs
form_id
list[tabs]
Retrieve all tabs for form
POST
/add_section
tab_id, section_id
tab_section_id
Create mapping in g2p_intake_form_ui_tab_sections
DELETE
/remove_section
tab_section_id
Empty
Remove section from tab
POST
/update_section
tab_section_id, section_order
tab_section_id
Update section order within tab
GET
/get_all_sections
tab_id
list[sections]
Retrieve sections for tab with metadata
10.2 G2PIntakeFormDataController
Endpoint Base: /intake-form-data/
POST
/save_intake_form_submission
form_id, section_payload[], submission_id (optional)
submission_id
Save form data in DRAFT status
POST
/finalize_intake_form_submission
submission_id
Empty
Set draft_status = FINAL
DELETE
/delete_intake_form_submission
submission_id
Empty
Delete submission and all section data
POST
/approve_intake_form_submission
submission_id
Empty
Set approval_status = APPROVED, trigger ingest and dedup
POST
/reject_intake_form_submission
submission_id
Empty
Set approval_status = REJECTED
GET
/get_intake_form_submission
submission_id, section_register_id
submission data
Retrieve submission records
GET
/search_in_intake_form_submissions
register_id, search_text, filter_schema, pagination
list[submissions]
Search across all submissions
10.3 Verification Controller Updates
Endpoint: /verification/
Changes:
Update submission model references when retrieving verification data
Ensure verification records link to submission_id
Update response model to include deduplication status fields
11. New Celery Jobs
11.1 Register Ingest Jobs
register_ingest_beat
Beat
Periodic
None
Emits submission_ids
register_ingest_worker
Worker
On-demand
submission_id
Processed submissions
11.2 Deduplication Jobs
deduplication_intake_forms_beat
Beat
Periodic
None
Emits submission_ids
deduplication_intake_forms_worker
Worker
On-demand
submission_id
Dedup results
deduplication_register_beat
Beat
Periodic
None
Emits submission_ids
deduplication_register_worker
Worker
On-demand
submission_id
Dedup results
12. Change Request vs Intake Form Workflow
12.1 Intake Form Workflow (New Records)
12.2 Change Request Workflow (Edits)
12.3 Key Distinction
Purpose
New record creation
Record modifications
Submission
Form submission
Change request submission
Approval
Staff approval before ingest
Standard CR approval process
Register Update
Via ingest process
Direct CR process
Versioning
Created as V0
Version increments
Deduplication
Yes (on approval)
No
13. Functional Changes Summary
13.1 Section Decoupling
Before:
Sections tightly coupled to tabs
Sections cannot be reused across tabs
Tab is required to define a section
After:
Sections defined independently
Sections mapped to tabs via junction table
Sections reusable across multiple tabs in register and intake forms
Section exists regardless of tab association
13.2 Intake Form Independence
Before:
Intake form submission could modify existing records
Changes submitted as change requests
After:
Intake form ONLY for new record creation
No modifications through intake form
All edits via change request process
Intake form with approval = direct register insertion
13.3 Ingest Pipeline
Before:
Incoming messages routed to change requests
Section resolution via semantic expression
After:
Incoming messages routed to intake forms
Form must be marked as
used_only_for_ingestion_pipeline = TRUEForm resolution via updated semantic expression
Partner-sourced submissions auto-processed
13.4 Deduplication
Before:
No built-in deduplication
After:
Two parallel deduplication processes
Results stored for UI display
Configurable match thresholds
Tracks dedup status across submission lifecycle
14. Data Flow Diagram
15. Implementation Notes
All dynamic class resolution follows pattern:
{module}.{register_mnemonic}Section reuse enabled through junction tables for both register and intake form tabs
Deduplication configurable per domain via configuration
Ingest process idempotent; can safely retry failed submissions
Submission model tracks multiple asynchronous processes (ingest, dedup intake, dedup register)
Abstract base class pattern allows domain-specific intake form extensions
Last updated
Was this helpful?