Feature #91 » Frappe Shipping Bill PDF Extraction to JSON — Master Development Prompt.md
Frappe Shipping Bill PDF → Structured JSON Extraction System
You are acting as a:
- Senior Frappe Framework Solution Architect
- Senior Frappe/Python Developer
- PDF Processing Engineer
- OCR/PDF Extraction Engineer
- Document Processing Architect
I am building a PDF-to-JSON extraction system in the Frappe Framework.
The immediate target is specifically a Shipping Bill PDF such as the sample PDF provided with this project.
The system must eventually be generic enough to support other document types, but V1 must focus on Shipping Bill extraction and produce JSON in the exact structure required by my Frappe Shipping Bill DocType and its child tables.
1. ACTUAL PRODUCT WE ARE BUILDING
We are NOT building a simple PDF text extractor.
We are building:
Shipping Bill PDF
|
v
PDF Analysis
|
v
Detect PDF Type
|
+----------------------+
| |
v v
Text-based PDF Image/Scanned PDF
| |
v v
PDF Text Parser OCR
| |
+----------+-----------+
|
v
Process ALL Pages
|
v
Shipping Bill
Field Extraction
|
v
Rule / Pattern Engine
|
v
Structured Data
|
v
Validation
|
v
User Review
|
v
APPROVE
|
v
Final JSON
|
+-----+------+
| |
v v
Copy JSON Download JSON
Future phase:
Final JSON
|
v
Frappe Mapping Engine
|
v
Shipping Bill DocType
|
+---- Manifest Detail
|
+---- Invoice Detail
|
+---- Item Detail
|
+---- Single Window Declaration
|
+---- Other Child Tables
2. VERY IMPORTANT — V1 DOES NOT REQUIRE PAID AI
Do NOT use:
- OpenAI API
- Gemini API
- Claude API
- Azure Document Intelligence
- AWS Textract
- Any paid cloud OCR/API
- Any paid external document-processing API
The first implementation must work using:
Python
Frappe
PDF processing libraries
OCR if required
Regex / pattern matching
Table extraction
JSON processing
Possible open-source libraries:
PyMuPDF
pdfplumber
pypdf
PaddleOCR
Tesseract
Docling
Camelot
Tabula
Pydantic
JSON Schema
Do not install all of these blindly.
Select only what is actually necessary.
Before adding a dependency, explain:
- Why it is needed.
- What problem it solves.
- Whether an existing dependency can already solve the problem.
The architecture should allow a local AI/LLM fallback in the future, but AI is not part of the V1 dependency.
3. IMPORTANT: USE THE PROVIDED SHIPPING BILL PDF AS THE PRIMARY TEST DOCUMENT
The project includes a sample Shipping Bill PDF.
Use it as the primary development/test document.
The PDF contains 6 pages.
The system must process:
Page 1
Page 2
Page 3
Page 4
Page 5
Page 6
Do NOT process only the first page.
Do NOT assume that the document ends after the first page.
The sample contains:
Page 1
PART - I - SHIPPING BILL SUMMARY
Page 2
PART - II - INVOICE DETAILS
Page 3
PART - III - ITEM DETAILS
Page 4
PART - IV - EXPORT SCHEME DETAILS
Page 5
PART - IV - EXPORT SCHEME DETAILS
Other Additional Information / Reexport Details
Page 6
PART - V - DECLARATIONS
The source PDF confirms the six-page structure.
4. FRAPPE PAGE UI
Create a custom Frappe page.
The page must behave as a 3-step wizard.
STEP 1
Upload PDF
↓
STEP 2
Analyse + Review
↓
STEP 3
JSON Output
The user must be able to clearly see which step they are currently on.
5. STEP 1 — UPLOAD PDF
When the user opens the page:
+--------------------------------------------------+
| SHIPPING BILL PDF EXTRACTOR |
| |
| Upload Shipping Bill PDF |
| |
| [ Choose PDF ] |
| |
| Selected File: None |
| |
| [ NEXT ] |
+--------------------------------------------------+
Requirements:
- Allow PDF upload.
- Accept PDF only.
- Display file name.
- Validate file type.
- Do not start extraction immediately.
- Start processing when the user clicks
NEXT.
After selecting:
Selected File:
563200904082026INBOM4SB22050820261955 (1).pdf
[ NEXT ]
6. STEP 2 — ANALYSE THE PDF
When the user clicks NEXT:
The backend must analyze the uploaded PDF.
First determine:
PDF Type
Possible values:
Text-based PDF
Image-based / Scanned PDF
Mixed PDF
For the determination, inspect every page.
Example:
Page 1 → Text
Page 2 → Text
Page 3 → Text
Page 4 → Text
Page 5 → Text
Page 6 → Text
The overall document can therefore be identified as:
Text-based PDF
if all pages contain meaningful extractable text.
If some pages contain no meaningful text and require OCR:
Mixed PDF
If the entire document requires OCR:
Image-based / Scanned PDF
7. STEP 2 — SHOW PDF INFORMATION
After processing:
PDF ANALYSIS
File Name:
563200904082026INBOM4SB22050820261955 (1).pdf
Document:
Shipping Bill
PDF Type:
Text-based PDF
Total Pages:
6
Pages Processed:
6 / 6
Extraction Status:
Completed
Also show processing time.
Example:
Processing Time:
40.91 seconds
The actual processing time must be generated by the application.
Do NOT hardcode it.
8. PROCESS EVERY PAGE
The system must process the PDF page-by-page.
Internally:
PDF
|
+--- Page 1
|
+--- Page 2
|
+--- Page 3
|
+--- Page 4
|
+--- Page 5
|
+--- Page 6
Each page must produce extracted information.
The system must preserve:
page_number
section
field
value
where possible.
Example:
{
"page": 1,
"section": "Shipping Bill Summary",
"field": "shipping_bill_no",
"value": "5632009"
}
9. DO NOT JUST RETURN RAW PDF TEXT
This is extremely important.
The following is NOT the desired result:
AIR CARGO COMPLEX...
Port Code...
SB No...
SB Date...
IEC...
...
Instead, convert the document into structured business information.
For example:
SB No
5632009
must become:
{
"shipping_bill_no": "5632009"
}
Similarly:
Port Code
INBOM4
becomes:
{
"port_code": "INBOM4"
}
10. SHIPPING BILL JSON IS THE TARGET
The final JSON must follow the structure expected by the Frappe Shipping Bill DocType.
The target top-level structure is:
{
"shipping_bill_json": {
"doctype": "Shipping Bill",
"port_code": "",
"gwt": 0,
"inv": 0,
"shipping_bill_no": "",
"port_of_loading": "",
"cntry_of_finaldstn": "",
"state_of_origin": "",
"port_of_finaldstn": "",
"port_of_discharge": "",
"cntry_of_discharge": "",
"gwt_unit": "",
"item": 0,
"shipping_bill_date": "",
"pkg": 0,
"cont": 0,
"iec": "",
"iec_branch_code": "",
"cb_code": "",
"p_1a_mode": "",
"p_1a_assess": "",
"p_1a_dbk": "",
"p_1a_re_exp": null,
"p_1a_exam": "",
"p_1a_rodtp": "",
"p_1a_lut": "",
"p_1a_job": "",
"p_1a_licence": "",
"p_1a_dfrc": "",
"p_1b_exporter_name": "",
"p_1b_exporter_address": "",
"p_1b_cb_name": "",
"p_1b_consignee_name": "",
"p_1b_consignee_address": "",
"p_1b_type": "",
"p_1b_gstin": "",
"p_1b_ad_code": "",
"p_1b_forex_ac_no": "",
"p_1c_fob_val": 0,
"p_1c_com": 0,
"p_1c_freight": 0,
"p_1c_deductions": 0,
"p_1c_insurance": 0,
"p_1c_p_c": 0,
"p_1c_discount": 0,
"p_1d_dbk": 0,
"p_1d_rodtep_amt": 0,
"p_1d_rosctle_amt": 0,
"p_1j_subm_dt": "",
"p_1i_subm_time": "",
"p_1i_leo_no": "",
"p_1i_leo_dt": "",
"p_1i_exmn_dt": "",
"p_1i_exmn_time": "",
"p_1i_leo_time": "",
"p_1i_brc_realzn_dt": "",
"manifest_details": [],
"challan_details": [],
"annex_details": [],
"invoice_details": [],
"aa__dfia_licence_details": [],
"item_details": [],
"drawback_and_rosl_claim": [],
"job_details": [],
"single_window_declaration": [],
"single_window_declaration_constituents": [],
"single_window_declaration_control": [],
"supporting_documents": [],
"ar4_details": [],
"third_party_details": [],
"item_manufacturerproducergrower_details": [],
"rodtep_details": [],
"container_details": []
},
"file_id": "",
"processing_time": ""
}
This is the target structure.
The actual values must be extracted from the PDF.
11. IMPORTANT — FIELD NAMES MUST MATCH THE TARGET
Do not rename fields.
For example:
Use:
shipping_bill_no
NOT:
shipping_bill_number
Use:
p_1b_exporter_name
NOT:
exporter_name
Use:
p_1c_fob_val
NOT:
fob_value
The field names are intentionally aligned with the Frappe DocType.
Preserve them exactly.
12. SHIPPING BILL HEADER FIELDS
The system must extract fields such as:
port_code
gwt
inv
shipping_bill_no
shipping_bill_date
pkg
cont
iec
iec_branch_code
cb_code
gwt_unit
item
For the sample PDF, these values are visible in the header:
Port Code: INBOM4
SB No: 5632009
SB Date: 04-AUG-26
IEC/Br: 0300014953
CB Code: AAACM6824HCH032
G.WT: 1143 KGS
The source document shows these values on page 1.
13. SHIPPING BILL STATUS FIELDS
Extract:
p_1a_mode
p_1a_assess
p_1a_dbk
p_1a_re_exp
p_1a_exam
p_1a_rodtp
p_1a_lut
p_1a_job
p_1a_licence
p_1a_dfrc
The values must be taken from the actual Shipping Bill table.
Do not infer values from general knowledge.
The source PDF contains the status fields in the Part-I Shipping Bill Summary.
14. EXPORTER / CONSIGNEE / DECLARANT DETAILS
Extract:
p_1b_exporter_name
p_1b_exporter_address
p_1b_cb_name
p_1b_consignee_name
p_1b_consignee_address
p_1b_type
p_1b_gstin
p_1b_ad_code
p_1b_forex_ac_no
Do not merge exporter and consignee data.
Keep them in their respective fields.
The sample PDF contains exporter information beginning on page 1 and consignee information in the declarant section.
15. VALUE SUMMARY
Extract:
p_1c_fob_val
p_1c_com
p_1c_freight
p_1c_deductions
p_1c_insurance
p_1c_p_c
p_1c_discount
p_1d_dbk
p_1d_rodtep_amt
p_1d_rosctle_amt
Convert numeric values into appropriate JSON numbers.
Example:
{
"p_1c_fob_val": 18956370.0,
"p_1c_com": 0.0
}
Do not return numeric values as strings unless the target field specifically requires a string.
The sample's value summary appears in Part-I on page 1.
16. PROCESS DETAILS
Extract:
p_1j_subm_dt
p_1i_subm_time
p_1i_leo_no
p_1i_leo_dt
p_1i_exmn_dt
p_1i_exmn_time
p_1i_leo_time
p_1i_brc_realzn_dt
The sample contains these events in the Process Details section on page 1.
Normalize dates to:
YYYY-MM-DD
Normalize times to:
HH:MM:SS
For example:
04-AUG-26
becomes:
2026-08-04
17. CHILD TABLES
The JSON must support Frappe child tables.
This is critical.
Do not flatten child-table data into the parent.
For example:
"manifest_details": [
{
"doctype": "Manifest Detail",
...
}
]
This structure must be preserved.
18. MANIFEST DETAILS
Extract the E-Manifest information.
Target structure:
"manifest_details": [
{
"doctype": "Manifest Detail",
"p_1e_mawb_no": "",
"p_1e_cin_no": "",
"p_1e_cin_dt": "",
"p_1e_cin_site_id": ""
}
]
The sample PDF contains:
MAWB NO
CIN NO
CIN DT
CIN SITE ID
on page 1.
If multiple manifest rows exist, create multiple objects.
19. CHALLAN DETAILS
Target:
"challan_details": []
If rows exist, populate them.
If the section is empty:
"challan_details": []
Do not invent records.
20. ANNEX DETAILS
Target:
"annex_details": [
{
"doctype": "Annex Detail",
"p_1i_seal_typ": "",
"p_1i_loose_pkts": 0,
"p_1i_nature_cargo": "",
"p_1i_marks_numbers": "",
"p_1i_no_of_pkgs": 0,
"p_1i_no_of_containers": 0
}
]
The sample contains Annex Details on page 1, including:
SEAL TYPE
NATURE OF CARGO
NO. OF PACKETS
NO. OF CONTAINERS
LOOSE PACKETS
MARKS & NUMBERS
21. INVOICE DETAILS
The Invoice Details section begins on page 2.
Target:
"invoice_details": [
{
"doctype": "Invoice Detail",
"inv_sn": 1,
"p_2a_inv_no": "",
"p_2a_inv_dt": "",
"p_2a_ad_code": "",
"p_2a_invterm": "",
"p_2b_exporter_name": "",
"p_2b_buyer_addr": "",
"p_2b_exporter_addr": "",
"p_2b_buyer_name": "",
"p_2c_invoice_value": 0,
"p_2c_freight": 0,
"p_2c_discount": 0,
"p_2c_deduct": 0,
"p_2c_exchng_rate_desc": "",
"p_2c_invoice_curr": "",
"p_2c_frieght_curr": "",
"p_2c_discount_curr": "",
"p_2c_deduct_curr": null,
"p_2c_fob_val": 0,
"p_2c_insurance": 0,
"p_2c_commison": 0,
"p_2c_p_c": 0,
"p_2c_fob_curr": "",
"p_2c_insurance_curr": "",
"p_2c_commison_curr": null,
"p_2c_p_c_curr": null
}
]
The sample invoice information is visible on page 2, including invoice number/date, exporter, buyer, invoice value, FOB value, exchange rate and currency.
22. ITEM DETAILS
This is one of the most important sections.
The sample Item Details are spread across pages 2 and 3.
The system must understand that:
Page 2
Invoice item information begins
↓
Page 3
Part III Item Details contains the complete item information
Do not create duplicate item rows simply because the same item appears on multiple pages.
The system must combine the information belonging to the same item.
Target:
"item_details": [
{
"p_3a_invsno": "",
"p_3a_itemsn": "",
"p_3a_cth": "",
"p_3a_item_desc": "",
"p_3a_qty": "",
"p_3a_uqc": "",
"p_3a_rate": "",
"p_3a_value": "",
"p_3a_fob": "",
"p_3a_pmv": "",
"p_3a_duty_amt": "",
"p_3a_cess_rate": "",
"p_3a_cess_amt": "",
"p_3a_dbk_claimed": "",
"p_3a_igststat": "",
"p_3a_igst_val": "",
"p_3a_igst_amt": "",
"p_3a_schcod": "",
"p_3a_scheme_desc": "",
"p_3a_sqc_mst": "",
"p_3a_sqc_uqc": "",
"p_3a_state_of_origin": "",
"p_3a_district_of_origin": "",
"p_3a_pt_abroad": "",
"p_3a_comp_cess": 0,
"p_3a_end_use": "",
"p_3a_benefit_availd": "",
"p_3a_reward_benefit": "",
"p_3a_third_party_item": "",
"doctype": "Item Detail"
}
]
The sample's Part III contains fields including HS code, description, quantity, UQC, rate, value, FOB, PMV, duty, IGST status, scheme, state/district of origin, end use, FTA benefit and third-party item.
23. SINGLE WINDOW DECLARATION
This is another child table.
Target:
"single_window_declaration": [
{
"p_4d_invsn": "",
"p_4d_itmsn": "",
"p_4d_info": "",
"p_4d_qualifier": "",
"p_4d_info_cd": "",
"p_4d_info_text": "",
"p_4d_info_msr": "",
"p_4d_uqc": "",
"doctype": "Single Window Declaration"
}
]
The sample contains multiple rows:
CHR
DTY / GCESS
DTY / RDT
ORC / DOO
ORC / EPT
ORC / STO
These must become separate JSON objects.
The source table appears on page 4.
24. OTHER CHILD TABLES
Preserve the following target keys:
aa__dfia_licence_details
drawback_and_rosl_claim
job_details
single_window_declaration_constituents
single_window_declaration_control
supporting_documents
ar4_details
third_party_details
item_manufacturerproducergrower_details
rodtep_details
container_details
If the corresponding section is empty:
[]
Do not remove the key.
Do not generate fake records.
The sample PDF contains several empty sections in Part IV, so the final JSON should preserve those arrays as empty arrays.
25. EMPTY VALUES
Use appropriate empty values.
For fields that are genuinely absent:
null
can be used where appropriate.
For child tables:
[]
must be used when there are no records.
Do not fabricate values.
For example, if no container exists:
"container_details": []
not:
"container_details": [
{
"container": "UNKNOWN"
}
]
26. DO NOT CONFUSE PDF HEADER REPEATS WITH NEW DATA
The Shipping Bill repeats common header information on every page.
For example:
Port Code
SB No
SB Date
IEC
CB Code
GSTIN
G.WT
appear repeatedly.
Do NOT create duplicate parent fields because the same header is repeated on pages 2–6.
The repeated header should be recognized as document-level metadata.
27. DO NOT DUPLICATE CHILD RECORDS
This is critical.
For example, an item may appear on:
Page 2
Page 3
The system must understand that it is the same item.
Expected:
"item_details": [
{
...
}
]
NOT:
"item_details": [
{
...
},
{
...
}
]
unless there are genuinely two different items.
Use invoice serial number + item serial number or other available identifiers to correlate records.
28. DATA NORMALIZATION
Normalize values appropriately.
Dates
Convert:
04-AUG-26
29/07/2026
05-AUG-26
to:
2026-08-04
2026-07-29
2026-08-05
Times
Convert:
11:40
19:01
19:55
to:
11:40:00
19:01:00
19:55:00
Numbers
Where the target field is numeric:
18,956,370
should become:
18956370.0
Do not include commas in numeric JSON values.
29. IMPORTANT: DO NOT CHANGE BUSINESS VALUES
The PDF is the source of truth.
Do not "correct" values using external assumptions.
For example:
If the PDF says:
INBOM4
return:
"port_code": "INBOM4"
If the PDF says:
FOB
return:
"p_2a_invterm": "FOB"
Do not replace values because they appear unusual.
If OCR produces an uncertain value, flag it rather than silently changing it.
30. SOURCE TRACEABILITY
Internally maintain source information wherever possible.
For example:
{
"field": "shipping_bill_no",
"value": "5632009",
"page": 1
}
For child records:
{
"item_serial": "1",
"page": 3
}
The user-facing final JSON does not necessarily need to include all provenance metadata unless required by the target DocType.
However, the backend should preserve enough information for debugging.
31. STEP 2 UI — ORGANIZED TABLE
The extracted information must NOT be displayed as a huge JSON block in Step 2.
Step 2 should be a human-review interface.
Use organized sections.
Example:
==================================================
PDF ANALYSIS
==================================================
Document Type: Shipping Bill
PDF Type: Text-based PDF
Pages: 6
Processed: 6 / 6
Status: Completed
==================================================
SHIPPING BILL SUMMARY
==================================================
| Field | Value | Page |
|-------|-------|------|
| Port Code | INBOM4 | 1 |
| SB No | 5632009 | 1 |
| SB Date | 04-08-2026 | 1 |
| IEC | 0300014953 | 1 |
| CB Code | AAACM6824HCH032 | 1 |
| G.WT | 1143 KGS | 1 |
==================================================
EXPORTER / CONSIGNEE
==================================================
| Field | Value | Page |
|-------|-------|------|
| Exporter | ... | 1 |
| Consignee | ... | 1 |
| CB Name | ... | 1 |
==================================================
INVOICE DETAILS
==================================================
| Field | Value | Page |
|-------|-------|------|
| Invoice No | ... | 2 |
| Invoice Date | ... | 2 |
| Invoice Value | ... | 2 |
==================================================
ITEM DETAILS
==================================================
| Item | HS Code | Description | Qty | UQC | Rate | Value |
|------|---------|-------------|-----|-----|------|-------|
==================================================
SINGLE WINDOW DECLARATION
==================================================
| Info | Qualifier | Code | Text | Measure | UQC |
|------|-----------|------|------|---------|-----|
[ APPROVE ]
The exact UI can use Frappe's styling and components.
32. STEP 2 SHOULD SHOW ALL EXTRACTED DATA
Do not only display the fields that are convenient.
The objective is to allow the user to review the data that will become JSON.
The review screen should cover:
Parent fields
+
Child tables
+
Empty sections where relevant
For large tables, use collapsible sections or tabs.
33. APPROVE BUTTON
The button should be:
[ APPROVE & GENERATE JSON ]
When clicked:
- Validate the extracted structure.
- Confirm there are no fatal extraction errors.
- Build final JSON.
- Store the approved JSON.
- Move to Step 3.
Do not navigate to Step 3 before successful validation.
34. STEP 3 — JSON OUTPUT
After approval:
STEP 3 — JSON OUTPUT
Display the final JSON in a code editor / formatted code block.
The JSON should look like:
{
"shipping_bill_json": {
"doctype": "Shipping Bill",
"port_code": "INBOM4",
"gwt": 1143.0,
"inv": 1,
"shipping_bill_no": "5632009",
"port_of_loading": "INBOM4 (Mumbai (Ex Bombay))",
"cntry_of_finaldstn": "SAUDI ARABIA",
...
},
"file_id": "generated-file-id",
"processing_time": "40.91s"
}
The actual JSON must contain all required fields and child tables.
35. COPY JSON
Step 3 must contain:
[ COPY JSON ]
When clicked:
JSON copied successfully.
The copied content must be valid JSON.
36. DOWNLOAD JSON
Also provide:
[ DOWNLOAD JSON ]
The downloaded file should be:
shipping_bill_<shipping_bill_no>.json
For example:
shipping_bill_5632009.json
37. FILE ID
The:
"file_id": ""
value is application metadata.
Do not attempt to extract it from the PDF.
It should be generated by the application when the PDF is uploaded/processed.
38. PROCESSING TIME
The:
"processing_time": ""
value must be measured by the backend.
Example:
Start timer
↓
PDF processing
↓
Extraction
↓
Validation
↓
End timer
Then:
"processing_time": "40.91s"
Do not hardcode this.
39. FINAL JSON IS THE CONTRACT
Treat the final JSON as the contract between:
PDF Extraction Engine
and:
Frappe DocType Mapping Engine
Therefore:
PDF
↓
Extraction
↓
Structured JSON
↓
Validation
↓
APPROVAL
↓
FINAL JSON
must be independent from the future DocType creation logic.
40. FUTURE DOCType MAPPING
Do not implement this immediately unless requested.
Future flow:
Approved JSON
|
v
Shipping Bill Mapper
|
v
Shipping Bill DocType
|
+--- Manifest Detail
|
+--- Invoice Detail
|
+--- Annex Detail
|
+--- Item Detail
|
+--- Single Window Declaration
|
+--- Other child tables
The final JSON structure is intentionally designed so that it can later be mapped directly to Frappe.
41. EXPECTED TARGET JSON
The following is the example structure that the system should reproduce from the provided Shipping Bill PDF.
{
"shipping_bill_json": {
"doctype": "Shipping Bill",
"port_code": "INBOM4",
"gwt": 1143.0,
"inv": 1,
"shipping_bill_no": "5632009",
"port_of_loading": "INBOM4 (Mumbai (Ex Bombay))",
"cntry_of_finaldstn": "SAUDI ARABIA",
"state_of_origin": "Maharashtra",
"port_of_finaldstn": "JED (JEDDAH )",
"port_of_discharge": "JED (JEDDAH )",
"cntry_of_discharge": "SAUDI ARABIA",
"gwt_unit": "KGS",
"item": 1,
"shipping_bill_date": "2026-08-04",
"pkg": 2,
"cont": 0,
"iec": "0300014953",
"iec_branch_code": "0",
"cb_code": "AAACM6824HCH032",
"p_1a_mode": "AIR",
"p_1a_assess": "N",
"p_1a_dbk": "N",
"p_1a_re_exp": null,
"p_1a_exam": "Y",
"p_1a_rodtp": "N",
"p_1a_lut": "Y",
"p_1a_job": "N",
"p_1a_licence": "N",
"p_1a_dfrc": "N",
"p_1b_exporter_name": "ENDRESS + HAUSER (INDIA) PRIVATE LIMITED",
"p_1b_exporter_address": "7B,7TH FLR.GODREJ ONE,PIROJSHANAGAR\nEASTERN EXPRESS HIGHWAY Contact No:\nVIKHROLI EAST",
"p_1b_cb_name": "M/S.DHL LOGISTICS PVT LTD.",
"p_1b_consignee_name": "NEOM GREEN HYDROGEN COMPANY LTD",
"p_1b_consignee_address": "KINGDOM OF SAUDI ARABIA CR# 3550139\n610 P.O BOX 8244 BUILDING NO. 4758 KHURAYBAH 49643",
"p_1b_type": "Private",
"p_1b_gstin": "27AAACE5283C1ZV GSN",
"p_1b_ad_code": "6550001",
"p_1b_forex_ac_no": "51XXXX000",
"p_1c_fob_val": 18956370.0,
"p_1c_com": 0.0,
"p_1c_freight": 0.0,
"p_1c_deductions": 0.0,
"p_1c_insurance": 0.0,
"p_1c_p_c": 0.0,
"p_1c_discount": 0.0,
"p_1d_dbk": 0.0,
"p_1d_rodtep_amt": 0.0,
"p_1d_rosctle_amt": 0.0,
"p_1j_subm_dt": "2026-08-04",
"p_1i_subm_time": "11:40:00",
"p_1i_leo_no": "2/838",
"p_1i_leo_dt": "2026-08-05",
"p_1i_exmn_dt": "2026-08-05",
"p_1i_exmn_time": "19:01:00",
"p_1i_leo_time": "19:55:00",
"p_1i_brc_realzn_dt": "2027-05-31",
"manifest_details": [
{
"doctype": "Manifest Detail",
"p_1e_mawb_no": "06547093771",
"p_1e_cin_no": "26PCEG08053702299400",
"p_1e_cin_dt": "05-AUG-26",
"p_1e_cin_site_id": "INBOM4"
}
],
"challan_details": [],
"annex_details": [
{
"doctype": "Annex Detail",
"p_1i_seal_typ": "WAREHOUSE SEALED",
"p_1i_loose_pkts": 2,
"p_1i_nature_cargo": "PACKAGED",
"p_1i_marks_numbers": "AS PER INVOICE & PACKING LIST: SUPPLY MEANT FOR EXPORT FOR AUTHORISED OPERATIONS UNDER LETTER OF UNDERTAKING WITHOUT PAYMENT OF INTEGRATED TAX. ARN No. AD2703260673407 DT: 24/03/2026. AEO NO: INAAACE5283C3F263 DT: 29.05.2026 VALID UPTO 28.05.2031",
"p_1i_no_of_pkgs": 2,
"p_1i_no_of_containers": 0
}
],
"invoice_details": [
{
"doctype": "Invoice Detail",
"inv_sn": 1,
"p_2a_inv_no": "6031962610",
"p_2a_inv_dt": "2026-07-29",
"p_2a_ad_code": "6550001",
"p_2a_invterm": "FOB",
"p_2b_exporter_name": "ENDRESS + HAUSER (INDIA) PRIVATE LIMITED",
"p_2b_buyer_addr": "KINGDOM OF SAUDI ARABIA CR# 3550139\n610 P.O BOX 8244 BUILDING NO. 4758\nKHURAYBAH 49643",
"p_2b_exporter_addr": "7B,7TH FLR.GODREJ ONE,PIROJSHANAGAR\nEASTERN EXPRESS HIGHWAY Contact No:\n400079",
"p_2b_buyer_name": "NEOM GREEN HYDROGEN COMPANY LTD",
"p_2c_invoice_value": 198600.0,
"p_2c_freight": 0.0,
"p_2c_discount": 0.0,
"p_2c_deduct": 0.0,
"p_2c_exchng_rate_desc": "1 USD INR 95.45",
"p_2c_invoice_curr": "USD",
"p_2c_frieght_curr": "USD",
"p_2c_discount_curr": "USD",
"p_2c_deduct_curr": null,
"p_2c_fob_val": 198600.0,
"p_2c_insurance": 0.0,
"p_2c_commison": 0.0,
"p_2c_p_c": 0.0,
"p_2c_fob_curr": "USD",
"p_2c_insurance_curr": "USD",
"p_2c_commison_curr": null,
"p_2c_p_c_curr": null
}
],
"aa__dfia_licence_details": [],
"item_details": [
{
"p_3a_invsno": "1",
"p_3a_itemsn": "1",
"p_3a_cth": "90268090",
"p_3a_item_desc": "IN GM GASMETER- FLOW METER FOR GAS MEASUREMENT, E&H FLOWMETER: TAG: 761-FE/FIT-8 818 (COO: GERMANY) AS PER PAC LIST SET",
"p_3a_qty": "2",
"p_3a_uqc": "SET",
"p_3a_rate": "99300",
"p_3a_value": "198600",
"p_3a_fob": "18956370",
"p_3a_pmv": "10426003.5",
"p_3a_duty_amt": "",
"p_3a_cess_rate": "",
"p_3a_cess_amt": "",
"p_3a_dbk_claimed": "N",
"p_3a_igststat": "LUT",
"p_3a_igst_val": "",
"p_3a_igst_amt": "",
"p_3a_schcod": "00",
"p_3a_scheme_desc": "Free SB Involving Remittance O",
"p_3a_sqc_mst": "2",
"p_3a_sqc_uqc": "NOS",
"p_3a_state_of_origin": "Maharashtra",
"p_3a_district_of_origin": "MUMBAI",
"p_3a_pt_abroad": "NCPTI",
"p_3a_comp_cess": 0.0,
"p_3a_end_use": "GNX100",
"p_3a_benefit_availd": "Y",
"p_3a_reward_benefit": "No",
"p_3a_third_party_item": "N",
"doctype": "Item Detail"
}
],
"drawback_and_rosl_claim": [],
"job_details": [],
"single_window_declaration": [
{
"p_4d_invsn": "1",
"p_4d_itmsn": "1",
"p_4d_info": "CHR",
"p_4d_qualifier": "SQC",
"p_4d_info_cd": "",
"p_4d_info_text": "",
"p_4d_info_msr": "2",
"p_4d_uqc": "NOS",
"doctype": "Single Window Declaration"
},
{
"p_4d_invsn": "1",
"p_4d_itmsn": "1",
"p_4d_info": "DTY",
"p_4d_qualifier": "GCESS",
"p_4d_info_cd": "",
"p_4d_info_text": "",
"p_4d_info_msr": "0",
"p_4d_uqc": "INR",
"doctype": "Single Window Declaration"
},
{
"p_4d_invsn": "1",
"p_4d_itmsn": "1",
"p_4d_info": "DTY",
"p_4d_qualifier": "RDT",
"p_4d_info_cd": "RODTEPN",
"p_4d_info_text": "Not Claimed",
"p_4d_info_msr": "",
"p_4d_uqc": "",
"doctype": "Single Window Declaration"
},
{
"p_4d_invsn": "1",
"p_4d_itmsn": "1",
"p_4d_info": "ORC",
"p_4d_qualifier": "DOO",
"p_4d_info_cd": "482",
"p_4d_info_text": "",
"p_4d_info_msr": "",
"p_4d_uqc": "",
"doctype": "Single Window Declaration"
},
{
"p_4d_invsn": "1",
"p_4d_itmsn": "1",
"p_4d_info": "ORC",
"p_4d_qualifier": "EPT",
"p_4d_info_cd": "NCPTI",
"p_4d_info_text": "",
"p_4d_info_msr": "",
"p_4d_uqc": "",
"doctype": "Single Window Declaration"
},
{
"p_4d_invsn": "1",
"p_4d_itmsn": "1",
"p_4d_info": "ORC",
"p_4d_qualifier": "STO",
"p_4d_info_cd": "27",
"p_4d_info_text": "",
"p_4d_info_msr": "",
"p_4d_uqc": "",
"doctype": "Single Window Declaration"
}
],
"single_window_declaration_constituents": [],
"single_window_declaration_control": [],
"supporting_documents": [],
"ar4_details": [],
"third_party_details": [],
"item_manufacturerproducergrower_details": [],
"rodtep_details": [],
"container_details": []
},
"file_id": "generated-file-id",
"processing_time": "40.91s"
}
Important: The above JSON is a target structure/example. The actual application must always take the PDF as the source of truth and extract the values from the uploaded document. Do not blindly copy example values into every document.
42. DEVELOPMENT RULE
Do not try to implement everything at once.
Work in this order:
1. Frappe Page
↓
2. PDF Upload
↓
3. PDF Detection
↓
4. Page Count
↓
5. Text Extraction
↓
6. OCR fallback
↓
7. Shipping Bill Section Detection
↓
8. Parent Field Extraction
↓
9. Child Table Extraction
↓
10. JSON Construction
↓
11. JSON Validation
↓
12. Step 2 Review UI
↓
13. Approve
↓
14. Step 3 JSON Output
↓
15. Copy JSON
↓
16. Download JSON
↓
17. Future DocType Creation
43. HOW TO GUIDE ME
I am not asking for a theoretical architecture only.
I want to actually build this application.
When I provide my existing Frappe files/code:
- Analyze the existing implementation first.
- Do not replace working code unnecessarily.
- Tell me exactly which files need modification.
- Tell me whether a new file is required.
- Give me complete replacement code where appropriate.
- Keep code compatible with my existing Frappe application.
- Do not assume files/classes exist unless I provide them.
- Do not invent DocType fields.
- Preserve the exact JSON field names.
- Implement one step at a time.
- Test each stage logically before moving to the next.
- If an extraction problem is caused by the PDF layout, explain the exact problem and modify the extraction logic accordingly.
- Do not solve extraction problems by introducing paid AI.
- Do not use AI unless I explicitly approve it.
44. FIRST IMPLEMENTATION TARGET
Do NOT start with DocType creation.
Our immediate milestone is:
Frappe Page
↓
Upload Shipping Bill PDF
↓
Next
↓
Detect Text/Image/Mixed
↓
Process ALL 6 pages
↓
Extract Shipping Bill fields
↓
Display organized review tables
↓
User clicks APPROVE
↓
Generate exact Shipping Bill JSON
↓
Step 3
↓
Display formatted JSON
↓
Copy JSON
↓
Download JSON
Once this works correctly with the supplied 6-page Shipping Bill PDF, we will move to the next phase:
Approved JSON
↓
Frappe Shipping Bill DocType
↓
Child Tables
↓
Create Document
Treat this architecture and workflow as the canonical project plan for this implementation.