# NSDL SEZ BOE/DTA Automation — Architecture & Runbook

*TEST stack · ap-south-1 · `nsdl-sez-boe-automation-test`*

A two-Lambda AWS pipeline that logs into NSDL's SEZ Online portal every day, downloads Bill of
Entry and DTA Sale trade data, quality-checks and stages it per client, then pushes it into each
client's Canofy/Frappe site as `FTWZ BOE Import` / `FTWZ DTA Sale` records with the original NSDL
PDF attached.

| | | | | | |
|---|---|---|---|---|---|
| **2** Lambda functions | **3** DynamoDB tables | **1** external portal (NSDL) | **6** QC checks per pipeline | **900s** Lambda timeout (max) | **N** client sites (multi-tenant) |

---

## Contents

1. [Overview](#00--overview)
2. [Architecture at a glance](#01--architecture)
3. [Record lifecycle](#02--data-model)
4. [Phase A — Fetch](#03--phase-a)
5. [Phase B — Deliver](#04--phase-b)
6. [Watchdog](#05--watchdog)
7. [The Frappe side](#06--frappe-integration)
8. [Configuration reference](#07--reference)
9. [Hardening notes](#08--hardening)
10. [Setup checklist](#09--setup)
11. [Open items](#10--open-items)

---

## 00 — Overview

Every SEZ/FTWZ unit's import (Bill of Entry) and domestic-tariff-area sale (DTA Sale) transactions
are recorded on NSDL's government SEZ Online portal, one entry at a time, with no bulk export API
— only a browser UI. This automation replaces the manual "log in, export, re-key into Frappe,
print, re-attach the PDF" routine with two scheduled Lambda functions either side of a DynamoDB
staging table.

**Phase A — Fetch.** Once a day, logs into NSDL, exports BE + DTA for a rolling 7-day window,
quality-checks every request, and stages new records per client in DynamoDB.
*(PhaseAFunction · EventBridge daily)*

**Phase B — Deliver.** Triggered per-client by that client's own Frappe site. Pushes staged
records into Canofy, then logs back into NSDL to fetch and attach the official PDF for each
record. *(PhaseBFunction · API Gateway)*

**Watchdog.** Runs hourly. Flags any client whose records have sat in `STAGED` for more than 24
hours — either a dead Frappe scheduler or a chronically failing push. *(WatchdogFunction ·
`rate(1 hour)`)*

> **Multi-tenant by design.** One AWS deployment serves every client. Phase A stages records for
> *all* configured clients from a single NSDL login; Phase B is called separately by each client's
> own Frappe site, identifying itself via an `X-Frappe-Client` header so Phase B knows whose
> records to push and which Frappe credentials to use.

---

## 01 — Architecture

Every resource in the deployed CloudFormation stack, and how data moves between them.

```mermaid
flowchart LR
    NSDL[["NSDL SEZ Online Portal<br/>(external, no API)"]]

    subgraph AWS["AWS · nsdl-sez-boe-automation-test"]
      direction TB
      EB["EventBridge<br/>cron, daily"] --> PA["PhaseAFunction<br/>Playwright + EasyOCR"]
      PA --> S3["S3 RawFilesBucket<br/>raw BE/DTA exports"]
      PA --> RCT["RunControlTable<br/>per-day idempotency"]
      PA --> SRT[("StagedRecordsTable<br/>per-record status")]
      PA --> RGT["RunGateTable<br/>per-client QC summary"]

      APIGW["API Gateway<br/>/sez/deliver-pending"] --> PB["PhaseBFunction<br/>Playwright + EasyOCR"]
      PB --> SRT
      PB -.async failure.-> DLQ["SQS DLQ"] --> ALM1["CloudWatch Alarm"]

      WD["WatchdogFunction<br/>hourly"] --> SRT
      WD --> SNS(["SNS AlertsTopic"])
      ALM1 --> SNS
      ALM2["CloudWatch Alarm<br/>PhaseA Errors"] --> SNS
      PA -.errors.-> ALM2

      SEC[("Secrets Manager<br/>NSDL creds + per-client Frappe creds")]
      PA --> SEC
      PB --> SEC
    end

    PA <--> NSDL
    PB <--> NSDL
    PB <--> FRAPPE["Client Frappe / Canofy site<br/>FTWZ BOE Import · FTWZ DTA Sale"]
    FRAPPE -- "daily scheduled job" --> APIGW
    SNS --> HUMAN["Email / Slack<br/>(human subscribes post-deploy)"]
```

*Fig. 1 — Every resource in the CloudFormation stack (`template.yaml`) and the two external
systems it bridges: NSDL and each client's Frappe site.*

**Why two identical-looking Lambdas.** Both PhaseA and PhaseB ship the same heavy container image
(Playwright + Chromium + EasyOCR + pandas/opencv, ~4GB) because both need a real NSDL browser
session — Phase A to export data, Phase B to fetch each record's official PDF. Neither is a
lightweight HTTP function.

**Why DynamoDB sits in the middle.** Phase A never talks to Frappe, and Phase B never talks to
NSDL's export screens. `StagedRecordsTable` is the sole handoff — this is what lets Phase A run
once for every client while each client's Phase B call stays independent and retry-safe.

---

## 02 — Data model

Every NSDL request (identified by its `request_id`) becomes exactly one row in
`StagedRecordsTable`, hashed on `request_id` with a `client_name-status-index` GSI for Phase B's
per-client queries.

**STAGED → PUSHED → ATTACHED** / **QC_MISMATCH** / *(error_type set, status unchanged)*

```mermaid
stateDiagram-v2
    [*] --> STAGED: Phase A stages a new,<br/>QC-passed request_id
    STAGED --> QC_MISMATCH: check_6 IEC consistency<br/>failed for this group
    STAGED --> PUSHED: Phase B — Frappe record<br/>created or found (get_record)
    PUSHED --> ATTACHED: Phase B — NSDL PDF<br/>fetched and attached
    STAGED --> STAGED: push failed —<br/>error_type/retry_count set,<br/>status unchanged, retried next run
    PUSHED --> PUSHED: PDF fetch/attach failed —<br/>same retry-in-place pattern
    QC_MISMATCH --> [*]
    ATTACHED --> [*]
```

*Fig. 2 — A failed push or PDF attach never moves the record to a dead-end status: it stays
exactly where it was, with `error_type` / `error_message` / `retry_count` set, so the next
invocation retries it automatically. The Watchdog is what catches a record stuck like this for too
long.*

| Field | Meaning |
|---|---|
| `request_id` | NSDL's own request ID — the table's hash key, and the natural idempotency key across every stage. |
| `client_name` | Which onboarded client this belongs to — the GSI hash key for Phase B's per-client queries. |
| `pipeline` | `BE` or `DTA` — decides the Frappe doctype; both now share one PDF path. |
| `status` | STAGED · PUSHED · ATTACHED · QC_MISMATCH (see Fig. 2). |
| `payload` | The full JSON-safe Frappe document body, built once at staging time and never re-derived. |
| `frappe_docname` | Set on PUSHED — the created/found Frappe document's real name, needed for the PDF-attach call. |
| `error_type` / `error_message` / `retry_count` | Set by a failed attempt; status is deliberately left unchanged (see Fig. 2 caption). |
| `ttl` | Only ever set on ATTACHED (`STAGED_TTL_DAYS`, default 60) — non-terminal rows must never expire mid-retry. |

---

## 03 — Phase A

`app/phase_a_fetch/` · one Lambda, one daily invocation, every client at once.

```mermaid
sequenceDiagram
    participant EB as EventBridge
    participant PA as PhaseAFunction
    participant RCT as RunControlTable
    participant NSDL as NSDL Portal
    participant S3 as S3
    participant Parse as Parser + QC
    participant SRT as StagedRecordsTable
    participant RGT as RunGateTable

    EB->>PA: cron daily · {force_refresh:false}
    PA->>RCT: get_run_control(run_date)
    alt already PASSED today
        RCT-->>PA: reuse existing S3 files
    else not yet run
        PA->>NSDL: login (Playwright + EasyOCR CAPTCHA)
        PA->>NSDL: export BILL OF ENTRY + DTA SALE
        PA->>S3: store raw exports
        PA->>RCT: mark PASSED
    end
    PA->>Parse: read_nsdl_export, partition_by_client(IEC map)
    Parse->>Parse: 6 QC checks per client per pipeline
    Parse->>SRT: put_staged_record (new request_ids only)
    Parse->>RGT: put_run_gate (per-client QC summary)
```

*Fig. 3 — One Phase A invocation, from EventBridge's static payload to two DynamoDB writes.*

### Trigger & the date window

EventBridge's `Schedule` target can only carry a static `Input`, so it always sends
`{"force_refresh": false}` — no `run_date`. `lambda_handler.py` defaults it to today's UTC date,
then computes the actual NSDL query window:

```
to_date   = run_date - 1 day      # NSDL's own data lags by one day
from_date = to_date - 7 days       # rolling 7-day window
# entered into NSDL's date fields as DD/MM/YYYY
```

> **The one-day lag is load-bearing.** NSDL's export for "today" is never fully populated yet.
> Querying up through `run_date` itself silently returns incomplete data — this was found and
> fixed against the real portal, not assumed.

### Login & export

`nsdl_download.py` drives a real headless Chromium session: fills username/password, reads the
CAPTCHA via `ocr_engine.py`, logs in through the shared login (`shared/nsdl_login.py`), then exports
both modules (`NSDL_MODULE_BE=17`, `NSDL_MODULE_DTA=26`) to tab-delimited `.xls` files.

**CAPTCHA.** `ocr_engine.py` crops the border, builds a darkness map (so the dark letters survive while
grey noise curves do not), removes speckles, and renders several clean variants (glyphs re-spaced with
even gaps, at different thicknesses). EasyOCR's *recogniser only* reads each variant's text box (the slow,
error-prone text detector is skipped), the readings are voted character by character, and the case of
shape-identical letters (c/C, s/S, w/W ...) is decided from glyph height. A reading is only submitted
when it has the expected length (5) and enough variants agree -- otherwise NSDL is asked for a fresh
CAPTCHA, so a doubtful guess never costs a login attempt.

**Email-OTP step.** NSDL's login requires an emailed OTP after credentials + CAPTCHA. The login is
implemented once in `shared/nsdl_login.py` and used by both Phase A and Phase B. Before clicking Login it
pulls the OTP mailbox until a pull has really completed, then snapshots the existing `Communication`
records on the OTP Frappe site (`shared/otp_mailbox.py`, REST API) and notes the Frappe server time of the
click -- *OTP request #1*. It then waits (up to `NSDL_OTP_WAIT_SECONDS`, never past the OTP's 10-minute
validity) for a mail that is new, was *sent after that request* (the mail's own date and the original
"Sent:" time inside a forwarded mail are both checked), carries the `<OTP> is your One Time Password`
sentence for the right NSDL User ID, and has not been used. A late mail of an earlier, already invalidated
OTP is therefore rejected rather than submitted. A resend (after NSDL's 30 s cooldown) happens only after
NSDL has *rejected* an OTP, never merely because the mail is slow. Limits stay below NSDL's lockout
(5 wrong OTPs = 30 minutes): 2 submits and 1 resend by default. A lockout message, rejected credentials or
no usable OTP mail raises a non-retryable login error, and Phase A's async invoke has
`MaximumRetryAttempts: 0` so a retry can't deepen a lockout. Every portal message is logged and written
with screenshots to `login_audit.jsonl`.

**Missing data.** NSDL produces no file when a module has no requests in the window. Each module is
exported independently; a module with no data (NSDL says so, or no file within
`NSDL_EXPORT_TIMEOUT_SECONDS` with no error shown) is a normal result, the other module is still exported,
and the run's `exports` summary reports each module's status. A real failure still fails the run.

### Parse, partition, six QC checks

`nsdl_parser.py` groups rows by `REQUEST ID`, resolves each row's `CLIENT IEC` against the
secret's reverse IEC→client map (an IEC mapped to two different clients aborts the entire run —
the one error class this pipeline never resolves silently), and runs six checks per pipeline per
client:

| # | Check | What it verifies |
|---|---|---|
| 1 | Header count | Input rows vs. grouped output headers reconcile. |
| 2 | Invoice count | Every invoice line survives grouping. |
| 3 | Item count | Every item line survives grouping. |
| 4 | Shipment count | Non-null shipment rows only (per BE/DTA drop rule). |
| 5 | Sum checks | Quantity, BCD, assessable value totals match pre/post transform. |
| 6 | IEC consistency | Every row in one request_id agrees on CLIENT IEC after normalization. |

A failed check 6 group is staged with status `QC_MISMATCH` and excluded from the batch. Any other
failing check either aborts staging for that client/pipeline (`STEP2_QC_STRICT=true`, the default)
or stages anyway with a logged warning.

---

## 04 — Phase B

`app/phase_b_deliver/` · one Lambda, invoked independently per client.

```mermaid
sequenceDiagram
    participant FR as Client Frappe site
    participant NSR as NSDL Sync Run doc
    participant APIGW as API Gateway
    participant PB as PhaseBFunction
    participant SRT as StagedRecordsTable
    participant CAN as Canofy (Frappe API)
    participant NSDL as NSDL Portal

    FR->>NSR: insert(status=Triggered)
    FR->>APIGW: POST /sez/deliver-pending<br/>X-Frappe-Client, {sync_run_name}
    APIGW->>PB: invoke — Event (async), 202 immediately
    PB->>SRT: query_actionable_by_client (STAGED + PUSHED)
    loop each STAGED record
        PB->>CAN: get_record → found? reuse : create_record
        PB->>SRT: mark_pushed(frappe_docname)
    end
    PB->>NSDL: login (once for the whole batch)
    loop pending PDFs, capped at MAX_PDF_ATTACHES_PER_INVOCATION
        PB->>NSDL: fetch print-PDF for request_id
        PB->>CAN: attach_pdf
        PB->>SRT: mark_attached(ttl)
        Note over PB,NSDL: every 15th download: 120s<br/>keep-alive-scroll pause
    end
    PB->>NSR: update_audit_doc(status, counts)
```

*Fig. 4 — One Phase B invocation for one client. The async 202 means Frappe never waits on this;
the real outcome lands later via the NSDL Sync Run doc update.*

### Why the call is asynchronous

API Gateway's REST integration caps a *synchronous* Lambda proxy at 29 seconds. A real NSDL login
plus throttled PDF downloads routinely runs minutes. The integration sets
`X-Amz-Invocation-Type: Event`, so Frappe's scheduled job gets a bare `202 {"accepted": true}` the
instant AWS accepts the trigger — the actual delivery result arrives later, written back onto the
same `NSDL Sync Run` document.

### Idempotent push to Frappe

Before creating anything, Phase B calls `frappe.get_record(doctype, request_id)`. Only if Frappe
has nothing for that `request_id` does it call `create_record`. This is the only check that
touches Frappe's real state — DynamoDB's own status is what decides whether Phase B even looks at
a record at all.

> **Proven under an actual duplicate invocation.** A production Lambda timeout mid-run triggered
> an automatic retry with the same 64-record batch. The retry pushed **zero** duplicate Frappe
> records — every one was already `PUSHED` — and correctly resumed PDF-attach for only the ones
> still outstanding.

### PDF fetch & attach — one path for BE and DTA

`pdf_fetch.py` drives `Search Request → request_id → Print` on NSDL and renders the result to PDF
via the browser's own print-to-PDF. BE and DTA requests use the *identical* navigation — an
earlier build assumed DTA had no printable document at all; that assumption was wrong and has
since been corrected (see [Hardening](#08--hardening)).

Every `PDF_PAUSE_EVERY` (15) downloads, the batch pauses `PDF_PAUSE_SECONDS` (120s) — not idle, but
scrolling the page gently, because a fully idle tab was observed tripping NSDL's own session
timeout mid-run. A hard `MAX_PDF_ATTACHES_PER_INVOCATION` (30) cap keeps one invocation inside
Lambda's 900s ceiling on a large backlog; anything past the cap stays `PUSHED` for the next
invocation.

---

## 05 — Watchdog

`app/watchdog/` · a plain zip Lambda, no browser needed.

Every hour, scans `StagedRecordsTable` for rows still `STAGED` more than 24 hours after
`staged_at`, groups the count by client, and publishes one SNS message naming every affected
client. One check, two real failure modes: a client's Frappe scheduler has silently stopped
calling Phase B at all, *or* Phase B is repeatedly failing to push a specific record (which — by
design — never changes that record's status, so it looks identical to "nobody's asking" from
here).

---

## 06 — Frappe integration

What lives in the client's own Canofy/Frappe app (`canofy_tcs`), not in this AWS stack.

```mermaid
flowchart LR
    HOOKS["hooks.py<br/>scheduler_events.daily"] --> JOB["trigger_nsdl_sync()"]
    JOB --> CFG[("API Configuration - Canofy<br/>doc: NSDL SEZ DELIVER PENDING")]
    CFG -.reads url / api_key / api_secret<br/>+ X-Frappe-Client header.-> JOB
    JOB --> NSR["NSDL Sync Run<br/>insert(status=Triggered)"]
    JOB --> POST["make_post_request()"]
    POST --> AWS["AWS API Gateway"]
    AWS -. later, async .-> NSR2["Phase B updates the<br/>same NSDL Sync Run doc"]
```

*Fig. 5 — Everything this integration needs — endpoint, credentials, and this site's own client
identity — lives in one existing doctype record; nothing lives in `site_config.json`.*

**NSDL Sync Run (new DocType).** One audit row per trigger: `run_date`, `client_name`, `status`
(Triggered → Delivered / No Pending Records / Failed), `records_delivered`, `pdfs_attached`,
`error_message`. Created by the Frappe job, updated later by Phase B.
*(`canofy_tcs/import_documents/doctype/nsdl_sync_run/`)*

**API Configuration - Canofy (existing DocType, reused).** One record named
`NSDL SEZ DELIVER PENDING` holds the deployed API URL, the API key/secret, *and* this site's
`X-Frappe-Client` value inside its existing "Additional Static Headers" JSON field — no new field,
no file-system config. *(`canofy_tcs/canofy_tcs/doctype/api_configuration___canofy/`)*

Deliberately **not** routed through the existing `Canofy API Call` auto-dispatch controller — that
controller only ever fires for API configs whose name/service contains `"EWAY"` (hardcoded), so it
would silently do nothing for an NSDL config without editing shared, already-in-production code.
`nsdl_sync_scheduled_job.py` instead reuses the two genuinely generic pieces:
`frappe.integrations.utils.make_post_request` and the `API Configuration - Canofy` doctype as a
config store.

---

## 07 — Reference

Every value that shapes this system's behavior, and where it lives.

### CloudFormation parameters — human-confirmed, never guessed

| Parameter | Purpose | Current (TEST) |
|---|---|---|
| `Environment` | TEST or PROD — suffixes every named resource so both stacks can coexist. | `TEST` |
| `S3RetentionDays` | Days before a raw NSDL export expires from S3. No default — pending a real SEZ/customs recordkeeping minimum. | `90` |
| `DailyScheduleCronExpression` | EventBridge schedule for Phase A. No default — needs a confirmed business UTC time. | `cron(0 9 * * ? *)` |

### PhaseAFunction environment

| Variable | Value / source | Notes |
|---|---|---|
| `RAW_FILES_BUCKET` | `!Ref RawFilesBucket` | Physical S3 bucket name |
| `RUN_CONTROL_TABLE` | `!Ref RunControlTable` | Per-day idempotency |
| `STAGED_RECORDS_TABLE` | `!Ref StagedRecordsTable` | Shared with Phase B |
| `RUN_GATE_TABLE` | `!Ref RunGateTable` | Per-client QC summary view |
| `NSDL_BOE_SECRET_NAME` | `!Ref NsdlBoeSecrets` (ARN) | boto3 accepts ARN or name |
| `NSDL_BASE_URL` | `sezonline-ndml.co.in/User/Login.aspx` | fw_verify's proven default |
| `NSDL_MODULE_BE` / `NSDL_MODULE_DTA` | `17` / `26` | NSDL's own module dropdown values |
| `NSDL_ACCOUNT_KEY` | `siddhartha_ftwz` | Key into the secret's NSDL_ACCOUNTS block |
| `MAX_LOGIN_ATTEMPTS` | `8` | Login attempt ceiling (a rejected-credentials or lockout message stops immediately) |
| `NSDL_OTP_ENABLED` | `true` | Email-OTP login step on/off |
| `NSDL_OTP_WAIT_SECONDS` | `540` | Max wait for the OTP mail (capped by the OTP's 10-minute validity); no resend while waiting |
| `NSDL_OTP_RESEND_ON_TIMEOUT` | `false` | Resend only after NSDL rejects an OTP -- a resend invalidates the previous OTP |
| `NSDL_CAPTCHA_EXPECTED_LENGTH` | `5` | A CAPTCHA reading is only submitted at this length with enough OCR agreement |
| `TREAT_EXPORT_TIMEOUT_AS_NO_DATA` | `true` | NSDL sends no file for a module with no data; a silent timeout = no data |
| `NSDL_OTP_MAX_SUBMIT_ATTEMPTS` / `NSDL_OTP_MAX_RESENDS` | `2` / `1` | Kept below NSDL's 5-wrong-OTP lockout |
| `STEP2_QC_STRICT` | `true` | Withhold a client/pipeline's records on any failed check |
| `HOME` | `/tmp` | Real Lambda sandbox user's own $HOME is read-only — see Hardening |

### PhaseBFunction environment

| Variable | Value / source | Notes |
|---|---|---|
| `STAGED_RECORDS_TABLE`, `NSDL_BOE_SECRET_NAME`, `NSDL_BASE_URL`, `NSDL_ACCOUNT_KEY`, `MAX_LOGIN_ATTEMPTS`, `NSDL_OTP_*`, `NSDL_CAPTCHA_*` | | Same as Phase A |
| `STAGED_TTL_DAYS` | `60` | TTL applied only at ATTACHED |
| `PDF_PAUSE_EVERY` / `PDF_PAUSE_SECONDS` | `15` / `120` | Throttle to avoid NSDL session expiry |
| `MAX_PDF_ATTACHES_PER_INVOCATION` | `30` | Caps one invocation's PDF work under the 900s ceiling |
| `HOME` | `/tmp` | Same real-Lambda constraint as Phase A |

### Secrets Manager shape — `nsdl-boe-automation-secrets-{Environment}`

```json
{
  "NSDL_ACCOUNTS": {
    "siddhartha_ftwz": {
      "NSDL_USERNAME": "",
      "NSDL_PASSWORD": "",
      "entity_id": "",           // NSDL's own SEZ-unit ID, shown per-row in every export
      "OTP_FRAPPE_URL": "",      // Frappe site whose Communication doctype receives the NSDL OTP mail
      "OTP_FRAPPE_API_KEY": "",
      "OTP_FRAPPE_API_SECRET": "",
      "OTP_EMAIL_ACCOUNT": ""    // Email Account name on that site (used for pull_emails)
    }
  },
  "CLIENTS": {
    "eap": {
      "client_iec_codes": ["300014953", "0300014953"],
      "FRAPPE_URL": "https://eapx.canofy.tech",
      "api_key": "",
      "api_secret": ""
    }
  }
}
```

Real credentials are never generated or guessed by this codebase — populated post-deploy via
`aws secretsmanager put-secret-value`. An IEC listed under two clients aborts the whole Phase A run
rather than resolving silently.

---

## 08 — Hardening

Every one of these passed cleanly under `sam local invoke` and then failed the first time it ran
as the actually-deployed Lambda — the gap between a normal Docker container and AWS Lambda's
sandboxed execution environment. Fixed and verified against real invocations, in the order found.

| # | Symptom | Root cause | Fix |
|---|---|---|---|
| 1 | numpy bool/int not JSON-serializable, crashing a DynamoDB write | `_dynamo_safe()` only handled `float` | Extended to `np.bool_` / `np.integer` |
| 2 | Lambda return-value marshaling error | DynamoDB's `Decimal` + raw numpy QC values in the response body | `make_json_safe()` + explicit `int(version)` |
| 3 | `BrowserContext.new_page`: target closed | Lambda's real `/dev/shm` is far smaller than a local container's | `--disable-dev-shm-usage` |
| 4 | Playwright rejects a raw `--user-data-dir` arg | `.launch()` doesn't accept that flag directly | Switched to `launch_persistent_context()` |
| 5 | `DropAllCapabilitiesOnCurrentThread()` check failed | Chromium's zygote fork model isn't permitted in Lambda's container | `--no-zygote --disable-gpu` |
| 6 | `[Errno 30] Read-only file system: /home/sbx_user1051` | Lambda runs as a non-root sandbox user; only `/tmp` is writable, but EasyOCR/torch/Chromium all assume a writable `$HOME` | `HOME=/tmp` set globally as a Lambda env var |
| 7 | `Object of type Decimal is not JSON serializable` pushing to Frappe | boto3 always deserializes DynamoDB numbers as `Decimal` on read | `make_json_safe()` extended for `Decimal`, applied at the read boundary |
| 8 | DTA Sale records never got a PDF | Wrong assumption baked into an earlier build — NSDL's print flow is identical for BE and DTA | Both pipelines now wait at `PUSHED` and share one PDF-attach path |
| 9 | Incomplete data for "today" | NSDL's own export lags one day behind | `to_date = run_date - 1` |
| 10 | 900s hard timeout on a large PDF backlog | ~9s/PDF plus periodic 120s pauses adds up fast | `MAX_PDF_ATTACHES_PER_INVOCATION` cap; excess safely deferred (idempotent resume) |

---

## 09 — Setup

What's actually done vs. still outstanding.

1. **Deploy the AWS stack** — ✅ done. `sam build && sam deploy` with `S3RetentionDays` and
   `DailyScheduleCronExpression` supplied.
2. **Populate real secrets** — ✅ done. NSDL TEST credentials + the eap client's Frappe URL/API
   key/secret, via `put-secret-value`.
3. **Verify Phase A + Phase B end-to-end** — ✅ done. Real NSDL login, real export, real Frappe
   push, real PDF attach — all verified against the actual deployed Lambdas, not just local
   invokes.
4. **Add NSDL Sync Run + register the scheduled job** — ✅ done, local dev bench. Migrated onto
   the local `frappe.local` dev site. Not yet deployed to whatever actually hosts
   `eapx.canofy.tech` in production.
5. **Deploy those same file changes to production Frappe** — ⬜ outstanding. Real production
   hosting path for `canofy_tcs` is not yet identified.
6. **Create the API Configuration - Canofy record** — ⬜ outstanding. Named
   `NSDL SEZ DELIVER PENDING`, with the real deployed API URL, the real API Gateway key value, and
   `{"X-Frappe-Client": "eap"}` in its headers field.
7. **Subscribe to AlertsTopic** — ⬜ outstanding. Nobody receives Watchdog/error alarms until a
   real email or Slack endpoint is subscribed.

---

## 10 — Open items

**No self-healing against a manually-deleted Frappe doc.** DynamoDB's `ATTACHED` status is
permanent ground truth — if a Frappe record is deleted by hand afterward, nothing re-checks or
re-creates it automatically.

**MAX_PDF_ATTACHES_PER_INVOCATION=30 is a first estimate.** Conservative, based on one real
64-record/31-PDF run. Worth revisiting once real multi-day, multi-client volume data exists.

**Cold-start EasyOCR model download.** A fresh execution environment's `/tmp` starts empty, so
every cold start re-downloads the OCR detection model (~minutes) before it can even attempt a
CAPTCHA.

**Unmapped IEC volume.** Real NSDL data in this window included large row counts under IECs not
yet configured for any onboarded client — correctly skipped, but worth checking whether any belong
to a client to onboard.

---

*Generated from the deployed `nsdl-sez-boe-automation-test` stack and its source under
`D:\Client Projects\EAP\NSDL_SEZ_AUTOMATION\nsdl-sez-boe-automation`. Every figure and code block
above was verified against a real running system, not derived from the build spec alone.*