Actions

Copy link

Feature #147

open

AS AS

Email-OTP login for NSDL in Phase A and Phase B, with the OTP read from Frappe

Feature #147: Email-OTP login for NSDL in Phase A and Phase B, with the OTP read from Frappe

Added by Akshath Shettigara 7 days ago.

Status:

New

Priority:

Normal

Assignee:

Akshath Shettigara

Target version:

Backlogs

Start date:

10/15/2026

Due date:

% Done:

Estimated time:

Description

Email-OTP login for NSDL in Phase A and Phase B, with the OTP read from Frappe¶

NSDL now sends a one-time password by email after credentials and CAPTCHA; both Lambdas read it from a Frappe site's Communication records and complete the login.

1. Summary¶

NSDL's login page gained a second factor. The mail goes to a mailbox that a Frappe site pulls into its Communication doctype. This story ports the user's reference login into two shared modules, shared/nsdl_login.py and shared/otp_mailbox.py, used by Phase A (export) and Phase B (PDF fetch). After the login click the Lambda reads new Communication records over the Frappe REST API, extracts the OTP by pattern, and submits it, with resend and lockout handling. Settings come from the existing Secrets Manager secret and Lambda environment variables. Verified live on the TEST stack: login on the first attempt.

2. Requirement¶

#	Requirement	Source	Status
R1	Pick up the OTP from the Frappe Communication doctype by pattern	user (2026-10-11)	Done
R2	Frappe URL, API key and secret and the Email Account name configurable in the same AWS secret	user	Done
R3	One login implementation for Phase A and Phase B	design	Done
R4	A failed run must not push NSDL toward its lockout	NSDL rules	Done (see Decisions)
R5	Missing OTP settings stop the run before touching NSDL	design	Done

NSDL rules the code follows: an OTP is valid for 10 minutes, a resend is allowed after 30 seconds, a new OTP invalidates the old one, and 5 wrong OTPs lock the account for 30 minutes.

3. Decisions¶

Decision	Options considered	Why this one	Date
Where the code lives	Per-Lambda copies; shared modules	One flow, used by both functions	2026-10-11
Phase A retries	AWS default of 2 retries; 0	A retry repeats the login and could add wrong-OTP attempts	2026-10-11
Only mails after the click count	Latest mail overall; mails created after the Login click	Avoids submitting an older, still-listed OTP (freshness anchored to the click)	2026-10-11
Wait before resending	90 s with 3 resends; 240 s with 1 resend and 2 submit attempts	Mails took 2 to 4 minutes; a short wait plus a resend made the first mail stale	2026-10-12

4. Logic & Workflow¶

fill credentials -> solve CAPTCHA -> snapshot Communication -> click Login (anchor time) -> pull mail -> poll Communication for mails newer than the anchor -> extract OTP -> submit -> success / resend / stop

Before the login, the mailbox is checked once: the Frappe site must answer and the Email Account must exist. A bad setting stops the run before NSDL is touched.
After the Login click, the Lambda asks Frappe to pull emails and reads Communication records created after the click.
The OTP is extracted by pattern from the mail text (forms such as "NNNNNN is your OTP" and "OTP ... NNNNNN").
If the OTP is rejected, the flow follows NSDL's rules: resend only after 30 seconds, never beyond the configured resend and submit limits.
A lockout page is recognised and is not retried automatically.

5. Files & Components Changed¶

File / resource	Type	Change	Purpose
`shared/nsdl_login.py`	Code	New (about 450 lines)	Credentials, CAPTCHA, OTP login flow
`shared/otp_mailbox.py`	Code	New (about 450 lines)	Frappe mailbox access and OTP extraction
`shared/secrets_loader.py`	Code	Docstring for the new keys	Documentation
`app/phase_a_fetch/nsdl_download.py`, `lambda_handler.py`	Code	Use the shared login; label login and OTP errors	Phase A
`app/phase_b_deliver/pdf_fetch.py`, `lambda_handler.py`	Code	`login()` is now a thin wrapper	Phase B
`app/phase_a_fetch/requirements.txt`	Config	Add `requests`	Image rebuild needed
`template.yaml`	SAM template	`NSDL_OTP_*` variables on both functions; Phase A `MaximumRetryAttempts: 0`	Settings
`VERSION`, `README.md`, architecture document	Docs	`1.1.0-beta`; secret shape and settings	Documentation

6. Configuration & Deployment¶

Add four keys to the NSDL_ACCOUNTS.<account> block of the secret (values entered by a human, not stored in the repo):

Key	Meaning
`OTP_FRAPPE_URL`	Frappe site that receives the NSDL mail
`OTP_FRAPPE_API_KEY`, `OTP_FRAPPE_API_SECRET`	API user for that site
`OTP_EMAIL_ACCOUNT`	Name of the Email Account that holds the mailbox

The API user needs read access to Communication and Email Account, and permission to pull emails.

Environment variable	Purpose	Value after this story
`NSDL_OTP_ENABLED`	Turn the OTP step on	`true`
`NSDL_OTP_WAIT_SECONDS`	How long to wait for the mail	240 (later raised, see `nsdl-login-otp-captcha-update`)
`NSDL_OTP_MAX_RESENDS`	Resend limit	1
`NSDL_OTP_MAX_SUBMIT_ATTEMPTS`	Submit limit	2

Deployed to the TEST stack through the pipeline (nsdl-github-actions-deployment).

7. Testing¶

Test	Data / Site	Expected	Actual	Status
Static checks	Local	Ruff and compile pass	Passed	Pass
OTP pattern and config builder	Sample sentences, missing keys	OTP found; error names missing keys	As expected	Pass
Live Phase A	TEST stack, real NSDL, 2026-10-11	Login with OTP, export, QC	HTTP 200, login on attempt 1, 549 BE and 878 DTA rows, QC passed	Pass
Live Phase A for `wud`, 2026-10-12 (two runs)	Real NSDL	Login	Failed: mail 3 to 4 minutes late, resend invalidated the first code, 2 wrong OTPs each time	Fail, fixed by the wait change
After the wait change	Real NSDL	Login	Login worked in later runs (see `nsdl-login-otp-captcha-update` and `nsdl-otp-mailbox-resilience`)	Pass

8. Known Limitations & Open Items¶

Mail delivery delay sits outside the Lambda (NSDL to mailbox to forwarding rule to Gmail to Frappe pull); intermittent missing mails are an open item (see nsdl-eventbridge-scheduling).
Settings changed live with update-function-configuration are reset by the next deploy unless the template has the same values.
Two wrong OTPs were used on 2026-10-12; whether the lockout counter reset was not confirmed.
Branch name still has the placeholder AI-TBD.

9. Status¶

Item	Value
Branch	`feature/AI-TBD-nsdl-otp-login`
Commits	`7213244`, `ab985f6`
PR	Pull requests into `test` created by the user; numbers not confirmed
Version	`1.1.0-beta`
Deployed to	TEST stack

10. References¶

Related stories: nsdl-login-otp-captcha-update, nsdl-otp-mailbox-resilience, nsdl-github-actions-deployment, nsdl-architecture-documentation.

Pushed to Redmine on 15 Oct 2026, 16:35 from nsdl-email-otp-login_20261015-1616_20261015-163532.md (README created 15 Oct 2026, 16:35).

Files

README.md (6.93 KB) README.md

README: Email-OTP login for NSDL in Phase A and Phase B, with the OTP read from Frappe

Akshath Shettigara, 10/15/2026 04:35 PM

No data to display