Project

General

Profile

Download (6.93 KB)

Feature #147 ยป README.md

README: Email-OTP login for NSDL in Phase A and Phase B, with the OTP read from Frappe - Akshath Shettigara, 10/15/2026 04:35 PM

 

type: redmine-readme
project: Canofy Process Standardization
subject: Email-OTP login for NSDL in Phase A and Phase B, with the OTP read from Frappe
story_key: nsdl-email-otp-login
generated_at: 2026-10-15T16:16:51+05:30
story_type: feature

Email-OTP login for NSDL in Phase A and Phase B, with the OTP read from Frappe

NSDL now sends a one-time password by email after credentials and CAPTCHA; both Lambdas read it from a Frappe site's Communication records and complete the login.

1. Summary

NSDL's login page gained a second factor. The mail goes to a mailbox that a Frappe site pulls into its Communication doctype. This story ports the user's reference login into two shared modules, shared/nsdl_login.py and shared/otp_mailbox.py, used by Phase A (export) and Phase B (PDF fetch). After the login click the Lambda reads new Communication records over the Frappe REST API, extracts the OTP by pattern, and submits it, with resend and lockout handling. Settings come from the existing Secrets Manager secret and Lambda environment variables. Verified live on the TEST stack: login on the first attempt.

2. Requirement

# Requirement Source Status
R1 Pick up the OTP from the Frappe Communication doctype by pattern user (2026-10-11) Done
R2 Frappe URL, API key and secret and the Email Account name configurable in the same AWS secret user Done
R3 One login implementation for Phase A and Phase B design Done
R4 A failed run must not push NSDL toward its lockout NSDL rules Done (see Decisions)
R5 Missing OTP settings stop the run before touching NSDL design Done

NSDL rules the code follows: an OTP is valid for 10 minutes, a resend is allowed after 30 seconds, a new OTP invalidates the old one, and 5 wrong OTPs lock the account for 30 minutes.

3. Decisions

Decision Options considered Why this one Date
Where the code lives Per-Lambda copies; shared modules One flow, used by both functions 2026-10-11
Phase A retries AWS default of 2 retries; 0 A retry repeats the login and could add wrong-OTP attempts 2026-10-11
Only mails after the click count Latest mail overall; mails created after the Login click Avoids submitting an older, still-listed OTP (freshness anchored to the click) 2026-10-11
Wait before resending 90 s with 3 resends; 240 s with 1 resend and 2 submit attempts Mails took 2 to 4 minutes; a short wait plus a resend made the first mail stale 2026-10-12

4. Logic & Workflow

fill credentials -> solve CAPTCHA -> snapshot Communication -> click Login (anchor time) -> pull mail -> poll Communication for mails newer than the anchor -> extract OTP -> submit -> success / resend / stop

  1. Before the login, the mailbox is checked once: the Frappe site must answer and the Email Account must exist. A bad setting stops the run before NSDL is touched.
  2. After the Login click, the Lambda asks Frappe to pull emails and reads Communication records created after the click.
  3. The OTP is extracted by pattern from the mail text (forms such as "NNNNNN is your OTP" and "OTP ... NNNNNN").
  4. If the OTP is rejected, the flow follows NSDL's rules: resend only after 30 seconds, never beyond the configured resend and submit limits.
  5. A lockout page is recognised and is not retried automatically.

5. Files & Components Changed

File / resource Type Change Purpose
shared/nsdl_login.py Code New (about 450 lines) Credentials, CAPTCHA, OTP login flow
shared/otp_mailbox.py Code New (about 450 lines) Frappe mailbox access and OTP extraction
shared/secrets_loader.py Code Docstring for the new keys Documentation
app/phase_a_fetch/nsdl_download.py, lambda_handler.py Code Use the shared login; label login and OTP errors Phase A
app/phase_b_deliver/pdf_fetch.py, lambda_handler.py Code login() is now a thin wrapper Phase B
app/phase_a_fetch/requirements.txt Config Add requests Image rebuild needed
template.yaml SAM template NSDL_OTP_* variables on both functions; Phase A MaximumRetryAttempts: 0 Settings
VERSION, README.md, architecture document Docs 1.1.0-beta; secret shape and settings Documentation

6. Configuration & Deployment

Add four keys to the NSDL_ACCOUNTS.<account> block of the secret (values entered by a human, not stored in the repo):

Key Meaning
OTP_FRAPPE_URL Frappe site that receives the NSDL mail
OTP_FRAPPE_API_KEY, OTP_FRAPPE_API_SECRET API user for that site
OTP_EMAIL_ACCOUNT Name of the Email Account that holds the mailbox

The API user needs read access to Communication and Email Account, and permission to pull emails.

Environment variable Purpose Value after this story
NSDL_OTP_ENABLED Turn the OTP step on true
NSDL_OTP_WAIT_SECONDS How long to wait for the mail 240 (later raised, see nsdl-login-otp-captcha-update)
NSDL_OTP_MAX_RESENDS Resend limit 1
NSDL_OTP_MAX_SUBMIT_ATTEMPTS Submit limit 2

Deployed to the TEST stack through the pipeline (nsdl-github-actions-deployment).

7. Testing

Test Data / Site Expected Actual Status
Static checks Local Ruff and compile pass Passed Pass
OTP pattern and config builder Sample sentences, missing keys OTP found; error names missing keys As expected Pass
Live Phase A TEST stack, real NSDL, 2026-10-11 Login with OTP, export, QC HTTP 200, login on attempt 1, 549 BE and 878 DTA rows, QC passed Pass
Live Phase A for wud, 2026-10-12 (two runs) Real NSDL Login Failed: mail 3 to 4 minutes late, resend invalidated the first code, 2 wrong OTPs each time Fail, fixed by the wait change
After the wait change Real NSDL Login Login worked in later runs (see nsdl-login-otp-captcha-update and nsdl-otp-mailbox-resilience) Pass

8. Known Limitations & Open Items

  • Mail delivery delay sits outside the Lambda (NSDL to mailbox to forwarding rule to Gmail to Frappe pull); intermittent missing mails are an open item (see nsdl-eventbridge-scheduling).
  • Settings changed live with update-function-configuration are reset by the next deploy unless the template has the same values.
  • Two wrong OTPs were used on 2026-10-12; whether the lockout counter reset was not confirmed.
  • Branch name still has the placeholder AI-TBD.

9. Status

Item Value
Branch feature/AI-TBD-nsdl-otp-login
Commits 7213244, ab985f6
PR Pull requests into test created by the user; numbers not confirmed
Version 1.1.0-beta
Deployed to TEST stack

10. References

Related stories: nsdl-login-otp-captcha-update, nsdl-otp-mailbox-resilience, nsdl-github-actions-deployment, nsdl-architecture-documentation.

    (1-1/1)