Project

General

Profile

Actions
Copy link

Feature #148

open
AS AS

Schedule Phase A at 06:30 IST and Phase B per client from AWS EventBridge

Feature #148: Schedule Phase A at 06:30 IST and Phase B per client from AWS EventBridge

Added by Akshath Shettigara 7 days ago.

Status:
New
Priority:
Normal
Assignee:
Akshath Shettigara
Target version:
Backlogs
Start date:
10/15/2026
Due date:
% Done:

0%

Estimated time:

Description

Schedule Phase A at 06:30 IST and Phase B per client from AWS EventBridge

The daily fetch and the delivery to each client's Frappe site now start from AWS on fixed IST times, with no Frappe-side scheduler needed.

1. Summary

Phase A ran from one EventBridge rule whose time had been changed by hand in AWS and no longer matched the stack parameter or the deploy workflow. Phase B was started only by each Frappe site's own scheduled job. This story sets Phase A to 06:30 IST everywhere (live rule and workflow value), and adds one EventBridge rule per client for Phase B (three runs per client per day), so delivery and backlog retries run without Frappe. Verified live on the TEST stack: all rules enabled, and the first two slots fired on time. The first slots with real work are the next morning's.

2. Requirement

# Requirement Source Status
R1 Know and control the Phase A schedule user (2026-10-13) Done
R2 Phase A runs at 06:30 IST user Done
R3 Phase B runs from AWS after Phase A, without a Frappe scheduler user (2026-10-15) Done (TEST)
R4 Three Phase B runs per client per day, no extra safety run user Done
R5 The live rule, the stack parameter and the deploy workflow must agree finding on 2026-10-13 Done for Phase A

3. Decisions

Decision Options considered Why this one Date
Phase A time 04:00 IST (first choice); 06:30 IST 04:00 IST would make the run date the previous UTC date and shorten the window by one day; the user then chose 06:30 IST 2026-10-13
Phase B trigger Dispatcher Lambda with a lock; one rule per client with a fixed input The rules need only template changes and Phase B already accepts a client name; the dispatcher was planned but not built 2026-10-15
Number of runs 5 per day; 3 per client per day New work arrives once a day after Phase A; later runs only retry or clear a backlog (30 PDFs per run) 2026-10-15
Spacing Same time for both clients; 30 minutes apart One NSDL account is shared, and two simultaneous logins can pick up each other's OTP mail 2026-10-15
Stacks All; TEST only The rules name TEST's clients; PROD clients and times are a separate decision 2026-10-15

4. Logic & Workflow

Run IST UTC cron Input
Phase A 06:30 daily cron(0 1 * * ? *) {"force_refresh": false}
Phase B eap 07:00, 11:00, 15:00 cron(30 1,5,9 * * ? *) {"client_name": "eap"}
Phase B wud 07:30, 11:30, 15:30 cron(0 2,6,10 * * ? *) {"client_name": "wud"}
  1. EventBridge schedules are in UTC; 01:00 UTC is 06:30 IST on the same calendar date, so the run date and the 8-day window behave as before.
  2. A Phase B run with nothing pending returns in about a second without logging in to NSDL.
  3. A run is capped at 15 minutes by Lambda; the later slots work through a backlog and retry a failed login.
  4. There is no lock. The spacing does not cover manual runs or Frappe's own job: that job should be disabled once these rules are relied on.

5. Files & Components Changed

File / resource Type Change Purpose
Live EventBridge rule for Phase A AWS cron(0 3 * * ? *) to cron(30 22 * * ? *) to cron(0 1 * * ? *) Schedule
.github/workflows/deploy-test.yml Workflow Phase A cron parameter cron(0 1 * * ? *) Keep the value on deploy
template.yaml SAM template One rule and one Lambda permission per client, conditional on the TEST stack (59 lines) Phase B triggers
README.md Docs Schedule section (20 lines) Documentation

6. Configuration & Deployment

  • A new client needs its own pair of resources (rule and permission) in the template.
  • Production has no scheduled Phase B runs; its clients and times are undecided.
  • Deployed through the pipeline (stack updated 2026-10-15 06:54 UTC).
  • deploy-prod.yml takes its cron from a GitHub variable and was not changed.
  • AWS-started runs create no NSDL Sync Run documents, because Frappe is not involved.

7. Testing

Test Data / Site Expected Actual Status
sam validate --lint Template Valid Valid Pass
IAM policy simulator for the deploy role New rules and permission Allowed Allowed; generated names are 59 characters, limit 64 Pass
Rule listing after deploy TEST stack Four rules enabled with the expected cron and input Phase A, Phase B eap, Phase B wud, watchdog all ENABLED as expected Pass
Phase B resource policy TEST stack Entry per new rule Present; the API Gateway entry is still there Pass
Slot 15:00 IST (eap) Live, nothing pending One run, no login 09:30:41 UTC, 7.5 s, "No staged records" Pass
Slot 15:30 IST (wud) Live, nothing pending One run, no login 10:00:29 UTC, "No staged records" Pass
Slots with real work (07:00, 07:30 IST) Live Delivery after Phase A Not yet run at the time of writing Not confirmed

8. Known Limitations & Open Items

  • Intermittent missing OTP mail: scheduled Phase A runs at 06:30 IST failed on three mornings (13, 14, 15 October) for different reasons (13 and 15: no OTP mail; 14: mailbox permission error). A cause outside the Lambda is suspected (forwarding or mail import); not confirmed. Moving the schedule later in the morning is an option raised to the user.
  • No lock between Phase A, Phase B slots, manual runs and Frappe's own job; a dispatcher with a shared NSDL login lease was proposed on 2026-10-14 and not built. A run is capped at 30 PDFs and 3 runs per client per day, so a backlog above that takes more than a day.
  • A failed login returns normally, so nothing alerts; a log-based alarm was suggested and not added.
  • Phase B's asynchronous retry setting applies only if the function throws, not to a failed login.
  • Disabling the Frappe-side trigger is the user's responsibility; not confirmed done.
  • Not confirmed: PROD schedules; the 11:00 and 15:00 IST slots with work.

9. Status

Item Value
Branch feature/AI-TBD-nsdl-otp-login
Commits 7856560, 4358b99
PR Created and merged by the user for each; numbers not confirmed
Version 1.1.0-beta
Deployed to TEST stack

10. References

Related stories: nsdl-otp-mailbox-resilience, nsdl-email-otp-login, nsdl-frappe-trigger-sync-run (the Frappe-side trigger), nsdl-github-actions-deployment, nsdl-pdf-attachment-be-dta (the 30-PDF cap).

Pushed to Redmine on 15 Oct 2026, 16:35 from nsdl-eventbridge-scheduling_20261015-1616_20261015-163533.md (README created 15 Oct 2026, 16:35).

Files

README.md (6.74 KB) README.md README: Schedule Phase A at 06:30 IST and Phase B per client from AWS EventBridge Akshath Shettigara, 10/15/2026 04:35 PM

No data to display

Actions
Copy link

Also available in: PDF Atom