Feature #34
Updated by Suresh Kumar about 1 month ago
### # Canofy Control Centre — Frontend Development Journal This document explains **what was built, why, and how**, across all development phases of the Canofy Control Centre frontend (Next.js) and its wiring to the Frappe backend (`canofy_control_centre`). It's meant as a reference for anyone picking up this codebase — including a future version of the person who built it. --- ## 1. How this app is put together Before the phases make sense, it helps to understand the shape of the app: - Something will **Frontend**: Next.js (App Router) + TypeScript + Tailwind + SWR for data fetching. Lives entirely under `src/`. - **Backend**: an existing Frappe app, `canofy_control_centre`, with its own doctypes (AWS Server, Client Site, Change Request, etc.) and a handful of custom whitelisted Python methods (`api/*.py`, `page/*/*.py`). - **The bridge**: every Next.js route under `src/app/api/**/route.ts` is a thin server-side proxy. It either calls one of the backend's custom methods, or — increasingly, as the app grew — calls Frappe's own generic `frappe.client.get_list` / `set_value` / `insert` methods directly. This means most new features **don't require touching backend Python code at all**; they just compose existing, already-permissioned generic endpoints. - **Mock fallback**: every API route checks `process.env.FRAPPE_URL` first. If it's not set, the route returns realistic mock data from `src/lib/mock-data.ts` instead of calling Frappe. This is what lets the entire UI run and be clicked through with zero backend connected — every page in every phase was built and verified this way before ever touching a live Frappe instance. - **Client-side data layer**: `src/lib/api.ts` wraps every API route in an SWR hook (`useXxxData()`) or a plain async function (`doXxxAction()`). Pages never call `fetch()` directly — they import from here. - **Types**: `src/lib/types.ts` mirrors the *exact* shape returned by each backend method/route, verified field-by-field against the real doctype JSON files, not guessed. Keeping this pattern consistent across all four phases is what makes the app predictable to extend — every new page follows the same recipe: `type → mock data → API route → SWR hook → page component`. --- ## 2. Phase 1 — Backend wiring + UI foundation **Goal:** get the dashboard and core entity pages (Servers, Clients, Credentials, Version Matrix, Upgrade Candidates) rendering real data from the Frappe backend, and establish the UI system everything else would build on. **What was built:** - `src/lib/frappe-client.ts` — the server-side-only Axios client that holds the Frappe API key/secret and a `FRAPPE_METHODS` map of every backend endpoint this app calls. Never imported into a `"use client"` component. - `src/lib/types.ts` and `src/lib/mock-data.ts` — established for the dashboard first (`DashboardData` and friends), matching `control_centre_home.py` exactly. - The **API proxy layer**: `src/app/api/dashboard`, `/clients`, `/servers`, `/servers/[id]`, `/credentials` (+ `reveal`/`rotate`), `/versions/*` — each following the mock-fallback pattern described above. - The **dashboard page** (`src/app/dashboard/page.tsx`) and its component breakdown in `src/components/dashboard/*` (SummaryCards, ServerHealthGrid, AlertsFeed, VersionDistribution, BackupStatus, RecentDeployments, ExpiryWatchlist). - Core entity pages: `clients/page.tsx`, `servers/page.tsx` + `servers/[id]/page.tsx` (with a live 24h trend chart via Recharts), `credentials/page.tsx` (masked reveal/rotate), `version-matrix/page.tsx`, `upgrade-candidates/page.tsx`. - The **shared UI kit** (`src/components/ui/*`): `Card`, `Badge`, `ProgressBar`, `EmptyState`, `DropdownChip`, `Sparkline` — every later page reuses these instead of inventing new patterns. - App shell: `layout.tsx`, `Sidebar.tsx`, `TopBar.tsx`. **Key decision:** API routes always resolve field names by reading the backend's actual doctype JSON / Python source first, never by guessing from the frontend side. This discipline is what kept the "hook it up to real data" step low-risk in every later phase. --- ## 3. Phase 2 — Dark theme **Goal:** apply a consistent dark visual theme across every page built in Phase 1. **What was built:** - All color, spacing, and font tokens were centralized as CSS custom properties in `src/app/globals.css` (`--bg`, `--surface`, `--text-primary`, `--ok` / `--warn` / `--crit` / `--info`, brand accent + indigo, etc.), then mapped into `tailwind.config.ts` so every component uses semantic utility classes (`bg-surface`, `text-text-dim`, `border-border-soft`) instead of raw hex values. - Every page/component from Phase 1 was restyled against these tokens. **Why it mattered later:** because no component ever hardcodes a color, the Phase 4 **theme toggle** (light/dark switch) only needed one new CSS block — zero component changes. This is a direct payoff of doing Phase 2 properly. --- ## 4. Phase 3 — New v2 modules **Goal:** build pages for three new backend modules that didn't exist in the original app: CI/CD Pipeline tracking, Client Onboarding tracking, and GitHub version drift detection. **What was built:** ### CI/CD Pipelines (`/pipelines`, `/pipelines/[id]`) - Lists `Deployment Pipeline` records with live status polling (15s on the list, 4s on an in-progress pipeline's detail page — stages move fast). - **"Start Pipeline"** modal only offers **Approved** Change Requests (`/api/pipelines/eligible-change-requests`) — the approval gate is enforced by the backend (`api/pipeline.py:start_pipeline`), the UI just reflects it honestly rather than pretending to bypass it. - Detail page shows stage-by-stage progress (backup → snapshot → git-pull → migrate → restart) with rollback action. ### Client Onboarding (`/onboarding`, `/onboarding/[id]`) - Same shape as Pipelines: list + polling detail page, 14-stage automated provisioning engine (EC2 → S3 → IAM → bench → site → apps), with per-stage retry on failure. ### GitHub Sync (`/github`) - Repository list with "Sync Now" action (`api/github.py:sync_repositories`), plus a per-client Git status view (`get_client_git_status`) showing whether a site's deployed commit matches its recorded ref — the "version drift" detector. **Pattern reused from Phase 1:** every one of these follows the same mock-fallback → API route → SWR hook → page recipe. Nothing new was invented architecturally; the app just grew sideways. --- ## 5. Phase 4 — Backlog: Toasts, Client Detail, Change Requests, Search This phase had an explicit constraint: **do not touch Phase 1–3 pages or any backend Python code.** Everything below was built additively. ### Toast Notifications - `src/components/ui/Toast.tsx` — a `ToastProvider` (mounted once in `layout.tsx`) + a `useToast()` hook (`.success()` / `.error()` / `.info()`) usable from any client component. Falls back to no-ops if used outside the provider, so it can never crash a page. - Wired into the two new Phase 4 pages (Change Requests, and available for any future page) without touching Phase 1–3 action handlers. ### Client Detail Page (`/clients/[id]`) - There was no existing backend method for "everything about one client," so rather than write new Python, `src/app/api/clients/[id]/route.ts` composes the view from **eight parallel `frappe.client.get_list` calls** (servers, sites, credentials, incidents, alerts, change requests, version map) plus one batched call for installed apps (a child table of Client Site, queried directly by `parenttype`/`parent` — the same trick used for Site Detail later). - Trade-off, documented honestly: this is more round-trips than the single-Python-method pattern used elsewhere (e.g. `get_server_detail`). If it ever needs to match that architecture, the clean fix is a `get_client_detail()` backend method — a small, additive change, not a rewrite. - Client cards on `/clients` now link here (the one necessary touch to Phase 1's `clients/page.tsx` — wrapping the card in an `<a>`). ### Change Request Queue (`/change-requests`) - Full lifecycle UI for `Change Request`: status tabs (Draft → Pending Approval → Approved → In Progress → Completed / Rejected / Rolled Back), a "New Change Request" creation modal (`frappe.client.insert`), and Submit/Approve/Reject actions (`frappe.client.set_value` on `status`) — mirroring exactly how `alerts/ack` and `alerts/resolve` already did status updates in Phase 1. - Approved requests link straight into the existing Pipelines "Start Pipeline" flow from Phase 3. - **Known constraint (by design, not a bug):** per the backend's own permission matrix, only System Manager / Senior Engineer / Support Lead can Approve/Reject (Write permission on Change Request) — Support Engineer can only Create. The UI doesn't try to route around this. ### Global Search (TopBar) - The search box in `TopBar.tsx` was purely decorative before this phase. It now debounces input, calls `/api/search`, and shows a results dropdown spanning Clients, Servers, Change Requests, Pipelines, and Onboarding Requests. `⌘K` / `Ctrl+K` focuses it from anywhere; `Esc` closes it. - `/api/search` uses simple `like` filters against each doctype's human-readable field (`company_name`, `server_name`, `title`, `pipeline_name`, `client`) — same generic-`get_list` philosophy as everything else. --- ## 6. Phase 4 extension — Theme toggle, Alerts, Backups, Sites A follow-up round, same "don't touch Phase 1–3" constraint, closing out sidebar links that had existed as dead-ends since Phase 1 (`/alerts`, `/backups`) and adding one brand-new module (`/sites`). ### Light/Dark theme toggle - Because Phase 2 centralized every color into CSS variables, this needed only one addition: an `html[data-theme="light"]` override block in `globals.css`, plus `src/components/theme/ThemeProvider.tsx` (context + `localStorage` persistence) and a tiny inline script in `layout.tsx`'s `<head>` that applies the saved theme *before* paint, avoiding a dark-flash-then-light flicker on reload. - The Sun/Moon button already existed in `TopBar.tsx` (decorative since Phase 1) — it now calls `toggleTheme()`. ### Alerts page (`/alerts`) - The `/api/alerts` route (list) and `/api/alerts/ack` + `/api/alerts/resolve` (actions) already existed from Phase 1 — used by the dashboard's alerts feed — but no full-page UI consumed them. This page adds that: separate Health Alerts and Incidents tables, with Acknowledge / Resolve actions and toast feedback. ### Backups page (`/backups`) - New `/api/backups` route (full `Backup Log` list via `get_list`) — the dashboard only ever showed a summary + failed list, never the full log. Page adds status summary cards, a status filter, and a Retry action for failed backups (reusing the existing `/api/backup/retry` endpoint from Phase 1). ### Sites (`/sites`, `/sites/[id]`) — new sidebar tab - New `Client Site` list page: searchable/filterable, shows app count and live up/down status per site. - New Site Detail page answers, in order: **which client owns this site, which server it runs on, how many other clients share that server** (via the server's `Shared Server Client` child table — only shown when `server_type = Shared`), **which apps are installed and at what version** (via the `Installed App` child table), plus creation date, backup/uptime status, recent backups, and credentials. - Same architectural trade-off as Client Detail: composed from `get_list` calls (including two direct child-table queries) rather than a new backend method — flagged as something to verify permissions-wise once connected to a real Frappe instance. --- ## 7. Directory map (as of the end of Phase 4) ``` src/ ├── app/ │ ├── alerts/page.tsx Phase 4 ext — Alerts & Incidents │ ├── backups/page.tsx Phase 4 ext — Backup Log │ ├── change-requests/page.tsx Phase 4 — Change Request Queue │ ├── clients/page.tsx Phase 1 (links to detail added in Phase 4) │ ├── clients/[id]/page.tsx Phase 4 — Client Detail │ ├── credentials/page.tsx Phase 1 │ ├── dashboard/page.tsx Phase 1 │ ├── github/page.tsx Phase 3 │ ├── onboarding/page.tsx, [id]/page.tsx Phase 3 │ ├── pipelines/page.tsx, [id]/page.tsx Phase 3 │ ├── servers/page.tsx, [id]/page.tsx Phase 1 │ ├── sites/page.tsx Phase 4 ext — Sites list │ ├── sites/[id]/page.tsx Phase 4 ext — Site Detail │ ├── upgrade-candidates/page.tsx Phase 1 │ ├── version-matrix/page.tsx Phase 1 │ ├── layout.tsx Phase 1 (+ Toast/Theme providers, Phase 4) │ ├── globals.css Phase 2 (dark) + Phase 4 ext (light) │ └── api/**/route.ts One per feature above, same pattern throughout ├── components/ │ ├── dashboard/* Phase 1 │ ├── layout/Sidebar.tsx, TopBar.tsx Phase 1 (+ search/theme, Phase 4) │ ├── theme/ThemeProvider.tsx Phase 4 ext │ └── ui/* Phase 1 (+ Toast.tsx, Phase 4) └── lib/ ├── frappe-client.ts Phase 1 (+ insertDoc method, Phase 4) ├── types.ts Grown incrementally every phase ├── mock-data.ts Grown incrementally every phase └── api.ts Grown incrementally every phase ``` --- ## 8. Connecting to a real Frappe backend Every route already supports it — set these three in `.env.local` and mock data is replaced by live data everywhere, with no code changes: ``` FRAPPE_URL=https://your-canofy-instance.example.com FRAPPE_API_KEY=... FRAPPE_API_SECRET=... ``` Two things to verify on first real connection (flagged honestly, not guaranteed, since only mock-mode has been tested end-to-end): 1. **Role permissions** — Change Request Approve/Reject needs Write access (System Manager / Senior Engineer / Support Lead). If the API key's user is only a Support Engineer, those two actions will fail with a Frappe permission error — by design, not a bug. 2. **Child-table queries** — Installed App and Shared Server Client are queried directly (`parenttype`/`parent` filters) in the Client Detail and Site Detail routes. This is an unusual access pattern for a child table; if it 403s on your instance, the fix is additive (either a permission rule on the child doctype, or switching that call to fetch the full parent document instead). --- ## 9. What was deliberately *not* changed To respect "don't touch anything else" across every phase after Phase 1: - No backend Python file was ever modified — every new feature uses either an existing custom method or Frappe's own generic `get_list` / `set_value` / `insert`. - Existing Phase 1–3 page logic was never rewritten — only additive touches were made (a provider wrapped around `children`, a nav link added, a card wrapped in a link), each called out explicitly when it happened.