Customer onboarding SOP

Example customer: Belimo (BEL)

1. [0. Before you start](#0-before-you-start)
2. [1. Identity and access (IAM)](#1-identity-and-access-iam)
3. [2. Launch the EC2 instance](#2-launch-the-ec2-instance)
4. [3. Security group](#3-security-group)
5. [4. EBS volume: encryption check and tags](#4-ebs-volume-encryption-check-and-tags)
6. [5. Elastic IP](#5-elastic-ip)
7. [6. First SSM connection](#6-first-ssm-connection)
8. [7. S3 bucket](#7-s3-bucket)
9. [8. Give the OCR server access to the bucket](#8-give-the-ocr-server-access-to-the-bucket)
10. [9. Snapshot backups (Data Lifecycle Manager)](#9-snapshot-backups-data-lifecycle-manager)
11. [10. DNS record](#10-dns-record)
12. [11. Server baseline](#11-server-baseline)
13. [12. System packages, frappe user and certbot](#12-system-packages-frappe-user-and-certbot)
14. [13. MariaDB](#13-mariadb)
15. [14. Install bench and initialise Frappe](#14-install-bench-and-initialise-frappe)
16. [15. Read the version lock from a reference server](#15-read-the-version-lock-from-a-reference-server)
17. [16. Download the Canofy apps](#16-download-the-canofy-apps)
18. [17. Golden AMI checkpoint](#17-golden-ami-checkpoint)
19. [18. Create the site and install the apps](#18-create-the-site-and-install-the-apps)
20. [19. Production mode (nginx and supervisor)](#19-production-mode-nginx-and-supervisor)
21. [20. SSL (Let's Encrypt)](#20-ssl-lets-encrypt)
22. [21. Daily database backups to S3](#21-daily-database-backups-to-s3)
23. [22. Canofy configuration (Setup Sheet 1.1–2.2, scripted)](#22-canofy-configuration-setup-sheet-1122-scripted)
24. [23. Users and passwords](#23-users-and-passwords)
25. [24. Microsoft 365 email (ICEGATE mailbox)](#24-microsoft-365-email-icegate-mailbox)
26. [25. OCR service](#25-ocr-service)
27. [26. Go-live checklist and clean-up](#26-go-live-checklist-and-clean-up)
28. [27. Customer handover email](#27-customer-handover-email)
29. [28. Troubleshooting](#28-troubleshooting)
30. [29. Maintenance](#29-maintenance)
31. [Appendices](#appendices)
32. [A. canofy_site_setup.py](#a-canofy_site_setuppy)
33. [B. bel_setup.json](#b-bel_setupjson)
34. [C. canofy_entra_app.sh](#c-canofy_entra_appsh)
35. [D. ocr_add_customer.py](#d-ocr_add_customerpy)
36. [E. Screenshot register](#e-screenshot-register)

# Canofy customer onboarding SOP

Standard procedure for bringing a new customer onto a dedicated Canofy (Frappe) production instance on AWS, end to end, with **SSM-only** server access. Every step is written with the real values from the first customer onboarded this way, **Belimo Automation India Pvt. Ltd. (code BEL)**. Replace them with the new customer's values from the variables table in section 0.3.

## 0. Before you start

### 0.1 How to read this SOP

Each step starts with a tag that tells you **where** to run it:

AWS console AWS CloudShell Server via SSM, as ssm-user Server via SSM, as frappe OCR server, as ubuntu GoDaddy Microsoft Entra Canofy in the browser GitHub

- Commands are written **one per box** with a copy button. Paste one box at a time.
- The SSM browser terminal can merge pasted multi-line text. This SOP therefore never uses heredocs (`<<EOF`): every file is written with a single-line command, or through S3.
- Lines that contain credentials start with a **space**, so they aren't saved in shell history.
- Screenshots come from the Belimo onboarding. Account IDs, tenant/client IDs and secret values are masked.

> **Note:** Never paste passwords, client secrets, tokens or the OCR encryption key into chat, tickets, email or this document. Store them only in the password manager.

### 0.2 Access you need

| Access | Used for | Belimo note |
| --- | --- | --- |
| AWS console in `ap-south-1` with IAM, EC2, S3, DLM and Systems Manager rights | Phases 1–9 | Operator account |
| GoDaddy access to `canofy.tech` DNS | Phase 10 | Needs manager access |
| GitHub read access to `arnk-canofy/canofy_TCS`, `arnk-canofy/canofy_ddit`, `arnk-canofy/priv-frappe-s3-file-storage` and the customer app repo | Phase 16 | Collaborator only, so a temporary classic token was used |
| The customer's Microsoft 365 mailbox login (`no-reply-<code>@canofy.tech`) | Phase 24 | The tenant allows normal users to register apps |
| SSM access to the OCR server (user `ubuntu`) | Phase 25 |  |
| A reference production server (e.g. MIL) | Phase 15 | Used to read versions and app remotes |
| Password manager | Throughout |  |

### 0.3 Customer variables

Fill in this table first. Every later step uses these values.

| Variable | Rule | Belimo value |
| --- | --- | --- |
| Customer legal name |  | Belimo Automation India Private Limited |
| Customer code | 3 letters, upper case | `BEL` |
| Site / sub-domain | `<code>.canofy.tech` (lower case) | `bel.canofy.tech` |
| EC2 Name tag | `#LIVE Production <Client Name> [<CODE>]` | `#LIVE Production Belimo Automation India [BEL]` |
| IAM role | `EC2-Canofy-<CODE>-SSM-Prod` | `EC2-Canofy-BEL-SSM-Prod` |
| IAM S3 policy | `S3-<CODE>` | `S3-BEL` |
| S3 bucket | `<code>.canofy.tech` (lower case) | `bel.canofy.tech` |
| S3 folder (app) | `<CODE>`, must match what OCR expects | `BEL` |
| Instance type | SOP default `t3a.large` | `t3a.medium` (accepted deviation) |
| Root disk | 40 GB recommended | 20 GB (expansion pending) |
| Elastic IP | allocated in phase 5 | `3.6.128.118` |
| Private IP | shown after launch | `172.31.10.27` |
| Mailbox | `no-reply-<code>@canofy.tech` | `no-reply-bel@canofy.tech` |
| Entra app | `Canofy-<CODE>` | `Canofy-BEL` |
| Customer app repo |  | `canofytech/canofy_custom_BEL` (app `custom_belimo`) |
| OCR server | shared | `172.31.35.206:5000` |

### 0.4 Version lock

New customers must run the same versions as the reference production server. Fill this in during phase 15.

| Component | Version / branch | Belimo |
| --- | --- | --- |
| frappe | pinned release tag | `v15.72.5` |
| canofy_tcs | branch | `development` → 2.3.0 (`4e82f42`) |
| s3_file_storage | branch | `main` → 0.0.1 (`163256d`) |
| canofy_ddit | branch, bench only (not installed on the site) | `main` → 1.0.2 (`782a128`) |
| Customer app | branch | `custom_belimo` `main` → 1.2.3 |
| Python / Node |  | 3.11 / 20 |
| MariaDB | Ubuntu 22.04 default | 10.6 |

## 1. Identity and access (IAM)

### 1.1 Create the S3 policy for the customer bucket

AWS console IAM → Policies → **Create policy** → **JSON**. Paste this, with the bucket name replaced:

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "BucketLevel",
      "Effect": "Allow",
      "Action": ["s3:ListBucket", "s3:GetBucketLocation"],
      "Resource": "arn:aws:s3:::bel.canofy.tech"
    },
    {
      "Sid": "ObjectLevel",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::bel.canofy.tech/*"
    }
  ]
}
```

Name it `S3-BEL`, then **Create policy**. The bucket doesn't need to exist yet.

### 1.2 Create the EC2 role

AWS console IAM → Roles → **Create role** → Trusted entity **AWS service**, use case **EC2**. Attach:

- `AmazonSSMManagedInstanceCore` (or your equivalent SSM policy); this is mandatory, because SSM is the only way in;
- `S3-BEL`.

Name it `EC2-Canofy-BEL-SSM-Prod`, add tags `Customer=BEL` and `Environment=Production`, then create it. An EC2 instance can only have **one** role, which is why both policies go on the same one.

> **Belimo:** the role was created with a custom SSM policy equivalent to `AmazonSSMManagedInstanceCore`, plus `S3-BEL`.

## 2. Launch the EC2 instance

### 2.1 Choose the AMI

AWS console EC2 → **Launch instance** → AMI search.

- **Preferred:** **Community AMIs**, search `ubuntu-jammy-22.04-amd64-server`, owner **099720109477** (Canonical), 64-bit x86, newest date.
- **Alternative:** a Canofy golden AMI (phase 17), once one exists.
- **Avoid** the Marketplace listing shown below. It works, but it adds a Marketplace product code to every AMI made from the instance.

You can also get the current Canonical AMI ID from CloudShell:

AWS CloudShell

```bash
aws ssm get-parameter --name /aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id --region ap-south-1 --query Parameter.Value --output text
```

Figure 1 SS-05a: the Marketplace "Subscribe" page for Ubuntu 22.04. Close it and use the Canonical Community AMI instead.

### 2.2 Launch settings

| Setting | Value |
| --- | --- |
| Name | `#LIVE Production Belimo Automation India` (add `[BEL]` to match the volume naming) |
| Tags | `Customer=BEL`, `Environment=Production`, `Owner=Canofy` |
| Instance type | `t3a.large` (SOP). Belimo uses `t3a.medium`. |
| Key pair | **Proceed without a key pair** |
| Network | production VPC (`vpc-cadaeca2`), public subnet, auto-assign public IP on |
| Security group | the customer security group (phase 3) |
| Storage | gp3, **encrypted** (KMS `aws/ebs`), 40 GB recommended |
| IAM instance profile | `EC2-Canofy-BEL-SSM-Prod` |
| Termination protection | Enable |
| Stop protection | Enable |
| Metadata | **V2 only (token required)** recommended. Belimo kept "optional" for now. |

> **Warning:** IMDSv2 protects the instance-role credentials from SSRF attacks through the application. If you leave it optional, record it as a deviation and fix it before the ISO audit (Actions → Instance settings → Modify instance metadata options; no downtime).

### 2.3 Verify the instance

AWS console Open the instance → **Details**. Check: Running, instance type, IAM role, no key pair, IMDSv2 setting.

Figure 2 SS-06: instance summary after launch: Running, t3a.medium, role EC2-Canofy-BEL-SSM-Prod, auto-assigned public IP.

## 3. Security group

AWS console EC2 → instance → **Security** tab → open the security group → **Inbound rules**.

| Type | Port | Source | Keep? |
| --- | --- | --- | --- |
| HTTP | 80 | 0.0.0.0/0 | Yes: Let's Encrypt validation and the redirect to HTTPS |
| HTTPS | 443 | 0.0.0.0/0 | Yes: the application |
| SSH | 22 | any | **Delete**. SSM doesn't need it. |

Outbound: **All traffic, 0.0.0.0/0** (needed for SSM, apt, GitHub and S3). If the instance is on the VPC's `default` group, create a dedicated group instead of editing the shared one.

**Screenshot to capture, SS-04:** Security group inbound rules showing only 80 and 443

## 4. EBS volume: encryption check and tags

AWS console Instance → **Storage** → click the volume ID.

1. Check **Encryption = Encrypted**, KMS `aws/ebs`, type gp3.
2. **Tags → Manage tags**:

| Key | Value |
| --- | --- |
| `Name` | `#LIVE Production Belimo Automation India [BEL]` |
| `Customer` | `BEL` |
| `Environment` | `Production` |
| `Backup` | `Daily` |

The `Name` value matters: the 60-day snapshot policy (phase 9) finds volumes by this tag.

> **Warning:** you can't encrypt an existing volume in place. If it shows "Not encrypted", stop. Snapshot it, copy the snapshot with encryption, and replace the volume before installing anything.

Figure 3 SS-07: volume vol-0cc1d95a7394d274a: 20 GiB gp3, 3000 IOPS, Encrypted with aws/ebs. Note the Customer tag here still holds a wrong value, which was corrected afterwards.

## 5. Elastic IP

AWS console EC2 → **Elastic IPs** → **Allocate**. Tags: `Name=EIP-BEL-PROD`, `Customer=BEL`. Then **Actions → Associate**, choose the instance and its private IP, and click Associate.

Record the IP in the variables table. AWS charges for every public IPv4 address, so release it if the instance is ever terminated.

Figure 4 SS-08: the Elastic IP 3.6.128.118 now replaces the auto-assigned address.

## 6. First SSM connection

AWS console Systems Manager → **Fleet Manager**. The instance must show **Online**. Then EC2 → instance → **Connect → Session Manager → Connect**.

Server via SSM, as ssm-user

```bash
whoami
```

```bash
lsb_release -a
```

```bash
sudo snap services amazon-ssm-agent
```

```bash
df -h / && free -h
```

```bash
curl -s https://checkip.amazonaws.com
```

Expected: `ssm-user`, Ubuntu 22.04.x (jammy), agent `enabled active`, and the Elastic IP as the outbound address.

Figure 5 SS-10: first SSM session checks: ssm-user, Ubuntu 22.04.5, agent active, outbound IP 3.6.128.118.

## 7. S3 bucket

### 7.1 Create the bucket

AWS console S3 → **Create bucket**:

| Setting | Value |
| --- | --- |
| Region | ap-south-1 |
| Name | `bel.canofy.tech` (bucket names must be lower case) |
| Object ownership | ACLs disabled |
| Block all public access | On (all four) |
| Versioning | Enable |
| Default encryption | SSE-S3, Bucket Key enabled |
| Tags | `Customer=BEL`, `Environment=Production` (mind the capital C) |

> **Note:** bucket names with dots (`bel.canofy.tech`) can cause TLS certificate mismatches in some S3 clients, because of virtual-hosted addressing. They work with the AWS CLI, boto3 and the Canofy OCR service (SLC uses `slc.canofy.tech`). A dash-style name such as `canofy-bel` avoids the issue completely.

Figure 6 SS-11: bucket properties: versioning enabled, SSE-S3 with Bucket Key, SSE-C blocked. The tag key "customer" must be "Customer".

### 7.2 Allow HTTPS only

AWS console Bucket → **Permissions → Bucket policy → Edit**:

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyInsecureTransport",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": ["arn:aws:s3:::bel.canofy.tech", "arn:aws:s3:::bel.canofy.tech/*"],
      "Condition": { "Bool": { "aws:SecureTransport": "false" } }
    }
  ]
}
```

**Screenshot to capture, SS-12:** Bucket Permissions tab with Block Public Access on and the policy saved

### 7.3 Lifecycle rules

AWS console Bucket → **Management → Create lifecycle rule**.

**Rule 1, `BEL-cleanup-old-versions`**: applies to all objects. Tick only:

- **Permanently delete noncurrent versions** → 90 days;
- **Delete expired object delete markers**, and **Delete incomplete multipart uploads** → 7 days.

> **Warning:** never tick "Expire current versions" on an all-objects rule. It would delete the customer's live documents.

**Rule 2, `BEL-expire-db-backups`** (create it after phase 21): prefix filter **`_backups/`**. Expire current versions → 30 days; permanently delete noncurrent versions → 1 day.

**Screenshot to capture, SS-14:** Both lifecycle rules listed on the Management tab

### 7.4 Test bucket access from the server

Server via SSM, as ssm-user

```bash
sudo snap install aws-cli --classic
```

```bash
aws sts get-caller-identity
```

```bash
echo "ssm-test" > /tmp/t.txt && aws s3 cp /tmp/t.txt s3://bel.canofy.tech/BEL/_healthcheck.txt --region ap-south-1 && aws s3 ls s3://bel.canofy.tech/BEL/ --region ap-south-1 && aws s3 rm s3://bel.canofy.tech/BEL/_healthcheck.txt --region ap-south-1
```

Expected: `assumed-role/EC2-Canofy-BEL-SSM-Prod/...`, then `upload:`, the listing, and `delete:`, with no Access Denied.

Figure 7 SS-17: role identity and S3 upload, list and delete, with no access keys on the server.

## 8. Give the OCR server access to the bucket

AWS console IAM → Policies → **OCR-production-S3** → **Edit → JSON**.

The policy has **two** statements: an **Allow** that lists the customer buckets, and a **Deny with NotResource** that blocks every bucket *not* listed. Add the new bucket to **both**, directly after an existing line that already ends with a comma:

```text
"arn:aws:s3:::bel.canofy.tech",
"arn:aws:s3:::bel.canofy.tech/*",
```

Click **Next → Save changes** and wait for the green "updated" banner. Then confirm in CloudShell that the default version contains the bucket 4 times:

AWS CloudShell

```bash
ARN=$(aws iam list-policies --scope Local --query "Policies[?PolicyName=='OCR-production-S3'].Arn" --output text); VER=$(aws iam get-policy --policy-arn "$ARN" --query Policy.DefaultVersionId --output text); echo "$VER"; aws iam get-policy-version --policy-arn "$ARN" --version-id "$VER" --query PolicyVersion.Document --output json | grep -c "bel.canofy.tech"
```

> **Warning:** in the Belimo run, the first edit was never saved (screenshot below, with no BEL lines). Always confirm with the count of 4, not with the editor view.

Figure 8 Before: the saved policy has no bel.canofy.tech lines, so OCR is blocked by the Deny statement.

Figure 9 SS-13: after: BEL lines 20–21 (Allow) and 40–41 (Deny/NotResource), "updated" banner, Errors 0.

> **Note:** the editor flags `"arn:aws:s3:::mil.canofy.tech/"` (trailing slash) as redundant. That's a pre-existing MIL typo. Fix it as a separate change, by removing only the trailing slash and never the `/*` line.

Figure 10 The suggestion markers on lines 22–23 come from the MIL trailing-slash typo, not from the new entries.

## 9. Snapshot backups (Data Lifecycle Manager)

AWS console EC2 → **Lifecycle Manager**.

Figure 11 Existing policies: an enabled default 14-day policy for all volumes, and the enabled 60_day_production_instances policy.

Open **`60_day_production_instances`** → **Modify**. Under target resource tags, add **Key `Name`**, Value **exactly** the volume's Name tag (`#LIVE Production Belimo Automation India [BEL]`), then save.

> **Warning:** the key must be `Name`, as for the other customers. In the screenshot below, the BEL entry was first added with key `BEL`, which matches no volume. It was corrected.

Figure 12 SS-15: target tags of the 60-day policy. Every customer uses key "Name" with the volume name. The "BEL" key entry is the mistake to avoid.

The next day, check EC2 → **Snapshots** for a `BEL-Prod` snapshot tagged `dlm:managed`.

## 10. DNS record

GoDaddy My Products → `canofy.tech` → **DNS** → **Add New Record**: Type **A**, Name **`bel`** (not the full domain), Value **`3.6.128.118`**, TTL 600 s. Check that no other `bel` record exists.

Server via SSM, as ssm-user

```bash
getent hosts bel.canofy.tech
```

Expected: `3.6.128.118`. SSL (phase 20) can't be issued until this resolves.

**Screenshot to capture, SS-16:** GoDaddy DNS record list showing the bel A record

## 11. Server baseline

Server via SSM, as ssm-user Start a tmux session first, so a dropped browser tab doesn't kill long-running commands. If you get disconnected, reattach with `tmux attach -t setup`.

```bash
sudo apt install -y tmux && tmux new -s setup
```

```bash
sudo apt update && sudo NEEDRESTART_MODE=a apt -y upgrade
```

```bash
sudo hostnamectl set-hostname bel-prod && echo "127.0.1.1 bel-prod" | sudo tee -a /etc/hosts && echo "preserve_hostname: true" | sudo tee /etc/cloud/cloud.cfg.d/99-hostname.cfg && sudo timedatectl set-timezone Asia/Kolkata
```

```bash
sudo fallocate -l 4G /swapfile && sudo chmod 600 /swapfile && sudo mkswap /swapfile && sudo swapon /swapfile && echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab && echo 'vm.swappiness=10' | sudo tee /etc/sysctl.d/99-swappiness.conf && sudo sysctl -p /etc/sysctl.d/99-swappiness.conf
```

```bash
sudo reboot
```

Reconnect, then check:

```bash
hostname; timedatectl | grep "Time zone"; free -h; swapon --show; df -h /
```

Figure 13 SS-11a: hostname bel-prod, timezone Asia/Kolkata (IST).

Figure 14 SS-11b: 4.0 GB swap active on /swapfile after the reboot.

## 12. System packages, frappe user and certbot

Server via SSM, as ssm-user

```bash
sudo adduser --disabled-password --gecos "" frappe && sudo usermod -aG sudo frappe && sudo chmod 755 /home/frappe
```

```bash
sudo NEEDRESTART_MODE=a apt install -y git curl software-properties-common build-essential pkg-config mariadb-server mariadb-client libmysqlclient-dev redis-server nginx supervisor xvfb libfontconfig fontconfig python3-dev python3-pip python3-setuptools python3-venv cron
```

```bash
sudo add-apt-repository -y ppa:deadsnakes/ppa && sudo apt update && sudo NEEDRESTART_MODE=a apt install -y python3.11 python3.11-dev python3.11-venv
```

```bash
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - && sudo NEEDRESTART_MODE=a apt install -y nodejs && sudo npm install -g yarn
```

```bash
cd /tmp && wget https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6.1-2/wkhtmltox_0.12.6.1-2.jammy_amd64.deb && sudo NEEDRESTART_MODE=a apt install -y ./wkhtmltox_0.12.6.1-2.jammy_amd64.deb && cd ~
```

Install **certbot now**. `bench setup sudoers` (which runs inside `setup production`) fails if certbot is missing.

```bash
sudo snap install --classic certbot && sudo ln -sf /snap/bin/certbot /usr/bin/certbot && certbot --version
```

Temporary passwordless sudo for `frappe`. The `frappe` user has no password (SSM only), and `bench init` calls `sudo supervisorctl`. **Remove this file at go-live** (phase 26).

```bash
echo "frappe ALL=(ALL) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/90-frappe-setup && sudo chmod 440 /etc/sudoers.d/90-frappe-setup && sudo visudo -c
```

Check the versions:

```bash
id frappe; python3.11 --version; node -v; yarn -v; mariadb --version; redis-server --version; nginx -v; wkhtmltopdf --version
```

Figure 15 SS-SW-01: frappe in the sudo group, Python 3.11.16, Node v20.20.2, yarn 1.22.22, MariaDB 10.6.23, Redis 6.0.16, nginx 1.18.0, wkhtmltopdf 0.12.6.1 with patched Qt.

## 13. MariaDB

Server via SSM, as ssm-user Write the config with **one line** (heredocs get mangled in the SSM terminal). It includes the two settings Canofy's wide tables need: `innodb_strict_mode = OFF` and `innodb_default_row_format = dynamic`.

```bash
sudo bash -c 'printf "%s\n" "[mysqld]" "character-set-client-handshake = FALSE" "character-set-server = utf8mb4" "collation-server = utf8mb4_unicode_ci" "bind-address = 127.0.0.1" "innodb_buffer_pool_size = 1G" "innodb_log_file_size = 256M" "max_allowed_packet = 256M" "innodb_strict_mode = OFF" "innodb_default_row_format = dynamic" "" "[mysql]" "default-character-set = utf8mb4" > /etc/mysql/mariadb.conf.d/99-frappe.cnf'
```

```bash
sudo cat -A /etc/mysql/mariadb.conf.d/99-frappe.cnf
```

Every line must end in `$`, with no `^M` and no merged lines. Then restart:

```bash
sudo systemctl restart mariadb && sudo systemctl status mariadb --no-pager | head -5
```

Figure 16 SS-DB-01: the config written with printf (one setting per line) and MariaDB active (running). The first lines show the earlier merged "256M\[mysql\]" that broke the service.

Secure the root account. Save the password in the password manager as `<CODE>-Prod MariaDB root`, and avoid `' " \ $` and spaces in it.

```bash
sudo mysql_secure_installation
```

Answers: Enter (no current password) → **n** (unix_socket) → **Y** (set a root password) → **Y** → **Y** → **Y** → **Y**.

```bash
sudo systemctl disable --now redis-server
```

```bash
sudo mysql -e "SHOW VARIABLES WHERE Variable_name IN ('character_set_server','collation_server','innodb_buffer_pool_size','bind_address','innodb_strict_mode');"
```

```bash
mysql -u root -p -e "SELECT VERSION();"
```

Figure 17 SS-DB-02: secure installation done; utf8mb4, 127.0.0.1, 1 GB buffer pool; root password login works; system Redis inactive.

## 14. Install bench and initialise Frappe

Server via SSM, as ssm-user

```bash
sudo -H pip3 install frappe-bench && bench --version
```

Server via SSM, as frappe Switch user inside tmux:

```bash
sudo -iu frappe
```

```bash
bench init --frappe-branch v15.72.5 --python python3.11 frappe-bench
```

Wait for `SUCCESS: Bench frappe-bench initialized`. This takes 10–20 minutes. Then add the missing `setuptools` package, without which `bench new-site` fails with `No module named 'pkg_resources'`:

```bash
cd ~/frappe-bench && uv pip install --python ./env/bin/python "setuptools<81" && ./env/bin/python -c "import pkg_resources; print('pkg_resources OK')"
```

```bash
bench version && ./env/bin/python --version && df -h / && free -h
```

Figure 18 SS-19: bench initialised, frappe 15.72.5, Python 3.11.16.

## 15. Read the version lock from a reference server

Server via SSM, as frappe On an **existing** production server (Belimo used MIL), as `frappe`, run:

```bash
cd ~/frappe-bench && bench version && ./env/bin/python --version && node -v && cat sites/apps.txt
```

Figure 19 Reference versions on MIL: frappe 15.72.5, canofy_tcs 2.2.1, canofy_ddit 1.0.2, s3_file_storage 0.0.1, custom_mil_power (MIL only), Python 3.10.12, Node 18.20.8.

For each app, record the remote, branch and commit. Switch user first, and then run one app per line. Running it through `sudo -iu frappe bash -c '…'` loses the loop variable.

```bash
cd apps/canofy_tcs && git remote -v && git branch --show-current && git log -1 --oneline && cd ~/frappe-bench
```

Repeat for `canofy_ddit` and `s3_file_storage`.

Figure 20 App remotes on MIL: canofy_TCS on development (e5c1cf96), canofy_ddit main (v1.0.1 tag, 782a128), priv-frappe-s3-file-storage main (163256d).

## 16. Download the Canofy apps

The repos are private. **Preferred:** read-only deploy keys (one per repo), added by a repo **admin**. **If you only have collaborator access:** use a short-lived classic token that is never written to disk (below).

Figure 21 Without credentials, git reports "could not read Username", which means the repo is private.

### 16.1 Temporary classic token (collaborator access)

GitHub Settings → Developer settings → **Tokens (classic)** → Generate: note `<CODE>-setup-temp`, expiry 7 days, scope **`repo`**. If the organisation uses SSO, click **Configure SSO → Authorize**.

Server via SSM, as frappe In `~/frappe-bench`:

```bash
read -s GH_TOKEN; echo "length: ${#GH_TOKEN}"
```

```bash
export GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=http.https://github.com/.extraheader GIT_CONFIG_VALUE_0="AUTHORIZATION: basic $(printf 'x-access-token:%s' "$GH_TOKEN" | base64 -w0)"
```

```bash
git ls-remote https://github.com/arnk-canofy/canofy_TCS.git HEAD
```

```bash
bench get-app canofy_ddit https://github.com/arnk-canofy/canofy_ddit.git --branch main
```

```bash
bench get-app s3_file_storage https://github.com/arnk-canofy/priv-frappe-s3-file-storage.git --branch main
```

```bash
bench get-app canofy_tcs https://github.com/arnk-canofy/canofy_TCS.git --branch development
```

```bash
unset GH_TOKEN GIT_CONFIG_COUNT GIT_CONFIG_KEY_0 GIT_CONFIG_VALUE_0
```

The customer app is downloaded **after** the golden-AMI checkpoint (phase 17), in phase 18.

`supervisor group frappe: ERROR (no such group)` warnings after `get-app` are expected at this stage.

```bash
for a in canofy_ddit s3_file_storage canofy_tcs; do echo "== $a"; git -C apps/$a branch --show-current; git -C apps/$a log -1 --oneline; git -C apps/$a remote get-url upstream; done
```

```bash
grep -rl "ghp_" ~/.gitconfig ~/.git-credentials apps/*/.git/config 2>/dev/null; echo "token check done"
```

Then **delete the token** on GitHub.

Figure 22 SS-21: versions and commits after download. canofy_tcs on development came in at 2.3.0, newer than MIL's 2.2.1, which is expected when following a branch.

### 16.2 Deploy keys (permanent)

Server via SSM, as frappe Create one key per repo, plus an SSH config that tells git which key belongs to which repo:

```bash
mkdir -p ~/.ssh && chmod 700 ~/.ssh && for k in canofy_tcs canofy_ddit s3_file_storage; do ssh-keygen -t ed25519 -N "" -C "bel-prod $k" -f ~/.ssh/deploy_$k; done
```

```bash
printf '%s\n' 'Host gh-canofy-tcs' '  HostName github.com' '  User git' '  IdentityFile ~/.ssh/deploy_canofy_tcs' '  IdentitiesOnly yes' '' 'Host gh-canofy-ddit' '  HostName github.com' '  User git' '  IdentityFile ~/.ssh/deploy_canofy_ddit' '  IdentitiesOnly yes' '' 'Host gh-s3-file-storage' '  HostName github.com' '  User git' '  IdentityFile ~/.ssh/deploy_s3_file_storage' '  IdentitiesOnly yes' > ~/.ssh/config && chmod 600 ~/.ssh/config
```

A repo **admin** adds each `.pub` key under Repo → Settings → **Deploy keys** (read-only). Collaborators don't see this page; the Actions secrets page below is the wrong place.

Figure 23 Settings → Secrets and variables → Actions is NOT where deploy keys go. Deploy keys need admin rights on the repo.

Then point the remotes at the keys:

```bash
git -C apps/canofy_tcs remote set-url upstream git@gh-canofy-tcs:arnk-canofy/canofy_TCS.git
```

Do the same for the other repos. Test each key with `ssh -T gh-canofy-tcs`.

> **Belimo:** keys were generated, but an admin hasn't added them yet. That's an open item.

## 17. Golden AMI checkpoint

This is the point to capture a reusable image: everything common to all customers is installed, and nothing customer-specific exists yet (no site, SSL, DNS or customer app).

### 17.1 Raw image of the source server

Server via SSM, as frappe Confirm that no site exists yet (the folder should only contain `apps.json`, `apps.txt`, `assets` and `common_site_config.json`):

```bash
ls ~/frappe-bench/sites/
```

AWS console Instance → **Actions → Image and templates → Create image**: name `TMP-<CODE>-raw-<YYYYMMDD>`, **Reboot** ticked, tag `Purpose=temp`. Wait until it shows **Available**.

> **Belimo:** `TMP-BEL-raw-20260924` was created. Parts 17.2–17.4 are still pending.

### 17.2 Builder instance and clean-up

AWS console Launch `TMP-AMI-Builder` from the raw image (no key pair, auto-assign public IP, no protections, no Customer/Backup tags).

Server via SSM, as ssm-user On the **builder**, first make sure you're not on the production server:

```bash
cat /var/lib/cloud/data/instance-id
```

It must **not** be the production instance ID (BEL: `i-0be6b206aecd2807e`). Then:

```bash
sudo rm -f /home/frappe/.ssh/deploy_* /home/frappe/.ssh/config /home/frappe/.ssh/known_hosts && sudo rm -f /etc/cloud/cloud.cfg.d/99-hostname.cfg && sudo sed -i '/bel-prod/d' /etc/hosts
```

```bash
sudo rm -f /home/frappe/.bash_history /home/ssm-user/.bash_history /home/ubuntu/.bash_history /root/.bash_history && sudo apt clean && sudo rm -rf /tmp/* /var/tmp/*
```

```bash
sudo journalctl --vacuum-time=1s && sudo find /var/log -type f -name "*.gz" -delete && sudo find /var/log -type f -exec truncate -s 0 {} \;
```

If the raw image was taken **before** these fixes, apply them now: the setuptools fix (phase 14), the MariaDB strict-mode lines (phase 13), certbot (phase 12) and the nginx log format file (phase 20).

```bash
sudo cloud-init clean --logs && sudo truncate -s 0 /etc/machine-id && echo "SANITISED - stop this instance from the console"
```

### 17.3 Create the golden AMI

AWS console **Stop** the builder from the console. Don't reboot it from the terminal: a reboot would undo the clean-up. Then create the image `Canofy-Frappe-v15-Golden-<YYYYMMDD>` with tags `Type=GoldenAMI`, `FrappeVersion`, `CanofyTCS`, `BaseOS=Ubuntu22.04`.

### 17.4 Clean up

Terminate the builder, and deregister the raw AMI and delete its snapshot. The raw image contains the deploy keys, so never share it.

**When using the golden AMI for a new customer:** set the hostname, **change the MariaDB root password**, create new deploy keys and pull the latest code, then continue from phase 18.

## 18. Create the site and install the apps

Server via SSM, as frappe

```bash
cd ~/frappe-bench && bench config dns_multitenant on
```

```bash
bench new-site bel.canofy.tech
```

Enter the MariaDB root password, then a new **Administrator** password (save it in the password manager).

```bash
bench --site bel.canofy.tech install-app s3_file_storage
```

```bash
bench --site bel.canofy.tech install-app canofy_tcs
```

Figure 24 canofy_tcs installed: DocTypes 100%, Address customisations, Dashboard.

### 18.1 Customer app

Download it with the same token method as phase 16 (the Belimo repo is under the `canofytech` organisation):

```bash
GIT_TERMINAL_PROMPT=0 git ls-remote --symref https://github.com/canofytech/canofy_custom_BEL.git HEAD
```

```bash
bench get-app https://github.com/canofytech/canofy_custom_BEL.git --branch main && ls apps/
```

The Python app name can differ from the repo name (Belimo: **`custom_belimo`**). Frappe 15.72.5 resolves `required_apps` on GitHub under the frappe/erpnext organisations, and fails for private dependencies. So comment the line out temporarily, install the app, and then restore the file:

```bash
grep -n "required_apps" apps/custom_belimo/custom_belimo/hooks.py
```

```bash
sed -i 's/^required_apps/# required_apps/' apps/custom_belimo/custom_belimo/hooks.py && bench --site bel.canofy.tech install-app custom_belimo
```

```bash
git -C apps/custom_belimo checkout -- custom_belimo/hooks.py && git -C apps/custom_belimo status --short && echo "hooks restored"
```

### 18.2 Site settings and checks

```bash
bench --site bel.canofy.tech set-config host_name "https://bel.canofy.tech" && bench --site bel.canofy.tech enable-scheduler && bench --site bel.canofy.tech set-maintenance-mode off
```

```bash
bench --site bel.canofy.tech set-config -p server_script_enabled 1
```

```bash
bench --site bel.canofy.tech list-apps
```

Figure 25 SS-22: apps on the BEL site: frappe 15.72.5 (pinned tag), canofy_tcs 2.3.0 development, s3_file_storage 0.0.1, custom_belimo 1.2.3. canofy_ddit stays in the bench only.

`bench doctor` reports `Connection refused 127.0.0.1:11000` until production mode is set up. That's expected at this point.

## 19. Production mode (nginx and supervisor)

Server via SSM, as frappe Size the web workers for the instance. Use 3 on t3a.medium:

```bash
bench config set-common-config -c gunicorn_workers 3
```

```bash
sudo bench setup production frappe --yes
```

Check that **all three parts** worked. In the Belimo run, `setup production` stopped silently before linking supervisor (because certbot was missing).

```bash
ls -la /etc/supervisor/conf.d/ /etc/nginx/conf.d/ && sudo ls /etc/sudoers.d/
```

You should see `frappe-bench.conf` in **both** folders, and a `frappe` file in sudoers.d. If the supervisor link is missing:

```bash
sudo ln -sf /home/frappe/frappe-bench/config/supervisor.conf /etc/supervisor/conf.d/frappe-bench.conf && grep -q chown /etc/supervisor/supervisord.conf || sudo sed -i '/^\[unix_http_server\]/a chown=frappe:frappe' /etc/supervisor/supervisord.conf
```

```bash
sudo systemctl restart supervisor && sleep 8 && sudo supervisorctl status
```

Remove nginx's default welcome page:

```bash
sudo rm -f /etc/nginx/sites-enabled/default && sudo nginx -t && sudo systemctl reload nginx
```

```bash
curl -sI -H "Host: bel.canofy.tech" http://127.0.0.1 | head -3 && bench doctor
```

Figure 26 SS-23: sudoers set up, setup production complete, all 7 processes RUNNING, HTTP 200.

## 20. SSL (Let's Encrypt)

Server via SSM, as ssm-user DNS must resolve first (phase 10). Define the `main` log format **before** enabling SSL. Bench's SSL nginx template uses it, and Ubuntu doesn't define it:

```bash
echo "log_format main '\$remote_addr - \$remote_user [\$time_local] \"\$request\" \$status \$body_bytes_sent \"\$http_referer\" \"\$http_user_agent\" \"\$http_x_forwarded_for\"';" | sudo tee /etc/nginx/conf.d/00-log-format.conf
```

```bash
sudo certbot certonly --nginx -d bel.canofy.tech -m devops@canofy.tech --agree-tos --no-eff-email --deploy-hook "systemctl reload nginx"
```

```bash
sudo -iu frappe bash -c 'cd ~/frappe-bench && bench --site bel.canofy.tech set-config ssl_certificate /etc/letsencrypt/live/bel.canofy.tech/fullchain.pem && bench --site bel.canofy.tech set-config ssl_certificate_key /etc/letsencrypt/live/bel.canofy.tech/privkey.pem && bench setup nginx --yes'
```

```bash
sudo nginx -t && sudo systemctl reload nginx && curl -sI https://bel.canofy.tech | head -1 && curl -sI http://bel.canofy.tech | head -1 && sudo certbot renew --dry-run
```

Expected: `200` over https, `301` over http, and `all simulated renewals succeeded`.

Figure 27 SS-25: log format file, nginx test OK, HTTPS 200, HTTP 301, renewal dry-run succeeded.

Figure 28 SS-24: login page served at https://bel.canofy.tech, before branding.

## 21. Daily database backups to S3

Server via SSM, as frappe The script is written with a single line:

```bash
printf '%s\n' '#!/bin/bash' 'set -e' 'export PATH=/usr/local/bin:/usr/bin:/bin:/snap/bin' 'cd /home/frappe/frappe-bench' 'echo "=== $(date) start"' 'bench --site bel.canofy.tech backup --compress' 'aws s3 sync sites/bel.canofy.tech/private/backups/ s3://bel.canofy.tech/_backups/ --only-show-errors --region ap-south-1' 'echo "=== $(date) done"' > ~/backup_to_s3.sh && chmod 700 ~/backup_to_s3.sh
```

```bash
~/backup_to_s3.sh && aws s3 ls s3://bel.canofy.tech/_backups/ --region ap-south-1 --human-readable
```

```bash
(crontab -l 2>/dev/null; echo '30 1 * * * /home/frappe/backup_to_s3.sh >> /home/frappe/frappe-bench/logs/backup_to_s3.log 2>&1') | crontab - && crontab -l
```

Then create lifecycle Rule 2 for `_backups/` (phase 7.3).

Figure 29 SS-26: scheduler enabled; bench's 6-hourly local backup plus the 01:30 S3 job; test backup (330.2 KiB) in \_backups/.

## 22. Canofy configuration (Setup Sheet 1.1–2.2, scripted)

The script `canofy_site_setup.py` (appendix A) applies Website Settings, System Settings, Navbar, Core Apps, Attachments and S3 File Settings from a per-customer JSON file (appendix B). It runs everything in **one transaction**, rolls back on any error, and is safe to re-run.

### 22.1 Prepare the files

1. Copy `bel_setup.json` to `<code>_setup.json` and change: `customer_code`, the `s3_settings` bucket and folder, `core_apps.apps` (the customer app), and anything else customer-specific.
2. Keep `"enabled": false` on the `connected_app` and `email_account` sections unless you set up email through the script (phase 24 uses the manual path).
3. AWS console Upload to `s3://bel.canofy.tech/_setup/`: the `.py`, the `.json`, and a folder **`branding/`** containing `Logo 150x150.png`, `Canofy logo 150px.png` and `Canofy Favicon1.png` (names must match exactly).

> **Warning:** browsers rename repeated downloads (for example `canofy_site_setup (1).py`). Upload the newest file, and check the version on the server with `grep`.

### 22.2 Download and dry-run

Server via SSM, as frappe

```bash
mkdir -p ~/canofy_setup && aws s3 cp s3://bel.canofy.tech/_setup/ ~/canofy_setup/ --recursive --region ap-south-1 && ls -R ~/canofy_setup
```

If the logos landed next to the script instead of in `branding/`:

```bash
mkdir -p ~/canofy_setup/branding && mv ~/canofy_setup/*.png ~/canofy_setup/branding/
```

```bash
cd ~/frappe-bench/sites && ../env/bin/python ~/canofy_setup/canofy_site_setup.py apply --site bel.canofy.tech --config ~/canofy_setup/bel_setup.json --dry-run
```

The dry run ends with `DRY RUN finished - nothing was saved.` If it reports `unknown field(s)` or `Could not find`, correct the JSON (the valid fields are listed in the error) and run it again.

### 22.3 Apply

```bash
../env/bin/python ~/canofy_setup/canofy_site_setup.py apply --site bel.canofy.tech --config ~/canofy_setup/bel_setup.json
```

Figure 30 SS-AUTO-01: Belimo apply output: logos placed as local public files, settings changed, help menu, Core Apps, 5 Attachments, S3 File Settings, "APPLIED and committed".

Run the same command once more. Every section should report "already up to date", "already present" or "exists, skipped".

Canofy in the browser Hard-refresh with Ctrl+Shift+R and check: the Canofy logo on the login page and navbar; the favicon; Help → About Canofy; no "Login with Email Link". Then attach a test file to any record, and check that it appears in S3:

Server via SSM, as frappe

```bash
aws s3 ls s3://bel.canofy.tech/BEL/ --recursive --region ap-south-1 | tail -3
```

**Screenshot to capture, SS-27:** Login page and desk with Canofy branding

## 23. Users and passwords

Server via SSM, as frappe Set or rotate the Administrator password. The command prompts for it, so it isn't saved in shell history.

```bash
bench --site bel.canofy.tech set-admin-password --logout-all-sessions
```

```bash
bench --site bel.canofy.tech set-password no-reply-bel@canofy.tech --logout-all-sessions
```

Customer users, roles and the Business Partner master are loaded from the customer's data (Setup Sheet 5–7).

## 24. Microsoft 365 email (ICEGATE mailbox)

This follows the existing Canofy pattern (`Canofy-COS`, `-BOS`, `-EAP`, `-LAX`, `-SLC`): one Entra app per customer, Microsoft Graph **delegated** permissions, and a Frappe Connected App plus an Email Account. Incoming mail lands in **Icegate Intimation**.

### 24.1 Mailbox

The mailbox `no-reply-<code>@canofy.tech` is created by the M365 admin, with display name `Canofy Support [<Customer>]` and **IMAP** enabled. Authenticated SMTP must also be enabled if Canofy sends mail from it.

### 24.2 Entra app registration

Microsoft Entra Sign in at entra.microsoft.com as the mailbox user (or an admin). Belimo's tenant allows normal users to register apps.

Figure 31 Signed in as no-reply-bel: Canofy Technologies tenant, display name "Canofy Support \[Belimo\]", no roles needed.

1. **App registrations → New registration**: name `Canofy-BEL`, account type **Single tenant**, redirect URI **Web**. You'll replace it in 24.4 with the value Frappe generates, so any placeholder will do for now.
2. Note the **Application (client) ID** and the **Directory (tenant) ID**.

Figure 32 SS-34: Canofy-BEL overview (IDs masked): single tenant, 1 web redirect URI, Activated.

3. **API permissions → Add a permission → Microsoft Graph → Delegated**: add `IMAP.AccessAsUser.All`, `SMTP.Send` and `offline_access` (`User.Read` is usually there already). "Grant admin consent" isn't needed, because the user consents when signing in.

Figure 33 SS-35: Microsoft Graph (4), all delegated, admin consent not required.

4. **Certificates & secrets → New client secret**: description `Canofy-BEL-secret`, expiry per policy. **Copy the Value immediately** into the password manager, together with the expiry date.

Figure 34 SS-36: Canofy-BEL-secret, expires 25 Sep 2027 (value and secret ID masked).

5. **Overview → Endpoints**: copy **OpenID Connect metadata document** `https://login.microsoftonline.com/<tenant-id>/v2.0/.well-known/openid-configuration`.

> **Note:** if the app was registered by the mailbox account, ask an admin to add themselves under **Owners**. Put the secret expiry in the calendar, about two weeks early. When the secret expires, mail pull stops without warning.

### 24.3 Canofy user for the mailbox

Canofy in the browser Logged in as Administrator, go to **User → Add**: email `no-reply-bel@canofy.tech`, first name `Canofy Support`, last name `[Belimo]`, **Send Welcome Email unticked**. Save, add the role **System Manager**, and set a strong Canofy password (phase 23). This is the **connected user**: the Microsoft token is stored under this Canofy user.

### 24.4 Connected App

Canofy in the browser **Connected App → Add**:

| Field | Value |
| --- | --- |
| Provider Name | `Microsoft 365` |
| OpenID Configuration | the URL from 24.2, then click **Get OpenID Configuration** |
| Client ID | the Canofy-BEL client ID |
| Client Secret | the secret value |
| Scopes | `https://outlook.office.com/IMAP.AccessAsUser.All`, `https://outlook.office.com/SMTP.Send`, `offline_access` |

Save, then copy the generated **Redirect URI** (`https://bel.canofy.tech/api/method/frappe.integrations.doctype.connected_app.connected_app.callback/<name>`).

Microsoft Entra Canofy-BEL → **Authentication → Web → Redirect URIs**: replace the placeholder with that exact URI, then Save.

**Screenshot to capture, SS-37:** Saved Connected App, and the Entra Authentication page with the matching redirect URI

### 24.5 Email Account

Canofy in the browser **Email Account → Add**. Leave incoming and outgoing **off** for now, because saving with them on tests the connection, which fails before authorization.

| Field | Value |
| --- | --- |
| Email Address / Account Name | `no-reply-bel@canofy.tech` / `No Reply Bel` |
| Authentication Method | OAuth; Connected App from 24.4; Connected User `no-reply-bel@canofy.tech` |
| Incoming | IMAP, SSL, `outlook.office365.com`, port 993, sync **UNSEEN**, initial sync 250, attachment limit 50, create contacts on |
| IMAP Folder | Folder `INBOX` → Append To **`Icegate Intimation`** (this is the ICEGATE filtering) |
| Outgoing | `smtp.office365.com`, port 587, TLS; always use the account email address and name as sender; unsubscribe message on; track status on |

### 24.6 Authorize, enable and test

1. Canofy in the browser In a **private window**, log in to Canofy as **`no-reply-bel@canofy.tech`**, open the Email Account, and click **Authorize API Access**. Sign in to Microsoft as the mailbox and **Accept**.
2. Tick Enable Incoming, Default Incoming, Enable Outgoing and Default Outgoing, then **Save**. The save succeeding means the connection test passed.
3. Send a test email to the mailbox. Go to **Scheduled Job Type → `email_account.pull` → Execute**, then check **Icegate Intimation** and **Communication** for the new record.

**Screenshot to capture, SS-38:** Icegate Intimation record created from the test email

> **Belimo:** email was activated. The end-to-end ICEGATE test still has to be recorded.

## 25. OCR service

The shared OCR server (`172.31.35.206:5000`, `POST /generate_ocr`, gunicorn service `ocr_app`) identifies each ERP server by its **private IP** in `property.json`. It checks the username and password the ERP server sends (stored Fernet-encrypted with the `ENCRYPTION_KEY` in `.env`), then reads PDFs from that customer's bucket.

### 25.1 Network

AWS console OCR server's security group → Inbound → add **TCP 5000** from **`172.31.10.27/32`**, description `BEL bel.canofy.tech`.

Server via SSM, as ssm-user Test from the customer server. Any HTTP code means the server is reachable; `000` means it's blocked.

```bash
curl -s -m 5 -o /dev/null -w "%{http_code}\n" -X POST http://172.31.35.206:5000/generate_ocr -H "Content-Type: application/json" -d "{}"
```

### 25.2 Find the live property.json

OCR server, as ubuntu

```bash
systemctl cat ocr_app.service | grep -E "WorkingDirectory|ExecStart"
```

Figure 35 The service runs gunicorn from /home/ubuntu/canofy_OCR, so the live file is /home/ubuntu/canofy_OCR/property.json.

Figure 36 Several copies exist (backups and a stray /home/property.json). Edit only the copy in the service's working directory.

Work as `ubuntu`. As `ssm-user`, you get "Permission denied" on `/home/ubuntu`.

```bash
sudo -iu ubuntu
```

Figure 37 ssm-user can't list /home/ubuntu; switch with sudo -iu ubuntu first.

### 25.3 Add the customer entry

OCR server, as ubuntu As `ubuntu`, back up the file first:

```bash
cp ~/canofy_OCR/property.json ~/canofy_OCR/property.json.bak.$(date +%Y%m%d-%H%M) && ls -la ~/canofy_OCR/property.json*
```

Generate credentials, encrypted with the **existing** key. Never change `ENCRYPTION_KEY`: that would break every customer. This prints the two Fernet tokens and the three `bench` commands for the customer server. Don't share the output.

```bash
cd ~/canofy_OCR && /usr/bin/python3 -c "import secrets,base64;from cryptography.fernet import Fernet;k=[l.split('=',1)[1].strip().strip('\"') for l in open('.env') if l.startswith('ENCRYPTION_KEY=')][0];f=Fernet(k.encode());u='bel_ocr_'+secrets.token_hex(3);p=secrets.token_urlsafe(24);print('ENC_USERNAME =',f.encrypt(u.encode()).decode());print('ENC_PASSWORD =',f.encrypt(p.encode()).decode());print(' bench --site bel.canofy.tech set-config ocr_username',base64.b64encode(u.encode()).decode());print(' bench --site bel.canofy.tech set-config ocr_password',base64.b64encode(p.encode()).decode());print(' bench --site bel.canofy.tech set-config ocr_url http://172.31.35.206:5000/generate_ocr')"
```

Edit the file with `nano ~/canofy_OCR/property.json` (paste with right-click, save with Ctrl+O and Enter, exit with Ctrl+X). Add a comma after the last entry, then:

```json
"172.31.10.27": {
    "BUCKET NAME": "bel.canofy.tech",
    "username": "PASTE_ENC_USERNAME",
    "password": "PASTE_ENC_PASSWORD"
}
```

Validate the file (this prints only IPs, buckets and True/False):

```bash
cd ~/canofy_OCR && /usr/bin/python3 -c "import json;from cryptography.fernet import Fernet;k=[l.split('=',1)[1].strip().strip('\"') for l in open('.env') if l.startswith('ENCRYPTION_KEY=')][0];f=Fernet(k.encode());d=json.load(open('property.json'));[print(ip,'->',v.get('BUCKET NAME'),'| decrypts:',all(bool(f.decrypt(str(v[x]).strip('b').strip(\"'\").encode())) for x in ('username','password'))) for ip,v in d.items()]"
```

If it fails, restore the backup with `cp "$(ls -t ~/canofy_OCR/property.json.bak.* | head -1)" ~/canofy_OCR/property.json`.

Alternative: `ocr_add_customer.py` (appendix D) does all of 25.3 with a dry run and a protection against overwriting an existing entry.

### 25.4 Customer site config and restart

Server via SSM, as frappe In `~/frappe-bench`, paste the three `bench … set-config` lines (each starts with a space). Then check that the keys exist, without showing their values:

```bash
grep -o '"ocr_[a-z]*"' sites/bel.canofy.tech/site_config.json && bench restart
```

To build the base64 values by hand, use `printf '%s' 'value' | base64`. `echo` adds a newline that breaks the match.

OCR server, as ubuntu Restart the OCR service at a quiet moment (it's shared with all customers):

```bash
sudo systemctl restart ocr_app.service && sudo systemctl status ocr_app.service --no-pager | head -5
```

### 25.5 End-to-end test

Canofy in the browser Run OCR on a real document in the customer site, the way users do. Then check the OCR server log:

OCR server, as ubuntu

```bash
sudo journalctl -u ocr_app.service --since "15 minutes ago" --no-pager | tail -80
```

> **Belimo (open):** BEL reaches OCR, but OCR returns `500 INTERNAL SERVER ERROR` (`boe_bridge.request_ocr_data`). Use the checklist in section 28 to find the cause.

## 26. Go-live checklist and clean-up

| # | Item | Belimo |
| --- | --- | --- |
| 1 | Delete `/etc/sudoers.d/90-frappe-setup` (`sudo rm` then `sudo visudo -c`) | Open |
| 2 | Deploy keys added by an admin; remotes switched; temporary GitHub tokens deleted | Open |
| 3 | Disk expanded to 40 GB (EC2 Modify volume, then `sudo growpart /dev/nvme0n1 1 && sudo resize2fs /dev/nvme0n1p1`) | Open |
| 4 | IMDSv2 set to required | Open (deviation) |
| 5 | Session Manager logging enabled | Open |
| 6 | Administrator password rotated and strong; real Administrator email set before 2FA | Open |
| 7 | 2FA enabled (System Settings, method Email) after email works | Open |
| 8 | ICEGATE email end-to-end test recorded | Open |
| 9 | OCR end-to-end test passes | **Failing** (HTTP 500) |
| 10 | First DLM snapshot visible | To check |
| 11 | `_backups/` lifecycle rule created; bucket tag `Customer` fixed | To confirm |
| 12 | `_setup/` deleted from the bucket | Open |
| 13 | Golden AMI parts 17.2–17.4 completed; raw image deleted | Open |
| 14 | Customer handover email sent (section 27) | Draft ready |

## 27. Customer handover email

Send this once UAT can start. Credentials are always sent separately, never in this email.

```text
Subject: Canofy Production Environment Ready – <Customer Legal Name>

Dear <Customer Contact Name>,

We are pleased to inform you that the dedicated Canofy production environment for <Customer Legal Name> has been set up and is ready for your team.

Access details
- Application URL: https://<code>.canofy.tech
- Login credentials will be shared separately with each nominated user. For security reasons, we never send passwords by email.

What has been configured
- A dedicated, secured production instance with HTTPS encryption
- Secure cloud storage for all document attachments
- A dedicated mailbox for ICEGATE intimation emails: no-reply-<code>@canofy.tech
- Automated OCR processing of customs documents
- Automated daily backups

Security policies in place
- Automatic logout after 30 minutes of inactivity
- One active session per user
- The account is temporarily locked after 3 consecutive failed login attempts
- Two-factor authentication (email OTP) will be enabled before go-live

Next steps
1. Please share the names, email IDs and roles of the users who need access.
2. Please share your Business Partner master and item details.
3. Please make sure ICEGATE intimation emails are routed to no-reply-<code>@canofy.tech.
4. UAT window: <start> to <end>.
5. Go-live: <date>, after UAT sign-off.

Warm regards,
<Name>, <Designation>
Canofy Technologies Private Limited
```

## 28. Troubleshooting

Every issue below was hit during the Belimo onboarding. The steps above already include the fixes.

| Symptom | Cause | Fix |
| --- | --- | --- |
| MariaDB won't start after writing the config | The SSM terminal merged heredoc lines (`256M[mysql]`) | Phase 13 `printf` command; check with `cat -A` |
| `bench init` asks for a sudo password, then fails at `sudo supervisorctl status` | `frappe` has no password | `90-frappe-setup` NOPASSWD (phase 12); answer **y** to the rollback, then rerun |
| `could not read Username` / `Password authentication is not supported` | Private repo; a password was entered instead of a token, or the paste failed | Phase 16.1 token method (check the printed length: 40) |
| Fine-grained token can't see the repos | Token owned by your personal account, not the organisation | Use a classic token with `repo` scope, 7 days |
| `new-site`: `No module named 'pkg_resources'` | The uv-built environment lacks setuptools | Phase 14 setuptools fix; `bench drop-site <site> --force --no-backup`, then recreate |
| `Row size too large (> 8126)` on `tabBOE Items` | MariaDB strict mode | `innodb_strict_mode = OFF` (phase 13); then `install-app` |
| `InvalidRemoteException` (GitHub 404 on `erpnext/canofy_tcs`) | `required_apps` lookup in Frappe 15.72.5 | Phase 18.1 comment-out workaround |
| `bench doctor`: `Connection refused 127.0.0.1:11000` | Production mode not set up yet | Phase 19 |
| `502 Bad Gateway`; `supervisorctl status` is empty | `setup production` stopped at the sudoers step (no certbot), so supervisor wasn't linked | Install certbot; phase 19 manual link and `chown` |
| `Certbot is not installed` during `setup sudoers` | certbot missing | Phase 12 certbot step |
| `unknown log format "main"` | Ubuntu nginx doesn't define it | Phase 20 `00-log-format.conf` |
| Setup script: `LinkValidationError` (FTWZ BOE / DTA) | Sheet labels used instead of DocType names | Use `FTWZ BOE Import` and `FTWZ DTA Sale` |
| Setup script: `TypeError ... got 'NoneType'` in s3_file_storage | The S3 hook ran on logo upload before the S3 settings existed | Use the current script, which places logos without hooks |
| `grep -c placed` returns 0 | An old copy of the script was uploaded | Upload the newest download (watch for "(1)") |
| `set-config -p ... true` raises `malformed node` | `-p` expects a Python literal | Use `1` or `0` |
| OCR returns 500 | Under investigation for BEL | See the checklist below |

**OCR 500 checklist.** Compare the service start time with the `property.json` change time, and restart if the file is newer (`systemctl show ocr_app.service -p ActiveEnterTimestamp`). Check that the new entry decrypts (phase 25.3 validation). Check that the base64-decoded `ocr_username` on the customer server matches the OCR username exactly. Check that the PDF exists under the S3 path the OCR log mentions (`aws s3 ls s3://bel.canofy.tech/BEL/ --recursive`), and that the folder case matches (MIL uses lower-case `mil`). Read the traceback in `journalctl -u ocr_app.service`.

### Error screenshots

Figure 38 MariaDB failed: heredoc lines merged by the SSM terminal.

Figure 39 bench init stuck at a sudo password prompt for frappe.

Figure 40 bench init traceback at sudo supervisorctl status, with the rollback prompt.

Figure 41 The bench folder wasn't found on another server: always locate it first with find … -name apps.txt.

Figure 42 Loop variables are lost inside sudo -iu … bash -c '…': switch user first.

Figure 43 bench get-app failed authentication: a password or failed paste instead of a token.

Figure 44 Expected "no such group" warning after get-app, before production mode.

Figure 45 new-site failed: No module named pkg_resources (Dropbox Settings import).

Figure 46 install-app failed: Row size too large on tabBOE Items.

Figure 47 install-app custom_belimo: InvalidRemoteException.

Figure 48 Traceback showing the required_apps lookup on GitHub (org erpnext → 404).

Figure 49 bench doctor: Redis 11000 refused before production mode.

Figure 50 The same Redis error after a setup production that didn't complete.

Figure 51 502 Bad Gateway: nginx linked, but /etc/supervisor/conf.d is empty.

Figure 52 supervisorctl status shows no processes.

Figure 53 After the manual supervisor link and chown: 7 processes RUNNING, HTTP 200.

Figure 54 setup sudoers failed: certbot not installed.

Figure 55 Certificate issued, but nginx -t fails with unknown log format "main".

Figure 56 Only the logos reached S3, in the wrong folder, so the script was missing.

Figure 57 Dry run: System Settings, Navbar and Core Apps OK; Attachments failed on FTWZ names (rolled back).

Figure 58 Apply failed in the s3_file_storage hook before the S3 settings existed (rolled back).

Figure 59 grep -c placed = 0: an old script was still in S3.

Figure 60 The OCR policy editor with BEL lines added. Without Next → Save changes, this isn't saved.

Figure 61 set-config -p server_script_enabled true raises a malformed node error; use 1.

## 29. Maintenance

| Task | When | How |
| --- | --- | --- |
| Entra client secret renewal | Before expiry (BEL: 25 Sep 2027) | New secret → Connected App → Client Secret → Save; no re-authorization needed |
| SSL certificate | Automatic (certbot timer) | Check with `sudo certbot renew --dry-run` |
| Backups | Weekly | `aws s3 ls s3://bel.canofy.tech/_backups/`; check `logs/backup_to_s3.log` |
| Disk | Monthly | `df -h /`; expand before 80% |
| Frappe / app updates | Planned change | Update the version lock, `bench update` on a test copy first; rebuild the golden AMI |

## Appendices

The scripts referenced in this SOP, as used for Belimo. Keep the master copies in the Canofy DevOps repository.

## A. canofy_site_setup.py

Phase 22 (and optional email sections). Run from `~/frappe-bench/sites` with `../env/bin/python`.

```python
#!/usr/bin/env python3
"""
Canofy site setup automation  (Setup Sheet 1.1, 1.2, 1.3, 2, 2.2)

Run as the frappe user, FROM THE sites FOLDER, with the bench's Python:

  cd ~/frappe-bench/sites
  ../env/bin/python ~/canofy_setup/canofy_site_setup.py inspect --site bel.canofy.tech
  ../env/bin/python ~/canofy_setup/canofy_site_setup.py apply   --site bel.canofy.tech --config ~/canofy_setup/bel_setup.json --dry-run
  ../env/bin/python ~/canofy_setup/canofy_site_setup.py apply   --site bel.canofy.tech --config ~/canofy_setup/bel_setup.json

Modes
  inspect   Print the real field names of the doctypes this script touches
            (and any extra ones given with --doctype). Changes nothing.
  apply     Apply the JSON config. Safe to re-run: unchanged values are left
            alone, existing records are skipped, logos are reused.
            With --dry-run everything is checked and printed, then rolled back.

Config conventions
  "@<filename>"   -> upload <branding_dir>/<filename> as a PUBLIC file and use its URL
  keys starting with "_" are comments and are ignored
  a section with "enabled": false is skipped
"""
import argparse
import json
import os
import sys

LAYOUT_TYPES = {"Section Break", "Column Break", "Tab Break", "HTML", "Button", "Fold", "Heading"}
DEFAULT_INSPECT = ["Website Settings", "System Settings", "Navbar Settings", "Navbar Item", "Core Apps", "Attachments", "S3 File Settings", "Connected App", "Email Account"]


def log(msg=""):
    print(msg, flush=True)


def clean(d):
    """Drop comment keys (starting with '_') and the 'enabled' switch."""
    return {k: v for k, v in d.items() if not k.startswith("_") and k != "enabled"}


class Ctx:
    def __init__(self, frappe, cfg, dry_run, new_secret=False):
        self.frappe = frappe
        self.cfg = cfg
        self.dry_run = dry_run
        self.new_secret = new_secret
        self.branding_dir = os.path.expanduser(cfg.get("branding_dir", "~/canofy_setup/branding"))
        self.file_cache = {}

    # ---------- files ----------
    def resolve(self, value):
        if isinstance(value, str) and value.startswith("@"):
            return self.upload_public_file(os.path.join(self.branding_dir, value[1:]))
        return value

    def upload_public_file(self, path):
        frappe = self.frappe
        if path in self.file_cache:
            return self.file_cache[path]
        if not os.path.isfile(path):
            raise SystemExit(f"ERROR: logo file not found: {path}")
        fname = os.path.basename(path)
        existing = frappe.get_all(
            "File",
            filters={"file_name": fname, "is_private": 0, "attached_to_doctype": ["is", "not set"]},
            fields=["name", "file_url"],
            limit=1,
        )
        if existing:
            url = existing[0].file_url
            log(f"  = file '{fname}' already uploaded -> {url}")
        elif self.dry_run:
            url = f"/files/{fname}"
            log(f"  + would upload '{fname}' (public) -> {url}")
        else:
            # Place the file the way Frappe stores public files, and register it
            # WITHOUT running File hooks. The s3_file_storage app hooks every File
            # insert and would try to push branding to S3 (and fails before S3
            # settings exist). Branding must stay local + public for the login page.
            import hashlib
            import shutil
            from frappe.utils import get_files_path

            with open(path, "rb") as f:
                content = f.read()
            dest = get_files_path(fname)  # sites/<site>/public/files/<fname>
            if not os.path.exists(dest):
                shutil.copyfile(path, dest)
            doc = frappe.new_doc("File")
            doc.update({
                "file_name": fname,
                "file_url": f"/files/{fname}",
                "is_private": 0,
                "folder": "Home",
                "file_size": len(content),
                "content_hash": hashlib.md5(content).hexdigest(),
            })
            doc.set_new_name()
            doc.set_user_and_timestamp()
            doc.db_insert()
            url = doc.file_url
            log(f"  + placed '{fname}' as local public file -> {url}")
        self.file_cache[path] = url
        return url


# ---------- helpers ----------
def require_doctype(frappe, dt):
    if not frappe.db.exists("DocType", dt):
        raise SystemExit(f"ERROR: DocType '{dt}' does not exist on this site. Run 'inspect' to find the right name.")
    return frappe.get_meta(dt)


def check_fields(meta, keys, where):
    bad = [k for k in keys if not meta.has_field(k)]
    if bad:
        valid = ", ".join(df.fieldname for df in meta.fields if df.fieldtype not in LAYOUT_TYPES)
        raise SystemExit(f"ERROR: {where}: unknown field(s) {bad}.\n  Valid fields: {valid}")


def same(a, b):
    norm = lambda x: "" if x is None else str(x)
    return norm(a) == norm(b)


# ---------- sections ----------
def set_single(ctx, dt, values):
    frappe = ctx.frappe
    values = clean(values)
    meta = require_doctype(frappe, dt)
    check_fields(meta, values.keys(), dt)
    doc = frappe.get_single(dt)
    changed = []
    for key, raw in values.items():
        new = ctx.resolve(raw)
        old = doc.get(key)
        if not same(old, new):
            doc.set(key, new)
            changed.append((key, old, new))
    if not changed:
        log(f"  {dt}: already up to date")
        return
    for key, old, new in changed:
        log(f"  {dt}.{key}: {old!r} -> {new!r}")
    doc.save(ignore_permissions=True)


def navbar_help(ctx, cfg):
    frappe = ctx.frappe
    item_meta = require_doctype(frappe, "Navbar Item")
    doc = frappe.get_single("Navbar Settings")
    rows = {row.item_label: row for row in doc.get("help_dropdown") or []}
    changed = False

    for label in cfg.get("hide", []):
        row = rows.get(label)
        if not row:
            log(f"  help item '{label}' not found (skipped)")
        elif not row.hidden:
            row.hidden = 1
            changed = True
            log(f"  hide help item '{label}'")
        else:
            log(f"  help item '{label}' already hidden")

    for item in cfg.get("add", []):
        item = clean(item)
        check_fields(item_meta, item.keys(), "Navbar Item")
        row = rows.get(item["item_label"])
        if row:
            diffs = {k: v for k, v in item.items() if not same(row.get(k), v)}
            if diffs:
                row.update(diffs)
                changed = True
                log(f"  update help item '{item['item_label']}': {diffs}")
            else:
                log(f"  help item '{item['item_label']}' already present")
        else:
            doc.append("help_dropdown", item)
            changed = True
            log(f"  add help item '{item['item_label']}'")

    if changed:
        doc.save(ignore_permissions=True)
    else:
        log("  Navbar help menu: already up to date")


def core_apps(ctx, cfg):
    """Fill the Core Apps table from the apps installed on this bench.
    Title, version and git branch are read from the bench itself, so re-running
    this after an upgrade refreshes the shown versions."""
    frappe = ctx.frappe
    from frappe.utils.change_log import get_versions

    dt = cfg.get("doctype", "Core Apps")
    table = cfg.get("table_field", "apps")
    meta = require_doctype(frappe, dt)
    check_fields(meta, [table], dt)
    child_meta = frappe.get_meta(meta.get_field(table).options)

    installed = frappe.get_installed_apps()
    versions = get_versions()
    wanted = []
    for app in cfg["apps"]:
        if app not in installed:
            raise SystemExit(f"ERROR: Core Apps: '{app}' is not installed on this site")
        v = versions.get(app, {})
        row = {
            "app_name": app,
            "title": v.get("title") or app,
            "app_version": v.get("version") or "",
            "git_branch": v.get("branch") or "",
        }
        wanted.append({k: val for k, val in row.items() if child_meta.has_field(k)})

    doc = frappe.get_single(dt)
    current = doc.get(table) or []
    up_to_date = len(current) == len(wanted) and all(
        all(same(r.get(k), w[k]) for k in w) for r, w in zip(current, wanted)
    )
    if up_to_date:
        log(f"  {dt}: already up to date")
        return
    doc.set(table, [])
    for w in wanted:
        doc.append(table, w)
        log(f"  {dt} row: {w}")
    doc.save(ignore_permissions=True)


def connected_app(ctx, cfg):
    """Create/update a Connected App with a FIXED name, so its redirect URI is
    known in advance and matches the one registered in Entra. The client secret
    is asked at a hidden prompt (never stored in the JSON)."""
    import getpass

    frappe = ctx.frappe
    cfg = clean(cfg)
    name = cfg.pop("name")
    scopes = cfg.pop("scopes", [])
    meta = require_doctype(frappe, "Connected App")
    check_fields(meta, list(cfg.keys()) + ["scopes"], "Connected App")
    scope_meta = frappe.get_meta(meta.get_field("scopes").options)
    check_fields(scope_meta, ["scope"], meta.get_field("scopes").options)

    # Fill the endpoint fields from the OpenID configuration document
    endpoints = {}
    if cfg.get("openid_configuration"):
        import requests

        data = requests.get(cfg["openid_configuration"], timeout=20).json()
        mapping = {
            "authorization_uri": "authorization_endpoint",
            "token_uri": "token_endpoint",
            "revocation_uri": "revocation_endpoint",
            "userinfo_uri": "userinfo_endpoint",
            "introspection_uri": "introspection_endpoint",
        }
        for field, key in mapping.items():
            if meta.has_field(field) and data.get(key):
                endpoints[field] = data[key]

    exists = frappe.db.exists("Connected App", name)
    doc = frappe.get_doc("Connected App", name) if exists else frappe.new_doc("Connected App")
    changed = []
    for key, val in {**cfg, **endpoints}.items():
        if not same(doc.get(key), val):
            doc.set(key, val)
            changed.append(key)
    if [r.scope for r in doc.get("scopes") or []] != scopes:
        doc.set("scopes", [{"scope": s} for s in scopes])
        changed.append("scopes")

    if not exists or ctx.new_secret:
        if ctx.dry_run:
            log("  would prompt for the client secret (hidden)")
        else:
            secret = getpass.getpass("  Paste the Entra client secret VALUE (hidden), then Enter: ").strip()
            if not secret:
                raise SystemExit("ERROR: empty client secret")
            doc.client_secret = secret
            changed.append("client_secret")

    if not changed:
        log(f"  Connected App '{name}': already up to date")
    else:
        log(f"  Connected App '{name}': {'update' if exists else 'create'} -> {', '.join(changed)}")
        if exists:
            doc.save(ignore_permissions=True)
        else:
            try:
                doc.insert(ignore_permissions=True, set_name=name)
            except TypeError:
                doc.name = name
                doc.flags.name_set = True
                doc.insert(ignore_permissions=True)
    log(f"  scopes      : {scopes}")
    log(f"  redirect URI: {doc.get('redirect_uri')}")
    if not ctx.dry_run and doc.name != name:
        log(f"  WARNING: Frappe named it '{doc.name}', not '{name}'. Put the redirect URI above into Entra.")


def email_account(ctx, cfg):
    """Create/update an OAuth Email Account (mirrors MIL). Incoming/outgoing are
    switched on only once a Microsoft token exists for connected_app + user,
    i.e. after 'Authorize API Access' has been done once in the browser."""
    frappe = ctx.frappe
    cfg = clean(cfg)
    later = cfg.pop("enable_after_authorize", {})
    meta = require_doctype(frappe, "Email Account")
    check_fields(meta, list(cfg.keys()) + list(later.keys()), "Email Account")
    name = cfg["email_account_name"]
    ca, user = cfg.get("connected_app"), cfg.get("connected_user")

    authorized = False
    try:
        authorized = bool(ca and user and frappe.db.exists("Token Cache", {"connected_app": ca, "user": user}))
    except Exception as e:  # field names differ on some versions
        log(f"  (could not check Token Cache: {e})")

    values = dict(cfg)
    if authorized:
        values.update(later)

    exists = frappe.db.exists("Email Account", name)
    doc = frappe.get_doc("Email Account", name) if exists else frappe.new_doc("Email Account")
    changed = []
    for key, val in values.items():
        if isinstance(val, list):  # child table, e.g. imap_folder
            cur = [{k: r.get(k) for k in (val[0].keys() if val else [])} for r in doc.get(key) or []]
            if cur != val:
                doc.set(key, val)
                changed.append(key)
        elif not same(doc.get(key), val):
            doc.set(key, val)
            changed.append(key)

    if not changed:
        log(f"  Email Account '{name}': already up to date")
    else:
        log(f"  Email Account '{name}': {'update' if exists else 'create'} -> {', '.join(changed)}")
        if exists:
            doc.save(ignore_permissions=True)
        else:
            doc.insert(ignore_permissions=True)

    if authorized:
        log(f"  Microsoft token found for {user}: incoming/outgoing settings applied.")
    else:
        log("  NEXT (manual, once): log in to the site as " + str(user) + ", open Email Account '" + name + "',")
        log("  click 'Authorize API Access' and sign in as " + str(cfg.get("email_id")) + ". Then re-run this script.")


def upsert_records(ctx, cfg):
    frappe = ctx.frappe
    dt, key = cfg["doctype"], cfg["match_field"]
    meta = require_doctype(frappe, dt)
    for rec in cfg["records"]:
        rec = clean(rec)
        check_fields(meta, rec.keys(), dt)
        existing = frappe.db.get_value(dt, {key: rec[key]}, "name")
        if existing:
            log(f"  = {dt} '{rec[key]}' exists ({existing}), skipped")
            continue
        doc = frappe.get_doc({"doctype": dt, **rec})
        doc.insert(ignore_permissions=True)
        log(f"  + {dt} '{rec[key]}' created")


# ---------- modes ----------
def run_inspect(frappe, extra):
    names = DEFAULT_INSPECT + extra
    for pattern in ("%Attach%", "%S3%"):
        names += frappe.get_all("DocType", filters={"name": ["like", pattern], "istable": 0}, pluck="name")
    seen = set()
    for dt in names:
        if dt in seen:
            continue
        seen.add(dt)
        if not frappe.db.exists("DocType", dt):
            log(f"\n## {dt}: NOT FOUND")
            continue
        meta = frappe.get_meta(dt)
        log(f"\n## {dt}   (single={meta.issingle}, module={meta.module})")
        for df in meta.fields:
            if df.fieldtype in LAYOUT_TYPES:
                continue
            opts = ""
            if df.options and df.fieldtype in ("Link", "Table", "Table MultiSelect", "Select", "Dynamic Link"):
                opts = "  -> " + str(df.options).replace("\n", " | ")
            req = " *required" if df.reqd else ""
            log(f"  {df.fieldname:38} {df.fieldtype:18} {df.label or ''}{req}{opts}")


def run_apply(frappe, cfg, dry_run, new_secret=False):
    ctx = Ctx(frappe, cfg, dry_run, new_secret)
    steps = [
        ("1.1 Website Settings", "website_settings", lambda c: set_single(ctx, "Website Settings", c)),
        ("1.2 System Settings", "system_settings", lambda c: set_single(ctx, "System Settings", c)),
        ("1.3 Navbar Settings", "navbar_settings", lambda c: set_single(ctx, "Navbar Settings", c)),
        ("1.3 Navbar help menu", "navbar_help", lambda c: navbar_help(ctx, c)),
        ("1.3 Core Apps", "core_apps", lambda c: core_apps(ctx, c)),
        ("2   Attachment List", "attachment_list", lambda c: upsert_records(ctx, c)),
        ("2.2 S3 File Attachment", "s3_settings", lambda c: set_single(ctx, c["doctype"], c["values"])),
        ("3   Connected App (Microsoft 365)", "connected_app", lambda c: connected_app(ctx, c)),
        ("3   Email Account (ICEGATE mailbox)", "email_account", lambda c: email_account(ctx, c)),
    ]
    for title, key, fn in steps:
        section = cfg.get(key)
        log(f"\n[{title}]")
        if not section:
            log("  (not in config, skipped)")
        elif section.get("enabled") is False:
            log("  (enabled: false, skipped)")
        else:
            fn(section)


def main():
    p = argparse.ArgumentParser(description="Canofy site setup automation")
    p.add_argument("mode", choices=["inspect", "apply"])
    p.add_argument("--site", required=True)
    p.add_argument("--config", help="customer JSON config (apply mode)")
    p.add_argument("--dry-run", action="store_true", help="check and print, then roll back")
    p.add_argument("--doctype", action="append", default=[], help="extra doctype to inspect (repeatable)")
    p.add_argument("--new-secret", action="store_true", help="prompt for a new Connected App client secret")
    a = p.parse_args()

    if not os.path.isfile(os.path.join(os.getcwd(), a.site, "site_config.json")):
        sys.exit("ERROR: run this from ~/frappe-bench/sites  (site folder not found in the current directory)")
    if a.mode == "apply" and not a.config:
        sys.exit("ERROR: apply needs --config <file.json>")

    import frappe

    frappe.init(site=a.site, sites_path=".")
    frappe.connect()
    frappe.set_user("Administrator")
    try:
        if a.mode == "inspect":
            run_inspect(frappe, a.doctype)
        else:
            with open(os.path.expanduser(a.config)) as f:
                cfg = json.load(f)
            log(f"Site: {a.site}   Customer: {cfg.get('customer_code', '?')}   {'DRY RUN' if a.dry_run else 'APPLY'}")
            run_apply(frappe, cfg, a.dry_run, a.new_secret)
            if a.dry_run:
                frappe.db.rollback()
                log("\nDRY RUN finished - nothing was saved.")
            else:
                frappe.db.commit()
                frappe.clear_cache()
                log("\nAPPLIED and committed. Hard-refresh the browser (Ctrl+Shift+R).")
    except SystemExit:
        frappe.db.rollback()
        raise
    except Exception:
        frappe.db.rollback()
        log("\nFAILED - all changes rolled back.")
        raise
    finally:
        frappe.destroy()


if __name__ == "__main__":
    main()

```

## B. bel_setup.json

Customer config for Belimo. Copy it per customer. The email sections are disabled by default.

```json
{
  "_about": "Canofy site setup config for BEL (Belimo Automation India). Keys starting with _ are comments.",
  "customer_code": "BEL",
  "branding_dir": "~/canofy_setup/branding",
  "website_settings": {
    "_sheet": "1.1 Website Settings",
    "app_name": "Canofy",
    "app_logo": "@Logo 150x150.png",
    "banner_image": "@Canofy logo 150px.png",
    "splash_image": "@Canofy logo 150px.png",
    "favicon": "@Canofy Favicon1.png"
  },
  "system_settings": {
    "_sheet": "1.2 System Settings (2FA is enabled later, after email works)",
    "enable_onboarding": 1,
    "login_with_email_link": 0,
    "session_expiry": "00:30",
    "deny_multiple_sessions": 1,
    "allow_login_using_user_name": 1,
    "allow_consecutive_login_attempts": 3,
    "allow_login_after_fail": 120
  },
  "navbar_settings": {
    "_sheet": "1.3 Navbar Settings",
    "app_logo": "@Logo 150x150.png"
  },
  "navbar_help": {
    "_sheet": "1.3 Navbar help dropdown",
    "hide": [
      "About",
      "Frappe Support"
    ],
    "add": [
      {
        "item_label": "About Canofy",
        "item_type": "Action",
        "action": "frappe.ui.toolbar.show_custom_about()"
      }
    ]
  },
  "core_apps": {
    "_sheet": "1.3 Core Apps. Title, version and branch are read from the bench automatically.",
    "doctype": "Core Apps",
    "table_field": "apps",
    "apps": [
      "canofy_tcs",
      "s3_file_storage",
      "custom_belimo"
    ]
  },
  "attachment_list": {
    "_sheet": "2 Attachment List (doctype 'Attachments', child 'Attachments Required For')",
    "doctype": "Attachments",
    "match_field": "attachment_name",
    "records": [
      {
        "attachment_name": "AWB_HBL",
        "attachments_required_for": [
          {
            "doctype_name": "Bill of Entry"
          },
          {
            "doctype_name": "FTWZ BOE Import"
          },
          {
            "doctype_name": "FTWZ DTA Sale"
          }
        ]
      },
      {
        "attachment_name": "Invoice",
        "attachments_required_for": [
          {
            "doctype_name": "Bill of Entry"
          },
          {
            "doctype_name": "FTWZ BOE Import"
          },
          {
            "doctype_name": "FTWZ DTA Sale"
          }
        ]
      },
      {
        "attachment_name": "Bill of Entry",
        "attachments_required_for": [
          {
            "doctype_name": "Bill of Entry"
          },
          {
            "doctype_name": "FTWZ BOE Import"
          },
          {
            "doctype_name": "FTWZ DTA Sale"
          }
        ]
      },
      {
        "attachment_name": "Packing List",
        "attachments_required_for": [
          {
            "doctype_name": "Bill of Entry"
          },
          {
            "doctype_name": "FTWZ BOE Import"
          },
          {
            "doctype_name": "FTWZ DTA Sale"
          }
        ]
      },
      {
        "attachment_name": "Attachment Type",
        "attachments_required_for": [
          {
            "doctype_name": "Bill of Entry"
          }
        ]
      }
    ]
  },
  "s3_settings": {
    "_sheet": "2.2 S3 File Settings. No AWS keys: the instance role provides access.",
    "doctype": "S3 File Settings",
    "values": {
      "bucket_name": "bel.canofy.tech",
      "region_name": "ap-south-1",
      "folder_name": "BEL",
      "signed_url_expiry_time": 300,
      "delete_file_from_cloud": 0,
      "download_file_without_signed_url": 0,
      "ignore_mapped_doctypes": 1
    }
  },
  "connected_app": {
    "enabled": false,
    "_note": "Set to true only if email is configured by script instead of the manual SOP path (phase 24).",
    "_sheet": "3 Email linking - Connected App. Fixed name = matches the Entra redirect URI. Secret is asked at a hidden prompt.",
    "name": "canofy-bel-m365",
    "provider_name": "Microsoft 365",
    "openid_configuration": "https://login.microsoftonline.com/<TENANT_ID>/v2.0/.well-known/openid-configuration",
    "client_id": "<CLIENT_ID of Canofy-CODE app>",
    "scopes": [
      "https://outlook.office.com/IMAP.AccessAsUser.All",
      "https://outlook.office.com/SMTP.Send",
      "offline_access"
    ]
  },
  "email_account": {
    "enabled": false,
    "_note": "Set to true only if email is configured by script instead of the manual SOP path (phase 24).",
    "_sheet": "3 Email Account - mirrors MIL 'No Reply Mil'. INBOX -> Icegate Intimation is the ICEGATE filtering.",
    "email_account_name": "No Reply Bel",
    "email_id": "no-reply-bel@canofy.tech",
    "auth_method": "OAuth",
    "connected_app": "canofy-bel-m365",
    "connected_user": "Administrator",
    "use_imap": 1,
    "use_ssl": 1,
    "use_starttls": 0,
    "email_server": "outlook.office365.com",
    "incoming_port": "993",
    "attachment_limit": 50,
    "email_sync_option": "UNSEEN",
    "initial_sync_count": "250",
    "append_emails_to_sent_folder": 0,
    "create_contact": 1,
    "enable_automatic_linking": 0,
    "use_tls": 1,
    "use_ssl_for_outgoing": 0,
    "smtp_server": "smtp.office365.com",
    "smtp_port": "587",
    "always_use_account_email_id_as_sender": 1,
    "always_use_account_name_as_sender_name": 1,
    "send_unsubscribe_message": 1,
    "track_email_status": 1,
    "no_smtp_authentication": 0,
    "imap_folder": [
      {
        "folder_name": "INBOX",
        "append_to": "Icegate Intimation"
      }
    ],
    "enable_after_authorize": {
      "enable_incoming": 1,
      "default_incoming": 1,
      "enable_outgoing": 1,
      "default_outgoing": 1
    }
  }
}
```

## C. canofy_entra_app.sh

Optional scripted alternative to phase 24.2, for an operator with Azure CLI access to the Canofy tenant: `bash canofy_entra_app.sh BEL bel.canofy.tech`.

```bash
#!/usr/bin/env bash
# ---------------------------------------------------------------------------
# Canofy - Microsoft Entra app registration for a customer's mail integration
# Matches the existing pattern (Canofy-COS/BOS/EAP/LAX/SLC):
#   single tenant, Microsoft Graph delegated: IMAP.AccessAsUser.All,
#   offline_access, SMTP.Send, User.Read; Web redirect = Frappe Connected App
#   callback; client secret "Canofy-<CODE>-secret".
#
# Run in Azure Cloud Shell (Bash) or any machine with Azure CLI, signed in as
# an admin of the Canofy tenant:
#   az login --allow-no-subscriptions      # not needed in Azure Cloud Shell
#   bash canofy_entra_app.sh BEL bel.canofy.tech
#   bash canofy_entra_app.sh BEL bel.canofy.tech --admin-consent   # optional
#
# Safe to re-run: reuses the app if it exists, only adds what is missing.
# A NEW secret is created only with --new-secret (or on first creation).
# ---------------------------------------------------------------------------
set -euo pipefail

CODE="${1:?Usage: $0 <CODE> <site-domain> [--admin-consent] [--new-secret]}"
SITE="${2:?Usage: $0 <CODE> <site-domain> [--admin-consent] [--new-secret]}"
shift 2
ADMIN_CONSENT=0; NEW_SECRET=0
for a in "$@"; do
  case "$a" in
    --admin-consent) ADMIN_CONSENT=1 ;;
    --new-secret)    NEW_SECRET=1 ;;
    *) echo "Unknown option: $a"; exit 1 ;;
  esac
done

CODE_UP=$(echo "$CODE" | tr '[:lower:]' '[:upper:]')
CODE_LO=$(echo "$CODE" | tr '[:upper:]' '[:lower:]')
APP_NAME="Canofy-${CODE_UP}"
SECRET_NAME="Canofy-${CODE_UP}-secret"
CONNECTED_APP="canofy-${CODE_LO}-m365"          # fixed Connected App name in Frappe
REDIRECT_URI="https://${SITE}/api/method/frappe.integrations.doctype.connected_app.connected_app.callback/${CONNECTED_APP}"
SECRET_DAYS="${SECRET_DAYS:-180}"               # override: SECRET_DAYS=365 bash ...
GRAPH="00000003-0000-0000-c000-000000000000"    # Microsoft Graph appId (fixed)
SCOPES=("IMAP.AccessAsUser.All" "offline_access" "SMTP.Send" "User.Read")

TENANT_ID=$(az account show --query tenantId -o tsv)
echo "Tenant      : $(az account show --query tenantDisplayName -o tsv 2>/dev/null || true) ($TENANT_ID)"
echo "App         : $APP_NAME"
echo "Redirect URI: $REDIRECT_URI"
echo

# 1) App registration (reuse if it exists)
APP_ID=$(az ad app list --display-name "$APP_NAME" --query "[0].appId" -o tsv)
CREATED=0
if [ -z "$APP_ID" ]; then
  APP_ID=$(az ad app create --display-name "$APP_NAME" --sign-in-audience AzureADMyOrg \
            --web-redirect-uris "$REDIRECT_URI" --query appId -o tsv)
  CREATED=1
  echo "+ created app registration $APP_NAME"
else
  echo "= app registration exists"
  # make sure the redirect URI is present (keep any existing ones)
  EXISTING=$(az ad app show --id "$APP_ID" --query "web.redirectUris" -o tsv | tr '\t' '\n')
  if ! echo "$EXISTING" | grep -qxF "$REDIRECT_URI"; then
    # shellcheck disable=SC2086
    az ad app update --id "$APP_ID" --web-redirect-uris $EXISTING "$REDIRECT_URI"
    echo "+ added redirect URI"
  else
    echo "= redirect URI already set"
  fi
fi

# 2) Delegated Microsoft Graph permissions (IDs looked up, not hard-coded)
CURRENT=$(az ad app show --id "$APP_ID" \
  --query "requiredResourceAccess[?resourceAppId=='$GRAPH'].resourceAccess[].id" -o tsv | tr '\t' '\n')
for S in "${SCOPES[@]}"; do
  SID=$(az ad sp show --id "$GRAPH" --query "oauth2PermissionScopes[?value=='$S'].id | [0]" -o tsv)
  if [ -z "$SID" ]; then echo "ERROR: Graph scope $S not found"; exit 1; fi
  if echo "$CURRENT" | grep -qxF "$SID"; then
    echo "= permission $S already present"
  else
    az ad app permission add --id "$APP_ID" --api "$GRAPH" --api-permissions "${SID}=Scope" >/dev/null 2>&1
    echo "+ added permission $S (delegated)"
  fi
done

# 3) Service principal (the "Enterprise application" entry used for consent)
if [ -z "$(az ad sp list --filter "appId eq '$APP_ID'" --query "[0].id" -o tsv)" ]; then
  az ad sp create --id "$APP_ID" >/dev/null
  echo "+ created service principal"
fi

# 4) Optional tenant-wide admin consent (only if user consent is blocked)
if [ "$ADMIN_CONSENT" = "1" ]; then
  sleep 10   # give Entra a moment to see new permissions
  az ad app permission admin-consent --id "$APP_ID"
  echo "+ admin consent granted"
fi

# 5) Client secret (only on first creation or with --new-secret)
SECRET=""
if [ "$CREATED" = "1" ] || [ "$NEW_SECRET" = "1" ]; then
  END_DATE=$(date -u -d "+${SECRET_DAYS} days" +%Y-%m-%dT%H:%M:%SZ)
  SECRET=$(az ad app credential reset --id "$APP_ID" --append --display-name "$SECRET_NAME" \
            --end-date "$END_DATE" --query password -o tsv 2>/dev/null)
  echo "+ created client secret '$SECRET_NAME' (expires $END_DATE)"
else
  echo "= secret not changed (use --new-secret to create one)"
fi

cat <<EOF

================ VALUES FOR THE FRAPPE CONNECTED APP ================
Connected App name : $CONNECTED_APP
Client ID          : $APP_ID
OpenID config URL  : https://login.microsoftonline.com/$TENANT_ID/v2.0/.well-known/openid-configuration
Redirect URI       : $REDIRECT_URI
EOF
if [ -n "$SECRET" ]; then
cat <<EOF
Client SECRET      : $SECRET
   ^ shown ONCE. Save it in the password manager now, with its expiry date.
     Do not paste it into chat, tickets or files.
EOF
fi
echo "====================================================================="

```

## D. ocr_add_customer.py

Optional scripted alternative to phase 25.3. Run on the OCR server as `ubuntu`: `/usr/bin/python3 ocr_add_customer.py --app-dir /home/ubuntu/canofy_OCR --code BEL --ip 172.31.10.27 --bucket bel.canofy.tech --site bel.canofy.tech --ocr-url http://172.31.35.206:5000/generate_ocr --dry-run`.

```python
#!/usr/bin/env python3
"""
Canofy OCR - add a customer (ERP server) to property.json

Run ON THE OCR SERVER, with the same Python the OCR service uses:

  /usr/bin/python3 ocr_add_customer.py --code BEL --ip 172.31.10.27 --bucket bel.canofy.tech --site bel.canofy.tech --dry-run
  /usr/bin/python3 ocr_add_customer.py --code BEL --ip 172.31.10.27 --bucket bel.canofy.tech --site bel.canofy.tech

What it does
  * reads ENCRYPTION_KEY from ~/canofy_OCR/.env (never typed, never printed)
  * generates a random OCR username + password for this customer
  * Fernet-encrypts them exactly like encrypt.py, and checks they decrypt
  * refuses to overwrite an existing IP entry (use --replace to rotate)
  * backs up property.json, writes the new entry, validates the JSON
  * prints the 3 'bench set-config' commands to run on the ERP server
    (ocr_username / ocr_password base64-encoded, ocr_url)
"""
import argparse
import base64
import json
import os
import secrets
import shutil
import socket
import sys
from datetime import datetime

from cryptography.fernet import Fernet


def read_env_key(env_path):
    with open(env_path) as f:
        for line in f:
            line = line.strip()
            if line.startswith("ENCRYPTION_KEY="):
                return line.split("=", 1)[1].strip().strip('"').strip("'")
    sys.exit(f"ERROR: ENCRYPTION_KEY not found in {env_path}")


def own_private_ip():
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    try:
        s.connect(("10.255.255.255", 1))
        return s.getsockname()[0]
    finally:
        s.close()


def main():
    home = os.path.expanduser("~")
    p = argparse.ArgumentParser()
    p.add_argument("--code", required=True, help="customer code, e.g. BEL")
    p.add_argument("--ip", required=True, help="ERP server PRIVATE IP, e.g. 172.31.10.27")
    p.add_argument("--bucket", required=True, help="customer S3 bucket, e.g. bel.canofy.tech")
    p.add_argument("--site", required=True, help="Frappe site name, e.g. bel.canofy.tech")
    p.add_argument("--app-dir", default=os.path.join(home, "canofy_OCR"))
    p.add_argument("--ocr-url", help="default: http://<this server private IP>:5000/generate_ocr")
    p.add_argument("--replace", action="store_true", help="rotate credentials of an existing IP entry")
    p.add_argument("--dry-run", action="store_true")
    a = p.parse_args()

    prop_path = os.path.join(a.app_dir, "property.json")
    env_path = os.path.join(a.app_dir, ".env")
    for path in (prop_path, env_path):
        if not os.path.isfile(path):
            sys.exit(f"ERROR: not found: {path}  (use --app-dir if the OCR app lives elsewhere)")

    fernet = Fernet(read_env_key(env_path).encode())
    with open(prop_path) as f:
        props = json.load(f)
    if not isinstance(props, dict):
        sys.exit("ERROR: property.json is not a JSON object")

    # Report existing entries (IP -> bucket) and sanity-check their format
    print(f"property.json has {len(props)} entries:")
    for ip, entry in props.items():
        print(f"  {ip:16} -> {entry.get('BUCKET NAME')}")
        u = str(entry.get("username", ""))
        if u.startswith("b'") or u.startswith('b"'):
            print("    NOTE: this entry stores values as b'...' text; new entry uses plain token text")
        try:
            fernet.decrypt(u.strip("b'\"").encode())
        except Exception:
            print("    WARNING: this entry does not decrypt with the current key")

    if a.ip in props and not a.replace:
        sys.exit(f"ERROR: {a.ip} already exists (bucket {props[a.ip].get('BUCKET NAME')}). Use --replace to rotate.")
    for ip, entry in props.items():
        if entry.get("BUCKET NAME") == a.bucket and ip != a.ip:
            print(f"WARNING: bucket {a.bucket} is already used by {ip}")

    username = f"{a.code.lower()}_ocr_{secrets.token_hex(3)}"
    password = secrets.token_urlsafe(24)
    enc_user = fernet.encrypt(username.encode()).decode()
    enc_pass = fernet.encrypt(password.encode()).decode()
    assert fernet.decrypt(enc_user.encode()).decode() == username
    assert fernet.decrypt(enc_pass.encode()).decode() == password

    new_entry = {"BUCKET NAME": a.bucket, "username": enc_user, "password": enc_pass}
    ocr_url = a.ocr_url or f"http://{own_private_ip()}:5000/generate_ocr"

    print(f"\nNew entry for {a.ip}: BUCKET NAME={a.bucket}, username/password = Fernet tokens (verified)")
    if a.dry_run:
        print("\nDRY RUN - property.json not changed.")
        return

    backup = f"{prop_path}.bak.{datetime.now():%Y%m%d-%H%M%S}"
    shutil.copy2(prop_path, backup)
    props[a.ip] = new_entry
    tmp = prop_path + ".tmp"
    with open(tmp, "w") as f:
        json.dump(props, f, indent=4)
    with open(tmp) as f:
        json.load(f)  # validate before replacing
    os.replace(tmp, prop_path)
    print(f"Backup : {backup}")
    print(f"Written: {prop_path}  ({len(props)} entries)")

    b64 = lambda s: base64.b64encode(s.encode()).decode()
    print("\n=========== RUN THESE ON THE ERP SERVER (as frappe, in ~/frappe-bench) ===========")
    print(f"bench --site {a.site} set-config ocr_username {b64(username)}")
    print(f"bench --site {a.site} set-config ocr_password {b64(password)}")
    print(f"bench --site {a.site} set-config ocr_url {ocr_url}")
    print("===================================================================================")
    print("These are credentials: copy them straight into the ERP session. Do not paste them")
    print("into chat, tickets or email. Then restart the OCR service:")
    print("  sudo systemctl restart ocr_app.service && sudo systemctl status ocr_app.service --no-pager")


if __name__ == "__main__":
    main()

```

## E. Screenshot register

Figures embedded from the Belimo onboarding, with sensitive identifiers masked, and screenshots still to capture for the SOP.

| Figure | Content | Status |
| --- | --- | --- |
| 1 | SS-05a: the Marketplace "Subscribe" page for Ubuntu 22.04. Close it and use the Canonical Community AMI instead. | embedded |
| 2 | SS-06: instance summary after launch: Running, t3a.medium, role EC2-Canofy-BEL-SSM-Prod, auto-assigned public IP. | embedded |
| 3 | SS-07: volume vol-0cc1d95a7394d274a: 20 GiB gp3, 3000 IOPS, Encrypted with aws/ebs. Note the Customer tag here still holds a wrong value, which was corrected afterwards. | embedded |
| 4 | SS-08: the Elastic IP 3.6.128.118 now replaces the auto-assigned address. | embedded |
| 5 | SS-10: first SSM session checks: ssm-user, Ubuntu 22.04.5, agent active, outbound IP 3.6.128.118. | embedded |
| 6 | SS-11: bucket properties: versioning enabled, SSE-S3 with Bucket Key, SSE-C blocked. The tag key "customer" must be "Customer". | embedded |
| 7 | SS-17: role identity and S3 upload, list and delete, with no access keys on the server. | embedded |
| 8 | Before: the saved policy has no bel.canofy.tech lines, so OCR is blocked by the Deny statement. | embedded |
| 9 | SS-13: after: BEL lines 20–21 (Allow) and 40–41 (Deny/NotResource), "updated" banner, Errors 0. | embedded |
| 10 | The suggestion markers on lines 22–23 come from the MIL trailing-slash typo, not from the new entries. | embedded |
| 11 | Existing policies: an enabled default 14-day policy for all volumes, and the enabled 60_day_production_instances policy. | embedded |
| 12 | SS-15: target tags of the 60-day policy. Every customer uses key "Name" with the volume name. The "BEL" key entry is the mistake to avoid. | embedded |
| 13 | SS-11a: hostname bel-prod, timezone Asia/Kolkata (IST). | embedded |
| 14 | SS-11b: 4.0 GB swap active on /swapfile after the reboot. | embedded |
| 15 | SS-SW-01: frappe in the sudo group, Python 3.11.16, Node v20.20.2, yarn 1.22.22, MariaDB 10.6.23, Redis 6.0.16, nginx 1.18.0, wkhtmltopdf 0.12.6.1 with patched Qt. | embedded |
| 16 | SS-DB-01: the config written with printf (one setting per line) and MariaDB active (running). The first lines show the earlier merged "256M\[mysql\]" that broke the service. | embedded |
| 17 | SS-DB-02: secure installation done; utf8mb4, 127.0.0.1, 1 GB buffer pool; root password login works; system Redis inactive. | embedded |
| 18 | SS-19: bench initialised, frappe 15.72.5, Python 3.11.16. | embedded |
| 19 | Reference versions on MIL: frappe 15.72.5, canofy_tcs 2.2.1, canofy_ddit 1.0.2, s3_file_storage 0.0.1, custom_mil_power (MIL only), Python 3.10.12, Node 18.20.8. | embedded |
| 20 | App remotes on MIL: canofy_TCS on development (e5c1cf96), canofy_ddit main (v1.0.1 tag, 782a128), priv-frappe-s3-file-storage main (163256d). | embedded |
| 21 | Without credentials, git reports "could not read Username", which means the repo is private. | embedded |
| 22 | SS-21: versions and commits after download. canofy_tcs on development came in at 2.3.0, newer than MIL's 2.2.1, which is expected when following a branch. | embedded |
| 23 | Settings → Secrets and variables → Actions is NOT where deploy keys go. Deploy keys need admin rights on the repo. | embedded |
| 24 | canofy_tcs installed: DocTypes 100%, Address customisations, Dashboard. | embedded |
| 25 | SS-22: apps on the BEL site: frappe 15.72.5 (pinned tag), canofy_tcs 2.3.0 development, s3_file_storage 0.0.1, custom_belimo 1.2.3. canofy_ddit stays in the bench only. | embedded |
| 26 | SS-23: sudoers set up, setup production complete, all 7 processes RUNNING, HTTP 200. | embedded |
| 27 | SS-25: log format file, nginx test OK, HTTPS 200, HTTP 301, renewal dry-run succeeded. | embedded |
| 28 | SS-24: login page served at https://bel.canofy.tech, before branding. | embedded |
| 29 | SS-26: scheduler enabled; bench's 6-hourly local backup plus the 01:30 S3 job; test backup (330.2 KiB) in \_backups/. | embedded |
| 30 | SS-AUTO-01: Belimo apply output: logos placed as local public files, settings changed, help menu, Core Apps, 5 Attachments, S3 File Settings, "APPLIED and committed". | embedded |
| 31 | Signed in as no-reply-bel: Canofy Technologies tenant, display name "Canofy Support \[Belimo\]", no roles needed. | embedded |
| 32 | SS-34: Canofy-BEL overview (IDs masked): single tenant, 1 web redirect URI, Activated. | embedded |
| 33 | SS-35: Microsoft Graph (4), all delegated, admin consent not required. | embedded |
| 34 | SS-36: Canofy-BEL-secret, expires 25 Sep 2027 (value and secret ID masked). | embedded |
| 35 | The service runs gunicorn from /home/ubuntu/canofy_OCR, so the live file is /home/ubuntu/canofy_OCR/property.json. | embedded |
| 36 | Several copies exist (backups and a stray /home/property.json). Edit only the copy in the service's working directory. | embedded |
| 37 | ssm-user can't list /home/ubuntu; switch with sudo -iu ubuntu first. | embedded |
| 38 | MariaDB failed: heredoc lines merged by the SSM terminal. | embedded |
| 39 | bench init stuck at a sudo password prompt for frappe. | embedded |
| 40 | bench init traceback at sudo supervisorctl status, with the rollback prompt. | embedded |
| 41 | The bench folder wasn't found on another server: always locate it first with find … -name apps.txt. | embedded |
| 42 | Loop variables are lost inside sudo -iu … bash -c '…': switch user first. | embedded |
| 43 | bench get-app failed authentication: a password or failed paste instead of a token. | embedded |
| 44 | Expected "no such group" warning after get-app, before production mode. | embedded |
| 45 | new-site failed: No module named pkg_resources (Dropbox Settings import). | embedded |
| 46 | install-app failed: Row size too large on tabBOE Items. | embedded |
| 47 | install-app custom_belimo: InvalidRemoteException. | embedded |
| 48 | Traceback showing the required_apps lookup on GitHub (org erpnext → 404). | embedded |
| 49 | bench doctor: Redis 11000 refused before production mode. | embedded |
| 50 | The same Redis error after a setup production that didn't complete. | embedded |
| 51 | 502 Bad Gateway: nginx linked, but /etc/supervisor/conf.d is empty. | embedded |
| 52 | supervisorctl status shows no processes. | embedded |
| 53 | After the manual supervisor link and chown: 7 processes RUNNING, HTTP 200. | embedded |
| 54 | setup sudoers failed: certbot not installed. | embedded |
| 55 | Certificate issued, but nginx -t fails with unknown log format "main". | embedded |
| 56 | Only the logos reached S3, in the wrong folder, so the script was missing. | embedded |
| 57 | Dry run: System Settings, Navbar and Core Apps OK; Attachments failed on FTWZ names (rolled back). | embedded |
| 58 | Apply failed in the s3_file_storage hook before the S3 settings existed (rolled back). | embedded |
| 59 | grep -c placed = 0: an old script was still in S3. | embedded |
| 60 | The OCR policy editor with BEL lines added. Without Next → Save changes, this isn't saved. | embedded |
| 61 | set-config -p server_script_enabled true raises a malformed node error; use 1. | embedded |
| — | SS-04: Security group inbound rules showing only 80 and 443 | to capture |
| — | SS-12: Bucket Permissions tab with Block Public Access on and the policy saved | to capture |
| — | SS-14: Both lifecycle rules listed on the Management tab | to capture |
| — | SS-16: GoDaddy DNS record list showing the bel A record | to capture |
| — | SS-27: Login page and desk with Canofy branding | to capture |
| — | SS-37: Saved Connected App, and the Entra Authentication page with the matching redirect URI | to capture |
| — | SS-38: Icegate Intimation record created from the test email | to capture |