Running a volunteer literacy programme on Elixir

Summary

A foundation runs a volunteer-led literacy programme in Kenyan primary schools. Volunteers, known internally as field teachers, run teaching sessions in clusters of schools, and are paid per day taught.

The system behind that programme holds the people (staff, volunteers, learners), the places (schools grouped into clusters), the calendar (academic years and terms), what happened (attendance, assessments, trainings) and the money (monthly pay runs settled over M-Pesa, plus airtime).

I have been the lead engineer on it since 2018. It is a single Phoenix application, deployed to a self-hosted production environment, and it has been carrying real payroll every month for eight years.

The problem

The programme runs where the connectivity is poor, the paperwork is manual and the money is real. Three things had to be true at once.

Most of the interesting engineering sits at the seam between those three: a data pipeline that cannot be trusted, feeding a payroll that cannot be wrong, in a system where the audit trail is a first-class feature rather than a logging afterthought.

What I built

Attendance ingestion from the field

Tablet punches land in a CSPro database and are pulled in on a schedule. The pipeline is deliberately unforgiving: a record that cannot be matched to a volunteer, a school, a term or a valid time is not silently dropped and not half-written. It is recorded as a sync failure with a specific reason, and surfaced on an admin page where somebody can retry it, ignore it, or export the batch as CSV.

That page exists because the failures are mostly not bugs. They are a volunteer posted to a school the roster does not agree with, a punch dated a day the term had already ended, a tablet with a clock 24 hours out. Those are decisions for a person, so the software's job is to state the reason in a sentence the person can act on, and then get out of the way.

On top of the raw punches sit attendance flags: punches that are individually valid but suspicious in context. A checkout before midday. A punch whose GPS puts the volunteer far from the school they are assigned to. A session dated differently from the check-in it belongs to. Each flag can be dismissed with a reason or resolved by an edit, and both paths are signed.

A payments ledger with real gates

Pay is a pure function of attendance plus trainings, testing and allowances, minus deductions. That calculation lives in one module of pure functions, which makes it property-testable and makes the prose documentation impossible to drift from the code without a test failing.

Around that calculation is a month lifecycle: a month opens, accumulates attendance, gets summarised and sent for approval, is approved, and only then can anybody be paid. Two separate gates guard writes into a month: an ingestion gate, for records arriving on their own from the sync, and an edit gate, for a person deliberately changing a day and signing for it. Ingestion is never more permissive than editing.

Settlement itself is driven by uploading the mobile money provider's real bulk payment statement: a line becomes a payment only where the statement reports success, the number belongs to that volunteer, and the amount matches what they are still owed. Everything else comes back with a reason attached, and nothing is written for it. Re-uploading the same file records nothing twice.

A month closes itself when the last volunteer settles, rather than waiting for somebody to remember to press a button. "Settled" is two things, not one: some of what a volunteer is owed is paid as cash over M-Pesa and some as an airtime top-up, tracked as separate balances, and the month closes only when both clear.

Authorisation and audit as product features

Access control follows the organisation's actual org chart: six staff roles from the head of department down to the team leader who answers for one school, plus a read-only role for the volunteer portal. Authorisation is never a role check. It is a permission (may this user do X) crossed with a data scope (which rows do they reach: their own school, their own cluster, everything), enforced at the route's mount hook, again inside the action, and again in the context's authorizer.

The scope axis has one rule that matters more than the rest: an unresolvable scope returns no rows, never all of them. A cluster-scoped user with no posting this year sees nothing. Widening on missing data is how scoped systems leak.

Every user action that changes something records an audit entry naming the actor, the record, the before and after. Those entries drive a searchable audit trail page, a per-user activity view on the profile, and the provenance shown next to a flagged or edited attendance day. Building it as one shared mechanism, rather than per-feature logging, meant later features inherited it for free.

The rest of the surface

• Documents and OCR. Volunteer certificates and IDs upload to S3, get variants generated by Lambda, and run through Textract, with findings raised only where the document contradicts what is on file.
• Messaging. Two-way WhatsApp, SMS campaigns with per-user approval, and airtime disbursement, with a transfer state machine and a balance gate in front of it.
• Learner outcomes. Assessments, per-term and per-year report cards, school performance and learner progress views, and a promotion flow that carries results forward to the next grade.
• A volunteer portal. Passwordless sign-in by phone or email code, opening a read-only view scoped to the learners in the classes that volunteer actually teaches, plus their own payslips.
• Operations. Oban for all periodic work, PromEx with Prometheus and Grafana for metrics, OpenTelemetry tracing, structured logging, and Ansible for deploys.

How it is built

One Phoenix app, contexts by domain. Attendances, Payments, Volunteers, Audit and so on, with the web layer split between classic controllers and LiveView. No microservices: the coordination cost would have dwarfed the benefit for a team this size.

Pure functions pulled out of the contexts. Anything that is really a calculation (pay arithmetic, pagination windows, month-state classification, data normalisation) is extracted into a module with no database access and covered by property tests. Four latent bugs surfaced the week that suite went in, all of them in code that unit tests had called green for years.

Testing at every level, as a standing rule. The end-to-end suite went from nine minutes to under a minute by self-hosting fonts, parallelising specs, replaying authenticated sessions and sharding in CI. That is the difference between a suite people run and a suite people skip, and on a system that pays people it is the difference that matters.

Enforced consistency. Styler and Credo run in a pre-push hook and in CI, so formatting and idiom are never review topics.

What this bought

What I can say without a number: the parts of the system that used to be a spreadsheet and a WhatsApp thread are now a workflow with an owner, a deadline and a record.

What I would tell someone building the same thing