Professional Education Case study
Fluence
Science-backed psychedelic-therapy education, professional certificates and clinical training for healthcare professionals.
Professional Education
Fluence Fluence
Lead Engineer
System map · Processing pipeline
Engineering approach
- Every score traceable Linked to the moment in the session that supports it
- Six inspectable stages A wrong result is attributable to one of them
Analysis pipeline
Transcript
Submitted with an API key
Segmentation
Each stage inspectable
Quote extraction
Located in the transcript
Evaluation
Scored components per criterion
Report
A webhook notifies the caller
Evidence, not opinion
- Every score linked to quoted evidence
- Per-stage caching
- Programme-specific criteria
- Failures attributable to a stage
Engineering
Dashboard engineering · Backend architecture · Analysis pipeline · Course delivery
Public product · Proprietary source code
- Role
- Lead Engineer
- Disclosure
- Public product · Proprietary source code
- Links
- Product website
The problem
What the system had to solve.
Clinical training in psychedelic therapy is assessed on how a practitioner conducts a session — whether they held a boundary, followed the participant, responded to distress appropriately. That has always required an experienced instructor to sit through a recording and form a view, which does not scale and is inconsistent between assessors.
A model producing a score is not an acceptable substitute in a clinical training context unless every judgement can be traced back to what was actually said in the session.
What I owned
The responsibility behind the work.
Led engineering for the product: the dashboard experience, the backend behind course delivery and user management, and the multi-stage analysis pipeline that turns a session transcript into a defensible assessment.
Led engineering for the core EdTech product, developing the dashboard experience and a scalable backend for course delivery, user management, learning workflows and platform operations.
Engineering approach
How it was built.
Analysis is decomposed into stages — segmentation, annotation, quote extraction, evaluation, debrief analysis and summarisation — each with an inspectable output, so a wrong result is attributable to a stage. Evaluations are stored as scored components linked to quoted evidence from the transcript rather than as scores with a comment, assessment criteria are configuration per training programme, and model responses are cached so re-analysis does not repeat a full pipeline run.
Analysis pipeline
- Transcript
- Segment
- Extract
- Evaluate
- Report
Assessing a conversation
Why this is harder than grading a test.
Clinical training in this field is assessed on how a practitioner conducts a session: whether they held a boundary, followed the participant rather than leading them, responded to distress appropriately. That judgement has always required an experienced instructor to sit through a recording and form a view.
It does not scale, it is inconsistent between instructors, and the feedback arrives long after the session it concerns. But the alternative — a model producing a score — is unacceptable in a clinical training context unless the score can be traced to what was actually said.
The requirement is not a grade. It is a defensible assessment: every judgement attached to the moment in the transcript that supports it.
A multi-stage analysis pipeline
One model call cannot do this.
A session transcript is long, and asking one model call to read it and produce a scored assessment gives an answer that cannot be checked. The analysis is decomposed into stages, each with a narrow job and an inspectable output, so a wrong result can be traced to the stage that produced it.
Analysis stages
- Transcript Timed, speaker-attributed
- Segmentation Identify session phases and their boundaries
- Annotation Add context to the transcript
- Quote extraction Pull candidate evidence per criterion
- Evaluation Assess against each criterion
- Debrief analysis Assess the discussion afterwards
- Summarisation Overall performance
- Report
Segmentation first matters: an assessment criterion that applies to one phase of a session should not be scored against what happened in another.
Evidence, not opinion
The data model is the accountability mechanism.
Evaluations are not stored as scores with a comment. They are stored as structured components, each attached to the evidence that supports it, and that evidence is a quote taken from the transcript — often as an exchange rather than a single line, because what a practitioner said only means something alongside what it was responding to.
From transcript to defensible judgement
- Transcript
- Quote exchange
- Quote
- Evidence
- Evaluation component
- Evaluation
Two ways to store an assessment
Deliberately avoided
- A score and a paragraph
- No link to what was said
- Cannot be checked or appealed
- Disagreement is unresolvable
Implemented
- Scored components per criterion
- Each backed by quoted evidence
- Evidence located in the transcript
- A disagreement can be examined
Different programmes, different standards
Assessment criteria as configuration.
Training programmes do not share one rubric. Each has its own criteria, its own vocabulary and its own view of what competent practice looks like. The pipeline is therefore parameterised by criteria set: the stages are the same, and what each stage is looking for is configuration rather than code.
This is what makes the system a product rather than one organisation’s assessment script. Onboarding a programme means expressing its standard, not forking the analyser.
Caching model work
A multi-stage pipeline is expensive by construction.
Six stages over a long transcript is a lot of model calls, and re-running an analysis during development or after a prompt change would repeat all of them. Model responses are cached, so re-analysis, iteration on a later stage, and reproducing a previous result do not each cost a full pipeline run.
What caching buys
- Iterating on a late stage without re-running early ones
- Reproducing a previous analysis
- Bounded cost per transcript
- Faster development cycles
A service, not a script
What makes it usable by another system.
The analysis runs behind an API with key-based access and webhook subscriptions, so a learning platform can submit a transcript and be told when the report is ready rather than holding a connection open for the length of a multi-stage pipeline.
Integration path
- Transcript submitted with an API key
- Accepted for analysis
- Pipeline runs
- Report stored
- Webhook notifies the caller
Also recorded
- Session and summary records
- Observations and recommendations
- Run logs for each analysis
Engineering scope
What I worked on.
- Core dashboard experience
- Course-delivery workflows
- User and learner management
- Scalable backend foundations
- Platform operations
Why these decisions mattered
The trade-offs behind the analysis.
Stages over one large call
Evidence in the data model
Quote exchanges, not single lines
Criteria as configuration
Cache model responses
Asynchronous with webhooks
Impact
What the design makes possible.
Every scored judgement can be traced to the moment in the session that supports it, which is what makes an automated assessment defensible enough to use in clinical training.
Technology
The technical surface.
- Dashboard engineering
- Backend architecture
- Course delivery
- User management