MediVault AI — Offline Clinical Intelligence Platform
MediVault AI is an offline-first clinical intelligence platform that captures doctor-patient consultations via live browser recording or audio upload and uses Speech-to-Text and Large Language Models to automatically generate structured SOAP notes, build a searchable clinical knowledge base, and enable natural-language querying over approved notes and uploaded clinical guidelines.
Developed as an open-source blueprint under the Cloud2 Labs Innovation Hub, MediVault AI demonstrates how locally-running Whisper models and local LLMs, orchestrated through Flowise, can be combined into a practical, privacy-preserving clinical documentation and decision-support workflow.
It showcases a production-style containerised architecture for capturing audio in the browser, performing gap-based speaker diarization, generating specialty-aware SOAP notes with ICD-10 and CPT codes, and building a persistent RAG knowledge base with Flowise serving as the visual orchestration and LLM chain layer without any patient data leaving the clinician's environment.
What It Demonstrates
MediVault AI illustrates how to:
- Capture browser audio live via the MediaRecorder API or upload WAV and MP3 files
- Transcribe speech using a locally-running Whisper container with no cloud speech-to-text dependency
- Perform contextual speaker diarization (Doctor vs. Patient) by sending timestamped audio segments to a local LLM then model determines speaker roles and combines adjacent same-speaker turns into coherent dialogue
- Orchestrate SOAP note generation and Clinical QA as named Flowise chatflows automatically provisioned on every startup. so the full LLM chain topology is visible and inspectable in the Flowise visual canvas
- Generate structured SOAP notes (Chief Complaint, Subjective, Objective, Assessment, Plan) using specialty-aware LLM prompting
- Surface ICD-10 diagnosis codes and CPT procedure codes directly within the generated clinical note
- Provide an interactive human-in-the-loop editor allowing clinicians to edit each SOAP section individually before approval
- Approve notes into a persistent ChromaDB vector knowledge base via direct Python client embedding, with Flowise's ConversationalRetrievalQAChain reading from the same collection for semantic retrieval
- Answer clinical questions via Retrieval-Augmented Generation through a Flowise QA flow backed by ChromaDB and local Ollama embeddings, returning cited source documents with document type and patient reference
- Ingest PDF clinical guidelines into the knowledge base alongside approved SOAP notes
- Run entirely offline using Ollama and local Whisper zero cloud dependency in default mode
- Ensure all patient audio, transcripts, and notes remain on-premises with no external data transmission
- Designed for healthcare providers, telemedicine platforms, clinical documentation teams, and health-tech innovation groups, MediVault AI serves as a reference implementation for offline AI-driven clinical scribing and knowledge management.
Key Capabilities
Automated SOAP Notes via Flowise LLMChain
Translates raw conversational transcripts into clinically formatted notes with Chief Complaint, Subjective, Objective, Assessment, and Plan sections. SOAP generation runs through a dedicated Flowise LLMChain flow connected to a local ChatOllama model, the full chain is auto-provisioned at startup and inspectable live in the Flowise canvas. Specialty-aware prompting tailors terminology and structure across Cardiology, Neurology, Orthopaedics, Psychiatry, and seven other medical specialties.
LLM Based Contextual Speaker Diarization
Sends all timestamped audio segments to the local LLM in a single prompt. The model determines whether each segment belongs to the Doctor or Patient based on clinical context clues: greetings, exam findings, symptom descriptions and combines consecutive same-speaker segments into coherent turns. No multi-channel audio hardware required. Clinicians can manually reassign any turn or edit transcript text before SOAP generation
ICD-10 & CPT Code
Extraction
Analyses the finalised SOAP note to suggest relevant ICD-10 diagnosis codes and CPT procedure codes inline, directly within the generated note. Assists medical coding departments and reduces manual billing effort without leaving the clinical workflow.
Clinical QA via Flowise ConversationalRetrievalQAChain
Approved SOAP notes and uploaded PDF clinical guidelines are embedded using local Ollama embeddings and stored in a ChromaDB vector store. Clinical QA runs through a dedicated Flowise ConversationalRetrievalQAChain flow with ChromaDB retriever, BufferMemory for conversation history, and a local ChatOllama model returning answers with cited source documents and document type badges. The complete RAG chain topology is visible and editable in the Flowise canvas.
Zero-Storage, On-Premises Architecture
Audio is processed in-memory and never written to permanent storage beyond the clinician's approved session. All SOAP notes, transcripts, and embeddings remain entirely on-premises. No patient PII or audio reaches any external service in the default offline configuration.
Flowise as the Orchestration and Inspection Layer
All LLM workflows SOAP generation, Clinical QA, and KB upsert are defined as Flowise chatflows and automatically provisioned on every API startup. Flowise visual canvas is always a live, accurate reflection of the running architecture: clinicians and developers can open Flowise at any time to inspect prompt templates, trace chain execution, adjust LLM parameters, or swap models without touching code.
Containerised Architecture
Uses a fully orchestrated Docker Compose deployment — React frontend, FastAPI backend, Flowise orchestration layer, ChromaDB vector store, and Whisper ASR all start with a single docker compose up command. No manual installation required beyond Docker and Ollama on the host machine.