Lecture → Living Mind-Map
Built while running — the team could not sit at a keyboard
Speak through a lecture and the concepts worth diagramming grow into an interactive map, live — extracted every few seconds and laid out by a physics simulation that never lets a card overlap.
- Time to first diagram, on my branch
- ~20s→~3s
- partials forwarded and force-committed; the submitted build kept the 20s interval
- Comprehension levels per concept
- 3
- Intuition · Mechanism · Rigour
- Card overlap at 20 nodes
- 0 pairs
- physics settled, pairwise bounding-box check
- Build time
- 1 day
- team of three, on the move
Figures with a dotted underline carry their source. Hover one to see whether it was verified in code or reported in project documentation.
Context
The situation
RUN/HACK is a hackathon with an unusual constraint: only the teammate currently out on the running loop may touch the build, hands-free, by voice from a phone, with cloud agents doing the typing. The moment nobody on a team is running, their agents pause — and anything slower than seven minutes per kilometre does not count as running. Teams are scored on what they built multiplied by how far they ran.
The problem
A lecture arrives as a linear stream, but the thing you actually want from it is a structure — which concepts exist, how they relate, which are processes and which are definitions. That structure only becomes visible after the lecture has finished, which is exactly too late.
What I built
Build the map while the lecture is still running. Stream the transcript to the backend continuously, and every extraction cycle hand the model both the new text and the graph built so far, asking for the complete updated graph rather than a diff. Reusing ids across cycles is what makes accumulation safe: a concept covered in minute two is still on the canvas in minute forty.
A lecture is linear and a mind-map is not, which is why note-taking is hard: the structure only becomes visible once the lecture is over. This turns speech into a concept graph while the lecture is still happening. The transcript streams to the backend over a WebSocket; every few seconds it is sent to a model along with the graph built so far, and the model returns the complete updated graph, reusing node ids so nothing already covered is silently dropped or redefined. The frontend merges each graph into a running state and lays it out with a force simulation plus a custom rectangle-collision force, so cards never overlap however dense the map gets.
Architecture
How it fits together
- Speechinput
lecture audio, or pasted text
- Transcriptiondeterministic
partials forwarded immediately
- WebSocketdeterministic
transcript state per connection
- Graph extractionlanguage model
prior graph fed back into the prompt
- Never-delete mergeguard
ids reused, nothing dropped
- Force layoutdeterministic
rect-collision, zero overlap
- Living mapoutput
flip-cards, three levels, animated processes
Key decisions
The choices that shaped it
- 01
Return the whole graph, not a diff
Each extraction cycle sends the model the transcript and the map built so far, and asks for the complete updated graph with ids reused for anything already covered. A diff-based protocol would have been cheaper and would have quietly lost concepts every time the model forgot one; returning the whole graph makes accumulation the default and deletion something the merge layer has to allow explicitly.
- 02
Partials, not final results
The browser speech engine only fires on final results, which meant nothing reached the extractor until a sentence had settled and then nothing again until the next 20-second cycle. Forwarding partial transcripts immediately and force-committing on a short timer brought the first diagram down from roughly twenty seconds to a few — which is the difference between a demo that looks live and one that looks broken.
- 03
Three levels, one map
Every concept carries an Intuition, a Mechanism and a Rigour description. The same map then serves someone meeting the material for the first time and someone revising it, without building two artefacts — and switching between them is a spoken intent, because on this build nobody had a free hand.
- 04
A model chain across separate quota pools
Each model name has its own free-tier quota, so the backend tries a fallback chain rather than one model. A quota-exhausted model mid-demo takes the app down otherwise — which it nearly did once, and the chain is why it did not.
- 05
The reliable path won, and my branches lost
With about two hours left the team reverted to the known-good MVP. That call took my streaming-transcription path, the three comprehension levels and the voice-driven level switching out of what we submitted — they sit on named branches, unmerged, rather than in the demo that placed. Under a live demo the question is not which implementation is better but which one you have watched survive, and on that question the revert was right. It is the part of this project I would defend, and the part that cost me the most.
commit: Revert to pre-hackathon working MVP
Evidence
What it actually looked like
Captured from the running application. The lecture text was pasted in and the concepts, the categories and the edge labels all came back from a live extraction — five nodes and four edges in 3.2 seconds.
Interface
The pitch happened where the build did: on the track, at the end of the run window.
EventRecognition
Result
Europe’s first running hackathon — teams of three, built hands-free while running
100 places · teams of three · selected to pitch
Selected by the judges from the day’s demo videos to pitch in the closing round.
The result is context. What the system does, and how it was measured, is everything above this.
My contribution
One of three. Built the realtime transcript path and the comprehension levels — on branches that were not merged into what we submitted.
What I owned
- A streaming transcription path against a dedicated speech API, alongside the browser engine — branch `elevenlabs-realtime-stt`
- Low-latency transcript delivery: forward partial results immediately and force-commit on a short timer instead of waiting out the extraction interval
- Tracing every speech-recognition event and surviving a dropped connection without losing the accumulated transcript
- Three comprehension levels per concept — Intuition, Mechanism, Rigour — branch `comprehension-levels`
- Voice-driven level switching, a spoken intent moving the whole map between levels — branch `voice-levels`
- The handover, demo and pitch documents the team ran the final hour on, including the lecture script whose extraction output was pinned and verified
What I did not
- The graph extraction service and its never-delete merge safety net
- The force-directed layout, the rectangle-collision force and the flip-card rendering
- Per-node follow-up Q&A, on-demand image generation, and the widget, quiz, video and summary generators
Stack and limits
Built with
- Python
- FastAPI
- WebSockets
- JavaScript
- d3-force
- Gemini
- ElevenLabs
- Web Speech API
Techniques
- Streaming speech-to-text with partial forwarding
- Stateful graph extraction with id reuse
- Force-directed layout with a custom collision force
- Voice intent routing
- Model fallback across quota pools
What it is not
- One day, three people, and part of that day spent running. The extraction interval is a conservative guess against free-tier rate limits, not a measured optimum.
- Every model call runs inline on the request thread. Fine for one user, not for several.
- Browser speech recognition could not be verified under automation; the paste-in text box is the path that was proven end to end.
- What we submitted is the team’s MVP plus the extraction, layout and generator work my teammates built. My own branches were not merged into it — the screenshot on this page is the product that placed, not the product I contributed code to.
Next
July – August 2026
Omnivision
Assistive tools describe scenes but carry no memory, and cannot tell recognising something from being confident about it. I built the part that refuses.
Read case study→July 2026
Obligation Ledger
The model proposes a termination date. The service throws it away and recomputes it from verified quotes. Nothing is approvable until every value has been found on its page.
Read case study→