Cited answers over 1,490 SEBI circulars, or an honest abstention.
- RAG
- Evals
- SEBI
Proves AI evaluation design · Retrieval (RAG) product design · Trust and compliance by design (India) · Communicating AI quality honestly
Last public activity
Lecture Lens is an open-source macOS app that turns the online lectures you attend into a timecoded transcript, slide-linked notes and flashcards, using models that run on your own Mac. Nothing is uploaded, copy-protected video is never recorded, and every prompt or model change can be checked with a local evaluation.

Tick = the 3 Oct baseline, before the judge scored the answer body only; same generator and prompt versions. Eval run summaries are in the private prototype; the harness and statistics are public. Transcript completeness is from one measured live class. Source: Evaluation guide ↗
How it started: from the private Screenpipe Study prototype to Lecture Lens · How it is built: architecture and decisions
A working professional in a part-time online course, with recorded lectures during the week, a live class at the weekend, and little time to re-watch either.
| You do | Lecture Lens does |
|---|---|
| Watch a recorded lecture in the browser | Records the audio, pauses screen capture on copy-protected players, logs which page you were on |
| Sit in a live class in a meeting app | Captures slides and audio, picks the distinct slides, describes them with a vision model |
| Click Build | Writes a timecoded transcript, notes with slide and time citations, and flashcards |
| Ask a question | Answers from your own captures, with sources |
| Change a prompt or a model | Runs a local evaluation so you can see whether quality moved |
Each bet had a measure before it had a feature. Numbers are pass counts from the prototype's eval runs, with 95% Wilson intervals because the samples are small.
| Bet | Measure | Result | Decision |
|---|---|---|---|
| Fix the measurement before the model | Answer relevancy and faithfulness, 43 questions, same generator and prompts | Relevancy 31 → 41 of 43 (72% → 95%; intervals 57–83% and 85–99% don't overlap). Faithfulness 37 → 43 of 43, 3 Oct to 14 Oct | Kept: the judge scores the answer body, not its source list. Errored judge calls are counted apart from failures (10 Oct) |
| Notes you test yourself on beat summaries | Notes coverage and faithfulness, 15 cases | 15 of 15 on both (3 Oct and 14 Oct) | Shipped recall-first notes: three questions before the notes |
| Flashcards worth reviewing | G-Eval card quality, 15 cases | 11 then 10 of 15 (67%, interval 42–85%) | Weak spot. Near-duplicate cards dropped (10 Oct); quality is the next prompt target |
| Say "not covered yet" instead of guessing | Questions the class had not answered yet, 8 cases | Hedging rose from 2 to 7 of 8, but agreement with the reference fell from 6 to 1 of 8 | No winner on 8 cases. The suite exposes a real trade-off, so it stays and grows |
| Questions to ask in a live class | 24 suggested questions | Teaching value 24 of 24; answerable from the class so far 10 of 24 (42%) | Open: good questions, but too many ask about material not yet shown |
| A model inside Chrome instead of a model server | Same suite, Gemini Nano, 36 questions and 12 card sets (5 Nov) | Correctness 32 of 36 (89%) vs 39 of 43 (91%) locally; notes 12 of 12; card quality 3 of 12 | Studio ships for answers and notes; cards stay on the local model for now (ADR-006) |
| Fast slide reading without "thinking" | Slide-vision checks on dense, high-resolution slides | Invented content in 0 of 5 hallucination checks with thinking off, 3 of 5 with thinking on | Thinking off for slides (ADR-001) |
I reviewed the six SwiftUI screens against usability heuristics and found 11 issues: 2 blockers, 5 major and 4 minor. The worst was a green "frames arriving" light driven by process status while nothing was being saved.
Three fixes (light driven by frames saved, no eval runs during a live class, show the settings in effect) cover every fault seen in the first live class.
| Output per lecture | What it is |
|---|---|
transcript.md | Timecoded speech-to-text |
slides.md | Distinct slides with a description and on-screen text |
notes.md | Notes with time and slide citations, action items and open questions |
cards.tsv | Flashcards for a spaced-repetition app |
manifest.json | What was built, from what, with which models and prompt versions |
Every prompt or model change can be measured locally. The judge is a different model from the generator, and the judge itself is benchmarked.
| Suite | What it checks | Time |
|---|---|---|
| Smoke | 6 fixed cases: 2 questions, 2 notes, 2 card sets | ~12 min |
| Full | All cases, scored with DeepEval faithfulness, answer relevancy and G-Eval criteria | 1 h+ |
| Judge benchmark | Labelled claim/context pairs: how often is the judge right? | ~10 min |
| Slide reading | Synthetic slides with exact answers | minutes |
Screenshots of the macOS app (11 Nov 2026) showing a short demo lecture on how tides work. The slides, notes and cards were made up for these screenshots and are labelled as invented, so no course material appears. Select a screenshot to open it at full size.

The same notes, flashcards, answers and live-class questions, made by a model running inside Chrome instead of a local model server.

| Decision | Reason | Call |
|---|---|---|
| Pause screen capture on DRM players, keep audio | Protected video goes black while screen capture runs. The protection is respected; transcripts come from audio (ADR-004). | Shipped |
| Three-layer content guard | A public repo about a private activity: ignore rules, a hashed blocklist scan on every commit, and a guarded publish script. | Shipped |
| Evals refuse to run during a live class | Evaluation swaps the model and would disrupt capture. | Shipped |
| Record everything, filter later | Breaks the privacy rule and the protected-video constraint. | Dropped |
| Longer notes and weekly digests | Feels productive; the evidence says summaries you didn't write do little. | Dropped |
| Hosted evaluation dashboards | Captured text would leave the machine. | Dropped |
Employers trust skills they can verify. Each claim below links to its place in the skills matrix, and every piece of evidence is public.
See the product artifacts: brief, eval spec, metrics tree, decisions
Last public activity 6 November 2026 · checked 5 November 2026
Lecture Lens is an open-source macOS app and toolkit by Ian Pinto that turns online lectures into a timecoded transcript, slide-linked notes with citations, and flashcards. It runs entirely on an Apple silicon Mac with local models, and nothing is uploaded.
macOS 15 or later on Apple silicon, with 32 GB of memory or more being realistic for the models; screenpipe installed separately for capture; a local OpenAI-compatible model server with a text model, a vision-capable model and a second model as evaluation judge; Python 3.11+, ffmpeg and Xcode command line tools.
No. On DRM-protected course players, screen capture pauses and only audio is kept, so the video stays watchable. Notes and flashcards for those lectures are built from the audio transcript.
A local evaluation harness generates answers, notes and cards, then swaps in a different local model as judge and scores them with DeepEval metrics such as faithfulness and answer relevancy. A separate judge benchmark measures how often the judge itself is right.
Three layers: .gitignore excludes capture output and recordings; a content guard scans every commit for blocked names stored as hashes, forbidden file types, secrets and personal data; and plain git push is disabled in favour of a publish script that runs the guard, which CI runs again.
Cited answers over 1,490 SEBI circulars, or an honest abstention.
Proves AI evaluation design · Retrieval (RAG) product design · Trust and compliance by design (India) · Communicating AI quality honestly
Last public activity
A zero-knowledge, offline-first password manager with passkey unlock and on-device AI.
Proves Privacy and security architecture · On-device and local AI · Technical program delivery
Last public activity
Passkey-derived vault key wrapping as an npm library and a research preprint.
Proves Privacy and security architecture · Communicating AI quality honestly
Last public activity
Search Python code in plain English, then get a one-line explanation of each function.
Proves AI evaluation design · Retrieval (RAG) product design · Communicating AI quality honestly
Last public activity