From Screenpipe Study to Lecture Lens: a private prototype becomes a product
Lecture Lens began as Screenpipe Study, a private toolkit I built in September 2026 to capture the online course I was taking, on my own Mac. In five weeks it went from a personal script set to an open-source product with a native app, an evaluation suite and an in-browser model, with a content guard keeping course material out of the public repository.
Why build a private study tool first?
The job
As a working professional in a part-time course, I attended recorded lectures and weekend live classes with little time to re-watch either. I wanted every class captured and turned into material I could trust, without uploading anything.
The bet
Build it for one user, on one machine, with real classes, and let evidence decide what generalises. Deterministic work (segmenting, de-duplicating, file layout) stays in code; the model is used only for judgement, and every output cites a timecode, slide or frame.
How did it evolve, week by week?
From the prototype's 55 commits and the public repository's history.
| Date | Milestone |
|---|---|
| 29 Sep 2026 | Private prototype "Screenpipe Study" gets its own repository: capture control, a grounded study command line and a lecture pipeline |
| 29 Sep 2026 | Engine change 1: tap one application’s own audio, so a lecture plays without room noise or other apps’ sounds |
| 29 Sep 2026 | Slide-vision decision record, chosen on a realistic stress set rather than clean slides |
| 30 Sep 2026 | Engine changes 2 and 3: pause screen capture on protected video, keep audio, and release capture immediately |
| 30 Sep 2026 | Native SwiftUI app, DeepEval suite and a judge benchmark |
| 3 Oct 2026 | First full DeepEval run of answers, notes and flashcards |
| 4 Oct 2026 | Public open-source release as Lecture Lens, with a content guard |
| 10 Oct 2026 | Live-class follower: re-transcription, slide log and Q&A threads |
| 29 Oct 2026 | Judge calibration on a blind holdout; the simpler judge prompt kept |
| 5 Nov 2026 | Lecture Lens Studio runs the model inside Chrome; fixes now flow back from the public port to the prototype |
What did the prototype prove before release?
Measured on the prototype, before any public claim was made.
Realistic frames changed the answer
- 21 of 21 checks passed on production-style captures, median 5.3 s per slide
- Cropping to the player cut image tokens by 48%
- "Thinking on" looked safer on clean slides but passed only 15 of 21 realistic ones, so it was rejected
- A guard discards any description where the image provably never reached the model, after a silent text-only load invented slide content
Study outputs, scored by a second model
- Notes: 15 of 15 cases passed (faithfulness 0.989)
- Answers: 26 of 43 passed (faithfulness 0.924, relevancy 0.826)
- Flashcards: 9 of 15 passed, the weakest output
- Median answer time 2.3 s on a local model
The judge was tested too
- On a blind holdout, the simpler judge prompt agreed with labels 80% of the time (Cohen's kappa 0.44)
- A revised prompt also scored 80% but kappa 0.29, so the simpler one stayed
- Lesson: an automated judge needs its own evaluation before its scores mean anything
What did I change in the screenpipe engine?
Three small changes on a local branch of the open-source recorder, each switched on by an environment variable so the difference from upstream stays easy to review.
| Change | Date | Size | Why |
|---|---|---|---|
| App audio tap | 29 Sep 2026 | 4 unit tests | Record one app's own sound (the browser playing the lecture), not the room or other apps |
| Configurable protected-video pages | 30 Sep 2026 | +83 lines, 3 files | Pause screen capture on listed player pages and keep audio, because screen capture blanks protected video while audio does not |
| Immediate capture release | 30 Sep 2026 | +105 lines, 2 files | Release capture as soon as the pause starts, and keep pausing while the protected page is still visible |
These changes are not yet upstream; a pull-request description is drafted. Read the builder's note on respecting protected video.
How was a private tool made safe to open-source?
Left behind
- Captured lectures, logs and personal tools
- Course, institution and platform names, replaced with generic ones
- Machine-specific paths and settings
Added
- A six-layer content guard on every commit and push
- Neutral test fixtures and documentation
- A sync path both ways: since 5 Nov 2026, fixes made in the public port flow back to the prototype
What are the product lessons?
- Start with one real user. Building for my own classes surfaced problems no spec would have: black video, room noise, judges that disagree with people.
- Test on realistic inputs. The clean slide set pointed to the wrong setting; only production-style frames showed it.
- Evaluate the evaluator. A model judge is a measuring instrument, and it needs calibration like any other.
- Design for publishing early. Generic names and a content guard made the move from private to public a port, not a rewrite.
Read the Lecture Lens case study · See its product artifacts
What do people ask about the origin story?
What was Screenpipe Study?
A private toolkit Ian Pinto built in September 2026 to capture the online course he was taking on his own Mac, using the open-source screenpipe recorder and local models to turn lectures into cited answers, notes and flashcards. Lecture Lens is its public, open-source successor.
Why not open-source the prototype directly?
It was full of private context: course and platform names, captured lectures, logs and personal tools. Lecture Lens was rebuilt from it with generic names, a content guard on every commit, and the private folders left behind.
What did the prototype prove before release?
On production-style slide captures the chosen vision setup passed 21 of 21 checks at a median 5.3 seconds, with 48% fewer image tokens after cropping. The first full DeepEval run passed 15 of 15 note cases, 26 of 43 question-answering cases and 9 of 15 flashcard cases.
Did the engine changes go upstream?
Not yet. Three small changes to the screenpipe engine (app audio tap, configurable protected-video pages, immediate capture release) live on a local branch, configured by environment variables to keep the change reviewable. A pull-request description has been drafted.