AINPM.in / Work / Lecture Lens / Origin story
Case study · Origin story · Updated 11 Nov 2026

From Screenpipe Study to Lecture Lens: a private prototype becomes a product

Lecture Lens began as Screenpipe Study, a private toolkit I built in September 2026 to capture the online course I was taking, on my own Mac. In five weeks it went from a personal script set to an open-source product with a native app, an evaluation suite and an in-browser model, with a content guard keeping course material out of the public repository.

01 · Why

Why build a private study tool first?

The job

As a working professional in a part-time course, I attended recorded lectures and weekend live classes with little time to re-watch either. I wanted every class captured and turned into material I could trust, without uploading anything.

The bet

Build it for one user, on one machine, with real classes, and let evidence decide what generalises. Deterministic work (segmenting, de-duplicating, file layout) stays in code; the model is used only for judgement, and every output cites a timecode, slide or frame.

02 · Timeline

How did it evolve, week by week?

From the prototype's 55 commits and the public repository's history.

DateMilestone
29 Sep 2026Private prototype "Screenpipe Study" gets its own repository: capture control, a grounded study command line and a lecture pipeline
29 Sep 2026Engine change 1: tap one application’s own audio, so a lecture plays without room noise or other apps’ sounds
29 Sep 2026Slide-vision decision record, chosen on a realistic stress set rather than clean slides
30 Sep 2026Engine changes 2 and 3: pause screen capture on protected video, keep audio, and release capture immediately
30 Sep 2026Native SwiftUI app, DeepEval suite and a judge benchmark
3 Oct 2026First full DeepEval run of answers, notes and flashcards
4 Oct 2026Public open-source release as Lecture Lens, with a content guard
10 Oct 2026Live-class follower: re-transcription, slide log and Q&A threads
29 Oct 2026Judge calibration on a blind holdout; the simpler judge prompt kept
5 Nov 2026Lecture Lens Studio runs the model inside Chrome; fixes now flow back from the public port to the prototype
03 · Evidence

What did the prototype prove before release?

Measured on the prototype, before any public claim was made.

Slide vision · 29 Sep 2026

Realistic frames changed the answer

  • 21 of 21 checks passed on production-style captures, median 5.3 s per slide
  • Cropping to the player cut image tokens by 48%
  • "Thinking on" looked safer on clean slides but passed only 15 of 21 realistic ones, so it was rejected
  • A guard discards any description where the image provably never reached the model, after a silent text-only load invented slide content
DeepEval · 3 Oct 2026

Study outputs, scored by a second model

  • Notes: 15 of 15 cases passed (faithfulness 0.989)
  • Answers: 26 of 43 passed (faithfulness 0.924, relevancy 0.826)
  • Flashcards: 9 of 15 passed, the weakest output
  • Median answer time 2.3 s on a local model
Judge calibration · 29 Oct 2026

The judge was tested too

  • On a blind holdout, the simpler judge prompt agreed with labels 80% of the time (Cohen's kappa 0.44)
  • A revised prompt also scored 80% but kappa 0.29, so the simpler one stayed
  • Lesson: an automated judge needs its own evaluation before its scores mean anything
04 · Engine

What did I change in the screenpipe engine?

Three small changes on a local branch of the open-source recorder, each switched on by an environment variable so the difference from upstream stays easy to review.

ChangeDateSizeWhy
App audio tap29 Sep 20264 unit testsRecord one app's own sound (the browser playing the lecture), not the room or other apps
Configurable protected-video pages30 Sep 2026+83 lines, 3 filesPause screen capture on listed player pages and keep audio, because screen capture blanks protected video while audio does not
Immediate capture release30 Sep 2026+105 lines, 2 filesRelease capture as soon as the pause starts, and keep pausing while the protected page is still visible

These changes are not yet upstream; a pull-request description is drafted. Read the builder's note on respecting protected video.

05 · Going public

How was a private tool made safe to open-source?

Left behind

  • Captured lectures, logs and personal tools
  • Course, institution and platform names, replaced with generic ones
  • Machine-specific paths and settings

Added

  • A six-layer content guard on every commit and push
  • Neutral test fixtures and documentation
  • A sync path both ways: since 5 Nov 2026, fixes made in the public port flow back to the prototype
06 · Lessons

What are the product lessons?

  1. Start with one real user. Building for my own classes surfaced problems no spec would have: black video, room noise, judges that disagree with people.
  2. Test on realistic inputs. The clean slide set pointed to the wrong setting; only production-style frames showed it.
  3. Evaluate the evaluator. A model judge is a measuring instrument, and it needs calibration like any other.
  4. Design for publishing early. Generic names and a content guard made the move from private to public a port, not a rewrite.

Read the Lecture Lens case study · See its product artifacts

FAQ

What do people ask about the origin story?

What was Screenpipe Study?

A private toolkit Ian Pinto built in September 2026 to capture the online course he was taking on his own Mac, using the open-source screenpipe recorder and local models to turn lectures into cited answers, notes and flashcards. Lecture Lens is its public, open-source successor.

Why not open-source the prototype directly?

It was full of private context: course and platform names, captured lectures, logs and personal tools. Lecture Lens was rebuilt from it with generic names, a content guard on every commit, and the private folders left behind.

What did the prototype prove before release?

On production-style slide captures the chosen vision setup passed 21 of 21 checks at a median 5.3 seconds, with 48% fewer image tokens after cropping. The first full DeepEval run passed 15 of 15 note cases, 26 of 43 question-answering cases and 9 of 15 flashcard cases.

Did the engine changes go upstream?

Not yet. Three small changes to the screenpipe engine (app audio tap, configurable protected-video pages, immediate capture release) live on a local branch, configured by environment variables to keep the change reviewable. A pull-request description has been drafted.