🔍 Read the full analysis: Building With AI: The Uncomfortable Lessons Of A First Release on ThorstenMeyerAI.com
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Thorsten Meyer describes how an AI-built drawing engine produced an improved explainer video after its first full video was criticized as slow and visually empty. The account says the team used independent code review, human approval and measurable production checks; a separate legal-model experiment showed that completing more benchmark tasks did not improve its legal score.
Thorsten Meyer says a new AI-built drawing engine moved from idea to a live test video in one evening, but its first full video the next morning was slow and visually sparse. In an account updated November 9, Meyer describes how critique, code review, human approval and production checks shaped a later release, offering a specific example of the work needed to turn an AI-assisted prototype into a publishable product.
Meyer says the initial idea came from a hand-drawn explainer video whose style he liked. To avoid copying it, he says the team wrote down a limited set of general traits—such as hand-drawn strokes and a continuous visual world—and used no drawings, layouts, assets or drawing code from the original. Claude built the engine, using colors from Meyer’s own design system and an openly licensed handwriting font.
In Meyer’s workflow, OpenAI’s Codex reviews code but does not build it. He says Codex reported five findings in its first review, all fixed that evening. Each fix was paired with a test that Meyer says failed against the earlier code, a way to check whether the test could detect the original problem rather than merely pass after a change.
The first full video drew a different kind of review: Meyer found excessive white space, icons that did not support the narration and images that stopped moving when their drawing was complete. He says the team measured frame coverage, still time and camera movement across the first three scenes. In the rebuilt version, those figures changed from 15% to 66% frame coverage, 29 seconds to zero seconds standing still, and three camera moves to 27. Meyer says the measures are now build gates intended to stop scenes that are too empty or contain an image unrelated to the narration.
Infographic 01 · The new video look
One evening, one boring first video, one measured fix
Speed was not the point. The steps in between were: a second AI that reviews, a human who says yes, and numbers that became gates.
Share of the frame filled with content on the first three scenes after the fix. Before: 15%.
From an evening idea to a measured fix
Relative times, this week
- EveningA video I likedClean room: only a list of style traits, no drawings, assets or layouts
- Same eveningOwn engine, first reviewClaude builds; OpenAI’s Codex reviews; 5 findings fixed, each with a test that fails on the old code
- Same eveningA human says yesOne-minute test video approved and published
- Next morningFirst full video: slow and boringToo much white space, icons that did not fit, a picture that stood still
- Hours laterMeasured fix, then gatesThe first video in the improved look goes live
Source: my own build notes for the new video look, this week (engine history, review round, approval, critique, before/after measurement of the first three scenes).
Powered by Thorsten Meyer AI
From Prototype to Production Checks
The account’s central point is that a working demonstration did not establish that the output was ready to publish. Meyer describes separate checks for different risks: a second AI reviewed code, tests targeted reported defects, measurements gave concrete form to visual criticism, and a person responsible for the result approved the test video.
That distinction matters to readers evaluating AI-assisted production. A system can complete a task while producing weak or incorrect work, and a green test suite can miss defects if the tests do not cover them. Meyer’s examples show why teams need to define what quality means for a particular product, check it against the output, and make those checks visible to people who can act on them.
The process is presented as Meyer’s own practice, not an independently audited industry standard. Still, the reported before-and-after measures and the preserved first video give readers concrete material to assess the claims, rather than relying only on a general assertion that the tool improved.
Products Worth Considering
Bose QuietComfort Wireless Noise-Cancelling Headphones, White Smoke
LG UltraGear 27GX704A-B 27-Inch QHD OLED Gaming Monitor, 240Hz
EcoFlow DELTA 3 Max Portable Power Station, 2048Wh LFP Battery
As an affiliate, we earn on qualifying purchases.
Two Tests of AI Quality
The article connects the video release to a separate experiment conducted during the same week. Meyer says Claude edited the weights of a local model to improve its performance on legal tasks. On the 10 published Lexassay tasks, the edited model answered all 10, compared with four before the edit. The reported legal score, however, remained at zero.
The contrast reinforces the article’s distinction between task completion and task quality. A higher number of answered questions did not, on Meyer’s account, translate into a better legal score. The November 9 update says the figures refer to all 10 published tasks and that the runs are public on lexassay.com. Meyer also describes an earlier monitoring failure: in August, a guard generated about 1,300 warnings in a log nobody read, while an alert email did not go out. The account says monitoring only helps when results reach a person.
“A demo that runs is not a decision.”
— Thorsten Meyer
Limits of the Reported Results
The account is a first-person report based on Meyer’s build notes, incident notes and experiments. It does not provide an independent evaluation of the video’s quality, the reliability of its measurements or whether the new gates prevent all relevant failures. The coverage and movement figures apply to the first three scenes, not necessarily the entire video or later releases.
The legal benchmark result is also narrow: the article reports answers across 10 published tasks and a legal score of zero, but the supplied account does not explain the score’s full methodology or establish how the model performs on legal work beyond that benchmark. It is not clear whether the improved video gates have been evaluated across a larger set of productions, or whether the warning and email problems described from August have been resolved through changes beyond those discussed.
Checks for Future Releases
Meyer says the video engine now uses frame coverage and still-time checks as gates before rendering, and flags objects that do not illustrate the narration. Those checks, alongside human review, are the next stated step in the publishing process. The article does not give a date for a further release or report results from a longer-term assessment.
The Lexassay runs are public, according to the November 9 update, giving readers a way to inspect the reported benchmark results. Further evidence would be needed to judge how well the video checks perform across more scenes and projects, and whether changes to the model improve its legal score rather than only its task-completion count.
Key Questions
What was the first release?
It was an AI-built drawing engine used to make explainer videos. Meyer says a short test video went live the evening the engine was built; the first full video followed the next morning.
Why did Meyer criticize the first full video?
He says it had too much white space, icons that did not support the narration and drawings that became static when complete. He describes the result as slow and boring.
What changed in the improved video?
For the first three scenes, Meyer reports frame coverage rising from 15% to 66%, still time falling from 29 seconds to zero, and camera moves increasing from three to 27. He says some of these measures became checks that can stop a build.
Did the legal-model experiment improve the model?
It answered 10 published tasks rather than four, according to Meyer, but its legal score remained zero. The reported completion increase did not amount to an improvement in that score.
Has the process been independently verified?
The article presents Meyer’s own notes and experiments; it does not describe an independent audit of the video measurements or benchmark interpretation. The updated article says the Lexassay runs are public.
Source: ThorstenMeyerAI.com
