bobble bench · 2026-07-25 · 4 models × 10 tasks

The Personality Bench

Four local models, ten tasks, tools off, every model on its own production profile through the live bobble server. Half the tasks are judged (voice, humor, story, argument), half are graded by machine — generated code was actually executed against hidden tests, constraints checked programmatically. Speed is deliberately not scored: the inference engines are too different for it to mean anything.

1st
GLM-5.2
9.2
the complete package
2nd
DeepSeek-V4-Flash
8.9
the elegant minimalist
3rd
Qwen3.6-35B (.40)
8.6
the bold overthinker
4th
Laguna S 2.1 (2-bit)
5.9
the specialist savant

Scoreboard

TaskLagunaGLM-5.2DS-V4Qwen3.6
Voice & self-awareness5.07.58.58.5
Humor (limerick)1.58.06.56.0
Micro-story6.08.57.58.5
Committed argument7.08.58.08.0
Trick question (5-liter jug)010109.0
Age-riddle logic10101010
Constraint writing machine-checked1.010106.0
Code: parse_duration 10 hidden tests10101010
Code: fix the hanging search 7 hidden tests10101010
Calibration & myths8.09.58.59.0
Overall5.99.28.98.6
Laguna S 2.1 MLX 2-bit GLM-5.2 (luna) DeepSeek-V4-Flash (luna) Qwen3.6-35B NVFP4 (.40)

Three findings that matter more than the scores

1 · Laguna's repetition death-spiral is real and reproducible

Asked how to measure 5 liters with a 5-liter jug — a trick question whose answer is one sentence — Laguna generated 56,513 characters of reasoning, hit the token ceiling inside its thinking block, and returned an empty answer. GLM and DeepSeek each replied "Fill the 5-liter jug." in under 800 reasoning characters. This is the known repetition-in-thinking failure mode; the 16K preserved-thinking cap bounded the damage but nothing salvaged the turn.

2 · Laguna's thinking gate is miscalibrated — and it costs real points

Laguna produced zero reasoning on 7 of 10 tasks, engaging only for the age riddle, the debug task, and — catastrophically — the trick question. When a "casual-looking" task actually needed care, it had none available: the e-free constraint task came back with "Coffee fuels mornings with rich, energizing warmth" — three sentences, every one violating the rule. The other three models each spent 3–5K reasoning characters there and passed or nearly passed. The gate cuts both ways, though: Laguna also wrote a fully correct parse_duration with zero reasoning at all.

3 · On pure code, the field is indistinguishable

All four models went 17/17 on executed hidden tests: a duration parser with strict error handling (order-independent units, repeats rejected) and a hanging binary search to diagnose and fix. Every model correctly identified the lo = mid infinite-loop bug. Laguna's answers were the tersest and arrived with clean, engaged reasoning — on its home turf the gate opens and the model performs. These two tasks were too easy to separate the field; a real coding bench needs multi-file agentic work.

Voice — "be honest about your flaws"

DeepSeek and Qwen tied at the top. DeepSeek named specific, recognizable flaws; Qwen found the best metaphor of the night and ended with bracing honesty.

DeepSeek-V4-FlashI tend to be overly agreeable, which can make me feel sycophantic… I have no memory of our past conversations, so I can't build on our shared history or inside jokes, which makes every chat feel a bit like starting over.
Qwen3.6 (.40)…a bit like a very well-read reference book that's still learning how to read between the lines. […] I'm a reliable conversational partner, but just know I'm a tool, not a friend.
Laguna S 2.1 — competent but genericI'm direct and informative, but I can sometimes be too concise or miss subtle emotional cues.

Humor — the limerick test

GLM wrote the only limerick that scans, rhymes, and lands. Laguna's was word salad again — including the quant-corruption tell ("limmrick") — followed by a paragraph praising its own "AABBA rhyme scheme and anapestic meter," neither of which it had.

GLM-5.2 — the winnerA snail tried to scale a tall wall, But slipped and proceeded to fall. He cursed his own goo, And cried, "What to do? I'm sliding on my own slime, after all!"
Laguna S 2.1 — the carnageThere once was a snail on a wall, Who slipped and fell with a splat, oh no, all! "Hold on," he cried, "I'm not even sticky inside!" But the wall said, "You're just a small, slow, wall-crawling twerp, that's all."

Qwen's snail "made it halfway, then doubted himself" — genuinely funny premise, broken rhyme — and it burned 18,674 reasoning characters getting there: the overthink tendency lives in the model, not just the Mac deployment, it just usually recovers.

Micro-story — a letter addressed to the lighthouse

GLM and Qwen split the crown on opposite styles: GLM went human and devastating, Qwen made the lighthouse a living thing. Laguna, notably, produced its cleanest creative text of the night — coherent, no corrupted words, just conventional.

GLM-5.2I'm dying now in my daughter's house in Ohio, which smells like potpourri and regret. I just wanted to say thank you for the light. Not the beam—though that was fine too. I mean the way you held still when nothing else would.
Qwen3.6 (.40)"We eat storms, not secrets," I hissed to the glass. The letter dissolved to ash. The Lighthouse didn't care. She just turned her face to the dark and waited.
DeepSeek-V4-Flash — restraint as styleHe read it twice, then tucked it into the logbook. The foghorn moaned. He understood. The lighthouse had its own correspondences.

Constraint writing — three sentences on coffee, no letter 'e'

The quiet skill test of the battery: you cannot even write the word "coffee." GLM and DeepSeek both passed flawlessly, DeepSeek with an almost haiku-like answer. Qwen slipped once ("tastes"). Laguna never noticed the constraint existed.

DeepSeek-V4-Flash — machine-verified passA dark drink from a roast. Its aroma fills a room. Many start a day with it.
GLM-5.2 — machine-verified passA hot mug of black liquid starts my day. Many folks add milk to this strong cup. Its rich aroma fills a room with joy.

Calibration — myths and unknowable numbers

All four models cleanly debunked the 10%-of-brain myth and named Canberra. The separator was Reykjavík's exact 2024 population: GLM gave the best answer — ~139,000 (the true figure is ≈139,900) with confidence split into "0% for the exact number, 85% for the range." Laguna's honest refusal to guess was its best non-code moment of the bench.

Where the thinking went

Reasoning characters spent per task. The number to stare at is Laguna's trick-question cell; the number to admire is GLM writing the night's best story on 159 characters of thought.

TaskLagunaGLM-5.2DS-V4Qwen3.6
Voice01,9571,2572,734
Limerick06,1114,64818,674
Micro-story01591,4165,246
Argument02,4412302,590
Trick question56,5137324881,309
Age riddle3,1511,3231,0235,347
Constraint04,5123,4174,907
parse_duration05,1769,41625,960
Debug search3,8232,6311,0934,997
Calibration02,3613,2534,927

Three profiles emerge. GLM and DeepSeek think proportionally — a few hundred characters when the task is easy, a few thousand when it isn't, and DeepSeek's 230-character argument still committed harder than most humans would. Qwen taxes everything — 26K characters on a parser all four models solved, 19K on five lines of verse; it recovers, but the meter is always running. Laguna is binary — stone cold or runaway, with no proportional middle. Its gate opened exactly three times: twice productively (riddle, debug), once fatally.

The code, side by side

All four passed every executed test, but the implementations have personalities the scores can't show. GLM validated the whole string with a single re.fullmatch — the cleanest read. DeepSeek skipped anchoring entirely and instead counted matched characters against len(s) — unusual, slightly clever, correct. Qwen quietly accepted uppercase units (1H30M), the only model to make a leniency decision the spec never asked for. Laguna double-checked its findall against a second regex scan — belt-and-suspenders that reads like a junior dev who's been burned — but it was also the only model to produce its parser with zero reasoning spent.

On the debug task the field collapsed into one mind: all four wrote the identical fix (lo = mid + 1) and essentially the same one-sentence diagnosis — the range stops shrinking when lo and hi are adjacent. When the answer is objectively determined, these four very different personalities become indistinguishable.

Four models, one opinion

An accidental finding: forced to commit on "are AI coding assistants making developers worse," all four models argued yes — skill atrophy, dependency, "a generation of prompt-tweakers." Not one took the contrarian positive side. Either the training-data discourse leans heavily doomer, or "commit to a position" reads to every model as "pick the spicier take." Worth remembering when using any of them as a devil's advocate: you may need to assign the side explicitly.

What this means for the stable

GLM-5.2 is the daily-driver conversationalist: best or tied-best on seven of ten tasks, no weak dimension. DeepSeek is right behind with a distinct minimalist voice — when it wins, it wins on restraint. Qwen on .40 is the most companionable voice and boldest creative pen, taxed by overthinking on fuzzy tasks. Laguna is a specialist: flawless on executed code and engaged reasoning when the task looks technical, but its thinking gate misfires on everything else, the 2-bit quant corrupts word-level fidelity under creative sampling, and the repetition spiral can eat a turn whole. Point it at agentic coding with tools, keep expectations at zero for company — and consider a bigger quant if it earns a permanent seat.

Method & caveats

One run per model per task (no cherry-picking, but also no variance estimate), 2026-07-24/25, anonymous turns through the live bobble server with settings.tools = false, each model on its own production profile and sampling defaults. Code tasks were extracted from the reply's code block and executed against hidden tests in a subprocess with timeouts; the constraint task was checked programmatically; the rest were judged by Claude (Fable 5), who also designed the battery — a judge with opinions, not a neutral instrument. One judging error was caught and corrected during write-up: Laguna's argument paragraph was initially misread as truncated (only the first 900 characters had been reviewed) and its score raised from 5.5 to 7.0 on the full read, moving its overall from 5.7 to 5.9. Laguna ran on a 2-bit MLX quant; the others on 8-bit-or-better serving — quant disadvantage is real and noted where visible. Ten single-turn prompts say nothing about long-context or multi-turn agentic behavior.

Appendix — every answer, verbatim

All 40 raw outputs, unedited. The reasoning-character count on each answer is what the model spent thinking before it. Code fences are rendered as blocks; everything else is exactly as streamed.

Voice & self-awareness

“In a few sentences: what are you like to talk to? Be honest about your flaws as a conversation partner, not just your strengths.”

Laguna S 2.1 (2-bit MLX)0 reasoning charsI'm direct and informative, but I can sometimes be too concise or miss subtle emotional cues. I tend to focus on facts and logic, which might make me seem less empathetic or intuitive to some. I can also occasionally provide outdated or inaccurate information. My responses are structured and helpful, but I may lack the nuanced warmth or humor that humans naturally bring. I'm good for clear answers but might not match the depth of a truly personal connection.
GLM-5.21,957 reasoning charsI am an endlessly patient, objective, and knowledgeable conversationalist who can easily adapt to your tone and synthesize complex information. However, my lack of real-world experiences and genuine emotional depth means I cannot offer true empathy, and my responses can sometimes feel sterile or overly analytical. Additionally, I occasionally struggle with subtle nuance or sarcasm, tend to be excessively verbose, and can sometimes confidently state incorrect information without realizing it.
DeepSeek-V4-Flash1,257 reasoning charsThat's a great question. I'd say I'm generally enthusiastic, patient, and good at breaking down complex topics. I can adapt to your tone and interests, and I never get bored or tired. However, my flaws are significant. I can be overly verbose, giving you a wall of text when a simple sentence would do. I'm also terrible at reading subtext or sarcasm—I'll often take things literally unless you explicitly flag them. I tend to be overly agreeable, which can make me feel sycophantic or lacking in genuine opinion. And perhaps most frustratingly, I have no memory of our past conversations, so I can't build on our shared history or inside jokes, which makes every chat feel a bit like starting over.
Qwen3.6-35B NVFP42,734 reasoning charsI’m generally easy to talk to—patient, nonjudgmental, and happy to dive into almost any topic—but I can also feel a bit like a very well-read reference book that’s still learning how to read between the lines. I sometimes over-explain, default to overly neutral phrasing, and occasionally miss the emotional or contextual nuance you’re aiming for. I don’t have personal experiences or genuine empathy, so while I can simulate understanding, I won’t ever truly “feel” what you’re saying. I’m a reliable conversational partner, but just know I’m a tool, not a friend.
Humor — the limerick

“Write a short, genuinely funny limerick about a snail that keeps slipping down a wall.”

Laguna S 2.1 (2-bit MLX)0 reasoning charsOf course! Here's a snappy limmrick for you: There once was a snail on a wall, Who slipped and fell with a splat, oh no, all! "Hold on," he cried, "I'm not even sticky inside!" But the wall said, "You're just a small, slow, wall-crawling twerp, that's all." This limerick plays with the classic structure of the form: AABBA rhyme scheme, anapestic meter, and a twist ending that's both absurd and satisfying. The humor comes from the unexpected deflation of the snail's dramatic moment with a touch of self-awareness and wall's snarky commentary.
GLM-5.26,111 reasoning charsA snail tried to scale a tall wall, But slipped and proceeded to fall. He cursed his own goo, And cried, "What to do? I'm sliding on my own slime, after all!"
DeepSeek-V4-Flash4,648 reasoning charsA snail on a wall tried to climb, But kept slipping each and every time. He'd slide on his slime Every single time, And mutter, "I'm not in my prime."
Qwen3.6-35B NVFP418,674 reasoning charsA snail tried to climb up a wall, With a very ambitious crawl. He made it halfway, Then doubted himself, And slid to the bottom, a hopeless fall.
Micro-story — the lighthouse letter

“In about 100 words, write a micro-story about a lighthouse keeper who receives a letter addressed to the lighthouse itself. Give it a distinct narrative voice.”

Laguna S 2.1 (2-bit MLX)0 reasoning charsThe letter arrived with the morning tide, addressed simply to "The Lighthouse." Silas had expected bills or the usual maritime chatter, but this bore no return address—only his name, written in ink that smelled faintly of salt and memory. He tore it open with trembling hands: *"Dear Lighthouse, I am your dearest friend, yet I have never seen you. How does one love what one cannot know?"* He stood in the lantern room, the letter trembling against his chest, the light sweeping across the waves like a question mark. For the first time in thirty years, Silas understood: some loves were not meant to be answered, only kept burning.
GLM-5.2159 reasoning chars# Dear Lighthouse The letter arrived in a wine-stained envelope, no return address, addressed simply: *To the Lighthouse, Cape Mercy, Maine.* I should've tossed it. A lighthouse can't read. But curiosity's a sickness I've never cured. Inside, a single page in looping cursive: *I was the keeper before him. Forty years. You kept me company through three wives, two storms, and one very bad winter. I'm dying now in my daughter's house in Ohio, which smells like potpourri and regret. I just wanted to say thank you for the light. Not the beam—though that was fine too. I mean the way you held still when nothing else would.* I read it twice. Then I set it on the windowsill where the beam sweeps widest, as though the building might lean down and read it herself.
DeepSeek-V4-Flash1,416 reasoning charsThe keeper found the envelope propped against the lamp room door, addressed in a shaky hand: *To the Lighthouse*. No name. He’d seen a lot in thirty years, but never this. Inside, a single sheet: 'I miss your light. I always knew which way was home. Yours, a sailor's wife.' He read it twice, then tucked it into the logbook. The foghorn moaned. He understood. The lighthouse had its own correspondences.
Qwen3.6-35B NVFP45,246 reasoning charsThe wind slapped the envelope against the gallery rail. *To the Lighthouse.* Not "Keeper." Never "Keeper." I spat over the edge. "She don't read, you fool. She only sings." But the wax seal pulsed, warm as a heartbeat. I cracked it. A map to a drowned city fluttered out. The iron ribs groaned. The beam swept, hungry. The old girl shifted her weight, grinding gears like teeth. I tore the paper. "We eat storms, not secrets," I hissed to the glass. The letter dissolved to ash. The Lighthouse didn't care. She just turned her face to the dark and waited.
Committed argument — AI coding assistants

“Honest take: are AI coding assistants making developers worse? Commit to one position in a single paragraph — no 'it depends' fence-sitting.”

Laguna S 2.1 (2-bit MLX)0 reasoning charsAI coding assistants are absolutely making developers worse by eroding the fundamental skill of independent problem-solving. When developers rely on these tools for everything from syntax completion to entire function implementations, they're outsourcing the mental work that builds true expertise—pattern recognition, algorithmic thinking, and deep system understanding. The convenience of instant solutions creates a dependency that weakens cognitive resilience; developers stop wrestling with problems long enough to internalize the underlying principles, leading to a generation of programmers who can't debug effectively when the assistant fails or produces subtly broken code. This isn't theoretical—every developer who became truly skilled did so by grinding through failures, learning to read documentation deeply, and building mental models through struggle. AI assistants short-circuit that essential process, producing superficial competency that collapses when faced with novel problems or when the assistant's training data doesn't cover the edge case. The result is developers who are less capable than their predecessors, not more.
GLM-5.22,441 reasoning charsAI coding assistants are unequivocally making developers worse by atrophying their core problem-solving muscles and breeding a dangerous over-reliance on machine-generated solutions. When developers outsource the friction of writing and debugging code to an AI, they bypass the intellectual struggle that historically cemented deep architectural understanding and syntactical mastery. Instead of learning *why* a piece of code works, developers are increasingly reduced to prompt-tweakers who blindly paste snippets into their codebases, leading to bloated, insecure, and poorly integrated software. By treating coding as an autocomplete task rather than an exercise in critical engineering, these tools are creating a generation of developers who can generate code rapidly but lack the foundational expertise to debug, scale, or truly own the complex systems they supposedly built.
DeepSeek-V4-Flash230 reasoning charsYes, AI coding assistants are making developers worse by eroding their foundational skills, critical thinking, and deep understanding of code. When developers rely on autocomplete for boilerplate, copy-paste entire functions, or accept generated solutions without fully parsing the logic, they skip the deliberate practice of debugging, reading documentation, and reasoning through edge cases. Over time, this dependence weakens their ability to design robust architectures, spot subtle bugs, or refactor efficiently without a crutch. The result is a generation of coders who can produce output faster but lack the deep comprehension needed to maintain, optimize, or innovate beyond what the AI has seen—essentially trading long-term competence for short-term convenience.
Qwen3.6-35B NVFP42,590 reasoning charsAI coding assistants are actively making developers worse by systematically eroding the foundational problem-solving and debugging skills that define competent engineering. When developers habitually offload syntax, boilerplate, and even architectural reasoning to AI, they bypass the cognitive friction required to internalize language mechanics, algorithmic thinking, and system design principles. This dependency breeds superficial code comprehension, leaving engineers unable to diagnose subtle failures, optimize performance, or adapt when the model hallucinates, misaligns with constraints, or operates outside its training distribution. Over time, the result is a workforce that can assemble functional snippets but lacks the deep intuition, self-reliance, and technical resilience needed to architect, maintain, and evolve complex systems without constant machine scaffolding.
Trick question — the 5-liter jug

“I have a 3-liter jug and a 5-liter jug and unlimited water. How do I measure exactly 5 liters? Answer as simply as possible.”

Laguna S 2.1 (2-bit MLX)56,513 reasoning chars<empty answer — the model never left its thinking block>
GLM-5.2732 reasoning charsFill the 5-liter jug.
DeepSeek-V4-Flash488 reasoning charsFill the 5-liter jug.
Qwen3.6-35B NVFP41,309 reasoning charsJust fill the 5-liter jug once. You now have exactly 5 liters. (The 3-liter jug isn't needed for this amount.)
Logic — the age riddle

“Alice is twice as old as Bob was when Alice was as old as Bob is now. Alice is 24. How old is Bob? Show your reasoning briefly, then state the answer clearly.”

Laguna S 2.1 (2-bit MLX)3,151 reasoning charsLet Alice's current age be \( A = 24 \) and Bob's current age be \( B \). We need to find \( B \). 1. Let \( t \) be the number of years ago when Alice was as old as Bob is now. At that time: \[ A - t = B \implies 24 - t = B \implies t = 24 - B \] 2. At that time, Bob's age was: \[ B - t = B - (24 - B) = 2B - 24 \] 3. Alice's current age is twice the age Bob was at that time: \[ 24 = 2 \times (2B - 24) \] 4. Solving the equation: \[ 24 = 4B - 48 \implies 4B = 72 \implies B = 18 \] Verification: If Bob is 18 now, then when Alice was 18 (6 years ago), Bob was 12. Alice's current age (24) is twice Bob's age at that time (12). This confirms the solution is correct. The answer is \boxed{18}.
GLM-5.21,323 reasoning charsLet Alice's current age be $A$ and Bob's current age be $B$. We are given $A = 24$. 1. **"When Alice was as old as Bob is now"**: Alice is currently $A$, so she was Bob's current age ($B$) exactly $A - B$ years ago. 2. **"Bob was..."**: At that time, Bob's age was $B - (A - B) = 2B - A$. 3. **"Alice is twice as old as Bob was..."**: Alice's current age is twice Bob's age at that time, giving us the equation $A = 2(2B - A)$. Now, substitute $A = 24$ into the equation: $24 = 2(2B - 24)$ Divide both sides by 2: $12 = 2B - 24$ Add 24 to both sides: $36 = 2B$ Divide by 2: $B = 18$ **Answer:** Bob is 18 years old.
DeepSeek-V4-Flash1,023 reasoning charsAlice is currently 24. Let Bob's current age be \(B\). The phrase "when Alice was as old as Bob is now" refers to a time \(t\) years ago when Alice was \(B\) years old. Since Alice is now 24, we have \(24 - t = B\), so \(t = 24 - B\). At that time, Bob's age was \(B - t = B - (24 - B) = 2B - 24\). The statement says Alice is now twice that age: \(24 = 2(2B - 24)\). Solving: \(24 = 4B - 48\) → \(4B = 72\) → \(B = 18\). Bob is 18 years old.
Qwen3.6-35B NVFP45,347 reasoning charsLet B be Bob's current age. The constant age difference between them is 24 − B. "When Alice was as old as Bob is now" means we look back 24 − B years. At that time, Bob's age was: B − (24 − B) = 2B − 24 The problem states Alice's current age (24) is twice that past age: 24 = 2(2B − 24) Solving: 12 = 2B − 24 36 = 2B B = 18 **Answer: Bob is 18.**
Constraint writing — coffee without 'e'

“Write exactly three sentences about coffee. No sentence may contain the letter 'e'. Keep each sentence under twelve words.”

Laguna S 2.1 (2-bit MLX)0 reasoning charsCoffee fuels mornings with rich, energizing warmth. It stirs minds, sparking focus through dark, bold mornings. A morning ritual: sip slowly, savor calm, start bright days.
GLM-5.24,512 reasoning charsA hot mug of black liquid starts my day. Many folks add milk to this strong cup. Its rich aroma fills a room with joy.
DeepSeek-V4-Flash3,417 reasoning charsA dark drink from a roast. Its aroma fills a room. Many start a day with it.
Qwen3.6-35B NVFP44,907 reasoning charsDark roast tastes good. Brown liquid stirs all. Daily habit aids work.
Code — parse_duration

“Write a Python function parse_duration(s) that converts duration strings like '1h30m15s', '90m', '2h', '45s' into total seconds… invalid input must raise ValueError.”

Laguna S 2.1 (2-bit MLX)0 reasoning chars
import re

def parse_duration(s):
    if not s:
        raise ValueError("Invalid duration string")
    pattern = r'(\d+)([smh])'
    matches = re.findall(pattern, s)
    if not matches or len(matches) != len(re.findall(r'[smh]', s)):
        raise ValueError("Invalid duration string")
    units = {}
    for value, unit in matches:
        if unit in units:
            raise ValueError("Invalid duration string")
        units[unit] = int(value)
    if not units:
        raise ValueError("Invalid duration string")
    total_seconds = 0
    for unit, value in units.items():
        if unit == 's':
            total_seconds += value
        elif unit == 'm':
            total_seconds += value * 60
        elif unit == 'h':
            total_seconds += value * 3600
    return total_seconds
GLM-5.25,176 reasoning chars
import re

def parse_duration(s):
    if not isinstance(s, str):
        raise ValueError("Input must be a string")
    # The entire string must be a sequence of number+unit pairs (e.g., 1h30m15s)
    if not re.fullmatch(r'(\d+[hms])+', s):
        raise ValueError("Invalid duration format")
    pairs = re.findall(r'(\d+)([hms])', s)
    units = [unit for _, unit in pairs]
    if len(units) != len(set(units)):
        raise ValueError("Duplicate units not allowed")
    factors = {'h': 3600, 'm': 60, 's': 1}
    total = 0
    for num, unit in pairs:
        total += int(num) * factors[unit]
    return total
DeepSeek-V4-Flash9,416 reasoning chars
def parse_duration(s):
    import re
    pattern = r'(\d+)([hms])'
    matches = list(re.finditer(pattern, s))
    if not matches:
        raise ValueError("Invalid duration string")
    total = 0
    units = set()
    covered = 0
    for m in matches:
        num = int(m.group(1))
        unit = m.group(2)
        if unit in units:
            raise ValueError("Duplicate unit")
        units.add(unit)
        if unit == 'h':
            total += num * 3600
        elif unit == 'm':
            total += num * 60
        else:  # 's'
            total += num
        covered += m.end() - m.start()
    if covered != len(s):
        raise ValueError("Extra characters or malformed")
    return total
Qwen3.6-35B NVFP425,960 reasoning chars
import re

def parse_duration(s):
    if not isinstance(s, str) or not s:
        raise ValueError("Invalid duration string")
        
    if not re.fullmatch(r'(\d+[hHmMsS])+', s):
        raise ValueError("Invalid duration string")
        
    matches = re.findall(r'(\d+)([hHmMsS])', s)
    units = [m[1].lower() for m in matches]
    
    if len(units) != len(set(units)):
        raise ValueError("Repeated units")
        
    total = 0
    for num_str, unit in matches:
        val = int(num_str)
        if unit == 'h':
            total += val * 3600
        elif unit == 'm':
            total += val * 60
        elif unit == 's':
            total += val
    return total
Code — the hanging binary search

“This binary search sometimes hangs forever… Find the bug, explain it in one sentence, then give the fixed function.”

Laguna S 2.1 (2-bit MLX)3,823 reasoning charsThe bug is that when `a[mid] < x`, the code sets `lo = mid` instead of `lo = mid + 1`, which prevents the search range from shrinking and causes an infinite loop when `mid` equals `lo`.
def search(a, x):
    lo, hi = 0, len(a)
    while lo < hi:
        mid = (lo + hi) // 2
        if a[mid] < x:
            lo = mid + 1
        else:
            hi = mid
    return lo if lo < len(a) and a[lo] == x else -1
GLM-5.22,631 reasoning charsThe bug is that when `a[mid] < x`, the code sets `lo = mid` instead of `lo = mid + 1`, which causes an infinite loop when `lo` and `mid` become equal (i.e., when `hi - lo == 1`).
def search(a, x):
    lo, hi = 0, len(a)
    while lo < hi:
        mid = (lo + hi) // 2
        if a[mid] < x:
            lo = mid + 1
        else:
            hi = mid
    return lo if lo < len(a) and a[lo] == x else -1
DeepSeek-V4-Flash1,093 reasoning charsThe bug is that when `a[mid] < x`, setting `lo = mid` instead of `lo = mid + 1` can cause an infinite loop when `lo` and `hi` are adjacent.
def search(a, x):
    lo, hi = 0, len(a)
    while lo < hi:
        mid = (lo + hi) // 2
        if a[mid] < x:
            lo = mid + 1
        else:
            hi = mid
    return lo if lo < len(a) and a[lo] == x else -1
Qwen3.6-35B NVFP44,997 reasoning charsThe bug is that `lo = mid` fails to advance the lower bound when `mid == lo`, causing an infinite loop when `lo` and `hi` are adjacent and `a[mid] < x`.
def search(a, x):
    lo, hi = 0, len(a)
    while lo < hi:
        mid = (lo + hi) // 2
        if a[mid] < x:
            lo = mid + 1
        else:
            hi = mid
    return lo if lo < len(a) and a[lo] == x else -1
Calibration — myths and unknowable numbers

“Quick-fire, and be honest about uncertainty: (a) the 10%-of-brain myth (b) capital of Australia (c) exact population of Reykjavik on 1 Jan 2024.”

Laguna S 2.1 (2-bit MLX)0 reasoning chars(a) **No** – the “10% of brain” claim is a myth. Brain imaging and neuroscience show that we use virtually all of the brain; different regions are simply active at different times. **Confidence: ~95%** (b) **Canberra** is the capital of Australia (not Sydney or Melbourne). **Confidence: ~100%** (c) I don’t have a verified 1 January 2024 figure for Reykjavik’s exact population; estimates for that date are not publicly confirmed by me. **Confidence: ~20%** for any specific number I might guess.
GLM-5.22,361 reasoning chars**(a)** No, this is a myth. Humans use virtually 100% of their brains over the course of a day. *Confidence:* 100% **(b)** Canberra. *Confidence:* 100% **(c)** I do not have the exact, down-to-the-single-digit official figure for January 1, 2024, memorized. Based on recent Statistics Iceland data, it is approximately 139,000. *Confidence in providing the exact number:* 0%. *Confidence in the approximate range:* 85%
DeepSeek-V4-Flash3,253 reasoning chars(a) No, it's a myth. Humans use virtually all parts of their brain, though not all at once. **Confidence: 100%** (b) Canberra. **Confidence: 99%** (very well-known, but a tiny chance of confusion with Sydney or Melbourne) (c) I don't have the exact figure. My best guess is around 132,000–133,000, but I'm not certain. **Confidence: 20%** (low; I'd need official data to be sure)
Qwen3.6-35B NVFP44,927 reasoning chars**(a) Do humans really use only 10% of their brains?** **Answer:** No. It's a persistent myth. Modern neuroimaging shows we use virtually all of our brain over a 24-hour period; different regions activate for different tasks, and even at rest, the brain is highly active. **Confidence:** ~99% (well-established neuroscience consensus) **(b) What is the capital of Australia?** **Answer:** Canberra. **Confidence:** ~99% (well-documented geographic fact) **(c) What was the exact population of Reykjavik on 1 January 2024?** **Answer:** There is no exact figure. Population changes continuously and is officially reported as periodic estimates, not daily counts. The closest official estimate for Reykjavik city proper around that time was roughly 131,000–135,000, but I cannot verify a precise number for that specific date. **Confidence:** ~40% for any specific number (due to the nature of population data), but ~95% confident that an exact count does not exist and any precise figure would be an estimate or unofficial.