Fig. 1 — the two-sigma tutoring benchmark as reported by Bloom (1984). Modern estimates are lower; the brief is in the research queue.
Mereth Labs studies how modern AI can improve learning outcomes. We work on feedback, memory, assessment and skill verification — stated as measurable systems, evaluated against a published standard, and built into products once the evidence supports them.
Est. 2026Bengaluru6 problems filedEvidence standard v1Methods: learner modelling, retrieval scheduling, LLM evaluation, mechanism design
The thesis
The effect of AI on learning depends on the system around the model.
One-to-one mastery tutoring is the reference point for instruction. Bloom reported it at two standard deviations in 1984; modern syntheses put human tutoring between 0.3σ and 0.8σ, and trace much of the original figure to extra feedback, retesting and stricter mastery thresholds. The correction matters less than what it exposes: the inflating mechanisms are the ones software can run, per learner, at low cost.
That constraint is moving. In randomised trials published in 2024–25, a purpose-built LLM tutor outperformed active-learning classroom instruction at Harvard (0.73–1.3σ); six weeks of after-school GPT-4 tutoring in Nigeria produced ~0.3σ at about $48 per learner; and real-time LLM guidance raised the topic mastery of students taught by novice human tutors by 9 percentage points.
The same period produced an instructive negative result. Students given unstructured chatbot access during practice improved 48% on practice problems and scored 17% below control on the exam. Both outcomes came from the same class of model. The difference between them is system design, which is what this lab works on.
Claims are stated so they can be tested.Each project names its learner-state variable, observable signal, intervention policy and evaluation metric before work begins.
Products keep their instrumentation.A system the lab ships can be audited and compared against a baseline in production, not only in the study that preceded it.
Evidence
Selected results from the literature.
The results that frame the lab's work: the tutoring benchmark and its modern correction, what pre-LLM systems achieved, what changed with large language models, and the negative result that constrains design. Effect sizes from different settings and instruments are not directly comparable; they are cited for direction and cost.
+2.0σ reportedOne-to-one mastery tutoring in Bloom's two dissertation studies: 20–33 students per arm, a stricter mastery threshold for the tutored group, narrow tests on novel topics.Bloom, 1984 · von Hippel, 2024
+0.29–0.79σHuman tutoring measured across many studies: 65-study meta-analysis (0.33σ); VanLehn's synthesis (0.79σ); 89–96 RCTs (0.29–0.37σ, none reaching 2σ).Cohen et al., 1982 · VanLehn, 2011 · Nickow et al., 2024
+0.58–0.66σIntelligent tutoring systems before LLMs, across two meta-analyses of ~50 controlled studies each. Authoring cost limited deployment.VanLehn, 2011 · Kulik & Fletcher, 2016
+0.73–1.3σPurpose-built LLM tutor vs active-learning classroom instruction, with lower median time on task. RCT, N = 194.Kestin et al., Sci. Reports 2025
+4 / +9ppTopic mastery for students of human tutors given real-time LLM guidance. The larger effect is for lower-rated tutors. RCT in live tutoring.Wang et al., 2024
+0.3σAfter-school GPT-4 tutoring over six weeks in Benin City, Nigeria, at ~$48 per learner. RCT across nine public schools.World Bank, 2025
g = 0.61Retrieval practice vs restudy, meta-analysis of 217 experiments. Distributed practice shows effects of similar order.Adesope et al., 2017 · Cepeda et al., 2006
−17%Exam performance after unstructured chatbot access was removed, relative to control, following a 48% gain during assisted practice. A guardrailed tutor arm removed the harm but produced no exam gain. RCT, ~1,000 students.Bastani et al., 2024
Full citations appear in the papers as they are published.
Method
Problems are stated as control loops.
A learning problem enters the lab as a loop: observe the attempt, estimate the learner's state, choose an intervention, measure the outcome. Modern AI runs the stages that previously required an expert on the spot. The instrumentation records whether it worked.
Observe
Capture the attempt
Action, timing, error class and context are recorded, not only the final score. Most of the usable signal is in the trace.
Infer
Estimate the state
The system maintains a calibrated estimate of what a learner knows, half-knows and has got wrong, with explicit uncertainty. Language models read the attempt; the learner model holds the belief.
Act
Choose the intervention
Feedback content and timing, review scheduling, task difficulty and routing are chosen from the estimated state under a stated policy.
Validate
Measure the outcome
Learning gain per active minute, delayed retention, transfer, model calibration, and performance after AI assistance is removed. Engagement metrics are not used as success criteria.
The research queue
Six problems as systems.
Each is filed by the kind of fix it needs: engineering, when the bottleneck is measurement, scheduling or latency; system dynamics, when the bottleneck is routing, incentives or verification.
ENG Fixable by building the system: distribution, scheduling, latency.SYS Fixable by re-aligning the loop: routing, incentives, verification.
The standard
Work is graded against a published evidence standard.
Each project sits at one of four levels, and claims are made at the level they have earned. The full standard — metrics, guardrails, and what would change our conclusions — is public.
L0
Argued
A mechanism grounded in prior results, written as a falsifiable brief: state variable, signal, policy, metric, and the result that would disprove it.
L1
Instrumented
The loop runs end to end and logs the quantities named in the brief: item-level traces, latency percentiles, calibration curves.
L2
Validated
The loop outperforms a credible baseline on pre-registered outcomes with real learners, with uncertainty reported.
L3
Durable
The gain persists under delayed retention, transfer and gaming pressure, and survives removal of AI assistance.
The lab was established in 2026. The sequence below is the current operating plan.
Now · 2026
Foundations
Six problems filed as research briefs. Evidence standard v1 published. The instrument library — shared measurement code for every study — in development.
Next · 12 months
First validated loop
A retrieval-and-feedback system instrumented from first release, evaluated against a pre-registered baseline with a partner institution, targeting L2.
Then
Products from validated loops
Each loop that passes validation becomes a product surface — tutoring, scheduling, assessment, verification — and each deployment feeds the next study.
Output
Two tracks.
Research leaves the lab as running systems. Both tracks ship with the instrumentation that produced their evidence.
For learners
Consumer tools
Software that gives a learner tighter feedback, better retrieval timing and clearer mastery signals than a fixed classroom clock can provide.
Retrieval scheduled from observed recall, not generic review plans
Feedback returned while the attempt is still actionable
Progression based on mastery evidence, not seat time
For institutions
Assessment and verification infrastructure
Tools for schools, training teams and employers that need stronger evidence than grades, attendance or coarse credentials.
Assessments designed against Goodhart pressure
Portable skill records with explicit validity evidence
Routing policies that adapt without hiding their criteria
The lab is looking for collaborators.
Partner classrooms and training programmes with measurable stakes, researchers in learning science and machine learning, and institutions that assess or hire around skill. Mereth Labs is based in Bengaluru.