Kodexa Luau Benchmarks

Comparing 18 AI models across 21 Luau coding tasks on accuracy, cost, and speed.

Models 18
Top Score 99.1% Cursor Grok 4.6
Speed Leader 96.9% @ 02:04 mimo-v2.5-pro (#3)
Fastest 01:57 gpt-oss-20b
Lowest Cost $0.006 gpt-oss-20b
Note: These results are based on our proprietary benchmark suite, which is actively evolving with more difficult, real-world Luau engineering tasks and improved scoring reliability. Actual performance may vary depending on your specific codebase, prompts, and runtime environment.

Cost vs. Score

Pass score against inference cost. Bubble size indicates speed (larger is faster).

Score Breakdown

95% – 100% Score 5 Models
90% – 95% Score 3 Models
Below 90% Score 10 Models
Key takeaway: mimo-v2.5-pro achieved #3 overall (96.9%) in 02:04 for only $0.0136.

Score (%)

Pass rate across all 21 tasks

Time Taken (mm:ss)

Total run time (faster is better)

Score per $1 Spent

Accuracy points per dollar (log scale)

Leaderboard

Click any model to view details.

Rank Model Score (%) Cost (21 tasks) Time Score / $ Action

Compare Models

Side-by-side comparison between any two models.

Model A
VS
Model B

Metric Differences

Radar Profile

Cost Calculator

Estimate your monthly spend and time based on task volume.

Monthly Task Volume

Tasks / Month 2,100 tasks
210 21k 100k 210k
Presets:

Estimated Monthly Cost & Time

Suggestions

Suggest models or tasks to test next. Submissions are sent directly to Discord.

Suggest a Model

Suggest a Task

Requested Models

4

Proposed Tasks

4

About the Benchmark

How models were tested and evaluated.

Overview & Disclaimer

Luau is a typed scripting language derived from Lua 5.1, widely used in game engines and simulation systems. Most LLMs struggle with Luau because of its strict type checker, custom metamethods, and memory buffer operations.

Disclaimer: This is based on our own benchmark suite, which is continually improving and becoming more reliable with additional, more difficult, and real-world Luau tasks. Real-world usage and practical performance may differ depending on your codebase, prompt structure, and execution environment.
Strict Typing: Luau type packs, singleton types, and non-nil narrowing.
Metatables: Metamethod protocols (__iter, __len, __mode).
Memory Buffers: SIMD vector operations and direct buffer manipulations.

Test Setup

All models were evaluated using the same prompt template at temperature 0.0:

Environment Luau 0.6x Strict Mode
Suite Size 21 Standardized Tasks
Grading Type Check + Unit Tests
Metrics Cost ($), Score (%), Time (s)
Score 99.10%
Cost (21 tasks) $0.27
Time 03:11
Score / $ 367

Summary
Notification