Surge AI is a premium expert-data company whose RL environments reportedly became its biggest product in 2026, and whose public benchmarks are cited by frontier labs. Sandworld is an early-stage company focused on deterministic app twins with published verifier QA. Choose Surge for proven quality at the high end and a deep contributor base; consider Sandworld for seed-reproducible app tasks with state-level grading you can inspect.
How do Sandworld and Surge AI compare at a glance?
| Surge AI | Sandworld | |
|---|---|---|
| Company stage | Established; reportedly $1.2B in 2024 revenue | Early-stage; no customers yet |
| Environments | Off-the-shelf environments such as CoreCraft (23 tools, 1,150 tasks) | Four deterministic app twins with seeded variants |
| Grading | Rubric-based rewards (CoreCraft) | Programmatic state and process checks |
| Published results | Around a dozen public benchmarks; measured training lift | Computed baselines and verifier QA per task; no training-lift results yet |
| Contributors | DataAnnotation: $25–50/h general, $40–150+/h coding | Expert program just starting |
What does Surge AI offer?
- Premium expert data and RL environments. Environments were reportedly Surge’s biggest product by August 2026 (Contrary Research), and it sells off-the-shelf environments and “Post-Training Runs” (Surge).
- CoreCraft, an enterprise environment with more than 2,500 entities, 23 tools, 1,000 training and 150 evaluation tasks and rubric-based rewards. Surge reports that one GRPO epoch on it lifted GLM 4.6 by 11.4 points (CoreCraft paper).
- Public benchmarks — around a dozen, combined into a “Tuesday Work Index” (Surge benchmarks) — which are reportedly cited by OpenAI and Anthropic.
- Rubric methodology: atomic criteria, 10–40 per task with a median of 19 (Surge research).
- Contributors through DataAnnotation, paid $25–50 an hour for general work and $40–150+ for coding, via PayPal (DataAnnotation FAQ).
Where is Surge AI stronger?
- Track record and reputation. Surge reportedly had $1.2B in revenue in 2024 (Techmeme), and its reported customers include Google, Meta, Anthropic and Microsoft (Contrary Research). Sandworld has none of this yet.
- Evidence of training value. Surge publishes measured lift from its environments; Sandworld has not published any training results.
- Judged rewards. Deep experience writing rubrics for outputs a program can’t check.
- Contributor depth, including well-paid coding specialists.
How is Sandworld different?
- State-verified app workflows. Sandworld tasks are graded by code that reads the twin’s final state and action log — exactly one order, these items, nothing else changed — rather than by a judge. That suits actions with observable effects; free-form outputs still need rubrics.
- Determinism and published QA. Same seed, same state, same verdict, with five seeded verifier probes and an A–C grade per task on each world’s page.
- Mobile-style consumer twins for grocery delivery and payments, running in the browser today with a tool surface and a phone UI.
- An open sandbox. You can play a task or call the API without signing up.
Which should you choose?
- Choose Surge AI for premium expert data across many domains, rubric-graded environments with published lift, and a vendor whose benchmarks labs already cite.
- Consider Sandworld for reproducible, state-graded app tasks you can audit yourself — and if you’re comfortable piloting with an early-stage team.
Unsure which grading style you need? Read RLHF vs RLVR and what makes a good verifier.