Unity Test Framework Property-Based Fuzz Hangs Editor on 1M FsCheck Iterations - How to Fix
Problem: You added FsCheck.Unity to fuzz your save serializer. The [Property] test is configured for 1,000,000 iterations. Clicking Run in the Unity Test Runner hangs the Editor for minutes—or until you force-quit—while the progress bar never finishes.
Who is affected now: Micro-studios building 2026 partner-cert evidence packets after adopting property-based save fuzzing. Reviewers ask for fuzz coverage; tutorials copy 1M iteration counts meant for headless CI, not interactive Editor sessions.
Fastest safe fix: Cap Editor-mode tests at 10,000 iterations (or fewer), run the 1M sweep only in batchmode CI, silence FsCheck verbose logging, and replay a shrunk seed corpus on every local run instead of re-exploring the full space.
Direct answer
The hang is not a “broken” FsCheck install—the main thread is doing a million allocations, GC stalls, and console writes. Unity’s Test Runner runs [Property] tests on the Editor thread unless you explicitly use a headless player or -batchmode test job. Treat 1M as a release-gate CI workload, not a developer smoke test.
Why this issue spikes in 2026
- Partner reviewers increasingly request save-fuzz evidence in Q3 2026 intake packets.
- FsCheck.Unity samples often show large
MaxTestvalues without Editor vs CI split. - Save DTOs grew heavier (inventory dictionaries, Steam Cloud sync fields) → GC every iteration hurts more.
- Teams run fuzz beside local Whisper playtest VOD triage overnight—Editor hangs block the same machine.
Pair with 15 Free Save-System Corruption Fuzz Testing Resources and Q3 cert intake mock audit.
Symptoms and search phrases
- Test Runner stuck on one test; CPU low; RAM climbing.
- Only happens above ~100k iterations.
FsCheckverbose output floods the Console.- Cancel does nothing until killing
Unity.exe. - Same test passes quickly when you lower
MaxTestto 100. - CI (batchmode) completes; Editor never does.
Root causes (check in order)
- Iteration count too high for Editor (1M on main thread).
- Per-iteration allocations in
Save()/Load()without object pooling. - FsCheck verbose logging synchronously on main thread.
- Burst/Jobs disabled in test assembly but property loop still heavy.
- Editor test running Play Mode domain reload each iteration.
- Duplicate test assemblies registering the same property twice.
Fastest safe fix path
Step 1 — Split Editor smoke vs CI sweep
| Lane | Max iterations | Where |
|---|---|---|
| Editor smoke | 10,000 (or 1,000 while iterating) | Test Runner window |
| CI release gate | 1,000,000 | -batchmode -runTests |
Example pattern:
#if UNITY_EDITOR
const int MaxIterations = 10_000;
#else
const int MaxIterations = 1_000_000;
#endif
Or drive both from an environment variable your CI sets:
var max = int.TryParse(Environment.GetEnvironmentVariable("SAVE_FUZZ_MAX"), out var n)
? n : 10_000;
Step 2 — Silence FsCheck output in Editor
FsCheckConfig.Default = FsCheckConfig.Default
.WithVerbosity(VerbosityLevel.Quiet);
Verbose shrinking traces belong in CI artifacts, not the Editor Console.
Step 3 — Preserve regression seeds
When CI finds a failure, FsCheck emits a shrunk counter-example. Commit it:
Tests/SaveFuzz/Seeds/
inventory_overflow_2026-05-24.json
Add a non-property [Test] that replays only seeds in Editor (< 1 second):
[Test]
public void ReplayKnownSaveFuzzSeeds()
{
foreach (var seed in SeedLoader.LoadAll("Tests/SaveFuzz/Seeds"))
SaveRoundTripProperty.Hold(seed);
}
Step 4 — Run 1M only headless
Unity Cloud Build or a local script:
Unity.exe -batchmode -nographics -projectPath . ^
-runTests -testPlatform editmode ^
-testResults Logs/save-fuzz-1m.xml ^
-testFilter "SaveRoundTripProperty"
Set SAVE_FUZZ_MAX=1000000 in the CI job environment.
Success check: Editor smoke finishes under 60 seconds; CI publishes save-fuzz-1m.xml + seed artifacts.
Step 5 — Reduce per-iteration cost
- Disable domain reload between tests if safe (
Enter Play Mode Options). - Reuse buffers; avoid
new byte[1024 * 1024]inside the property. - Fuzz DTO serialize/deserialize without full scene load when possible.
- Turn off Burst in the test assembly if mixing with main-thread serializers.
save_fuzz_editor_receipt_v1.json
{
"schema": "save_fuzz_editor_receipt_v1",
"editor_max_iterations": 10000,
"ci_max_iterations": 1000000,
"editor_duration_seconds": 42,
"ci_artifact": "Logs/save-fuzz-1m.xml",
"seed_corpus_count": 3,
"device_used": "n/a",
"pass": true
}
Store beside release evidence when attaching fuzz proof to partner packets.
Beginner path
- Copy the save fuzz blog’s round-trip property with 100 iterations.
- Confirm green in Test Runner.
- Raise to 10,000 only after it stays fast.
- Ask your build person to add the 1M job—do not click it locally.
Working dev path
- Same assembly, two
[Category]tags:SmokevsReleaseGate. - CI runs
-testCategory ReleaseGatenightly; PRs runSmokeonly. - Log
save_fuzz_editor_receipt_v1.jsonfrom CI; attach to BUILD_RECEIPT extensions when serialization changes.
Prevention
- Never commit
MaxTest = 1_000_000without#ifor CI env guard. - Code-review rule: property tests must declare Editor cap in the PR description.
- Pin FsCheck + FsCheck.Unity versions in
Packages/manifest.json. - Add Test Runner timeout (Unity 6 supports per-test timeout in some runners—use CI kill switch as backstop).
- Link QA README to this help + save fuzz resource list.
Troubleshooting
| Symptom | Fix |
|---|---|
| Still slow at 10k | Profile allocations; shrink generator complexity |
| CI passes, Editor fails | Different defines; sync SAVE_FUZZ_MAX |
| OOM after minutes | Lower CI iterations temporarily; fix leak in Load() |
| Hang without FsCheck | Plain [Test] infinite loop—check while(true) in serializer |
| Play Mode tests hang | Move fuzz to Edit Mode tests |
FAQ
Is 1M iterations required for cert?
Reviewers want evidence you fuzz, not a specific count. A 1M CI artifact plus seed corpus beats a hung Editor screenshot.
Can I use Unity Test Framework without FsCheck?
Yes—start with fixed fixtures; add FsCheck when round-trip properties are stable.
Does this apply to Godot GUT?
Godot has different runners; see the save fuzz blog’s GUT section. This help is Unity + FsCheck specific.
Cloud vs local Whisper triage?
Unrelated lane—see local Whisper playtest VOD triage and Whisper/ffmpeg resource list when overnight transcription stalls.
Related links
- Your First Save-System Corruption Test - Property-Based Fuzzing (2026)
- 15 Free Save-System Corruption Fuzz Testing Resources
- Q3 2026 Cert Intake Mock Audit
- Wednesday Demo Build Smoke Ritual
- FsCheck documentation
- Unity Test Framework manual
Run 10k in the Editor, 1M in CI—never the reverse.