osu!mania 4K RL Bot

It never touches the running game: no input injection, no memory reading, no modification. An independent fan project, not affiliated with osu! or ppy Pty Ltd.

• Speed

  Learning needs a huge number of plays. Real-time gameplay would make that impractical, so the game had to be simulated faster than real time.

• Faithfulness

  The simulator has to judge presses the way the game does — hit windows, hold notes, misses — or the learned timing won’t transfer to a replay.

• Signal

  The agent only learns what the reward encourages. A badly shaped reward teaches it to mash keys rather than to keep time.

• Reliability

  Long training runs, many models and many versions need the same care as any other software: nothing overwritten, nothing silently lost.

• +

  HitJudgement weight (MAX/300 → 50) plus a bonus of up to 0.25 for precise timing

• +

  Completed hold+0.3

• −

  Miss−1

• −

  Broken hold−0.5 for releasing too early

• −

  Stray pressA small penalty for pressing with no note to hit

Observation

11 values per lane × 4 lanes, looking 1 second ahead

Action

Four binary key states per tick — up→down is a press, down→up a release

Decision tick

10 ms by default

Policy

Stable-Baselines3 PPO with an MLP policy — two 128-unit hidden layers by default

Interface

A custom Gymnasium environment: reset, step, observation and action spaces

Judging

osu!mania stable hit windows — MAX, 300, 200, 100, 50, miss — scaled by each map’s Overall Difficulty

Hold notes

Press to start, release near the tail; letting go too early breaks the hold

Clips

Training samples 20-second clips (by default) from across the map set; evaluation plays whole maps

Realism knob

Optional Gaussian timing jitter, so a perfect-timing bot still has something to learn

Recording

Every press and release can be logged with the exact timestamps the judging used — the raw material for replays

1. Beatmap
2. Parser
3. 4K simulation
4. PPO agent
5. Training
6. Model version
7. Evaluation
8. Replay generation
9. osu! stable

Maps in

Loose .osu files, .osz mapsets or .zip packs. Only 4K maps with at least 50 notes are used; everything else is listed with the reason it was skipped

Parallel games

Several simulator instances run in their own processes at once

Background worker

Training runs in a separate process, reporting live progress, losses, recent accuracy and ETA

Control

Pause, save a checkpoint now, stop & save, or cancel

Resume

Continuing restores weights, optimiser state, step counter and learning-rate schedule, and warns when a changed setting could hurt

• Active

  The version used by default for testing — you choose it.

• Latest

  The most recently created version.

• Best

  Highest validation accuracy on held-out maps, unless picked by hand.

• Protected

  Kept by any cleanup; deleting needs extra confirmation.

Saving is atomic: weights go to a temp folder, are verified, get their metadata, and only then are renamed into place. A version being trained from or tested is locked until that finishes.

Evaluation

Inference on full maps with metrics for timing, lanes, holds, misses and extra presses

Validation

Maps are deterministically held out from training so “best” means best on unseen maps

Comparison

Compare any two versions side by side from the version history

Baseline

A scripted oracle gives the ceiling for a given timing-noise level

Test suite

pytest covering versioning and lineage, safe deletion, failed saves preserving the previous version, map validation, .osr round-trips — and a real training worker process (marked slow)

From the README’s reference run on 2 CPU cores: 700k steps (about 7 minutes) took the synthetic demo maps from about 6% to about 89% simulator accuracy, still improving. Simulator accuracy is not an osu! score — osu! re-judges a replay with its own rules.

1. Record the play

   The simulator logs each key press and release with its timestamp.

2. Encode the frames

   Events become mania replay frames — time deltas and a key bitmask — compressed with LZMA.

3. Write the file

   Header, judgement counts, score, combo and the beatmap’s MD5, in the .osr binary layout.

4. Hand it to osu!

   The app finds osu! stable and opens the replay. If the map isn’t imported yet, it first hands osu! a byte-identical .osz built from the map’s own files.

Replay generation, not control: the app only gives osu! a file, the same as double-clicking one. Generated replays are kept in a history with their metrics and survive deleting the model.

Dashboard

Totals, last trained and best model, training status, quick actions

Create Model

Maps, training length, learning rate, batch size, checkpoints, seed, advanced PPO settings — then a summary

Existing Models

Rename, archive, delete or open a model

Version History

The lineage tree and every per-version action

Continue Training

Resume from the active, best or latest version, an older one, or a checkpoint

Training

Live progress, losses, recent accuracy, checkpoints and ETA

Test Model

Watch the bot play live in a 4K playfield, or generate a replay directly

Replays

History with metrics; open in osu!, show in folder, regenerate, delete

Settings

Storage location and usage, osu! folder, defaults, import of older CLI runs, logs

• Evaluation that presses keys

  Always taking the single most likely action per tick would never press a key, so evaluation samples actions the way training does.

• Byte-identical maps

  A replay is tied to its beatmap by MD5, so any re-downloaded or edited map breaks it. Mapsets handed to osu! are built from the map’s own files.

• Long-running training

  Training lives in a spawned worker with its own child processes, so the interface stays responsive and a stop still saves a clean version.

• Never losing a model

  Atomic saves, immutable versions, locks on versions in use and confirmations on deletion — each covered by tests.

• PPO itself was the off-the-shelf part. Almost all of the real work was in the environment: what the agent observes, how often it decides, and what it is rewarded for. Designing a custom Gymnasium environment taught me that the environment is the specification of the problem.
• Observation design mattered more than I expected. Square-root scaling the time-to-hit gives the network its finest resolution exactly where timing decisions happen, and handing it structured lane data rather than pixels kept the problem small enough to train on a CPU.
• Rewards are an argument with the optimiser. Without small penalties for stray presses, mashing becomes a strategy; without timing jitter, a perfect-timing policy has nothing left to learn. Making “improving” mean something took deliberate design.
• Some behaviour only shows up in evaluation. Taking the single most likely action each tick would never press a key at all — a reminder to evaluate a stochastic policy the way it was trained.
• An ML project quickly becomes a model-lifecycle project. Checkpoints, resumable optimiser state, immutable versions and lineage made experiments reproducible and safe to branch from, and I tested that infrastructure as carefully as any other software rather than trusting it.
• Working with real file formats was its own lesson: parsing .osu beatmaps, unpacking .osz and .zip archives without touching the originals, and writing a binary .osr with LZMA-compressed frames that another program accepts byte for byte.
• Finally, running long training jobs behind a desktop interface — worker processes, live progress, pause and safe stop — showed me how much engineering sits between “a training script works” and “someone can actually use it”.

Known limitsSimulator accuracy is not an osu! score — osu! re-judges replays, mostly differing on 50/100 edges and hold releases. 4K only. osu! stable is the only supported client; osu!lazer is untested. Developed and tested on Windows; macOS and Linux paths exist but are untested.