# Universal DOS Game Reconstruction Playbook ### A Game-Agnostic Engineering Guide to Bit-Exact (1:1) Clean-Room Reverse Engineering, Differential CPU Harnessing, and Modern Recompilation --- ## 1. Core Philosophy: Bit-Exact Clean-Room Re-engineering Porting a legacy DOS title to modern architectures (Python, C++, Rust, WebAssembly) typically falls into one of two paradigms: 1. **Approximate Recreation ("By Eye"):** Recreating physics, AI, and rules by playing the game in DOSBox or reading high-level disassembly. * *The Problem:* Subpixel physics, collision corner cases, fixed-point integer rounding, and subtle AI timers inevitably diverge. The resulting game *feels* different and fails historical preservation standards. 2. **Differential Re-engineering (Dual-Execution Parity):** Executing the original 16-bit x86 binary alongside the newly reconstructed modern engine, feeding identical inputs, and asserting that the state of both systems is **bit-for-bit identical on every single simulation tick**. ```mermaid flowchart TD ROM["Original DOS Executable (.EXE / .COM)"] --> DISASM["Static Triage (Ghidra / IDA / objdump)"] ROM --> UNICORN["Differential CPU Sandbox (Unicorn x86 Engine)"] DISASM --> CODE["Reconstructed Engine (Python / C++ / Rust)"] UNICORN --> HARNESS["Differential Assertion Harness"] CODE --> HARNESS HARNESS -- "Memory, Registers & State Match?" --> PASS["Bit-Exact Parity Confirmed"] HARNESS -- "Mismatch at Tick N" --> DEBUG["Automated Register / Memory Delta Report"] DEBUG --> CODE ``` > [!IMPORTANT] > **The Golden Rule of 1:1 Parity:** > Do not guess formulas. If an algorithm's behavior is ambiguous, isolate its entry point in an x86 CPU emulator, feed it controlled register/memory states, and verify that the re-implemented function reproduces the exact register and memory mutations across all value domains. --- ## 2. Universal Tooling Stack A complete clean-room reconstruction requires four decoupled layers: ### 2.1. Static Analysis & Binary Triage * **Disassemblers:** `objdump` / `ndisasm` for rapid machine code inspection (`-m i8086 -M intel`). * **Interactive Decompilers:** Ghidra or IDA Pro for mapping call graphs, data segment references (`DGROUP`), indirect jump tables (`switch/case`), and runtime compiler signatures (e.g., Borland C++, Microsoft C, Watcom C). * **Binary Container Unpackers:** Custom scripts for parsing DOS executable headers (MZ / LE / Linear Executable / COM), segment relocation tables, and legacy compression schemes (LZSS, RLE, Huffman, PKWARE DCL). ### 2.2. Execution Sandbox & CPU Emulation * **CPU Emulation Core:** **Unicorn Engine** (lightweight x86 16-bit real mode and 32-bit protected mode emulator based on QEMU). * **Interception Framework:** CPU hooks (`UC_HOOK_CODE`, `UC_HOOK_INTR`) for capturing hardware port I/O (`IN`/`OUT`), BIOS/DOS software interrupts (`INT 10h`, `INT 21h`, `INT 09h`), and external driver far-calls. ### 2.3. Audio Synthesis & Hardware Emulation * **FM Synthesis Core:** **Nuked-OPL3** (or Nuked-OPL2) compiled as a shared library (`.so` / `.dll`). Translates direct register writes to ports `0x388`/`0x389` (AdLib / Sound Blaster) into a 16-bit stereo PCM stream (44,100 Hz). * **PCM Sample Engine:** Direct Sound Blaster DSP register emulation (`0x220..0x22C`) for digitized 8-bit mono audio samples. * **Low-Latency Streaming:** Modern SDL2 audio queue (`SDL_QueueAudio`) or native OS audio backends. ### 2.4. Presentation & Windowing * Modern surface rendering (SDL2 / Pygame / WebGL) with integer scaling, 4:3 aspect-ratio correction, and virtual framebuffers decoupling internal simulation resolution (e.g., 320x200 / 320x240) from host display resolutions. --- ## 3. The Differential Execution Harness Architecture The differential harness executes the original DOS machine code inside an emulated memory space. Below is the game-agnostic architecture for a 16-bit DOS real-mode sandbox. ### 3.1. Real-Mode Memory Layout In x86 16-bit real mode, physical memory addresses are calculated as: $$\text{Physical Address} = (\text{Segment} \times 16) + \text{Offset}$$ A standard 2 MB virtual address space accommodates the entire legacy environment: ``` 0x000000 ┌──────────────────────────────────────────────┐ │ Interrupt Vector Table (IVT, 256 vectors * 4B)│ 0x000400 ├──────────────────────────────────────────────┤ │ BIOS Data Area (BDA, 256 bytes) │ 0x000500 ├──────────────────────────────────────────────┤ │ DOS Conventional RAM Free Pool │ 0x01DB00 ├──────────────────────────────────────────────┤ │ Application Program Image (CS, Code Segments)│ 0x05C090 ├──────────────────────────────────────────────┤ │ Application Data Segment (DS / DGROUP) │ 0x090000 ├──────────────────────────────────────────────┤ │ Stack Segment (SS:SP = 0x9000:0xFFFE) │ 0x0A0000 ├──────────────────────────────────────────────┤ │ Video RAM (VRAM - Mode 13h / Mode X / Planar)│ 0x0FFFFF └──────────────────────────────────────────────┘ ``` ### 3.2. Reusable Unicorn Real-Mode Harness (Python Template) ```python import struct from unicorn import Uc, UC_ARCH_X86, UC_MODE_16, UC_HOOK_CODE, UC_HOOK_INTR from unicorn.x86_const import ( UC_X86_REG_CS, UC_X86_REG_IP, UC_X86_REG_DS, UC_X86_REG_ES, UC_X86_REG_SS, UC_X86_REG_SP, UC_X86_REG_BP, UC_X86_REG_AX, UC_X86_REG_BX, UC_X86_REG_CX, UC_X86_REG_DX, UC_X86_REG_SI, UC_X86_REG_DI ) class RealModeHarness: """Game-agnostic execution sandbox for 16-bit DOS binaries.""" def __init__(self, memory_mb: int = 2): self.mem_size = memory_mb * 1024 * 1024 self.uc = Uc(UC_ARCH_X86, UC_MODE_16) self.uc.mem_map(0, self.mem_size) # Setup standard stack at top of conventional memory self.ss = 0x9000 self.sp = 0xFFFE self.uc.reg_write(UC_X86_REG_SS, self.ss) self.uc.reg_write(UC_X86_REG_SP, self.sp) # Intercept hooks self.io_reads = {} self.io_writes = {} self.external_calls = {} self.uc.hook_add(UC_HOOK_CODE, self._on_code) def load_image(self, segment: int, data: bytes): """Loads executable code or data at the specified segment.""" phys_addr = segment * 16 self.uc.mem_write(phys_addr, data) def stub_far_call(self, target_seg: int, target_off: int, handler_fn=None): """Intercepts far calls to external libraries/drivers and simulates RETF.""" phys_addr = target_seg * 16 + target_off self.external_calls[phys_addr] = handler_fn def _on_code(self, uc, address, size, user_data): # 1. Handle stubbed library calls (far calls) if address in self.external_calls: handler = self.external_calls[address] if handler: handler(self) # Read far return address from stack and simulate RETF sp = uc.reg_read(UC_X86_REG_SP) ss = uc.reg_read(UC_X86_REG_SS) ret_ip, ret_cs = struct.unpack_from(" P2["Phase 2: Physics Core Isolation"] P2 --> P3["Phase 3: Entity Machines & AI"] P3 --> P4["Phase 4: Audio Synthesis & Timers"] P4 --> P5["Phase 5: Lifecycle & Frontend"] ``` ### Phase 1: Binary & Asset Triage 1. **Identify the Compiler & Runtime (CRT):** * Inspect the entry point (`CS:IP` from MZ header). Identify Borland `C0.ASM`, Microsoft CRT, or custom assembly loaders. Borland C++ binaries typically locate DGROUP at a fixed segment following code segments. 2. **Resource Extraction:** * Dump static palette tables (VGA 6-bit DAC values `0..63`). Convert via `(val * 255) // 63` or `val << 2`. * Unpack graphics buffers. Distinguish between packed pixel formats (Mode 13h linear `320x200`) and planar bitplane formats (Mode X unchained 4-plane memory at `0xA000`). * Extract level descriptor arrays and tile lookup indices. ### Phase 2: Physics Core Isolation 1. **Map the Fixed-Point Physics Loop:** * DOS physics calculations almost universally use fixed-point arithmetic instead of floating-point units (which were rare on 286/386 systems). Typical formats: $1/16$ px (4 fractional bits), $1/32$ px (5 fractional bits), or $16.16$ fixed-point. 2. **Isolate the Primary Entity Step:** * Trace the physics routine (inputs -> velocity integration -> tile collision mask -> snap/bounce). * Wire the procedure into the differential harness. * Run Monte Carlo fuzzing: generate 50,000 randomized input sequences (joypad + terrain configurations) and assert that Python positions match x86 positions on every tick. ### Phase 3: Entity Dispatch & Actor State Machines 1. **Decode the Entity Table Structure:** * DOS games typically manage entities via an array of uniform structures in `DGROUP` (e.g., 32 to 64 bytes per actor). * Locate the handler jump table (`call word ptr [bx + state_offset]`). 2. **Translate Jump Tables into Reconstructed State Machines:** * Convert indirect assembly function pointer tables into explicit state enumerations: ```python class EntityState(Enum): INIT = 0x08 PATROL = 0x0A ALERT = 0x0E ATTACK = 0x12 STUNNED = 0x16 DEAD = 0x1E ``` 3. **Differential Actor Verification:** * Step each actor type independently across its entire state lifecycle, verifying sprite index, velocity delta, and hit-box boundaries. ### Phase 4: Audio Driver, Hardware Timers & Static Asset Export 1. **Deconstruct the Low-Level Sound Engine:** * DOS titles typically use an AdLib/Sound Blaster driver executing via timer interrupt (`INT 08h`). * Identify whether the driver reprograms the Intel 8253/8254 PIT (Programmable Interval Timer) channel 0. 2. **The PIT Timing Trap:** * The standard BIOS timer ticks at **18.2 Hz** (divisor 0). * Game music drivers often reprogram PIT Channel 0 to divisors like `6211` ($\approx 192.1\text{ Hz}$) or `1193` ($\approx 1000\text{ Hz}$). * **The Solution:** Do not bind music playback to video frame rate. Use a dedicated sub-accumulator ticking the audio engine at the exact hardware PIT frequency. 3. **Eliminating Heavy CPU Emulators & Native C Compilation (100% Python Distribution):** * *The Problem:* Shipping Unicorn (heavy x86 CPU emulator) and compiling custom C libraries (e.g. NukedOPL) introduces compilation hurdles, toolchain requirements (GCC/MinGW/Clang), and platform friction on Windows/macOS/Linux. * *The Modern Architecture:* - **Sequencing:** Parse the DOS audio bytecodes/events into native Python sequence tables (`game/audio_data.py`), completely eliminating the need for an x86 CPU emulator (`Unicorn`). - **FM Synthesis:** Use standard, cross-platform PyPI packages with prebuilt wheels (such as `PyOPL`, `pip install pyopl`) that require zero compiler setup across Windows, macOS, and Linux. - **Asset Export:** Pre-render all music tracks (seamless loop lengths) and SFX to standard CD-quality `.wav` or `.ogg` files (`assets/audio/`). - **Runtime:** Stream audio via standard host audio (`pygame.mixer` or `SDL2`). - Result: **Zero CPU emulation, zero Unicorn dependency, zero native C compilation**, and 100% bit-exact authentic sound. 4. **Sound Effect (SFX) Multi-Stage Sequencing & Acoustic Release Decay:** * *The Multi-Stage Trap:* Sound effects are often mistakenly assumed to be short, single-shot pulses lasting $< 0.5\text{ s}$. In reality, retro DOS sound effects are frequently multi-stage sequences (e.g., primary impact at $t=2$, pause while in air, secondary rebound at $t=422$, and rolling rubble with final mute at $t=1262$). Hardcoding small tick limits (e.g. 400 ticks) cuts off the latter half of the sound effect. * *Dynamic Event Scanning:* Emulate sound effects with a generous scan window (e.g. 2,000–3,000 ticks) and locate the true terminal write where the driver issues the final Key-Off (`0x00` on bit 5 of `0xB0..0xB8`) and total level attenuation mute (`TL=0x3F` on `0x40..0x55`). * *Acoustic Decay Tail:* When Key-Off is issued, Yamaha OPL2 FM operators transition to the Release state (`RR` in registers `0x80..0x95`). Halting sample generation immediately upon the final register write creates a harsh, abrupt step-cutoff. Always append a release decay tail (300–400 ms) and apply a gentle 50 ms taper to zero at the trailing boundary to ensure sample values end at exactly `0`. ```python class PitAudioAccumulator: """Ensures audio driver ticks at authentic PIT timer frequency.""" def __init__(self, pit_frequency_hz: float = 192.107): self.pit_dt = 1.0 / pit_frequency_hz self.accumulator = 0.0 def advance(self, real_dt: float, sound_driver_tick_fn): self.accumulator += real_dt while self.accumulator >= self.pit_dt: sound_driver_tick_fn() self.accumulator -= self.pit_dt ``` ### Phase 5: Lifecycle & Frontend Integration 1. **State Flow Mapping:** * Trace the global finite state machine: $$\text{Intro Cutscene} \longrightarrow \text{Title Menu} \longrightarrow \text{Level Loading} \longrightarrow \text{Game Loop} \longrightarrow \text{Level Transition} \longrightarrow \text{Game Over / Victory}$$ 2. **Binary Data Persistence:** * Map save-game structures and high-score files. * If the DOS game XOR-encrypts or checksums save files (e.g., `XOR 0xFF` on high-score bytes), re-implement the exact byte manipulation so existing legacy save files load seamlessly. --- ## 5. Architectural Pitfalls & Root-Cause Analysis These 7 failure modes represent the most frequent sources of discrepancies in DOS-to-modern ports: > [!WARNING] > ### Pitfall 1: Procedure Coverage vs. Integration Lifecycle Blind Spots > A test harness verifying that 100% of individual assembly procedures produce identical math when called in isolation does **not** prove that the game loop connects them correctly. > * *Failure Example:* An entity collision function works identically, but the main game loop fails to invoke the sound trigger event or resets a multiplier on the wrong screen transition. > * *Mitigation:* Always complement procedural unit tests with long-running, multi-level **Scenario Tests (End-to-End)** that simulate real player death, level completions, and menu exits. > [!CAUTION] > ### Pitfall 2: Silent Audio Loss via Emulator Stubs > When constructing a CPU differential harness, external driver far-calls (such as `play_sfx` or `start_music`) are routinely stubbed out with `RETF` to prevent the CPU emulator from getting bogged down in low-level sound card hardware registers. > * *Failure Example:* The CPU emulator succeeds because the stub returns cleanly, but the developer forgets to wire that specific trigger into the modern audio queue. The game runs with missing sound effects. > * *Mitigation:* Build a static audit listing **every single call site** to the audio entry points in the original binary disassembly. Assert that each call site corresponds to an event dispatched in the modern engine. > [!IMPORTANT] > ### Pitfall 3: Level Transition Audio State Leaks > In DOS games, the main loop often re-arms background music at the beginning of a level reload. > * *Failure Example:* A modern port starts background music in `start_game()`, but when the player loses a life and restarts the level, the level-card display silences the audio, and no routine restarts the track. > * *Mitigation:* Track the entry-point address of restart loops in disassembly. If the DOS assembly jumps to a location that calls `start_music(TRACK_ID)` before re-entering the main loop, ensure the modern loop mirrors that exact call. > [!NOTE] > ### Pitfall 4: 16-Bit Integer Wrapping and Subpixel Boundary Inaccuracies > In 16-bit x86 code, registers wrap at 65,536 (`0xFFFF`), and signed comparisons (`JL`, `JLE`) behave differently from unsigned comparisons (`JB`, `JBE`). > * *Failure Example:* Translating `CMP AX, BX / JLE target` to high-level code as `if ax < bx:` instead of `if ax <= bx:` causes boundary conditions (e.g., surface water line, platform edge) to trigger one pixel early or late. > * *Mitigation:* Always enforce explicit 16-bit masking (`val & 0xFFFF`) on accumulator registers and translate conditional jumps strictly according to x86 flag definitions (ZF, SF, OF, CF). > [!TIP] > ### Pitfall 5: Display Refresh Rate Bottlenecks (70 Hz vs. 35 Hz) > Standard VGA Mode 13h and Mode X refresh at **70.086 Hz** ($25.175\text{ MHz} / (800\text{ clocks} \times 449\text{ lines})$). The DOS main loop polls VBLANK (`IN AL, 0x3DA / TEST AL, 8`). > * *The Issue:* On original 286/386 hardware, complex planar rendering often exceeded 14.2 milliseconds per frame. As a result, the game regularly missed the vertical retrace and dropped to **35 FPS** (every second VBLANK). > * On a modern machine running the code at an unconstrained 70.086 Hz, the game will feel significantly faster than players remember from childhood hardware. > * *Mitigation:* Keep internal game simulation strictly deterministic at the native rate (70.086 Hz) using a fixed-timestep clock, but provide a configurable timing mode or half-rate throttle for players seeking period-accurate 386 performance. > [!CAUTION] > ### Pitfall 6: Dynamic PIT Timer Mismatches & Song Loop Truncation ("Plumkanie") > Sound drivers in DOS rarely run at a fixed 18.2 Hz or 192 Hz. Track headers often dynamically re-program the PIT Timer 0 frequency (e.g. via opcode `0x60`) to custom rates (e.g. 560 Hz, 770 Hz, 840 Hz) and play elaborate loops spanning 50,000–80,000 ticks (1–2 minutes). > * *Failure Example:* Extracting audio with an idle/default PIT frequency (e.g. 192 Hz) and an arbitrary truncated tick count (e.g. 840 ticks) causes music to play at 1/4 original speed with only 1.5 seconds of the intro repeating, producing broken, disjointed "plumkanie". Superficial test harnesses using mocks (`MockSoundDriver`) miss this completely because they only check if `start_music()` was invoked. > * *Mitigation:* Extract audio by dynamically capturing the PIT reload register configured by the driver. Verify exact mathematical periodicity across tens of thousands of ticks to ensure the full song loop is captured. Assert track duration (>= 60–100s) and signal RMS/peak energy in the automated test harness. > [!WARNING] > ### Pitfall 7: Sound Effect Premature Truncation & Release Envelope Clipping > Sound effects in DOS titles often contain multi-stage mechanical behaviors (e.g., falling rocks bouncing multiple times, water splashing in successive waves, crash crunch followed by metallic reverberation) spanning over 1,000 hardware ticks (1.5–2.5 seconds). > * *Failure Example:* Assuming all SFX are short (< 0.5s) and capping the capture loop at an arbitrary threshold (e.g., 400 ticks). Multi-stage sound effects lose their secondary bounces and fadeouts. Additionally, stopping rendering immediately upon the final register write slices the FM release envelope mid-decay, creating an unnatural, clipped cutoff ("efekty są ucięte"). > * *Mitigation:* > 1. **Dynamic Capture:** Run emulation until the sound driver writes the final Key-Off and volume mute commands (scanning 2,000+ ticks). Never guess tick counts. > 2. **Release Envelope Tail:** Append 300–400 ms of synthesized audio after the final register write to allow FM operator envelopes to decay naturally into silence. > 3. **Zero-Crossing Taper:** Smoothly taper the final 50 ms of the release tail to absolute zero to prevent DC-offset pops or clicks. > 4. **Harness Assertion:** In automated tests, verify both the expected duration range (e.g., 1.2–2.0s for impacts/boulders) and assert that trailing audio samples are strictly zero (`max(abs(trailing_samples)) == 0`). --- ## 6. Deterministic Fixed-Timestep Clock Template Modern host operating systems run with variable refresh rates, multitasking jitter, and sleep latency. The game loop must decouple wall-clock time from internal simulation ticks: ```python import time class FixedTimestepClock: """Deterministic game loop clock modeled on DOS hardware VBLANK timing.""" def __init__(self, target_hz: float = 70.0866, max_ticks_per_frame: int = 4): self.tick_ns = round(1e9 / target_hz) self.max_ticks = max_ticks_per_frame self.accumulator_ns = 0 self.total_ticks = 0 def advance(self, dt_seconds: float) -> int: """Accumulates elapsed wall-clock time and returns logic ticks to execute.""" dt_ns = max(0, round(dt_seconds * 1e9)) self.accumulator_ns += dt_ns # Calculate pending whole ticks ticks = self.accumulator_ns // self.tick_ns if ticks > self.max_ticks: # Drop backlog to prevent spiral of death during backgrounding / debug pause ticks = self.max_ticks self.accumulator_ns = 0 else: self.accumulator_ns -= ticks * self.tick_ns self.total_ticks += ticks return ticks def reset(self): """Call upon unpausing or level reload to prevent time bursts.""" self.accumulator_ns = 0 ``` --- ## 7. Comprehensive End-to-End Verification Checklist Before certifying a DOS reconstruction as **1:1 Bit-Exact Complete**, verify all 8 test suites: ### 1. Long-Run Physics Determinism * [ ] Run the differential harness over $\ge 50,000$ continuous ticks with randomized and scripted inputs. * [ ] Assert $0$ mismatches across position ($X, Y$), velocity ($VX, VY$), and collision states. ### 2. Rare State Machine Edge-Cases * [ ] Verify actor knockback, hitstun, and recovery states. * [ ] Verify entity destruction, rubble/shrapnel countdown timers, and despawn cycles. * [ ] Verify environmental submergence, buoyancy physics, and out-of-bounds checks. ### 3. Complete Audio Call-Site Parity * [ ] Conduct a binary-wide search for all `call play_sfx` instruction sites. * [ ] Construct automated integration tests ensuring every single audio call site dispatches its associated SFX event during gameplay. ### 4. Audio Driver Parity & FM Synthesis * [ ] Capture raw OPL register streams (`0x388/0x389`) from the x86 binary and assert identical byte output from the modern driver. * [ ] Verify PIT timer accumulation ($192\text{ Hz}$ or equivalent) to prevent tempo dragging. ### 5. Session Lifecycle & Input Abort * [ ] Test player death down to 0 lives: verify game over transition, delay timers, and failure audio. * [ ] Test mid-game input abort (e.g., ESC key): verify clean score carry-over and life zeroing without process termination. * [ ] Test pause toggle: verify audio continuity during pause and zero physics jumps on unpause. ### 6. Persistence & Serialization * [ ] Save and load high-score tables and game save files. * [ ] Assert identical byte-for-byte outputs against legacy DOS disk formats (including XOR masks and padding). ### 7. Game Completion (Outro & Victory) * [ ] Simulate completion of the final level. * [ ] Verify that victory cutscenes execute their complete animation script and play the authentic ending music track. ### 8. Cooperative & Difficulty Modes * [ ] Verify multi-player modes: test split input handling, shared life pools, and player-to-player collision boxes. * [ ] Verify difficulty modifiers: test crash thresholds, fuel/energy drain rate multipliers, and score bonuses. --- ## 8. Summary A true 1:1 reconstruction does not rely on approximations. By establishing an x86 CPU differential harness, cataloging every procedure, intercepting hardware registers, and enforcing strict dual-execution assertions, legacy DOS titles can be preserved in modern software stacks with absolute mathematical fidelity.