# Universal DOS Game Reconstruction Playbook ### A Game-Agnostic Engineering Guide to Bit-Exact (1:1) Clean-Room Reverse Engineering, Differential CPU Harnessing, and Modern Recompilation --- ## 1. Core Philosophy: Bit-Exact Clean-Room Re-engineering Porting a legacy DOS title to modern architectures (Python, C++, Rust, WebAssembly) typically falls into one of two paradigms: 1. **Approximate Recreation ("By Eye"):** Recreating physics, AI, and rules by playing the game in DOSBox or reading high-level disassembly. * *The Problem:* Subpixel physics, collision corner cases, fixed-point integer rounding, and subtle AI timers inevitably diverge. The resulting game *feels* different and fails historical preservation standards. 2. **Differential Re-engineering (Dual-Execution Parity):** Executing the original 16-bit x86 binary alongside the newly reconstructed modern engine, feeding identical inputs, and asserting that the state of both systems is **bit-for-bit identical on every single simulation tick**. ```mermaid flowchart TD ROM["Original DOS Executable (.EXE / .COM)"] --> DISASM["Static Triage (Ghidra / IDA / objdump)"] ROM --> UNICORN["Differential CPU Sandbox (Unicorn x86 Engine)"] DISASM --> CODE["Reconstructed Engine (Python / C++ / Rust)"] UNICORN --> HARNESS["Differential Assertion Harness"] CODE --> HARNESS HARNESS -- "Memory, Registers & State Match?" --> PASS["Bit-Exact Parity Confirmed"] HARNESS -- "Mismatch at Tick N" --> DEBUG["Automated Register / Memory Delta Report"] DEBUG --> CODE ``` > [!IMPORTANT] > **The Golden Rule of 1:1 Parity:** > Do not guess formulas. If an algorithm's behavior is ambiguous, isolate its entry point in an x86 CPU emulator, feed it controlled register/memory states, and verify that the re-implemented function reproduces the exact register and memory mutations across all value domains. --- ## 2. Universal Tooling Stack A complete clean-room reconstruction requires four decoupled layers: ### 2.1. Static Analysis & Binary Triage * **Disassemblers:** `objdump` / `ndisasm` for rapid machine code inspection (`-m i8086 -M intel`). * **Interactive Decompilers:** Ghidra or IDA Pro for mapping call graphs, data segment references (`DGROUP`), indirect jump tables (`switch/case`), and runtime compiler signatures (e.g., Borland C++, Microsoft C, Watcom C). * **Binary Container Unpackers:** Custom scripts for parsing DOS executable headers (MZ / LE / Linear Executable / COM), segment relocation tables, and legacy compression schemes (LZSS, RLE, Huffman, PKWARE DCL). ### 2.2. Execution Sandbox & CPU Emulation * **CPU Emulation Core:** **Unicorn Engine** (lightweight x86 16-bit real mode and 32-bit protected mode emulator based on QEMU). * **Interception Framework:** CPU hooks (`UC_HOOK_CODE`, `UC_HOOK_INTR`) for capturing hardware port I/O (`IN`/`OUT`), BIOS/DOS software interrupts (`INT 10h`, `INT 21h`, `INT 09h`), and external driver far-calls. ### 2.3. Audio Synthesis & Hardware Emulation * **FM Synthesis Core:** **Nuked-OPL3** (or Nuked-OPL2) compiled as a shared library (`.so` / `.dll`). Translates direct register writes to ports `0x388`/`0x389` (AdLib / Sound Blaster) into a 16-bit stereo PCM stream (44,100 Hz). * **PCM Sample Engine:** Direct Sound Blaster DSP register emulation (`0x220..0x22C`) for digitized 8-bit mono audio samples. * **Low-Latency Streaming:** Modern SDL2 audio queue (`SDL_QueueAudio`) or native OS audio backends. ### 2.4. Presentation & Windowing * Modern surface rendering (SDL2 / Pygame / WebGL) with integer scaling, 4:3 aspect-ratio correction, and virtual framebuffers decoupling internal simulation resolution (e.g., 320x200 / 320x240) from host display resolutions. --- ## 3. The Differential Execution Harness Architecture The differential harness executes the original DOS machine code inside an emulated memory space. Below is the game-agnostic architecture for a 16-bit DOS real-mode sandbox. ### 3.1. Real-Mode Memory Layout In x86 16-bit real mode, physical memory addresses are calculated as: $$\text{Physical Address} = (\text{Segment} \times 16) + \text{Offset}$$ A standard 2 MB virtual address space accommodates the entire legacy environment: ``` 0x000000 ┌──────────────────────────────────────────────┐ │ Interrupt Vector Table (IVT, 256 vectors * 4B)│ 0x000400 ├──────────────────────────────────────────────┤ │ BIOS Data Area (BDA, 256 bytes) │ 0x000500 ├──────────────────────────────────────────────┤ │ DOS Conventional RAM Free Pool │ 0x01DB00 ├──────────────────────────────────────────────┤ │ Application Program Image (CS, Code Segments)│ 0x05C090 ├──────────────────────────────────────────────┤ │ Application Data Segment (DS / DGROUP) │ 0x090000 ├──────────────────────────────────────────────┤ │ Stack Segment (SS:SP = 0x9000:0xFFFE) │ 0x0A0000 ├──────────────────────────────────────────────┤ │ Video RAM (VRAM - Mode 13h / Mode X / Planar)│ 0x0FFFFF └──────────────────────────────────────────────┘ ``` ### 3.2. Reusable Unicorn Real-Mode Harness (Python Template) ```python import struct from unicorn import Uc, UC_ARCH_X86, UC_MODE_16, UC_HOOK_CODE, UC_HOOK_INTR from unicorn.x86_const import ( UC_X86_REG_CS, UC_X86_REG_IP, UC_X86_REG_DS, UC_X86_REG_ES, UC_X86_REG_SS, UC_X86_REG_SP, UC_X86_REG_BP, UC_X86_REG_AX, UC_X86_REG_BX, UC_X86_REG_CX, UC_X86_REG_DX, UC_X86_REG_SI, UC_X86_REG_DI ) class RealModeHarness: """Game-agnostic execution sandbox for 16-bit DOS binaries.""" def __init__(self, memory_mb: int = 2): self.mem_size = memory_mb * 1024 * 1024 self.uc = Uc(UC_ARCH_X86, UC_MODE_16) self.uc.mem_map(0, self.mem_size) # Setup standard stack at top of conventional memory self.ss = 0x9000 self.sp = 0xFFFE self.uc.reg_write(UC_X86_REG_SS, self.ss) self.uc.reg_write(UC_X86_REG_SP, self.sp) # Intercept hooks self.io_reads = {} self.io_writes = {} self.external_calls = {} self.uc.hook_add(UC_HOOK_CODE, self._on_code) def load_image(self, segment: int, data: bytes): """Loads executable code or data at the specified segment.""" phys_addr = segment * 16 self.uc.mem_write(phys_addr, data) def stub_far_call(self, target_seg: int, target_off: int, handler_fn=None): """Intercepts far calls to external libraries/drivers and simulates RETF.""" phys_addr = target_seg * 16 + target_off self.external_calls[phys_addr] = handler_fn def _on_code(self, uc, address, size, user_data): # 1. Handle stubbed library calls (far calls) if address in self.external_calls: handler = self.external_calls[address] if handler: handler(self) # Read far return address from stack and simulate RETF sp = uc.reg_read(UC_X86_REG_SP) ss = uc.reg_read(UC_X86_REG_SS) ret_ip, ret_cs = struct.unpack_from(" P2["Phase 2: Physics Core Isolation"] P2 --> P3["Phase 3: Entity Machines & AI"] P3 --> P4["Phase 4: Audio Synthesis & Timers"] P4 --> P5["Phase 5: Lifecycle & Frontend"] ``` ### Phase 1: Binary & Asset Triage 1. **Identify the Compiler & Runtime (CRT):** * Inspect the entry point (`CS:IP` from MZ header). Identify Borland `C0.ASM`, Microsoft CRT, or custom assembly loaders. Borland C++ binaries typically locate DGROUP at a fixed segment following code segments. 2. **Resource Extraction:** * Dump static palette tables (VGA 6-bit DAC values `0..63`). Convert via `(val * 255) // 63` or `val << 2`. * Unpack graphics buffers. Distinguish between packed pixel formats (Mode 13h linear `320x200`) and planar bitplane formats (Mode X unchained 4-plane memory at `0xA000`). * Extract level descriptor arrays and tile lookup indices. ### Phase 2: Physics Core Isolation 1. **Map the Fixed-Point Physics Loop:** * DOS physics calculations almost universally use fixed-point arithmetic instead of floating-point units (which were rare on 286/386 systems). Typical formats: $1/16$ px (4 fractional bits), $1/32$ px (5 fractional bits), or $16.16$ fixed-point. 2. **Isolate the Primary Entity Step:** * Trace the physics routine (inputs -> velocity integration -> tile collision mask -> snap/bounce). * Wire the procedure into the differential harness. * Run Monte Carlo fuzzing: generate 50,000 randomized input sequences (joypad + terrain configurations) and assert that Python positions match x86 positions on every tick. ### Phase 3: Entity Dispatch & Actor State Machines 1. **Decode the Entity Table Structure:** * DOS games typically manage entities via an array of uniform structures in `DGROUP` (e.g., 32 to 64 bytes per actor). * Locate the handler jump table (`call word ptr [bx + state_offset]`). 2. **Translate Jump Tables into Reconstructed State Machines:** * Convert indirect assembly function pointer tables into explicit state enumerations: ```python class EntityState(Enum): INIT = 0x08 PATROL = 0x0A ALERT = 0x0E ATTACK = 0x12 STUNNED = 0x16 DEAD = 0x1E ``` 3. **Differential Actor Verification:** * Step each actor type independently across its entire state lifecycle, verifying sprite index, velocity delta, and hit-box boundaries. ### Phase 4: Audio Driver & Hardware Timer Parity 1. **Deconstruct the Low-Level Sound Engine:** * DOS titles typically use an AdLib/Sound Blaster driver executing via timer interrupt (`INT 08h`). * Identify whether the driver reprograms the Intel 8253/8254 PIT (Programmable Interval Timer) channel 0. 2. **The PIT Timing Trap:** * The standard BIOS timer ticks at **18.2 Hz** (divisor 0). * Game music drivers often reprogram PIT Channel 0 to divisors like `6211` ($\approx 192.1\text{ Hz}$) or `1193` ($\approx 1000\text{ Hz}$). * **The Solution:** Do not bind music playback to video frame rate. Use a dedicated sub-accumulator ticking the audio engine at the exact hardware PIT frequency. ```python class PitAudioAccumulator: """Ensures audio driver ticks at authentic PIT timer frequency.""" def __init__(self, pit_frequency_hz: float = 192.107): self.pit_dt = 1.0 / pit_frequency_hz self.accumulator = 0.0 def advance(self, real_dt: float, sound_driver_tick_fn): self.accumulator += real_dt while self.accumulator >= self.pit_dt: sound_driver_tick_fn() self.accumulator -= self.pit_dt ``` ### Phase 5: Lifecycle & Frontend Integration 1. **State Flow Mapping:** * Trace the global finite state machine: $$\text{Intro Cutscene} \longrightarrow \text{Title Menu} \longrightarrow \text{Level Loading} \longrightarrow \text{Game Loop} \longrightarrow \text{Level Transition} \longrightarrow \text{Game Over / Victory}$$ 2. **Binary Data Persistence:** * Map save-game structures and high-score files. * If the DOS game XOR-encrypts or checksums save files (e.g., `XOR 0xFF` on high-score bytes), re-implement the exact byte manipulation so existing legacy save files load seamlessly. --- ## 5. Architectural Pitfalls & Root-Cause Analysis These 5 failure modes represent the most frequent sources of discrepancies in DOS-to-modern ports: > [!WARNING] > ### Pitfall 1: Procedure Coverage vs. Integration Lifecycle Blind Spots > A test harness verifying that 100% of individual assembly procedures produce identical math when called in isolation does **not** prove that the game loop connects them correctly. > * *Failure Example:* An entity collision function works identically, but the main game loop fails to invoke the sound trigger event or resets a multiplier on the wrong screen transition. > * *Mitigation:* Always complement procedural unit tests with long-running, multi-level **Scenario Tests (End-to-End)** that simulate real player death, level completions, and menu exits. > [!CAUTION] > ### Pitfall 2: Silent Audio Loss via Emulator Stubs > When constructing a CPU differential harness, external driver far-calls (such as `play_sfx` or `start_music`) are routinely stubbed out with `RETF` to prevent the CPU emulator from getting bogged down in low-level sound card hardware registers. > * *Failure Example:* The CPU emulator succeeds because the stub returns cleanly, but the developer forgets to wire that specific trigger into the modern audio queue. The game runs with missing sound effects. > * *Mitigation:* Build a static audit listing **every single call site** to the audio entry points in the original binary disassembly. Assert that each call site corresponds to an event dispatched in the modern engine. > [!IMPORTANT] > ### Pitfall 3: Level Transition Audio State Leaks > In DOS games, the main loop often re-arms background music at the beginning of a level reload. > * *Failure Example:* A modern port starts background music in `start_game()`, but when the player loses a life and restarts the level, the level-card display silences the audio, and no routine restarts the track. > * *Mitigation:* Track the entry-point address of restart loops in disassembly. If the DOS assembly jumps to a location that calls `start_music(TRACK_ID)` before re-entering the main loop, ensure the modern loop mirrors that exact call. > [!NOTE] > ### Pitfall 4: 16-Bit Integer Wrapping and Subpixel Boundary Inaccuracies > In 16-bit x86 code, registers wrap at 65,536 (`0xFFFF`), and signed comparisons (`JL`, `JLE`) behave differently from unsigned comparisons (`JB`, `JBE`). > * *Failure Example:* Translating `CMP AX, BX / JLE target` to high-level code as `if ax < bx:` instead of `if ax <= bx:` causes boundary conditions (e.g., surface water line, platform edge) to trigger one pixel early or late. > * *Mitigation:* Always enforce explicit 16-bit masking (`val & 0xFFFF`) on accumulator registers and translate conditional jumps strictly according to x86 flag definitions (ZF, SF, OF, CF). > [!TIP] > ### Pitfall 5: Display Refresh Rate Bottlenecks (70 Hz vs. 35 Hz) > Standard VGA Mode 13h and Mode X refresh at **70.086 Hz** ($25.175\text{ MHz} / (800\text{ clocks} \times 449\text{ lines})$). The DOS main loop polls VBLANK (`IN AL, 0x3DA / TEST AL, 8`). > * *The Issue:* On original 286/386 hardware, complex planar rendering often exceeded 14.2 milliseconds per frame. As a result, the game regularly missed the vertical retrace and dropped to **35 FPS** (every second VBLANK). > * On a modern machine running the code at an unconstrained 70.086 Hz, the game will feel significantly faster than players remember from childhood hardware. > * *Mitigation:* Keep internal game simulation strictly deterministic at the native rate (70.086 Hz) using a fixed-timestep clock, but provide a configurable timing mode or half-rate throttle for players seeking period-accurate 386 performance. --- ## 6. Deterministic Fixed-Timestep Clock Template Modern host operating systems run with variable refresh rates, multitasking jitter, and sleep latency. The game loop must decouple wall-clock time from internal simulation ticks: ```python import time class FixedTimestepClock: """Deterministic game loop clock modeled on DOS hardware VBLANK timing.""" def __init__(self, target_hz: float = 70.0866, max_ticks_per_frame: int = 4): self.tick_ns = round(1e9 / target_hz) self.max_ticks = max_ticks_per_frame self.accumulator_ns = 0 self.total_ticks = 0 def advance(self, dt_seconds: float) -> int: """Accumulates elapsed wall-clock time and returns logic ticks to execute.""" dt_ns = max(0, round(dt_seconds * 1e9)) self.accumulator_ns += dt_ns # Calculate pending whole ticks ticks = self.accumulator_ns // self.tick_ns if ticks > self.max_ticks: # Drop backlog to prevent spiral of death during backgrounding / debug pause ticks = self.max_ticks self.accumulator_ns = 0 else: self.accumulator_ns -= ticks * self.tick_ns self.total_ticks += ticks return ticks def reset(self): """Call upon unpausing or level reload to prevent time bursts.""" self.accumulator_ns = 0 ``` --- ## 7. Comprehensive End-to-End Verification Checklist Before certifying a DOS reconstruction as **1:1 Bit-Exact Complete**, verify all 8 test suites: ### 1. Long-Run Physics Determinism * [ ] Run the differential harness over $\ge 50,000$ continuous ticks with randomized and scripted inputs. * [ ] Assert $0$ mismatches across position ($X, Y$), velocity ($VX, VY$), and collision states. ### 2. Rare State Machine Edge-Cases * [ ] Verify actor knockback, hitstun, and recovery states. * [ ] Verify entity destruction, rubble/shrapnel countdown timers, and despawn cycles. * [ ] Verify environmental submergence, buoyancy physics, and out-of-bounds checks. ### 3. Complete Audio Call-Site Parity * [ ] Conduct a binary-wide search for all `call play_sfx` instruction sites. * [ ] Construct automated integration tests ensuring every single audio call site dispatches its associated SFX event during gameplay. ### 4. Audio Driver Parity & FM Synthesis * [ ] Capture raw OPL register streams (`0x388/0x389`) from the x86 binary and assert identical byte output from the modern driver. * [ ] Verify PIT timer accumulation ($192\text{ Hz}$ or equivalent) to prevent tempo dragging. ### 5. Session Lifecycle & Input Abort * [ ] Test player death down to 0 lives: verify game over transition, delay timers, and failure audio. * [ ] Test mid-game input abort (e.g., ESC key): verify clean score carry-over and life zeroing without process termination. * [ ] Test pause toggle: verify audio continuity during pause and zero physics jumps on unpause. ### 6. Persistence & Serialization * [ ] Save and load high-score tables and game save files. * [ ] Assert identical byte-for-byte outputs against legacy DOS disk formats (including XOR masks and padding). ### 7. Game Completion (Outro & Victory) * [ ] Simulate completion of the final level. * [ ] Verify that victory cutscenes execute their complete animation script and play the authentic ending music track. ### 8. Cooperative & Difficulty Modes * [ ] Verify multi-player modes: test split input handling, shared life pools, and player-to-player collision boxes. * [ ] Verify difficulty modifiers: test crash thresholds, fuel/energy drain rate multipliers, and score bonuses. --- ## 8. Summary A true 1:1 reconstruction does not rely on approximations. By establishing an x86 CPU differential harness, cataloging every procedure, intercepting hardware registers, and enforcing strict dual-execution assertions, legacy DOS titles can be preserved in modern software stacks with absolute mathematical fidelity.