Bug: Wizardry 7 crashes before getting to intro videos

[dicene]

Describe the bug
Playing around with Wizardry 7 decompilation in my free time and stumbled on this extremely interesting project. I was hoping that since Wiz7 was released the same year as Dune it would be fairly similar as far as compatibility goes, but that doesn't seem to be the case.

Upon running Spice86 with the Wizardry 7 (dos version, not the Windows focused Gold version) executable, from within the folder Wizardry 7 is installed in, Spice86 is crashing without getting far enough to even see the Intro videos begin to play.

If there's any additional info I can provide beyond this, let me know. I'm assuming this is happening because of un-implemented functionality and if I can get pointed in the right direction, it's possible I might be able to contribute something to the project towards that end.

To Reproduce
Steps to reproduce the behavior:

1. Attempt to run Wizardry 7's executable DS.EXE in Spice86. (the command I used to run it is C:\DSAVANT>c:\spice86\spice86 --Ems -l -e ds.exe.
2. DS.EXE begins but doesn't make it far enough to display any graphics or play any audio.

Expected behavior
DS.EXE to make it further into the application before crashing.

Desktop:

• OS: Windows 10
• Spice86 from the current latest tag. This commit: 94ec50d
• This is using the dos version of Wizardry 7 obtained from Steam and used with DOSBox.

Additional context
Attached is a Verbose log from my attempt to run it. I believe the --Ems option is necessary for Wiz7, but no other options that I used seemed to make a difference on getting it to run.

spice86_wiz7_verboselog.txt

[maximilien-noal]

Hello,

Thank you for this report.

Indeed there is a lot of work still in making Spice86 work with more games. PRs are welcome since the primary focus is set on Dune until it is completely reverse engineered.

There are a lot of code bases to draw inspiration from, such as:

• dosbox-staging for DOS / bios / hardware emu
• glabios for bios level code
• Freedos kernel for DOS interrupts
• otvdm for DOS int21h and other DOS interrupts

And documentation such as EGA/VGA a programmer's reference guide, and Undocumented DOS books.

Do you have a copy to of the game I can download ? (edit: I'll get it from Steam)

Also did you try debugging it with gdb and compare it with dosbox-staging debugger ?

The -f option is also useful in order to catch unemulated ports early.

I have a branch named fix/Detroit that implemented more dos and bios services. It doesn't fix Detroit (the game) ATM, but it might be useful as a headstart.

[maximilien-noal]

Indeed, the branch fix/detroit has a few improvements compared to master with this game:

• no undefined interrupt handler
• no InvalidOpCodeException
• the logs are saved in Spice86.log

Still, the game exits really early with this message:

Unable to load display driver

Also, I tested the game with dosbox staging and the following settings:

ems=false
xms=false
umb=false

and it ran fine in DOSBox.

[dicene]

I've done a little poking around and I think I'm getting a little more familiar with DOS applications and Spice. My initial hangup with reversing Wizardry 7's DOS version is that I'm not very familiar with DOS specific details, like the various INT 21 options, and my confusion trying to understand what I'm seeing when I open DOS applications in Ghidra or Radare2. Regular PE applications look logical by comparison. I've had a LOT more luck with reversing the "Gold" version of Wizardry7 which is a standard Windows application and basically functions just fine on modern Windows. There's even some old remnants of the DOS code in there, functions that read from original VGA files instead of the BMPs that Gold uses.

The first bump I hit was the unhandled CLTS command. I think I see enough info to figure out how to implement something simple like a register being cleared. I tried to just skip it without making any other change and ran into this file reading attempt that is a little bit suspect...

Edit 1: Figured out with procmon that it should be loading "vga.drv" and substituting that in for the garbage string it had gets me closer, until I hit this guy Int was called but vector was not initialized for vectorNumber=0x8 ::::: Cycles=94401 CS:IP=0x0:0x57A7/0x57A7 EAX=0x0 EBX=0x5 ECX=0x0 EDX=0x100 ESI=0x2D87 EDI=0xFA00 EBP=0xFC96 ESP=0xFC92 SS=0x1FF6 DS=0x1FF6 ES=0x0 FS=0x0 GS=0x0 flags=0x7246 ( I Z P ). I suspect that fixing the filename is just fixing a symptom and there's actually something going on wrong prior to that that is causing the string to be read from the wrong place or something. I'll try and do more comparing vs DOSBox when I can.

One thing I'm a little confused by. While stopped at the breakpoint for that OpenFile call, I tried to pin down where I was in the code. I checked the CS and IP and it pointed me to like f000:0044 or something like that. I think that translates to f0044 (which was also the Physical IP when I check that), but that doesn't line up with DS.EXE which isn't long enough to have an f0044. I check the bytes themselves with _memory.Ram.Read(f0044) - _memory.Ram.Read(f0048) and find bytes fe 38 21 cf fe, but I can't find those bytes in DS.EXE or any of the other 3 files that it had loaded by that point (vinit.ovr, SCENARIO.HDR, and play.drv). Am I misunderstanding something obvious here?

Edit 2: With the little hacks I've made to it, it does appear to get far enough to blank the screen finally, so that's cool. 😅

[maximilien-noal]

I've done a little poking around and I think I'm getting a little more familiar with DOS applications and Spice. My initial hangup with reversing Wizardry 7's DOS version is that I'm not very familiar with DOS specific details, like the various INT 21 options, and my confusion trying to understand what I'm seeing when I open DOS applications in Ghidra or Radare2.

Yeah, Win32 uses a flat memory model, and applications are... a lot more civilized there.

On DOS, the application uses a Segmented Memory model, and a DOS application controls everything, and sometimes you deal with hand written ASM, not something easier to read like a C compiler would produce. Fun.

CLTS

Do you mean a CLS command ?
Edit: Oh it's an x86 instruction to go into protected mode.

About VGA:

On top of the "VGA Black Book" (see "Michael Abrash's Graphics Programming Black Book Special Edition"), and looking at the DOSBox Staging or PCem code, the Bochs VGA Bios is also a good reference: https://www.nongnu.org/vgabios/

Thankfully the VGA/CGA/EGA/Text mode implementation, while not 100% complete, is pretty extensive thanks a lot to @JorisVanEijden

I suspect that fixing the filename is just fixing a symptom and there's actually something going on wrong prior to that that is causing the string to be read from the wrong place or something. I'll try and do more comparing vs DOSBox when I can.

That would be my guess too.

Am I misunderstanding something obvious here?

The application can relocate code or have self modifying code, which is why it doesn't correspond to the file(s) on disk. Also, the file on disk might be packed too. That's why you can set the entry point to start you jourrney from there. Your point of reference is the program's entry point segment.
All of this is also why Spice86 produces a memory dump.

INT21H

Yes, it's one of the most important interrupt handlers of DOS. A good source of comparison is FreeDOS and DOSBox Staging.
There's a lot of INT21H functions to implement still (most notably TSR support and 0x4B LOAD AND EXEC), and more generally a lot of BIOS and DOS functions missing too.

Therefore, a breakpoint on the line of code where the emulator doesn't find an interrupt handler and throws an exception could be helpful.
The -f command line option also forces the emulator to stop if a port write or read is unimplemented.
Also a breakpoint in the GetInterruptVector method of our INT21H handler helps catch things that are not implemented there yet.
If a game gets 0:0 from there its bad news.

There is also probably entries missing in our interrupt table, which is something to keep in mind.
If the game uses the FPU, we only report it as not available.

The branch fix/detroit implements FPU emulation placeholders (even if the game itself does not use the FPU) that this game overrides (with the help of Get/Set interrupt vectors methods) with the MS C runtime software FPU implementation.
(DOSBox doesn't seem to define that but the game Detroit still finds interrupt vectors to replace there. I haven't figured this out yet.)

All of this to say, while I think this game has unique needs, I suspect there is a common set of causes between those too games not booting up.

Edit: If you prefer to read Java instead of C++, there is a modern version of JDOSBox here: https://github.com/Tennessene/jDOSBox
Java has far fewer keywords than C++ or C#, I find it easier to follow sometimes.

[JorisVanEijden]

The first bump I hit was the unhandled CLTS command. I think I see enough info to figure out how to implement something simple like a register being cleared. I tried to just skip it without making any other change and

Encountering a CLTS instruction in a game is probably a sign of mis-disassembly. Like trying to disassemble data or starting disam in the middle of an instruction.

ran into this file reading attempt that is a little bit suspect... Figured out with procmon that it should be loading "vga.drv"

Indeed, looks like the pointer to the string is not containing the correct value. It can be tricky to find out where the origin lies.

and substituting that in for the garbage string it had gets me closer, until I hit this guy Int was called but vector was not initialized for vectorNumber=0x8 ::::: Cycles=94401 CS:IP=0x0:0x57A7/0x57A7 EAX=0x0 EBX=0x5 ECX=0x0 EDX=0x100 ESI=0x2D87 EDI=0xFA00 EBP=0xFC96 ESP=0xFC92 SS=0x1FF6 DS=0x1FF6 ES=0x0 FS=0x0 GS=0x0 flags=0x7246 ( I Z P ). I

This indicates that INT 08 was triggered, but the IVT (Interrupt Vector Table) does not contain a valid address to call for that interrupt number.

One thing I'm a little confused by. While stopped at the breakpoint for that OpenFile call, I tried to pin down where I was in the code. I checked the CS and IP and it pointed me to like f000:0044 or something like that. I think that translates to f0044 (which was also the Physical IP when I check that), but that doesn't line up with DS.EXE which isn't long enough to have an f0044. I check the bytes themselves with _memory.Ram.Read(f0044) - _memory.Ram.Read(f0048) and find bytes fe 38 21 cf fe,

The spice emulator has a special fake instruction "FE 38 xx" that triggers a switch from emulating assembly to executing C# code. Specifically it will run the callback that is registered for the number in xx, in your case 21.
The Spice "bios" puts one of these at F000:0000 for every registered interrupt, and fills the IVT with pointers to these addresses.

So let's say the cpu encounters an INT 21 instrcution. It will look into the IVT for a pointer for that interrupt. It will find F000:0044 there and jump to it. There it encounters the FE38 21 code so it looks into it's own registered callbacks and will find Spice86.Core.Emulator.InterruptHandlers.Dos.DosInt21Handler there and will call the Run() method on it.

[maximilien-noal]

@dicene any news ?