Reverse engineering Packet Format(s)
This is a reference/guide that covers reverse engineering a DDON packet format (via transferring symbols from an old debug version of the game).
- IDA Pro 7.5 (+HexRays decompiler for x86/x64)
- PS4 module loader IDA plugin
- Dumped/unpacked version of the PC client (v03.04.007)
- Debug PS4 client (v02020005) [2016-12-21]
In addition, this document expects existing knowledge/experience with reverse engineering and IDA.
Special steps are needed to load the DWARF debug symbols for the PS4 version:
- Ensure the IDA PS4 module loader plugin is installed.
- Load the
DDBORBIS.elffile and allow it to fully load and finish auto-analysis. - Go to
Edit->Plugins->Load DWARF fileand select theDDORBIS.elffile.- This will take a long time, even on high-end computers. Be patient.
- Open the unpacked PC client with IDA and let the auto-analysis completely finish.
- Go to
View->Local Types, right click on entry area and selectInsert, then paste the following definition:
struct CPacket {
void *vft;
unsigned __int8 *data_start;
unsigned __int8 *data_end;
unsigned __int8 *read_offset;
unsigned __int8 *write_offset;
};- Go to the following addresses and rename the functions:
CPacket::ReadInt64 => 0x008BC660
CPacket::Read8byte => 0x008BC6A0
CPacket::Read4byte_ => 0x008BC480
CPacket::Read4byte => 0x008BC4D0
CPacket::ReadUint16 => 0x008BC440
CPacket::ReadByte => 0x008BC410
CPacket::ReadBool => 0x008BC6E0
CPacket::ReadLengthPrefixedMtString => 0x008BC5B0
CPacket::ReadBuffer => 0x008BC750
NOTE: These were what I named the functions in my IDB before the debug symbols were found.
Spend a moment to understand and take note of how these functions work, as you might need to RE many other type-specific reader functions. Using CPacket::Read4byte as an example:
The first argument is the class instance this pointer, and should be typed to CPacket*. The second argument is the output pointer:
The function gets the pointer to the current data index in the packet reader, swaps the endianness and stores it to the output points, then increments the reader and returns the number of bytes read.
First, take the packet name or ID and find it's handler or writer address on the following document:
https://github.com/Andoryuuta/DDON_RE/blob/master/packet_docs/GamePackets.md
In this example, we will use be reverse engineering S2C_STAGE_GET_STAGE_LIST_RES. The handler address is at 0x7fd9e0 in the PC client.
As this is a game server packet (S2C not L2C), this handler is a member method of the cSeedGameSvConnection, so I've called it cSeedGameSvConnection::OnPacket_S2C_STAGE_GET_STAGE_LIST_RES.
The first function argument is the this pointer, second argument is the CPacket* packetReader. Like such:
Find the first function call that uses the packetReader within the function. I've renamed it to S2C_STAGE_GET_STAGE_LIST_RES::Read:
After renaming and retyping the second argument to CPacket* packetReader, we are finally at the packet parser function.
The PS4 debug client is a much older version of the game, so the packet names have changed a bit. Search for different substrings of the PC packet name in the IDA functions window until you find the packet.
For example, the PC packet name S2C_STAGE_GET_STAGE_LIST_RES does not directly appear in the PS4 binary. However, searching for GET_STAGE_LIST_RES does yield results:
Unlike the PC handler, the PS4 handler directly parses the packet (without a sub parser function):
NOTE: The PS4's read functions (e.g. nPacket::Read_0, nPacket::Read_1, nPacket::Read_...) do not include the type or byte size in the name. These are auto-generated names from somewhere during the build process for this generic/templated C++ read function. I recommend renaming these functions to their correct types as determined by their second argument type.
For example, nPacket::Read_0 has an output pointer type of u16, so I will be renaming this to nPacket::Read_u16.
Now that we have the packet parser code located in both the PC and PS4 client, we can begin to compare the two and transfer names.
Right away, we can see that that there are three function calls that take our packet reader:
Based on the number of function calls that use the packet reader (and order), we can make an educated guess and copy the name of the nPacket::Read_CDataStageInfo_ function over to our PC client.
However, when we compare the first two fields there is a difference in size:
- PC:
- 4 byte
- 4 byte
- PS4:
- 2 byte (uint16)
- 1 byte (uint8)
This discrepancy is due to change in the network protocol over the years. These two fields are parsed at the start of every packet from server -> client, and represent error and result (some other functions have these properly named on the PS4 client). At some point in the past few years, these fields were updated to both being uint32 (as we see on the PC client).
Now that we have the first two fields reconciled, we can look deeper into the nPacket::Read_CDataStageInfo_ function. This function takes in a MtTypedArray, which means it's a list of serialized objects.
It is standard for these MtTypedArray serializers to start with a uint32 field containing the count of objects in the list/array:
Comparing further into the function, we find that the PC version only has 1 function call using the packetReader, while the PS4 version has 6 calls:
This is due to function inlining. The single function call on the PC client has been directly expanded into the function on the PS4 version. If we go into the call on the PC client and compare again, we see code that is much more similar in nature:
We now know that the PC function (sub_77A130) is used to read a single CDataStageInfo entity, and name it as such (CPacket::Read_CDataStageInfo) .
On the PS4 side, we now want to get symbols to determine what these fields actually represent. To do so, we simply need to change the type of the allocated CDataStageInfo memory. Click on the output variable of the operator_new call, press the y hotkey and enter the type CDataStageInfo*:
After the appropriate type is set, we can now see each field name:
Then we can copy over the field & function names as appropriate (again -- educated guesses based on the datatype sizes and order. If a field was added, removed, or had it's data type changed, these would not align perfectly like this).
Now that we have the fields for the CDataStageInfo figured out, we repeat the same process for the sub-entity CDataStageAttribute:
Oh no! We've run into an issue, the PC version has 11 bools, while the PS4 version only had 9. How do we know which field is which?
We cant.
This is one of the many scenarios in which something has changed in the netcode, but we cannot easily transfer symbols between the two. For now, we will just note that the PC version has 11 bools, and document the possible names from the PS4 client. Later on, we can manually try each bool and figure out why functionality it causes the client to take, and work from there.
After transferring the symbols (where possible) from the PS4 client, we can now note down the exact format (field sizes and order) for the PC client:
typedef struct {
uint32 Error;
uint32 Result;
List<CDataStageInfo> StageInfoList;
} S2C_STAGE_GET_STAGE_LIST_RES;
typedef struct {
uint32 ID;
uint32 StageNo;
uint32 RandomStageGroupID;
uint32 Type;
CDataStageAttribute StageAttribute;
bool IsAutoSetBloodEnemy;
} CDataStageInfo;
typedef struct {
/*
Changed since PS4, unable to know where bools were added in the struct.
Possible names:
IsSolo
IsEnablePartyFunc
IsAdventureCountKeep
IsEnableCraft
IsEnableStorage
IsEnableStorageInCharge
IsNotSessionReturn
IsEnableBaggage
IsClanBase
*/
bool unk0;
bool unk1;
bool unk2;
bool unk3;
bool unk4;
bool unk5;
bool unk6;
bool unk7;
bool unk8;
bool unk9;
bool unk10;
} CDataStageAttribute;Each of these structs should then be converted into a Entity serializer struct/class at Arrowgene.Ddon.Shared/Entity/Structure and then registered within Arrowgene.Ddon.Shared/Entity/EntitySerializer.cs.