A blazing-fast DBC/DB2 reader for World of Warcraft. Name is totally not stolen from LordJZ's old 4.3.4 library.
The library supports WDBC (Vanilla+), WDB2 (Cata+), WDB5 (WoD+), and WDB6 (Legion+) file formats.
Since PTR build 7.3.0.24473, the format of WDB6 files has once again changed - but this time, without a version bump. The solution the library implements to work around it is designed in an attempt to maintain backwards-compatibility with DB2 files from previous builds. This was implemented in commit e346d6565, and while that build fixes a relatively big bug in WDB2 handling, it will not be deployed to NuGet at least until 7.3.0 hits live.
TL;DR: If you are not actively datamining PTR, stick to the NuGet package. I can only hope Blizzard pushes out WDB7 before 7.3.0 rolls out, for I have no idea how to efficiently handle this sudden change. -- 8/20/17
Regular instantiation goes by
var instance = new Storage<AreaTriggerEntry>("AreaTrigger.db2");Note that Storage<T> behaves like a Dictionary<int, T>.
For all file formats, you should provide reference-type structures to the library. Fields (and not properties, though that is subject to changes in the future) will get loaded in the order they are defined.
Bidimentional arrays are not supported; neither are collections.
Nothing special here, just mark arrays with MarshalAsAttribute, specifying the size.
public sealed class AreaTableEntry
{
[MarshalAs(UnManagedType.ByValArray, SizeConst = ...)]
public uint[] Flags;
public string ZoneName;
public float AmbientMultiplier;
public string AreaName;
public ushort MapID;
...
}Due to the nature of the WDB5 format, arrays do not always need an explicit size defined through attributes. Size is most of the time inferred from the file.
However, the library has difficulty to determine the size of arrays at the end of the record, because every record in a WDB5 file is aligned to the size of its largest field. You will need to decorate your array with MarshalAsAttribute most of the time, but if all the fields in that file are of the same size, the attribute will be ignored.
To put it simply, decorating an array with MarshalAsAttribute is necessary if your structure ends with that array.
public sealed class AreaTableEntry
{
// MarshalAsAttribute here is ignored
[MarshalAs(UnManagedType.ByValArray, SizeConst = ...)]
public uint[] Flags;
public string ZoneName;
public float AmbientMultiplier;
public string AreaName;
public ushort MapID;
...
// Necessary 9 out of 10 times
[MarshalAs(UnManagedType.ByValArray, SizeConst = ...)]
public int[] Element;
}Everything related to WDB5 still applies. This format mostly introduces field reordering, because of the birth of a « common » block (which I will now refer to as « off-stream »). Given the added complexity this introduces, you should notice a rough 30% loading speed decrease, averaging. I will wait until I have more structures available to run proper benchmarks.
Starting with version 1.1.0.0, DBFilesClient.NET now allows you to use somewhat complex types in your structures declaration.
To expand on the feature, simply declare a type that inherits IObjectType<T>, where T is the value type of the underlying type on the record.
A common example usage of such a feature would be foreign keys.
public class ForeignKey<T, U> : IObjectType<U>
{
public ForeignKey(U underlyingValue) : base(underlyingValue) { }
public T Value => ...;
}Which you would use like this:
public sealed class AreaTableEntry
{
...
public ForeignKey<MapEntry, ushort> MapID;
...
}With added effort, assuming you had access to some sort of representation of the CAS file system, you would be able to map a field to a BLTE file entry.
Other examples of IObjectType<T> usages can be found under the test project.
Here are a few examples (randomly selected) of loading speeds for various files. Expected record count in the tables below is calculated depending on field meta:
- If the file has an offset map, we count there.
- Otherwise, we use the record count in header.
We then add the amount of entries in the copy table.
Seed measurements as of commit af4909c4927d7fdec51660e9b029b13d92cfa39d
Made on an i7-7700HQ with 16 GB of RAM (SSD).
File name Average time to load Minimum time Maximum time Record count
------------------------------------------------------------------------------------------------------------
CreatureDisplayInfo 00:00:00.0604927 00:00:00.0537344 00:00:00.0668479 64001
CriteriaTree 00:00:00.0318721 00:00:00.0201606 00:00:00.0481298 45800
Item 00:00:00.0266174 00:00:00.0201141 00:00:00.0378913 122686
ItemModifiedAppearance 00:00:00.0249730 00:00:00.0135512 00:00:00.0490496 81961
ItemSearchName 00:00:00.0789017 00:00:00.0626334 00:00:00.1036024 79394
ItemSpecOverride 00:00:00.0079788 00:00:00.0058986 00:00:00.0161451 38956
SoundKit 00:00:00.0656281 00:00:00.0535618 00:00:00.0776808 74960
SpellCastTimes 00:00:00.0019794 00:00:00.0001766 00:00:00.0179368 132
SpellCategories 00:00:00.0084619 00:00:00.0067918 00:00:00.0110348 41938
SpellEffect 00:00:00.4155679 00:00:00.3989355 00:00:00.4488714 256034
Spell 00:00:00.2458318 00:00:00.2188421 00:00:00.2792958 171012
SpellInterrupts 00:00:00.0219506 00:00:00.0127948 00:00:00.0357718 50246
SpellItemEnchantmentCondition 00:00:00.0004443 00:00:00.0003922 00:00:00.0006907 0
SpellMisc 00:00:00.1156524 00:00:00.1025456 00:00:00.1197430 171805
SpellXSpellVisual 00:00:00.0549506 00:00:00.0424872 00:00:00.0714835 104795
TaxiPathNode 00:00:00.0287585 00:00:00.0230101 00:00:00.0396076 71763
WMOAreaTable 00:00:00.0181253 00:00:00.0113962 00:00:00.0277448 39087
Speed measurements as of commit f3d097706200a8f50b2236d3f68877e310e1f141
Measurements were made on an i3-2310M with 8 GB of DDR3 RAM (HDD).
File name Average time to load Minimum time Maximum time Record count
------------------------------------------------------------------------------------------------------------
CreatureDisplayInfo 00:00:00.0254500 00:00:00.0187491 00:00:00.0368231 62608
CriteriaTree 00:00:00.0203885 00:00:00.0119250 00:00:00.0289922 43746
Item 00:00:00.0155547 00:00:00.0121462 00:00:00.0249945 118770
ItemModifiedAppearance 00:00:00.0152652 00:00:00.0072048 00:00:00.0218214 79494
ItemSearchName 00:00:00.0470748 00:00:00.0370949 00:00:00.0605512 77309
Item-sparse 00:00:00.1563485 00:00:00.1467743 00:00:00.1667554 94843
ItemSpecOverride 00:00:00.0038236 00:00:00.0022122 00:00:00.0122970 36656
SoundKit 00:00:00.0422319 00:00:00.0309875 00:00:00.0479300 69770
SpellCastTimes 00:00:00.0000941 00:00:00.0000809 00:00:00.0001874 131
SpellCategories 00:00:00.0101107 00:00:00.0040866 00:00:00.0213984 39917
SpellEffect 00:00:00.2042076 00:00:00.1916380 00:00:00.2202517 241443
Spell 00:00:00.1686459 00:00:00.1529934 00:00:00.2087546 161626
SpellInterrupts 00:00:00.0139325 00:00:00.0071885 00:00:00.0253924 47680
SpellItemEnchantmentCondition 00:00:00.0000229 00:00:00.0000157 00:00:00.0000785 0
SpellMisc 00:00:00.0602413 00:00:00.0479208 00:00:00.0774596 162447
SpellXSpellVisual 00:00:00.0181881 00:00:00.0128776 00:00:00.0328005 99374
TaxiPathNode 00:00:00.0215235 00:00:00.0157172 00:00:00.0290410 70052
WMOAreaTable 00:00:00.0129732 00:00:00.0076368 00:00:00.0204752 38163
File specs for the various flavors of DB2 and DBC files can be found here.
In no particuliar order:
- #modcraft on QuakeNet.
- Kevin Montrose for Sigil (up to version 1.0.0.2).