Skip to content

Commit

Permalink
adding WSBR0001 header to song chunk, removing some useless header fr…
Browse files Browse the repository at this point in the history 
…om Maj7 chunk. some code cleanup.

The primary justification for the WSBR0001 header is issue #4. Any external players that want to detect/read a WS song blob can now have a method of doing so, with room for format versioning. So it's really not used within the WS codebase, but useful for players to understand the chunk. Nothing in the WS code is actually reading/parsing/reacting to this value.

I have chosen to use a character based version ("0001" instead of uint8_t[]{0,0,0,1}), because it can be read in a text editor easily, and is unlikely to lead to problems. If versions do eventually exceed this, it can be adapted (hex -> other format -> extend the header, etc.).

Apologies for yet another megacommit littered with changes of various relevance.
  • Loading branch information
@thenfour
thenfour committed Apr 11, 2023
1 parent c96681f commit 2b532f1
Show file tree
Hide file tree
Showing 8 changed files with 6,595 additions and 8,615 deletions.
76 changes: 76 additions & 0 deletions README-tenfour.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@



tenfour's changes for revision 2023
-----------------------------------

* project mgr
* more info shown in project mgr (compressed size, and breakdown by midi lane / device / track)
* ability to copy project info as text
* ability to copy complete render .hpp file
* warnings for unsupported features
* selectable options for song bounds
* Pitchbend and midi CC automation is now supported.
* wavesabre tag now marks the beginning of a song chunk
* EXE music project
* VST improvements
* Now using ImGui for its flexibility
* Ability to type in values
* Shift+drag = fine adjust
* Smasher now shows level meters for input, output, compression, and threshold.
* You can see now how compressible a device chunk will be.
* VST chunks are now different than minsize chunks; this means VSTs are stored as text-based JSON key-value pairs therefore:
* can be copied/pasted from clipboard, or saved/loaded from disk
* will remain compatible no matter how the minsize serialization format evolves.
* New devices
* Maj7 synthesizer (this is the biggest change IMO)
* Width device for stereo field adjustments including narrowing of bass freqs
* Scissor now supports tanh saturation
* Size optimizations
* MIDI LANES
* Track names can now have directives; the first directive is "#fixedvelocity" where velocities are left out of the chunk.
* De-interleaving event / midi note / velocity, in order to put like data with like.
* Time values now stored as var-length ints instead of int32.
* PARAM chunks
* Now storing params as diff of default value
* optimization stage lets the device set values explicitly to 0 when they're not used
* params are now stored as 16-bit signed values which are slightly more compressible than floats.
* defaults now stored in code as a single blob
* CODE SIZE
* Biquad filter / other hand-optimizations
* Using memset to set arrays of floats to 0
* Using a new method of accessing params which reduces amount of code (a few kb, compressed!), and unifies a lot of params (ParamAccessor)
* Rendering speed optimizations
* Deep tracks now prioritized for processing first
* Proper non-busy locking in the node graph runner (rewrite of graph runner)


Maj7
----

A polyphonic megasynthesizer specifically designed for 32kb executable music.

* 4 fully high quality (polyblep bandlimiting) oscillators
* Hard-sync and FM supported on all oscillators
* 4 samplers which can use loaded samples or GM.DLS, each with key zone (i.e. drum kit support)
* 4 LFOs, including noise with adjustable HP and LP
* 10 modulation envelopes (DAHDSR, one-shot, note retrig, each stage with adjustable curve)
* 40 user-configurable modulations
* Each with range, curve, mapping settings
* Each with a side-chain (think Serum) which attenuates the mod strength based on another mod source (e.g. modulate env -> volume, and side-chain with velocity)
* Full 4op FM matrix
* 2 stereo filters available, with key tracking frequency support
* 7 macro knobs (think NI Massive) which can be used as modulation sources
* It's fairly performant, but there are quality settings for big projects

So this is a LOT of features, and therefore it's a lot of code. Something like 9kb of code, compressed. On the other hand it replaces at least 4 existing WaveSabre devices.

techniques

Wish list
---------

* pitch modulation in delay plugin
* tests


4 changes: 0 additions & 4 deletions Vsts/Maj7/Maj7Vst.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,6 @@ int GetMinifiedChunk(M7::Maj7* p, void** data)
// 0.0000001
M7::Serializer s;

s.WriteUInt32(M7::Maj7::gChunkTag);
s.WriteUByte((uint8_t)M7::Maj7::ChunkFormat::Minified);
s.WriteUByte(M7::Maj7::gChunkVersion);

auto defaultParamCache = GenerateDefaultParamCache();

for (int i = 0; i < (int)M7::ParamIndices::NumParams; ++i) {
Expand Down
24 changes: 1 addition & 23 deletions Vsts/Maj7/Maj7Vst.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,27 +37,7 @@ inline int Maj7SetVstChunk(M7::Maj7* p, void* data, int byteSize)
return 0;
}

bool tagSatisfied = false;
bool versionSatisfied = false;
while (true) {
auto ch = maj7Obj.GetNextObjectItem();
if (ch.mKeyName == "Tag") {
if (ch.mNumericValue.Get<DWORD>() != M7::Maj7::gChunkTag) {
return 0; //invalid tag
}
tagSatisfied = true;
}
if (ch.mKeyName == "Version") {
if (ch.mNumericValue.Get<uint8_t>() != M7::Maj7::gChunkVersion) {
return 0; // unknown version
}
versionSatisfied = true;
}
if (ch.IsEOF())
break;
}
if (!tagSatisfied || !versionSatisfied)
return 0;
maj7Obj.EnsureClosed();

auto paramsObj = doc.GetNextObjectItem(); // assumes these are in this order. ya probably should not.
if (paramsObj.IsEOF()) {
Expand Down Expand Up @@ -156,8 +136,6 @@ class Maj7Vst : public WaveSabreVstLib::VstPlug

auto maj7Element = doc.Object_MakeKey("Maj7");
maj7Element.BeginObject();
maj7Element.Object_MakeKey("Tag").WriteNumberValue(M7::Maj7::gChunkTag);
maj7Element.Object_MakeKey("Version").WriteNumberValue(M7::Maj7::gChunkVersion);
maj7Element.Object_MakeKey("Format").WriteStringValue(diff ? "DIFF values" : "Absolute values");

auto paramsElement = doc.Object_MakeKey("params");
Expand Down
106 changes: 5 additions & 101 deletions WaveSabreConvert/Serializer.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -281,19 +281,14 @@ public void WriteMidiLane(int iMidiLane, byte n)
}
}

// things not worth doing:
// creating a note map which indexes note values to a map. it actually causes bloat.
BinaryOutput CreateBinary(Song song)
{
BinaryOutput writer = new BinaryOutput();
//int minTimeFromLastEvent = int.MaxValue;
//int maxTimeFromLastEvent = int.MinValue;
Dictionary<int, int> timestampByteCounts = new Dictionary<int, int>();
timestampByteCounts[0] = 0;
timestampByteCounts[1] = 0;
timestampByteCounts[2] = 0;
timestampByteCounts[3] = 0;
timestampByteCounts[4] = 0;

// song header.
// TODO: Organize header version numbers in some structured way. But currently it's not easy to predict what
// would be the best way to do this so stay simple until the next change is required, and clean this up.
writer.Write(("WSBR0001".Select(ch => Convert.ToByte(ch)).ToArray()));

// song settings
writer.Write(song.Tempo);
Expand All @@ -309,69 +304,6 @@ BinaryOutput CreateBinary(Song song)
writer.Write(device.Id, device.Chunk);
}

// serialize note map. this maps note value key to a real note value. the idea is that the most common notes will output as 0.
// considering tracks like drums / hihats / arps / staccato basslines have so many repeated notes, this works.
Dictionary<int, int> noteCounts = new Dictionary<int, int>();
for (int i = 0; i < 128; ++i)
{
noteCounts[i] = 0;
}

for (int iMidiLane = 0; iMidiLane < song.MidiLanes.Count; ++iMidiLane)
{
var midiLane = song.MidiLanes[iMidiLane];
foreach (var e in midiLane.MidiEvents)
{
switch (e.Type)
{
case EventType.NoteOff:
case EventType.NoteOn:
noteCounts[e.Note]++;
break;
}
}
}

// note map maps key => note value
//var s = noteCounts.OrderByDescending(kv => kv.Value).ToList(); // list of [notevalue, count] ordered desc by count
//Dictionary<int, int> noteValToK = new Dictionary<int, int>(); // map notevalue to serializeable key
//int noteMapSize = s.Count(kv => kv.Value > 0);
//writer.Write(noteMapSize);
//for (int i = 0; i < noteMapSize; ++i)
//{
// writer.Write(s[i].Key);
//}
//for (int i = 0; i < 128; ++i)
//{
// noteValToK[s[i].Key] = i;
//}
//for (int i = 0; i < noteMapSize; ++i)
//{
// writer.Write(s[i].Key);
// noteValToK[s[i].Key] = i;
//}

var usedNotes = noteCounts.Where(kv => kv.Value > 0).Select(kv => kv.Key).OrderBy(o => o).ToList();
//var noteValueToKMap = noteCounts.OrderBy(kv => kv.Value > 0 ? 0 : 1).ThenBy(kv => kv.Key).Select(kv => kv.Key).ToList(); // list of [notevalue, count] ordered by note value (no special ordering basically)
int noteMapSize = usedNotes.Count;
//writer.Write((byte)noteMapSize);
for (int i = 0; i < noteMapSize; ++i)
{
//writer.Write((byte)usedNotes[i]);
}
var noteValueToKMap = new Dictionary<int, int>(); // inverse of usedNotes; maps notevalue to index.
Dictionary<int, int> noteKeyCounts = new Dictionary<int, int>();
for (int noteVal = 0; noteVal < 128; ++noteVal)
{
noteValueToKMap[noteVal] = 0;
int foundIndex = usedNotes.FindIndex(v => v == noteVal);
if (foundIndex >= 0)
{
noteValueToKMap[noteVal] = foundIndex;
}
noteKeyCounts[noteVal] = 0;
}

// serialize all midi lanes
writer.Write(song.MidiLanes.Count);
for (int iMidiLane = 0; iMidiLane < song.MidiLanes.Count; ++ iMidiLane)
Expand All @@ -390,28 +322,6 @@ BinaryOutput CreateBinary(Song song)
{
Console.WriteLine($"Negative time deltas break things. wut?");
}
//writer.WriteMidiLane(iMidiLane, e.TimeFromLastEvent);
// some things:
// 1. delta times tend to be either very big or very small. suggests the need for a variable length integer.
// 2. the event type is 2 bits and we're trying to pack them somewhere.
// 3. delta times can be pretty big, we should support uint32 sizes.

// so timefromlastevent is actually a 31-bit signed int where only positive part is used. but in theory we should
// support the full 32-bits. it's not a problem when using varlen ints.
// byte stream: [eec-----][c-------][c-------][c-------][00------]
// ||| | | |
// ||| | | |set to 1 to read the next byte. if 0, then the number is 26-bits (0-67,108,863). if 1 then it's 32 bits
// ||| | |set to 1 to read the next byte. if 0, then the number is 19-bits (0-524,287)
// ||| |set to 1 to read the next byte. if 0, then the number is 12-bits (0-4095)
// |||set 1 to read the next byte. if 0 then it's a 5-bit value 0-31
// ||
// ||event type

// those bits get placed into the int in that order (first bit = highest bit; we just apply & shift as we go.
//const uint _5bitMask = (1 << 5) - 1;
//const uint _12bitMask = (1 << 12) - 1;
//const uint _19bitMask = (1 << 19) - 1;
//const uint _26bitMask = (1 << 26) - 1;
int eventType = (int)e.Type;
if (eventType < 0 || eventType > 3)
{
Expand All @@ -437,11 +347,6 @@ BinaryOutput CreateBinary(Song song)
{
continue;
}
//int newCurrent = e.Velocity;
//byte b = Utils.ByteDeltaEncode(currentVel, newCurrent);
//currentVel = newCurrent;
//writer.WriteMidiLane(iMidiLane, b);
//writer.WriteMidiLane(iMidiLane, (byte)(e.Velocity / 2));
writer.WriteMidiLane(iMidiLane, (byte)e.Velocity);
break;
case EventType.CC:
Expand All @@ -455,7 +360,6 @@ BinaryOutput CreateBinary(Song song)

// serialize each track
writer.Write(song.Tracks.Count);
//foreach (var track in song.Tracks)
for (int iTrack = 0; iTrack < song.Tracks.Count; ++ iTrack)
{
var track = song.Tracks[iTrack];
Expand Down
14 changes: 0 additions & 14 deletions WaveSabreCore/include/WaveSabreCore/Maj7.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -258,14 +258,6 @@ namespace WaveSabreCore

struct Maj7 : public Maj7SynthDevice
{
static constexpr DWORD gChunkTag = MAKEFOURCC('M', 'a', 'j', '7');
static constexpr uint8_t gChunkVersion = 0;
enum class ChunkFormat : uint8_t
{
Minified,
Count,
};

static constexpr size_t gOscillatorCount = 4;
static constexpr size_t gFMMatrixSize = gOscillatorCount * (gOscillatorCount - 1);
static constexpr size_t gSamplerCount = 4;
Expand Down Expand Up @@ -501,12 +493,6 @@ namespace WaveSabreCore
virtual void SetChunk(void* data, int size) override
{
Deserializer ds{ (const uint8_t*)data };
auto tag = ds.ReadUInt32();
if (tag != gChunkTag) return;
auto format = (ChunkFormat)ds.ReadUByte();
if (format != ChunkFormat::Minified) return;
auto version = ds.ReadUByte();
if (version != gChunkVersion) return; // in the future maybe support other versions? but probably not in the minified code
SetMaj7StyleChunk(ds);
for (auto& s : mSamplerDevices) {
s.Deserialize(ds);
Expand Down
4 changes: 2 additions & 2 deletions WaveSabreExecutableMusicPlayer/config.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

//static constexpr uint32_t gSongPaddingMS = 3000; // additional padding to add at the end of the track to account for fadeouts whatever
static constexpr uint32_t gBlockSizeSamples = 256; // don't try to bite off too much; modulations will be too loose. try to replicate DAW behavior. Note these are samples not frames. 128 sample buffer probably means 256 here.
#define TEXT_INTRO "(song title here)\r\nby tenfour/RBBS for Revision 2023\r\n\r\n"
#define TEXT_INTRO "Bright Velvet\r\nby tenfour/RBBS for Revision 2023\r\n\r\n"

// these are not about optimization & compiler options but about features.
// enable one at a time.
Expand Down Expand Up @@ -33,7 +33,7 @@ struct
COLORREF PrecalcTextShadowColor = PrecalcProgressForeground;
} gColorScheme;

static constexpr Rect grcWindow{ 0, 0, 1400, 600 };
static constexpr Rect grcWindow{ 0, 0, 500, 500 };
static constexpr Rect grcText{ grcWindow.Offset(20, 10) };// { 0, 0, 400, 400 };
static constexpr Rect grcWaveform{ grcWindow };// { 0, 0, 1400, 600 };
static constexpr uint32_t gGeneralSleepPeriodMS = 30;
Expand Down
Loading

0 comments on commit 2b532f1

Please sign in to comment.