Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scanning Architecture Refactor + Additional CacheHandler Documentation #217

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

sonicfind
Copy link
Collaborator

@sonicfind sonicfind commented Sep 30, 2024

  • Add DTAEntry type that siphons the old ParseDTA code - providing an avenue to pre-loading information for a DTA node before loading a new song entry.

  • Streamline the hierarchy of construction to the most basic form. All the processing of data that produces the member variables for new entries will be performed solely through static functions. Additionally, we will now mark all the values in any of the SongEntry types as readonly. This was already the case for the "ini` subtypes, but the RBCON structures prior to this change required the live modification of an existing entry. The only exception is Song Length, as processing a missing length value requires loading the mixer - which requires setting all subtype information.

  • Provide SongMetadata & RBMetadata types with constructors & Serialize() functions holding the code that used to exist in the SongEntry & RBCONEntry constructors and Serialize() functions.

  • Streamline the Serialize() functions so that they all share the same signature - allowing for proper virtualization. For the ini subtypes, this meant moving the relative path and EntryType boolean writes to the IniGroup SerializeEntries function instead.

  • To add to the above: instead of passing around a BinaryWriter wrapper around the underlying MemoryStream, I've added StreamExtensions to implement directly writing other values to streams - allowing use to pass the streams themselves to all the serialize functions.

  • Alter the return types of the Midi & .Chart scanning functions to return the tick resolution of the underlying file. We require that value so that we can calculate the sustain cutoff or hopo thresholds where necessary. By setting the values during the scan, we dodge the necessity of re-evaluating the values at song load time - which also means that the songcache more accurately depicts the true state of the songs it contains.

  • Replace the ParseSettings variable in song entries with a trimmed down LoaderSettings variable. This new type only carries the two threshold types and the overdrive midinote. On song load, we can gather all the other values used in ParseSettings through the other states present in the entry instance. Additionally, the appropriate values to use for song loading are calculated during the scanning process instead of before every load. The only adjustments after the fact should be singled out to state changes in the loading code (i.e. possible +1 tick additions).

  • Add new ScanResult entries to catch songs that try to be funny and set their resolutions to zero.

  • Provide YARGTextContainer with a Null variable similar to FixedArray alongside an IsActive property to allow the simplification of functions that had to use a boolean variable as the return type with a container as an out variable.

  • Add DTAEntry type

  • Optimize the RBCON update and upgrade utilization:
    -- Firstly, instead of holding all the updates and upgrades in separate lists during the scanning of new items, use the new CONModification type to combine the updates and upgrades that apply to the same DTA node name into a single instance. This , combined with the new DTAEntry type, allows for more-optimal deferment of the processing of DTA data without the downside of re-evaluating the data for song entries that share the same node name. However, upgrades loaded from cache will be held in a separate list.
    -- Secondly, disallow separate SongUpdate folders to hold references to the same node names. Only the update with the most recent write date will be used to modify files. This simplified the DTA parsing scheme so that no looping occurs over multiple update nodes at different locations.

  • Re-convert CONFile to a static class that contains a function that generates a List<CONFileListing> from a CON. The "TryGetListing" method becomes an extension on that type.

@sonicfind sonicfind force-pushed the DTA-Rework branch 2 times, most recently from 659529b to 8e95d63 Compare October 6, 2024 04:52
+ Add DTAEntry type that siphons the old ParseDTA code - providing an avenue to pre-loading information for a DTA node before loading a new song entry.

+ Streamline the hierarchy of construction to the most basic form. All the processing of data that produces the member variables for new entries will be performed solely through static functions. Additionally, we will now mark all the values in any of the `SongEntry` types as `readonly`. This was already the case for the "ini` subtypes, but the RBCON structures prior to this change required the live modification of an existing entry. The only exception is Song Length, as processing a missing length value requires loading the mixer - which requires setting all subtype information.

+ Provide `SongMetadata` & `RBMetadata` types with constructors & `Serialize()` functions holding the code that used to exist in the `SongEntry` & `RBCONEntry` constructors and `Serialize()` functions.

+ Streamline the `Serialize()` functions so that they all share the same signature - allowing for proper virtualization. For the ini subtypes, this meant moving the relative path and EntryType boolean writes to the `IniGroup` `SerializeEntries` function instead.

+ To add to the above: instead of passing around a BinaryWriter wrapper around the underlying MemoryStream, I've added StreamExtensions to implement directly writing other values to streams - allowing use to pass the streams themselves to all the serialize functions.

+ Alter the return types of the Midi & .Chart scanning functions to return the tick resolution of the underlying file. We require that value so that we can calculate the sustain cutoff or hopo thresholds where necessary. By setting the values during the scan, we dodge the necessity of re-evaluating the values at song load time - which also means that the songcache more accurately depicts the true state of the songs it contains.

+ Replace the ParseSettings variable in song entries with a trimmed down LoaderSettings variable. This new type only carries the two threshold types and the overdrive midinote. On song load, we cna gather all the other values used in ParseSettings through the other states present in the entry instance.

+ Add new `ScanResult` entries to catch songs that try to be funny and set their resolutions to zero.

+ Provide `YARGTextContainer` with a `Null` variable similar to FixedArray alongside an `IsActive` property to allow the simplification of functions that had to use a boolean variable as the return type with a container as an `out` variable.

+ Add DTAEntry type

+ Optimize the RBCON update and upgrade utilization:
++ Firstly, instead of holding all the updates and upgrades in separate lists during the scanning of new items, use the new CONModification type to combine the updates and upgrades that apply to the same DTA node name into a single instance. This , combined with the new DTAEntry type, allows for more-optimal deferment of the processing of DTA data without the downside of re-evaluating the data for song entries that share the same node name. However, upgrades loaded from cache will be held in a separate list.
++Secondly, disallow separate SongUpdate folders to hold references to the same node names. Only the update with the most recent write date will be used to modify files. This simplified the DTA parsing scheme so that no looping occurs over multiple update nodes at different locations.

+ Re-convert CONFile to a static class that contains a function that generates a `List<CONFileListing>` from a CON. The "TryGetListing" method becomes an extension on that type.
Realized that `Resolution` wasn't actually being parsed, but placing it in the same big dictionary made no sense.

+ Changes (and fixes) the Debug-only validation pattern
+ Changes `ZeroResolution` to `InvalidResolution`
Since the scanning process now pre-calculates HopoThreshold and SustainCutoffThreshold, we need to adjust the reader code so that it doesn't re-calculate the thresholds to incorrect values when called from a song entry.

+ Remove `HopoFreq_FoF` & `EighthNoteHopo` ParseSettings variables
+ Remove `GetHopoThreshold`-like functions
+ Convert MoonSong resolution to uint
@EliteAsian123
Copy link
Member

HUGE. Unfortunately, I don't really have the time to look through all of the code but a lot of these are good changes overall!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants