A basic Music tracker inside gd5?

[blurymind]

Description

There is a new type of game engines out there that many indie developers seem to be hopping on.
The fantasy console.
https://tic.computer/
https://www.lexaloffle.com/pico-8.php
https://store.steampowered.com/app/767240/BASIC8/
...and many others

The allure of these engines imo is in having everything required to create a game in one IDE - a map editor, a code editor, a sprite editor and ..a music tracker with a sound effect generator! That makes them very fun to use, because the developer can quickly explore all aspects of their game idea- including the music. The music tracker may not be very advanced- but it is simple to use and really fast at getting results.
examples:
https://www.youtube.com/watch?v=gQf4vkQXJUM
https://www.youtube.com/watch?v=HPu-Jkdf8uQ

Gdevelop now has it's own built in pixel editor, which almost puts in the fantasy console scene, but minus any restrictions imposed by design. There is potential here :D

Now this may sound a bit nuts at first, but please hear me out.
This is something that can be easily added to gdevelop, once you can insert metadata to game objects - much like how piskel layer data could be inserted as metadata.
In the same vain, the actual music tracker data can be stored as a json string and be edited directly inside gdevelop embedding a tracker/ sound effect generator.

Solution suggested

My suggestion here is once I sort out storing layer piskel data, to try to embed one of the open source js trackers, so one can make music for their game directly inside gdevelop.

For music creation
There already are a few html5 trackers that are tiny and and can be embedded via electron:
This one is my favorite:
http://www.stef.be/bassoontracker/?file=demomods%2Fdemomusic.mod
https://github.com/steffest/BassoonTracker

https://github.com/mbitsnbites/soundbox has to be second best atm. There is an online demo here:
http://sb.bitsnbites.eu/

other ones:
https://github.com/gridsound/daw

https://www.igorski.nl/experiment/efflux/

So in summary, I believe that if we can add built in capabilities to gdevelop to also rapid prototype music, it would make the editor even more fun for creative types. People who are into fantasy consoles will jump ship to gdevelop, as it will give them a similar experience but without the limitations.

The thing is,I still need to explore js music trackers before deciding if this is worth doing

In any case, what do you guys think? Is this a crazy idea? Do you know of any html5 trackers that would be amazing for gamedev?

[ddabrahim]

I'm not against to turn GD in to an all-in-one tool, only would like to comment on that GD is compete with fantasy consoles and consoles becoming more popular because of the built-in tools.

I don't think it is the case. The consoles are not popular only because of the tools built-in but also because the purpose of the console is to have fun and enjoy your time with the console like you do with a real console or did back in the old Amiga days. Many talented and experienced programmers do jump in to the fantasy consoles because of the culture, nostalgia, and to have fun. It is unlikely to happen to GD simply because GD is meant to be used to make games for today hardware and today market while the consoles are meant to be used to make games for a fantasy hardware with limited resolution, input, colors, memory..etc and share them on a fantasy market usually with full source for free. Fantasy consoles are completely different culture, I don't think GD ever going to compete with fantasy consoles simply because GD is about real game development for real hardware and real market, fantasy consoles are about playing games in the first place and to make games for a fantasy hardware and share them on a fantasy market. GD doesn't have this. If GD would have a market to share games that people can download, play and change and redistribute completely FREE, maybe then we could say GD compete with fantasy consoles but it is not going to happen because people not going to share their games with full source for free as long GD can export to real platforms. GD and other game dev tools are completely different culture. I know some console can export to HTML5 to let people share their creations outside the fantasy market, but the limitations for input, memory, resolution..etc still apply which is make them less suitable for targeting real market and it is not the point of the fantasy consoles anyway.

[blurymind]

@ddabrahim You make a good point. We dont have a board where people can share playable games.
On the bright side, we do have a lot of demos.
Pico8's board is excellent imo, but so is tic80:
https://tic.computer/play
https://www.lexaloffle.com/bbs/?cat=7#sub=2&mode=carts

We can use itch.io instead for now

[4ian]

GDevelop by itself used not to provide creation tools for images/sounds but it's true that having Piskel integrated make life easier for prototyping and even for 8bit/pixel art, so the whole experience of game creation is enhanced.

While I still think, as ddabrahim pointed, that GDevelop is more about "real" game development than fantasy console (which are a great thing but the way! I wasn't aware it was existing), it can be interesting to have third parties tools like Piskel if:

• It is high quality: great, "app-like" user experience, ability to export the created resource (music here).
• It is easily embeddable without hacks.
• It is easy to remove (yes, that's an important criteria :) The idea is that GDevelop must not rely on it to work).
• It is maintained/supported (because we don't have the man power to support anything else than GDevelop which is already a very huge piece of software).
• It provides a lot of value for users.
• It is open-source.

This is something that can be easily added to gdevelop, once you can insert metadata to game objects - much like how piskel layer data could be inserted as metadata.
In the same vain, the actual music tracker data can be stored as a json string and be edited directly inside gdevelop embedding a tracker/ sound effect generator.

This could indeed be working like this :)

[blurymind]

Thank you @4ian :) It is certainly to have it as an option - like piskel. I agree on all of the points.

It is high quality: great, "app-like" user experience, ability to export the created resource (music here).

The ones I looked into are a mixed bag. Bassoon tracker seems to be most maintained and best in terms of quality, but I need to use it more to be sure.

It is easily embeddable without hacks.

This will need to be researched, I think that point is important too.

It is easy to remove (yes, that's an important criteria :) The idea is that GDevelop must not rely on it to work).

Thanks to you it can be- you laid down the foundations with piskelbridge. The tracker will be in the very same way completely optional. It wont change how music is currently loaded.

It is maintained/supported (because we don't have the man power to support anything else than GDevelop which is already a very huge piece of software).

Bassoon seems to be the best option again - as its code seems to be getting more updates, but it would be good to keep this in the air and look around.
A simpler but with less features is http://sb.bitsnbites.eu/ - it is much easier to learn too
I found a bug in it however mbitsnbites/soundbox#63

It provides a lot of value for users.

That is the biggest reason to consider it for me. I strongly believe that it does provide great value and that can clearly be seen by all the wonderful music that is made inside tic80 and pico8 and shared across the web. If we find a tracker that has the same simplicity and charm- it can easily start the same trend with gdevelop projects

It is open-source.

Without it being open source, it won't be possible to embed imo

From the two I think that the sound effects generator will be easier to integrate and is also easier to pick.
The tracker needs a bit more research as it is much more complex.

Both would of course first need the ability to store metadata into sound resources. The metadata feature can be quite valuable further down the road imo

[blurymind]

Update on Bassoon tracker. I posted a feature request on their tracker and notified the dev about my intention to bundle a copy of his/her tracker:
steffest/BassoonTracker#7

After looking more into it, the tracker seems to already have a great community of users and a lot of community made content - both in terms of songs and instruments.

[4ian]

Cool :)

Something that may not be big, but beware of the phrasing "it lacks the fundamental ability" => "fundamental" is subjective as it seems to work for a lot of people without it - and that could bother the maintainers/authors :)
Better just say that rendering to wav/ogg or mp3 would be a great addition, unlocking integration with other softwares like GDevelop and potentially help usage in indie games.
(More generally, especially when it comes to open-source projects, it's good to avoid things that could be seen as negativity like "this thing is missing". Rather ask "What about adding this?". Maybe the authors even already thought about it but don't have time/don't think it's a good idea/is too much work/don't know that people are even interested).

[4ian]

@blurymind Also maybe I missed something but I can read in the README:

export to .wav and .mod

🤔

[zatsme]

Yes, the Web demo can export wav files 😋

[blurymind]

@zatsme for some reason I couldn't get it to render a working audio file of the demo songs. The author of bassoon clarified that due to memory limitations it is constrained to rendering only one of the 4 pattern channels.

That said he is actually quite a cool guy and offered some solutions to rendering all 4. I will explore the options and see what can be done. The web demo is incredible. It can even fetch community made music and instruments straight from the editor. Bassoon is hands down the best html5 tracker atm imo

[zatsme]

After reading the issue you referenced I wonder why we can't access the mod file directly from gdevelop, this would seem the easiest solution as no change is required on the tracker side, and it would give gdevelop access to more music files in the mod format. 🤔

[4ian]

The current audio engine is Howler.js - it is based on webaudio API and has great support for all browsers (see more on https://howlerjs.com/)
.mod files are different from traditional wav/mp3/ogg and other audio format as it's not directly audio but need to be processed and converted to audio before being played. There might be existing libraries for playing these songs but that implies:

• Finding and adding the library
• Ensuring that it works properly, especially on iOS, Safari and Android phones.
• Re-work the audio engine in the game engine to either use Howler.js or the mod sounds.
• Support changing volume, pitch, pausing, etc..

[blurymind]

I think of finding a library to render the mod file to a wav during the export process in this case, so the game engine uses wav/ogg in the end.
It would be better if we don't have to re-work the audio engine.. The only advantage in having support for mod files in the engine would be in making rhythm games, where note timing is important.
I think godot engine actually still supports mod files, mainly because the main developer is experienced in writing music trackers and used to actually be a sound software programmer

I found a few js modplayers so far:
https://www.npmjs.com/package/protracker
https://github.com/jhalme/webaudio-mod-player
What is needed in this case is something that will render the mod into a wav/ogg - that is hard to find
In any case this conversation should be on bassoon's issue tracker :)

I will give it a few days and in the meantime play with integrating the sound effects generator-as its easier to solve

[blurymind]

Actually more advantages in using mod files by the game engine appear when you consider being able to dynamically change the playback speed, the volume of some channels and even pitch.It could be used to create atmosphere, but as @4ian says I dont know if there is a good multiplatform library to process them on the engine side

[blurymind]

I moved out all the sound effects posts to another issue post for feedback - so as not to clutter this one.

Some update on Bassoon - the developer of the tracker has looked into saving wav files and suggested some solutions:

If we want to use bassoon we have three options

• add support for mod playback in gdevelop.js and expose it to the newide (he even offered a mod player we can use in the engine)
• render the songs server side then download them
• bundle gdevelop with openmpt binaries - different to each platform and do the rendering in a separate process

All of this is due to memory restrictions on the browser side

See this post for better explanation:
steffest/BassoonTracker#7 (comment)

I personally agree with him on:

As 30 years of history tells us: tracker music and games are a good combo :-)

If I knew more about gd.js, I would try to add support for mod files in it. For that somebody more experienced is required. If we get the export part solved, embedding bassoon will be the easy part. I think I can do it within a week

[zatsme]

That bassoon tracker player would be too awesome for us to use. Mod playback built into gdevelop would be a killer feature for many 😍😍😍

[blurymind]

That bassoon tracker player would be too awesome for us to use. Mod playback built into gdevelop would be a killer feature for many 😍😍😍

Adding a mod player to the engine seems like the best approach to this, but it also requires some work on the engine side and testing if it functions on all platforms. Adding the tracker itself to the newIde won't be hard to do and we actually have its developer's blessing too - much like we did with piskel

Bassoon's dev made a really good point about the perks of having support for mod/xm music:

... keeping the file in .mod or .xm format makes sense too as this would be a much (much!) smaller file.
For example: demomusic.mod is 50kb - rendering it to wav is 50MB - times 1000 !
Of course a game would include it as .mp3, but that still would be a few MB, easily x10 or x20 the file size.

Reduced file size would be beneficial for both web and mobile platform where that is important. Google has a limit on APK file size you can publish

In order to load mod files, the js extension api needs to be able to include them in published projects and have access to them. Here is the documentation:
http://www.stef.be/bassoontracker/player/
I might be wrong, but I think the js extension API still can not do this to files inside the game's project folder:
Tracker.load("url/to/file.mod")

Of course the option to go with soundbox instead is still here, but soundbox has much less features and the actual performance in it is not as good. Bassoon really is the best html5 tracker out there imo

[4ian]

I might be wrong, but I think the js extension API still can not do this to files inside the game's project folder:

For the game engine, I think all of this should be doable using extensions. You can use resources of type "audio" and get their filenames from resources in the game engine.

• You need to include the js library for reading/playing .mod file (addIncludeFile in JS extensions)
• You need to add actions to play the musics, potentially conditions too.
• When you want to get the file, you should be able to get the URL to it (see _getFileFromSoundName in howler-sound-manager.js)

[blurymind]

@4ian thank you for explaining this. I will give it a try once the sound effects generator is complete and I add support for storing layers in piskel. With this feature and the awesome new reusable functions feature gdevelop5 is becoming a very powerful engine 😺

[blurymind]

@4ian since basson can not export mp3/wav files without relying on an external server and internet connection, we will have to go with mod files alone here. In that case I wonder where to store the metadata to. Can I make a custom gd resource (mod) and set/get metadata on it - all via the js extension api?
Should we have mod files as a resource type in the resource manager? They are kind of like a mixture of xml and binary audio data - they are very efficient in terms of size/footprint. In that case I need to register them as a new resource type

[blurymind]

@4ian presented a temporary mod file loading workaround, so I can start writing an extension:
#1112 (comment)

Will give this a try once the dialogue tree extension is ready

[4ian]

Cool! With an extension to play the mod files and an external editor to have the tracker inside the IDE, that will be really great 😄

[Antar74]

Hello, is this thread still active? I just started with GDevelop this month and would love to have music-making options from within GDevelop. What actions should be taken to achieve this? How can one help?

[blurymind]

An extension that plays mod files needs to first be implemented. It will wrap bassoon tracker's player library. A new mod file resource type has to be added to the ide. That should be easy to do, just got a bit sidetracked with the tilemap extension. If someone wants to pick this one up, I can do some mentoring on it? Once GD can play mod files, another pr has to be done to bundle bassoon tracker to create and edit them- similar to how jfxr is bundled

…

On Sat, Mar 28, 2020, 10:44 PM Antar Kuri ***@***.***> wrote: Hello, is this thread still active? I just started with GDevelop this month and would love to have music-making options from within GDevelop. What actions should be taken to achieve this? How can one help? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#689 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABRRWVI2G524JDGJQNOT2F3RJZ4VHANCNFSM4FZOHTLQ> .