Support for Tilemaps objects (tilesets and Tiled files)

[blurymind]

See also the card on the roadmap on Trello

If you're interested in implementing this, read the discussion so far. In particular, there are important decisions to be made about which library to use, how to integrate this in the editor and how to ensure performance.

Thanks,
@4ian

below is the original post/discussion:

---

With the new ide we are still missing a very important object- the tilemap. Its important because it would help reduce the memory the game is using- allowing is to make much bigger maps.
The workflow of using tilesets and tilemaps is also much nicer and is a standard for 2d games.

My proposal is to allow to import tilemaps made in tiled- since that is now an open source industry/indie standard when it comes to 2d games. Most other game engines support it too (godot,defold,phaser,unity, etc) and has a wide use in the gamedev community.
https://www.mapeditor.org/
https://steemit.com/gamedev/@tensaix2j/tutorial-part-6-how-to-create-a-html5-game-using-tiled-map-editor-js-and-canvas

What's great about going this way is also that we dont have to start from scratch! Since tiled is such a popular tilemap editor and file type, it already has an implementation for pixi.js and cocos
https://github.com/riebel/pixi-tiledmap
https://github.com/pixijs/pixi-tilemap

NPM:
https://www.npmjs.com/package/pixi-tiled
https://www.npmjs.com/package/pixi-tiledmap
https://www.npmjs.com/package/tiled-to-pixi
https://www.npmjs.com/package/tiled-utils
and many more...
https://www.npmjs.com/search?q=tiled%20pixi

There are also many tiled files ready to be used for testing here:
https://github.com/bjorn/tiled/tree/master/examples
We can even use them in example gdevelop projects :D

Tiled exports both to json and xml btw in terms of formatting- making it great for an html5 engine.
http://docs.mapeditor.org/en/stable/reference/json-map-format/

example of rendering a tiled file in phaser
https://phaser.io/examples/v2/loader/load-tilemap-json

So why reinvent the wheel. This will give us the basic tilemap object inside gdevelop5 plus all the perks of being able to use tiled- an editor that has smart autotiles and many advanced features that have been developed for many years.
Later on we can add ability to create and author the tilemaps inside the newIde - but with laid groundwork for how these objects are structured and the perk of having support for this industry standard

There is a bounty on this feature
https://www.bountysource.com/issues/58876395-support-for-tilemaps-objects-tilesets-and-tiled-files

EDIT: 2 years later the starter of this issue ticket learned javascript and implemented it himself. Pr still pending of course :)

PR here: #1901

[blurymind]

found this repository, which could also be useful
https://github.com/kittykatattack/tileUtilities

[4ian]

Hey 👋 I've put this issue on the roadmap (so that anyone can vote for it!) there: https://trello.com/c/vJacNvXq/65-support-for-tilemaps-objects-gdevelop-5-html5-games

I'm closing this issue as open issues on Github is now be dedicated to bugs only :) (But it's still possible to continue the discussion on this issue even if it's closed)

[blurymind]

there is a cool html5 tilemap editor here:
https://github.com/less-xx/tile-map-editor
but unfortunately it can not import tiled files (xml or json)
it does both standard and isometric maps.

One thing that I would like to see improved upon gdevelop4 is having the tilemap editing happen inside the scene editor and not as a separate window. I found that to be a bit inconvenient in the implementation of tilemaps in gd4

[blurymind]

The developers of rpg maker mv have actually contributed a tilemap renderer to pixi.js and it is an officially in the pixi git:
https://github.com/pixijs/pixi-tilemap

A potential problem of using it would be any imposed limitations/dependencies from rpg maker mv. At least for now it looks maintained - with the last update being a week ago. Rpg maker mv does have a huge userbase

[4ian]

pixi-tilemap could be interesting. Ideally we can find something that is well maintained, or at least with an implementation that have a good quality.
Two things that may not be handled and so we'll have to check how to do it is:

• Collision handling.
• For really huge tilemap, we would ideally avoid to render all tiles if they are not displayed on screen.

[Lizard-13]

Zero time to work on this one, but for tiles we can store a lot of sprites in a common PIXI container or a particles container for more speed. Then always hide all the sprites/tiles but leave visible the ones in front of the camera. For extra performance, it should be possible to store the tiles in a 2D array, and map the camera top-left and bottom-right points into array indices. I did something like this in python years ago.

Also, because the permissive PIXI rendering system we can use arbitrary Z-orders for each layer in the same tilemap. I mean we can create a tilemap object with three layers, and put one layer at Z-index = -1, other at Z-index = 0 and the other at Z = 100 (instead of creating three different tilemap objects).

Finally, I like the in-scene object editors idea, An object editor should be able to define custom panels on the editor, and instances should handle clicks, drags, end clicks, etc.
A lot of work but a nice long-term goal.

[4ian]

Finally, I like the in-scene object editors idea, An object editor should be able to define custom panels on the editor, and instances should handle clicks, drags, end clicks, etc.

Yep, I think it's fair :) Ideally we would have an easy to use API (not too large/not too permissive) to allow edition from the scene editor.

[blurymind]

I was trying to build a new gdevelop demo from a free set that is on itch.io - for the showreel video.
But realized that the character sprites and map elements are all on tilesets.
https://ansimuz.itch.io/sunnyland-woods

This made me realize how limited we are when not having this in the editor. One way this can be tackled in a lower hanging fruit kind of a way is to simply add the ability to set a visible area of a sprite object. That way what I can do from there is add a tileset importer that imports the tileset as a single resource, adds it to a number of frames, then on each frame sets the visible square area.
This is how godot does both tilesets and animated sprites btw.

If the game engine side supports setting a square visible area for sprites, I can start working on adding a tileset import/cutter and a mask visible area editor to the newIDE as a first stepping stone.

[blurymind]

@4ian we can reuse the functionality of the collision masking editor to set visible area of a sprite inside a tileset.
see for example how godot does this in that way, but with an area rect
https://www.gamefromscratch.com/post/2015/04/28/Godot-Engine-Tutorial-Part-8-Tilemaps.aspx

If gdevelop.js allows that for sprite resources, I can create an importer for the newIde that generates the masks automatically or generates area rects. An area rect is basically just x1,y1,x2,y2

We could from there on make it so that for example the collision masking editor could optionally just let you use the visible area mask as collision bounds as well.

[blurymind]

Edit hitboxes can be repurposed to edit visible areas and expanded to be used to set the areas of a tileset like this:

in the example I am setting the visible area of frame#1

by default the visible area could be the bounds of the image,unless the image is a tileset. In that case the visible area can be generated by the importer code, which sliced it

[zatsme]

To me this seems closely related to spritesheets which are also missing from GD5. Maybe this can be implemented so both tileset and spritesheets are added.

[blurymind]

I think this will be the best way to do it if we are to reuse whats already in gd5's ide.

The hitbox editor's UI can be improved too - I would like to give it more area for showing bigger images, add zoom in/out/pan.

[4ian]

The hitbox editor's UI can be improved too - I would like to give it more area for showing bigger images, add zoom in/out/pan.

Yep, that's something that is asked for a lot: https://trello.com/c/kPRc19zV/147-add-support-for-zooming-in-out-when-editing-points-hit-boxes-or-in-a-preview-of-a-sprite-in-the-sprite-editor

Feel free to give it a try :)

For using spritesheets, Pixi.js supports setting a frame/area on the texture that is displayed - this is I think the "official" Pixi.js way of using spritesheet so that would make sense to expose it in GD.

If gdevelop.js allows that for sprite resources, I can create an importer for the newIde that generates the masks automatically or generates area rects. An area rect is basically just x1,y1,x2,y2

I'm not sure how it would work, do we really need a new "Sprite Resource"? Resource should represents "physical" files that the game rely on. So a Spritesheet would be one resource (an image, like other textures), even if we can have editors to ease the selection of the areas in the spritesheets :)

[4ian]

This being said, both features of spritesheet and tileset/tilemap are relatively independent.
They are in two different cards by the way on the roadmap:
https://trello.com/c/dmElE214/103-support-for-spritesheet-textures
https://trello.com/c/vJacNvXq/65-support-for-tilemaps-objects-tilesets-and-tiled-files-gdevelop-5-html5-games

From what I can see, people are more asking for now for tilemaps - so this would be for now a more impactful addition to GDevelop.

[blurymind]

@4ian I need to have the pixijs functionality somehow exposed to the engine right? I mean once a visible area is set in the editor, the engine itself needs to respect that too

I think that we dont need new type of resource, just the ability for the current game objects to have a visible area mask

[4ian]

I mean once a visible area is set in the editor, the engine itself needs to respect that too

Yes of course :) This is at which point you should inspect how gdjs.SpriteRuntimeObject is done to see how images are used and in particular gdjs.SpriteRuntimeObjectPixiRenderer which is responsible for creating and updating the PIXI.Sprite of the object. This is where support for displaying some area of an image should be added. For now, images are loaded using gdjs.SpriteRuntimeObjectPixiRenderer.getAnimationFrame from the image manager, but without any information about area.

I think that we dont need new type of resource, just the ability for the current game objects to have a visible area mask

Yes. Though I've just seen in Pixi.js that there is a PIXI.SpriteSheet that is a helper class that can be given a JSON (see http://pixijs.download/dev/docs/PIXI.Spritesheet.html) and load all the textures from the spritesheet (using the same base texture, i.e, the same image file): https://github.com/pixijs/pixi.js/blob/dev/src/core/textures/Spritesheet.js#L238

So maybe you were right when speaking about resources, every sprite in a spritesheet could be a different resource (we could enhance ImageResource to have not only the file, like now, but also the area). Ideally we would support importing json from TexturePacker, Shoebox or Spritesheet.js like PIXI.SpriteSheet.
This could be a tool that open the JSON and create the associated resources.

If we do this, we still need to study if this will work with other objects (i.e: will a ImageResource with an area work for a TiledSprite/PanelSprite?).

[blurymind]

Interesting, the pixi spritesheet object supports loading the mask data from a json file. Thiswill be particularly useful when we want to load it from something like texturepacker

See example:
atlasExample.zip

[blurymind]

Another thing I could do to approach this is to add something like
https://www.npmjs.com/package/unpack-texturepacker
to the editor - to allow unpacking atlases and spritesheets

and
https://www.npmjs.com/package/tileset-slicer

slicing a tileset into files has two downsides:

• you are de-optimising the game
• you are creating more files

It will be the easiest way to approach importing tilesets with no need to touch the engine- just the editor

I can add this as a command to the resourcemanager if you are interested :)

[HarshRajSinghania]

Please do it as fast as possible..... It would be amazing.....

[roracle]

I am going to come in and say this is a feature I really need. Linked here is a 64bit version (Linux and Windows) of the game I've been working on in Construct 2, and I would love to be developing it in GDevelop, or even then if I end up finishing it in C2, I'd eventually like to do all my projects in GDevelop, as it's easier to keep up with.

In order for this whole system to work, I have about 5 layers of active graphics, 1 that is invisible, the bottom being the water, the topmost being anything above the player, and other layers for ground, type, and solids. Plus another for the player themselves.

I hope to see tilemap support, including painting, implemented.
IF IT CAN BE DONE LIKE RPG MAKER where the main tile is your bg fill (water, for example) and you select a grass icon, and it automatically uses the 9 tiles like system to fill the edges, that would be even better.

https://drive.google.com/open?id=1LYlSTF-t2ncgbQYKD2PrzV8aWjba6ufq

[blurymind]

Rpg maker uses a concept called autotiles. It is something that also Godot already has. There are a few JavaScript implementations of autotiling I've seen before http://blog.rpgmakerweb.com/tutorials/anatomy-of-an-autotile/

…

On Fri, Sep 6, 2019, 6:31 PM roracle ***@***.***> wrote: I am going to come in and say this is a feature I really need. Linked here is a 64bit version (Linux and Windows) of the game I've been working on in Construct 2, and I would love to be developing it in GDevelop, or even then if I end up finishing it in C2, I'd eventually like to do all my projects in GDevelop, as it's easier to keep up with. I hope to see tilemap support, including painting, implemented. IF IT CAN BE DONE LIKE RPG MAKER where the main tile is your bg fill (water, for example) and you select a grass icon, and it automatically uses the 9 tiles like system to fill the edges, that would be even better. https://drive.google.com/open?id=1LYlSTF-t2ncgbQYKD2PrzV8aWjba6ufq <http://url> — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#503?email_source=notifications&email_token=ABRRWVO3TUGRWA2FBJ3TXDLQIKAVPA5CNFSM4FB4DSFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6DLXMY#issuecomment-528923571>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABRRWVJMR6N5IUOLJNBIOVTQIKAVPANCNFSM4FB4DSFA> .

[Silver-Streak]

I see this was closed but with no note, so it may have been lack of activity.

I posted this over on trello, so I wanted to at least throw it on here too.

It looks like some folks have already done Tilemap and Tileset support for PixiJS
https://github.com/pixijs/pixi-tilemap

Which may help with a lot of the work needed to implement this.

[4ian]

Yes this library is a strong contender to be the one used to implement a tilemap object.

This object is more complex than others because it needs changes in the IDE so that you can paint the map directly in the scene editor and it needs particular attention to collision handling and performance. In particular, collision handling is being slightly reworked in GD, work can start after on this object :)

[Silver-Streak]

I'd like to post a bounty on Bountysource for Tileset and Tilemap/Atlas support, It seems like this issue is the most descriptive/filled out issue GDevelop has on it.

Any chance this could be reopened so I could post the bounty, or should I post a new issue to cover it?

[blurymind]

@4ian @Silver-Streak Do you feel this is ok to close now? Further improvements could be a separate task. Should we wait for the release with this feature to come out? (next release that is)

[Silver-Streak]

@blurymind I think it would make sense to close this once the release is live, because that the "issue" (and bounty) is solved, and any problemsd/enhancements requested at that point would be new items. Does that sound reasonable?

[blurymind]

Yup I was thinking the same thing 😅

…

On Thu, 21 Jan 2021, 13:29 Silver-Streak, ***@***.***> wrote: @blurymind <https://github.com/blurymind> I think it would make sense to close this once the release is live, because that the "issue" (and bounty) is solved, and any problemsd/enhancements requested at that point would be new items. Does that sound reasonable? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#503 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABRRWVO3MFKGXTGKZQE7FXDS3AT2JANCNFSM4FB4DSFA> .

[Silver-Streak]

Speaking of which, it is probably worthwhile to note that @Entr0py404 tested the nightly build and is seeing tile rotation being ignored completely in a map they made with tiled map editor.

I asked them to comment on @4ian pr about tile rotation, but it doesn't appear that was done yet so I wanted to call it out.

They are making the tilemap in Tiled directly, and while flipping horizontally or vertically works fine with the object/preview, rotating in any direction is ignored by the object in GD5.

[4ian]

Would be good to verify if it's still the case after my PR, was it a recent nightly build?

Surely we should then clarify in the doc what is supported and what is not. Though rotation should work, because it's expressed by tiled as a combination of flipping and flipping diagonally - unless we're not speaking about 90 degrees rotations?

[Entr0py404]

Gdevelop version used:
https://gdevelop-releases.s3.amazonaws.com/master/commit/ff55b8b6a05c3ce7821dc5e28b77f3779b7f3a4a/GDevelop%205%20Setup%205.0.0-beta103.exe

Objectbox is offset.

Tilemap is not anti-aliasing and ignored rotated tiles. (Tilemap on top, Exported as image bottom, also bugged object list lol)

Flipping works but not rotation, the one on the ground is horizontally flipped.

[blurymind]

can you share your project folder zipped? I can have a look maybe tonight? :)
Is that snowman made of multiple tiles? Kind of looks like its rotating them individually rather than as a group - the tileset and tilemap json needs to be inspected here.
Maybe the bug is that it is flipping them horizontally rather than vertically in these rotated cases, but its hard to say without having the source project

Or maybe rotation support and others could be done in a separate ticket?

[Entr0py404]

TileMap Test.zip

"Is that snowman made of multiple tiles?", yes it is

[blurymind]

TileMap Test.zip

"Is that snowman made of multiple tiles?", yes it is

thank you. How does it look rotated in tiled? :)

[Entr0py404]

Tilemaps are in the zip

[blurymind]

it looks like its flipping them in the wrong direction, interesting :)
@4ian should I try to fix this rotation problem in another pr?

[4ian]

it looks like its flipping them in the wrong direction, interesting :)
@4ian should I try to fix this rotation problem in another pr?

Yes please! Maybe one direction is incorrect in the flipping code (I may have inverted horizontal and vertical flip!).

[4ian]

@blurymind I went ahead and did the fix!
See 2222aeb and the link I've put in the code to understand these "magic numbers" :)

We would still need to find:

• why the selection rectangle is offset.
• add a checkbox property to activate or not antialiasing? Right now it's always disabled.

[blurymind]

@4ian that was quick! :) thank you.

on anti-aliasing - do we want that only on rotated tiles? In that case not sure if a checkbox that does it to the whole tilemap would be a good fit

on the rectangle - I am guessing that it is not updating the size or not computing it properly due to something in the tilemap or tileset. Needs to be investigated

[4ian]

do we want that only on rotated tiles? In that case not sure if a checkbox that does it to the whole tilemap would be a good fit

No I'm thinking of the whole tilemap.
Actually I see that we have texture.baseTexture.scaleMode = PIXI.SCALE_MODES.NEAREST; (i.e: "pixel perfect" on the base texture, i.e: the atlas). We should probably remove this because this is chosen in the resources editor.

on the rectangle - I am guessing that it is not updating the size or not computing it properly due to something in the tilemap or tileset. Needs to be investigated

Yeah, adding a few logs and seeing how is generated the tilemap should help finding the issue.

[4ian]

@blurymind I also went ahead and remove the NEAREST applied to the texture.
See d6a94a3

If you want to take a look at the object rectangle, I won't have time to do this tonight.

The result of the "anti-aliasing" (nearest texture mode) change can be seen as you can now enable or disable the smoothing of the texture in the resources, like for sprites and other objects. See for example the different in the desert tilemap:

(Not smoothed on top, smoothed on bottom).
Let me know if you find something for the last issue :)

[blurymind]

@4ian thank you for removing it. I completely forgot that gdevelop already handles this in the resources editor 😄

I will have a look at the box problem tomorrow. I think initially it wasnt even drawing the box unless you do a selection around it. Could this be somehow related?

[4ian]

I think initially it wasnt even drawing the box unless you do a selection around it. Could this be somehow related?

This was because there was a missing function to detect if a point is inside.

For the box problem, the box should be drawn according to RenderedTileMapInstance.prototype.getDefaultWidth (and the same for the height), so might be good to look there if what is returned is correct.

[Silver-Streak]

@blurymind In anticipation of this nearing completion, the bounty has now been bumped up to a final total of $200.

[4ian]

Nice! I hope to be able to make a new testing release soon with this.

[blurymind]

@Silver-Streak thank you! Much appreciated :) I am investigating the incorrect box size bug btw. Hope to have a pr soon

I think it does return the correct height, its just offsetting the box on Y for some reason.

EDIT: oh nevermind, it is actually cutting a bit of the height. this._pixiObject.height is off, but why on that map?

[Silver-Streak]

@Silver-Streak thank you! Much appreciated :) I am investigating the incorrect box size bug btw. Hope to have a pr soon

I think it does return the correct height, its just offsetting the box on Y for some reason.

EDIT: oh nevermind, it is actually cutting a bit of the height. this._pixiObject.height is off, but why on that map?

Excited to see this go live, although I'm waiting for the update servers to get b104 so I can test this out more thoroughly.

What do we think next steps would be for 2021 around this?

• Is it feasible to integrate LDtk (obviously as a separate pr/issue/bounty/etc)? I remember that there would need to be some sort of parser made for their files or something similar.
• How realistic is it to add collision layer imports?

[blurymind]

What do we think next steps would be for 2021 around this?

• Is it feasible to integrate LDtk (obviously as a separate pr/issue/bounty/etc)? I remember that there would need to be some sort of parser made for their files or something similar.
• How realistic is it to add collision layer imports?

Both are potentially big tasks that should have their own new issue tickets, but both are feasible imo. Which of the two is more important to you?

The collision layer imports could use the shape data from tiled to create collisions - you can create your collision objects inside tiled and gdevelop can identify them as collisions somehow (collision groups?)

The ldtk editor - what is tricky about it is that LDTK is very powerful - it lets you edit several maps in a single file. Even lets you describe game objects and other things. The other tricky thing is that it can only read/write tiled as xml (tmx files)
https://ldtk.io/docs/game-dev/loading-ldtk-in-your-game/exporting-tiled-tmx/
so we need to add support for parsing xml files to gdevelop - as a file resource
OR
add another parser for ldk json files, which hold more data and would be better supported.
The trickiest is allowing a resource in gdevelop to load other resources that it depends on (via relative paths). An ldtk tilemap json for example has relative paths in its data that link to image resources and other json resource (tilesets). Its not a problem atm because we only allow one tileset per tilemap - but it would be with ldtk which lets you have many.

That said are you ok to close this ticket, as we now support reading tiled files? :)

[Silver-Streak]

Both are potentially big tasks that should have their own new issue tickets, but both are feasible imo. Which of the two is more important to you?

@blurymind Honestly, the editor is probably a bigger deal. (There is an in-engine way to deal with collisions for now, there is not an editor for tilemaps in the engine right now.)

LDTK does support json output but it is its own format. Would there be a way to transcribe that output > to Tiled format JSON > to import into the pixi-tilemap extension you implemented?

And yes, I'd definitely think both of these would be new requests/new bounties/etc. I'm just wanting to understand what should be considered "request-able" at all so I'm not putting in feature requests for things that aren't possible 😄

[blurymind]

The ldtk support would be more exciting for me to implement too. In any case, please open new issues for these things and we will add them to the trello board.

I think we can close this ticket now :)
let me know if you would like to reopen it anyways

[blurymind]

LDTK github ticket here
#2002

collisions feature ticket here:
#2268

Lets continue those discussions there 😄

[Silver-Streak]

@blurymind if you haven't already, please ensure you go to https://www.bountysource.com/issues/58876395-support-for-tilemaps-objects-tilesets-and-tiled-files and claim your bounty. This works wonderfully.

[blurymind]

@Silver-Streak claimed it, thank you for supporting this work! 👍 I will try to fix any bugs that people find with it and hope to also see ldtk bundled in the future. If we get that in, gdevelop will probably have the best level editor of all game engines. 😸
The godot guys are doing some interesting stuff with their tilemaps too. Been following it