[Feature requests] Useful rgbgfx behavior flags

[Rangi42]

Most of these are provided by pret's tools/gfx.c post-processor for rgbgfx output.

• Omit all blank tiles from the end, the beginning, and/or throughout the image ([Feature request] Specify a color to skip tiles which are entirely that color #1484) (this would let graphics be resized and padded at either end to look good)
• Omit horizontally and/or vertically mirrored tiles (currently -m omits both)
• Lay out tiles in 8x16 order suitable for OAM graphics (Allow arbitrary "dedup unit" sizes in RGBGFX #1005) (currently have to make the image 2 tiles tall and use -h)
• Specify a list of tile indexes or coordinates (tools/gfx.c uses indexes) to preserve despite and mirror/blank-omitting flags
• Ignore an embedded indexed palette if one exists, so only the RGB color values will be used to determine the 2bpp values (no need to worry if your graphics editor has created a PLTE chunk)
• Add a given offset to the tile indexes in an outputted tilemap

[pinobatch]

Lay out tiles in 8x16 order suitable for OAM graphics

Seconded. For comparison, when I wrote my own tool (pilbmp2nes.py, later rewritten in C as pngtochr), I didn't include -h as a separate switch. It instead takes arbitrary height of each strip in pixels (which could be set to 16 for 8x16 or the image height for RGBGFX -h behavior).

only the RGB color values will be used to determine the 2bpp values

How would this work? Would the caller pass a BGP value and then the converter assign indices to RGB color values based on luminance (.30R+.59G+.11B)?

[Rangi42]

How would this work?

From the rgbgfx docs:

1. If the file has an embedded palette, that palette's color and order are used.
2. If not, and the image only contains shades of gray, rgbgfx maps them to the indices appropriate for each shade. [...]
3. If the image has color (or the grayscale method failed), the colors are sorted from lightest to darkest.

This proposed flag would skip step 1. So a hypothetical image where PLTE index 0 is RGB #000000, index 1 is #555555, index 2 is also #555555, and index 3 is #FFFFFF, would just be treated as a grayscale image with black, dark gray, and white; instead of turning index 0 into white, index 3 into black, and treating indexes 1 and 2 as separate grays.

[pinobatch]

Does the concept of "indices appropriate to each shade" assume BGP=OBP0=OBP1=$E4 (%11100100, or 0 is white 1 is light gray 2 is dark gray 3 is black)? Because it appears steps 2 and 3 would fail with sprite cels containing opaque white.

[ISSOtm]

After some discussion, rather than the "skip embedded palette" flag, the following could be of use:

• A flag to dump the palette RGBGFX ended up with (post --fuzzy, for example) as text, which could then be used as an external palette spec as-is (Allow RGBGFX to take an external palette specification #487); also as an image.
• --fuzzy would allow treating similar enough colors as identical. Two proposed behaviors: one with a parameter, specifying a distance (distance function TBD) below which colors would be merged; and a parameter-less one, only usable with a CLI palette specified, that would coerce any unknown color into the closest (by the same distance) color. If using the parameter and the palette was specified on the CLI, indicate the threshold that would cause it to be merged.

[pinobatch]

Proposed shortcut forms for specifying a grayscale palette on the command line, implying an appropriate fuzziness:

• Hexadecimal: --bgp $E4 or --bgp $1B
• Binary: --bgp %11100100 or --bgp %00011011
  And have it emit a diagnostic if not all four 2-byte fields have distinct values.

[ISSOtm]

This is equivalent to what I just mentioned in another issue.

[ISSOtm]

For --fuzzy, what distance function should be used between colors? What range should it have on the CLI?

For the range, I'm thinking 0-255 integers, as it's what other numeric arguments use, and it can be essentially arbitrary.
For the distance function, I'm no expert in colors, but I'd guess sqrt(Δchroma² + Δluma²)?

[Rangi42]

First, --fuzzy makes sense iff rgbgfx has to construct its own palette index. If the input image has four palettes already, even if some entries are similar or identical, that's the palette it would use (right?).

Second, input PNGs generally have 8-bit color depth while output palettes have 5-bit, so at the least I'd expect allowing fuzziness as a binary switch to just ignore the low 3 bits of R, G, and B.

If the user wants even more fuzziness than that, it could be a number of low bits to drop, so 0 to 7 (or higher if they have a 16-bit-depth PNG for some reason).

Alternatively, sqrt(Δchroma² + Δluma²) is also fine, but I don't know how much of an improvement it would be for typical non-photorealistic input images; especially since the perceived colors will be different on different Game Boy screens and with different emulators' color adjustments. (So a formula meant to be based on perceived differences won't always be accurate; maybe go with the simpler-to-explain one of ignoring low bits.)

[ISSOtm]

--fuzzy makes sense even if a palette was specified on the CLI, in case it doesn't exactly match the color's images. (Param-less --fuzzy is intended for that exact use case, actually; I'm not sure what its semantics would be for an "auto-palette" invocation.)

Currently, all channels are truncated to 8-bit before any comparison, and I'm reluctant to simply truncate to 5-bit due to color curves potentially requiring further conversion before such a truncation. The behavior I have in mind is, if the image contains too many colors, compute the fuzziness threshold that would make it valid; if it's "sufficiently close" (TBD) to the specified threshold, notify the user of it.

[Rangi42]

That sounds fine.

For specifying CLI palettes, can they be specified as 8-bit or 5-bit? So if the user supplies a PNG palette, those colors would have to be exact unless --fuzzy is sufficient; but if they supply a .gbcpal, then the input's low 3 bits would be ignored for purposes of comparing the image and palette colors. (And if there were still too many colors, the fuzzy-matching attempt would not ignore those bits.)

[ISSOtm]

CLI palettes shall be specified either textually (in-line or from a file), or from a PNG file; in the latter case, colors can be specified exactly, and in the former, they shall be specified as 8-bit. (I'm not sure how to disambiguate an 8-bit spec from a 5-bit one, especially in (r,g,b,a) format.

I'm currently designing the palette input: either it's in-line -p <palette>, or it's from a file (-p @<path>). The file should either be a PNG (as determined by the first 8 magic bytes, though I fear there might be overlap with the alternative), or a binary file containing raw palettes.

My current design problems are:

• What textual format to use. I currently have in mind the following BNF:

  <palettes> = <palette> <sep> <palettes>
  <sep> = \n | ;
  <palette> = <color> | <color> <palette>
  <color> = (<num> <num> <num>) | (<num> <num> <num> <num>) | #<hex_color>
  

  What needs to be additionally figured out is whether <num>s are 8- or 5-bit, and whether <hex_color>s are 8- or 5-bit at well. (5-bit makes less sense for an input palette that might contain alpha.) I haven't thought of a good way to disambiguate this. Perhaps add a prefix-less | <hex_555> to <color>?
• If there might be any ambiguity between a file containing raw RGB555/RGB888 palettes, and a PNG magic header.
• If we want to also support a raw binary file as input (e.g. what RGBGFX would itself output), and how to disambiguate it from a textual file in that case. Or maybe only allow binary files for @, and rely on shell expansions for textual input files?

[Rangi42]

RGB555 could just be specified as a single number, the 2-byte encoding as displayed in BGB's VRAM viewer. So (R=15,G=6,B=3) would be 0ccf (or 0x0ccf or $0ccf if the base needs indicating, with 3279 being base-10). This would be equivalent to #783018, but also to #7B3119, or in-between #7A3019, etc.

For inline palettes, I'd suggest a comma to separate the three/four channels if spaces separate the colors, or vice-versa. No need for parentheses. Then #RRGGBB or #RRGGBBAA would be 8-bit hex colors, and could allow $xxxx for 5-bit input.

[ISSOtm]

There's no point in specifying palettes as a whole in decimal, since all displays are either per-channel (either decimal or hex), or global in hex. Hence, I guess #rrggbbaa/#rrggbb versus pal5 should be good enough. Or $pal5?

[Rangi42]

Explicit $ and # is more clear.

[ISSOtm]

So:

palettes = palette { ( ";" | "\n" ) { "\n" } palettes } ;
palette = color [ , color [ , color [ , color ] ] ] ;
color = "(" num num num [ num ] ")" | "#" rgba888 | "$" rgb555;

What about the file ambiguities?

[Rangi42]

Allowing only binary input for -p @<path> and only textual for -p <palette> sounds fine to me. So yeah, you'd use shell expansions to pass some pal.txt.

[ISSOtm]

Would it be interesting to allow taking tiles as input, and only using those? I think this might be useful if a tileset has already been generated, and one wants to ensure that the generated tilemap matches that tileset. (E.g. if several maps share a given tileset, you'd want tile IDs to match the tileset's, not the image's own.)

I had planned to also allow taking a tilemap as input, to allow finer control over duplication, but I'm less sure about the use cases for that.

[Rangi42]

Taking an input tileset to generate a tilemap definitely sounds useful. Taking a tilemap as input is also potentially useful: there are a few places where pret uses gfx.c's --preserve option to list tile IDs that shouldn't be removed as duplicates, because most duplicates are removed but a few aren't. An input tilemap would be easier to use than that.

[ISSOtm]

It seems really specific: if you already know the tilemap, in which cases would you need to change the input image? If you don't change the image, there's no point in using RGBGFX, is there?

[Rangi42]

Here are a couple examples:

• pokered's gfx/intro/gengar.png goes with three tilemaps in gfx/intro; however, the 2bpp needs to include two duplicate tiles. So we just don't remove any duplicates with rgbgfx, and post-process it with gfx.c to --preserve 0x19,0x76.
• pokecrystal's gfx/mobile/electro_ball.png has one duplicate tile, so we use --preserve=0x39. The alternative is having a messy-looking PNG that's manually deduplicated except for that tile.

You're right that it's a niche use case though. Plus neither of those goes with one single tilemap that could conveniently be used as rsgbgfx input. And there are other cases where it's convenient for us to store tiles in multiple PNGs, some of which need --preserve, and the Makefile combines them into one 2bpp for tilemapping purposes. So even with such an option, we'd probably keep gfx.c for postprocessing.

[ISSOtm]

Yeah, these look like use cases for matching disassemblies, which I'm not sure I want rsgbgfx to target. I'll try to keep those in mind, and incorporate them directly if they don't require too much effort; otherwise, we'll discuss their inclusion at a later stage.

[ISSOtm]

Would it be interesting to allow taking tiles as input, and only using those? I think this might be useful if a tileset has already been generated, and one wants to ensure that the generated tilemap matches that tileset. (E.g. if several maps share a given tileset, you'd want tile IDs to match the tileset's, not the image's own.)

I've tried designing this feature, and it seems very difficult at least if a palette spec is not given; otherwise, rgbgfx might decide on a different palette set (e.g. sorting the colours differently) and then the tiles won't be matched.

So I think a first implementation should be fine if it required -c to also be passed?

Additionally, I think there should be a "strict" mode where only the predefined tiles are allowed, and a "seed" mode where more tiles are allowed to be generated. Both seem like they should have use cases.

[ISSOtm]

I guess that depends on whether the "seed tileset" is provided as a PNG (has inherent colours, but may require its own set of processing flags), or as tile data (no colours, but easier to manipulate)... I'm leaning more towards the latter, personally.

[Rangi42]

Plan: -X can only mirror horizontally, -Y vertically (-m still mirrors both).

[Rangi42]

All we need to resolve this issue is -B/--background <color>.

[Rangi42]

Some of these points have been done, some have their own individual issues, and some (e.g. a list of tile indexes to preserve) will not be implemented.