Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve explanation on how to migrate from 151 to 152 color management #30305

Open
philipswan opened this issue Jan 11, 2025 · 7 comments
Open

Comments

@philipswan
Copy link

philipswan commented Jan 11, 2025

Description

I feel that certain revisions to instructions provided in "Updates to Color Management in three.js r152" would benefit the community.

Solution

Merge the following text into the current explanation where appropriate...

1. Renderer Settings

Property Old Name New Name Default Behavior Change?
Renderer output encoding renderer.outputEncoding renderer.outputColorSpace Yes: From THREE.LinearEncoding to THREE.SRGBColorSpace (r152).
Tone mapping renderer.toneMapping Same No: Default remains THREE.NoToneMapping.

Texture Properties

Property Old Name New Name Default Behavior Change?
Texture encoding texture.encoding texture.colorSpace No: Default remains THREE.LinearEncoding (non-color textures).
sRGB textures THREE.sRGBEncoding THREE.SRGBColorSpace No: Only the name changed.
Linear textures THREE.LinearEncoding THREE.LinearSRGBColorSpace No: Only the name changed.

Shader Changes

Function/Include Old Name New Name Default Behavior Change?
GLSL fragment includes <encodings_pars_fragment> <color_space_pars_fragment> No: Name changed; behavior remains the same.
GLSL linear-to-output function linearToOutputTexel() linearToColorSpace() No: Name changed; behavior remains the same.
GLSL sRGB-to-linear function sRGBToLinear() srgbToLinear() No: Name changed; behavior remains the same.

Material Properties

Property Old Behavior New Behavior Default Behavior Change?
Colors and textures on materials Assumed sRGB Assumed sRGB Yes: Colors are now converted from sRGB to Linear by default.

Global Defaults

Property Old Behavior New Behavior Default Behavior Change?
Color management toggle THREE.ColorManagement.enabled Same Yes: Default is now true (enabled).

Migration Steps

Previously inputs, outputs, and blending were all done in the sRGB color space (Note: "colorSpace" was formerly called "encoding"). With the improved methodology, three.js still accepts sRGB inputs, but now it converts them to linear. All blending is done in linear colorSpace. Finally, the renderer converts the finished scene back to the sRBG color space.

Here's what you need to do:

  1. Set the color space of textures with color explicitly to THREE.SRGBColorSpace. Textures that use non-color data (e.g., normal maps, height maps) can be left as-is, as they default to THREE.LinearSRGBColorSpace.

  2. For all fragment shaders, if the old fragment shader produced sRGB values, under the new methodology it needs to generate linear values. To adapt the shader, apply the linearToOutputTexel conversion function before assigning the pixel value to gl_FragColor. Here's an example:

#include <color_space_pars_fragment>

// Old (output assumed to be sRGB):
gl_FragColor = vec4(color, 1.0);

// New (convert to linear space):
gl_FragColor = vec4(linearToOutputTexel(color), 1.0);
This ensures the shader aligns with the updated linear workflow.

<!--EndFragment-->

You can also include colorspace_fragment after assigning gl_FragColor.

gl_FragColor = vec4(color, 1.0);
#include <colorspace_fragment>

Alternatives

If any of the proposed new instructions are inaccurate please correct them.

Explaining what the following two lines do in more detail might help.

#include <tonemapping_fragment>
#include <encodings_fragment>

Additional context

No response

@philipswan philipswan changed the title Explanation of how to migration from 151 to 152 color management Improve explanation on how to migrate from 151 to 152 color management Jan 11, 2025
@donmccurdy
Copy link
Collaborator

donmccurdy commented Jan 13, 2025

Hi @philipswan! Thanks for sharing the feedback and writing up these tables. I'm not sure if I'm eager to make major changes to the migration guide at this stage, but I'd at least be happy to offer some thoughts on the text here, and to include a link out to it from the current migration post if that feels helpful.

Renderer Settings

This section looks accurate to me.

Texture Properties

LinearEncoding has been replaced by not one but two enum values: NoColorSpace and LinearSRGBColorSpace.

The default for THREE.TextureLoader is NoColorSpace. NoColorSpace is suitable for things like normal maps and metalness maps: they aren't color!

LinearSRGBColorSpace is likely to be what your .hdr and .exr files use; these are still color. The default for HDR loaders is LinearSRGBColorSpace (since that's almost always what they'll use). Some loaders (like KTX2Loader) identify the color space from the file itself.

In current workflows NoColorSpace and LinearSRGBColorSpace have the same effect on the renderer — no conversion is made — but for future wide gamut and HDR workflows that may not be true... so the difference is important.

Shader Changes

Correct about the chunk names. I would not necessarily advise relying on internal GLSL function names, these may change.

Material Properties

We assume sRGB and convert to Linear-sRGB only for hexadecimal and CSS-style color representations. More details in:

Migration Steps

All blending is done in linear colorSpace. Finally, the renderer converts the finished scene back to the sRBG color space.

I wish it were this simple. In general this is the goal. We cannot do alpha blending in Linear-sRGB color space without post-processing. Otherwise, we render in Linear-sRGB, convert to sRGB, then blend and composite. In the WebGPURenderer I believe the behavior is currently more as you describe, as is more ideal.

Set the color space of textures with color explicitly to THREE.SRGBColorSpace. Textures that use non-color data (e.g., normal maps, height maps) can be left as-is, as they default to THREE.LinearSRGBColorSpace.

See above. In general it's very likely that your PNG and JPG color textures are sRGB, but this is an authoring choice and we do support other values. If in doubt use sRGB yes.

For non-color data use NoColorSpace. For HDR textures the defaults are probably fine, and typically this is LinearSRGBColorSpace (which is also the working color space for rendering).

To adapt the shader, apply the sRGBToLinear conversion function before assigning the pixel value to gl_FragColor

Possibly a typo here: the rendering takes place in Linear-sRGB, so at output the shader converts from Linear-sRGB to sRGB, not the other way around. You could instead include colorspace_fragment just after assigning gl_FragColor a Linear-sRGB value, which will convert it to the output color space (by default, sRGB).

@philipswan
Copy link
Author

@donmccurdy Yes, I get that its always tricky to make the trade between polishing documentation and getting the next major improvement out. That said, I don't think we want the community to feel that the cost of keeping three.js up-to-date is high. I remember feeling that way about Unity back in the day.

Ten+ years ago I had considerable expertise on blending and color spaces, but I have to admit that I'm rusty now. My recollection is that a lot of people were getting it wrong back then.

One trick that I do remember is that it was helpful to create a test where you have a top layer (anti-aliased text of col1 on a background of col2, typically with pre-multiplied alpha) which you attempt to blend onto a background of col1. Then test scaling the top layer image before compositing it. The correct result is a flat image of col1. If you can see an outline of the text in the composited result, then there's something wrong with how the scaling or blending algorithms are handling the alpha.

Thanks for spotting the typo - I'm getting dyslexic!

@philipswan
Copy link
Author

I'm reflecting on part 2 of my Migration Steps section and I think that we want the outputs of shaders to be linear as the renderer should automatically convert the final composited image to SRGB. Could you clarify what you meant by "We cannot do alpha blending in Linear-sRGB color space without post-processing."? "Linear-sRGB color space" is kind of an oxymoron.

@donmccurdy
Copy link
Collaborator

donmccurdy commented Jan 13, 2025

I think that we want the outputs of shaders to be linear as the renderer should automatically convert the final composited image to SRGB...

In a post-processing workflow, yes, conversion of the composited image to sRGB is a separate pass over the entire image at the end. If you are not using post-processing, then after your shader there is nothing else. The output of each shader is written to the drawing buffer, so we have no choice but to blend/composite in sRGB space, three.js doesn't apply another pass over the entire image for performance reasons.

You may of course also choose to set up your post-processing pipeline differently. There are multiple diverging post-processing implementations for three.js today, so a more complete explanation of color management — with post-processing enabled — is more effort than I can carve out of nights and weekends at the moment. :)

"Linear-sRGB color space" is kind of an oxymoron.

There are any number of possible linear color spaces, so I'm using the term "Linear-sRGB" as a shorthand for the particular color space with linear transfer functions, Rec. 709 "sRGB" primaries, and D65 white point. As compared to (for example) the "Linear-P3" color space which has linear transfer functions, wide gamut P3 primaries, and D65 white point. More on this in the color management guide. A more precise name for the three.js working color space would be “Linear-Rec709-D65” but I've generally assumed that would confuse readers more.

@Mugen87
Copy link
Collaborator

Mugen87 commented Jan 13, 2025

In the WebGPURenderer I believe the behavior is currently more as you describe, as is more ideal.

To confirm, yes. Tone mapping and color space conversion is not applied inline anymore but uniformly to the entire image in a separate pass. Hence, alpha blending with or without post processing produces equal results since it's always done in linear-sRGB color space.

When not using PostProcessing, tone mapping and color space conversion is applied as a separate internal render pass by WebGPURenderer. If you are using PostProcessing, this is also done as a internal separate pass but potentially combined with previous effects from the effect chain.

@philipswan
Copy link
Author

I probably shouldn't have added, "Linear-sRGB is kind of an oxymoron." It looks like it's a legit term defined by the W3C.

What I'd like to understand is the "We cannot do alpha blending in Linear-sRGB color space without post-processing." part.

@donmccurdy said, "In a post-processing workflow, yes, conversion of the composited image to sRGB is a separate pass over the entire image at the end." So this implies that it's not what three.js does by default.

But the color management docs define: a) a "working color space" where "Rendering, interpolation, and many other operations must be performed in an open domain linear working color space.", and b) a "Output color space" where "Output to a display device, image, or video may involve conversion from the open domain Linear-sRGB working color space to another color space. This conversion may be performed in the main render pass (WebGLRenderer.outputColorSpace), or during post-processing."
(emphasis added)

Also, @Mugen87 said, "Tone mapping and color space conversion is not applied inline anymore but uniformly to the entire image in a separate pass."

There are a bunch of terms and concepts here that in my mind don't perfectly line up. But, I'll hazard a guess. Is it...

The default behavior prior to R??? was that the output of each shader was written directly to the drawing buffer, with Three.js opting to perform blending and compositing in sRGB space. This approach was chosen to avoid the performance overhead of an additional final pass over the entire image. However, blending in sRGB space was fundamentally incorrect for lighting calculations, as its non-linear nature prevented developers from implementing algorithms that accurately aligned with the physics of light behavior.

As of R???, Three.js introduced a significant change by rendering into an intermediate buffer called the "?" that operates in the working color space. This working color space is unconstrained linear (which is different from sRGB), with Rec. 709 primaries (same as sRGB) and a D65 white point (also the same as sRGB). This configuration is commonly known in the industry as "linear-sRGB."

This improvement enables Three.js to perform all pixel manipulation operations in a linear color space, preserving the physical accuracy of lighting interactions. There is now an additional final pass that copies the rendered image from the working buffer (term?) to the drawing buffer. Tone mapping and color space conversion from linear-sRGB to the color space of the drawing buffer (typically sRGB) are now performed in this pass.

@donmccurdy
Copy link
Collaborator

The default behavior prior to R??? was that the output of each shader was written directly to the drawing buffer ...

This remains true for all versions of three.js today, unless you're using the (still experimental) WebGPURenderer. The improved behavior described in the later part of your comment currently applies only to WebGPURenderer, but can be also be achieved in WebGLRenderer with a (opt-in) post-processing pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants