Why is "Display P3 - Green Primary" different from the ICC profile?

[hminle]

Hi Colour Team,

I compare Display P3 color profile in my macbook and your models of Display P3, and found that, the Green primary is quite different:

Why is that?
Could you provide any detail here?

Thank,

[sobotka]

Greetings Hoang. Welcome.

I believe what you are likely spotting is the discrepancy between the Display P3 specification's definitions of the primaries and the ICC protocol driven values. If you adapt the values you are seeing in your ColorSync utility from ICC D50 to the standardized white point for Display P3 being D65 xy coordinates of 0.3127, 0.3290 you will likely discover that the values will match.

The reason is that the ICC V4 protocol has a fixed and constant Profile Connection Space. When using XYZ as the Profile Connection Space, all values must be chromatically adapted to ICC D50. Hence the Display P3 values ColorSync displays should in fact be the chromatically adapted ICC protocol values.

./iccinspector.py ../Apple-ICC-Peeler/iccs/Display\ P3.icc 
                 Profile Size:     548            
           Preferred CMM Type:     "appl"
              Profile Version:     4.0.0
       Profile / Device Class:     "mntr", "Display device profile"
      Profile Data Colorspace:     "RGB ", "RGB"
                  Profile PCS:     "XYZ ", "nCIEXYZ or PCSXYZ"
        Date and Time Created:     2017-07-07 13:22:32
       Profile File Signature:     "acsp"
             Primary Platform:     "APPL", "Apple Computer, Inc."
                Profile Flags:     0
          Device Manufacturer:     "APPL"
                 Device Model:     ""
            Device Attributes:     0
                  Profile PCS:     0, "Perceptual"
               PCS Illuminant:     [X: 0.964202880859375, Y: 1.000000000000000, Z: 0.824905395507812][x: 0.345702921980228, y: 0.358537532756706, Y: 1.000000000000000]
              Profile Creator:     "appl"
                   Profile ID:     ca1a9582257f104d389913d5d1ea1582
                     Reserved:     00000000000000000000000000000000000000000000000000000000
              Tag Table Count:     10
               Tag Signatures:     "desc",  Offset: 252,     Size: 101,     ["desc", 0, 11, "Display P3", 0, 0, "", 0, 0, ""]
                                   "cprt",  Offset: 356,     Size: 35,      ["text", 0, "Copyright Apple Inc., 2017"]
                                   "wtpt",  Offset: 392,     Size: 20,      ["XYZ ", 0, [X: 0.950454711914062, Y: 1.000000000000000, Z: 1.089050292968750][x: 0.312700492477296, y: 0.329000938769158, Y: 1.000000000000000]]
                                   "rXYZ",  Offset: 412,     Size: 20,      ["XYZ ", 0, [X: 0.515121459960938, Y: 0.241195678710938, Z: -0.001052856445312][x: 0.682041335838536, y: 0.319352688041699, Y: 0.241195678710938]]
                                   "gXYZ",  Offset: 432,     Size: 20,      ["XYZ ", 0, [X: 0.291976928710938, Y: 0.692245483398438, Z: 0.041885375976562][x: 0.284548009576635, y: 0.674632325605603, Y: 0.692245483398438]]
                                   "bXYZ",  Offset: 452,     Size: 20,      ["XYZ ", 0, [X: 0.157104492187500, Y: 0.066574096679688, Z: 0.784072875976562][x: 0.155896069287142, y: 0.066062019259887, Y: 0.066574096679688]]

[KelSolaar]

Hi,

I haven't looked at how they get the normalised XYZ values but indeed as @sobotka says, you need to chromatically adapt the primaries from D65 to D50:

import colour
>>> colour.xy_to_XYZ(
...     colour.chromatically_adapted_primaries(
...         colour.models.DISPLAY_P3_COLOURSPACE.primaries,
...         colour.ILLUMINANTS['CIE 1931 2 Degree Standard Observer']['D65'],
...         colour.ILLUMINANTS['CIE 1931 2 Degree Standard Observer']['D50'])) * 0.692
array([[1.48184488, 0.69200000, -0.00388932],
       [0.29343346, 0.69200000, 0.03142031],
       [1.72315629, 0.69200000, 8.89448088]])

Green matches quite well here now!

[sobotka]

In many ICC's you'll also find a chad tag, which is the chromatic adaptation matrix used to derive the ICC D50 aligned values. Using that matrix, you can typically figure out what the native values were by routing the primaries through the matrix.

Here's a proof of principle using my unfinished iccinspector Python tool:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import colour
import array
import sys
import struct
import numpy
from iccinspector import iccinspector
import argparse


if __name__ == "__main__":
    try:
        parser = argparse.ArgumentParser(
            prog="Apple Adapted Primaries"
        )
        parser.add_argument(
            "iccfile",
            type=argparse.FileType("rb")
        )

        args = parser.parse_args()

        numpy.set_printoptions(15)

        with args.iccfile as f:
            s = memoryview(f.read())

            iccFile = iccinspector.iccProfile()
            iccFile.read(s)

            chad = iccFile.tags["tag"][
                numpy.where(iccFile.tags["signature"] == "chad")
            ][0]

            CAT_D65_to_D50 = numpy.reshape(chad.type.value, (3, 3))

            rXYZ = iccFile.tags["tag"][
                numpy.where(iccFile.tags["signature"] == "rXYZ")
            ][0]
            gXYZ = iccFile.tags["tag"][
                numpy.where(iccFile.tags["signature"] == "gXYZ")
            ][0]
            bXYZ = iccFile.tags["tag"][
                numpy.where(iccFile.tags["signature"] == "bXYZ")
            ][0]

            AppleP3D50toXYZ = numpy.transpose(
                [
                    rXYZ.type.value[0].XYZ,
                    gXYZ.type.value[0].XYZ,
                    bXYZ.type.value[0].XYZ
                ]
            )

            adapted_XYZ = numpy.matmul(
                numpy.linalg.inv(CAT_D65_to_D50),
                AppleP3D50toXYZ
            )
            print(
                "The xyY coordinates adapted via the `chad` tag: \n{}".format(
                    colour.XYZ_to_xyY(adapted_XYZ.T)
                )
            )

    except Exception as ex:
        raise ex

And the output:

./basic-proof-of-principle-adaptation.py ./iccs/Display\ P3.icc 
The xyY coordinates adapted via the `chad` tag: 
[[ 0.68001343314178   0.320010797932105  0.228979429053466]
 [ 0.26499931293778   0.690002466701867  0.691741223367124]
 [ 0.149993732755441  0.059991279046883  0.079273583521213]]

The discrepancy between the values listed here and the specification's standard values ultimately comes down to the quantisation within the ICC file itself.

[hminle]

Thank guys, I got it now.

[hminle]

Hi, to conclude, if I use Photoshop convert profile my image to Display P3 in my macbook, then I use colour to visualize its CIE diagram, what exactly should I do in my code?

[sobotka]

@hminle sadly, the answer is "It depends".

The problem is that on ICC V4 enabled software, there is no longer a means to reliably get an absolute colourimetric output. So if the goal is to get to known values, you are out of luck with the ICC chain now thanks to V4's exceptionally poor design choice here.

Better would be to use the Colour science library to load the image from a known state and transform it with granular control to the XYZ domain, and onward to xyY as required for your plots.

Not sure if that helps. The question depends on what exactly you are trying to achieve and your contexts.

[hminle]

Hi @sobotka , my goal is I want to develop some algorithm that can convert images from a small gamut to a bigger one like Display P3 and compare it with Photoshop. Then I need to calculate deltaE between my output and Photoshop output. That's my context. But I am not sure, if I use colour library to compare their delta E, is it good or not?

[sobotka]

Colour is excellent. ICC V4 sadly is missing that crucial ability to get a sense of the values mapped under an ICC absolute colorimetric approach.

That means specifically that the white point in the output can’t be firmly known. As such, evaluations will be off. If you limit yourself to ICC V2, absolute colourimetric results are possible, assuming the ICC attributes are kept firmly in view.

[hminle]

Hi @sobotka , so if I create a custom profile in Photoshop like this:

and I convert my sRGB image to this profile using Photoshop.
Then I use colour to compare it with my output:

reconstructed_img = ...
photoshop_img = ... # Read photoshop Display P3 image converted by above profile
reconstructed_img = colour.models.RGB_to_XYZ(reconstructed_img, colour.ILLUMINANTS['CIE 1931 2 Degree Standard Observer']['D65'], colour.ILLUMINANTS['CIE 1931 2 Degree Standard Observer']['D65'], colour.models.DISPLAY_P3_COLOURSPACE.RGB_to_XYZ_matrix)
reconstructed_img = colour.models.XYZ_to_Lab(reconstructed_img)
photoshop_img = colour.models(photoshop_img, colour.ILLUMINANTS['CIE 1931 2 Degree Standard Observer']['D65'], colour.ILLUMINANTS['CIE 1931 2 Degree Standard Observer']['D65'], colour.models.DISPLAY_P3_COLOURSPACE.RGB_to_XYZ_matrix)
photoshop_img = colour.models.XYZ_to_Lab(photoshop_img)
colour.delta_E(reconstructed_img, photoshop_img, method='CIE 2000')

Am I right?

[Myndex]

....The other problem is Z of XYZ of red color is negative when adapted to D50 (as you can see on first picture). Dunno why, either Apple did an oopsie or it is just like that when adapted with "chad".

It's not Apple who are just using the established P3 primaries. You'll see the same thing happen with DCI P3 and the 6300K WP, and for that matter, happens with some other WGC spaces when adapting between D65 and D50.

[Myndex]

Valerii, if you dig into ICC you'll see they themselves say that (paraphrasing) "while the spec indicates clamping at zero, it is advisable to allow negative numbers in XYZ when transiting the PCS... etc etc".

As for different opinions: calculate the matrix yourself using only the chromaticity coordinates and you will derive the same thing.

As far as who supports or does not support what exactly? If you are using V4 ICC profiles you are required to CAT into D50. It's one of the reasons I avoid ICC for things like video, and stick to LUTs in OCIO.

There are colorspaces today that have their imaginary primaries in negative positions, as was the case back in 1931. XYZ was created for mathematical convenience: make everything positive as that is mathematically simpler when you have to do all your calculations on a slide rule.

[Myndex]

@ValZapod

I mean this. https://bugzilla.mozilla.org/show_bug.cgi?id=1250461 In chrome it is still present.

First that issue has literally NOTHING to do with the TRC. It has to do with the primaries being farther apart/different coordinates.

Second, most browsers are only supporting v2 profiles (if at all), and unsupported profiles tend to get ignored, and the result interpreted as sRGB or just as display.

And in v2 profiles, the media whitepoint would NOT necessarily be D50, if the device/space was D65, then D65 would be indicated, and the spec just required to adapt into D50. This caused more confusion, so in the V4 spec they changed to make everything to be pre-adapted to D50.

This is one of several reasons that ICC is not a good choice for D65->D65 conversions.

[sobotka]

I think you missed Andrew’s point? All of which are well stated and on the mark, but perhaps burying the lede a bit…

v4 is a bedshit, as it doesn’t allow for a white point beyond ICC D50?

And good luck relying on the chad tag…

[sobotka]

The problem is that ICC V4 is such a catastrophic bed shit that the chad tag essentially becomes soaked in ambiguity. I don’t think there’s a way out of V4’s abject failure unless ICCMax somehow comes back from the land of the dead.

[sobotka]

You are ignoring the most important aspect which is what version of ICC is being harnessed. Without that, the values are going to be inconsistent and meaningless.

[hminle]

You are ignoring the most important aspect which is what version of ICC is being harnessed. Without that, the values are going to be inconsistent and meaningless.

But as what you said earlier, it's just the difference between D50 and D65, right? What am I missing here?
So what should I do? Do you have any suggestion?

Btw, I have checked this link: http://www.color.org/chardata/rgb/DCIP3.xalter
They provide DCI P3 with D65 whitepoint --> meaning that it is Display P3.
What if I install this profile for Photoshop?

Thanks

[sobotka]

I don’t have an answer as the problem is subtly complex when trying to deal with absolute values in an ICC system of unknowns.

For more information, this post from Graeme Gill of ArgyllCMS is extremely informative for folks unfamiliar with the subject.

In terms of your goal of gamut mapping, not to discourage you, but the halls of colour are littered with the corpses of Good Ideas that attempted and failed miserably to gamut map upwards to a wider gamut.

With that said, if I were attempting such a thing, I would:

1. Avoid ICCs and consumer tools altogether.
2. Deal in strict colourimetric ground truths.
3. Use Colour
4. Explicitly state the constraints for such a gamut mapping intent. Display referred, scene referred, etc.

[hminle]

Hi @sobotka , so you mean that even I use ICC v4 color profile (http://www.color.org/chardata/rgb/DCIP3.xalter) in Photoshop, I still cannot draw its CIE diagram in Colour library, right?

[sobotka]

You can of course use Colour. The validity of the values is the issue.

The problem is that it is very tricky to impossible to get the native colourimetry of something out of an ICC profile with V4, as there is no way to get absolute colourimetric output. It’s all relative now to ICC D50, essentially obfuscating the true colourimetry.

Read that link on the arts tag for more clarity here.

[hminle]

You are ignoring the most important aspect which is what version of ICC is being harnessed. Without that, the values are going to be inconsistent and meaningless.

Hi @sobotka , yes I've read the article.
So if I do know I am using ICC V4, then my code above is valid, right?

[sobotka]

ICC V4 is the problem.

[hminle]

ICC V4 is the problem.

even if I use perceptual rendering intent in Photoshop?

[sobotka]

Perceptual would be even worse. There are four rendering intents, two of which are potentially science based:

1. Absolute colourimetric
2. Relative colourimetric
3. Perceptual
4. Saturation

Items 1. and 2. are the only potential colour science approaches, where two is adapted to some unique white.

Items 3. and 4. are no rules, vendor secret sauce.

[hminle]

Thank @sobotka a lot, I learned a lot from you.
Wow, this field is the whole new universe for me.