Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mismatch lens detection with Canon RF 24-105 versions (STM / USM) in darktable #2057

Open
GitHubbero opened this issue Jan 19, 2022 · 22 comments · May be fixed by #2235
Open

Mismatch lens detection with Canon RF 24-105 versions (STM / USM) in darktable #2057

GitHubbero opened this issue Jan 19, 2022 · 22 comments · May be fixed by #2235
Assignees
Labels
bug lens Issue related to lens detection
Milestone

Comments

@GitHubbero
Copy link

I apologize, if the issue is already on GitHub, but i couldn't find anything.

While working with darktable, i saw it detects my lens wrong.
I used for the photo the Canon RF 24-105 F4-7.1 IS STM
but darktable detects it as Canon RF 24-105 F4L IS USM

I am sorry, if the problem is minor, but i can't work properly with exiv2 to check it myself. So i thought the problem of the lens detection have to be from exiv2 and its tags, because the lens in the exif-data is correctly (checked with exiftool and DPP4).
Hope you could help me. And if there is something unclear, text me.
Thank you all.

@GitHubbero GitHubbero added the bug label Jan 19, 2022
@postscript-dev postscript-dev added lens Issue related to lens detection and removed bug labels Jan 19, 2022
@postscript-dev
Copy link
Collaborator

Looking at the source code, there is an entry for:

61182, "Canon RF 24-105mm F4L IS USM"

If Exiv2 incorrectly identifies a lens then there are two solutions:

  1. You can create an Exiv2 lens configuration file on your computer, which will override any existing value for that lens ID (i.e., 61182).
  2. Add the lens data into Exiv2 - which has a global effect. To do this, you need to submit an image that we can add to the project for testing (under a GPL 2.0 license). If you are concerned about privacy, then leave the lens cap when taking a photo. After this is submitted, I will make the change to Exiv2.

@postscript-dev postscript-dev self-assigned this Jan 19, 2022
@GitHubbero
Copy link
Author

Thank you @postscript-dev ,
just to clarify, the "RF ... F4L" is detected but wrong. It should be the "RF...F4-7.1"
So the Lens code should be "61182 //25" but is wrongly "61182 //2"

As I am using Windows and can't figure out, how to use exiv2, maybe you can check the lens detection of exiv2?
Just to make sure that the problem is not from exiv2 but at darktable itself.

Thank you so much at this point

Canon_RF_24-105mm_F__4-7 1_IS_STM--105mm-1__125-F__7 1
"

@postscript-dev
Copy link
Collaborator

@GitHubbero: Usually in posts like yours, the lens value is missing and it is a simple case of adding an entry and a test file. However, this time it is different, as your lens is already defined in Exiv2.

The Exiv2 sample program mentioned in https://www.exiv2.org/manpage.html, is a small command line tool that enables viewing and editing of image metadata. I can use this to output the lens tag from your test file:

$ exiv2 --Print kt --key Exif.CanonCs.LensType Canon_RF_24-105_F4-7.1_IS_STM.jpg
Exif.CanonCs.LensType                         Unknown Lens (61182)

Although the lens is defined in Exiv2, it is currently not being found and this is something that I will fix. However, even if I submitted this today, due to release dates, it could still take more than 6 months before arriving in darktable. There is another solution though, that will only take a couple of minutes for you to complete.

If you add a lens configuration file to your computer then this will fix the problem on your machine. To do this in Windows:

  1. Open a new document in any text editor (e.g., Notepad, Wordpad, Libre Office Writer, MS Word, etc)
  2. Copy the following into the program:
[canon]
61182=Canon RF 24-105mm F4-7.1 IS STM
  1. Save the file in your home directory (i.e., C:\Users\<Your Windows username>, e.g., C:\Users\postscript-dev) as a text file (.txt) called: exiv2.ini . Some text editors automatically add .txt on the end of any filename and such a file will need to be renamed to exiv2.ini
  2. The new definition will now be displayed for that user in any software using the Exiv2 library (e.g. darktable, GIMP)

If I rerun the test from above, you can see that the output has changed.

$ exiv2 --Print kt --key Exif.CanonCs.LensType Canon_RF_24-105_F4-7.1_IS_STM.jpg
Exif.CanonCs.LensType                         Canon RF 24-105mm F4-7.1 IS STM

To make the correct value appear in darktable:

  1. Start/restart darktable
  2. Delete any existing files that have the incorrect lens value
  3. Re-add the files and now the lens entry of the image information will use the correct value

I don't use darktable much, so there may be a simpler way to refresh the files than what I have shown.

@GitHubbero
Copy link
Author

Thank you for your great work.
I will run the test tomorrow, and will inform you about the result.
Thanks a lot

@EC1000
Copy link

EC1000 commented Jan 23, 2022

I should be corrected by #1692 👍 (at least, the Canon RF 50mm F1.8 version is now correctly identified while it was identified as the F1.2 version with exiv2 0.27.5)

Note that all Canon RF lenses have the same LensType=61182.

@GitHubbero
Copy link
Author

GitHubbero commented Jan 23, 2022

[canon]
61182=Canon RF 24-105mm F4-7.1 IS STM

Thank you @EC1000 . I knew that the RF lenses have all the same type, but they are different with the addition e.g. //2 aren't they?
Is there a way to implement these differentiations into the manual exiv2.ini (from @postscript-dev ) for the time until the update? (I just need the defferentiations of 2 or 3 lenses)
Or can i somehow update the main data by myself? I don't use exiv2 on my own, just darktable.

I found exiv2.mo files in my share folder of darktable. I assume i have to exchange them with the new one, am I right?
But where can i find the newest file in the exiv2/main branch?

@postscript-dev
Copy link
Collaborator

@EC1000:

Note that all Canon RF lenses have the same LensType=61182.

I have no experience with Canon cameras or lenses, so this is helpful.

@GitHubbero:

I knew that the RF lenses have all the same type, but they are different with the addition e.g. //2 aren't they?

The //2 is only a comment and is ignored by the software. If not using a lens configuration file, Exiv2 will produce a list of lenses that match the definitions that it has.

Is there a way to implement these differentiations into the manual exiv2.ini (from @postscript-dev ) for the time until the update? (I just need the defferentiations of 2 or 3 lenses)

Out of interest, what are your other lenses?

@GitHubbero
Copy link
Author

GitHubbero commented Jan 23, 2022

Out of interest, what are your other lenses?

There is the RF24-240 4-6.3 and RF 100-400 5.6-8

@EC1000
Copy link

EC1000 commented Jan 23, 2022

What exiv2 version are you using?

If you have three RF lenses that are not identified, you can find a possible solution here (set 61182=000 in .exiv2 file, so exiv2/dt will pick the LensModel name instead of the LensType; note the small difference: without "Canon" at the beginning).

All should be ok with future exiv2 V1.00 version :)

@postscript-dev postscript-dev added this to the v1.00 milestone Jan 23, 2022
@GitHubbero
Copy link
Author

If you have three RF lenses that are not identified, you can find a possible solution here

On the funny side, my 100-400 lens is identified correctly (couldn't figure out how it's with the 24-240 because it's just on my wishlist by now)
So I thought the problem has to be with maybe the naming of the lens or the detection. Following the correctly identification of the one lens, there should no problem with the single Lens-ID for all RF lenses

I couldn't find out which version I am using, because I just have darktable installed. But it should be 0.27.5, at least 0.27.4.

@clanmills
Copy link
Collaborator

It's very likely that the darktable "About" box (or preference panel) will list the version of every library.

@GitHubbero
Copy link
Author

GitHubbero commented Jan 23, 2022

It's really weird, because there is no "about" box in darktable. I searched for it before, and also now but i can't find something like that. Nor anything like this in the options/preferences
But instead I compared the date of the used exiv2.mo file of darktable and it was just 2 days "newer" than the newest version of exiv2 (23.10.2021 / 21.10.2021) So I assume it's the current version v0.27.5

@clanmills
Copy link
Collaborator

Some of the darktable folks are on the Exiv2 chat server and they'll advise. When I retired from Exiv2 last year, I left the chat room. Time for me to leave Exiv2 to other great young guys like @postscript-dev.

@postscript-dev
Copy link
Collaborator

@GitHubbero: Sorry for the delay replying, I needed to finish some other jobs first. I don't have much free time this week but your issue is the next task that I intend to work on.

@clanmills
Copy link
Collaborator

clanmills commented Feb 6, 2022

@postscript-dev Thanks for getting involved with this, Peter. I know you're doing lots of good stuff for Exiv2 AND you have a life to live! I'm sure @GitHubbero appreciates your efforts.

@GitHubbero I've thought of other ways to check the version of exiv2 being used by darktable.

  1. use $ lsof | grep darktable | grep exiv2
    On my Linux VM (polluted by building exiv2) it shows libexiv2.so.0.27.5.1

  2. If you are using an anonymous exiv2 shared object, grep it for the string '0.27.5' (which is baked into the shared object) by the compiler/linker: For example:

539 rmills@rmillsm1-ubuntu:~/gnu/github/exiv2/0.27-maintenance/build/lib $ ls -l
total 4616
lrwxrwxrwx 1 rmills rmills      14 Feb  6 11:18 libexiv2.so -> libexiv2.so.27
-rwxrwxr-x 1 rmills rmills 3675944 Feb  6 11:18 libexiv2.so.0.27.5
lrwxrwxrwx 1 rmills rmills      18 Feb  6 11:18 libexiv2.so.27 -> libexiv2.so.0.27.5
-rw-rw-r-- 1 rmills rmills 1046168 Feb  6 11:17 libexiv2-xmp.a
540 rmills@rmillsm1-ubuntu:~/gnu/github/exiv2/0.27-maintenance/build/lib $ grep 0.27.5 libexiv2.so.0.27.5 
Binary file libexiv2.so.0.27.5 matches
541 rmills@rmillsm1-ubuntu:~/gnu/github/exiv2/0.27-maintenance/build/lib $ grep 0.27.4 libexiv2.so.0.27.5
542 rmills@rmillsm1-ubuntu:~/gnu/github/exiv2/0.27-maintenance/build/lib $

@GitHubbero
Copy link
Author

@postscript-dev I appreciate definitely you effort. And there is no pressure by my side.

And thank you @clanmills for your thoughts, but I use Windows as platform and so it's always a bit harder around the darktable environment. Sorry for that.
But I can't imagine, that the newest darktable version (which I installed only about 4 weeks ago) uses an old exiv2 version. Therefore I assume I have the exiv2 version 0.27.5, but haven't got a proof.

@clanmills
Copy link
Collaborator

@GitHubbero I forgot you're using Windows. BTW I like Windows (but don't tell anybody). Use procexp.exe to determine which exiv2.dll is being used. https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer We'll have to update the exiv2 build to add a public "version" to the DLL properties which would be visible in procexp.exe

If your version of darktable supports BMFF (Canon .CR3, Apple .HEIC), then it's using exiv2 v0.27.4 or later.

Using grep, or dumpbin will probably reveal the version string baked into exiv2.dll.

@GitHubbero
Copy link
Author

If your version of darktable supports BMFF (Canon .CR3, Apple .HEIC), then it's using exiv2 v0.27.4 or later.

Yes I am working with .CR3 and darktable shows me correctly the image data. Except the lens data of RF 24-105

@postscript-dev
Copy link
Collaborator

@GitHubbero:
I am sorry for not working on your issue sooner. Thanks for being patient. I have not had much free time recently.

@hassec:
While working on this issue, I have been looking at the src/canonmn_int.cpp particularly the printCsLensTypeByMetadata() function. I see that you updated this recently and I have a question. In

exiv2/src/canonmn_int.cpp

Lines 2781 to 2784 in b2b6d83

if (flMin != exifFlMin || flMax != exifFlMax || exifAperMax < (aperMaxShort - .1 * tc) ||
exifAperMax > (aperMaxTele + .1 * tc)) {
continue;
}
f

where does the 0.1 value come from?

@GitHubbero
Copy link
Author

@GitHubbero: I am sorry for not working on your issue sooner. Thanks for being patient. I have not had much free time recently.

@postscript-dev you don't have to apologize. I am glad that you take the trouble to solve the problem.
Thank you all so much.

postscript-dev added a commit to postscript-dev/exiv2 that referenced this issue May 11, 2022
Previously, when translating `Exif.CanonCs.LensType`,
`Exif.CanonCs.MaxAperture` was used as the upper bound to find a
potential match. In Exiv2#2057, the test file produced a value outside of
that range and the lens was not identified.

The fix uses `Exif.CanonCs.MinAperture` instead.
@postscript-dev postscript-dev linked a pull request May 11, 2022 that will close this issue
@postscript-dev
Copy link
Collaborator

@GitHubbero:
For the past 3 months, I haven't been able to contribute to Exiv2 as I wanted to. Now I have more time and can finish jobs such as yours that are still outstanding. Thanks for being very patient.

I have submitted a PR to fix your problem (#2235) which will now output:

$ exiv2 --Print kyyvt --key Exif.CanonCs.LensType Canon_RF_24-105_F4-7.1_IS_STM.jpg
Exif.CanonCs.LensType                        Short      61182  Canon RF 24-105mm F4L IS USM *OR* Canon RF 24-105mm F4-7.1 IS STM

Providing a list of these two lenses is the best we can do at the moment.

When the PR is merged, the issue will be fixed on the main branch.

Thanks again.

@kmilos
Copy link
Collaborator

kmilos commented May 12, 2022

FWIW, I have dropped using Exif.CanonCs.LensType in darktable when 61182 is detected, it uses vendor Exif.Canon.LensModel for RF lenses now on master (and the upcoming 4.0).

postscript-dev added a commit to postscript-dev/exiv2 that referenced this issue May 27, 2022
This reverts commit d8d5a3a.

Revert to fix the lens detection in a different way.
postscript-dev added a commit to postscript-dev/exiv2 that referenced this issue May 27, 2022
The `Canon RF 24-105 F4-7.1 IS STM` lens is reported as using an
`Exif.CanonCs.MaxAperture` of `F7.3` instead of `F7.1`. As a result,
the translation of `Exif.CanonCs.LensType` is missing.

Fix updates the `CanonEV()` decode function to provide the correct
value.
@kevinbackhouse kevinbackhouse modified the milestones: v0.28.0, Backlog Nov 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug lens Issue related to lens detection
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants