Shutter speed adjustment

[takanotume24]

Hello. Thanks for all your support.

I am using ARKit mode on RTAB-MAP iOS version. I checked the images stored in the database and found that some photos were shaky. Probably due to insufficient shutter speed.

If the photos are shaky, does this affect map construction or positioning accuracy?

I am not familiar with camera controls in iOS, but is it possible to control shutter speed and ISO value from the app side?

If so, I think that allowing the user to fix the shutter speed and ISO value would improve positioning accuracy.
(Or maybe we could make sure they aren't affected)

Just an idea I came up with while using it. Please refer to it if you like.

[matlabbe]

I am not sure if we can change the camera parameters for the Main camera from ARKit API. When images are blurry, it will affect loop closure detection performance and also the texturing quality of the 3D model on RTAB-Map side. ARKit is using the UltraWide camera for positional tracking (VIO), so it would not be affected if the Main camera is blurry (unless the UltraWide camera can be also blurry on fast motions). Maybe in latest ARKit API we can have access to UltraWide camera instead of the Main camera. I see this post as a workaround (using AVCaptureMultiCamSession API), but not sure if it can work at the same time than ARKit VIO.

Other post referring an out of focus background issue: https://developer.apple.com/forums/thread/87909

[takanotume24]

I see that the shaky image affects the quality of the map. Thanks for letting me know.

Perhaps Apple has already taken into account the effects of shaky images to improve the accuracy of ARKit, so the ultra-wide camera side may have a faster shutter speed and higher ISO sensitivity.
If the developer cannot adjust the parameters of the main camera, but can still get a non-shaky image from the ultrawide camera, that might be promising!

I was unaware that the blurring could be caused by lens focal length issues. Thanks for sharing.
My gut feeling is that the images stored in the database seem to be in focus. I think the too slow shutter speed caused the shutter to stay open for too long, thus not capturing the fast movement.

[borongyuan]

Motion blur is more likely to occur in poorly lit scenes. For mobile phones, an alternative trick is to enable the flashlight.

I also observed some other non-traditional factors affecting shutter speed. For example, many mobile phone CMOS have the binning readout method. Generally speaking, using binning to read out a lower-resolution image may result in shorter shutter time and potentially higher image quality than reading out a image at the original resolution and then scaling it to the same size. I've also been consulting some hardware experts recently about the impact of binning readout on SNR.

[takanotume24]

I agree that brightening the environment is the most fundamental way to do this!
However, in vast indoor spaces, it can be difficult to illuminate with hand-held lights. We do not need a beautiful image with little noise, but one with a lot of noise but clear outlines, so I think it would be ideal to be able to manipulate the camera parameters.

I had no idea about the different shutter speeds depending on the resolution. Thanks for sharing the info.
I am curious if the same behavior is exhibited on iOS. If a developer requests a high resolution image and a low resolution image from the OS, it may end up providing a compressed version of the high resolution image, or as you say, it may actually read out the low resolution image.