-
Notifications
You must be signed in to change notification settings - Fork 6.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
drivers: video: introduce "GET" sub-operations #78603
base: main
Are you sure you want to change the base?
Conversation
There are various ways that the controls can be implemented in the drivers, such as the image sensors: Split functions for every controls. static int sensor_get_ctrl_exposure(const struct device *dev, uint32_t cid, uint32_t *value)
{
const struct sensor_config *cfg = dev->config;
uint16_t u16;
int ret;
switch (cid & VIDEO_CTRL_GET_MASK) {
case VIDEO_CTRL_GET_CUR:
ret = sensor_read_reg(&cfg->i2c, SENSOR_EXPOSURE_REG, &u16, 2);
*value = u16;
return ret;
case VIDEO_CTRL_GET_MIN:
*(uint32_t *)value = 0;
return 0;
case VIDEO_CTRL_GET_MAX:
*(uint32_t *)value = 320;
return 0;
case VIDEO_CTRL_GET_DEF:
*(uint32_t *)value = 9;
return 0;
default:
return -EINVAL;
}
}
static int sensor_get_ctrl(const struct device *dev, unsigned int cid, void *value)
{
switch (cid & ~VIDEO_GET_MASK) {
case VIDEO_CID_CAMERA_EXPOSURE:
return sensor_get_ctrl_exposure(dev, cid, value);
default:
return -ENOTSUP;
}
} Everything inline. static int sensor_get_ctrl(const struct device *dev, unsigned int cid, void *value)
{
const struct sensor_config *cfg = dev->config;
uint16_t u16;
int ret;
switch (cid) {
case VIDEO_CTRL_GET_CUR | VIDEO_CID_CAMERA_EXPOSURE:
ret = sensor_read_reg(&cfg->i2c, SENSOR_EXPOSURE_REG, &u16, 2);
*(uint32_t)value = u16;
return ret;
case VIDEO_CTRL_GET_MIN | VIDEO_CID_CAMERA_EXPOSURE:
*(uint32_t *)value = 0;
return 0;
case VIDEO_CTRL_GET_MAX | VIDEO_CID_CAMERA_EXPOSURE:
*(uint32_t *)value = 320;
return 0;
case VIDEO_CTRL_GET_DEF | VIDEO_CID_CAMERA_EXPOSURE:
*(uint32_t *)value = 9;
return 0;
default:
return -ENOTSUP;
}
} Helper function specifying the min/max/def. static int video_ctrl_range_u32(unsigned int cid, void *value, uint32_t min, uint32_t max, uint32_t def)
{
switch (cid & VIDEO_CTRL_GET_MASK) {
case VIDEO_CTRL_GET_MIN:
*(uint32_t *)value = min;
return 0;
case VIDEO_CTRL_GET_MAX:
*(uint32_t *)value = max;
return 0;
case VIDEO_CTRL_GET_DEF:
*(uint32_t *)value = def;
return 0;
case VIDEO_CTRL_GET_CUR:
return 1;
default:
return -ENOTSUP;
}
}
static int sensor_get_ctrl(const struct device *dev, unsigned int cid, void *value)
{
uint16_t u16;
int ret;
switch (cid & ~VIDEO_CTRL_GET_MASK) {
case VIDEO_CID_CAMERA_EXPOSURE:
ret = video_get_ctrl_range(cid, value, 0, 320, 9);
if (ret == 1) {
ret = sensor_read_reg(&cfg->i2c, SENSOR_EXPOSURE_REG, &u16, 2);
*(uint32_t)value = u16;
}
return ret;
default:
return -ENOTSUP;
}
} |
26f0023
to
fd5e76b
Compare
It is possible to know the default value by probing the systems at startup and storing the default value. |
The way Linux does this is to initialize new controls, which likely means memory allocation: The last code block above with Maybe adding a set of "CID wrappers" like this could help simplifying the min/max handling while not introducing any memory allocation or involve a complete subsystem? /* Standard helpers for simple situations fitting most controls */
int video_ctrl_range_u32(const struct device *dev, unsigned int cid, void *value,
uint32_t min, uint32_t max, uint32_t def);
int video_ctrl_range_i32(const struct device *dev, unsigned int cid, void *value,
int32_t min, int32_t max, int32_t def);
int video_ctrl_range_enum(const struct device *dev, unsigned int cid, void *value,
int min, int max, int def);
/* Customized helpers for customized structures */
int video_ctrl_range_roi(const struct device *dev, unsigned int cid, void *value,
struct video_ctrl_roi *min, video_ctrl_roi *max, video_ctrl_roi *def); |
fd5e76b
to
84b2763
Compare
84b2763
to
96a0c3e
Compare
Last block seems fair. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this approach is reasonable
96a0c3e
to
b6a2b3c
Compare
b6a2b3c
to
71de5b5
Compare
Force-push:
|
I think @ngphibang might have been planning something around the video API CID types, which might obsolete this PR. |
71de5b5
to
ec5aeb9
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@josuah Thanks for mentioning this ! Yes, I am working on improving the control aspect as well. This PR seems targeting part of this scope. But just go ahead merging this if we see that it's ok.
drivers/video/video_common.c
Outdated
return ret; | ||
} | ||
|
||
if (value < min || value > max) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you should also check min > max
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I first added it as an assert()
, but the MAX is not always fixed at build-time, i.e. could come from I2C read.
This can happen even when the software is correct => cannot be assert()
!
drivers/video/video_common.c
Outdated
@@ -83,3 +84,45 @@ void video_buffer_release(struct video_buffer *vbuf) | |||
VIDEO_COMMON_FREE(block->data); | |||
} | |||
} | |||
|
|||
int video_get_range_u32(unsigned int cid, uint32_t *value, uint32_t min, uint32_t max, uint32_t def) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we only check uint32_t ? What if the control values are of other types, e,g, int (control values can be negative), uint64_t (pixel_rate, link frequencies), menu (test pattern), etc. ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are right, this should be void *
like video_get_ctrl()
.
If some concept of type is to be introduced, this should be a separate PR.
Thank you for the help.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it cannot be uint64_t
(as it does not fit in a pointer in some architectures supported by Zephyr) but it can be uint64_t *
.
Maybe a first compromise is to support int32_t
and uint32_t
(or to be more correct, uintptr_t
and intptr_t
) is good enough for a first cycle? I am updating this branch to illustrate this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am waiting before introducing bool
, menu
, bitmap
and extra types as I tried to focus on the mechanism that enable fetching a range that would fit the current API.
Let me know if I should include them on this PR, I'd be glad to give it a try!
What if user specifies : |
This becomes #define VIDEO_CTRL_GET_CUR 0x00000000 /**< Get the current value */
#define VIDEO_CTRL_GET_MIN 0x00001000 /**< Get the minimum value */
#define VIDEO_CTRL_GET_MAX 0x00002000 /**< Get the maximum value */
#define VIDEO_CTRL_GET_DEF 0x00003000 /**< Get the default value */
#define VIDEO_CTRL_GET_MASK 0x0000f000 /**< Mask for get operations */ In the current implementation, this is not a bitmask of flags, but a field added inside of the CID that contains an integer value from 0 to 3. Is this too confusing? |
Maybe a better solution can be visible after starting to use it a bit. |
ec5aeb9
to
af11160
Compare
e40296b
to
2665a9a
Compare
Force-push: CI fixes. Developing:
Before submitting:
|
2665a9a
to
1a4dcb1
Compare
force-push: switch from |
The video_get_ctrl() API permits to retrieve a value from a device using standard CIDs from <zephyr/drivers/video-controls.h>. The CIDs do not come with a range information, and to know which value to apply to a video driver, knowing the minimum and maximum value before-hand is required. This prevents building generic layers that handle any video devices, such as protocols such as USB UVC, GigE Vision, or anything making use of "zephyr,camera" chosen node. This commit introduces extra flags added to the CIDs that indicates to the target device that instead of returning the current value, they should return the minimum, maximum, or default value instead, with the same type as the current value. The GET_CUR operation having a flag of 0x00, this makes all drivers implicitly support this new API, with an opt-in migration to also support the extra controls, correctly rejecting the unsupported extra operations by default. Signed-off-by: Josuah Demangeon <[email protected]>
This allows implementing the VIDEO_CTRL_GET_MIN/MAX/DEF controls on drivers with minimum modification of the drivers. A typical usage in a video_get_ctrl() implementation would be: ret = video_get_ctrl_range(cid, value, 0, 320, 9); if (ret <= 0) { return ret; } return sensor_read_u32(&cfg->i2c, EXPOSURE_REG, value, 2); A typical usage in a video_set_ctrl() implementation would be: ret = video_check_range_u32(dev, cid, (uint32_t)value); if (ret < 0) { return ret; } return sensor_read_reg(&cfg->i2c, EXPOSURE_REG, value, 2); Signed-off-by: Josuah Demangeon <[email protected]>
1a4dcb1
to
3bd46c6
Compare
force-push:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry to come back to this a bit late, but I think there is a couple of issues within this PR.
-
The 1st commit is to add some "helper" functions to do sanity check (range check) on the control values and these are intended to be used by drivers. But would it be really helpful to simplify the code ? Looking at the ov5640 driver, for example, instead of calling one line of code for
IN_RANGE
macro to check the range in everyset_ctrl()
, if I use these "helper", it's still the same amount of code :-). And every drivers still have to repeatedly do the same sanity check. For me, it's only helpful if we can delegate all the sanity check to the video framework, drivers have to do nothing. -
The 2nd commit is to add 4 macros which are intended to be combined with the CID parameters in
get_ctrl()
to query additional information for the min/max/default values beside the current control value. I think these information should be queried through another API, thequery_ctrl()
API rather than included in theget_ctrl()
. Thequery_ctrl()
API could query all things at once (min/max/default). This is not only for efficiency but also to keep things cleaner. I know that the UVC driver does it separately with GET_CUR, GET_MIN, GET_MAX, etc. but it does not mean we have to impose this constraint to everyone. You can always query all at the same time at the application level and call UVC separately for each value. And for a larger scope, theget_ctrl()
andquery_ctrl()
are something that can be also handled by the video framework, drivers don't need to implement these API.
These issues do not only come from the PR but also from the current video framework limitation which is not implemented to support this.
Thank you for taking the time to review this again.
Yes, no real advantage.
This would be nice.
Like this? struct video_ctrl_range {
void *min;
void *max;
void *cur; /* Did you mean to include this field in `query_ctrl()` or leave it to `get_ctrl()`? */
void *def; /* This one too? */
int type; /* Other fields for allowing batch handling? */
};
struct video_ctrl_range result;
video_ctrl_query(dev, VIDEO_CID_..., &result);
The UVC driver will be refactored as many times as needed. :)
I think on Linux, the controls are registered at init, and then this cached information is used at runtime. On Zephyr this could need some An alternative is to ask the information to the drivers every time An alternative is to pass the whole table, like it is done for struct video_ctrl_range {
unsigned int cid;
void *min, *max, *def;
};
const struct video_ctrl_range ctrls[] = {
{ .cid = VIDEO_CID_EXPOSURE, .min = (void *)0, .max = (void *)100, .def = (void *)50 },
{ .cid = VIDEO_CID_GAIN, .min = (void *)0, .max = (void *)100, .def = (void *)0 },
{0},
}; |
What is the way forward to propose an alternative? Did you plan to make a new PR? I'm glad to edit this one too. |
In general, controls need to be cached in memory because reading control value directly from registers each time
There are a few control types, in some contexts, need to be read at runtime directly from registers, such as gain, exposure, which we call volatile controls. They will need a dedicated API like
Yes, as being said, I am working on this. The main part is not about the API or structure defintions, etc but something like how to make the controls visible from a device to the framework and vice versa, how to merge controls from the each device throughout the pipeline, etc. But it's good because through this, we can make the framework do "something" on behalf of the drivers for the 1st time. There are some technical challenges and time constraint but I am finalizing the last parts, hope to push the PR soon. |
The Arducam Mega allows a selection of image sensors to be connected, so in this kind of scenario, which controls are supported could depend on which sensor is connected. NXP's
My bad, yes you did. I will wait the new PR and will close this one. |
I am not sure to understand this. Does it mean which sensor is selected is done at compile time ? or runtime ? and can be changed at any moment ? If it's done at compile time, it is as other camera sensors. |
Here is the pull request for it: The same single driver is expected to be used for all cameras, and the variability is at construction time: there are cameras variants: Arducam Mega SPI protocol has a "Camera model" field: In their PR, Ardcuam reads this "SENSOR_ID" to tell which controls and resolutions are available: switch (cam_id) {
case ARDUCAM_SENSOR_5MP_1: /* 5MP-1 */
fmts[8] = (struct video_format_cap)ARDUCAM_MEGA_VIDEO_FORMAT_CAP(
2592, 1944, VIDEO_PIX_FMT_RGB565);
fmts[17] = (struct video_format_cap)ARDUCAM_MEGA_VIDEO_FORMAT_CAP(
2592, 1944, VIDEO_PIX_FMT_JPEG);
fmts[26] = (struct video_format_cap)ARDUCAM_MEGA_VIDEO_FORMAT_CAP(
2592, 1944, VIDEO_PIX_FMT_YUYV);
support_resolution[8] = MEGA_RESOLUTION_WQXGA2;
drv_data->info = &mega_infos[0];
break;
case ARDUCAM_SENSOR_3MP_1: /* 3MP-1 */
...
} Then have defined for themselves several "camera profiles": static struct arducam_mega_info mega_infos[] = {{
.support_resolution = 7894,
.support_special_effects = 63,
.exposure_value_max = 30000,
.exposure_value_min = 1,
.gain_value_max = 1023,
.gain_value_min = 1,
.enable_focus = 1,
.enable_sharpness = 0,
.device_address = 0x78,
}, ...}; So it is all done at startup time and relies of a database like storage of sensor capabilities. They would update the driver when more sensors are supported. For now,
[EDIT: my bad, it looks like they wish it to be possible upon request of the application]
Where |
Thank you for these infos ! I was not involved in the reviewing of the Arducam Mega driver but I think there are several aspects need to be reconsidered. Just for the control part, we see:
The arducam mega info table:
They declared a new vendor CID (VIDEO_CID_ARDUCAM_INFO) just for showing a group of control default values and resolutions. Even the Arducam driver is common for a set of similar sensors, the code can be refactored to a common and particular parts. At hardware abstraction level, they can be considered as separate sensors. When sensor is changed (not sure at compile time or runtime), it is always possible to follow the normal process : stop stream / restart stream, control is re-initialized for the new sensor, etc. |
Yes good point, a very similar API but still different devices. |
An issue faced while working on UVC: how to know the min/max value of a video control ID (CID)?
The UVC protocol supports
GET_CUR
,GET_DEF
,GET_MIN
,GET_MAX
,GET_RES
operation to requst a value whose storage length is given byGET_LEN
.The
video_get_ctrl()
API permits to retrieve a value from a device using standard CIDs from<zephyr/drivers/video-controls.h>
. The CIDs do not define standard min/max values. This means knowing what value to apply needs knowing the particular sensor before-hand.This commit introduces extra flags added to the CIDs asking for the current, minimum, maximum, or default value, with the same type.
The
GET_CUR
operation having a flag of0x00
, this makes all drivers implicitly support this new API, with an opt-in migration to also support the extra controls, correctly rejecting the unsupported extra operations by default.Is this a reasonable way to solve this?