feature: New Pipeline Stages #2236
Conversation
product-auto-label
bot
added
size: l
| * @param fieldName The name of the field containing the array. | ||
| * @param options The {@code UnnestOptions} options. | ||
| * @return A new {@code Pipeline} object with this stage appended to the stage list. | ||
| */ |
There was a problem hiding this comment.
Can we document the use case for indexField? Or otherwise why someone would use this overload?
There was a problem hiding this comment.
I think we should focus on describing the mechanics of how the stage works, as opposed to why they may want to use this stage. The why belongs more in tutorials, guides or blogs, thereby keeping Stage descriptions more concise.
Here is the Stage Schemas as described by Query team: https://docs.google.com/document/d/1pvF6JH8Ulc2GSmFwC0XOjH1KdTZ_sg779XAN6NoRR7s/edit?resourcekey=0-gmt2NHpJrHPS-z4InY4ufw&tab=t.0#heading=h.7b8hsu3i0pnv
If we don't think there is value in an option/stage, we should raise it with Query team.
There was a problem hiding this comment.
I agree on documenting the mechanics. My comment was poorly worded, but I think my concern still stands. IMO, we don't clearly document what happens when indexField is set. At least it wasn't clear to me until I read the linked doc.
We could add something like this to the docs for this method or for UnnestOptions:
When indexField is provided, a new field is added to the emitted documents, storing the offset (starting at zero) into the array the expanded element is from.
There was a problem hiding this comment.
I have updated PR. @MarkDuckworth, please take another look.
| * @param fieldName The name of the field containing the array. | ||
| * @param options The {@code UnnestOptions} options. | ||
| * @return A new {@code Pipeline} object with this stage appended to the stage list. | ||
| */ |
There was a problem hiding this comment.
I think we should focus on describing the mechanics of how the stage works, as opposed to why they may want to use this stage. The why belongs more in tutorials, guides or blogs, thereby keeping Stage descriptions more concise.
Here is the Stage Schemas as described by Query team: https://docs.google.com/document/d/1pvF6JH8Ulc2GSmFwC0XOjH1KdTZ_sg779XAN6NoRR7s/edit?resourcekey=0-gmt2NHpJrHPS-z4InY4ufw&tab=t.0#heading=h.7b8hsu3i0pnv
If we don't think there is value in an option/stage, we should raise it with Query team.
| * @param fieldName The name of the field containing the array. | ||
| * @param options The {@code UnnestOptions} options. | ||
| * @return A new {@code Pipeline} object with this stage appended to the stage list. | ||
| */ |
There was a problem hiding this comment.
I agree on documenting the mechanics. My comment was poorly worded, but I think my concern still stands. IMO, we don't clearly document what happens when indexField is set. At least it wasn't clear to me until I read the linked doc.
We could add something like this to the docs for this method or for UnnestOptions:
When indexField is provided, a new field is added to the emitted documents, storing the offset (starting at zero) into the array the expanded element is from.
No description provided.