Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow non-nullable schemas to be created #604

Open
eddelbuettel opened this issue Sep 6, 2024 · 2 comments
Open

Allow non-nullable schemas to be created #604

eddelbuettel opened this issue Sep 6, 2024 · 2 comments

Comments

@eddelbuettel
Copy link
Contributor

We commonly use a combination of columns that are non-nullable and nullable. The current default of 'always nullable' from e.g. line 179 here is a little strict.

void ArrowSchemaInit(struct ArrowSchema* schema) {
schema->format = NULL;
schema->name = NULL;
schema->metadata = NULL;
schema->flags = ARROW_FLAG_NULLABLE;
schema->n_children = 0;
schema->children = NULL;
schema->dictionary = NULL;
schema->private_data = NULL;
schema->release = &ArrowSchemaReleaseInternal;
}

Obviously, we can add a local variant with an added bool which calls this and then, as needed, follows up with

 schema->flags &= ~ARROW_FLAG_NULLABLE;

But can you see a way to more cleanly allow this? There are a few spots that condition / undo but they seem like lower-level entry points. If I am missing something really obvious don't be shy and call me out 😀

@paleolimbot
Copy link
Member

That is a great point...it was this way initially due to the ArrowSchema's role as both "data type" and "field" (which is data type + name + nullability in most Arrow implementations). Having this be nullable by default is a bit safer since some implementations use the nulability flag to skip checking for nulls and it would result in incorrect behaviour to set that by default.

Obviously, we can add a local variant with an added bool which calls this and then, as needed, follows up with

This is probably your best bet for now, but it would be cheap to add ArrowSchemaSetNullability() (or maybe more generically, ArrowSchemaSetFlag(schema, flag, state)) for better readability when toggling the flag.

@eddelbuettel
Copy link
Contributor Author

Yep -- I actually like a simple pair of ArrowSchemaSetNullability() and ArrowSchemaIUnsetNullability() (or maybe even just the latter if the default is to set).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants