Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Could bigflow support either SQL or a familiar DataFrame API to query structured data. #48

Open
kelvict opened this issue Dec 21, 2017 · 3 comments
Open

Could bigflow support either SQL or a familiar DataFrame API to query structured data. #48

kelvict opened this issue Dec 21, 2017 · 3 comments
Assignees
@yshysh

Comments

@kelvict
Copy link

kelvict commented Dec 21, 2017

Could bigflow support either SQL or a familiar DataFrame API to query structured data.
@yshysh yshysh self-assigned this Jan 3, 2018
@yshysh
Copy link
Collaborator

yshysh commented Jan 3, 2018

Bigflow will support this follow-up.

@yshysh
Copy link
Collaborator

yshysh commented Jan 3, 2018

We have a version of Bigflow on schema, but this version needs to improve, for example, only supportting input, process sql is on the way.

@acmol
Copy link
Collaborator

acmol commented Jan 3, 2018 •
edited
Loading

We'd like to implement a Spark SQL familiar framework, but it's not a easy task.

We've implement a prototype, named Bigflow Schema, it's available inside Baidu.
But it has a lot of problems if there are too many fields. So it's not recommended to use.

More time is needed to consider, design, and implement this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yshysh yshysh

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@acmol @kelvict @yshysh