You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The existing FlightSQLExecutor executes a sql string. We should add support for executing a serialized Substrait plan. In the longer term , it should be easier to support more advanced pushdown for engines which support consuming Substrait.
The text was updated successfully, but these errors were encountered:
I plan to take a look at apache/datafusion#9299 upstream to get some more familiarity with how Datafusion serializes plans to substrait and what limitations it may have currently.
In the current approach we're federating the logical plan. So, this would likely be done earlier then the error mentioned in that issue.
It should be fairly simple to add Substrait-based federation to the FlightSQL example. There is fairly fleshed out import and export code for logical plans. My idea was to, in the short term, just have a boolean to determine if SQL or Substrait should be sent. The alternative/eventual goal could be to build out the Flight SQL introspection commands to automate that further.
Adding support for federating (part of) physical plans is another avenue that could be explored in the future.
edmondop
pushed a commit
to edmondop/datafusion-federation
that referenced
this issue
Jul 6, 2024
…uery when executing (datafusion-contrib#10)
* keep original qualified_field when wrap_projections
* only mutate the table reference when sending plan to execute
* formatting
* rename method
The existing FlightSQLExecutor executes a sql string. We should add support for executing a serialized Substrait plan. In the longer term , it should be easier to support more advanced pushdown for engines which support consuming Substrait.
The text was updated successfully, but these errors were encountered: