feat: add extension for logical_plan_builder

[zhuliquan]

Which issue does this PR close?

Closes #.

Rationale for this change

Support build extension for builder

What changes are included in this PR?

add function extension for LogicalPlanBuilder, which is aimed to build extension logical plan.

Are these changes tested?

yes, add test case plan_builder_from_extension

Are there any user-facing changes?

yes, user can build logical plan with extension node.

[jayzhan211]

Why do we need extension? Is there any example that shows we need this? I would be nice if there is end-to-end query in slt file

[zhuliquan]

Why do we need extension? Is there any example that shows we need this? I would be nice if there is end-to-end query in slt file

I think building a logical plan from SQL is a high-level API behavior exposed to user, and we also need to build a logical plan from a low-level API like building blocks, just like Flink I can build a stream through Flink-SQL, and we can also build a flow through basic operators. Because SQL is completely limiting to how LogicalPlan is built. This allows us to build logicalPlans not only from SQL, but also from user-defined DSL. If there are features that are not sufficient with the existing LogicalPlan, we need to extend it. Meanwhile we need a extension func to build it

[edmondop]

Why do we need extension? Is there any example that shows we need this? I would be nice if there is end-to-end query in slt file

I think building a logical plan from SQL is a high-level API behavior exposed to user, and we also need to build a logical plan from a low-level API like building blocks, just like Flink I can build a stream through Flink-SQL, and we can also build a flow through basic operators. Because SQL is completely limiting to how LogicalPlan is built. This allows us to build logicalPlans not only from SQL, but also from user-defined DSL. If there are features that are not sufficient with the existing LogicalPlan, we need to extend it. Meanwhile we need a extension func to build it

Following up on this, can we file an issue with this new feature @zhuliquan ? I think it will make easier to understand the context/scenario you have in mind

[alamb]

In general I think this PR is trying to make it easier to create LogicalPlan::Extension (aka user defined logical nodes)

I agree in general about the benefit of a ticket

What I would personally suggest is starting with a new example in datafusion-examples with a LogicalPlan::Extension node

perhaps we could show how to use that to implement some sort of non-relational operation. Maybe based on the following thing from influxdata:

https://github.com/influxdata/influxdb3_core/blob/1eaa4ed5ea147bc24db98d9686e457c124dfd5b7/iox_query/src/exec/schema_pivot.rs#L1-L20

@zhuliquan if you agree this is what you are trying to do, I would be happy to file the ticket to try and help this project along

[alamb]

Thanks @zhuliquan -- I know adding/improving examples is a bigger as than just an API