Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: query compiler proof of concept #5024

Open
wants to merge 67 commits into
base: main
Choose a base branch
from
Open

feat: query compiler proof of concept #5024

wants to merge 67 commits into from

Conversation

aqrln
Copy link
Member

@aqrln aqrln commented Oct 25, 2024

Separate query planning and execution and lower the query graph into a format executable by the client.

Translates, for example,

prisma.user.findUnique({
    where: {
        email: Prisma.Param("userEmail")
    },
    select: {
        val: true,
        posts: true,
        profile: true,
    }
})

to

let @parent = query «SELECT "public"."User"."id", "public"."User"."val" FROM
                     "public"."User" WHERE "public"."User"."email" = $1 OFFSET
                     $2»
              params [var(userEmail as String), const(BigInt(0))]
in let @parent$id = mapField id (get @parent)
   in join (get @parent)
      with (query «SELECT "public"."Post"."id", "public"."Post"."createdAt",
                   "public"."Post"."updatedAt", "public"."Post"."published",
                   "public"."Post"."title", "public"."Post"."content",
                   "public"."Post"."authorId" FROM "public"."Post" WHERE
                   "public"."Post"."authorId" IN /*
                   prisma-comma-repeatable-start */$1/*
                   prisma-comma-repeatable-end */ OFFSET $2»
            params [var(@parent$id as String),
                    const(BigInt(0))]) on left.(id) = right.(authorId) as posts,
           (unique (query «SELECT "public"."Profile"."userId" FROM
                           "public"."Profile" WHERE "public"."Profile"."userId"
                           IN /* prisma-comma-repeatable-start */$1/*
                           prisma-comma-repeatable-end */ LIMIT $2 OFFSET $3»
                    params [var(@parent$id as String), const(BigInt(1)),
                            const(BigInt(0))])) on left.(id) = right.(userId) as profile

which is then executed by the QueryInterpeter on the client.

Processing query graphs with more than one node is work in progress. Functions in the edges (data dependencies) can be executed at compile time thanks to the new Placeholder abstraction (we can algebraically substitute future values not yet fetched from the database into the inputs of downstream graph nodes and continue building the query with them before those values actually exist). Flow and computation nodes need to be redesigned though.

Client: prisma/prisma#25511

Copy link
Contributor

github-actions bot commented Oct 25, 2024

WASM Query Engine file Size

Engine This PR Base branch Diff
Postgres 2.117MiB 2.106MiB 11.146KiB
Postgres (gzip) 848.034KiB 844.559KiB 3.475KiB
Mysql 2.079MiB 2.068MiB 11.919KiB
Mysql (gzip) 834.060KiB 830.604KiB 3.457KiB
Sqlite 1.987MiB 1.976MiB 11.335KiB
Sqlite (gzip) 797.124KiB 793.008KiB 4.116KiB

Copy link

codspeed-hq bot commented Oct 26, 2024

CodSpeed Performance Report

Merging #5024 will not alter performance

Comparing compiler (7d1763d) with main (bc5b021)

Summary

✅ 11 untouched benchmarks

@aqrln aqrln added this to the 6.2.0 milestone Dec 19, 2024
@aqrln aqrln force-pushed the compiler branch 2 times, most recently from 2d4977c to 3668ee0 Compare December 20, 2024 11:51
@jkomyno jkomyno removed this from the 6.2.0 milestone Jan 7, 2025
@aqrln aqrln force-pushed the compiler branch 16 times, most recently from 9f44246 to b1c7caf Compare January 11, 2025 20:13
FGoessler and others added 11 commits January 15, 2025 20:55
* feat: generic query building

* fix: compiler example

* fix: comment out non pg variants for now

* fix: unused imports

* chore: remove currently unnecessary impl

* fix: enable postgres feature for now
* feat: generic query building

* fix: compiler example

* fix: comment out non pg variants for now

* fix: unused imports

* chore: remove currently unnecessary impl

* chore: extract query compiler crate

* feat: add query-compiler-wasm

* chore: cleanup

* chore: get rid of adapter object, pass connection info and add CI

* ci: update formatting job and pedantic check

* build: remove driver-adapters dependency from compiler-wasm for now and remove cfg target wasm

* chore: address comments

* ci: remove unneeded RUSTFLAGS

* chore: remove accidentally added file
@aqrln aqrln added this to the 6.3.0 milestone Jan 20, 2025
@aqrln aqrln marked this pull request as ready for review January 20, 2025 10:05
@aqrln aqrln requested a review from a team as a code owner January 20, 2025 10:05
@aqrln aqrln requested review from wmadden, FGoessler, jkomyno and jacek-prisma and removed request for a team and wmadden January 20, 2025 10:05
Comment on lines +56 to +63

if [[ "$OSTYPE" == "linux-gnu"* ]] && command -v lld &> /dev/null && [ ! -f .cargo/config.toml ]; then
mkdir -p .cargo
cat << EOF > .cargo/config.toml
[target.$(uname -m)-unknown-linux-gnu]
rustflags = ["-C", "link-arg=-fuse-ld=lld"]
EOF
fi
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated changes, can be dropped or extracted to a different PR

@@ -25,6 +28,12 @@ impl<'a> IntoSql<'a> for &'a Value<'a> {
ValueType::DateTime(val) => val.into_sql(),
ValueType::Date(val) => val.into_sql(),
ValueType::Time(val) => val.into_sql(),
ValueType::Var(name, _) => {
Copy link
Contributor

@jacek-prisma jacek-prisma Jan 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it might be worth introducing a separate type at this point, to guarantee we don't try to put one of these in an actual database query,
We could maybe have ValueType with a type parameter that turns Var into Never when we output a database query, or just a separate enum

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants