diff --git a/LICENSE b/LICENSE.txt similarity index 99% rename from LICENSE rename to LICENSE.txt index f49a4e1..05842e4 100644 --- a/LICENSE +++ b/LICENSE.txt @@ -186,7 +186,7 @@ same "printed page" as the copyright notice for easier identification within third-party archives. - Copyright [yyyy] [name of copyright owner] + Copyright 2023 Hasura Inc. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. diff --git a/README.md b/README.md index 356a6c1..e64efb3 100644 --- a/README.md +++ b/README.md @@ -1,155 +1,172 @@ -## Qdrant Connector +# Hasura Qdrant Connector + -The Qdrant Data Connector allows for connecting to a Qdrant database. This connector uses the [Typescript Data Connector SDK](https://github.com/hasura/ndc-sdk-typescript) and implements the [Data Connector Spec](https://github.com/hasura/ndc-spec). +[![Docs](https://img.shields.io/badge/docs-v3.x-brightgreen.svg?style=flat)](https://hasura.io/connectors/qdrant) +[![ndc-hub](https://img.shields.io/badge/ndc--hub-qdrant-blue.svg?style=flat)](https://hasura.io/connectors/qdrant) +[![License](https://img.shields.io/badge/license-Apache--2.0-purple.svg?style=flat)](./LICENSE.txt) +[![Status](https://img.shields.io/badge/status-alpha-yellow.svg?style=flat)](./README.md) -### Setting up the Qdrant connector using Hasura Cloud & a Qdrant database +The Hasura Qdrant Connector allows for connecting to a Qdrant database to give you an instant GraphQL API on top of your Qdrant data. -#### Step 1: Prerequisites +This connector is built using the [Typescript Data Connector SDK](https://github.com/hasura/ndc-sdk-typescript) and implements the [Data Connector Spec](https://github.com/hasura/ndc-spec). -1. Install the [new Hasura CLI](https://hasura.io/docs/3.0/cli/installation/) — to quickly and easily create and manage your Hasura projects and builds. -2. Install the [Hasura VS Code extension](https://marketplace.visualstudio.com/items?itemName=HasuraHQ.hasura) — with support for other editors coming soon! -3. Have a [Qdrant](https://qdrant.tech/) database — for supplying data to your API. +* [Connector information in the Hasura Hub](https://hasura.io/connectors/qdrant) +* [Hasura V3 Documentation](https://hasura.io/docs/3.0/index/) -#### Step 2: Login to Hasura +## Features -After our prerequisites are taken care of, login to Hasura Cloud with the CLI: +Below, you'll find a matrix of all supported features for the Qdrant connector: -`ddn login` +| Feature | Supported | Notes | +| ------------------------------- | --------- | ----- | +| Native Queries + Logical Models | ❌ | | +| Simple Object Query | ✅ | | +| Filter / Search | ✅ | | +| Simple Aggregation | ❌ | | +| Sort | ❌ | | +| Paginate | ✅ | Pagination offset field only works for documents with Integer ID's | +| Nested Objects | ✅ | | +| Nested Arrays | ✅ | | +| Nested Filtering | ❌ | | +| Nested Sorting | ❌ | | +| Nested Relationships | ❌ | | +| Vector Search | ✅ | | -This will open up a browser window and initiate an OAuth2 login flow. If the browser window doesn't open automatically, use the link shown in the terminal output to launch the flow. +## Before you get Started -#### Step 3: Create a new project +[Prerequisites or recommended steps before using the connector.] -We'll use the `create project` command to create a new project: +1. The [DDN CLI](https://hasura.io/docs/3.0/cli/installation) and [Docker](https://docs.docker.com/engine/install/) installed +2. A [supergraph](https://hasura.io/docs/3.0/getting-started/init-supergraph) +3. A [subgraph](https://hasura.io/docs/3.0/getting-started/init-subgraph) +4. Have a [Qdrant](https://qdrant.tech/) hosted database, or a locally running Qdrant database — for supplying data to your API. -`ddn create project --dir ./ddn` +The steps below explain how to Initialize and configure a connector for local development. You can learn how to deploy a +connector — after it's been configured — [here](https://hasura.io/docs/3.0/getting-started/deployment/deploy-a-connector). -#### Step 4: Add a connector manifest +## Using the Qdrant connector -Let's move into the project directory: +### Step 1: Authenticate your CLI session -`cd ddn` +```bash +ddn auth login +``` + +### Step 2: Initialize the connector -Create a subgraph: +```bash +ddn connector init qdrant --subgraph my_subgraph --hub-connector hasura/qdrant +``` -`ddn create subgraph qdrant` +In the snippet above, we've used the subgraph `my_subgraph` as an example; however, you should change this +value to match any subgraph which you've created in your project. -Then, create a connector manifest: -`ddn add connector-manifest qdrant_connector --subgraph qdrant --hub-connector hasura/qdrant --type cloud` +### Step 3: Modify the connector's port -#### Step 5: Edit the connector manifest +When you initialized your connector, the CLI generated a set of configuration files, including a Docker Compose file for +the connector. Typically, connectors default to port `8080`. Each time you add a connector, we recommend incrementing the published port by one to avoid port collisions. -You should have a connector manifest created at `ddn/qdrant/qdrant_connector/connector/qdrant_connector.build.hml` +As an example, if your connector's configuration is in `my_subgraph/connector/qdrant/docker-compose.qdrant.yaml`, you can modify the published port to reflect a value that isn't currently being used by any other connectors: ```yaml -kind: ConnectorManifest -version: v1 -spec: - supergraphManifests: - - base -definition: - name: qdrant_connector - type: cloud - connector: - type: hub - name: hasura/qdrant:v0.1.7 - deployments: - - context: . - env: - QDRANT_API_KEY: - value: "" - QDRANT_URL: - value: "" +ports: + - mode: ingress + target: 8080 + published: "8082" + protocol: tcp ``` -Fill in the value for the QDRANT_API_KEY and QDRANT_URL environment variables with your Qdrant credentials. +### Step 4: Add environment variables + +Now that our connector has been scaffolded out for us, we need to provide a connection string so that the data source can be introspected and the boilerplate configuration can be taken care of by the CLI. -(Make sure to save your changes to the file!) +The CLI has provided an `.env.local` file for our connector in the `my_subgraph/connector/qdrant` directory. We can add a key-value pair of `QDRANT_URL` along with the connection string itself to this file, and our connector will use this to connect to our database. If the Qdrant database has an API key you can also provide the environment variable for the `QDRANT_API_KEY` which allows the connector to authenticate. -#### Step 6: Start a development session -Start a Hasura dev session using the following command: +The file, after adding the `QDRANT_URL`, should look like this example if connecting to a Qdrant hosted database instance: -`ddn dev` +```env +OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://local.hasura.dev:4317 +OTEL_SERVICE_NAME=my_subgraph_qdrant +QDRANT_URL=https://7312d6c4-3f6c-432c-987c-34d7d96428ef.us-east4-0.gcp.cloud.qdrant.io +QDRANT_API_KEY=Ch8I... +``` -You should see something like this if the connector has been deployed successfully: +To connect to a locally running Qdrant instance you can then point the `QDRANT_URL` to the local database. Assuming the Qdrant database is running on port 6333 without any API key, you should be able to use this example: +```env +OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://local.hasura.dev:4317 +OTEL_SERVICE_NAME=my_subgraph_qdrant +QDRANT_URL=http://local.hasura.dev:6333 ``` -3:29PM INF Building SupergraphManifest "base"... -+---------------+-------------------------------------------------------------------------------------------------------+ -| Build Version | 39e8b49ed5 | -+---------------+-------------------------------------------------------------------------------------------------------+ -| API URL | https://allowing-sturgeon-9867-39e8b49ed5.ddn.hasura.app/graphql | -+---------------+-------------------------------------------------------------------------------------------------------+ -| Console URL | https://console.hasura.io/project/allowing-sturgeon-9867/environment/default/build/39e8b49ed5/graphql | -+---------------+-------------------------------------------------------------------------------------------------------+ -| Project Name | allowing-sturgeon-9867 | -+---------------+-------------------------------------------------------------------------------------------------------+ -| Description | Dev build - Mon, 15 Apr 2024 15:29:56 CDT | -+---------------+-------------------------------------------------------------------------------------------------------+ + +### Step 5: Introspect your data source + +With the connector configured, we can now use the CLI to introspect our database and create a source-specific configuration file for our connector. + +```bash +ddn connector introspect --connector my_subgraph/connector/qdrant/connector.yaml ``` -Navigate to your Console URL and you can issue a query or mutation. +### Step 6. Create the Hasura metadata -### Setting up the Qdrant connector locally (Coming Soon) +Hasura DDN uses a concept called "connector linking" to take [NDC-compliant](https://github.com/hasura/ndc-spec) +configuration JSON files for a data connector and transform them into an `hml` (Hasura Metadata Language) file as a +[`DataConnectorLink` metadata object](https://hasura.io/docs/3.0/supergraph-modeling/data-connectors#dataconnectorlink-dataconnectorlink). -Please keep an eye out for instructions on running things locally which will be coming soon. +Basically, metadata objects in `hml` files define our API. -### Qdrant Introspection Details: +First we need to create this `hml` file with the `connector-link add` command and then convert our configuration files +into `hml` syntax and add it to this file with the `connector-link update` command. -The current Qdrant introspection performed is naive, as it will simply poll the points for the first point, and assume the schema matches that point. In order to expose the schema via GraphQL all points must conform to the introspected schema. +Let's name the `hml` file the same as our connector, `qdrant`: -### Performing Joins +```bash +ddn connector-link add qdrant --subgraph my_subgraph +``` + +The new file is scaffolded out at `my_subgraph/metadata/qdrant/qdrant.hml`. + +### Step 7. Update the environment variables + +The generated file has two environment variables — one for reads and one for writes — that you'll need to add to your subgraph's `.env.my_subgraph` file. Each key is prefixed by the subgraph name, an underscore, and the name of the connector. Ensure the port value matches what is published in your connector's docker compose file. -Joins cannot be performed to the Qdrant connector, as joins with collection arguments are not supported. Joins CAN be performed from the Qdrant connector to another connector such as Postgres for example. +As an example: -### Qdrant Connector Usage Details +```env +MY_SUBGRAPH_QDRANT_READ_URL=http://local.hasura.dev: +MY_SUBGRAPH_QDRANT_WRITE_URL=http://local.hasura.dev: +``` + +These values are for the connector itself and utilize `local.hasura.dev` to ensure proper resolution within the docker container. -The Qdrant connector makes use of parameterized collections to allow for performing a vector search. +### Step 8. Start the connector's Docker Compose -Here is an example of a query that passes an array of positive and negative examples by ID. +Let's start our connector's Docker Compose file by running the following from inside the connector's subgraph: -```graphql -query MyQuery { - qdrant_album(args: {recommend: {positive: [1], negative: [2]}}) { - artistId - id - score - title - vector - } -} +```bash +docker compose -f docker-compose.qdrant.yaml up ``` -The above query gets recommendations for albums with AlbumID = 1 as a positive example and AlbumID = 2 as a negative example. +### Step 9. Update the new `DataConnectorLink` object -Here is an example of a query that performs a vector search. +Finally, now that our `DataConnectorLink` has the correct environment variables configured for the connector, we can run the update command to have the CLI look at the configuration JSON and transform it to reflect our database's schema in `hml` format. In a new terminal tab, run: -```graphql -query MyQuery { - qdrant_album(args: {search: {vector: [0.5]}}) { - artistId - id - score - title - vector - } -} +```bash +ddn connector-link update qdrant --subgraph my_subgraph ``` -This will perform a similarity search and surface the most relevant results to the provided vector. (Note: The vector inputs are likely to be the embeddings vectors. These will be a array of floats of varying size.) +After this command runs, you can open your `my_subgraph/metadata/qdrant.hml` file and see your metadata completely +scaffolded out for you 🎉 + +## Documentation + +View the full documentation for the Qdrant connector [here](./docs/index.md). -There are additional parameters that can be used to configure the Qdrant search such as: +## Contributing -* score_threshold - Define a minimal score threshold for the result. If defined, less similar results will not be returned. Score of the returned result might be higher or smaller than the threshold depending on the Distance function used. E.g. for cosine similarity only higher scores will be returned. +Check out our [contributing guide](./docs/contributing.md) for more details. -* params - * hnsw_ef - Params relevant to HNSW index Size of the beam in a beam-search. Larger the value - more accurate the result, more time required for search. - * exact - Default: false Search without approximation. If set to true, search may run long but with exact results. - * indexed_only - Default: false If enabled, the engine will only perform search among indexed or small segments. Using this option prevents slow searches in case of delayed index, but does not guarantee that all uploaded vectors will be included in search results - * quantization - Default null - * ignore - Default: false If true, quantized vectors are ignored. Default is false. - * rescore - Default: null If true, use original vectors to re-score top-k results. Might require more time in case if original vectors are stored on disk. If not set, qdrant decides automatically apply rescoring or not. - * oversampling - Default: null Oversampling factor for quantization. Default is 1.0. Defines how many extra vectors should be pre-selected using quantized index, and then re-scored using original vectors. For example, if oversampling is 2.4 and limit is 100, then 240 vectors will be pre-selected using quantized index, and then top-100 will be returned after re-scoring. +## License -For more information, please see the [official Qdrant documentation](https://qdrant.github.io/qdrant/redoc/index.html#tag/points/operation/search_points). \ No newline at end of file +The Qdrant connector is available under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). \ No newline at end of file diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..b976473 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,54 @@ +# General Architecture of the Qdrant Connector + +## Query Engine +The query engine's job is to take a `QueryRequest`, which contains information about the query a user would like to run, translate it it into an API call using the Qdrant API, execute it, and return the results as a `QueryResponse`. + +One place in particular that uses the Query Engine is the `/query` endpoint (defined in the `ndc-hub` repository). + +`/query` endpoints receives a `QueryRequest`, and calls the `plan_queries` function from the Query Engine in order to create a QueryPlan which includes the information needed to execute the query. It then calls the `perform_query` function using the QueryPlan (which is run against the Qdrant API) and gets back a `QueryResponse` which it can then return to the caller. + +API: + +```typescript +export async function planQueries(query: QueryRequest, collectionNames: string[], collectionFields: { [key: string]: string[] }, collectionVectors: {[k: string]: boolean}): Promise +``` + +```typescript +export async function performQueries( + state: State, + query: QueryRequest, + queryPlan: QueryPlan): Promise +``` + +Note that the response from this function should be in the format of an ndc-spec [QueryResponse](https://hasura.github.io/ndc-spec/reference/types.html#queryresponse) represented as JSON. + +### Query Planning +The query plan is essentially side-effect free - we use information from the request as well as the information about the metadata to translate the query request into an API call to run against the database. + +This process is currently found in the [src/handlers](/src/handlers/query.ts) directory in the query.ts file. The API is the following function: + +```typescript +export async function planQueries(query: QueryRequest, collectionNames: string[], collectionFields: { [key: string]: string[] }, collectionVectors: {[k: string]: boolean}): Promise +``` + +The `plan_queries` function returns a `QueryPlan` which functions as an execution plan. + +```typescript +export type QueryPlan = { + collectionName: string; + scrollQueries: ScrollRequest[]; + searchQueries: SearchRequest[]; + recommendQueries: RecommendRequest[]; + dropAggregateRows: string[]; + fieldAliases: {[key: string]: string} +}; +``` + +The incoming `QueryRequest` is used to construct an API call that ultimately wraps the Qdrant API. + +## Patterns and guiding principles + +Here are a few ideas I have about working with this connector. + +### KISS (Keep it simple stupid!) +Robust and full-featured connector implementations should preferably be written in Rust for performance purposes. For Community Connectors it is preferred to try to keep things simple where possible, all we are doing is mapping the Query Request to an API call. diff --git a/docs/code-of-conduct.md b/docs/code-of-conduct.md new file mode 100644 index 0000000..4d6ab54 --- /dev/null +++ b/docs/code-of-conduct.md @@ -0,0 +1,74 @@ +# Hasura Community Code of Conduct + +## Our Pledge + +In the interest of fostering an open and welcoming environment, we as +contributors and maintainers pledge to make participation in our project and +our community a harassment-free experience for everyone, regardless of age, body +size, disability, ethnicity, sex characteristics, gender identity and expression, +level of experience, education, socio-economic status, nationality, personal +appearance, race, religion, or sexual identity and orientation. + +## Our Standards + +Examples of behavior that contributes to creating a positive environment +include: + +- Using welcoming, inclusive and gender-neutral language (example: instead of "Hey guys", + you could use "Hey folks" or "Hey all") +- Being respectful of differing viewpoints and experiences +- Gracefully accepting constructive criticism +- Focusing on what is best for the community +- Showing empathy towards other community members + +Examples of unacceptable behavior by participants include: + +- The use of sexualized language or imagery and unwelcome sexual attention or + advances +- Trolling, insulting/derogatory comments, and personal or political attacks +- Public or private harassment +- Publishing others' private information, such as a physical or electronic + address, without explicit permission +- Other conduct which could reasonably be considered inappropriate in a + professional setting + +## Our Responsibilities + +Project maintainers are responsible for clarifying the standards of acceptable +behavior and are expected to take appropriate and fair corrective action in +response to any instances of unacceptable behavior. + +Project maintainers have the right and responsibility to remove, edit, or +reject comments, commits, code, wiki edits, issues, and other contributions +that are not aligned to this Code of Conduct, or to ban temporarily or +permanently any contributor for other behaviors that they deem inappropriate, +threatening, offensive, or harmful. + +## Scope + +This Code of Conduct applies both within project spaces and in public spaces +when an individual is representing the project or its community. Examples of +representing a project or community include using an official project e-mail +address, posting via an official social media account, or acting as an appointed +representative at an online or offline event. Representation of a project may be +further defined and clarified by project maintainers. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be +reported by contacting the project team at community@hasura.io. All +complaints will be reviewed and investigated and will result in a response that +is deemed necessary and appropriate to the circumstances. The project team is +obligated to maintain confidentiality with regard to the reporter of an incident. +Further details of specific enforcement policies may be posted separately. + +Project maintainers who do not follow or enforce the Code of Conduct in good +faith may face temporary or permanent repercussions as determined by other +members of the project's leadership. + +## Attribution + +This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, +available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html + +[homepage]: https://www.contributor-covenant.org \ No newline at end of file diff --git a/docs/contributing.md b/docs/contributing.md new file mode 100644 index 0000000..bcfd9cb --- /dev/null +++ b/docs/contributing.md @@ -0,0 +1,39 @@ +# Contributing + +_First_: if you feel insecure about how to start contributing, feel free to ask us on our +[Discord channel](https://discordapp.com/invite/hasura) in the #contrib channel. You can also just go ahead with your contribution and we'll give you feedback. Don't worry - the worst that can happen is that you'll be politely asked to change something. We appreciate any contributions, and we don't want a wall of rules to stand in the way of that. + +However, for those individuals who want a bit more guidance on the best way to contribute to the project, read on. This document will cover what we're looking for. By addressing the points below, the chances that we can quickly merge or address your contributions will increase. + +## 1. Code of conduct + +Please follow our [Code of conduct](./code-of-conduct.md) in the context of any contributions made to Hasura. + +## 2. CLA + +For all contributions, a CLA (Contributor License Agreement) needs to be signed +[here](https://cla-assistant.io/hasura/ndc-duckdb) before (or after) the pull request has been submitted. A bot will prompt contributors to sign the CLA via a pull request comment, if necessary. + +## 3. Ways of contributing + +### Reporting an Issue + +- Make sure you test against the latest released cloud version. It is possible that we may have already fixed the bug you're experiencing. +- Provide steps to reproduce the issue, including Database (e.g. Postgres) version and Hasura DDN version. +- Please include logs, if relevant. +- Create a [issue](https://github.com/hasura/ndc-duckdb/issues/new/choose). + +### Working on an issue + +- We use the [fork-and-branch git workflow](https://blog.scottlowe.org/2015/01/27/using-fork-branch-git-workflow/). +- Please make sure there is an issue associated with the work that you're doing. +- If you're working on an issue, please comment that you are doing so to prevent duplicate work by others also. +- Squash your commits and refer to the issue using `fix #` or `close #` in the commit message, at the end. For example: `resolve answers to everything (fix #42)` or `resolve answers to everything, fix #42` +- Rebase master with your branch before submitting a pull request. + +## 6. Commit messages + +- The first line should be a summary of the changes, not exceeding 50 characters, followed by an optional body which has more details about the changes. Refer to [this link](https://github.com/erlang/otp/wiki/writing-good-commit-messages) for more information on writing good commit messages. +- Use the imperative present tense: "add/fix/change", not "added/fixed/changed" nor "adds/fixes/changes". +- Don't capitalize the first letter of the summary line. +- Don't add a period/dot (.) at the end of the summary line. \ No newline at end of file diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000..3e1be28 --- /dev/null +++ b/docs/development.md @@ -0,0 +1,61 @@ +# Qdrant Connector Development + +### Prerequisites +1. Follow the steps in the [README](../README.md) +2. Install [NodeJS & npm](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) + +### Clone the repo + +In a new directory, clone the repo using: + +```git clone https://github.com/hasura/ndc-qdrant``` + +### Run the Introspection + +If you are working with a Qdrant hosted database, you can run the introspection using: + +```QDRANT_URL=https://7312d6c4-3f6c-432c-987c-34d7d96428ef.us-east4-0.gcp.cloud.qdrant.io QDRANT_API_KEY=Ch8I... ts-node generate-config``` + +This will generate a `config.json` file. + +### Run the Connector + +To start the connector on port 9094, for a Qdranto hosted database instance run: +```HASURA_CONNECTOR_PORT=9094 QDRANT_URL=https://7312d6c4-3f6c-432c-987c-34d7d96428ef.us-east4-0.gcp.cloud.qdrant.io QDRANT_API_KEY=Ch8I... ts-node ./src/index.ts serve --configuration=.``` + +### Attach the connector to the locally running engine + +There should a file located at `my_subgraph/.env.my_subgraph` that contains + +```env +MY_SUBGRAPH_QDRANT_READ_URL=http://local.hasura.dev: +MY_SUBGRAPH_QDRANT_WRITE_URL=http://local.hasura.dev: +``` + +Create a new .env file called `.env.my_subgraph.dev` and place the following values into it: + +```env +MY_SUBGRAPH_QDRANT_READ_URL=http://local.hasura.dev:9094 +MY_SUBGRAPH_QDRANT_WRITE_URL=http://local.hasura.dev:9094 +``` + +In your `supergraph.yaml` file change the env file to point to the dev file. + +``` + subgraphs: + my_subgraph: + generator: + rootPath: my_subgraph + # envFile: my_subgraph/.env.my_subgraph # Change the env file + envFile: my_subgraph/.env.my_subgraph.dev + includePaths: + - my_subgraph/metadata +``` + +Do a local supergraph build: + +```ddn supergraph build local --output-dir ./engine``` + +Mutations and Queries will now be issued against your locally running connector instance. + +Depending on the changes made to the connector, you may need to re-generate the config. The best way to do this is to regenerate the config locally then move that config into the supergraph. \ No newline at end of file diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..e39afea --- /dev/null +++ b/docs/index.md @@ -0,0 +1,14 @@ +# Qdrant Documentation + +The Qdrant connector is a [Hasura](https://hasura.io/) Native Data Connector. + +- [Usage](./usage/index.md) + - [Querying (for example)](./usage/querying.md) +- [Architecture](./architecture.md) +- [Development](./development.md) +- [Code of Conduct](./code-of-conduct.md) +- [Contributing](./contributing.md) +- [Production](./production.md) +- [Support](./support.md) +- [Limitations](./limitations.md) +- [Security](./security.md) \ No newline at end of file diff --git a/docs/limitations.md b/docs/limitations.md new file mode 100644 index 0000000..1b99509 --- /dev/null +++ b/docs/limitations.md @@ -0,0 +1,4 @@ +# Limitations of the Qdrant connector + +* Offset currently only works when document ID's are Integers, and will not work if String ID's are used. +* Currently, queries are generated using collections which engine auto-generates the types for, it might be better to dynamically generate prodecures/commands since this database doesn't fully comply with the NDC spec. \ No newline at end of file diff --git a/docs/logo.png b/docs/logo.png new file mode 100644 index 0000000..12cbdad Binary files /dev/null and b/docs/logo.png differ diff --git a/docs/production.md b/docs/production.md new file mode 100644 index 0000000..6b86134 --- /dev/null +++ b/docs/production.md @@ -0,0 +1,30 @@ +# Hasura Qdrant Connector in Production +We ship the Qdrant connectors as Docker images. + +You can look at the Dockerfile in the root of the repository to see how the image is built. + +The connector can be run via a docker-compose file: + +``` +services: + qdrant_qdrant: + build: + context: . + dockerfile_inline: |- + FROM ghcr.io/hasura/ndc-qdrant:v0.2.1 + COPY ./ /etc/connector + develop: + watch: + - path: ./ + action: sync+restart + target: /etc/connector + env_file: + - .env.local + extra_hosts: + - local.hasura.dev=host-gateway + ports: + - mode: ingress + target: 8080 + published: "8086" + protocol: tcp +``` \ No newline at end of file diff --git a/docs/security.md b/docs/security.md new file mode 100644 index 0000000..e6d4dd9 --- /dev/null +++ b/docs/security.md @@ -0,0 +1,30 @@ +## Reporting Vulnerabilities + +We’re extremely grateful for security researchers and users that report vulnerabilities to the Hasura Community. All reports are thoroughly investigated by a set of community volunteers and the Hasura team. + +To report a security issue, please email us at [security@hasura.io](mailto:security@hasura.io) with all the details, attaching all necessary information. + +### When Should I Report a Vulnerability? + +- You think you have discovered a potential security vulnerability in the Hasura GraphQL Engine or related components. +- You are unsure how a vulnerability affects the Hasura GraphQL Engine. +- You think you discovered a vulnerability in another project that Hasura GraphQL Engine depends on (e.g. Heroku, Docker, etc). +- You want to report any other security risk that could potentially harm Hasura GraphQL Engine users. + +### When Should I NOT Report a Vulnerability? + +- You need help tuning Hasura GraphQL Engine components for security. +- You need help applying security related updates. +- Your issue is not security related. + +## Security Vulnerability Response + +Each report is acknowledged and analyzed by the project's maintainers and the security team within 3 working days. + +The reporter will be kept updated at every stage of the issue's analysis and resolution (triage -> fix -> release). + +## Public Disclosure Timing + +A public disclosure date is negotiated by the Hasura product security team and the bug submitter. We prefer to fully disclose the bug as soon as possible once a user mitigation is available. It is reasonable to delay disclosure when the bug or the fix is not yet fully understood, the solution is not well-tested, or for vendor coordination. The timeframe for disclosure is from immediate (especially if it's already publicly known) to a few weeks. We expect the time-frame between a report to a public disclosure to typically be in the order of 7 days. The Hasura GraphQL Engine maintainers and the security team will take the final call on setting a disclosure date. + +(Some sections have been inspired and adapted from [https://github.com/kubernetes/website/blob/master/content/en/docs/reference/issues-security/security.md](https://github.com/kubernetes/website/blob/master/content/en/docs/reference/issues-security/security.md)). \ No newline at end of file diff --git a/docs/support.md b/docs/support.md new file mode 100644 index 0000000..f814927 --- /dev/null +++ b/docs/support.md @@ -0,0 +1,11 @@ +# Support & Troubleshooting + +The documentation and community will help you troubleshoot most issues. If you have encountered a bug or need to get in touch with us, you can contact us using one of the following channels: +* Support & feedback: [Discord](https://discord.gg/hasura) +* Issue & bug tracking: [GitHub issues](https://github.com/hasura/ndc=[connectorName]/issues) +* Follow product updates: [@HasuraHQ](https://twitter.com/hasurahq) +* Talk to us on our [website chat](https://hasura.io) + +We are committed to fostering an open and welcoming environment in the community. Please see the [Code of Conduct](code-of-conduct.md). + +If you want to report a security issue, please [read this](security.md). \ No newline at end of file diff --git a/docs/usage/index.md b/docs/usage/index.md new file mode 100644 index 0000000..b33c77e --- /dev/null +++ b/docs/usage/index.md @@ -0,0 +1,5 @@ +# Turso Usage Documentation + +Information on how to use the Turso connector. + +- [Querying](./querying.md) \ No newline at end of file diff --git a/docs/usage/querying.md b/docs/usage/querying.md new file mode 100644 index 0000000..c8f5218 --- /dev/null +++ b/docs/usage/querying.md @@ -0,0 +1,65 @@ +# Querying Example + +Hasura instrospects the database and allows for Querying a pre-existing Qdrant database. + +If you want to try these queries out yourself, please see [this supergraph](https://github.com/hasura/super_supergraph/tree/main). + +These examples use [these Qdrant collections](https://github.com/hasura/super_supergraph/tree/main/qdrant/connector/qdrant/initdb) which contains data from the Chinook Music database. + +All Qdrant queries must pass an "args" object. An empty args object will perform a scroll request. You can also use the args object to perform a vector search, or a recommendation. JOIN's can be performed from the result of a Qdrant query to other connectors, but currently parameterized joins aren't supported. (And wouldn't make much sense to support!) + +Fetch all Albums + +```graphql +query Query { + qdrant_album(args: {}) { + albumId + artistId + title + } +} +``` + +Fetch all Albums where Album ID = 1 + +```graphql +query Query { + qdrant_album(args: {}, where: {albumId: {eq: 1}}) { + albumId + artistId + title + } +} +``` + +Perform a vector search: + +``` +query Query { + qdrant_boolean(args: {search: {vector: [0.5, 0.5]}}) { + a + b + c + d + id + score + vector + } +} +``` + +Get reccomendations providing positive and negative examples by ID: + +``` +query Query { + qdrant_boolean(args: {recommend: {positive: [0], negative: [1]}}) { + a + b + c + d + id + score + vector + } +} +``` \ No newline at end of file