Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error getting documents: Error: 9 FAILED_PRECONDITION when using collectionGroup #1866

Open
dumitruPuggle opened this issue Jul 25, 2023 · 22 comments
Assignees
Labels
api: firestore Issues related to the googleapis/nodejs-firestore API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@dumitruPuggle
Copy link

Environment details

  • OS: 13.2.1 (22D68)
  • Node.js version: v18.15.0
  • npm version: v9.5.0
  • @google-cloud/firestore version: I was told to come to this repo to get the responses quickly, although I'm using firestore-admin 11.9.0

Steps to reproduce

  1. Use any collectionGroup and call get function
 db.collectionGroup("textTemplates")
  .where("id", "==", templateId)
  .get()
  1. You'll receive an error:
Google API requested!
 - URL: "https://oauth2.googleapis.com/token"
 - Be careful, this may be a production service.
>  Error getting documents:  Error: 9 FAILED_PRECONDITION: 
>      at callErrorFromStatus (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
>      at Object.onReceiveStatus (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/client.js:357:73)
>      at Object.onReceiveStatus (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:323:181)
>      at /Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/resolving-call.js:94:78
>      at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
>  for call at
>      at ServiceClientImpl.makeServerStreamRequest (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/client.js:340:32)
>      at ServiceClientImpl.<anonymous> (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
>      at /Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/build/src/v1/firestore_client.js:227:29
>      at /Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/node_modules/google-gax/build/src/streamingCalls/streamingApiCaller.js:38:28
>      at /Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16
>      at Object.request (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/node_modules/google-gax/build/src/streamingCalls/streaming.js:130:40)
>      at makeRequest (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/retry-request/index.js:141:28)
>      at retryRequest (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/retry-request/index.js:109:5)
>      at StreamProxy.setStream (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/node_modules/google-gax/build/src/streamingCalls/streaming.js:121:37)
>      at StreamingApiCaller.call (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/node_modules/google-gax/build/src/streamingCalls/streamingApiCaller.js:54:16)
>  Caused by: Error
>      at Query._get (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/build/src/reference.js:1717:23)
>      at Query.get (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/build/src/reference.js:1705:21)
>      at startTextSequencesHandler (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/lib/functions/callable/start-text-sequences.js:25:10)
>      at fixedLen (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/firebase-functions/lib/v2/providers/https.js:118:31)
>      at /Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/firebase-functions/lib/common/providers/https.js:467:32
>      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
>      at async runFunction (/Users/dumitrucucu/.nvm/versions/node/v18.15.0/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:506:9)
>      at async runHTTPS (/Users/dumitrucucu/.nvm/versions/node/v18.15.0/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:531:5)
>      at async /Users/dumitrucucu/.nvm/versions/node/v18.15.0/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:693:21 {
>    code: 9,
>    details: '',
>    metadata: Metadata { internalRepr: Map(0) {}, options: {} }
>  }

I created the necessary indexes on Firestore Console:
Screenshot 2023-07-25 at 17 42 50

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

@dumitruPuggle dumitruPuggle added priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Jul 25, 2023
@product-auto-label product-auto-label bot added the api: firestore Issues related to the googleapis/nodejs-firestore API. label Jul 25, 2023
@tom-andersen tom-andersen self-assigned this Jul 25, 2023
@tom-andersen
Copy link
Contributor

I suspect you are using OAuth 2.0.

See: https://firebase.google.com/docs/admin/setup

Note: OAuth 2.0 refresh tokens are not supported for connecting to Cloud Firestore.

@dumitruPuggle
Copy link
Author

I found the issue. All I had to do is to use orderBy, and after that I received an index link for firestore console.

@diginikkari
Copy link

I think that the main issue here is that the FAILED_PRECONDITION error is not including create query index -link.

@leonardopn
Copy link

My firebase-admin: ^11.10.1

Same problem running this code snippet in a deployed code:

static async getAllByIdProduct(id: string) {
	const query = await firestore
		.collectionGroup("modelos")
		.where("idsDeProduto", "array-contains", id)
		.get();

	return query.docs.map(doc => ({
		data: doc.data() as RoteiroModelDTO,
		ref: doc.ref as DocumentReference<RoteiroModelDTO>,
	}));
}

Error in GCP:

image

It really seems that the link to create the indexes should come, but it's just showing a generic error

@dconeybe
Copy link
Contributor

Thank you for this additional information. @tom-andersen will follow up early next week.

@tom-andersen
Copy link
Contributor

Update:

@leonardopn I verified the server logs, and the error message describing the problem along with link to create index was provided. But somewhere from the time that response is sent to cloud function and the output being logged, the error message is being stripped out. I see no reason for why the SDK should do this, so I wonder if there is something in the logging setup that causes this.

If you run the query locally on your development machine, I imagine the error with link should appear.

I will see if I can reproduce error with cloud function.

@dumitruPuggle
Copy link
Author

Hi guys

Was this issue resolved? I'm asking because I'm sill getting this error in production.

Screenshot 2023-09-19 at 23 02 31

@tom-andersen
Copy link
Contributor

Hi @dumitruPuggle

We still don't know why the text is stripped from the error. However, the resolution remains the same. Create the required composite index. For now you will need to run the code locally (or not within cloud function), and then there will be a helpful link as part of error.

@ivanmartinvalle
Copy link

Running into this also. As an alternative to running the query locally, I was able to get the index creation link by using the GCP Console with the Query Builder. But even if the link wasn't generated via that method, it does allow you to quickly test if a particular query would be supported, and then manually go through the process of adding the index in the GCP Console.

@dumitruPuggle
Copy link
Author

Hey guys,

I found an alternative solution to fix this problem.
Here are the steps:
Screenshot 2023-11-05 at 00 27 30

  1. Go to Firestore

  2. Click on Query builder tab

  3. Replicate the same query through UI:

Screenshot 2023-11-05 at 00 28 10

That's it. You get the index link in console.

@alexstanbury
Copy link

This is absolutely ridiculous, you can't warn us when developing and you 'don't know why the text is stripped from the error', so we can't even set up a proper alert in case a missing index gets through to production. Come on Google, do better.

@jelling
Copy link

jelling commented Apr 18, 2024

I was referred here by GCP support - which we pay for - so it's extra frustrating to see that a repeatable issue affecting basic functionality is known but receiving no attention. GCP support also referred me to this likely related issue that has been open since 2022.

It is impossible for me to believe that with all the smart people I know at Google that no one can figure out why an error message is getting truncated. So when can we expect a resolution for this?

I know no one the support team is going to lose sleep if we leave Firebase, but we have had so many similar Firebase support experiences - long open issues with basic functionality - that it's increasingly hard to believe that GCP sees Firebase as a priority.

@kossnocorp
Copy link

I believe that the issue should receive higher priority. It seems to have slipped through the cracks, and the team isn't really aware of the problem.

Not being able to receive the index link makes the collection group's DX hellish. Let's alone say we have to deploy to production and hope to catch the errors before users do, but we also have to guess which query is failing.

@MarkDuckworth forgive me for pinging you. I can see your commits in the main branch, so hopefully, you can access the actual issue priority.

@tom-andersen
Copy link
Contributor

Hi @kossnocorp,

I am trying to move this forward, since this has somewhat fallen between teams. So here are some updates:

  1. Logging is not the cause. The text message is stripped before exception is emitted as part of Firebase Function.
  2. I have produced a reproduction and shared it with Firebase Functions. They will investigate whether this happens as part of Firebase Function or further back in the system under a Cloud Function. Someone from FIrebase Functions will investigate further.

When we know more, I will update this issue.

@jelling
Copy link

jelling commented May 1, 2024 via email

@tom-andersen
Copy link
Contributor

This problem sometimes occurs in NodeJS 18 and 20, but not 16 from our observation.

For Cloud Functions and Firebase Functions, reducing the engine to NodeJS version 16 removes the problem.

@sirusbaladi
Copy link

orderBy

this fixed it for me.

@leblancmeneses
Copy link

I've been a Firebase user since 2016. This error did not include the link to create the missing index, making it difficult to manually create this collection group index.

image

This worked in the emulator, but in production, it failed because the emulator doesn't use indexes. Not having the link resulted in an unnecessary time sink. I'm lucky to have known what query I added and prior projects I could take the collection group index out to modify for this new application I was working on.

{
  "indexes": [],
  "fieldOverrides": [
    {
      "collectionGroup": "shortredirectlink_shortlink_campaigns_cg",
      "fieldPath": "short_code",
      "indexes": [
        {
          "order": "ASCENDING",
          "queryScope": "COLLECTION"
        },
        {
          "order": "DESCENDING",
          "queryScope": "COLLECTION"
        },
        {
          "arrayConfig": "CONTAINS",
          "queryScope": "COLLECTION"
        },
        {
          "order": "ASCENDING",
          "queryScope": "COLLECTION_GROUP"
        }
      ]
    }]
}

@dalenguyen
Copy link

Agree. I'm having the same issue today and the error log is not helpful.

@ohprettyhak
Copy link

I am still experiencing this issue today. It is not possible to create the index through the logs.

@marcinbunsch
Copy link

@dumitruPuggle thank you so much, I had the same issue - no orderBy means no error message. When I add orderBy, I get a link to create an index

@silasljennings
Copy link

I want to add to this thread. The solution I found for this was as simple (not so simple if you don't know) as going to the console query builder and executing the query I wanted to execute. In terms of running it on the collectionGroup, it will not execute but will give you the link to setup the indexes you are missing.

Screenshot 2024-12-19 at 10 31 57 PM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: firestore Issues related to the googleapis/nodejs-firestore API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests