How to use the Dataloader correctly w/ Mongoose?

[mareksuscak]

Hi there. I saw the official examples and the documentation but I still do not fully understand how to leverage the built-in dataloader to address a N+1 query issue when using the Mongoose adapter. We keep seeing N+1 queries even when pulling 1st-level connections. We did not provide batchScheduleFn and I'm not sure when exactly it's necessary. Looking at some other examples of using the dataloader, it's rarely mentioned. I am expecting to see 2 Mongoose queries - one to retrieve the list of objects and another one to retrieve another list of objects for relationships - is this even how it's expected it to work? In reality what I see is one query to pull a list of objects and another query for each object and connection that's being accessed which seems unnecessary.

[TriPSs]

I do not know much about the Mongoose one, but willing to take a look, could you share your code?

[TriPSs]

@mareksuscak think he means these.

[mareksuscak]

That is what I've found but I don't see anything dataloader-related in that file unless I'm misunderstanding.

[smolinari]

@mareksuscak - As I noted in my longer post above, the Mongoose reference query service isn't using dataloaders. It's using Mongoose's populate functionality, which makes sense to use instead of dataloaders. This one place where it is used.

nestjs-query/packages/query-mongoose/src/services/reference-query.service.ts

Line 155 in c865ddb

const populated = await foundEntity.populate({ path: relationName, match: filterQuery })

Why your example isn't using populate is a good question. Unfortunately, I can't dig deeper at the moment. But, maybe if you can think in terms of populate, you might also find a solution for this too. I don't know. 🤷

Scott

[mareksuscak]

Oh okay, thanks for the additional details @smolinari! We override the generated service with a custom one that extends MongooseQueryService - maybe we're not doing that properly or maybe we should extend the ReferenceQueryService instead? That would be the only thing that comes to my mind. Or the wiring between the ReferenceQueryService and the MongooseQueryService isn't working as it should.

[mareksuscak]

@TriPSs and @smolinari: I invested some time into troubleshooting why this wasn't working and finally figured it out. This was a bug in the implementation of the Mongoose adapter and there's a similar issue in the Typegoose adapter which I am not intending to fix myself but anyone's invited to fix it based on my fix which is here: #279

Can you please review?