Performances when a content has many routes

[EmmanuelVella]

Hi !

In my current project I have 4 menus for a total of 18 menu entries. Each of these menu entries are linked to a content, which has 4 routes (one for each locale). It takes about 2/3 seconds to render all the menus.

After some debugging, I found that when the getRouteLocale() method is called, it tries to find the good route, and the lazy loading of each one takes about 50/100ms for each.

Is there a way to solve this, maybe by forcing the loading a content with all its routes ?

[dbu]

i think it should be possible to do something about this. thanks for identifying what exactly the problem is. @JOiz was reporting the same problem.

i will try to figure out something to force load all menu nodes and all referenced documents and all referring routes in as few calls as possible. there is certainly room for improvement here, until we manage to solve #19

[lsmith77]

finding such issues and optimizing them should be much easier with doctrine/DoctrinePHPCRBundle#95

[EmmanuelVella]

👍

[dbu]

just to keep you updated: i created the following pull requests, largely helped by the logging:

• fetch referrer properties in one go jackalope/jackalope#193
• implementing support for the depthHint in Session::getNode jackalope/jackalope#192
• batch-fetch parent and child nodes. depth is numeric and can't be null doctrine/phpcr-odm#393
• agressively prefetch the menu as it needs to be completely loaded anyways #166

[dbu]

with these, i have pretty much run out of ideas for real optimization for now. we could try some caching which potentially would save a lot of calls (if the caching is faster than a jackrabbit, that is):
doctrine/phpcr-odm#392

@EmmanuelVella if you have any ideas what we could do further to reduce the number of requests, please tell. the main issue now seems to me that we can only get the referrers of a single node at a time, which can get cumbersome. if you could double check my various PR to ensure they also improve your situation, that would be great. (note that the whole prefetch thing is atm only implemented with jackalope-jackrabbit but not with doctrine dbal jackalope/jackalope-doctrine-dbal#157).

[EmmanuelVella]

@dbu This is great, thank you ! I'm going to make some tests and give you some feedback asap.

[EmmanuelVella]

@dbu I've made a little benchmark (using jackrabbit / jackalope in prod environment). Sadly, I almost have no performance boost.

Request number	w/ menus & w/o optimisations	w/ menus & w/ optimisations	w/o menus
1	0,874 ms	0,860 ms	0,382 ms
2	0,868 ms	0,817 ms	0,370 ms
3	0,926 ms	0,817 ms	0,385 ms
4	0,892 ms	0,835 ms	0,370 ms
5	0,874 ms	0,800 ms	0,372 ms

So I checked the code and it seems that the routes are still lazy loaded ! Is there something more to do to enable the prefetch ?

[EmmanuelVella]

In fact the prefetch is done (the $session->getNode($path, $this->getPrefetch()) is called and the returned json object seems ok), but the document manager still lazy loads the childrens when $menu = $dm->find(null, $path) is called.

[dbu]

thanks for the test. so obviously something is not yet good - did you use all branches i mentioned above? there is 2 in jackalope which you would need to locally merge, plus the one on phpcr-odm and #166 to get the thing actually activated. without jackalope/jackalope#192 the depthHint on getNode is not used at all.

and as said, with jackalope-doctrine-dbal, depth is not supported at all. i hope you tested with jackrabbit?

[lsmith77]

i am pretty sure we have fetch depth support in jackalope-doctrine-dbal

[dbu]

oh indeed you are right lukas, we have it. looking at the current implementation i think it makes fetch depth 0 unnecessary expensive but that should make no difference in these tests.

[EmmanuelVella]

@dbu Yes I got all the updates. The deep fetch is done (the request was something like /cms/menu/my-menu.10.json) and the returned object was containing the submenus. But it seems that the document manager didn't use this data as the collection wasn't initialized.

I'm going to do some more tests ! Thank you !

[dbu]

on phpcr-odm level its normal that it will still call getNode on phpcr. but jackalope should realize it has a cached version. from the json url i gather you where using jackalope jackrabbit, so theoretically it should have worked. what happens if you also configure a fetch depth of 1?

doctrine_phpcr:
    session:
        options:
            jackalope.fetch_depth: 1 # or 2 or 3

[lsmith77]

when testing this I would recommend to use the logging enabled version of DoctrinePHPCRBundle to see the actual requests handled by the transport layer.

[EmmanuelVella]

I'm doing some more tests !

@dbu Changing fetch depth option has no effect.
@lsmith77 How can I enable the logging in DoctrinePHPCRBundle ?

[lsmith77]

There is nothing much to do except for making sure you have this code doctrine/DoctrinePHPCRBundle#95

then you need to enable logging/profiling:

parameters:
    phpcr_backend:
        type: jackrabbit
        url: http://localhost:8080/server/
        logging: true
        profiling: true
    phpcr_workspace: sandbox
    phpcr_user: admin
    phpcr_pass: admin

also works with doctrine dbal

[dbu]

okay, so dbal prefetch on getNodes is not working i think. created a TODO comment and an issue jackalope/jackalope-doctrine-dbal#157