Skip to content

Commit

Permalink
[TASK] Adjust mount point indexing
Browse files Browse the repository at this point in the history
Mount point indexing and corresponding tests have been adjusted for
TYPO3 13. Mount points are supported in general and the mounted pages
will be indexed like standard pages.

But there are some points to consider:

*   Cross-site mounts require that `config.index_enable = 1` is set
    in the pagetree of the mounted page, as TYPO3 loads the
    TypoScript of the mounted page.
    This is probably a bug in the TYPO3 core and may change, due to
    changes in RootlineUtility->generateRootlineCache() the mount page
    pid isn't used any more.
*   Pages in a pagetree without a site configuration cannot be
    indexed, in fact TYPO3 currently can't mount a page from a page
    tree without a site configuration and an exeception occurs.

Test procedure in PageIndexerTest has been improved and now uses
the PageUriBuilder to build the url to index, instead of building
the url in the test itself.

Resolves: #4160
  • Loading branch information
dkd-friedrich committed Oct 16, 2024
1 parent 81959d9 commit 08e5ff0
Show file tree
Hide file tree
Showing 12 changed files with 305 additions and 147 deletions.
10 changes: 9 additions & 1 deletion Classes/Domain/Index/PageIndexer/PageUriBuilder.php
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,15 @@ protected function buildPageIndexingUriFromPageItemAndLanguageId(
int $language = 0,
string $mountPointParameter = '',
): UriInterface {
$site = $this->siteFinder->getSiteByPageId($item->getRecordUid());

if ($item->hasIndexingProperty('isMountedPage')) {
$recordUid = (int)$item->getIndexingProperty('mountPageDestination');
} else {
$recordUid = $item->getRecordUid();
}

$site = $this->siteFinder->getSiteByPageId($recordUid);

$parameters = [];

if ($language > 0) {
Expand Down
16 changes: 15 additions & 1 deletion Documentation/FAQ/Index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -693,7 +693,7 @@ Yes. If user is not logged in one can redirect to a login page with the help of

The reason for that:

With Typo3 10 and solr 11.0.1 it was possible for solr to read it's configuration from shortcut pages directly. With Typo3 11 and solr 11.5 these shortcuts are followed. And if the access to the destination of the shortcut is restricted, solr cannot read the configuration because solr is not logged in.
With TYPO3 10 and solr 11.0.1 it was possible for solr to read it's configuration from shortcut pages directly. With TYPO3 11 and solr 11.5 these shortcuts are followed. And if the access to the destination of the shortcut is restricted, solr cannot read the configuration because solr is not logged in.

The solution:

Expand Down Expand Up @@ -721,3 +721,17 @@ The container log did not contain any output relevant to the container quitting.
Changing to Docker Desktop as provider keep the container alive.

Relevant DDEV link: https://ddev.readthedocs.io/en/latest/users/providers/



Can mount points be indexed?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Mount points are supported in general and the mounted pages will be indexed like standard pages.

But there are some points to consider:

* Cross-site mounts require that `config.index_enable = 1` is set in the page tree of the mounted page
This might change as due to changes in RootlineUtility->generateRootlineCache the mount point pid isn't used for the rootline.
* Mounted pages from a pagetree without a site configuration cannot be indexed, in fact TYPO3 currently can't mount a page from a page tree without a site configuration and an exeception occurs

35 changes: 30 additions & 5 deletions Documentation/Releases/solr-release-13-0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,44 @@
Releases 13.0
=============


Release 13.0.0-beta-1
=====================

This is a first beta release for TYPO3 13.4 LTS

New in this release
-------------------

Adjust mount point indexing
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Mount point indexing and corresponding tests have been adjusted for TYPO3 13. Mount points are supported in general and the mounted pages will be indexed like standard pages.

But there are some points to consider:

* Cross-site mounts require that `config.index_enable = 1` is set in the pagetree of the mounted page, as TYPO3 loads the TypoScript of the mounted page
* Mounted pages from a pagetree without a site configuration cannot be indexed, in fact TYPO3 currently can't mount a page from a page tree without a site configuration and an exeception occurs


Release 13.0.0-alpha-1
======================

This is a first alpha release for upcoming TYPO3 13 LTS

Known Bugs
----------

* Mount pages cannot be indexed, see `#4160 <https://github.com/TYPO3-Solr/ext-solr/issues/4160>`__

New in this release
-------------------

Adjust mount point indexing
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Mount point indexing and corresponding tests have been adjusted for TYPO3 13. Mount points are supported in general and the mounted pages will be indexed like standard pages.

But there are some points to consider:

* Cross-site mounts require that `config.index_enable = 1` is set in the pagetree of the mounted page, as TYPO3 loads the TypoScript of the mounted page
* Mounted pages from a pagetree without a site configuration cannot be indexed, in fact TYPO3 currently can't mount a page from a page tree without a site configuration and an exeception occurs

!!! Upgrade to Apache Solr 9.7.0
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
36 changes: 22 additions & 14 deletions Tests/Integration/ConnectionManagerTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -189,24 +189,32 @@ public function exceptionIsThrownForUnAvailableSolrConnectionOnGetConnectionByPa
* ConnectionManager must use the connection for site(tree), where mount Point is defined.
* There is following scenario:
*
* [0]
* |
* ——[20] Shared-Pages (Folder)
* | |
* | ——[24] FirstShared
* | |
* | ——[25] first sub page from FirstShared
* | |
* | ——[26] second sub page from FirstShared
* |
* ——[ 1] Page (Root)
* |
* ——[14] Mount Point 1 (to [24] to show contents from)
* [0]
* |
* |—[211] Page3 (Root)
* | |
* | |—[24] FirstShared
* | |
* | |—[25] first sub page from FirstShared
* | |
* | |—[26] second sub page from FirstShared
* |
* |—[ 1] Page (Root)
* | |
* | |—[14] Mount Point 1 (to [24] to show contents from)
* |
* |—[111] Page2 (Root)
* |
* |—[34] Mount Point 2 (to [24] to show contents from)
*/
#[Test]
public function canFindSolrConnectionForMountedPageIfMountPointIsGiven(): void
{
self::markTestSkipped('@todo: Fix it. See: https://github.com/TYPO3-Solr/ext-solr/issues/4160');
self::markTestSkipped(
'@todo: Test currently fails in TYPO3 13.4 due to changes in RootlineUtility'
. ' RootlineUtility doesn\'t use the mount point pid in RootlineUtility->generateRootlineCache currently,'
. ' which is probably a bug. See: https://github.com/TYPO3-Solr/ext-solr/issues/4160'
);

$this->importCSVDataSet(__DIR__ . '/Fixtures/connection_for_mounted_page.csv');

Expand Down
67 changes: 10 additions & 57 deletions Tests/Integration/Fixtures/connection_for_mounted_page.csv
Original file line number Diff line number Diff line change
Expand Up @@ -2,75 +2,28 @@
#
# [0]
# |
# ——[20] Shared-Pages (Folder: Not root)
# |—[211] Page3 (Root)
# | |
# | —[24] FirstShared
# | |—[24] FirstShared
# | |
# | —[25] first sub page from FirstShared
# | |—[25] first sub page from FirstShared
# | |
# | —[26] second sub page from FirstShared
# | |—[26] second sub page from FirstShared
# |
# ——[ 1] Page (Root)
# |—[ 1] Page (Root)
# | |
# | —[14] Mount Point 1 (to [24] to show contents from)
# | |—[14] Mount Point 1 (to [24] to show contents from)
# |
# ——[ 2] Page2 (Root)
# |—[111] Page2 (Root)
# |
# —[34] Mount Point 2 (to [24] to show contents from)
# |—[34] Mount Point 2 (to [24] to show contents from)
"pages",
,"uid","pid","is_siteroot","doktype","mount_pid","mount_pid_ol","slug","title"
# Shared Pages tree
,20,0,0,254,0,0,"/","Shared-Pages"
,24,20,0,1,0,0,"/first-shared","FirstShared (Not root)"
,24,211,0,1,0,0,"/first-shared","FirstShared (Not root)"
,25,24,0,1,0,0,"/first-shared/first-subpage","first sub page from FirstShared (Not root)"
,26,24,0,1,0,0,"/first-shared/second-subpage","second sub page from FirstShared (Not root)"
# Site tree
,1,0,1,1,0,0,"/","Page (Root)"
,14,1,0,7,24,1,"/mount-point-1","Mount Point 1"
# Second Site tree
,2,0,1,1,0,0,"/","Second Site (Root)"
,34,2,0,7,24,1,"/mount-point-2","Mount Point 2"
"sys_template",
,"uid","pid","root","clear","sorting","config"
,1,1,1,3,100,"
page = PAGE
page.typeNum = 0
config.index_enable = 1

plugin.tx_solr {
enabled = 1

index {
fieldProcessingInstructions {
changed = timestampToIsoDate
created = timestampToIsoDate
endtime = timestampToUtcIsoDate
rootline = pageUidToHierarchy
}

queue {
// mapping tableName.fields.SolrFieldName => TableFieldName (+ cObj processing)
pages = 1
pages {
initialization = ApacheSolrForTypo3\Solr\IndexQueue\Initializer\Page

// allowed page types (doktype) when indexing records from pages
allowedPageTypes = 1,7

indexingPriority = 0

indexer = ApacheSolrForTypo3\Solr\IndexQueue\PageIndexer
indexer {
// add options for the indexer here
}

// Only index standard pages and mount points that are not overlayed.
additionalWhereClause = (doktype = 1 OR (doktype=7 AND mount_pid_ol=0)) AND no_search = 0

fields {
sortSubTitle_stringS = subtitle
}
}
}
}
}"
,34,111,0,7,24,1,"/mount-point-2","Mount Point 2"
Original file line number Diff line number Diff line change
@@ -1,24 +1,23 @@
# There is following scenario:
#
# [0]
# |
# ——[20] Shared-Pages (Not root)
# | |
# | ——[24] FirstShared (Not root)
# |
# ——[ 1] Page (Root)
# |—[ 1] Page (Root Testpage 1)
# |
# ——[14] Mount Point (to [24] to show contents from)
# |—[14] Mount Point (to [24] to show contents from)
# |
# |—[24] FirstShared
"pages",
,"uid","pid","is_siteroot","doktype","mount_pid","mount_pid_ol","slug","title"
# Site tree
,14,1,0,7,24,1,"/mount-point","Mount Point"
# Shared Pages tree
,20,0,0,254,0,0,"/","Shared-Pages"
,24,20,0,1,0,0,"/first-shared","FirstShared (Not root)"
,24,1,0,1,0,0,"/first-shared","FirstShared"
"tt_content",
,"uid","pid","colPos","CType","bodytext"
,99,24,0,"text","Some Lorem Ipsum conteint!"
"tx_solr_indexqueue_item",
,"uid","root","item_type","item_uid","indexing_configuration","changed","indexed","has_indexing_properties","indexing_priority","indexed","errors"
,4711,1,"pages",24,"pages",1449151778,0,0,0,0,0
,"uid","root","item_type","item_uid","indexing_configuration","changed","indexed","has_indexing_properties","indexing_priority","indexed","errors","pages_mountidentifier"
,4711,1,"pages",24,"pages",1449151778,0,1,0,0,0,"24-14-1"
"tx_solr_indexqueue_indexing_property",
,"uid","root","item_id","property_key","property_value"
,1,1,4711,"mountPageSource",24
,2,1,4711,"mountPageDestination",14
,3,1,4711,"isMountedPage",1
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# There is following scenario:
#
# [0]
# |
# |—[ 111] Page (Root Testpage 2)
# | |
# | |—[24] FirstShared
# |
# |—[ 1] Page (Root Testpage 1)
# |
# |—[14] Mount Point (to [24] to show contents from)
"pages",
,"uid","pid","is_siteroot","doktype","mount_pid","mount_pid_ol","slug","title"
,14,1,0,7,24,1,"/mount-point","Mount Point"
,24,111,0,1,0,0,"/first-shared","FirstShared"
"tt_content",
,"uid","pid","colPos","CType","bodytext"
,99,24,0,"text","Some Lorem Ipsum conteint!"
"tx_solr_indexqueue_item",
,"uid","root","item_type","item_uid","indexing_configuration","changed","indexed","has_indexing_properties","indexing_priority","indexed","errors","pages_mountidentifier"
,4711,1,"pages",24,"pages",1449151778,0,1,0,0,0,"24-14-1"
"tx_solr_indexqueue_indexing_property",
,"uid","root","item_id","property_key","property_value"
,1,1,4711,"mountPageSource",24
,2,1,4711,"mountPageDestination",14
,3,1,4711,"isMountedPage",1
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# There is following scenario:
#
# [0]
# |
# |—[20] Shared-Pages (Not root)
# | |
# | |—[24] FirstShared (Not root)
# |
# |—[ 1] Page (Root)
# |
# |—[14] Mount Point (to [24] to show contents from)
"pages",
,"uid","pid","is_siteroot","doktype","mount_pid","mount_pid_ol","slug","title"
,14,1,0,7,24,1,"/mount-point","Mount Point"
,24,20,0,1,0,0,"/first-shared","FirstShared"
"tt_content",
,"uid","pid","colPos","CType","bodytext"
,99,24,0,"text","Some Lorem Ipsum conteint!"
"tx_solr_indexqueue_item",
,"uid","root","item_type","item_uid","indexing_configuration","changed","indexed","has_indexing_properties","indexing_priority","indexed","errors","pages_mountidentifier"
,4711,1,"pages",24,"pages",1449151778,0,1,0,0,0,"24-14-1"
"tx_solr_indexqueue_indexing_property",
,"uid","root","item_id","property_key","property_value"
,1,1,4711,"mountPageSource",24
,2,1,4711,"mountPageDestination",14
,3,1,4711,"isMountedPage",1
Original file line number Diff line number Diff line change
@@ -1,29 +1,34 @@
# There is following scenario:
# [0]
# |
# ——[20] Shared-Pages (Not root)
# |—[111] 2nd page root
# | |
# | —[44] FirstShared (Not root)
# | |—[44] FirstShared (Not root)
# |
# —[ 1] Page (Root)
# |—[ 1] Page (Root)
# |
# ——[14] Mount Point (to [44] to show contents from)
# |
# ——[ 2] Page (Root)
# |—[14] Mount Point (to [44] to show contents from)
# |
# —[24] Mount Point (to [44] to show contents from)
# |—[24] Mount Point (to [44] to show contents from)
"pages",
,"uid","pid","is_siteroot","doktype","mount_pid","mount_pid_ol","slug","title"
# Site tree a
,14,1,0,7,44,1,"/mount-point","Mount Point"
,24,1,0,7,44,1,"/mount-point-2","Mount Point 2"
# Shared Pages tree
,20,0,0,254,0,0,"/shared-pages","Shared-Pages"
,44,20,0,1,0,0,"/first-shared","FirstShared (Not root)"
,44,111,0,1,0,0,"/first-shared","FirstShared (Not root)"
"tt_content",
,"uid","pid","colPos","CType","bodytext"
,99,44,0,"text","Some Lorem Ipsum conteint!"
"tx_solr_indexqueue_item",
,"uid","root","item_type","item_uid","indexing_configuration","changed","indexed","has_indexing_properties","indexing_priority","indexed","errors"
,4711,1,"pages",44,"pages",1449151778,0,0,0,0,0
,4712,1,"pages",44,"pages",1449151778,0,0,0,0,0
,4711,1,"pages",14,"pages",1449151778,0,1,0,0,0
,4712,1,"pages",24,"pages",1449151778,0,1,0,0,0
"tx_solr_indexqueue_indexing_property",
,"uid","root","item_id","property_key","property_value"
,1,1,4711,"mountPageSource",44
,2,1,4711,"mountPageDestination",14
,3,1,4711,"isMountedPage",1
,4,1,4712,"mountPageSource",44
,5,1,4712,"mountPageDestination",24
,6,1,4712,"isMountedPage",1
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"pages",
,"uid","pid","is_siteroot","doktype","slug","title","subtitle","crdate","tstamp"
,2,1,1,1,"/","hello solr","the subtitle",1449151778,1449151778
,2,1,1,1,"/hello-solr","hello solr","the subtitle",1449151778,1449151778
"tx_solr_indexqueue_item",
,"uid","root","item_type","item_uid","indexing_configuration","changed","indexed","has_indexing_properties","indexing_priority","indexed","errors"
,4711,1,"pages",2,"pages",1449151778,0,0,0,0,0
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"tx_solr_indexqueue_item",
,"uid","root","item_type","item_uid","indexing_configuration","changed","indexed","has_indexing_properties","indexing_priority","indexed","errors"
,4711,999,"pages",1636120156,"pages",1449151778,0,0,0,0,0
,4711,1,"pages",1636120156,"pages",1449151778,0,0,0,0,0
Loading

0 comments on commit 08e5ff0

Please sign in to comment.