Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TASK] Adjust mount point indexing #4195

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion Classes/Domain/Index/PageIndexer/PageUriBuilder.php
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,15 @@ protected function buildPageIndexingUriFromPageItemAndLanguageId(
int $language = 0,
string $mountPointParameter = '',
): UriInterface {
$site = $this->siteFinder->getSiteByPageId($item->getRecordUid());

if ($item->hasIndexingProperty('isMountedPage')) {
$recordUid = (int)$item->getIndexingProperty('mountPageDestination');
} else {
$recordUid = $item->getRecordUid();
}

$site = $this->siteFinder->getSiteByPageId($recordUid);

$parameters = [];

if ($language > 0) {
Expand Down
16 changes: 15 additions & 1 deletion Documentation/FAQ/Index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -693,7 +693,7 @@ Yes. If user is not logged in one can redirect to a login page with the help of

The reason for that:

With Typo3 10 and solr 11.0.1 it was possible for solr to read it's configuration from shortcut pages directly. With Typo3 11 and solr 11.5 these shortcuts are followed. And if the access to the destination of the shortcut is restricted, solr cannot read the configuration because solr is not logged in.
With TYPO3 10 and solr 11.0.1 it was possible for solr to read it's configuration from shortcut pages directly. With TYPO3 11 and solr 11.5 these shortcuts are followed. And if the access to the destination of the shortcut is restricted, solr cannot read the configuration because solr is not logged in.

The solution:

Expand Down Expand Up @@ -721,3 +721,17 @@ The container log did not contain any output relevant to the container quitting.
Changing to Docker Desktop as provider keep the container alive.

Relevant DDEV link: https://ddev.readthedocs.io/en/latest/users/providers/



Can mount points be indexed?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Mount points are supported in general and the mounted pages will be indexed like standard pages.

But there are some points to consider:

* Cross-site mounts require that `config.index_enable = 1` is set in the page tree of the mounted page
This might change as due to changes in RootlineUtility->generateRootlineCache the mount point pid isn't used for the rootline.
* Mounted pages from a pagetree without a site configuration cannot be indexed, in fact TYPO3 currently can't mount a page from a page tree without a site configuration and an exeception occurs

35 changes: 30 additions & 5 deletions Documentation/Releases/solr-release-13-0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,44 @@
Releases 13.0
=============


Release 13.0.0-beta-1
=====================

This is a first beta release for TYPO3 13.4 LTS

New in this release
-------------------

Adjust mount point indexing
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Mount point indexing and corresponding tests have been adjusted for TYPO3 13. Mount points are supported in general and the mounted pages will be indexed like standard pages.

But there are some points to consider:

* Cross-site mounts require that `config.index_enable = 1` is set in the pagetree of the mounted page, as TYPO3 loads the TypoScript of the mounted page
* Mounted pages from a pagetree without a site configuration cannot be indexed, in fact TYPO3 currently can't mount a page from a page tree without a site configuration and an exeception occurs


Release 13.0.0-alpha-1
======================

This is a first alpha release for upcoming TYPO3 13 LTS

Known Bugs
----------

* Mount pages cannot be indexed, see `#4160 <https://github.com/TYPO3-Solr/ext-solr/issues/4160>`__

New in this release
-------------------

Adjust mount point indexing
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Mount point indexing and corresponding tests have been adjusted for TYPO3 13. Mount points are supported in general and the mounted pages will be indexed like standard pages.

But there are some points to consider:

* Cross-site mounts require that `config.index_enable = 1` is set in the pagetree of the mounted page, as TYPO3 loads the TypoScript of the mounted page
* Mounted pages from a pagetree without a site configuration cannot be indexed, in fact TYPO3 currently can't mount a page from a page tree without a site configuration and an exeception occurs

Comment on lines +36 to +45
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double pasted the copy?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, will remove the duplicate

!!! Upgrade to Apache Solr 9.7.0
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
42 changes: 22 additions & 20 deletions Tests/Integration/ConnectionManagerTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,6 @@
*/
class ConnectionManagerTest extends IntegrationTestBase
{
/**
* @inheritdoc
* @todo: Remove unnecessary fixtures and remove that property as intended.
*/
protected bool $skipImportRootPagesAndTemplatesForConfiguredSites = true;

protected function setUp(): void
{
parent::setUp();
Expand Down Expand Up @@ -195,24 +189,32 @@ public function exceptionIsThrownForUnAvailableSolrConnectionOnGetConnectionByPa
* ConnectionManager must use the connection for site(tree), where mount Point is defined.
* There is following scenario:
*
* [0]
* |
* ——[20] Shared-Pages (Folder)
* | |
* | ——[24] FirstShared
* | |
* | ——[25] first sub page from FirstShared
* | |
* | ——[26] second sub page from FirstShared
* |
* ——[ 1] Page (Root)
* |
* ——[14] Mount Point 1 (to [24] to show contents from)
* [0]
* |
* |—[211] Page3 (Root)
* | |
* | |—[24] FirstShared
* | |
* | |—[25] first sub page from FirstShared
* | |
* | |—[26] second sub page from FirstShared
* |
* |—[ 1] Page (Root)
* | |
* | |—[14] Mount Point 1 (to [24] to show contents from)
* |
* |—[111] Page2 (Root)
* |
* |—[34] Mount Point 2 (to [24] to show contents from)
*/
#[Test]
public function canFindSolrConnectionForMountedPageIfMountPointIsGiven(): void
{
self::markTestSkipped('@todo: Fix it. See: https://github.com/TYPO3-Solr/ext-solr/issues/4160');
self::markTestSkipped(
'@todo: Test currently fails in TYPO3 13.4 due to changes in RootlineUtility'
. ' RootlineUtility doesn\'t use the mount point pid in RootlineUtility->generateRootlineCache currently,'
. ' which is probably a bug. See: https://github.com/TYPO3-Solr/ext-solr/issues/4160'
);

$this->importCSVDataSet(__DIR__ . '/Fixtures/connection_for_mounted_page.csv');

Expand Down
21 changes: 1 addition & 20 deletions Tests/Integration/Fixtures/connection_basic.csv
Original file line number Diff line number Diff line change
Expand Up @@ -18,28 +18,9 @@
#
"pages",
,"uid","pid","is_siteroot","doktype","mount_pid","mount_pid_ol","slug","title"
,1,0,1,1,0,0,"/","First site"
,11,1,0,7,24,1,"/subpage","Subpage of first site"
,111,0,1,1,0,0,"/","Second site"
,21,111,0,7,24,1,"/subpage","Subpage of second site"
# detached and non Root Page-Tree
,3,0,0,1,0,0,"/","Detached and non Root Page-Tree"
,31,3,0,7,24,1,"/subpage","Subpage 1 of Detached"
,32,3,0,7,24,1,"/subpage-2","Subpage 2 of Detached"
"sys_template",
,"uid","pid","root","clear","sorting","config"
,1,1,1,3,100,"
page = PAGE
page.typeNum = 0
plugin.tx_solr {
enabled = 1
}"
,111,111,1,3,100,"
page = PAGE
page.typeNum = 0

plugin.tx_solr {
enabled = 1
}"


,32,3,0,7,24,1,"/subpage-2","Subpage 2 of Detached"
67 changes: 10 additions & 57 deletions Tests/Integration/Fixtures/connection_for_mounted_page.csv
Original file line number Diff line number Diff line change
Expand Up @@ -2,75 +2,28 @@
#
# [0]
# |
# ——[20] Shared-Pages (Folder: Not root)
# |—[211] Page3 (Root)
# | |
# | —[24] FirstShared
# | |—[24] FirstShared
# | |
# | —[25] first sub page from FirstShared
# | |—[25] first sub page from FirstShared
# | |
# | —[26] second sub page from FirstShared
# | |—[26] second sub page from FirstShared
# |
# ——[ 1] Page (Root)
# |—[ 1] Page (Root)
# | |
# | —[14] Mount Point 1 (to [24] to show contents from)
# | |—[14] Mount Point 1 (to [24] to show contents from)
# |
# ——[ 2] Page2 (Root)
# |—[111] Page2 (Root)
# |
# —[34] Mount Point 2 (to [24] to show contents from)
# |—[34] Mount Point 2 (to [24] to show contents from)
"pages",
,"uid","pid","is_siteroot","doktype","mount_pid","mount_pid_ol","slug","title"
# Shared Pages tree
,20,0,0,254,0,0,"/","Shared-Pages"
,24,20,0,1,0,0,"/first-shared","FirstShared (Not root)"
,24,211,0,1,0,0,"/first-shared","FirstShared (Not root)"
,25,24,0,1,0,0,"/first-shared/first-subpage","first sub page from FirstShared (Not root)"
,26,24,0,1,0,0,"/first-shared/second-subpage","second sub page from FirstShared (Not root)"
# Site tree
,1,0,1,1,0,0,"/","Page (Root)"
,14,1,0,7,24,1,"/mount-point-1","Mount Point 1"
# Second Site tree
,2,0,1,1,0,0,"/","Second Site (Root)"
,34,2,0,7,24,1,"/mount-point-2","Mount Point 2"
"sys_template",
,"uid","pid","root","clear","sorting","config"
,1,1,1,3,100,"
page = PAGE
page.typeNum = 0
config.index_enable = 1

plugin.tx_solr {
enabled = 1

index {
fieldProcessingInstructions {
changed = timestampToIsoDate
created = timestampToIsoDate
endtime = timestampToUtcIsoDate
rootline = pageUidToHierarchy
}

queue {
// mapping tableName.fields.SolrFieldName => TableFieldName (+ cObj processing)
pages = 1
pages {
initialization = ApacheSolrForTypo3\Solr\IndexQueue\Initializer\Page

// allowed page types (doktype) when indexing records from pages
allowedPageTypes = 1,7

indexingPriority = 0

indexer = ApacheSolrForTypo3\Solr\IndexQueue\PageIndexer
indexer {
// add options for the indexer here
}

// Only index standard pages and mount points that are not overlayed.
additionalWhereClause = (doktype = 1 OR (doktype=7 AND mount_pid_ol=0)) AND no_search = 0

fields {
sortSubTitle_stringS = subtitle
}
}
}
}
}"
,34,111,0,7,24,1,"/mount-point-2","Mount Point 2"
Original file line number Diff line number Diff line change
@@ -1,24 +1,23 @@
# There is following scenario:
#
# [0]
# |
# ——[20] Shared-Pages (Not root)
# | |
# | ——[24] FirstShared (Not root)
# |
# ——[ 1] Page (Root)
# |—[ 1] Page (Root Testpage 1)
# |
# ——[14] Mount Point (to [24] to show contents from)
# |—[14] Mount Point (to [24] to show contents from)
# |
# |—[24] FirstShared
"pages",
,"uid","pid","is_siteroot","doktype","mount_pid","mount_pid_ol","slug","title"
# Site tree
,14,1,0,7,24,1,"/mount-point","Mount Point"
# Shared Pages tree
,20,0,0,254,0,0,"/","Shared-Pages"
,24,20,0,1,0,0,"/first-shared","FirstShared (Not root)"
,24,1,0,1,0,0,"/first-shared","FirstShared"
"tt_content",
,"uid","pid","colPos","CType","bodytext"
,99,24,0,"text","Some Lorem Ipsum conteint!"
"tx_solr_indexqueue_item",
,"uid","root","item_type","item_uid","indexing_configuration","changed","indexed","has_indexing_properties","indexing_priority","indexed","errors"
,4711,1,"pages",24,"pages",1449151778,0,0,0,0,0
,"uid","root","item_type","item_uid","indexing_configuration","changed","indexed","has_indexing_properties","indexing_priority","indexed","errors","pages_mountidentifier"
,4711,1,"pages",24,"pages",1449151778,0,1,0,0,0,"24-14-1"
"tx_solr_indexqueue_indexing_property",
,"uid","root","item_id","property_key","property_value"
,1,1,4711,"mountPageSource",24
,2,1,4711,"mountPageDestination",14
,3,1,4711,"isMountedPage",1
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# There is following scenario:
#
# [0]
# |
# |—[ 111] Page (Root Testpage 2)
# | |
# | |—[24] FirstShared
# |
# |—[ 1] Page (Root Testpage 1)
# |
# |—[14] Mount Point (to [24] to show contents from)
"pages",
,"uid","pid","is_siteroot","doktype","mount_pid","mount_pid_ol","slug","title"
,14,1,0,7,24,1,"/mount-point","Mount Point"
,24,111,0,1,0,0,"/first-shared","FirstShared"
"tt_content",
,"uid","pid","colPos","CType","bodytext"
,99,24,0,"text","Some Lorem Ipsum conteint!"
"tx_solr_indexqueue_item",
,"uid","root","item_type","item_uid","indexing_configuration","changed","indexed","has_indexing_properties","indexing_priority","indexed","errors","pages_mountidentifier"
,4711,1,"pages",24,"pages",1449151778,0,1,0,0,0,"24-14-1"
"tx_solr_indexqueue_indexing_property",
,"uid","root","item_id","property_key","property_value"
,1,1,4711,"mountPageSource",24
,2,1,4711,"mountPageDestination",14
,3,1,4711,"isMountedPage",1
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# There is following scenario:
#
# [0]
# |
# |—[20] Shared-Pages (Not root)
# | |
# | |—[24] FirstShared (Not root)
# |
# |—[ 1] Page (Root)
# |
# |—[14] Mount Point (to [24] to show contents from)
"pages",
,"uid","pid","is_siteroot","doktype","mount_pid","mount_pid_ol","slug","title"
,14,1,0,7,24,1,"/mount-point","Mount Point"
,24,20,0,1,0,0,"/first-shared","FirstShared"
"tt_content",
,"uid","pid","colPos","CType","bodytext"
,99,24,0,"text","Some Lorem Ipsum conteint!"
"tx_solr_indexqueue_item",
,"uid","root","item_type","item_uid","indexing_configuration","changed","indexed","has_indexing_properties","indexing_priority","indexed","errors","pages_mountidentifier"
,4711,1,"pages",24,"pages",1449151778,0,1,0,0,0,"24-14-1"
"tx_solr_indexqueue_indexing_property",
,"uid","root","item_id","property_key","property_value"
,1,1,4711,"mountPageSource",24
,2,1,4711,"mountPageDestination",14
,3,1,4711,"isMountedPage",1
Loading