Improve search index generation for PHP.net

[lhsazevedo]

Note

This is a companion PR to php/web-php#1084, but it is not dependent on it and can be merged independently.

Intro

PHD index entries can have both short (sdesc) and long (ldesc) descriptions. Often, the short description is empty, leading to the current fallback mechanism in getShortDescription and getLongDescription:

// phpdotnet/phd/Format.php
final public function getLongDescription($id, &$isLDesc = null) {
    if ($this->indexes[$id]["ldesc"]) {
        $isLDesc = true;
        return $this->indexes[$id]["ldesc"];
    } else {
        $isLDesc = false;
        return $this->indexes[$id]["sdesc"];
    }
}
final public function getShortDescription($id, &$isSDesc = null) {
    if ($this->indexes[$id]["sdesc"]) {
        $isSDesc = true;
        return $this->indexes[$id]["sdesc"];
    } else {
        $isSDesc = false;
        return $this->indexes[$id]["ldesc"];
    }
}

Problem

The current search index JSON generation doesn't utilize this fallback
mechanism, resulting in missing entries in PHP.net search results.

Solution

This PR addresses the issue by:

• Using the long description as the title when the short description is empty.
• Using the parent book title as the long description in such cases.
• Ignoring non-page entries (non-chunk entries) when generating the search index JSON.

Impact

• Improves search result completeness and accuracy.
• Reduces search index size by excluding irrelevant entries.

Examples

Currently, "PHP Manual > Language Reference > Types > String" is missing from search results due to an empty short description. This change will use "Strings" (the long description) as the title and "Language Reference" (the parent book title) as the long description.

search-index.json

 [
-  "",
+  "Strings",
   "language.types.string",
   "sect1"
 ],
-[
-  "",
-  "language.types.string",
-  "sect2"
-],

...
  [
-   "",
+   "Enumerations",
    "language.types.enumerations",
    "sect1"
  ],

search-description.json

-  "language.types.string": "Strings",
+  "language.types.string": "Language Reference",
...
-  "language.types.enumerations": "Enumerations",
+  "language.types.enumerations": "Language Reference",

Statistics

stat	before	after	change
Entries count	29,765	11,256	-62%
Entries lacking title (sdesc)	19,955	0	-100%
Search index size	1,383KB	693KB	-50%

Generated Search Index Diff Preview

Only the first 100 lines are shown for brevity.

 [
   [
-    "",
+    "Copyright",
     "copyright",
     "legalnotice"
   ],
   [
-    "",
-    "index",
-    "info"
-  ],
-  [
-    "",
-    "index",
-    "book"
-  ],
-  [
-    "",
-    "preface",
-    "section"
-  ],
-  [
-    "",
+    "Preface",
     "preface",
     "preface"
   ],
   [
-    "",
-    "intro-whatis",
-    "example"
-  ],
-  [
-    "",
+    "What is PHP?",
     "intro-whatis",
     "section"
   ],
   [
-    "",
+    "What can PHP do?",
     "intro-whatcando",
     "section"
   ],
   [
-    "",
+    "Introduction",
     "introduction",
     "chapter"
   ],
   [
-    "",
+    "What do I need?",
     "tutorial.requirements",
     "section"
   ],
   [
-    "",
-    "tutorial.firstpage",
-    "example"
-  ],
-  [
-    "",
-    "tutorial.firstpage",
-    "example"
-  ],
-  [
-    "",
+    "Your first PHP-enabled page",
     "tutorial.firstpage",
     "section"
   ],
   [
-    "",
-    "tutorial.useful",
-    "example"
-  ],
-  [
-    "",
-    "tutorial.useful",
-    "example"
-  ],
-  [
-    "",
-    "tutorial.useful",
-    "example"
-  ],
-  [
-    "",
+    "Something Useful",
     "tutorial.useful",
     "section"
   ],
   [
-    "",
-    "tutorial.forms",
-    "example"
-  ],

[kamil-tekiela]

Will this break the search if we merge it now?

[lhsazevedo]

No. It is safe to merge as it doesn't alter the JSON structure.

[lhsazevedo]

Thank you!

[kamil-tekiela]

Just curious. You said it would not affect the current functionality, right? Does that mean you changed something in the other PR that will avail of this change?

[lhsazevedo]

I said that it wouldn't break the current search, but you can already see the improved results on php.net.

Unfortunately, the current UI hides them under the last result group ("Other Matches"). You'll probably need to scroll down the menu to see it. You may also need to clear you local storage because the search index is cached for two weeks.

Try searching for "syntax", "types" or "operators". You should see some language reference results under "Other Matches". Those pages were missing from the index before because they don't have an sdesc.