Analytics: Create Collectors Layer

[jdotcms]

Parent Issue

#29476

User Story

As part of the Analytics infrastructure; we need to create a collectors layer
We can think on two kind collectors:

1. syn collectors; which are in charge of collect information from the request, session, app and so on but in the same request thread scope
2. async collectors; which are in charge of collect information from session, app and so on but in a separated thread, the intention behind this is to avoid to delay the current request thread execution by doing cpu times by gathering information
   In addition the clients might add collectors programatically by OSGI

Acceptance Criteria

We need to create a collector to gather information for the following events:

• PAGE_REQUEST: Triggered when a page is accessed.

• FILE_REQUEST: Triggered when any file is accessed. If a page is accessed and requests a file (e.g., an image, JS, CSS, etc.), two events will be triggered: one for the page as a PAGE_REQUEST and another for each file as a FILE_REQUEST. Files can be loaded using the following endpoints:

  • /dA/... Endpoint. More info here.
  • /contentAsset Endpoint. More info here.
  • /dotAsset Endpoint. More info here.
  • By directly using the file's URI in the URL.

• URL_MAP: Triggered when a URL map is used. In this case, two events are triggered: one for the URL_MAP and another for PAGE_REQUEST (the second one corresponds to the detailed page request).

• VANITY_REQUEST: Triggered when a vanity URL is used. If the vanity URL forwards internally, two events will be triggered: the first one as a VANITY_REQUEST, and the second one could be either a FILE_REQUEST or a PAGE_REQUEST, depending on the forward destination.

Proposed Objective

Core Features

Proposed Priority

Priority 3 - Average

External Links... Slack Conversations, Support Tickets, Figma Designs, etc.

No response

Assumptions & Initiation Needs

No response

Quality Assurance Notes & Workarounds

No response

Sub-Tasks & Estimates

No response

[wezell]

Just to be clear, we have these already. See:

https://github.com/dotCMS/core/tree/master/dotCMS/src/main/java/com/dotcms/visitor/filter/characteristics

These build a map of information from the request based on the user, session, etc.

[jdotcms]

PR

[freddyDOTCMS]

Note to QA: it is better test this issue together with #29862, if you want to test it alone then you are going to need to run the query directly in Clickhouse doing together allow you to run the query using CubeJS instead.

Steps to test (both issues together):

• Start dotCMS with the full starter
• Start Analytics to this just: Go to ""/core/docker/docker-compose-examples/analytics"" and run the command "docker-compose up", and wait a couple of minutes, Configure the Analitycs App in dotCMS with the follow parameters:

ClientIId: analytics-customer-customer1
Client Secret: testsecret
Analytics Config URL http://localhost:8088/c/customer1/cluster1/keys
Analytics Write URL http://localhost:8081/api/v1/event
Analytics Read URL http://localhost:4001/

• Run this script in the CH Database Create REQUEST DataModel in CubeJS Schema  #29862 (comment)
• Navigate by the dotCMS FE.
• Start cubsJS UI in "localhost:4001" and play around with it

[freddyDOTCMS]

Pass Internal QA: The CA data for PAGE, FILE, URL MAP and Vanity URL are been collecting as expected

We just have some problems collecting the referer, user-agent and persona field but we already work on this in another issue:

• Fix/Improve some stuff in CA #30179

[josemejias11]

Approved: Tested on trunk_a12617a, Docker, macOS 14.5, FF v126.0.1