With v4.5.0 web ui loads infinitely #5476

p5n · 2023-09-15T15:44:05Z

Steps to reproduce

nextcloud-app-calendar (1:4.4.5-1 -> 1:4.5.0-1)
try open calendar app

Expected behavior

Calendar opened

Actual behaviour

It never loads, database executes selects from oc_calendarobjects joining on oc_calendarobjects_props for 3 minutes

I have ~15 000 000 rows in calendarobjects-props and 1 500 000 in oc_schedulingobjects

~300 users has ~5-15 shared calendars

But for some users it loads successfully, cannot understand the difference

It surely regression because of rolling back to 4.4.5 fixes the problem

Calendar app version

4.5.0

CalDAV-clients used

firefox or any web browser

Browser

firefox 117.0.1

Client operating system

Arch Linux

Server operating system

Arch Linux

Web server

Apache

Database engine version

PostgreSQL

PHP engine version

PHP 8.1

Nextcloud version

27.0.2-1

Updated from an older installed version or fresh install

Updated from an older version

List of activated apps

Enabled:
  - activity: 2.19.0
  - admin_audit: 1.17.0
  - bookmarks: 13.1.1
  - calendar: 4.4.5
  - calendar_resource_management: 0.5.0
  - circles: 27.0.1
  - cloud_federation_api: 1.10.0
  - comments: 1.17.0
  - contacts: 5.4.0
  - contactsinteraction: 1.8.0
  - dashboard: 7.7.0
  - dav: 1.27.0
  - deck: 1.10.0
  - end_to_end_encryption: 1.13.1
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_external: 1.19.0
  - files_markdown: 2.3.6
  - files_pdfviewer: 2.8.0
  - files_rightclick: 1.6.0
  - files_sharing: 1.19.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - forms: 3.3.0
  - impersonate: 1.14.0
  - keeweb: 0.6.13
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - notes: 4.8.1
  - notifications: 2.15.0
  - notify_push: 0.6.3
  - oauth2: 1.15.1
  - onlyoffice: 8.1.0
  - photos: 2.3.0
  - polls: 3.4.1
  - previewgenerator: 5.1.0
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - related_resources: 1.2.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - spreed: 17.0.3
  - suspicious_login: 5.0.0
  - systemtags: 1.17.0
  - talk_matterbridge: 1.26.0
  - tasks: 0.15.0
  - text: 3.8.0
  - theming: 2.2.0
  - twofactor_backupcodes: 1.16.0
  - twofactor_totp: 9.0.0
  - user_ldap: 1.17.0
  - user_status: 1.7.0
  - viewer: 2.1.0
  - weather_status: 1.7.0
  - workflowengine: 2.9.0
Disabled:
  - bruteforcesettings: 2.7.0
  - encryption: 2.15.0 (installed 2.12.0)
  - firstrunwizard: 2.16.0 (installed 1.1)
  - mail: 3.2.5 (installed 3.2.1)
  - nextcloud_announcements: 1.16.0 (installed 1.1)
  - password_policy: 1.17.0 (installed 1.2.2)
  - recommendations: 1.6.0 (installed 0.5.0)
  - serverinfo: 1.17.0 (installed 1.7.0)
  - support: 1.10.0 (installed 1.0.1)
  - survey_client: 1.15.0 (installed 1.0.0)
  - updatenotification: 1.17.0 (installed 0.1.0)

Nextcloud configuration

{
    "system": {
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "27.0.2.1",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "loglevel": 2,
        "log_type": "file",
        "logfile": "\/var\/log\/nextcloud\/nextcloud.log",
        "log_query": false,
        "log_rotate_size": 268435456,
        "lastupdatedat": "1328756567.0597",
        "forcessl": true,
        "enablebackup": false,
        "maxZipInputSize": "20971520",
        "allowZipDownload": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "ldapIgnoreNamingRules": false,
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "defaultapp": "dashboard",
        "maintenance": false,
        "theme": "",
        "overwrite.cli.url": "https:\/\/cloud.dsr-corporation.com",
        "overwritehost": "cloud.dsr-corporation.com",
        "overwritewebroot": "\/",
        "trusted_domains": [
            "cloud.dsr-corporation.com"
        ],
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "appstoreenabled": false,
        "secret": "***REMOVED SENSITIVE VALUE***",
        "debug": false,
        "filelocking.enabled": true,
        "trashbin_retention_obligation": "disabled",
        "versions_retention_obligation": "disabled",
        "htaccess.RewriteBase": "\/",
        "ldapProviderFactory": "\\OCA\\User_LDAP\\LDAPProviderFactory",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "sendmail",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauthtype": "LOGIN",
        "simpleSignUpLink.shown": false,
        "onlyoffice": {
            "verify_peer_off": true
        },
        "app_install_overwrite": [
            "files_markdown",
            "onlyoffice",
            "tasks",
            "rainloop",
            "social",
            "keeweb",
            "impersonate",
            "talk_matterbridge",
            "twofactor_totp",
            "polls",
            "calendar_resource_management",
            "end_to_end_encryption",
            "forms",
            "previewgenerator"
        ],
        "mysql.utf8mb4": true,
        "mail_sendmailmode": "pipe",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "default_phone_region": "US"
    }
}

Web server error log

No logs, it hangs on php side waiting for sql

Log file

it is too big but it looks nothing related to calendar

Browser log

also no log, it waits for GET /index.php/apps/calendar/

Additional info

No response

The text was updated successfully, but these errors were encountered:

ChristophWurst · 2023-09-15T15:54:51Z

@miaulalala could you have a look?

ChristophWurst · 2023-09-15T16:58:29Z

Probably related to #5161

kiilo · 2023-09-19T07:59:11Z

related to that, nextcloud calender webui doesn't load
postgresql process is going through the roof, chewing all CPU cycles

kiilo · 2023-09-19T10:35:17Z

just some notes

rolling back to a previous release is not feasible, I tried it
iCal / thunderbird still syncs and events could be edited
ca 30 users standing in front of my door with forks in their hand ...

tcitworld · 2023-09-19T11:27:41Z

@p5n @kiilo Are you able to test the following patch to make sure we identify the issue:

diff --git a/lib/Controller/ViewController.php b/lib/Controller/ViewController.php
index fde305e5..7f670266 100644
--- a/lib/Controller/ViewController.php
+++ b/lib/Controller/ViewController.php
@@ -143,7 +143,7 @@ class ViewController extends Controller {
 		$this->initialStateService->provideInitialState('appointmentConfigs', $this->appointmentConfigService->getAllAppointmentConfigurations($this->userId));
 		$this->initialStateService->provideInitialState('disable_appointments', $disableAppointments);
 		$this->initialStateService->provideInitialState('can_subscribe_link', $canSubscribeLink);
-		$this->initialStateService->provideInitialState('categories', $this->categoriesService->getCategories($this->userId));
+		$this->initialStateService->provideInitialState('categories', []);
 		$this->initialStateService->provideInitialState('show_resources', $showResources);
 
 		return new TemplateResponse($this->appName, 'main');

The full path to file to edit is apps/calendar/lib/Controller/ViewController.php

kiilo · 2023-09-19T12:11:24Z

Yep that did the trick!!!

thanks a ton

miaulalala · 2023-09-19T13:36:43Z

Both are using PostgreSQL - is there anything specific about PostgreSQL that would cause these issues?

flawlessweb · 2023-09-21T02:17:06Z

I'm not using PosgresSQL and I have the same issue.

kiilo · 2023-09-22T09:47:14Z

This issue still exists with v4.5.1
Same workaround resolves temporarily, just no tags for calendar.

stefangweichinger · 2023-09-25T07:06:26Z

Upgraded to nextcloud-26.0.7, checked apps/calendar/lib/Controller/ViewController.php, patch is not there (expected), but Calendar seems to work in GUI now. I keep it this way for now.

miaulalala · 2023-09-25T13:10:33Z

@p5n @kiilo Are you able to test the following patch to make sure we identify the issue:

diff --git a/lib/Controller/ViewController.php b/lib/Controller/ViewController.php
index fde305e5..7f670266 100644
--- a/lib/Controller/ViewController.php
+++ b/lib/Controller/ViewController.php
@@ -143,7 +143,7 @@ class ViewController extends Controller {
 		$this->initialStateService->provideInitialState('appointmentConfigs', $this->appointmentConfigService->getAllAppointmentConfigurations($this->userId));
 		$this->initialStateService->provideInitialState('disable_appointments', $disableAppointments);
 		$this->initialStateService->provideInitialState('can_subscribe_link', $canSubscribeLink);
-		$this->initialStateService->provideInitialState('categories', $this->categoriesService->getCategories($this->userId));
+		$this->initialStateService->provideInitialState('categories', []);
 		$this->initialStateService->provideInitialState('show_resources', $showResources);
 
 		return new TemplateResponse($this->appName, 'main');

The full path to file to edit is apps/calendar/lib/Controller/ViewController.php

Shall we add the changes for 4.5.2 and disable the category feature this way?

cm-schl · 2023-09-26T12:38:56Z

Had the same problem with mariadb (error 500 like in #5479) - the change solved the problem, thank you 👍

p5n · 2023-09-26T15:36:40Z

Can I switch off something else to reduce cpu load?)

apachetop and pgtop looks more or less reasonable (~100 req per sec), but loopback busy by postgres, it is about 10Mb per sql query. More than 1Gb/s total.

bwm-ng

|         iface                   Rx                   Tx                Total
  ==============================================================================
               lo:     1119557.62 KB/s      1119557.62 KB/s      2239115.25 KB/s

pgtop

 Queries: 3.0G   qps: 231180613 Slow:    1.3G         Se/In/Up/De(%):    411/03/01/03 
             qps now:   89 Slow qps: 34.2  Threads:   11 (  16/   0) 169/00/01/00 
 Key Efficiency: 100.0%  Bps in/out:   0.0/  0.0   Now in/out:   0.0/  0.0

apachetop

All:           72 reqs (   3.8/sec)         18.4M (  993.5K/sec)     262.2K/req
2xx:      39 (54.9%) 3xx:       2 ( 2.8%) 4xx:    30 (42.3%) 5xx:     0 ( 0.0%)
R ( 20s):      72 reqs (   3.6/sec)         18.4M (  943.8K/sec)     262.2K/req
2xx:      39 (54.9%) 3xx:       2 ( 2.8%) 4xx:    30 (42.3%) 5xx:     0 ( 0.0%)

visually near half of requests are to /remote.php/dav/calendars/username/inbox/

p5n added bug 0. to triage Pending approval or rejection labels Sep 15, 2023

ChristophWurst added the regression Regression of a previous working feature label Sep 15, 2023

ChristophWurst added this to 💌 📅 👥 Groupware team Sep 15, 2023

ChristophWurst moved this to 📄 To do (~10 entries) in 💌 📅 👥 Groupware team Sep 15, 2023

ChristophWurst mentioned this issue Sep 19, 2023

Memory consumption since upgrade to 25.0.11 #5479

Closed

ChristophWurst mentioned this issue Sep 28, 2023

revert: Augment the category menu by system tags and already used categories #5498

Merged

miaulalala mentioned this issue Oct 2, 2023

Revert "revert: Augment the category menu by system tags and already used categories" #5509

Open

miaulalala self-assigned this Oct 2, 2023

miaulalala added the 2. developing Work in progress label Oct 2, 2023

miaulalala moved this from 📄 To do (~10 entries) to 🏗️ In progress in 💌 📅 👥 Groupware team Oct 2, 2023

miaulalala removed the 0. to triage Pending approval or rejection label Oct 2, 2023

miaulalala linked a pull request Oct 2, 2023 that will close this issue

revert: Augment the category menu by system tags and already used categories #5498

Merged

miaulalala closed this as completed in #5498 Oct 2, 2023

github-project-automation bot moved this from 🏗️ In progress to ☑️ Done in 💌 📅 👥 Groupware team Oct 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

With v4.5.0 web ui loads infinitely #5476

With v4.5.0 web ui loads infinitely #5476

p5n commented Sep 15, 2023

ChristophWurst commented Sep 15, 2023

ChristophWurst commented Sep 15, 2023

kiilo commented Sep 19, 2023

kiilo commented Sep 19, 2023 •

edited

Loading

tcitworld commented Sep 19, 2023 •

edited

Loading

kiilo commented Sep 19, 2023

miaulalala commented Sep 19, 2023

flawlessweb commented Sep 21, 2023

kiilo commented Sep 22, 2023

stefangweichinger commented Sep 25, 2023

miaulalala commented Sep 25, 2023

cm-schl commented Sep 26, 2023

p5n commented Sep 26, 2023

With v4.5.0 web ui loads infinitely #5476

With v4.5.0 web ui loads infinitely #5476

Comments

p5n commented Sep 15, 2023

Steps to reproduce

Expected behavior

Actual behaviour

Calendar app version

CalDAV-clients used

Browser

Client operating system

Server operating system

Web server

Database engine version

PHP engine version

Nextcloud version

Updated from an older installed version or fresh install

List of activated apps

Nextcloud configuration

Web server error log

Log file

Browser log

Additional info

ChristophWurst commented Sep 15, 2023

ChristophWurst commented Sep 15, 2023

kiilo commented Sep 19, 2023

kiilo commented Sep 19, 2023 • edited Loading

tcitworld commented Sep 19, 2023 • edited Loading

kiilo commented Sep 19, 2023

miaulalala commented Sep 19, 2023

flawlessweb commented Sep 21, 2023

kiilo commented Sep 22, 2023

stefangweichinger commented Sep 25, 2023

miaulalala commented Sep 25, 2023

cm-schl commented Sep 26, 2023

p5n commented Sep 26, 2023

kiilo commented Sep 19, 2023 •

edited

Loading

tcitworld commented Sep 19, 2023 •

edited

Loading