no such table: packages

[dlangille]

This error is intermittent. It occurs a few times a week. Last night it occurred on 12 ports

I've been running with export DEBUG_LEVEL=4 in poudriere.conf.

An example from today (full log)

===>   adwaita-icon-theme-42.0 depends on file: /usr/local/sbin/pkg - not found
===>   Installing existing package /packages/All/pkg-1.21.3.pkg
DBG(1)[83564]> PkgConfig: loading repositories in /etc/pkg/
DBG(1)[83564]> PKgConfig: loading /etc/pkg//FreeBSD.conf
DBG(1)[83564]> PkgConfig: parsing key 'FreeBSD'
DBG(1)[83564]> PkgConfig: parsing repository object FreeBSD
DBG(1)[83564]> PkgConfig: loading repositories in /usr/local/etc/pkg/repos/
DBG(4)[83564]> Pkgdb: running: 'PRAGMA user_version;'
DBG(4)[83564]> Pkgdb: running: 'BEGIN IMMEDIATE TRANSACTION '
DBG(4)[83564]> Pkgdb: executing 'CREATE TABLE licenses (id INTEGER PRIMARY KEY, license TEXT NOT NULL UNIQUE );CREATE TABLE pkg_licenses_assoc (package_id INTEGER REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE, license_id INTEGER REFERENCES licenses(id) ON DELETE RESTRICT ON UPDATE RESTRICT, PRIMARY KEY (package_id, license_id));CREATE VIEW pkg_licenses AS SELECT origin, license FROM packages INNER JOIN pkg_licenses_assoc ON packages.id = pkg_licenses_assoc.package_id INNER JOIN licenses ON pkg_licenses_assoc.license_id = licenses.id;CREATE TRIGGER license_insert INSTEAD OF INSERT ON pkg_licenses FOR EACH ROW BEGIN INSERT OR IGNORE INTO licenses(license) values (NEW.license);INSERT INTO pkg_licenses_assoc(package_id, license_id) VALUES ((SELECT id FROM packages where origin = NEW.origin), (SELECT id FROM categories WHERE name = NEW.name));END;'
DBG(4)[83564]> Pkgdb: executing 'PRAGMA user_version = 1;'
DBG(4)[83564]> Pkgdb: running: 'COMMIT TRANSACTION '
DBG(4)[83564]> Pkgdb: running: 'BEGIN IMMEDIATE TRANSACTION '
DBG(4)[83564]> Pkgdb: executing 'ALTER TABLE packages ADD licenselogic INTEGER NOT NULL DEFAULT(1);'
pkg-static: sqlite error while executing ALTER TABLE packages ADD licenselogic INTEGER NOT NULL DEFAULT(1); in file pkgdb.c:2353: no such table: packages
DBG(4)[83564]> Pkgdb: running: 'ROLLBACK TRANSACTION '
*** Error code 1

Stop.
make: stopped in /usr/ports/x11-themes/adwaita-icon-theme
=>> Cleaning up wrkdir
===>  Cleaning for adwaita-icon-theme-42.0
build of x11-themes/adwaita-icon-theme | adwaita-icon-theme-42.0 ended at Tue Jun 11 04:21:46 UTC 2024
build time: 00:00:04
!!! build failure encountered !!!

This build contains the 12 failures: https://services.unixathome.org/poudriere/build.html?mastername=140amd64-default-primary&build=2024-06-11_04h18m17s

[dlangille]

From https://services.unixathome.org/poudriere/data/140amd64-default-primary/2024-06-13_04h18m18s/logs/errors/qt6-tools-6.7.1.log

===>   qt6-tools-6.7.1 depends on file: /usr/local/sbin/pkg - not found
===>   Installing existing package /packages/All/pkg-1.21.3.pkg
DBG(1)[80596]> PkgConfig: loading repositories in /etc/pkg/
DBG(1)[80596]> PKgConfig: loading /etc/pkg//FreeBSD.conf
DBG(1)[80596]> PkgConfig: parsing key 'FreeBSD'
DBG(1)[80596]> PkgConfig: parsing repository object FreeBSD
DBG(1)[80596]> PkgConfig: loading repositories in /usr/local/etc/pkg/repos/
DBG(4)[80596]> Pkgdb: running: 'PRAGMA user_version;'
DBG(4)[80596]> Pkgdb: running: 'BEGIN IMMEDIATE TRANSACTION '
DBG(4)[80596]> Pkgdb: executing 'CREATE TABLE licenses (id INTEGER PRIMARY KEY, license TEXT NOT NULL UNIQUE );CREATE TABLE pkg_licenses_assoc (package_id INTEGER REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE, license_id INTEGER REFERENCES licenses(id) ON DELETE RESTRICT ON UPDATE RESTRICT, PRIMARY KEY (package_id, license_id));CREATE VIEW pkg_licenses AS SELECT origin, license FROM packages INNER JOIN pkg_licenses_assoc ON packages.id = pkg_licenses_assoc.package_id INNER JOIN licenses ON pkg_licenses_assoc.license_id = licenses.id;CREATE TRIGGER license_insert INSTEAD OF INSERT ON pkg_licenses FOR EACH ROW BEGIN INSERT OR IGNORE INTO licenses(license) values (NEW.license);INSERT INTO pkg_licenses_assoc(package_id, license_id) VALUES ((SELECT id FROM packages where origin = NEW.origin), (SELECT id FROM categories WHERE name = NEW.name));END;'
DBG(4)[80596]> Pkgdb: executing 'PRAGMA user_version = 1;'
DBG(4)[80596]> Pkgdb: running: 'COMMIT TRANSACTION '
DBG(4)[80596]> Pkgdb: running: 'BEGIN IMMEDIATE TRANSACTION '
DBG(4)[80596]> Pkgdb: executing 'ALTER TABLE packages ADD licenselogic INTEGER NOT NULL DEFAULT(1);'
pkg-static: sqlite error while executing ALTER TABLE packages ADD licenselogic INTEGER NOT NULL DEFAULT(1); in file pkgdb.c:2353: no such table: packages
DBG(4)[80596]> Pkgdb: running: 'ROLLBACK TRANSACTION '

[dlangille]

Running the build again, succeeds: https://services.unixathome.org/poudriere/data/140amd64-default-primary/2024-06-13_11h23m18s/logs/qt6-tools-6.7.1.log

When comparing the logs, the first differences occur after: ===> Installing existing package /packages/All/pkg-1.21.3.pkg

success:

===>   Installing existing package /packages/All/pkg-1.21.3.pkg
DBG(1)[86626]> PkgConfig: loading repositories in /etc/pkg/
DBG(1)[86626]> PKgConfig: loading /etc/pkg//FreeBSD.conf
DBG(1)[86626]> PkgConfig: parsing key 'FreeBSD'
DBG(1)[86626]> PkgConfig: parsing repository object FreeBSD
DBG(1)[86626]> PkgConfig: loading repositories in /usr/local/etc/pkg/repos/
DBG(4)[86626]> Pkgdb: executing 'PRAGMA journal_mode = TRUNCATE;PRAGMA synchronous = FULL;BEGIN;CREATE TABLE packages (id INTEGER PRIMARY KEY,origin TEXT ....

failure:

===>   Installing existing package /packages/All/pkg-1.21.3.pkg
DBG(1)[80596]> PkgConfig: loading repositories in /etc/pkg/
DBG(1)[80596]> PKgConfig: loading /etc/pkg//FreeBSD.conf
DBG(1)[80596]> PkgConfig: parsing key 'FreeBSD'
DBG(1)[80596]> PkgConfig: parsing repository object FreeBSD
DBG(1)[80596]> PkgConfig: loading repositories in /usr/local/etc/pkg/repos/
DBG(4)[80596]> Pkgdb: running: 'PRAGMA user_version;'
DBG(4)[80596]> Pkgdb: running: 'BEGIN IMMEDIATE TRANSACTION '
DBG(4)[80596]> Pkgdb: executing 'CREATE TABLE licenses (id INTEGER ....

That might be it. FYI, each of the 12 failures listed in the first comment of this issue contain "CREATE TABLE licenses".

Each of the five successful builds I checked: start with 'CREATE TABLE packages'

[dlangille]

Three more examples: https://services.unixathome.org/poudriere/build.html?mastername=140amd64-default-primary&build=2024-06-14_04h18m18s

[dlangille]

Adding more in case it helps find the pattern. Last night 15 packages failed to build with that error.

https://services.unixathome.org/poudriere/build.html?mastername=141amd64-default-primary&build=2024-06-20_04h21m07s

The failures were in the time frame UTC 04:39:26 - 04:44:41, with many at 04:44:25

[dlangille]

One failure: https://services.unixathome.org/poudriere/data/140amd64-default-primary/2024-06-28_04h18m15s/logs/errors/librenms-24.6.0,1.log

[dlangille]

8 failures: https://services.unixathome.org/poudriere/build.html?mastername=141amd64-default-primary&build=2024-07-03_04h18m13s