You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Whenever I start the conda-store services with docker compose up --build, I get integrity errors:
... <more startup stuff here>
conda-store-worker-1 | [2024-07-19 00:02:40,346: WARNING/ForkPoolWorker-4] [CondaStoreWorker] building conda_prefix=/var/lib/conda-store/filesystem/7ff0f78a-1721347282-1-python-flask-env took 77.196 [s]
conda-store-worker-1 | [2024-07-19 00:02:41,850: INFO/MainProcess] Task task_build_conda_env_export[build-1-conda-env-export] received
conda-store-worker-1 | [2024-07-19 00:02:41,852: INFO/MainProcess] Task task_build_conda_pack[build-1-conda-pack] received
conda-store-worker-1 | [2024-07-19 00:02:41,853: INFO/MainProcess] Task task_build_conda_docker[build-1-docker] received
conda-store-worker-1 | [2024-07-19 00:02:41,855: INFO/MainProcess] Task task_build_constructor_installer[build-1-constructor-installer] received
conda-store-worker-1 | [2024-07-19 00:02:41,855: INFO/ForkPoolWorker-4] Task task_build_conda_environment[build-1-environment] succeeded in 78.7562527419941s: None
conda-store-worker-1 | [2024-07-19 00:02:41,867: WARNING/ForkPoolWorker-4] /opt/conda-store-server/conda_store_server/_internal/build.py:415: UserWarning: Generating Docker images is currently not supported, see https://github.com/conda-incubator/conda-store/issues/666
conda-store-worker-1 | warnings.warn(
conda-store-worker-1 |
conda-store-worker-1 | [2024-07-19 00:02:41,868: INFO/ForkPoolWorker-4] Task task_build_conda_docker[build-1-docker] succeeded in 0.012042938004015014s: None
conda-store-worker-1 | [2024-07-19 00:02:41,899: WARNING/ForkPoolWorker-4] /opt/conda-store-server/conda_store_server/_internal/action/generate_constructor_installer.py:48: UserWarning: Installer generation requires constructor: https://github.com/conda/constructor
conda-store-worker-1 | warnings.warn(
conda-store-worker-1 |
conda-store-worker-1 | [2024-07-19 00:02:41,900: WARNING/ForkPoolWorker-4] [CondaStoreWorker] creating installer for conda environment=/var/lib/conda-store/filesystem/7ff0f78a-1721347282-1-python-flask-env took 0.018 [s]
conda-store-worker-1 | [2024-07-19 00:02:41,901: INFO/ForkPoolWorker-4] Task task_build_constructor_installer[build-1-constructor-installer] succeeded in 0.03257362500880845s: None
conda-store-worker-1 | [2024-07-19 00:02:43,334: INFO/ForkPoolWorker-2] Task task_build_conda_env_export[build-1-conda-env-export] succeeded in 1.4822059069992974s: None
conda-store-worker-1 | [2024-07-19 00:02:47,599: WARNING/ForkPoolWorker-3] [CondaStoreWorker] packaging archive of conda environment=/var/lib/conda-store/filesystem/7ff0f78a-1721347282-1-python-flask-env took 5.731 [s]
conda-store-worker-1 | [2024-07-19 00:02:47,604: INFO/ForkPoolWorker-3] Task task_build_conda_pack[build-1-conda-pack] succeeded in 5.751906952005811s: None
conda-store-worker-1 | [2024-07-19 00:02:47,605: INFO/MainProcess] Task task_update_storage_metrics[build-1] received
conda-store-worker-1 | [2024-07-19 00:02:47,628: INFO/ForkPoolWorker-2] Task task_update_storage_metrics[build-1] succeeded in 0.02287509001325816s: None
conda-store-server-1 | INFO: 127.0.0.1:38788 - "GET /conda-store/api/v1/ HTTP/1.1" 200 OK
conda-store-server-1 | INFO: 127.0.0.1:58982 - "GET /conda-store/api/v1/ HTTP/1.1" 200 OK
conda-store-worker-1 | [2024-07-19 00:03:01,985: INFO/ForkPoolWorker-5] packages to insert : 64960
postgres-1 | 2024-07-19 00:03:01.989 UTC [81] ERROR: duplicate key value violates unique constraint "_conda_package_uc"
postgres-1 | 2024-07-19 00:03:01.989 UTC [81] DETAIL: Key (channel_id, name, version)=(2, _libgcc_mutex, 0.1) already exists.
postgres-1 | 2024-07-19 00:03:01.989 UTC [81] STATEMENT: INSERT INTO conda_package (channel_id, license, name, version, summary) VALUES (2, 'None', '_libgcc_mutex', '0.1', 'Mutex for libgcc and libgcc-ng') RETURNING conda_package.id
conda-store-worker-1 | [2024-07-19 00:03:01,991: WARNING/ForkPoolWorker-5] (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "_conda_package_uc"
conda-store-worker-1 | DETAIL: Key (channel_id, name, version)=(2, _libgcc_mutex, 0.1) already exists.
conda-store-worker-1 |
conda-store-worker-1 | [SQL: INSERT INTO conda_package (channel_id, license, name, version, summary) VALUES (%(channel_id)s, %(license)s, %(name)s, %(version)s, %(summary)s) RETURNING conda_package.id]
conda-store-worker-1 | [parameters: {'channel_id': 2, 'license': 'None', 'name': '_libgcc_mutex', 'version': '0.1', 'summary': 'Mutex for libgcc and libgcc-ng'}]
conda-store-worker-1 | (Background on this error at: https://sqlalche.me/e/14/gkpj)
conda-store-worker-1 | [2024-07-19 00:03:01,996: ERROR/ForkPoolWorker-5] Task task_update_conda_channel[b07f7b56-0f96-43fb-a27b-65afaf22b1c6] raised unexpected: IntegrityError('(psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "_conda_package_uc"\nDETAIL: Key (channel_id, name, version)=(2, _libgcc_mutex, 0.1) already exists.\n')
conda-store-worker-1 | Traceback (most recent call last):
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
conda-store-worker-1 | self.dialect.do_execute(
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
conda-store-worker-1 | cursor.execute(statement, parameters)
conda-store-worker-1 | psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "_conda_package_uc"
conda-store-worker-1 | DETAIL: Key (channel_id, name, version)=(2, _libgcc_mutex, 0.1) already exists.
conda-store-worker-1 |
conda-store-worker-1 |
conda-store-worker-1 | The above exception was the direct cause of the following exception:
conda-store-worker-1 |
conda-store-worker-1 | Traceback (most recent call last):
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/celery/app/trace.py", line 453, in trace_task
conda-store-worker-1 | R = retval = fun(*args, **kwargs)
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/celery/app/trace.py", line 736, in __protected_call__
conda-store-worker-1 | return self.run(*args, **kwargs)
conda-store-worker-1 | File "/opt/conda-store-server/conda_store_server/_internal/worker/tasks.py", line 188, in task_update_conda_channel
conda-store-worker-1 | channel.update_packages(db, subdirs=settings.conda_platforms)
conda-store-worker-1 | File "/opt/conda-store-server/conda_store_server/_internal/orm.py", line 569, in update_packages
conda-store-worker-1 | raise e
conda-store-worker-1 | File "/opt/conda-store-server/conda_store_server/_internal/orm.py", line 564, in update_packages
conda-store-worker-1 | db.bulk_insert_mappings(CondaPackage, packages.values())
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3810, in bulk_insert_mappings
conda-store-worker-1 | self._bulk_save_mappings(
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3912, in _bulk_save_mappings
conda-store-worker-1 | with util.safe_reraise():
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
conda-store-worker-1 | compat.raise_(
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
conda-store-worker-1 | raise exception
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3901, in _bulk_save_mappings
conda-store-worker-1 | persistence._bulk_insert(
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/orm/persistence.py", line 107, in _bulk_insert
conda-store-worker-1 | _emit_insert_statements(
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements
conda-store-worker-1 | c = connection._execute_20(
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1710, in _execute_20
conda-store-worker-1 | return meth(self, args_10style, kwargs_10style, execution_options)
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
conda-store-worker-1 | return connection._execute_clauseelement(
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1577, in _execute_clauseelement
conda-store-worker-1 | ret = self._execute_context(
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1953, in _execute_context
conda-store-worker-1 | self._handle_dbapi_exception(
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2134, in _handle_dbapi_exception
conda-store-worker-1 | util.raise_(
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
conda-store-worker-1 | raise exception
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
conda-store-worker-1 | self.dialect.do_execute(
conda-store-worker-1 | File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
conda-store-worker-1 | cursor.execute(statement, parameters)
conda-store-worker-1 | sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "_conda_package_uc"
conda-store-worker-1 | DETAIL: Key (channel_id, name, version)=(2, _libgcc_mutex, 0.1) already exists.
conda-store-worker-1 |
conda-store-worker-1 | [SQL: INSERT INTO conda_package (channel_id, license, name, version, summary) VALUES (%(channel_id)s, %(license)s, %(name)s, %(version)s, %(summary)s) RETURNING conda_package.id]
conda-store-worker-1 | [parameters: {'channel_id': 2, 'license': 'None', 'name': '_libgcc_mutex', 'version': '0.1', 'summary': 'Mutex for libgcc and libgcc-ng'}]
conda-store-worker-1 | (Background on this error at: https://sqlalche.me/e/14/gkpj)
Even with fresh containers, this error always appears. It doesn't seem to affect anything, but getting error messages immediately upon starting up isn't helpful.
Expected behavior
No error appears.
How to Reproduce the problem?
docker compose up --build. I'm running Linux 6.6.39-1-lts x86_64.
I was looking into this issue some but was unable to reproduce it from a clean clone of the repo and running docker compose up --build. Thinking out loud for a bit here:
Digging into the error a bit more, I notice these lines in the debug output:
conda-store-worker-1 | File "/opt/conda-store-server/conda_store_server/_internal/worker/tasks.py", line 188, in task_update_conda_channel
The task_update_conda_channel is defined here. This should be looking at the repodata for each configured channel and inserting any new packages into the postgres db.
conda-store-worker-1 | [2024-07-19 00:03:01,985: INFO/ForkPoolWorker-5] packages to insert : 64960
This line I believe comes from the package updater. It's going to try to insert a bunch of packages, which makes sense as this is the first run and the database should be empty. Though, this seems like maybe a small amount if the main channel is included in your config.
conda-store-worker-1 | File "/opt/conda-store-server/conda_store_server/_internal/orm.py", line 569, in update_packages
conda-store-worker-1 | raise e
conda-store-worker-1 | File "/opt/conda-store-server/conda_store_server/_internal/orm.py", line 564, in update_packages
conda-store-worker-1 | db.bulk_insert_mappings(CondaPackage, packages.values())
This part is the error coming from the orm package updater. There is this little piece of logic here looks like it should be stopping any situation where duplicates get inserted into the database. Looks like this in particular is failing.
@peytondmurray in order to continue debugging this problem can you share:
the conda_platform setting you have (if modified from the default)
the list of channels you have enabled (if modified from the default)
Describe the bug
Whenever I start the conda-store services with
docker compose up --build
, I get integrity errors:Even with fresh containers, this error always appears. It doesn't seem to affect anything, but getting error messages immediately upon starting up isn't helpful.
Expected behavior
No error appears.
How to Reproduce the problem?
docker compose up --build
. I'm running Linux 6.6.39-1-lts x86_64.Here's the output of
pip list
Here's the output of
python -VV
:Python 3.10.0 | packaged by conda-forge | (default, Nov 20 2021, 02:24:10) [GCC 9.4.0]
The text was updated successfully, but these errors were encountered: