[ci] replace uses of 'mamba' with 'conda', use Python 12 for test-python-latest-job

[jameslamb]

Trying to fix #6539 (comment)

Starting with the 23.10 release of conda (about a year ago), the conda executable now uses the mamba solver by default... so there's no need to invoke the mamba executable any more:

• https://docs.conda.io/projects/conda/en/latest/release-notes.html#id55
• https://conda.org/blog/2023-11-06-conda-23-10-0-release/

We should be able to use conda {command} instead of mamba {command} without any loss of performance now, and I'm hoping it will fix the issue with mamba reported in the linked comment above.

Notes for Reviewers

Searched for all uses of mamba like this:

git grep 'mamba'

[jameslamb]

While testing this, I encountered another unrelated error on the test-python-latest job:

ERROR: Ignored the following yanked versions: 1.11.0, 1.14.0rc1
ERROR: Ignored the following versions that require a different python version: 1.21.2 Requires-Python >=3.7,<3.11; 1.21.3 Requires-Python >=3.7,<3.11; 1.21.4 Requires-Python >=3.7,<3.11; 1.21.5 Requires-Python >=3.7,<3.11; 1.21.6 Requires-Python >=3.7,<3.11; 1.6.2 Requires-Python >=3.7,<3.10; 1.6.3 Requires-Python >=3.7,<3.10; 1.7.0 Requires-Python >=3.7,<3.10; 1.7.1 Requires-Python >=3.7,<3.10; 1.7.2 Requires-Python >=3.7,<3.11; 1.7.3 Requires-Python >=3.7,<3.11; 1.8.0 Requires-Python >=3.8,<3.11; 1.8.0rc1 Requires-Python >=3.8,<3.11; 1.8.0rc2 Requires-Python >=3.8,<3.11; 1.8.0rc3 Requires-Python >=3.8,<3.11; 1.8.0rc4 Requires-Python >=3.8,<3.11; 1.8.1 Requires-Python >=3.8,<3.11
ERROR: Could not find a version that satisfies the requirement scipy>=1.15.0.dev0 (from versions: 0.8.0, 0.9.0, 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.12.1, 0.13.0, 0.13.1, 0.13.2, 0.13.3, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.16.0, 0.16.1, 0.17.0, 0.17.1, 0.18.0, 0.18.1, 0.19.0, 0.19.1, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.4.0, 1.4.1, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 1.5.4, 1.6.0, 1.6.1, 1.9.0rc1, 1.9.0rc2, 1.9.0rc3, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0rc1, 1.10.0rc2, 1.10.0, 1.10.1, 1.11.0rc1, 1.11.0rc2, 1.11.1, 1.11.2, 1.11.3, 1.11.4, 1.12.0rc1, 1.12.0rc2, 1.12.0, 1.13.0rc1, 1.13.0, 1.13.1, 1.14.0rc2, 1.14.0, 1.14.1)
ERROR: No matching distribution found for scipy>=1.15.0.dev0

Notice: A new release of pip is available: 24.0 -> 24.2
Notice: To update, run: pip install --upgrade pip

Looks to me that https://pypi.anaconda.org/scientific-python-nightly-wheels/simple/scipy/ does not have Python 3.11 manylinux x86_64 wheels.

Using Python 3.12 for that job should fix that... and is probably good anyway, since (for 2 more days) it's the latest released stable Python version.

[jameslamb]

Even with the Python version changes, I think something is just wrong with the scipy nightly wheels.

Collecting scipy>=1.15.0.dev0
ERROR: HTTP error 404 while getting https://pypi.anaconda.org/scientific-python-nightly-wheels/simple/scipy/1.15.0.dev0/scipy-1.15.0.dev0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (from https://pypi.anaconda.org/scientific-python-nightly-wheels/simple/scipy/)
ERROR: Could not install requirement scipy>=1.15.0.dev0 from https://pypi.anaconda.org/scientific-python-nightly-wheels/simple/scipy/1.15.0.dev0/scipy-1.15.0.dev0-cp312-cp312-manylinux_2_17_x[86](https://github.com/microsoft/LightGBM/actions/runs/11190881568/job/31113194716?pr=6663#step:4:87)_64.manylinux2014_x86_64.whl because of HTTP error 404 Client Error: Not Found for url: https://binstar-cio-packages-prod.s3.amazonaws.com/64774d9554a6f0c49626f52d/66fe64d51378ca9be9b3503c?response-content-disposition=attachment%3B%20filename%3D%22scipy-1.15.0.dev0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl%22%3B%20filename%2A%3DUTF-8%27%27scipy-1.15.0.dev0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl&response-content-type=application%2Foctet-stream&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAWUI46DZFLX5KC5CH%2F20241005%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241005T055925Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEJ7%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJHMEUCIQCbj0CPSobDdugrusKNCmxy6T0KdyUoymt4uAcSpK51NQIgfwIgBkqyhw6q7%2BWI90e

(build link)

[jameslamb]

It does look like switching from mamba to conda resolves the main CI issue this PR is trying to address 🎉

Upgrading to Python 3.12 did not help with the scipy wheels issue... I've reported it over at scipy/scipy#21623 (comment).

If that isn't resolved in a day or 2, we can temporarily stop using scipy nightlies.

[StrikerRUS]

Thank you very much for your work on this!

I remember I've already seen this scipy issue a week or two ago...

[jameslamb]

Thanks! I just manually re-ran the failed Python job and the scipy wheels were downloaded and installed successfully, so I guess it was just a temporary issue with those.