Update ecosystem build tooling (#901)

### Before submitting

Please complete the following checklist when submitting a PR:

- [x] All new features must include a unit test.
If you've fixed a bug or added code that should be tested, add a test to
the
      [`tests`](../tests) directory!

- [x] All new functions and code must be clearly commented and
documented.
If you do make documentation changes, make sure that the docs build and
      render correctly by running `make docs`.

- [x] Ensure that the test suite passes, by running `make test`.

- [x] Add a new entry to the `.github/CHANGELOG.md` file, summarizing
the
      change, and including a link back to the PR.

- [x] Ensure that code is properly formatted by running `make format`. 

When all the above are checked, delete everything above the dashed
line and fill in the pull request template.


------------------------------------------------------------------------------------------------------------

**Context:** Update Pybind11 to the latest release (2.13.5) for enabling
Numpy 2.0 support, amongst other fixes. In addition, an upgrade to CIBW
is required, as well as a new minimum version for MacOS support to 12.0

**Description of the Change:** Pybind11 updated to v2.13.5. CiBuildWheel
updated to 2.20. MacOS wheel builds updated to a minimum support of
12.0. Allow import failure to return as warning for debugging.

**Benefits:**

**Possible Drawbacks:**

**Related GitHub Issues:**

---------

Co-authored-by: ringo-but-quantum <[email protected]>

.github/CHANGELOG.md

@@ -16,6 +16,9 @@
 
 ### Breaking changes
 
+* Update MacOS wheel builds to require Monterey (12.0) or greater for x86_64 and ARM.
+  [(#901)](https://github.com/PennyLaneAI/pennylane-lightning/pull/901)
+
 * Remove PowerPC wheel build recipe for Lightning-Qubit.
   [(#902)](https://github.com/PennyLaneAI/pennylane-lightning/pull/902)
 
@@ -24,6 +27,9 @@
 
 ### Improvements
 
+* Update Pybind11 to 2.13.5.
+  [(#901)](https://github.com/PennyLaneAI/pennylane-lightning/pull/901)
+
 * Migrate wheels artifacts to v4.
   [(#893)](https://github.com/PennyLaneAI/pennylane-lightning/pull/893)
 

.github/workflows/wheel_linux_aarch64.yml

@@ -125,7 +125,7 @@ jobs:
           cp -rf ${{ github.workspace }}/Kokkos_install/${{ matrix.exec_model }}/* Kokkos/
 
       - name: Install dependencies
-        run: python -m pip install cibuildwheel~=2.16.0 tomlkit
+        run: python -m pip install cibuildwheel~=2.20.0 tomlkit
 
       - name: Configure pyproject.toml file
         run: PL_BACKEND="${{ matrix.pl_backend }}" python scripts/configure_pyproject_toml.py
@@ -144,7 +144,7 @@ jobs:
           # Python build settings
           CIBW_BEFORE_BUILD: |
             cat /etc/yum.conf | sed "s/\[main\]/\[main\]\ntimeout=5/g" > /etc/yum.conf
-            python -m pip install ninja cmake~=3.24.0
+            python -m pip install ninja cmake~=3.27.0
 
           CIBW_ENVIRONMENT: CMAKE_ARGS="-DENABLE_LAPACK=OFF"
 

.github/workflows/wheel_linux_aarch64_cuda.yml

@@ -49,7 +49,7 @@ jobs:
         uses: actions/checkout@v4
 
       - name: Install cibuildwheel
-        run: python -m pip install cibuildwheel~=2.16.0 tomlkit
+        run: python -m pip install cibuildwheel~=2.20.0 tomlkit
 
       - name: Configure pyproject.toml file
         run: PL_BACKEND="${{ matrix.pl_backend }}" python scripts/configure_pyproject_toml.py
@@ -67,7 +67,7 @@ jobs:
 
           # Python build settings
           CIBW_BEFORE_BUILD: |
-            python -m pip install ninja cmake~=3.24.3 auditwheel~=5.0 custatevec-cu${{ matrix.cuda_version }} cutensornet-cu${{ matrix.cuda_version }}
+            python -m pip install ninja cmake~=3.27.0 auditwheel~=5.0 custatevec-cu${{ matrix.cuda_version }} cutensornet-cu${{ matrix.cuda_version }}
             dnf clean all -y
             dnf install gcc-toolset-12 dnf-utils -y
             source /opt/rh/gcc-toolset-12/enable -y

.github/workflows/wheel_linux_x86_64.yml

@@ -141,7 +141,7 @@ jobs:
           dnf update -y && dnf install -y podman
 
       - name: Install dependencies
-        run: python3.10 -m pip install cibuildwheel~=2.16.0 tomlkit
+        run: python3.10 -m pip install cibuildwheel~=2.20.0 tomlkit
 
       - name: Configure pyproject.toml file
         run: PL_BACKEND="${{ matrix.pl_backend }}" python3.10 scripts/configure_pyproject_toml.py
@@ -159,7 +159,7 @@ jobs:
           # Python build settings
           CIBW_BEFORE_BUILD: |
             cat /etc/dnf.conf | sed "s/\[main\]/\[main\]\ntimeout=5/g" > /etc/dnf.conf
-            python -m pip install ninja cmake~=3.24.0 scipy
+            python -m pip install ninja cmake~=3.27.0 scipy
             dnf clean all -y
             dnf install gcc-toolset-13 -y
             source /opt/rh/gcc-toolset-13/enable -y

.github/workflows/wheel_linux_x86_64_cuda.yml

@@ -68,7 +68,7 @@ jobs:
           dnf update -y && dnf install -y podman
 
       - name: Install dependencies
-        run: python3.10 -m pip install cibuildwheel~=2.16.0 tomlkit
+        run: python3.10 -m pip install cibuildwheel~=2.20.0 tomlkit
 
       - name: Configure pyproject.toml file
         run: PL_BACKEND="${{ matrix.pl_backend }}" python3.10 scripts/configure_pyproject_toml.py
@@ -85,7 +85,7 @@ jobs:
 
           # Python build settings
           CIBW_BEFORE_BUILD: |
-            python -m pip install ninja cmake~=3.24.3 auditwheel~=5.0 custatevec-cu${{ matrix.cuda_version }} cutensornet-cu${{ matrix.cuda_version }}
+            python -m pip install ninja cmake~=3.27.0 auditwheel~=5.0 custatevec-cu${{ matrix.cuda_version }} cutensornet-cu${{ matrix.cuda_version }}
             dnf clean all -y
             dnf install gcc-toolset-12 dnf-utils -y
             source /opt/rh/gcc-toolset-12/enable -y

.github/workflows/wheel_macos_arm64.yml

@@ -24,6 +24,7 @@ env:
   ARCHS: 'arm64'
   PYTHON3_MIN_VERSION: "10"
   PYTHON3_MAX_VERSION: "12"
+  MACOSX_DEPLOYMENT_TARGET: 12.0
 
 concurrency:
   group: wheel_macos_arm64-${{ github.ref }}
@@ -81,7 +82,7 @@ jobs:
           python-version: '3.10'
 
       - name: Install dependencies
-        run: python -m pip install cibuildwheel~=2.16.0 tomlkit
+        run: python -m pip install cibuildwheel~=2.20.0 tomlkit
 
       - name: Configure pyproject.toml file
         run: PL_BACKEND="${{ matrix.pl_backend }}" python scripts/configure_pyproject_toml.py
@@ -96,7 +97,7 @@ jobs:
 
           # Python build settings
           CIBW_BEFORE_BUILD: |
-            python -m pip install pybind11 ninja cmake~=3.24.0 setuptools scipy
+            python -m pip install pybind11 ninja cmake~=3.27.0 setuptools scipy
 
           CIBW_ENVIRONMENT: |
             CMAKE_ARGS="-DCMAKE_CXX_COMPILER_TARGET=arm64-apple-macos11 -DCMAKE_SYSTEM_NAME=Darwin -DCMAKE_SYSTEM_PROCESSOR=ARM64 -DENABLE_OPENMP=OFF"

.github/workflows/wheel_macos_x86_64.yml

@@ -21,7 +21,7 @@ on:
   workflow_dispatch:
 
 env:
-  MACOSX_DEPLOYMENT_TARGET: 10.15
+  MACOSX_DEPLOYMENT_TARGET: 12.0
 
 concurrency:
   group: wheel_macos_x86_64-${{ github.ref }}
@@ -130,7 +130,7 @@ jobs:
           python-version: '3.10'
 
       - name: Install dependencies
-        run: python -m pip install cibuildwheel~=2.16.0 tomlkit
+        run: python -m pip install cibuildwheel~=2.20.0 tomlkit
 
       - name: Configure pyproject.toml file
         run: PL_BACKEND="${{ matrix.pl_backend }}" python scripts/configure_pyproject_toml.py
@@ -146,7 +146,7 @@ jobs:
 
           # Python build settings
           CIBW_BEFORE_BUILD: |
-            python -m pip install pybind11 ninja cmake~=3.24.0 setuptools scipy
+            python -m pip install pybind11 ninja cmake~=3.27.0 setuptools scipy
 
           PL_BACKEND: ${{ matrix.pl_backend }}
 

.github/workflows/wheel_win_x86_64.yml

@@ -119,7 +119,7 @@ jobs:
                     -Destination "D:\a\pennylane-lightning\pennylane-lightning\Kokkos" -Recurse -Force
 
       - name: Install dependencies
-        run: python -m pip install cibuildwheel~=2.16.0 wheel toml
+        run: python -m pip install cibuildwheel~=2.20.0 wheel toml
 
       - name: Configure pyproject.toml file
         env:
@@ -134,7 +134,7 @@ jobs:
 
           # Python build settings
           CIBW_BEFORE_BUILD: |
-            python -m pip install pybind11 cmake~=3.24.0 build
+            python -m pip install pybind11 cmake~=3.27.0 build
 
           CIBW_ENVIRONMENT: |
             CMAKE_ARGS="-DENABLE_LAPACK=OFF"

CMakeLists.txt

@@ -115,7 +115,7 @@ if(ENABLE_PYTHON)
     find_package(Python COMPONENTS Interpreter Development)
     FetchContent_Declare(pybind11
                         GIT_REPOSITORY https://github.com/pybind/pybind11.git
-                        GIT_TAG        v2.11.1
+                        GIT_TAG        v2.13.5
     )
     FetchContent_MakeAvailable(pybind11)
 endif()

pennylane_lightning/core/_version.py

@@ -16,4 +16,4 @@
    Version number (major.minor.patch[-label])
 """
 
-__version__ = "0.39.0-dev13"
+__version__ = "0.39.0-dev14"

pennylane_lightning/lightning_gpu/lightning_gpu.py

@@ -74,7 +74,8 @@
         )
 
         MPI_SUPPORT = True
-    except ImportError:
+    except ImportError as ex:
+        warn(str(ex), UserWarning)
         MPI_SUPPORT = False
 
     if find_library("custatevec") is None and not imp_util.find_spec(
@@ -90,8 +91,8 @@
         raise ValueError(f"CUDA device is an unsupported version: {get_gpu_arch()}")
 
     LGPU_CPP_BINARY_AVAILABLE = True
-except (ImportError, ValueError) as e:
-    warn(str(e), UserWarning)
+except (ImportError, ValueError) as ex:
+    warn(str(ex), UserWarning)
     backend_info = None
     LGPU_CPP_BINARY_AVAILABLE = False
 

pennylane_lightning/lightning_kokkos/lightning_kokkos.py

@@ -20,6 +20,7 @@
 from dataclasses import replace
 from pathlib import Path
 from typing import Optional
+from warnings import warn
 
 import numpy as np
 import pennylane as qml
@@ -56,7 +57,8 @@
     from pennylane_lightning.lightning_kokkos_ops import backend_info, print_configuration
 
     LK_CPP_BINARY_AVAILABLE = True
-except ImportError:
+except ImportError as ex:
+    warn(str(ex), UserWarning)
     LK_CPP_BINARY_AVAILABLE = False
     backend_info = None
 

pennylane_lightning/lightning_qubit/lightning_qubit.py

@@ -19,6 +19,7 @@
 from functools import reduce
 from pathlib import Path
 from typing import Optional, Sequence
+from warnings import warn
 
 import numpy as np
 import pennylane as qml
@@ -55,7 +56,8 @@
     from pennylane_lightning.lightning_qubit_ops import backend_info
 
     LQ_CPP_BINARY_AVAILABLE = True
-except ImportError:
+except ImportError as ex:
+    warn(str(ex), UserWarning)
     LQ_CPP_BINARY_AVAILABLE = False
 
 # The set of supported operations.

pennylane_lightning/lightning_tensor/lightning_tensor.py

@@ -18,6 +18,7 @@
 from dataclasses import replace
 from numbers import Number
 from typing import Callable, Optional, Sequence, Tuple, Union
+from warnings import warn
 
 import numpy as np
 import pennylane as qml
@@ -49,7 +50,9 @@
         raise ValueError(f"CUDA device is an unsupported version: {get_gpu_arch()}")
 
     LT_CPP_BINARY_AVAILABLE = True
-except ImportError:
+
+except ImportError as ex:
+    warn(str(ex), UserWarning)
     LT_CPP_BINARY_AVAILABLE = False
 
 Result_or_ResultBatch = Union[Result, ResultBatch]

pyproject.toml

@@ -1,5 +1,5 @@
 [build-system]
-requires = ["cmake~=3.24.0", "ninja; platform_system!='Windows'", "setuptools>=42", "tomli"]
+requires = ["cmake~=3.27.0", "ninja; platform_system!='Windows'", "setuptools>=42", "tomli"]
 build-backend = "setuptools.build_meta"
 
 [project]

scripts/configure_pyproject_toml.py

@@ -89,7 +89,7 @@ def parse_args():
     # Configure Build.
     # ------------------------
     requires = [
-        "cmake~=3.24.0",
+        "cmake~=3.27.0",
         "ninja; platform_system!='Windows'",
         "setuptools>=42",
         "tomli",

setup.py

@@ -94,9 +94,10 @@ def build_extension(self, ext: CMakeExtension):
         ]
         configure_args += (
             [f"-DPYTHON_EXECUTABLE={sys.executable}"]
-            if platform.system() == "Linux"
+            if platform.system() != "Darwin"
             else [f"-DPython_EXECUTABLE={sys.executable}"]
         )
+        configure_args += ["-DPYBIND11_FINDPYTHON=ON"]
 
         if platform.system() == "Windows":
             # As Ninja does not support long path for windows yet: