Migration of lightning.gpu device to the new device API (#853)

### Before submitting

Please complete the following checklist when submitting a PR:

- [X] All new features must include a unit test.
If you've fixed a bug or added code that should be tested, add a test to
the
      [`tests`](../tests) directory!

- [X] All new functions and code must be clearly commented and
documented.
If you do make documentation changes, make sure that the docs build and
      render correctly by running `make docs`.

- [X] Ensure that the test suite passes, by running `make test`.

- [X] Add a new entry to the `.github/CHANGELOG.md` file, summarizing
the
      change, and including a link back to the PR.

- [X] Ensure that code is properly formatted by running `make format`. 

When all the above are checked, delete everything above the dashed
line and fill in the pull request template.


------------------------------------------------------------------------------------------------------------

**Context:**
Following the design of *Lightning Qubit* and *Lightning Kokkos* that
use the New Device API, we can now migrate **Lightning GPU** device to
the new device API.

**Description of the Change:**
Migration of Lightning GPU with MPI support to integrate the New device
API. The list of main feature changes are
- Add the `MPIHandler` `state_vector`, `measurements`, and
`adjoint-jacobian` classes for `lightning.gpu`.
- Add the `simulate`, `jacobian`, `simulate_and_jacobian`, `vjp`, and
`simulate_and_vjp` methods to `lighting.gpu`.
- Add MPI support to `lightning.gpu` with the New device API.
- Update unit/integration tests for the new device API to work with
`lightning.gpu`
- Check the full support for sampling in full parity with Lightning
Qubit
- Replace the old device API for Lightning GPU.

**Benefits:**
Full integration of **Lightning GPU** with the new device API. 

**Possible Drawbacks:**

**Related GitHub Issues:**

[sc-70964] [sc-59219]

---------

Co-authored-by: ringo-but-quantum <[email protected]>
Co-authored-by: Vincent Michaud-Rioux <[email protected]>
Co-authored-by: Ali Asadi <[email protected]>
Co-authored-by: Amintor Dusko <[email protected]>

.github/CHANGELOG.md

@@ -21,6 +21,9 @@
 * Lightning-Kokkos migrated to the new device API.
   [(#810)](https://github.com/PennyLaneAI/pennylane-lightning/pull/810)
 
+* Lightning-GPU migrated to the new device API.
+  [(#853)](https://github.com/PennyLaneAI/pennylane-lightning/pull/853)
+
 ### Breaking changes
 
 * Deprecate PI gates implementation.

.github/workflows/wheel_noarch.yml

@@ -50,7 +50,6 @@ jobs:
         if: ${{ matrix.pl_backend == 'lightning_qubit'}}
         uses: actions/checkout@v4
 
-
       - uses: actions/setup-python@v5
         if: ${{ matrix.pl_backend == 'lightning_qubit'}}
         with:

MANIFEST.in

@@ -3,7 +3,7 @@ include cmake/*
 include requirements.txt
 include .github/CHANGELOG.md
 include pennylane_lightning/lightning_qubit/lightning_qubit.toml
-include pennylane_lightning/lightning_qpu/lightning_gpu.toml
+include pennylane_lightning/lightning_gpu/lightning_gpu.toml
 include pennylane_lightning/lightning_kokkos/lightning_kokkos.toml
 include pennylane_lightning/core/_version.py
 graft pennylane_lightning/core/src/

Makefile

@@ -35,9 +35,11 @@ help:
 	@echo "  test-cpp [verbose=1]     to run the C++ test suite (requires CMake)"
 	@echo "                           use with 'verbose=1' for building with verbose flag"
 	@echo "  test-cpp [target=?]      to run a specific C++ test target (requires CMake)."
+	@echo "  test-cpp-mpi [backend=?] to run the C++ test suite with MPI (requires CMake and MPI)"
+	@echo "                           Default: lightning_gpu"
 	@echo "  test-python [device=?]   to run the Python test suite"
 	@echo "                           Default: lightning.qubit"
-	@echo "  wheel [backend=?]        to configure and build Python wheels
+	@echo "  wheel [backend=?]        to configure and build Python wheels"
 	@echo "                           Default: lightning_qubit"
 	@echo "  coverage [device=?]      to generate a coverage report for python interface"
 	@echo "                           Default: lightning.qubit"
@@ -98,7 +100,7 @@ coverage-cpp:
 	lcov --directory . -b ../pennylane_lightning/core/src/ --capture --output-file coverage.info; \
 	genhtml coverage.info --output-directory out
 
-.PHONY: test-python test-builtin test-suite test-cpp
+.PHONY: test-python test-builtin test-suite test-cpp test-cpp-mpi
 test-python: test-builtin test-suite
 
 test-builtin:
@@ -124,6 +126,27 @@ else
 	cmake --build ./BuildTests $(VERBOSE) --target test
 endif
 
+test-cpp-mpi:
+	rm -rf ./BuildTests
+	cmake -BBuildTests -G Ninja \
+		  -DCMAKE_BUILD_TYPE=Debug \
+		  -DBUILD_TESTS=ON \
+		  -DENABLE_WARNINGS=ON \
+		  -DPL_BACKEND=lightning_gpu \
+		  -DENABLE_MPI=ON \
+		  $(OPTIONS)
+ifdef target
+	cmake --build ./BuildTests $(VERBOSE) --target $(target)
+	mpirun -np 2 ./BuildTests/$(target)
+else
+	cmake --build ./BuildTests $(VERBOSE)
+	for file in ./BuildTests/*_test_runner_mpi; do \
+		echo "Running $$file"; \
+		mpirun -np 2 $$file ; \
+	done
+endif
+
+
 .PHONY: format format-cpp format-python
 format: format-cpp format-python
 

doc/lightning_gpu/device.rst

@@ -11,9 +11,9 @@ A ``lightning.gpu`` device can be loaded using:
     import pennylane as qml
     dev = qml.device("lightning.gpu", wires=2)
 
-If the NVIDIA cuQuantum libraries are available, the above device will allow all operations to be performed on a CUDA capable GPU of generation SM 7.0 (Volta) and greater. If the libraries are not correctly installed, or available on path, the device will fall-back to ``lightning.qubit`` and perform all simulation on the CPU.
+If the NVIDIA cuQuantum libraries are available, the above device will allow all operations to be performed on a CUDA capable GPU of generation SM 7.0 (Volta) and greater. If the libraries are not correctly installed, or available on path, the device will raise an error.
 
-The ``lightning.gpu`` device also directly supports quantum circuit gradients using the adjoint differentiation method. This can be enabled at the PennyLane QNode level with:
+The ``lightning.gpu`` device supports quantum circuit gradients using the adjoint differentiation method by default. This can be enabled at the PennyLane QNode level with:
 
 .. code-block:: python
 
@@ -281,3 +281,6 @@ To enable the memory-optimized adjoint method with MPI support, ``batch_obs`` sh
     dev = qml.device('lightning.gpu', wires= n_wires, mpi=True, batch_obs=True)
 
 For the adjoint method, each MPI process will provide the overall simulation results.
+
+.. note::
+    The observable ``Projector``` does not have support with the multi-GPU backend.

mpitests/conftest.py

@@ -98,6 +98,13 @@ def get_device():
 # Device specification
 if device_name == "lightning.gpu":
     from pennylane_lightning.lightning_gpu import LightningGPU as LightningDevice
+    from pennylane_lightning.lightning_gpu._measurements import (
+        LightningGPUMeasurements as LightningMeasurements,
+    )
+    from pennylane_lightning.lightning_gpu._state_vector import (
+        LightningGPUStateVector as LightningStateVector,
+    )
+
 else:
     raise qml.DeviceError(f"The MPI tests do not apply to the {device_name} device.")