Add performance benchmarks for Qiskit Machine Learning

[adekusar-drl]

Description

Qiskit Applications (Optimization, Finance, Nature, ML) have some performance benchmarks implemented. By performance we understand various metrics like executing time, required memory and other specific metrics, e.g. score, that may arise in the benchmarks. Currently a few on them were added Qiskit Machine Learning, benchmarks for quantum neural networks. This project aims at building a set of such benchmarks for QuantumKernel.

In the project we want to do:

• Analyze the QuantumKernel code/features for potential benchmarks, e.g. tutorials and unit tests may be good places for a search for candidates.
• Re-use the existing code and implement new benchmarks
• Analyze the obtained results and suggest further improvement if any.

Current benchmarks are available here: https://qiskit.github.io/qiskit-app-benchmarks.

Deliverables

Performance benchmarks for QuantumKernel.

Mentors details

• Mentor 1
  • Name: Atsushi Matsuo
  • GitHub ID: @a-matsuo
  • What they do: Researcher at IBM Research Tokyo, Qiskit Optimization core developer

Number of mentees

1

Type of mentees

What is required:

• Good software engineering skills, ability to work with Git.
• Willingness to look through third party code.
• Proficiency in Python.
• Interested in Qiskit Machine Learning and quantum computing in general.

[adekusar-drl]

@HuangJunye I'd put this one on hold or closed in favor of #11. Or may be somebody else can become a mentor here.

[a-matsuo]

Hi @HuangJunye, I talked with @adekusar-drl and I'd like to take over this project.

[Gruntrexpewrus]

Hi, this project is very interesting!

[Gruntrexpewrus]

This is our presentation for the first checkpoint! Work is in progress :)
#4 Add performance benchmarks for Qiskit Machine Learning.pdf

[Gruntrexpewrus]

Our work in QAMP Spring 2022 consists in benchmarking and exploring the capability of Qiskit machine learning in regards to quantum kernel methods. We have an open draft pull request on qiskit-app-benchmarks and we are working to extract visualizations in airspeed velocity (asv). We test 2 main models, such as QuantumKernel and QuantumKernelTrainer, which differ for the absence or not of parametrized gates with backpropagation. Our tests are conducted in the realm of classification tasks (indeed after the application of the quantum kernel, a support vector machine method performs the prediction task). The work is developed in three main script using object oriented programming. In qk_base_benchmark.py we create the class that allows to test either the quantum kernel, or the quantum kernel trainer (basically a part of a variational classifier). In qk_benchmark.py we declare the options for our benchmarking and create a full script that will test the performance of the models based on time and accuracy. In qk_fit_benchmark.py we invesitigate how much time does it take to fit a good kernel (which in the variational one includes a backpropagation).
We think our work may be able to cast light on eventual bottlenecks or improvements to be obtained for the kernel methods, opening possibly some new research directions and enriching the pool of benchmarks for Qiskit.
Our experience has been positive so far and we enjoyed the discussions. Even though our works have been slowed because of special circumstances by both parts, we think that the code is already a useful starting point for a general benchmarking of kernel methods.

def setup(
        self, 
        dataset: str, 
        technique: str, 
        quantum_instance_name: str,
        optimizer: str,
        loss_function: str
    ) -> None:
        """Set up the benchmark."""
        self.train_features = self.datasets[dataset]["train_features"]
        self.train_labels = self.datasets[dataset]["train_labels"]
        #new
        n_qubits = self.train_features.shape[1]
        if technique == "QuantumKernel":
            self.model = self._construct_QuantumKernel_classical_classifier(quantum_instance_name= quantum_instance_name, 
                                                            optimizer = optimizer,
                                                            num_qubits = n_qubits) 
        elif technique == "QuantumKernelTraining":
            self.model = self._construct_QuantumKernelTrainer(quantum_instance_name= quantum_instance_name, 
                                                            optimizer= optimizer, 
                                                            loss_function = loss_function,
                                                            num_qubits = n_qubits, 
                ) 

    def _construct_QuantumKernel(
        self,
        num_inputs: int,
        quantum_instance_name: str,
        method: str,
    ) -> QuantumKernel:
        """Construct a QuantumKernel"""
        if method == "quantumclassical":
            feature_map = ZZFeatureMap(num_inputs, reps=2, entanglement="linear")
            #quantum kernel, not parametrized
            qk = QuantumKernel(feature_map=feature_map, quantum_instance=self.backends[quantum_instance_name])
            return qk
        elif method == "quantum":
            user_params = ParameterVector("θ", 1)
            fm0 = QuantumCircuit(num_inputs)
            for i in range(num_inputs):
                fm0.ry(user_params[0], i)
            fm1 = ZZFeatureMap(num_inputs, reps=2, entanglement="linear")
            feature_map = fm0.compose(fm1)
            #quantum kernel, parametrized
            qk = QuantumKernel(feature_map = feature_map, user_parameters=user_params, quantum_instance=self.backends[quantum_instance_name])
            return qk
        else:
            return ValueError(f"Unsupported method: {method}")

[Gruntrexpewrus]

Final Checkpoint!
Final Showcase.pdf