Number of qubits as single-number metric of Quantum Volume jobs

[cosenal]

We would like the quantum volume benchmark to be a single-number metric as defined in https://arxiv.org/pdf/1811.12926.
At the moment in metriq-gym we don't do the argmax part, that is, the job takes the number of qubits (width) and the depth, and returns a boolean representing whether the quantum computer is able to run the quantum volume circuit for those width x depth dimension. Instead, we want to iterate on num_qubits > 1 and stop when the condition is no longer satisfied. The max num_qubit that is obtained should be returned as a single-number metric of the benchmark.

[vprusso]

Nice. Yes, this is definitely what an external user would expect when running quantum volume, so we should make sure we align with that!

[WrathfulSpatula]

Like we discussed earlier today, the problem we still have is that there's no straightforward way to pass a job to IBMQ in a manner that works like "increment and repeat until failure," in terms of conditioning running additional, larger quantum volume jobs based on whether the last test just passed or failed when validated against simulation. Further, it also doesn't make sense, from an economic perspective, to have the quantum volume scripts immediately and without sequential dependency condition dispatch anything like a full series from 1 to 30 qubits.

We could do something like this: when polling returns a quantum volume job, check whether it passed or failed. If it passed, ask the user if they'd like to repeat with more qubits; if it failed, ask them if they'd like to repeat it with fewer qubits. This specific approach works, somewhat, but it might add relatively little value.

For now, let's put this issue aside. If we have a better idea for how to do that respects realistic economic costs, we'll revisit.

[cosenal]

Great summary of the discussion. Thanks @WrathfulSpatula