Submitted Successfully!
Thank you for your contribution! You can also upload a video entry or images related to this topic.
Ver. Summary Created by Modification Content Size Created at Operation
1 -- 2114 2022-11-04 10:06:51 |
2 format corrected. + 3 word(s) 2117 2022-11-07 03:54:15 |

Video Upload Options

Do you have a full video?


Are you sure to Delete?
If you have any further questions, please contact Encyclopedia Editorial Office.
Wang, J.;  Guo, G.;  Shan, Z. Benchmarking the Performance of a Quantum Computer. Encyclopedia. Available online: (accessed on 11 December 2023).
Wang J,  Guo G,  Shan Z. Benchmarking the Performance of a Quantum Computer. Encyclopedia. Available at: Accessed December 11, 2023.
Wang, Junchao, Guoping Guo, Zheng Shan. "Benchmarking the Performance of a Quantum Computer" Encyclopedia, (accessed December 11, 2023).
Wang, J.,  Guo, G., & Shan, Z.(2022, November 04). Benchmarking the Performance of a Quantum Computer. In Encyclopedia.
Wang, Junchao, et al. "Benchmarking the Performance of a Quantum Computer." Encyclopedia. Web. 04 November, 2022.
Benchmarking the Performance of a Quantum Computer

The quantum computer has been claimed to show more quantum advantage than the classical computer in solving some specific problems. Many companies and research institutes try to develop quantum computers with different physical implementations. Currently, most people only focus on the number of qubits in a quantum computer and consider it as a standard to evaluate the performance of the quantum computer intuitively. However, it is quite misleading in most times, especially for investors or governments. This is because the quantum computer works in a quite different way than classical computers. Thus, quantum benchmarking is of great importance. Currently, many quantum benchmarks are proposed from different aspects.

quantum computing quantum benchmark fidelity qubit quantum circuit

1. Overview of Quantum Benchmarks

In this research, the researchers classify the benchmarks into three categories: the physical benchmarks, the aggregated benchmarks, and application-level benchmarks. Most news and reports place emphasis on the number of qubits in a quantum processor, which is mostly misleading for those who are not familiar with quantum computing. Definitely, the number of qubits can directly decide the quantum computing power of a quantum computer. Some people intuitively think that the quantum computing power of a quantum computer grows exponentially with the number of qubits. For instance, in 2019, Google first demonstrated “quantum supremacy” with a Sycamore quantum processor having 53 qubits. However, apart from the number of qubits, the noise and the quantum property of the qubits can greatly affect the correctness of the results. Thus, apart from the number of qubits, there are other physical properties that most people are concerned about.
Physical benchmarks include tools, models, and algorithms to reflect the physical properties of a quantum processor. Typical physical indicators of quantum computers include T1, T2, single qubit gate fidelity, two qubit gate fidelity, and readout fidelity. The aggregated benchmarks can help the user to determine the performance of a quantum processor with only one or several parameters. The aggregated metrics can be calculated with randomly generated quantum circuits or estimated based on the basic physical properties of a quantum processor. Typical aggregated benchmarks include quantum volume (QV) and algorithmic qubits (AQ). The application-level benchmarks refer to the metrics obtained by running real-world applications on the quantum computer. Many existing works propose using real world applications to benchmark the quantum computer’s performance because they assume that random circuits cannot reflect a quantum computer’s performance accurately. An overview of the existing quantum benchmarks is shown in Figure 1.
Figure 1. Overview of the quantum benchmarks.

2. Physical Benchmarks

Different physical implementations are concerned with different aspects of a quantum computing system. For instance, the trapped ion-based quantum computer focuses more on the stability of the trap frequency, the duration of a gate operation, and the stability of the control lasers. The superconducting quantum computers’ performance is affected by the controllability and scalability of the system. Mostly, they are affected by the precision of the Josephson junction, anharmonicity, and gate duration [1].
In general, the quantum computation systems are concerned with the quantum correlations and controlling operation precision. In a superconducting quantum computer, generally researchers from the background of quantum information focus more on physical properties of quantum computers, such as the T1, T2, number of qubits, connectivity, single qubit gate fidelity, two qubit gate fidelity, and readout fidelity.
The indicators for quantum computers of IBM’s online quantum cloud (Table 1, from [2]) is shown in the following table.
Table 1. IBM quantum cloud’s performance metrics. Avg stands for average; N/A means not applicable.
Name Number of Qubits QV Avg.T1 (μs) Avg.T2 (μs) Avg.Readout Fidelity Avg.CNOT Fidelity
brooklyn 65 32 77.1686 74.6345 0.9682 0.9746
manhattan 65 32 110.1959 101.6078 0.9761 0.9543
hanoi 27 64 123.3959 93.4341 0.9837 0.991
sydney 27 32 266.1433 256.6081 0.9833 0.9898
peekskill 27 N/A 97.4474 107.0911 0.9821 0.9896
cairo 27 64 76.01 97.6543 0.9796 0.989
toronto 27 32 180.3614 155.1329 0.9869 0.9814
kolkata 27 128 70.3363 75.2432 0.9698 0.9536
mumbai 27 128 117.2574 92.1067 0.9484 0.9526
montreal 27 128 81.004 104.678 0.938 0.4972
guadalupe 16 32 132.6257 40.5357 0.977 0.9896
lagos 7 32 158.6 57.702 0.9697 0.9912
jakarta 7 16 74.214 104.008 0.9728 0.9895
perth 7 32 155.0078 92.217 0.9118 0.9894
casablanca 7 32 82.2681 96.0744 0.9696 0.9883
nairobi 7 32 86.5337 107.1733 0.9428 0.9878
quito 5 16 130.2629 100.9629 0.9859 0.9932
santiago 5 32 105.2286 98.9143 0.9633 0.9909
manila 5 32 100.56 101.29 0.9739 0.99
lima 5 8 84.0278 84.4122 0.9829 0.9891
belem 5 16 75.936 94.722 0.9676 0.9828
bogota 5 32 92.454 124.096 0.959 0.9794
armonk 1 1 118.1 149.22 0.967 N/A

4. Application-Based Benchmarks

The physical properties of a quantum computer can affect its performance. However, it is difficult to determine whether a quantum computer outperforms another only based on these properties. For instance, a quantum computer “A” has less qubits, but the qubits’ quality of another quantum computer “B” is higher. If a quantum application needs more qubits, then “A” is preferred. If a quantum application requires the qubits’ quality to be higher, then “B” is preferred. Therefore, some researchers propose to evaluate the performance of a quantum computer with a real-world quantum application.
A summary of the application-based quantum benchmarks is shown in Table 2. In Table 2, the researchers can see that most quantum benchmarks consider the typical combinational optimization problems and use variational quantum circuits (VQC) to solve the problem. This is mainly because the combinational optimization problems can be widely used in many real-world scenarios, such as traffic engineering and flight scheduling. Moreover, the variational quantum solutions, such as quantum approximation optimization algorithm (QAOA) and variational quantum eigensolver (VQE) are popular, due to the possibility to obtain a useful result on NISQ devices. Thus, most people believe that, in the NISQ era, the variational quantum solution will remain the most effective solution.
Table 2. Summary of the application-based quantum benchmarks.


  1. How Is the Quantum Computer Performance Assessed. Available online: (accessed on 9 July 2021).
  2. IBM Quantum Experience. Available online: (accessed on 17 November 2021).
  3. Bishop, L.S.; Bravyi, S.; Cross, A.; Gambetta, J.M.; Smolin, J. Quantum Volume. 2017. Available online: (accessed on 17 November 2021).
  4. Cross, A.W.; Bishop, L.S.; Sheldon, S.; Nation, P.D.; Gambetta, J.M. Validating quantum computers using randomized model circuits. Phys. Rev. A 2019, 100, 032328.
  5. Blume-Kohout, R.; Young, K.C. A volumetric framework for quantum computer benchmarks. Quantum 2020, 4, 362.
  6. IonQ Quantum Computer 4 Million Quantum Volume. Available online: (accessed on 28 March 2021).
  7. Scott Aaronson: Turn Down the Quantum Volume. Available online: (accessed on 1 March 2020).
  8. Scaling IonQ’s Quantum Computers: The Roadmap. Available online: (accessed on 1 December 2020).
  9. Algorithmic Qubit Calculator. Available online: (accessed on 1 March 2021).
  10. Proctor, T.; Rudinger, K.; Young, K.; Nielsen, E.; Blume-Kohout, R. Measuring the Capabilities of Quantum Computers. Nat. Phys. 2020, 18, 75–79.
  11. Wack, A.; Paik, H.; Javadi-Abhari, A.; Jurcevic, P.; Faro, I.; Gambetta, J.M.; Johnson, B.R. Quality, Speed, and Scale: Three key attributes to measure the performance of near-term quantum computers. arXiv 2021, arXiv:2110.14108.
  12. Mesman, K.; Al-Ars, Z.; Mller, M. QPack: Quantum Approximate Optimization Algorithms as universal benchmark for quantum computers. arXiv 2021, arXiv:2103.17193.
  13. qScore. Available online: (accessed on 1 June 2021).
  14. New Cambridge Quantum Algorithm Sets a Benchmark in Performance and Effectively Outperforms Existing Methods. Available online: (accessed on 1 June 2021).
  15. Dallaire-Demers, P.L.; Stęchły, M.; Gonthier, J.F.; Bashige, N.T.; Romero, J.; Cao, Y. An application benchmark for fermionic quantum simulations. Am. Phys. Soc. 2021, arXiv:2003.01862.
  16. Benedetti, M.; Garcia-Pintos, D.; Perdomo, O.; Leyton-Ortega, V.; Nam, Y.; Perdomo-Ortiz, A. A generative modeling approach for benchmarking and training shallow quantum circuits. NPJ Quantum Inf. 2019, 5, 45.
  17. Mills, D.; Sivarajah, S.; Scholten, T.L.; Duncan, R. Application-Motivated, Holistic Benchmarking of a Full Quantum Computing Stack. Quantum 2020, 5, 415.
  18. Lubinski, T.; Johri, S.; Varosy, P.; Coleman, J.; Zhao, L.; Necaise, J.; Baldwin, C.H.; Mayer, K.; Proctor, T. Application-Oriented Performance Benchmarks for Quantum Computing. arXiv 2021, arXiv:2110.03137.
  19. Dong, Y.; Lin, L. Random circuit block-encoded matrix and a proposal of quantum LINPACK benchmark. Phys. Rev. A 2020, 33, 062412.
  20. McCaskey, A.J.; Parks, Z.P.; Jakowski, J.; Moore, S.V.; Morris, T.; Humble, T.S.; Pooser, R. Quantum chemistry as a benchmark for near-term quantum computers. npj Quantum Inf. 2019, 5, 99.
  21. Li, A.; Krishnamoorthy, S. QASMBench: A Low-level QASM Benchmark Suite for NISQ Evaluation and Simulation. arXiv 2020, arXiv:2005.13018.
Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to : , ,
View Times: 983
Revisions: 2 times (View History)
Update Date: 07 Nov 2022