|
2 | 2 | "cells": [ |
3 | 3 | { |
4 | 4 | "cell_type": "markdown", |
| 5 | + "id": "4dbd4656-5e40-4d5a-a29b-b13817c19cf2", |
5 | 6 | "metadata": {}, |
6 | 7 | "source": [ |
7 | | - "# Compute resources and resource management \n", |
| 8 | + "# Compute resources and resource management\n", |
8 | 9 | "\n", |
9 | 10 | "\n", |
10 | 11 | "## Resource Model and Classical Resources\n", |
|
35 | 36 | "\n", |
36 | 37 | "For memory, the unit of measure would be Mega/Giga/Terabytes.\n", |
37 | 38 | "\n", |
38 | | - "For graphics cards and other accelerators, the unit of measure depends on the context. While their true computing capabilitiy is measured by fine-grained matrics - number of processing cores, memory size, and memory bandwidth, in high level discussions of cluster resources or job schedulings, GPUs and similar accelerators are can be quantifed at the device-level by the number of whole devices assigned (e.g. 3 GPUs). \n", |
| 39 | + "For graphics cards and other accelerators, the unit of measure depends on the context. While their true computing capabilitiy is measured by fine-grained matrics - number of processing cores, memory size, and memory bandwidth, in high level discussions of cluster resources or job schedulings, GPUs and similar accelerators are can be quantifed at the device-level by the number of whole devices assigned (e.g. 3 GPUs).\n", |
39 | 40 | "\n", |
40 | | - "Network/connectivity/busses are crucial aspect of any compute infrastructure as they dictate how fast data is transferred between compute components. From LPU to cache of CPU, to RAM, to PCI cards, to network connected devices; all of it is communication and it is crucial to have an accurate mental model of it to design highly optimized algorithms for HPC. \n", |
| 41 | + "Network/connectivity/busses are crucial aspect of any compute infrastructure as they dictate how fast data is transferred between compute components. From LPU to cache of CPU, to RAM, to PCI cards, to network connected devices; all of it is communication and it is crucial to have an accurate mental model of it to design highly optimized algorithms for HPC.\n", |
41 | 42 | "\n", |
42 | 43 | "\n", |
43 | 44 | "\n", |
|
72 | 73 | }, |
73 | 74 | { |
74 | 75 | "cell_type": "markdown", |
| 76 | + "id": "b9c79070-17a8-42cc-98ff-27f351df80d9", |
75 | 77 | "metadata": {}, |
76 | 78 | "source": [ |
77 | 79 | "## New Type of Resource: QPU (Quantum Processing Unit)\n", |
|
80 | 82 | "\n", |
81 | 83 | "### Definition of QPU\n", |
82 | 84 | "\n", |
83 | | - "* Quantum processing unit (QPU): A QPU includes all of the hardware responsible for accepting an executable quantum instruction set, or a quantum circuit, and returning an accurate answer. \n", |
| 85 | + "* Quantum processing unit (QPU): A QPU includes all of the hardware responsible for accepting an executable quantum instruction set, or a quantum circuit, and returning an accurate answer.\n", |
84 | 86 | "\n", |
85 | | - "That means the QPU includes one or more quantum chips (e.g. Heron), the multiple additional components in the dilution refrigerator such as the quantum amplifiers, the control electronics, and the classical compute required for tasks such as holding the instructions and waveforms in memory, accumulating results, and future error correction decoding. While a dilution refrigerator is required to perform these tasks, we exclude the dilution refrigerator from this definition to allow for the case of multiple QPUs in the same fridge. \n", |
| 87 | + "That means the QPU includes one or more quantum chips (e.g. Heron), the multiple additional components in the dilution refrigerator such as the quantum amplifiers, the control electronics, and the classical compute required for tasks such as holding the instructions and waveforms in memory, accumulating results, and future error correction decoding. While a dilution refrigerator is required to perform these tasks, we exclude the dilution refrigerator from this definition to allow for the case of multiple QPUs in the same fridge.\n", |
86 | 88 | "\n", |
87 | 89 | "* Quantum computer: A quantum computer is comprised of the QPU plus the classical compute which hosts the runtime environment.\n", |
88 | 90 | "\n", |
|
140 | 142 | "\n", |
141 | 143 | "### Scaling quantum resources\n", |
142 | 144 | "\n", |
143 | | - "Scaling of quantum resources can also be categorized into vertical and horizontal. \n", |
| 145 | + "Scaling of quantum resources can also be categorized into vertical and horizontal.\n", |
144 | 146 | "\n", |
145 | | - "* Vertical scaling would be increasing the number of qubits per chiplet or improving the fidelity of devices. \n", |
146 | | - "* Horizontal scaling would be connecting chips with couplers or with classical interconnect. \n", |
| 147 | + "* Vertical scaling would be increasing the number of qubits per chiplet or improving the fidelity of devices.\n", |
| 148 | + "* Horizontal scaling would be connecting chips with couplers or with classical interconnect.\n", |
147 | 149 | "\n", |
148 | 150 | "\n", |
149 | 151 | "\n", |
|
166 | 168 | }, |
167 | 169 | { |
168 | 170 | "cell_type": "markdown", |
| 171 | + "id": "dc79a919-7936-4ed1-b957-f4046f8fca6d", |
169 | 172 | "metadata": {}, |
170 | 173 | "source": [ |
171 | 174 | "## Resource Management\n", |
|
188 | 191 | "There are analogs of all these processes for quantum computers.\n", |
189 | 192 | "* Jobs are submitted by the user, leveraging Qiskit runtime, and typically using a Qiskit runtime primitive, like `Sampler`, `Estimator`, or others.\n", |
190 | 193 | "* The user selects from a list of backends to which they have access. The complete list of available backends can be seen [here](https://quantum.cloud.ibm.com/computers). It is common to simply use the least busy quantum computer. But there are cases where it might be important to use a specific one because of device layout considerations, replication of previous calculations, etc.\n", |
191 | | - "* The execution of quantum jobs is similar to the HPC case. Although some differences have already been outlined, a few are worth repeating here. QPUs are currently not generally located on the same node as classical compute resources but are connected through a network. This may have scheduling implications. Further, quantum computers may have substantial queue times, and these queue times vary, making precising control of timing difficult. This situation may be different for dedicated systems; that depends on the quantum computer's internal administration. \n", |
| 194 | + "* The execution of quantum jobs is similar to the HPC case. Although some differences have already been outlined, a few are worth repeating here. QPUs are currently not generally located on the same node as classical compute resources but are connected through a network. This may have scheduling implications. Further, quantum computers may have substantial queue times, and these queue times vary, making precising control of timing difficult. This situation may be different for dedicated systems; that depends on the quantum computer's internal administration.\n", |
192 | 195 | "\n", |
193 | 196 | "### Controlling/Managing - Workload Management\n", |
194 | 197 | "\n", |
|
197 | 200 | "* Job scheduling: The activity of the scheduler software in deciding which jobs run, when, and on which resources, managing priorities and queues to efficiently utilize the HPC system. Although this broad statement applies to quantum resources, there may be less control over timing than with other resources.\n", |
198 | 201 | "\n", |
199 | 202 | "\n", |
200 | | - "**Example:** \n", |
| 203 | + "**Example:**\n", |
201 | 204 | "\n", |
202 | 205 | "Consider a well-known task as a context for understanding resource management: finding the prime factors of large numbers. Let us further assume that the algorithm being used relies on brute force checking of every potential divisor. While this is ofen not the most efficient method, it is easy to understand how the workload might be managed.\n", |
203 | 206 | "\n", |
|
218 | 221 | "\n", |
219 | 222 | "High performance computing environments use special software to carry out these steps and manage resources. In the next section, we will learn about a widely-adopted resource management software systems: Slurm.\n", |
220 | 223 | "\n", |
221 | | - "**Example with quantum resources:** \n", |
| 224 | + "**Example with quantum resources:**\n", |
222 | 225 | "\n", |
223 | 226 | "A workflow that will be the subject of other lessons in this course is determining chemical ground states and energies using sample-based quantum diagonalization (SQD). This is covered in more detail in Lesson 4, and you can also visit [this course on SQD](https://quantum.cloud.ibm.com/learning/en/courses/quantum-diagonalization-algorithms) and related methods on [IBM Quantum Learning](https://quantum.cloud.ibm.com/learning/). All we need to know for this discussion is that the workflow involves:\n", |
224 | 227 | "* preparing a quantum circuit\n", |
|
264 | 267 | "\n", |
265 | 268 | "(a) Planning\n", |
266 | 269 | "(b) Controlling/managing\n", |
267 | | - "(c) Allocating/scheduling, \n", |
| 270 | + "(c) Allocating/scheduling,\n", |
268 | 271 | "\n", |
269 | 272 | "\n", |
270 | 273 | "</details>" |
271 | 274 | ] |
272 | 275 | }, |
273 | 276 | { |
274 | 277 | "cell_type": "markdown", |
| 278 | + "id": "7ff27439-296e-4dba-98e4-8cae2cc8c9d6", |
275 | 279 | "metadata": {}, |
276 | 280 | "source": [ |
277 | 281 | "## Software: Slurm\n", |
|
300 | 304 | }, |
301 | 305 | { |
302 | 306 | "cell_type": "markdown", |
| 307 | + "id": "4fea8c6e-a32e-45b6-ad8d-eb349e6cecc3", |
303 | 308 | "metadata": {}, |
304 | 309 | "source": [ |
305 | 310 | "```shell\n", |
|
320 | 325 | "Gres=qpu:1\n", |
321 | 326 | "NodeAddr=q1 NodeHostName=q1 Version=21.08.6\n", |
322 | 327 | "...\n", |
323 | | - " \n", |
| 328 | + "\n", |
324 | 329 | "```\n", |
325 | 330 | "\n", |
326 | 331 | "We have 2 partitions or node groups: normal and quantum. The normal partition is composed of nodes that have access to classical resources only. The quantum partition has access to quantum resources. You can see the details of each node by executing `scontrol show nodes`." |
327 | 332 | ] |
328 | 333 | }, |
329 | 334 | { |
330 | 335 | "cell_type": "markdown", |
| 336 | + "id": "20932bac-88de-49bd-9f58-eea29c31df45", |
331 | 337 | "metadata": {}, |
332 | 338 | "source": [ |
333 | 339 | "### Running simple hello world example in Slurm.\n", |
|
346 | 352 | }, |
347 | 353 | { |
348 | 354 | "cell_type": "markdown", |
| 355 | + "id": "cc826674-d703-4765-ab30-ecd0b9035007", |
349 | 356 | "metadata": {}, |
350 | 357 | "source": [ |
351 | 358 | "Now we need to tell the resource manager what resources we need to execute this program. Slurm provides a way to specify all metadata for the job via a submission script, which is just a shell script with Slurm-specific annotations. These annotations allow you to specify resource requirements, schedule parameters, and more. Let's create a `hello_world.sh` shell script for this.\n", |
|
433 | 440 | }, |
434 | 441 | { |
435 | 442 | "cell_type": "markdown", |
| 443 | + "id": "1f32cef4-3b71-49ea-b11b-70c19250bb9e", |
436 | 444 | "metadata": {}, |
437 | 445 | "source": [ |
438 | 446 | "### Running simple Qiskit hello world example in Slurm.\n", |
|
446 | 454 | }, |
447 | 455 | { |
448 | 456 | "cell_type": "markdown", |
| 457 | + "id": "7fc48590-2dca-4568-b729-6ae09dc41f81", |
449 | 458 | "metadata": {}, |
450 | 459 | "source": [ |
451 | 460 | "```python\n", |
|
455 | 464 | "from qiskit.quantum_info import SparsePauliOp\n", |
456 | 465 | "from qiskit.transpiler import generate_preset_pass_manager\n", |
457 | 466 | "from qiskit_ibm_runtime import EstimatorV2 as Estimator\n", |
458 | | - " \n", |
| 467 | + "\n", |
459 | 468 | "# Create a new circuit with two qubits\n", |
460 | 469 | "qc = QuantumCircuit(2)\n", |
461 | | - " \n", |
| 470 | + "\n", |
462 | 471 | "# Add a Hadamard gate to qubit 0\n", |
463 | 472 | "qc.h(0)\n", |
464 | | - " \n", |
| 473 | + "\n", |
465 | 474 | "# Perform a controlled-X gate on qubit 1, controlled by qubit 0\n", |
466 | 475 | "qc.cx(0, 1)\n", |
467 | 476 | "\n", |
|
470 | 479 | "\n", |
471 | 480 | "# switch to QRMI service\n", |
472 | 481 | "from qiskit_ibm_runtime import QiskitRuntimeService\n", |
473 | | - " \n", |
| 482 | + "\n", |
474 | 483 | "service = QiskitRuntimeService()\n", |
475 | | - " \n", |
| 484 | + "\n", |
476 | 485 | "backend = service.backend(\"...\")\n", |
477 | | - " \n", |
| 486 | + "\n", |
478 | 487 | "# Convert to an ISA circuit and layout-mapped observables.\n", |
479 | 488 | "pm = generate_preset_pass_manager(backend=backend, optimization_level=1)\n", |
480 | 489 | "isa_circuit = pm.run(qc)\n", |
481 | | - " \n", |
| 490 | + "\n", |
482 | 491 | "# Construct the Estimator instance.\n", |
483 | | - " \n", |
| 492 | + "\n", |
484 | 493 | "estimator = Estimator(mode=backend)\n", |
485 | 494 | "estimator.options.resilience_level = 1\n", |
486 | 495 | "estimator.options.default_shots = 5000\n", |
487 | | - " \n", |
| 496 | + "\n", |
488 | 497 | "mapped_observables = [\n", |
489 | 498 | " observable.apply_layout(isa_circuit.layout) for observable in observables\n", |
490 | 499 | "]\n", |
491 | | - " \n", |
| 500 | + "\n", |
492 | 501 | "# One pub, with one circuit to run against five different observables.\n", |
493 | 502 | "job = estimator.run([(isa_circuit, mapped_observables)])\n", |
494 | 503 | "\n", |
495 | 504 | "job_result = job.result()\n", |
496 | | - " \n", |
| 505 | + "\n", |
497 | 506 | "pub_result = job.result()[0]\n", |
498 | 507 | "\n", |
499 | 508 | "print(\"Result\", pub_result)\n", |
|
503 | 512 | }, |
504 | 513 | { |
505 | 514 | "cell_type": "markdown", |
| 515 | + "id": "3462d06f-767d-4586-8c95-99426df9c3b1", |
506 | 516 | "metadata": {}, |
507 | 517 | "source": [ |
508 | 518 | "Here we will use the quantum resource management interface (QRMI), a Slurm Spank plugin for Quantum resources and jobs support which was developed jointly by IBM, Pasqal, the The Hartree Center and RPI. We made a simple pauli-2-design circuit with random initial values and a simple observable and run it using Estimator to get the expectation value. To run it we will again need the submission script `hello_qiskit.sh`, which will have quantum resources as a requirement.\n", |
|
539 | 549 | "# 1 main hello_qiskit root R 0:01 1 q1\n", |
540 | 550 | "```\n", |
541 | 551 | "\n", |
542 | | - "And we will be able to explore logs and results after the job is finished. \n", |
| 552 | + "And we will be able to explore logs and results after the job is finished.\n", |
543 | 553 | "\n", |
544 | 554 | "```shell\n", |
545 | 555 | "$ cat hello_qiskit.out | grep Exp\n", |
|
550 | 560 | }, |
551 | 561 | { |
552 | 562 | "cell_type": "markdown", |
| 563 | + "id": "5a394dfb-17ce-41a9-85e1-7e4dcff4ef6c", |
553 | 564 | "metadata": {}, |
554 | 565 | "source": [ |
555 | 566 | "### Recap\n", |
|
563 | 574 | "All the code and scripts used in this chapter are available for you within our Github repository. [TempLink](https://github.ibm.com/Christopher-Porter1/hpc-course-demos/blob/main/source/updates.patch)\n", |
564 | 575 | "</div>" |
565 | 576 | ] |
566 | | - }, |
567 | | - { |
568 | | - "cell_type": "markdown", |
569 | | - "metadata": {}, |
570 | | - "source": [] |
571 | 577 | } |
572 | 578 | ], |
573 | 579 | "metadata": { |
574 | 580 | "description": "This lesson introduces classical and quantum computing resources and discusses their scaling.", |
575 | | - "title": "Computing resources", |
576 | 581 | "kernelspec": { |
577 | | - "display_name": "Python 3 (ipykernel)", |
| 582 | + "display_name": "Python 3", |
578 | 583 | "language": "python", |
579 | 584 | "name": "python3" |
580 | 585 | }, |
|
588 | 593 | "name": "python", |
589 | 594 | "nbconvert_exporter": "python", |
590 | 595 | "pygments_lexer": "ipython3", |
591 | | - "version": "3.11.11" |
| 596 | + "version": "3" |
592 | 597 | }, |
| 598 | + "title": "Computing resources", |
593 | 599 | "widgets": { |
594 | 600 | "application/vnd.jupyter.widget-state+json": { |
595 | 601 | "state": {}, |
|
0 commit comments