Skip to content

Commit 81e6016

Browse files
gjbexgjbex
committed
Adapt for Cython 3.x
1 parent c9e1ac0 commit 81e6016

File tree

8 files changed

+414
-17
lines changed

8 files changed

+414
-17
lines changed

‎source-code/cython/Primes/Makefile

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,12 @@
1-
VERSION = cpython-34m
2-
PRIMES_LIB = primes_cython.$(VERSION).so
1+
VERSION = cpython-311-x86_64-linux-gnu
2+
PRIMES_LIBS = primes_cython.$(VERSION).so primes_pure_python.$(VERSION).so
33

4-
all: $(PRIMES_LIB)
4+
all: $(PRIMES_LIBS)
55

6-
$(PRIMES_LIB): primes_cython.pyx
6+
$(PRIMES_LIBS): primes_cython.pyx primes_pure_python.py
77
python setup.py build_ext --inplace
88

99
clean:
1010
python setup.py clean
11-
rm -f primes_cython.c $(PRIMES_LIB)
11+
$(RM) primes_cython.c primes_pure_python.c $(PRIMES_LIBS)
12+
$(RM) -r build/

‎source-code/cython/Primes/README.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,14 @@ Motivating example for using Cython
33

44
## What is it?
55
1. `primes.py`: Python script to run either the Python, or the Cython
6-
version of copmuting a list of prime numbers less than the given
7-
number.
6+
version (both pyx and pure python) of copmuting a list of prime
7+
numbers less than the given number.
88
1. `primes_vanilla.py`: pure Python implementation of the primes function.
99
1. `primes_cython.pyx`: Cython implementation of the primes function.
10+
1. `primes_pure_python.py`: Cython pure Python implementation of the
11+
primes function.
1012
1. `setup.py`: Python build script.
1113
1. `Makefile`: make file to build the extension.
14+
1. `time_all.sh`: timings with hyperfine, **Note:** due to short runtimes
15+
these timings are donated by Python interpreter startup times and
16+
are hence skewed.

‎source-code/cython/Primes/primes.ipynb

Lines changed: 362 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,362 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "137ce0ef-cbea-4900-b764-0af2b5e98d3d",
6+
"metadata": {},
7+
"source": [
8+
"# Vanilla Python"
9+
]
10+
},
11+
{
12+
"cell_type": "markdown",
13+
"id": "740733be-9eba-4300-b20c-ff1e6a125b9c",
14+
"metadata": {},
15+
"source": [
16+
"To illustrate Cython, you can consider the following Python function that will compute the first `n` primes and return them as a list."
17+
]
18+
},
19+
{
20+
"cell_type": "code",
21+
"execution_count": 1,
22+
"id": "4c383365-765d-41f3-9913-e93e3a580e42",
23+
"metadata": {
24+
"tags": []
25+
},
26+
"outputs": [
27+
{
28+
"name": "stdout",
29+
"output_type": "stream",
30+
"text": [
31+
"def primes(kmax):\n",
32+
" p = [0]*1000\n",
33+
" result = []\n",
34+
" if kmax > 1000:\n",
35+
" kmax = 1000\n",
36+
" k = 0\n",
37+
" n = 2\n",
38+
" while k < kmax:\n",
39+
" i = 0\n",
40+
" while i < k and n % p[i] != 0:\n",
41+
" i = i + 1\n",
42+
" if i == k:\n",
43+
" p[k] = n\n",
44+
" k = k + 1\n",
45+
" result.append(n)\n",
46+
" n = n + 1\n",
47+
" return result\n"
48+
]
49+
}
50+
],
51+
"source": [
52+
"%cat primes_vanilla.py"
53+
]
54+
},
55+
{
56+
"cell_type": "markdown",
57+
"id": "df0fb7bd-3ae9-4e41-9986-39a41769628b",
58+
"metadata": {},
59+
"source": [
60+
"You can import the module and call the function using the `%timeit` magic to establish a baseline timning."
61+
]
62+
},
63+
{
64+
"cell_type": "code",
65+
"execution_count": 2,
66+
"id": "9057c260-3897-4e18-8be0-d9c1a3085892",
67+
"metadata": {
68+
"tags": []
69+
},
70+
"outputs": [],
71+
"source": [
72+
"import primes_vanilla"
73+
]
74+
},
75+
{
76+
"cell_type": "code",
77+
"execution_count": 3,
78+
"id": "a18ab705-dae5-47a0-bada-bcc857b921c9",
79+
"metadata": {
80+
"tags": []
81+
},
82+
"outputs": [
83+
{
84+
"name": "stdout",
85+
"output_type": "stream",
86+
"text": [
87+
"17.8 ms ± 104 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
88+
]
89+
}
90+
],
91+
"source": [
92+
"%timeit primes_vanilla.primes(1000)"
93+
]
94+
},
95+
{
96+
"cell_type": "markdown",
97+
"id": "88d2944f-0f05-4271-94eb-9edd5c602001",
98+
"metadata": {},
99+
"source": [
100+
"# Cython `.pyx` files"
101+
]
102+
},
103+
{
104+
"cell_type": "markdown",
105+
"id": "4d062abf-fad1-459f-920e-acb6a8b73753",
106+
"metadata": {},
107+
"source": [
108+
"A first approach to speed up this computation is rewriting this function in Cython. You can review the source code below."
109+
]
110+
},
111+
{
112+
"cell_type": "code",
113+
"execution_count": 4,
114+
"id": "7fadfc46-11b6-40be-a541-f14e6fa1d995",
115+
"metadata": {
116+
"tags": []
117+
},
118+
"outputs": [
119+
{
120+
"name": "stdout",
121+
"output_type": "stream",
122+
"text": [
123+
"def primes(int kmax):\n",
124+
" cdef int n, k, i\n",
125+
" cdef int p[1000]\n",
126+
" result = []\n",
127+
" if kmax > 1000:\n",
128+
" kmax = 1000\n",
129+
" k = 0\n",
130+
" n = 2\n",
131+
" while k < kmax:\n",
132+
" i = 0\n",
133+
" while i < k and n % p[i] != 0:\n",
134+
" i = i + 1\n",
135+
" if i == k:\n",
136+
" p[k] = n\n",
137+
" k = k + 1\n",
138+
" result.append(n)\n",
139+
" n = n + 1\n",
140+
" return result\n"
141+
]
142+
}
143+
],
144+
"source": [
145+
"%cat primes_cython.pyx"
146+
]
147+
},
148+
{
149+
"cell_type": "markdown",
150+
"id": "784f09a9-c76c-4601-8250-0f66349683ed",
151+
"metadata": {},
152+
"source": [
153+
"As you can see, the only changes to the original function are\n",
154+
"* the declarations of the types for the function's argument,\n",
155+
"* the declaration of the type of the variables `n`, `k`, `i`, and\n",
156+
"* replacing the `p` Python array by a C array of `int`.\n",
157+
"\n",
158+
"This code first needs to be compiled before it can be run. Fortunately, this can easily be done from a Jupyter notebook by using the `pyximport` module. The `install` function will ensure that for `.pyx` files, the `import` defined by `pyximport` will be used. We also specify the `language_level` to Python 3."
159+
]
160+
},
161+
{
162+
"cell_type": "code",
163+
"execution_count": 5,
164+
"id": "333e6db5-da37-483a-8287-34b299ba2cd0",
165+
"metadata": {
166+
"tags": []
167+
},
168+
"outputs": [],
169+
"source": [
170+
"import pyximport\n",
171+
"pyximport.install(pyximport=True, pyimport=True, language_level='3str');"
172+
]
173+
},
174+
{
175+
"cell_type": "markdown",
176+
"id": "3ef79c3b-e8d4-4a07-a361-02e6979dcd7a",
177+
"metadata": {},
178+
"source": [
179+
"Now you can import the Cython module that implements the `primes` function and time it for comparison with the vanilla Python implementation."
180+
]
181+
},
182+
{
183+
"cell_type": "code",
184+
"execution_count": 6,
185+
"id": "70ae637a-6a45-4def-baa2-20eaabadc448",
186+
"metadata": {
187+
"tags": []
188+
},
189+
"outputs": [],
190+
"source": [
191+
"import primes_cython"
192+
]
193+
},
194+
{
195+
"cell_type": "code",
196+
"execution_count": 7,
197+
"id": "9195d340-9a0f-41c1-a450-e4b66f155c05",
198+
"metadata": {
199+
"tags": []
200+
},
201+
"outputs": [
202+
{
203+
"name": "stdout",
204+
"output_type": "stream",
205+
"text": [
206+
"1.9 ms ± 2.25 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)\n"
207+
]
208+
}
209+
],
210+
"source": [
211+
"%timeit primes_cython.primes(1000)"
212+
]
213+
},
214+
{
215+
"cell_type": "markdown",
216+
"id": "2587cac1-4c24-4329-9a48-ad6f6b77202f",
217+
"metadata": {},
218+
"source": [
219+
"It is quite clear that the speedup is considerable for very little effort on your part."
220+
]
221+
},
222+
{
223+
"cell_type": "markdown",
224+
"id": "9f083fc0-d5a1-4fdf-9889-407c26c7cd48",
225+
"metadata": {
226+
"tags": []
227+
},
228+
"source": [
229+
"# Pure Python & Cython"
230+
]
231+
},
232+
{
233+
"cell_type": "markdown",
234+
"id": "99120581-7942-4949-8111-51aaeec6ee58",
235+
"metadata": {},
236+
"source": [
237+
"It is however also possible to use pure Python with type annotations to get a similar result."
238+
]
239+
},
240+
{
241+
"cell_type": "code",
242+
"execution_count": 8,
243+
"id": "0785d966-aec8-4a7c-95bc-4f305109ae3b",
244+
"metadata": {
245+
"tags": []
246+
},
247+
"outputs": [
248+
{
249+
"name": "stdout",
250+
"output_type": "stream",
251+
"text": [
252+
"import cython\n",
253+
"\n",
254+
"def primes(nb_primes: cython.int):\n",
255+
" i: cython.int\n",
256+
" p: cython.int[1000]\n",
257+
"\n",
258+
" if nb_primes > 1000:\n",
259+
" nb_primes = 1000\n",
260+
"\n",
261+
" if not cython.compiled: # Only if regular Python is running\n",
262+
" p = [0] * 1000 # Make p work almost like a C array\n",
263+
"\n",
264+
" len_p: cython.int = 0 # The current number of elements in p.\n",
265+
" n: cython.int = 2\n",
266+
" while len_p < nb_primes:\n",
267+
" # Is n prime?\n",
268+
" for i in p[:len_p]:\n",
269+
" if n % i == 0:\n",
270+
" break\n",
271+
"\n",
272+
" # If no break occurred in the loop, we have a prime.\n",
273+
" else:\n",
274+
" p[len_p] = n\n",
275+
" len_p += 1\n",
276+
" n += 1\n",
277+
"\n",
278+
" # Let's copy the result into a Python list:\n",
279+
" result_as_list = [prime for prime in p[:len_p]]\n",
280+
" return result_as_list"
281+
]
282+
}
283+
],
284+
"source": [
285+
"%cat primes_pure_python.py"
286+
]
287+
},
288+
{
289+
"cell_type": "markdown",
290+
"id": "a208daab-efa3-4d00-b327-8c71ce01591e",
291+
"metadata": {},
292+
"source": [
293+
"Note that\n",
294+
"* the `cython` module has to be imported,\n",
295+
"* the Cython types such as `cython.int` have to be specified, rather than `int`,\n",
296+
"* you can check whether the Python function has been compiled using `cython.compiled`."
297+
]
298+
},
299+
{
300+
"cell_type": "code",
301+
"execution_count": 9,
302+
"id": "338d4447-e6c6-4335-8715-102c19fa5bf0",
303+
"metadata": {
304+
"tags": []
305+
},
306+
"outputs": [],
307+
"source": [
308+
"import primes_pure_python"
309+
]
310+
},
311+
{
312+
"cell_type": "code",
313+
"execution_count": 10,
314+
"id": "a572342b-b458-463f-9b4f-9add15fabbf9",
315+
"metadata": {
316+
"tags": []
317+
},
318+
"outputs": [
319+
{
320+
"name": "stdout",
321+
"output_type": "stream",
322+
"text": [
323+
"1.9 ms ± 2.55 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)\n"
324+
]
325+
}
326+
],
327+
"source": [
328+
"%timeit primes_pure_python.primes(1000)"
329+
]
330+
},
331+
{
332+
"cell_type": "markdown",
333+
"id": "c1b1698a-e81d-4030-b3d7-1b39ba7e241c",
334+
"metadata": {},
335+
"source": [
336+
"The performance is almost identical to that of the `.pyx` file, and the code is pure Python."
337+
]
338+
}
339+
],
340+
"metadata": {
341+
"kernelspec": {
342+
"display_name": "Python 3 (ipykernel)",
343+
"language": "python",
344+
"name": "python3"
345+
},
346+
"language_info": {
347+
"codemirror_mode": {
348+
"name": "ipython",
349+
"version": 3
350+
},
351+
"file_extension": ".py",
352+
"mimetype": "text/x-python",
353+
"name": "python",
354+
"nbconvert_exporter": "python",
355+
"pygments_lexer": "ipython3",
356+
"version": "3.11.5"
357+
},
358+
"toc-autonumbering": true
359+
},
360+
"nbformat": 4,
361+
"nbformat_minor": 5
362+
}

0 commit comments

Comments
 (0)