Merge pull request #401 from realpython/python-312-perf-profiler

Python 3.12 Preview: perf Profiler - Initial Commit (Materials)

Author: KateFinegan

python-311/README.md

@@ -50,7 +50,7 @@ This section only contains brief instructions on how you can run the examples. S
 
 ### Improved Error Messages
 
-Run [`inverse.py](inverse.py) for a quick look at the improved tracebacks:
+Run [`inverse.py`](inverse.py) for a quick look at the improved tracebacks:
 
 ```console
 (venv) $ python inverse.py 

python-312/README.md

@@ -8,9 +8,12 @@ You need Python 3.12 installed to run these examples. See the following tutorial
 
 - [How Can You Install a Pre-Release Version of Python](https://realpython.com/python-pre-release/)
 
+Note that for the `perf` support, you'll need to build Python from source code with additional compiler flags enabled, as [explained below](#support-for-the-linux-perf-profiler).
+
 You can learn more about Python 3.12's new features in the following Real Python tutorials:
 
 - [Python 3.12 Preview: Ever Better Error Messages](https://realpython.com/python312-error-messages/)
+- [Python 3.12 Preview: Support For the Linux `perf` Profiler](https://realpython.com/python312-perf-profiler/)
 
 You'll find examples from all these tutorials in this repository.
 
@@ -20,7 +23,7 @@ This section only contains brief instructions on how you can run the examples. S
 
 ### Improved Error Messages
 
-Run [`encoder.py](encoder.py) to create an encoded message like the one shown in the tutorial. You can decode the message using [`decoder.py](decoder.py).
+Run [`encoder.py`](error-messages/encoder.py) to create an encoded message like the one shown in the tutorial. You can decode the message using [`decoder.py`](error-messages/decoder.py).
 
 You can swap the import statement to `import d from this` in either of the files to encounter the improved error message:
 
@@ -32,14 +35,93 @@ You can swap the import statement to `import d from this` in either of the files
 SyntaxError: Did you mean to use 'from ... import ...' instead?
 ```
 
-In [`local_self.py`](local_self.py), you can see a naive reproduction of another improved error message. Pick apart the example code to learn more about how this was implemented in Python 3.12. 
+In [`local_self.py`](error-messages/local_self.py), you can see a naive reproduction of another improved error message. Pick apart the example code to learn more about how this was implemented in Python 3.12. 
 
 See [Ever Better Error Messages in Python 3.12](https://realpython.com/python312-error-messages/) for more information.
 
+### Support For the Linux `perf` Profiler
+
+#### Setting Up
+
+You'll need to download, build, and install Python 3.12 from the source code with the frame pointer optimizations disabled:
+
+```shell
+$ git clone --branch v3.12.0b2 https://github.com/python/cpython.git
+$ cd cpython/
+$ export CFLAGS='-fno-omit-frame-pointer -mno-omit-leaf-frame-pointer'
+$ ./configure --prefix="$HOME/python-custom-build"
+$ make -j $(nproc)
+$ make install
+```
+
+Create and activate a new virtual environment based on Python 3.12:
+
+```shell
+$ "$HOME/python-custom-build/bin/python3" -m venv venv/ --prompt 'py3.12-custom'
+$ source venv/bin/activate
+```
+
+Install dependencies from the `requirements.txt` file into your virtual environment:
+
+```shell
+(py3.12-custom) $ python -m pip install -r requirements.txt
+```
+
+#### Using `perf` With Python
+
+Record Samples:
+
+```shell
+$ cd perf-profiler/
+$ sudo perf record -g -F max ../venv/bin/python -X perf benchmark.py
+```
+
+Display reports:
+
+```shell
+$ cd perf-profiler/
+$ sudo perf report
+$ sudo perf report --stdio -g
+$ sudo perf report --hierarchy --verbose --call-graph fractal --sort sample,dso
+```
+
+#### Rendering Flame Graphs
+
+Download Perl scripts and add them to your `$PATH` environment variable:
+
+```shell
+$ git clone [email protected]:brendangregg/FlameGraph.git
+$ export PATH="$(pwd)/FlameGraph:$PATH"
+```
+
+Generate the flame graph and save it to a local file:
+
+```shell
+$ cd perf-profiler/
+$ sudo perf script | stackcollapse-perf.pl | flamegraph.pl > flamegraph.svg
+```
+
+Open the flame graph in your default SVG viewer: (Use your web browser for the interactive features.)
+
+```shell
+$ xdg-open flamegraph.svg
+```
+
+Produce a pure-Python flame graph by filtering and processing the collapsed stack traces:
+
+```shell
+$ sudo perf script | stackcollapse-perf.pl > traces.txt
+$ cat traces.txt | python censor.py -m benchmark,PIL > traces_censored.txt
+$ cat traces_censored.txt | flamegraph.pl --minwidth 10 > flamegraph.svg
+```
+
+See [Support For the Linux `perf` Profiler in Python 3.12](https://realpython.com/python312-perf-profiler/) for more information.
+
 ## Authors
 
 - **Martin Breuss**, E-mail: [[email protected]]([email protected])
 - **Bartosz Zaczyński**, E-mail: [[email protected]]([email protected])
+- **Leodanis Pozo Ramos**, E-mail: [[email protected]]([email protected])
 - **Geir Arne Hjelle**, E-mail: [[email protected]]([email protected])
 
 ## License

python-312/decoder.py python-312/error-messages/decoder.pypython-312/decoder.py renamed to python-312/error-messages/decoder.py

@@ -0,0 +1,50 @@
+import sys
+import sysconfig
+from math import exp, log
+from statistics import mean
+
+from PIL import Image
+
+
+def check_perf_support():
+    if sys.version_info < (3, 12):
+        version = sysconfig.get_python_version()
+        raise RuntimeError(f"This is Python {version}, not 3.12 or later")
+
+    if not sysconfig.get_config_var("PY_HAVE_PERF_TRAMPOLINE"):
+        raise RuntimeError("Python doesn't support perf on this platform")
+
+    if not sys.is_stack_trampoline_active():
+        raise RuntimeError("Did you forget the '-X perf' option?")
+
+    cflags = sysconfig.get_config_var("CONFIGURE_CFLAGS")
+    if "-fno-omit-frame-pointer" not in cflags:
+        print("Python compiled without the frame pointer", file=sys.stderr)
+
+
+def main():
+    image = Image.open("image.jpg")
+    print("luminance =", get_average_luminance(image.getdata()))
+    image.show()
+
+
+def get_average_luminance(pixels):
+    return exp(mean(log(luminance(pixel) + 1e-9) for pixel in pixels))
+
+
+def luminance(pixel):
+    red, green, blue = tuple(linearize(c) for c in pixel)
+    return 0.2126 * red + 0.7152 * green + 0.0722 * blue
+
+
+def linearize(channel, gamma=2.2):
+    return (channel / 255) ** gamma
+
+
+if __name__ == "__main__":
+    try:
+        check_perf_support()
+    except RuntimeError as error:
+        print(error, file=sys.stderr)
+    else:
+        main()

python-312/encoder.py python-312/error-messages/encoder.pypython-312/encoder.py renamed to python-312/error-messages/encoder.py

@@ -0,0 +1,53 @@
+import sys
+import sysconfig
+from math import exp, log
+
+from PIL import Image
+
+
+def check_perf_support():
+    if sys.version_info < (3, 12):
+        version = sysconfig.get_python_version()
+        raise RuntimeError(f"This is Python {version}, not 3.12 or later")
+
+    if not sysconfig.get_config_var("PY_HAVE_PERF_TRAMPOLINE"):
+        raise RuntimeError("Python doesn't support perf on this platform")
+
+    if not sys.is_stack_trampoline_active():
+        raise RuntimeError("Did you forget the '-X perf' option?")
+
+    cflags = sysconfig.get_config_var("CONFIGURE_CFLAGS")
+    if "-fno-omit-frame-pointer" not in cflags:
+        print("Python compiled without the frame pointer", file=sys.stderr)
+
+
+def main():
+    image = Image.open("image.jpg")
+    print("luminance =", get_average_luminance(image.getdata()))
+    image.show()
+
+
+def get_average_luminance(pixels):
+    return exp(mean([log(luminance(pixel) + 1e-9) for pixel in pixels]))
+
+
+def mean(n):
+    return sum(n) / len(n)
+
+
+def luminance(pixel):
+    red, green, blue = tuple(linearize(c) for c in pixel)
+    return 0.2126 * red + 0.7152 * green + 0.0722 * blue
+
+
+def linearize(channel, gamma=2.2):
+    return (channel / 255) ** gamma
+
+
+if __name__ == "__main__":
+    try:
+        check_perf_support()
+    except RuntimeError as error:
+        print(error, file=sys.stderr)
+    else:
+        main()

python-312/local_self.py python-312/error-messages/local_self.pypython-312/local_self.py renamed to python-312/error-messages/local_self.py

@@ -0,0 +1,35 @@
+import sys
+from argparse import ArgumentParser
+
+
+def main(args):
+    censorship = censor(args.modules)
+    for line in sys.stdin:
+        stack_trace, num_samples = line.rsplit(maxsplit=1)
+        symbols = filter(censorship, stack_trace.split(";"))
+        censored_stack_trace = ";".join(symbols)
+        if censored_stack_trace:
+            print(censored_stack_trace, num_samples)
+
+
+def parse_args():
+    parser = ArgumentParser()
+    parser.add_argument(
+        "-m", "--modules", default=[], type=lambda s: s.split(",")
+    )
+    return parser.parse_args()
+
+
+def censor(modules):
+    def is_valid(symbol):
+        if not symbol.startswith("py::"):
+            return False
+        if modules:
+            return any(module in symbol for module in modules)
+        return True
+
+    return is_valid
+
+
+if __name__ == "__main__":
+    main(parse_args())

python-312/perf-profiler/benchmark.py

@@ -0,0 +1 @@
+Pillow==9.5.0