Further simplifications

config.yaml

@@ -3,15 +3,15 @@
 #------------------------------------------------------------
 
 # Which carpentry is this (swc, dc, lc, or cp)?
-# swc: Software Carpentry
+# swc: Software Carpentry - 
 # dc: Data Carpentry
 # lc: Library Carpentry
 # cp: Carpentries (to use for instructor training for instance)
 # incubator: The Carpentries Incubator
-carpentry: 'incubator'
+carpentry: 'swc'
 
 # Overall title for pages.
-title: 'Performance Profiling & Optimisation (Python)'
+title: 'Python Optimisation and Performance Profiling'
 
 # Date the lesson was created (YYYY-MM-DD, this is empty by default)
 created: 2024-02-01~ # FIXME
@@ -27,13 +27,13 @@ life_cycle: 'alpha'
 license: 'CC-BY 4.0'
 
 # Link to the source repository for this lesson
-source: 'https://github.com/RSE-Sheffield/pando-python'
+source: 'https://github.com/ICR-RSE-Group/carpentry-pando-python'
 
 # Default branch of your lesson
 branch: 'main'
 
 # Who to contact if there are any issues
-contact: 'robert.chisholm@sheffield.ac.uk'
+contact: 'mira.sarkis@icr.ac.uk'
 
 # Navigation ------------------------------------------------
 #
@@ -59,18 +59,17 @@ contact: '[email protected]'
 
 # Order of episodes in your lesson
 episodes: 
-- profiling-introduction.md
-- profiling-functions.md
-- short-break1.md
-- profiling-lines.md
-- profiling-conclusion.md
 - optimisation-introduction.md
 - optimisation-data-structures-algorithms.md
-- long-break1.md
 - optimisation-minimise-python.md
 - optimisation-use-latest.md
 - optimisation-memory.md
 - optimisation-conclusion.md
+- long-break1.md
+- profiling-introduction.md
+- profiling-functions.md
+- profiling-lines.md
+- profiling-conclusion.md
 
 # Information for Learners
 learners: 
@@ -91,5 +90,5 @@ profiles:
 # This space below is where custom yaml items (e.g. pinning
 # sandpaper and varnish versions) should live
 
-varnish: RSE-Sheffield/uos-varnish@main
-url: 'https://rse.shef.ac.uk/pando-python'
+#varnish: RSE-Sheffield/uos-varnish@main
+#url: 'https://icr-rse-group.github.io/carpentry-pando-python'

episodes/long-break1.md

@@ -1,5 +1,5 @@
 ---
-title: Break
+title: Lunch Break
 teaching: 0
 exercises: 0
 break: 60

episodes/optimisation-data-structures-algorithms.md

@@ -65,7 +65,7 @@ CPython for example uses [`newsize + (newsize >> 3) + 6`](https://github.com/pyt
 
 This has two implications:
 
-* If you are creating large static lists, they will use upto 12.5% excess memory.
+* If you are creating large static lists, they will use up to 12.5% excess memory.
 * If you are growing a list with `append()`, there will be large amounts of redundant allocations and copies as the list grows.
 
 ### List Comprehension
@@ -165,7 +165,7 @@ To retrieve or check for the existence of a key within a hashing data structure,
 
 ### Keys
 
-Keys will typically be a core Python type such as a number or string. However multiple of these can be combined as a Tuple to form a compound key, or a custom class can be used if the methods `__hash__()` and `__eq__()` have been implemented.
+Keys will typically be a core Python type such as a number or string. However, multiple of these can be combined as a Tuple to form a compound key, or a custom class can be used if the methods `__hash__()` and `__eq__()` have been implemented.
 
 You can implement `__hash__()` by utilising the ability for Python to hash tuples, avoiding the need to implement a bespoke hash function.
 
@@ -265,7 +265,7 @@ Constructing a set with a loop and `add()` (equivalent to a list's `append()`) c
 
 The naive list approach is 2200x times slower than the fastest approach, because of how many times the list is searched. This gap will only grow as the number of items increases.
 
-Sorting the input list reduces the cost of searching the output list significantly, however it is still 8x slower than the fastest approach. In part because around half of it's runtime is now spent sorting the list.
+Sorting the input list reduces the cost of searching the output list significantly, however it is still 8x slower than the fastest approach. In part because around half of its runtime is now spent sorting the list.
 
 ```output
 uniqueSet: 0.30ms
@@ -280,9 +280,9 @@ uniqueListSort: 2.67ms
 
 Independent of the performance to construct a unique set (as covered in the previous section), it's worth identifying the performance to search the data-structure to retrieve an item or check whether it exists.
 
-The performance of a hashing data structure is subject to the load factor and number of collisions. An item that hashes with no collision can be checked almost directly, whereas one with collisions will probe until it finds the correct item or an empty slot. In the worst possible case, whereby all insert items have collided this would mean checking every single item. In practice, hashing data-structures are designed to minimise the chances of this happening and most items should be found or identified as missing with a single access.
+The performance of a hashing data structure is subject to the load factor and number of collisions. An item that hashes with no collision can be checked almost directly, whereas one with collisions will probe until it finds the correct item or an empty slot. In the worst possible case, whereby all insert items have collided this would mean checking every single item. In practice, hashing data-structures are designed to minimise the chances of this happening and most items should be found or identified as missing with single access.
 
-In contrast if searching a list or array, the default approach is to start at the first item and check all subsequent items until the correct item has been found. If the correct item is not present, this will require the entire list to be checked. Therefore the worst-case is similar to that of the hashing data-structure, however it is guaranteed in cases where the item is missing. Similarly, on-average we would expect an item to be found half way through the list, meaning that an average search will require checking half of the items.
+In contrast, if searching a list or array, the default approach is to start at the first item and check all subsequent items until the correct item has been found. If the correct item is not present, this will require the entire list to be checked. Therefore, the worst-case is similar to that of the hashing data-structure, however it is guaranteed in cases where the item is missing. Similarly, on-average we would expect an item to be found halfway through the list, meaning that an average search will require checking half of the items.
 
 If however the list or array is sorted, a binary search can be used. A binary search divides the list in half and checks which half the target item would be found in, this continues recursively until the search is exhausted whereby the item should be found or dismissed. This is significantly faster than performing a linear search of the list, checking a total of `log N` items every time.
 
@@ -333,9 +333,7 @@ print(f"linear_search_list: {timeit(linear_search_list, number=repeats)-gen_time
 print(f"binary_search_list: {timeit(binary_search_list, number=repeats)-gen_time:.2f}ms")
 ```
 
-Searching the set is fastest performing 25,000 searches in 0.04ms.
-This is  followed by the binary search of the (sorted) list which is 145x slower, although the list has been filtered for duplicates. A list still containing duplicates would be longer, leading to a more expensive search.
-The linear search of the list is more than 56,600x slower than the fastest, it really shouldn't be used!
+Searching the set is the fastest, performing 25,000 searches in 0.04ms. This is followed by the binary search of the (sorted) list which is 145x slower, although the list has been filtered for duplicates. A list still containing duplicates would be longer, leading to a more expensive search. The linear search of the list is more than 56,600x slower than searching the set, it really shouldn't be used!
 
 ```output
 search_set: 0.04ms

episodes/optimisation-introduction.md

@@ -18,57 +18,41 @@ exercises: 0
 
 ## Introduction
 
-<!-- Enable you to look at hotspots identified by compiler, identify whether it's efficient -->
-Now that you're able to find the most expensive components of your code with profiling, it becomes time to learn how to identify whether that expense is reasonable.
-
+<!-- Changing the narrative: you'd better learn how to write a good code an what are the good practice -->
+Think about optimisation as the first step on your journey to writing high-performance code.
+It’s like a race: the faster you can go without taking unnecessary detours, the better. 
+Code optmisation is all about understanding the principles of efficiency in Python and being conscious of how small changes can yield massive improvements.
+
 <!-- Necessary to understand how code executes (to a degree) -->
-In order to optimise code for performance, it is necessary to have an understanding of what a computer is doing to execute it.
+These are the first steps in code optimisation: making better choices as you write your code and have an understanding of what a computer is doing to execute it.
 
 <!-- Goal is to give you a high level understanding of how your code executes. You don't need to be an expert, even a vague general understanding will leave you in a stronger position. -->
-Even a high-level understanding of how you code executes, such as how Python and the most common data-structures and algorithms are implemented, can help you to identify suboptimal approaches when programming. If you have learned to write code informally out of necessity, to get something to work, it's not uncommon to have collected some bad habits along the way.
+A high-level understanding of how your code executes, such as how Python and the most common data-structures and algorithms are implemented, can help you identify suboptimal approaches when programming. If you have learned to write code informally out of necessity, to get something to work, it's not uncommon to have collected some bad habits along the way.
 
 <!-- This is largely high-level/abstract knowledge applicable to the vast majority of programming languages, applies even more strongly if using compiled Python features like numba -->
 The remaining content is often abstract knowledge, that is transferable to the vast majority of programming languages. This is because the hardware architecture, data-structures and algorithms used are common to many languages and they hold some of the greatest influence over performance bottlenecks.
 
-## Premature Optimisation
-
-> Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: **premature optimization is the root of all evil**. Yet we should not pass up our opportunities in that critical 3%. - Donald Knuth
-
-This classic quote among computer scientists states; when considering optimisation it is important to focus on the potential impact, both to the performance and maintainability of the code.
-
-Profiling is a valuable tool in this cause. Should effort be expended to optimise a component which occupies 1% of the runtime? Or would that time be better spent focusing on the most expensive components?
-
-Advanced optimisations, mostly outside the scope of this course, can increase the cost of maintenance by obfuscating what code is doing. Even if you are a solo-developer working on private code, your future self should be able to easily comprehend your implementation.
+## Optimising code from scratch: trade-off between performance and maintainability
 
-Therefore, the balance between the impact to both performance and maintainability should be considered when optimising code.
+> Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: **premature optimisation is the root of all evil**. Yet we should not pass up our opportunities in that critical 3%. - Donald Knuth
 
-This is not to say, don't consider performance when first writing code. The selection of appropriate algorithms and data-structures covered in this course form good practice, simply don't fret over a need to micro-optimise every small component of the code that you write.
+This classic quote among computer scientists states; when considering optimisation it is important to focus on the potential impact, both to the performance and maintainability of the code. Advanced optimisations, mostly outside the scope of this course, can increase the cost of maintenance by obfuscating what code is doing. Even if you are a solo-developer working on private code, your future self should be able to easily comprehend your implementation. Therefore, the balance between the impact to both performance and maintainability should be considered when optimising code.
 
+This is not to say, don't consider performance when first writing code. The selection of appropriate algorithms and data-structures covered in this course form a good practice, simply don't fret over a need to micro-optimise every small component of the code that you write.
 
-## Ensuring Reproducible Results
+## Ensuring Reproducible Results when optimising an existing code
 
 <!-- This is also good practice when optimising your code, to ensure mistakes aren't made -->
-When optimising your code, you are making speculative changes. It's easy to make mistakes, many of which can be subtle. Therefore, it's important to have a strategy in place to check that the outputs remain correct.
+When optimising an existing code, you are making speculative changes. It's easy to make mistakes, many of which can be subtle. Therefore, it's important to have a strategy in place to check that the outputs remain correct.
 
-Testing is hopefully already a seamless part of your research software development process.
-Test can be used to clarify how your software should perform, ensuring that new features work as intended and protecting against unintended changes to old functionality.
-
-There are a plethora of methods for testing code.
+Testing is hopefully already a seamless part of your research software development process. Test can be used to clarify how your software should perform, ensuring that new features work as intended and protecting against unintended changes to old functionality.
 
 ## pytest Overview
 
-Most Python developers use the testing package [pytest](https://docs.pytest.org/en/latest/), it's a great place to get started if you're new to testing code.
+There are a plethora of methods for testing code. Most Python developers use the testing package [pytest](https://docs.pytest.org/en/latest/), it's a great place to get started if you're new to testing code. Tests should be created within a project's testing directory, by creating files named with the form `test_*.py` or `*_test.py`. pytest looks for these files, when running the test suite. Within the created test file, any functions named in the form `test*` are considered tests that will be executed by pytest. The `assert` keyword is used, to test whether a condition evaluates to `True`.
 
 Here's a quick example of how a test can be used to check your function's output against an expected value.
 
-Tests should be created within a project's testing directory, by creating files named with the form `test_*.py` or `*_test.py`.
-
-pytest looks for these files, when running the test suite.
-
-Within the created test file, any functions named in the form `test*` are considered tests that will be executed by pytest.
-
-The `assert` keyword is used, to test whether a condition evaluates to `True`.
-
 ```python
 # file: test_demonstration.py