Merge branch 'hjwp:main' into chapter-11-notes-starting-from-upstream

Author: zipperer

ER_sampleTOC.html

@@ -0,0 +1,50 @@
+<section data-type="preface">
+<h1>Brief Table of Contents (<em>Not Yet Final</em>)</h1>
+
+<p>Preface (AVAILABLE)</p>
+<p>Prerequisites and Assumptions (AVAILABLE)</p>
+<p>Companion Video (AVAILABLE)</p>
+<p><em>Acknowledgments (UNAVAILABLE)</em></p>
+<p>Part 1: The Basics of TDD and Django (AVAILABLE)</p>
+<p>Chapter 1: Getting Django Set Up Using a Functional Test (AVAILABLE)</p>
+<p>Chapter 2: Extending Our Functional Test Using the unittest Module (AVAILABLE)</p>
+<p>Chapter 3: Testing a Simple Home Page with Unit Tests (AVAILABLE)</p>
+<p>Chapter 4: What Are We Doing with All These Tests? (And, Refactoring) (AVAILABLE)</p>
+<p>Chapter 5: Saving User Input: Testing the Database (AVAILABLE)</p>
+<p>Chapter 6: Improving Functional Tests: Ensuring Isolation and Removing Voodoo Sleeps (AVAILABLE)</p>
+<p>Chapter 7: Working Incrementally (AVAILABLE)</p>
+<p>Part 2: Web Development Sine Qua Nons (AVAILABLE)</p>
+<p>Chapter 8: Prettification: Layout and Styling, and What to Test About It (AVAILABLE)</p>
+<p>Chapter 9: Deployment Part 1: Containerization akaDocker (AVAILABLE)</p>
+<p>Chapter 10: Getting to a Production-Ready Deployment (AVAILABLE)</p>
+<p>Chapter 11: Infrastructure As Code: Automated Deployments With Ansible (AVAILABLE)</p>
+<p><em>Chapter 12: Splitting Our Tests into Multiple Files, and a Generic Wait Helper (UNAVAILABLE)</em></p>
+<p><em>Chapter 13: Validation at the Database Layer (UNAVAILABLE)</em></p>
+<p><em>Chapter 14: A Simple Form (UNAVAILABLE)</em></p>
+<p><em>Chapter 15: More Advanced Forms (UNAVAILABLE)</em></p>
+<p><em>Chapter 16: Dipping Our Toes, Very Tentatively, into JavaScript (UNAVAILABLE)</em></p>
+<p><em>Chapter 17: Deploying Our New Code (UNAVAILABLE)</em></p>
+<p><em>Part 3: More Advanced Topics in Testing (UNAVAILABLE)</em></p>
+<p><em>Chapter 18: User Authentication, Spiking, and De-Spiking (UNAVAILABLE)</em></p>
+<p><em>Chapter 19: Using Mocks to Test External Dependencies or Reduce Duplication (UNAVAILABLE)</em></p>
+<p><em>Chapter 20: Test Fixtures and a Decorator for Explicit Waits (UNAVAILABLE)</em></p>
+<p><em>Chapter 21: Server-Side Debugging (UNAVAILABLE)</em></p>
+<p><em>Chapter 22: Finishing “My Lists”: Outside-In TDD (UNAVAILABLE)</em></p>
+<p><em>Chapter 23: Test Isolation, and “Listening to Your Tests” (UNAVAILABLE)</em></p>
+<p><em>Chapter 24: Continuous Integration (CI) (UNAVAILABLE)</em></p>
+<p><em>Chapter 25: The Token Social Bit, the Page Pattern, and an Exercise for the Reader (UNAVAILABLE)</em></p>
+<p><em>Chapter 26: Fast Tests, Slow Tests, and Hot Lava (UNAVAILABLE)</em></p>
+<p><em>Back Matter: Obey the Testing Goat! (UNAVAILABLE)</em></p>
+<p><em>App A: PythonAnywhere (UNAVAILABLE)</em></p>
+<p><em>App B: Django Class-Based Views (UNAVAILABLE)</em></p>
+<p><em>App C: Provisioning with Ansible (UNAVAILABLE)</em></p>
+<p><em>App D: Testing Database Migrations (UNAVAILABLE)</em></p>
+<p><em>App E: Behaviour-Driven Development (BDD) (UNAVAILABLE)</em></p>
+<p><em>App F: Building a REST API: JSON, Ajax, and Mocking with JavaScript (UNAVAILABLE)</em></p>
+<p><em>App G: Django-Rest-Framework (UNAVAILABLE)</em></p>
+<p><em>App H: Cheat Sheet (UNAVAILABLE)</em></p>
+<p><em>App I: What to Do Next (UNAVAILABLE)</em></p>
+<p><em>App J: Source Code Examples (UNAVAILABLE)</em></p>
+<p><em>Bibliography (UNAVAILABLE)</em></p>
+
+</section>

Makefile

@@ -52,7 +52,7 @@ build: $(HTML_PAGES) $(TMPDIR)
 
 .PHONY: install
 install: .venv/bin
-	which brew && brew install asciidoctor || apt install -y asciidoctor
+	which brew && brew install asciidoctor tree || apt install -y asciidoctor tree
 
 .PHONY: update-submodules
 update-submodules:

chapter_08_prettification.asciidoc

@@ -88,7 +88,7 @@ Here are a few things we might want:
 //      including Django documentation or learnpython.org
 //      Both look completely different today - not so ugly anymore
 
-* A nice large input field for adding new and existing lists
+* A nice large input field for adding to new and existing lists
 * A large, attention-grabbing, centered box to put it in
 
 ((("aesthetics, testing", seealso="design and layout testing")))

chapter_09_docker.asciidoc

@@ -61,7 +61,7 @@ and this is a very new draft.
 What that means is that I'd, really, really love feedback from readers.
 Please have a go at following along and let me know what you think!
 I'm [email protected], or you can open up
-https://github.com/hjwp/Book-TDD-Web-Dev-Python/issues[GitHub Issue]
+https://github.com/hjwp/Book-TDD-Web-Dev-Python/issues[GitHub Issues]
 or Pull Requests.
 
 Let me know how you got on, if you get stuck on anything,
@@ -861,6 +861,7 @@ The docker daemon lets you list all the currently running containers
 with `docker ps`:
 
 [role="skipme small-code"]
+[subs="quotes"]
 ----
 $ *docker ps*
 CONTAINER ID   IMAGE        COMMAND                  CREATED         STATUS         PORTS     NAMES
@@ -873,8 +874,9 @@ and a randomly-generated name (you can override that if you want to).
 We can use the ID or the name to kill the container with `docker kill`:
 
 [role="skipme"]
+[subs="quotes"]
 ----
-$ docker kill 0818e1b8e9bf
+$ *docker kill 0818e1b8e9bf*
 0818e1b8e9bf
 ----
 
@@ -904,7 +906,7 @@ What's going on here?  Time for a little debugging.
 
 
 
-=== Debugging a Container Networking Problems
+=== Debugging Container Networking Problems
 
 First, let's try and take a look ourselves, in our browser, by going to http://localhost:8888/:
 
@@ -955,7 +957,9 @@ Starting development server at http://127.0.0.1:8888/
 
 A quick run of the FT or check in our browser will show us that nope, that doesn't work either.
 Let's try an even lower-level smoke test, the traditional Unix utility `curl`.
-It's a command-line tool for making HTTP requests.  Try it on your own computer first:
+It's a command-line tool for making HTTP requests.footnote:[
+`curl` can do FTP and many other types of network requests too! Check out the https://man7.org/linux/man-pages/man1/curl.1.html[curl manual].]
+Try it on your own computer first:
 
 [role="ignore-errors"]
 [subs="specialcharacters,macros"]
@@ -1484,7 +1488,7 @@ Tests and small steps take some of the uncertainty out of deployment::
 
 Some typical pain points--networking, ports, static files, and the database::
     Moving from the local django development server to a container
-    is chance to rehearse the fiddliness of configuring networking
+    is a chance to rehearse the fiddliness of configuring networking
     in a deployed environment.
     It's also a chance to think about persistence and the database,
     and some configuration issues like static files.

chapter_10_production_readiness.asciidoc

@@ -800,6 +800,7 @@ RUN addgroup --system nonroot && adduser --system --group nonroot
 
 USER nonroot
 ----
+====
 
 
 === Configuring logging

chapter_11_ansible.asciidoc

@@ -6,6 +6,15 @@ ______________________________________________________________
 Automate, automate, automate.
 ______________________________________________________________
 
+.A Note for Early Release Readers
+****
+With Early Release ebooks, you get books in their earliest form—the author's raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.
+
+This will be the 11th chapter of the final book. The GitHub repo is available at https://github.com/hjwp/book-example.
+
+If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the author at [email protected].
+****
+
 // RITA: In this intro, please mention that you'll be using Ansible. Not only will it prep the reader, but it'll set you up to say "(or "become" in Ansible terminology)" in the User Accounts, SSH, and Privileges section before we actually get to the Ansible section.
 ((("deployment", "automating with Ansible", id="Dfarbric11")))
 ((("infrastructure as code")))
@@ -17,6 +26,14 @@ We _could_ do all these things manually,
 but a key insight of the modern infrastructure management
 is that automation really pays off in reducing maintenance burdens.
 
+// SEBASTIAN: IMO, above statement is not strong enough (my opinion)
+//  I'd add something that software development nowadays is made in short cycles with frequent deployments
+//  and automation is a MUST - the earlier one invests into it, the faster they can focus on doing what's
+//  actually giving value.
+//
+//  ALSO... to reassure readers - automating anything requires deep understanding, so they will not miss anything.
+//  Perhaps this is even more important. When I first read this, I was like "ohhh, I'm gonna miss some fun!"
+
 It's also key to making sure our tests give us true confidence over our deployments.
 If we go to the trouble of building a staging server,footnote:[
 What I'm calling a "staging" server, some people would
@@ -67,9 +84,26 @@ A few small things:
   billed needlessly.
 ////
 
+////
+SEBASTIAN overall notes
+All in all, I am not very fond of the current shape of this chapter.
 
-.🚧 Warning, chapter under construction 🚧
-*******************************************************************************
+The summary is great and the choice of technologies is the best I can imagine.
+
+However, I got lost several times while reading through the chapter.
+There are too many open loops. For example, SSH is mentioned but then we jump into all other technologies without seeing what SSH is and how it will play with the rest.
+
+mentioning of too many technologies (e.g. Puppet/Chef - IMHO not necessary in 2024).
+
+I think (my opinion) the chapter needs reorganizing so that readers can more quickly see a given piece of tech in action. I'd cut some content.
+
+If you like some more specific suggestions, I can spend more time and provide them.
+
+Also, it's mentioned that the server will be provisioned manually without automation, but then we get ansible infra/ansible-provision.yaml. I must say I'm not following when provisioning starts and where it ends. In my book (figuratively speaking), installing docker falls under the definition of provisioning (which was meant NOT to be automated) while the remaining steps, like exporting and reimporting docker image are something different (deployment?).
+////
+
+.Warning, chapter under construction
+****
 As part of my work on the third edition of the book,
 I'm making big changes to the deployment chapters.
 This chapter is still very fresh, but the content is all there,
@@ -81,8 +115,7 @@ https://github.com/hjwp/Book-TDD-Web-Dev-Python/issues[GitHub Issues]
 and Pull Requests.
 
 I hope you enjoy the new version!
-
-*******************************************************************************
+****
 
 
 === Getting a Domain Name
@@ -152,6 +185,7 @@ They're unlikely to ever go away,
 and knowing a bit about them will get you some respect
 from all the grizzled dinosaurs out there.
 
+// SEBASTIAN: Nice, and seem-to-be timeless choice!
 
 
 ==== Spinning Up a Server
@@ -309,10 +343,10 @@ rather than specifying a procedural series of steps to be followed one by one.
 
 ==== Installing Ansible
 
-Take a look at the  https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html [documentation] for instructions on how to install Ansible.
-// RITA: Please anchor all URLs to descriptive text.
-
-The simplest thing to do is to install Ansible into the virtualenv
+Take a look at the
+https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html[Ansible installation guide]
+for all the various options,
+but probably the simplest thing to do is to install Ansible into the virtualenv
 on our local machine:
 
 [subs="specialcharacters,quotes"]
@@ -908,6 +942,11 @@ but I wanted to keep this (already long) chapter as simple as possible.
 *******************************************************************************
 
 
+// SEBASTIAN: I feel the above section would make more sense if it was mentioned waaay earlier.
+//  I must say I got lost while reading about Chef, Puppet and suddenly I see some Ansible examples
+//  without any explanation how Ansible works, any diagrams etc.
+//  I'd also appreciate seeing some example of SSH first as this is a prerequisite to
+//  using Ansible. Currently, there are too many "open loops" and I think this chapter is hard to follow.
 
 Let's run the latest version of our playbook and see how our tests get on:
 
@@ -971,6 +1010,8 @@ selenium.common.exceptions.WebDriverException: Message: Reached error page:
 about:neterror?e=connectionFailure&u=http%3A//staging.ottg.co.uk/[...]
 ----
 
+// SEBASTIAN: It's awesome that by this moment by using `TEST_SERVER` one is able to run tests against "staging". Just wow!
+
 
 That `neterror` makes me think it's another networking problem.
 
@@ -1397,9 +1438,15 @@ which are a fairly typical set of steps for deployment in general
   and looking into techniques like red-green deployments.
 // CSANAD: we haven't mentioned the downtime so far
 
+// SEBASTIAN: Is red-green deployment a thing? I must admit it's the first time
+//  I see the name and so far I've only know blue-green. I also read there's red-black deployment,
+//  but am I looking for this wrong, so I cannot find anything about red-green? 🤔
+
 // TODO is there a better word than "switching across"?
 // CSANAD: I can only think of "releasing" or "deploying"
 
+// SEBASTIAN: How about simply "updating" or "changing"?
+
 Every single aspect of deployment can and probably should be automated.
 Here are a couple of general principles to think about
 when implementing infrastructure-as-code:
@@ -1415,5 +1462,6 @@ Declarative::
   rather than _how_ we should get there.
   This goes hand-in-hand with the idea of idempotence above.
 
+// SEBASTIAN: Okay, this summary is goldie 👌
 
 *******************************************************************************

chapter_12_organising_test_files.asciidoc

@@ -1,15 +1,6 @@
 [[chapter_12_organising_test_files]]
 == Splitting Our Tests into Multiple Files, and a Generic Wait Helper
 
-.🚧 Warning, chapter update in progress
-*******************************************************************************
-This chapter is currently in the process of being rewritten for the 3e.
-
-The code listings should all be valid,
-and work with Python3.12 + Django 4,
-but I haven't reviewed the chapter text in detail yet.
-
-*******************************************************************************
 
 Back to local development!
 The next feature we might like to implement is a little input validation.
@@ -62,6 +53,9 @@ def test_cannot_add_empty_list_items(self):
 ----
 ====
 
+// DAVID: I think we should spell out that you want us to add this to the test file. Also,
+// where? As a method of NewVisitorTestCase?
+
 That's all very well, but before we go any further--our
 functional tests file is beginning to get a little crowded.
 Let's split it out into several files, in which each has a single test method.
@@ -79,6 +73,8 @@ how to get there step by step.
 NOTE: We're back to local development now.
     Double check that the `TEST_SERVER` environment variable is unset in your terminal.
 
+// DAVID: might be nice to remind people how they check this?
+
 ((("unittest module", "skip test decorator")))
 ((("refactoring")))
 ((("decorators", "skip test decorator")))
@@ -98,6 +94,7 @@ from unittest import skip
     def test_cannot_add_empty_list_items(self):
 ----
 ====
+// DAVID: The indentation here looks weird - maybe better to include the class declaration too?
 
 This tells the test runner to ignore this test.
 You can see it works--if we rerun the tests,
@@ -171,6 +168,10 @@ https://www.oreilly.com/library/view/tidy-first/9781098151232/[Tidy First?]
 ((("test files", "splitting FTs into many")))
 We start putting each test into its own class, still in the same file:
 
+// DAVID: I would say this is too big a refactor all in one go.
+// A nicer way to do this would be to break out the base class first as one refactor, moving it into another file.
+// Then when we copy the files we could rename each class and delete the irrelevant methods.
+
 [role="sourcecode"]
 .src/functional_tests/tests.py (ch11l002)
 ====
@@ -204,6 +205,8 @@ class ItemValidationTest(FunctionalTest):
 ----
 ====
 
+// DAVID: I also have check_for_row_in_list_table as a method - maybe I should've removed it?
+
 At this point we can rerun the FTs and see they all still work:
 
 ----
@@ -383,6 +386,7 @@ when we're only interested in a single one.
 Although we need to remember to run all of them now and again, to check for regressions.
 Later in the book we'll set up a Continuous Integration (CI) server to run all the tests automatically,
 for example every time we push to master.
+// DAVID: "master" -> "our main branch"
 For now, a good prompt for running all the tests is "just before you do a commit",
 so let's get into that habit now:
 
@@ -462,6 +466,8 @@ might decide that building a specific helper method is overkill at this stage,
 but it might be nice to have some generic way of saying, in our tests, "wait
 until this assertion passes".  Something like this:
 
+// DAVID: might be worth reminding the reader about that helper method, e.g. what it's called.
+
 [role="sourcecode"]
 .src/functional_tests/test_list_item_validation.py (ch11l009)
 ====
@@ -511,6 +517,8 @@ and we'll adapt it slightly:
 ----
 ====
 
+// DAVID: I have `raise e` from that other method, not `raise`.
+
 <1> We make a copy of the method, but we name it `wait_for`,
     and we change its argument.  It is expecting to be passed a function.
 
@@ -594,6 +602,9 @@ and that can be executed later, and multiple times:
 
 Let's see our funky `wait_for` helper in action:
 
+// DAVID: Might be worth explicitly telling the reader to add that code earlier - I hadn't yet,
+// so I have to go back and find the right snippet.
+
 [subs="macros,verbatim"]
 ----
 $ pass:quotes[*python src/manage.py test functional_tests.test_list_item_validation*]
@@ -861,6 +872,8 @@ Use a tests folder::
     * You probably want a separate test file for each tested source code
       file. For Django, that's typically 'test_models.py', 'test_views.py', and
       'test_forms.py'.
+// DAVID: worth mentioning that mirroring test files to real files can pave the way for automatically running tests
+// relevant to your changes.
     * Have at least a placeholder test for 'every' function and class.
     ((("test files", "organizing and refactoring")))
 
@@ -871,8 +884,15 @@ Don't forget the "Refactor" in "Red, Green, Refactor"::
     ((("Red/Green/Refactor")))
 
 Don't refactor against failing tests::
-    * In general!
-    * But the FT you're currently working on doesn't count.
+    * The general rule is that you shouldn't mix refactoring and behaviour
+      change. Having green tests is our best guarantee that we aren't changing
+      behaviour. If you start refactoring against failing tests, it becomes much
+      harder to spot when you're accidentally introducing a regression.
+    * This applies strongly to unit tests. With functional tests, because we
+      often develop against red FTs anyway, it's sometimes more tempting to
+      refactor against failing tests. My suggestion is to avoid that temptation
+      and use an early return, so that it's 100% clear if, during a refactory,
+      you accidentally introduce a regression that's picked up in your FTs.
     * You can occasionally put a skip on a test which is testing something you
       haven't written yet.
     * More commonly, make a note of the refactor you want to do, finish what