Add compile_with_docker.bat

Author: PaulZC

Firmware/compile_with_docker.bat

@@ -0,0 +1,6 @@
+docker build -t rtk_everywhere_firmware --no-cache-filter deployment .
+docker create --name=rtk_everywhere rtk_everywhere_firmware:latest
+docker cp rtk_everywhere:/RTK_Everywhere.ino.bin .
+docker cp rtk_everywhere:/RTK_Everywhere.ino.elf .
+docker cp rtk_everywhere:/RTK_Everywhere.ino.bootloader.bin .
+docker container rm rtk_everywhere

docs/firmware_compile.md

@@ -14,12 +14,14 @@ You are welcome to clone or fork this repo and do the exact same thing yourself.
 
 If you run the [compilation workflow](https://github.com/sparkfun/SparkFun_RTK_Everywhere_Firmware/blob/main/.github/workflows/compile-rtk-everywhere.yml), it will compile the firmware and attempt to push the binary to the Binaries repo. This will fail as your account won't have the right permissions. The [non-release-build](https://github.com/sparkfun/SparkFun_RTK_Everywhere_Firmware/blob/main/.github/workflows/non-release-build.yml) is the one for you. The firmware binary will be attached as an Artifact to the workflow run. Navigate to Actions \ Non-Release Build, select the latest run of Non-Release Build, the binary is in the Artifacts.
 
-You can then use the [SparkFun RTK Firmware Uploader](https://github.com/sparkfun/SparkFun_RTK_Firmware_Uploader) to upload the binary onto the ESP32.
+You can then use (e.g.) the [SparkFun RTK Firmware Uploader](https://github.com/sparkfun/SparkFun_RTK_Firmware_Uploader) to upload the binary onto the ESP32.
 
 ## Using Docker
 
 Installing the correct version of the ESP32 core and of each required Arduino library, is tedious and error-prone. Especially on Windows. We've lost count of the number of times code compilation fails on our local machines, because we had the wrong ESP32 core installed, or forgot to patch libbt or libmbedcrypto... It is much easier to sandbox the firmware compilation using an environment like [Docker](https://www.docker.com/).
 
+Docker is open-source. It is our new favourite thing!
+
 Here is a step-by-step guide for how to install Docker and compile the firmware from scratch:
 
 ### Clone, fork or download the RTK Everywhere Firmware repo
@@ -56,20 +58,20 @@ For the real Wild West experience, you can also download a copy of the `release_
 
 ### Install Docker Desktop
 
-* Head to [Docker](https://www.docker.com/) and create an account. A free "Personal" account will cover occasional compilations of the firmware
+* **(Optional)** Head to [Docker](https://www.docker.com/) and create an account. A free "Personal" account will cover occasional compilations of the firmware
 * Download and install [Docker Desktop](https://docs.docker.com/get-started/get-docker/) - there are versions for Mac, Windows and Linux. You may need to restart to complete the installation.
-* Run the Desktop and sign in
+* Run the Desktop. If you don't sign in, it will run in Personal mode - which will cover occasional compilations of the firmware
 * On Windows, you may see an error saying "**WSL needs updating** Your version of Windows Subsystem for Linux (WSL) is too old". If you do:
     * Open a command prompt
 	* Type `wsl --update` to update WSL. At the time of writing, this installs Windows Subsystem for Linux 2.6.1
 	* Restart the Docker Desktop
-* If you are using Docker for the first time, the "What is a container?" and "How do I run a container?" demos are useful
+* If you are using Docker for the first time, the "What is a container?" and "How do I run a container?" demos are useful - _but not essential_
     * On Windows, you may want to give Docker Desktop permission to access to your Network, so it can access (e.g.) HTML ports
 	* You can Stop the container and Delete it when you are done
 * You may want to prevent Docker from running when your machine starts up
     * Uncheck "Start Docker Desktop when you sign in to your computer" in the Desktop settings
 
-### Running the Dockerfile to create an Image
+### Using Docker to create the firmware binary
 
 * **Make sure you have Docker Desktop running.** You don't need to be logged in, but it needs to be running.
 * Open a Command Prompt and `cd` into the SparkFun_RTK_Everywhere_Firmware folder
@@ -90,14 +92,31 @@ For the real Wild West experience, you can also download a copy of the `release_
 * `cd Firmware` and then `dir` again. You should see:
 
 ```
+    compile_with_docker.bat
     Dockerfile
     readme.md
     RTKEverywhere.csv
     RTKEverywhere_8MB.csv
+    RTK_Everywhere
 ```
 
-* The file we will be using is the `Dockerfile`.
-* Type:
+* The file that does most of the work is the `Dockerfile`
+
+* But, if you're short on time, run `compile_with_docker.bat`. It does everything for you:
+
+![Output of the compile batch file](./img/CompileSource/compile_me_batch_file.png)
+
+* Hey presto! You have your newly compiled firmware binary!
+
+You can then use (e.g.) the [SparkFun RTK Firmware Uploader](https://github.com/sparkfun/SparkFun_RTK_Firmware_Uploader) to upload the binary onto the ESP32.
+
+### Running the Dockerfile manually
+
+If you want to see and understand what's going on under the hood with the Dockerfile, Image and Container:
+
+Here is how to perform each step manually:
+
+* To begin, run the Dockerfile. Type:
 
 ```
 docker build -t rtk_everywhere_firmware .
@@ -121,15 +140,15 @@ docker build -t rtk_everywhere_firmware --progress=plain --no-cache .
 
 Building the full Image from scratch is slow, taking several minutes. You should only need to do it once - unless you make any changes to the Dockerfile.
 
-* When you make changes to the source code and want to recompile, use:
+* **When you make changes to the source code and want to recompile, use:**
 
 ```
 docker build -t rtk_everywhere_firmware --no-cache-filter deployment .
 ```
 
 This uses the cache for the `upstream` stage and avoids recreating the full ubuntu machine. But it ignores the cache for the `deployment` stage, ensuring the code is recompiled.
 
-### Access the firmware by running the Image
+### Access the firmware binary by running the Image
 
 In Docker Desktop, in the Images tab, you should now be able to see an Image named `rtk_everywhere_firmware`. We now need to Run that Image to access the firmware binary. Click the triangular Run icon under Actions:
 
@@ -161,12 +180,39 @@ If you need the `.elf` file so you can debug code crashes with me-no-dev's [ESP
 docker cp rtk_everywhere:/RTK_Everywhere.ino.elf .
 ```
 
+If you need the `.bootloader.bin` file so you can upload it with esptool:
+
+```
+docker cp rtk_everywhere:/RTK_Everywhere.ino.bootloader.bin .
+```
+
 If you want the files to appear in a more convenient directory, replace the single `.` with a folder path.
 
-Delete the `rtk_everywhere` container afterwards, to save disk space and so you can reuse the same container name next time.
+Delete the `rtk_everywhere` container afterwards, to save disk space and so you can reuse the same container name next time. If you forget, you will see an error:
+
+```Conflict. The container name "/rtk_container" is already in use by container. You have to remove (or rename) that container to be able to reuse that name.```
+
+* **Remember:** when you make changes to the source code and want to recompile, use:
+
+```
+docker build -t rtk_everywhere_firmware --no-cache-filter deployment .
+```
+
+* **Shortcut:** the `compile_with_docker.bat` batch file does everything for you:
+
+```
+docker build -t rtk_everywhere_firmware --no-cache-filter deployment .
+docker create --name=rtk_everywhere rtk_everywhere_firmware:latest
+docker cp rtk_everywhere:/RTK_Everywhere.ino.bin .
+docker cp rtk_everywhere:/RTK_Everywhere.ino.elf .
+docker cp rtk_everywhere:/RTK_Everywhere.ino.bootloader.bin .
+docker container rm rtk_everywhere
+```
 
 ## Compiling on Windows (Deprecated)
 
+**Note: we recommend using the Docker method. It is far easier and much less error-prone...**
+
 The SparkFun RTK Everywhere Firmware is compiled using Arduino CLI (currently [v1.0.4](https://github.com/arduino/arduino-cli/releases)). To compile:
 
 1. Install [Arduino CLI](https://github.com/arduino/arduino-cli/releases).
@@ -188,13 +234,26 @@ The SparkFun RTK Everywhere Firmware is compiled using Arduino CLI (currently [v
 
 	This will increase the program partitions, as well as the SPIFFs partition to utilize the full 16MB of flash (8MB in the case of the Postcard).
 
-6. Compile using the following command
+6. Patch the core (libmbedtls, NetworkEvents and libbt) - as shown in the workflow
 
-		arduino-cli compile 'Firmware/RTK_Everywhere' --build-property build.partitions=RTKEverywhere --build-property upload.maximum_size=4055040 --fqbn esp32:esp32:esp32:FlashSize=16M,PSRAM=enabled
+7. Update the web config `form.h` by running the two python scripts in the `Tools` folder:
 
-8. Once compiled, upload to the device using the following command where `[COM_PORT]` is the COM port on which the RTK device is located (ie `COM42`).
+```
+python index_html_zipper.py ../RTK_Everywhere/AP-Config/index.html ../RTK_Everywhere/form.h
+python main_js_zipper.py ../RTK_Everywhere/AP-Config/src/main.js ../RTK_Everywhere/form.h
+```
+
+8. Compile using the following command
 
+```
+		arduino-cli compile --fqbn "esp32:esp32:esp32":DebugLevel=none,PSRAM=enabled ./Firmware/RTK_Everywhere/RTK_Everywhere.ino --build-property build.partitions=RTKEverywhere --build-property upload.maximum_size=4055040 --build-property "compiler.cpp.extra_flags=-MMD -c" --export-binaries
+```
+
+9. Once compiled, upload to the device using the following command where `[COM_PORT]` is the COM port on which the RTK device is located (ie `COM42`).
+
+```
 		arduino-cli upload -p [COM_PORT] --fqbn esp32:esp32:esp32:UploadSpeed=512000,FlashSize=16M 'Firmware/RTK_Everywhere'
+```
 
 If you are seeing the error: