Add some adjustments while reviewing

TobiasNx · TobiasNx · commit 8ae66e2ae4b1 · 2025-05-15T12:17:34.000+02:00
diff --git a/docs/02_Introduction_into_Metafacture-Flux.md b/docs/02_Introduction_into_Metafacture-Flux.md
@@ -214,7 +214,7 @@ Luckily, we cannot only open the data we have in our `inputFile-content` field,
 
 Clear your playground and copy the following Flux workflow:
 
-```
+```text
 "https://openlibrary.org/books/OL2838758M.json"
 | open-http
 | as-lines
@@ -236,6 +236,7 @@ Let's take a look at what a Flux workflow does. The Flux workflow is a combinati
 6. Finally, we tell MF to `print` everything.
 
 So let's have a small recap of what we've done and learned so far:
+
 * We've played around with the Metafacture Playground.
 * We've learned that a Metafacture Flux workflow is a combination of modules with an inital text string or a variable.
 * We got to know different modules like `open-http`, `as-lines`. `decode-json`, `encode-yaml`, `print`
diff --git a/docs/03_Introduction_into_Metafacture-Fix.md b/docs/03_Introduction_into_Metafacture-Fix.md
@@ -96,13 +96,13 @@ Using a separate Fix file is recommended if you need to write many Fix functions
 To add more Fixes we can again edit the Fix file.
 Lets add these lines in front of the retain function:
 
-```
+```perl
 move_field("type.key", "pub_type")
 ```
 
 Also change the `retain` function so that you keep the new element `"pub_type"` instead of the not existing nested `"key"` element.
 
-```
+```perl
 move_field("type.key","pub_type")
 retain("title", "publish_date",  "notes.value", "pub_type")
 ```
@@ -121,7 +121,7 @@ notes:
 With `move_field` we moved and renamed an existing element.
 As next step add the following function before the `retain` function.
 
-```
+```perl
 replace_all("pub_type","/type/","")
 ```
 
@@ -142,7 +142,7 @@ We cleaned up the value of `"pub_type"` element for better readability.
 
 Metafacture contains many Fix functions to manipulate data. Also there are many Flux commands/modules that can be used.
 
-Check the documentation to get a complete list of [Flux commands](https://metafacture.github.io/metafacture-documentation/docs/flux/flux-commands.html) and [Fix functions](https://metafacture.github.io/metafacture-documentation/docs/fix/fix-functions.html). This post only presented a short introduction into Metafacture. In the next posts we will go deeper into its capabilities.
+Check the documentation to get a complete list of [Flux commands](https://metafacture.github.io/metafacture-documentation/docs/flux/flux-commands.html) and [Fix functions](https://metafacture.github.io/metafacture-documentation/docs/fix/Fix-functions.html). This post only presented a short introduction into Metafacture. In the next posts we will go deeper into its capabilities.
 
 Besides Fix functions you can also add as many comments and linebreaks as you want to a Fix.
 
@@ -169,7 +169,7 @@ retain("title", "publish_date", "pub_type")
 
 2) [Add a field with todays date called `"map_date"`.](https://metafacture.org/playground/?flux=%22https%3A//openlibrary.org/books/OL2838758M.json%22%0A%7C+open-http%0A%7C+as-lines%0A%7C+decode-json%0A%7C+fix+%28transformationFile%29%0A%7C+encode-yaml%0A%7C+print%0A%3B&transformation=move_field%28%22type.key%22%2C%22pub_type%22%29%0Areplace_all%28%22pub_type%22%2C%22/type/%22%2C%22%22%29%0A...%28%22mape_date%22%2C%22...%22%29%0Aretain%28%22title%22%2C+%22publish_date%22%2C+%22by_statement%22%2C+%22pub_type%22%29)
 
-Have a look at the [Fix functions](https://metafacture.org/metafacture-documentation/docs/fix/fix-functions.html). (Hint: you could use `add_field` or `timestamp`. And don't forget to add the new element to `retain`)
+Have a look at the [Fix functions](https://metafacture.org/metafacture-documentation/docs/fix/Fix-functions.html). (Hint: you could use `add_field` or `timestamp`. And don't forget to add the new element to `retain`)
 
 
 <details>
diff --git a/docs/05-More-Fix-Concepts.md b/docs/05-More-Fix-Concepts.md
@@ -7,9 +7,9 @@ parent: Tutorial
 
 # Lesson 5: More Fix concepts
 
-We already learned about simple Fixes aka *[Fix functions](https://metafacture.github.io/metafacture-documentation/docs/fix/fix-functions.html)* but there are three additional concepts in Fix: selector, conditionals and binds.
+We already learned about simple Fixes aka *[Fix functions](https://metafacture.github.io/metafacture-documentation/docs/fix/Fix-functions.html)* but there are three additional concepts in Fix: selector, conditionals and binds.
 
-These Fix concepts were introduced by Catmandu (see [functions](https://librecat.org/Catmandu/#functions), [selector](https://librecat.org/Catmandu/#selectors), [conditionals](https://librecat.org/Catmandu/#conditionals) and [binds](https://librecat.org/Catmandu/#binds)). Be aware that Metafacture Fix does not support all of the specific functions, selectors, conditionals and binds from Catmandu. Check the documentation for a full overview of the supported [Fix functions](https://metafacture.org/metafacture-documentation/docs/fix/fix-functions.html).
+These Fix concepts were introduced by Catmandu (see [functions](https://librecat.org/Catmandu/#functions), [selector](https://librecat.org/Catmandu/#selectors), [conditionals](https://librecat.org/Catmandu/#conditionals) and [binds](https://librecat.org/Catmandu/#binds)). Be aware that Metafacture Fix does not support all of the specific functions, selectors, conditionals and binds from Catmandu. Check the documentation for a full overview of the supported [Fix functions](https://metafacture.org/metafacture-documentation/docs/fix/Fix-functions.html).
 
 ## Additional concepts
 
@@ -47,11 +47,11 @@ Fix functions are used to add, change, remove or otherwise manipulate elements.
 
 The other three concepts help when you intend to use more complex transformations:
 
-*[Conditionals](https://metafacture.github.io/metafacture-documentation/docs/fix/fix-functions.html#conditionals)* are used to control the processing of Fix functions. The included Fix functions are not processed with every workflow but only under certain conditions.
+*[Conditionals](https://metafacture.github.io/metafacture-documentation/docs/fix/Fix-functions.html#conditionals)* are used to control the processing of Fix functions. The included Fix functions are not processed with every workflow but only under certain conditions.
 
-*[Selectors](https://metafacture.github.io/metafacture-documentation/docs/fix/fix-functions.html#selectors)* can be used to filter the records you want.
+*[Selectors](https://metafacture.github.io/metafacture-documentation/docs/fix/Fix-functions.html#selectors)* can be used to filter the records you want.
 
-*[Binds](https://metafacture.github.io/metafacture-documentation/docs/fix/fix-functions.html#binds)* are wrappers for one or more Fixes. They give extra control functionality for Fixes such as loops. All binds have the same syntax:
+*[Binds](https://metafacture.github.io/metafacture-documentation/docs/fix/Fix-functions.html#binds)* are wrappers for one or more Fixes. They give extra control functionality for Fixes such as loops. All binds have the same syntax:
 
 ```perl
 do Bind(params,...)
@@ -160,7 +160,7 @@ else
 end
 ```
 
-Metafacture supports lots of conditionals, find a list of all of them [here](https://metafacture.github.io/metafacture-documentation/docs/fix/fix-functions.html#conditionals).
+Metafacture supports lots of conditionals, find a list of all of them [here](https://metafacture.github.io/metafacture-documentation/docs/fix/Fix-functions.html#conditionals).
 
 ## Selectors
 
@@ -176,11 +176,11 @@ end
 
 Selectors work in combination with conditionals to define the conditions that you want to kick out.
 
-See the [list of supported selectors](https://metafacture.github.io/metafacture-documentation/docs/fix/fix-functions.html#selectors).
+See the [list of supported selectors](https://metafacture.github.io/metafacture-documentation/docs/fix/Fix-functions.html#selectors).
 
 ## Binds
 
-As mentioned above [Binds](https://metafacture.github.io/metafacture-documentation/docs/fix/fix-functions.html#binds) are wrappers for one or more Fixes. They give extra control functionality for Fixes such as loops. All binds have the same syntax:
+As mentioned above [Binds](https://metafacture.github.io/metafacture-documentation/docs/fix/Fix-functions.html#binds) are wrappers for one or more Fixes. They give extra control functionality for Fixes such as loops. All binds have the same syntax:
 
 ```perl
 do Bind(params,...)
@@ -247,7 +247,7 @@ end
 
 [See this example in the playground.](https://metafacture.org/playground/?flux=inputFile%0A%7C+open-file%0A%7C+as-records%0A%7C+decode-yaml%0A%7C+fix%28transformationFile%29%0A%7C+encode-yaml%0A%7C+print%0A%3B&transformation=do+list%28path%3A%22colours%5B%5D%22%2C%22var%22%3A%22%24i%22%29%0A++++if+any_equal%28%22%24i%22%2C%22green%22%29%0A++++++++upcase%28%22%24i%22%29%0A++++++++append%28%22%24i%22%2C%22+is+a+nice+color%22%29%0A++++++++copy_field%28%22%24i%22%2C%22result%5B%5D.%24append%22%29%0A++++end%0Aend&data=---%0Acolours%3A%0A+-+red%0A+-+yellow%0A+-+green)
 
-See the [list of supported binds](https://metafacture.github.io/metafacture-documentation/docs/fix/fix-functions.html#binds).
+See the [list of supported binds](https://metafacture.github.io/metafacture-documentation/docs/fix/Fix-functions.html#binds).
 
 TODO: Add excercises.
 
diff --git a/docs/06_MetafactureCLI.md b/docs/06_MetafactureCLI.md
@@ -9,7 +9,7 @@ parent: Tutorial
 
 ## Get Metafacture Runner as CLI Tool
 
-_This lesson requires basic practical knowledge of the command line and Shell.
+This lesson requires basic practical knowledge of the command line and Shell.
 If you want to get familiar with it, have a look at the great [intro to Unix Shell by Library Carpentry](https://librarycarpentry.github.io/lc-shell/) (Session 1 - 3). You could also have a look at the great [introdution by the Programming Historian to Powershell](https://programminghistorian.org/en/lessons/intro-to-powershell)_
 
 While we had fun with our Metafacture Playground another way to use Metafacture is by
@@ -28,6 +28,22 @@ In the folder you find the `flux.bat` and `flux.sh`
 
 The code below assumes you moved the resulting folder to your home directory and renamed it to `"metafacture"`.
 
+If you run
+
+Unix:
+
+```bash
+~/metafacture/flux.sh
+```
+
+or Windows:
+
+```bash
+~\metafacture\flux.bat
+```
+
+Metafacture will list all currently available Flux Commands.
+
 ## How to run Metafacture via CLI
 
 You can run your workflows:
@@ -74,7 +90,7 @@ To simplify the code examples we will be using unix paths for the terminal comma
 The result of running the Flux script via CLI should be the same as with the Playground.
 
 The Metafacture CLI tool expects a Flux file for every workflow.
-Our workflow only has the following Flux and no additional files since it is fetching data from the web and it has no fix transformations. The file should have the following content, defining the playground specific variables and the Flux workflow that you also saw in the playground. You can delete the playground specific variables since they are not needed so you woul end with this:
+Our workflow only has the following Flux and no additional files since it is fetching data from the web and it has no Fix transformations. The file should have the following content, defining the playground specific variables and the Flux workflow that you also saw in the playground. You can delete the playground specific variables since they are not needed so you would end with this:
 
 ```text
 "https://openlibrary.org/books/OL2838758M.json"
@@ -108,7 +124,7 @@ Run it again as shown above.
 
 It should output:
 
-```JSON
+```json
 {
   "professionOrOccupation" : [ {
     "id" : "https://d-nb.info/gnd/4629643-8",
@@ -276,7 +292,7 @@ It should output:
 }
 ```
 
-If we want to use Fix we need to reference the Fix file (in the playground we only referenced t via `| fix`):
+If we want to use Fix we need to reference the Fix file (in the playground we only referenced the variable `transformationFile` via `| fix`):
 
 ```text
 "path/to/your/file/11942150X.json"
@@ -329,7 +345,7 @@ FILE
 ;
 ```
 
-which you use like:
+Which you use like:
 
 ```bash
 ~/metafacture/flux.sh path/to/your.flux FILE="path/to/your/file.json"
diff --git a/docs/07_Processing_MARC.md b/docs/07_Processing_MARC.md
@@ -10,15 +10,15 @@ parent: Tutorial
 
 In the previous lessons we learned how we can use Metafacture to process structured data like JSON. In this lesson we will use Metafacture to process MARC metadata records. In this process we will see that MARC can be processed using FIX paths.
 
-[Transformation of MARC data with Mmetafacture can be used for multiple things, e.g. you could transform MARC binary files to MARC XML.](https://metafacture.org/playground/?flux=%22https%3A//raw.githubusercontent.com/metafacture/metafacture-tutorial/main/data/sample.mrc%22%0A%7C+open-http%0A%7C+as-lines%0A%7C+decode-marc21%28emitleaderaswhole%3D%22true%22%29%0A%7C+encode-marcxml%0A%7C+print%0A%3B)
+[Transformation of MARC data with Metafacture can be used for multiple things, e.g. you could transform MARC binary files to MARC XML.](https://metafacture.org/playground/?flux=%22https%3A//raw.githubusercontent.com/metafacture/metafacture-tutorial/main/data/sample.mrc%22%0A%7C+open-http%0A%7C+as-lines%0A%7C+decode-marc21%28emitleaderaswhole%3D%22true%22%29%0A%7C+encode-marcxml%0A%7C+print%0A%3B)
 
 As always, we will need to set up a small metafacture Flux script.
 
 Lets inspect a MARC file: https://raw.githubusercontent.com/metafacture/metafacture-tutorial/main/data/sample.marc
 
 Create the following Flux in a new file and name it e.g. `marc1.flux`:
 
-```
+```text
 "https://raw.githubusercontent.com/metafacture/metafacture-tutorial/main/data/sample.mrc"
 | open-http
 | as-lines
@@ -27,7 +27,8 @@ Create the following Flux in a new file and name it e.g. `marc1.flux`:
 ```
 
 Run this Flux via CLI, e.g.:
-```
+
+```bash
 /path/to/your/metafix-runner path/to/your/marc1.flux
 ```
 
@@ -352,7 +353,7 @@ You will see this as output:
 
 In the Fix above we mapped the field 245 to the title, and iterated over every subfield with the help of the list-bind and the `?`- wildcard. The ISBN is in the 020-field. Because MARC records can contain one or more 020 fields we created an isbn array with add_array and added the values using the isbn.$append syntax. Next we turned the isbn array back into a comma separated string using the join_field fix. As last step we deleted all the fields we didn’t need in the output with the `retain` syntax.
 
-Different versions of MARC-Serialization need different workflows: e.g. h[ere see an example of Aseq-MARC Files that are transformed to MARCxml.](https://test.metafacture.org/playground/?flux=%22https%3A//raw.githubusercontent.com/LibreCat/Catmandu-MARC/dev/t/rug01.aleph%22%0A%7C+open-http%0A%7C+as-lines%0A%7C+decode-aseq%0A%7C+merge-same-ids%0A%7C+encode-marcxml%0A%7C+print%0A%3B)
+Different versions of MARC-Serialization need different workflows: e.g. [here see an example of Aseq-MARC Files that are transformed to MARCxml.](https://test.metafacture.org/playground/?flux=%22https%3A//raw.githubusercontent.com/LibreCat/Catmandu-MARC/dev/t/rug01.aleph%22%0A%7C+open-http%0A%7C+as-lines%0A%7C+decode-aseq%0A%7C+merge-same-ids%0A%7C+encode-marcxml%0A%7C+print%0A%3B)
 
 In this post we demonstrated how to process MARC data. In the next post we will show some examples how Catmandu typically can be used to process library data.
 
diff --git a/docs/08_Harvest_data_with_OAI-PMH.md b/docs/08_Harvest_data_with_OAI-PMH.md
@@ -49,7 +49,7 @@ You can also harvest MARC data, serialize it to MARC-binary and store it in a fi
 ;
 ```
 
-You can also transform incoming data and store/index it with MongoDB or Elasticsearch. For the transformation you need to create a fix (see Lesson 3) in the playground or in a text editor:
+You can also transform incoming data and prepare it for indexing it with Elasticsearch. For the transformation you need to create a fix (see Lesson 3) in the playground or in a text editor:
 
 Add the following fixes to the file:
 
@@ -61,7 +61,7 @@ copy_field("260??.c","date")
 retain("_id","title","creator[]","date")
 ```
 
-Now you can run an ETL process (extract, transform, load) with this worklflow:
+Now you can run an ETL process (extract, transform, load) with this worklflow, we use `json-to-elasticsearch-bulk` to prepare the output for elastic search indexing:
 
 ```text
 "https://lib.ugent.be/oai"
diff --git a/docs/09_Working_with_CSV.md b/docs/09_Working_with_CSV.md
@@ -43,7 +43,6 @@ Convert the data to different serializations, like JSON, YAML and XML by decodin
 See that the elements have no literal names but only numbers.
 As the CSV has a header we need to add the option `(hasHeader="true")` to `decode-csv` in the Flux.
 
-
 You can extract specified fields while converting to another tabular format by using the Fix. This is quite handy for analysis of specific fields or to generate reports. In the following example we only keep three columns (`"ISBN"`,`"Title"`,`"Author"`):
 
 Flux:
@@ -60,7 +59,8 @@ Flux:
 ```
 
 With Fix:
-```
+
+```perl
 retain("ISBN","Title","Author")
 ```
 
@@ -82,7 +82,7 @@ Flux:
 
 Fix:
 
-```
+```perl
 replace_all("?","^\\$|\\$$","")
 ```
 
diff --git a/docs/10_Working_with_XML.md b/docs/10_Working_with_XML.md
@@ -103,7 +103,7 @@ title:
 [For our example above, to get rid of the value subfields in the yaml, we need to change the hirachy:](https://metafacture.org/playground/?flux=inputFile%0A%7C+open-file%0A%7C+decode-xml%0A%7C+handle-generic-xml%0A%7C+fix%28transformationFile%29%0A%7C+encode-yaml%0A%7C+print%0A%3B&transformation=move_field%28%22title.value%22%2C%22@title%22%29%0Amove_field%28%22@title%22%2C%22title%22%29%0Amove_field%28%22author.value%22%2C%22@author%22%29%0Amove_field%28%22@author%22%2C%22author%22%29%0Amove_field%28%22datePublished.value%22%2C%22@datePublished%22%29%0Amove_field%28%22@datePublished%22%2C%22datePublished%22%29&data=%3C%3Fxml+version%3D%221.0%22+encoding%3D%22utf-8%22%3F%3E%0A%3Crecord%3E%0A++%3Ctitle%3EGRM%3C/title%3E%0A++%3Cauthor%3ESibille+Berg%3C/author%3E%0A++%3CdatePublished%3E2019%3C/datePublished%3E%0A%3C/record%3E)
 
 
-```
+```text
 inputFile
 | open-file
 | decode-xml
@@ -115,6 +115,7 @@ inputFile
 ```
 
 with Fix:
+
 ```perl
 move_field("title.value","@title")
 move_field("@title","title")
@@ -135,6 +136,7 @@ inputFile
 | print
 ;
 ```
+
 results in:
 
 ```xml
diff --git a/docs/11_MARC_to_Dublin_Core.md b/docs/11_MARC_to_Dublin_Core.md
@@ -6,7 +6,8 @@ parent: Tutorial
 ---
 
 
-# Lesson 11 : From MARC to Dublin Core as Linked Open Usable Data (LOUD)
+## Lesson 11 : From MARC to Dublin Core as Linked Open Usable Data (LOUD)
+
 TODO: Use better example. But the following is missing isbns: https://github.com/metafacture/metafacture-examples/blob/master/Swissbib-Extensions/MARC-CSV/
 
 Today we will look a bit further into MARC processing with Metafacture. We already saw a bit of MARC processing and today we will transform MARC records into Dublin Core providing the data as linked open usable data.
