You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/06_MetafactureCLI.md
+25-34Lines changed: 25 additions & 34 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,30 +9,24 @@ parent: Tutorial
9
9
10
10
## Get Metafacture Runner as CLI Tool
11
11
12
-
Hint: This lesson requires basic practical knowledge of the command line and Shell.
13
-
If you want to get familiar with tit, have a look at the great intro to Unix Shell by Library Carpentry: https://librarycarpentry.github.io/lc-shell/ (Session 1 - 3) You could also have a look at the great introdution by the Programming Historian to Powershell: https://programminghistorian.org/en/lessons/intro-to-powershell
12
+
_This lesson requires basic practical knowledge of the command line and Shell.
13
+
If you want to get familiar with it, have a look at the great [intro to Unix Shell by Library Carpentry](https://librarycarpentry.github.io/lc-shell/) (Session 1 - 3). You could also have a look at the great [introdution by the Programming Historian to Powershell](https://programminghistorian.org/en/lessons/intro-to-powershell)_
14
14
15
-
While we had fun with our Metafacture Playground another way to use Metafacture is
16
-
the command line. For running a Metafacture flux process we need a terminal and installed JAVA 11 ore higher.
17
-
For creating and editing Flux and Fix files we need an texteditor like Codium/VS Code or others.
15
+
While we had fun with our Metafacture Playground another way to use Metafacture is by
16
+
the command line. For running a Metafacture Flux process we need a terminal and Java 11 ore higher.
17
+
For creating and editing Flux and Fix files we need a text editor like Codium/VS Code or others.
18
18
19
19
For this lesson basic knowledge of the commandline is recommended.
20
20
21
21
Check if Java 11 or higher is installed with `java -version` in your terminal.
22
-
If not, install JAVA 11 or higher.
22
+
If not, install Java 11 or higher.
23
23
24
-
To use Metafacture on the commandline we can download the latest distribution e.g.: `metafacture-core-7.0.0-dist.zip`:
Hint: If 7.0.0 is not published yet use the runner version of the [prerelease 7.0.0-rc1](https://github.com/metafacture/metafacture-core/releases/tag/metafacture-core-7.0.0-rc1).
24
+
To use Metafacture on the commandline [download the latest (pre-) release](https://github.com/metafacture/metafacture-core/releases).
29
25
30
26
Download `metafacture-core-$VERSION-dist.tar.gz` or the zip version and extract the archive to your choosen folder.
31
27
In the folder you find the `flux.bat` and `flux.sh`
32
28
33
-
The code below assumes you moved the resulting folder to your home directory and renamed it to `'metafacture'`
34
-
35
-
$ ~/metafacture/flux.sh # or flux.bat on Windows
29
+
The code below assumes you moved the resulting folder to your home directory and renamed it to `"metafacture"`.
36
30
37
31
## How to run Metafacture via CLI
38
32
@@ -50,7 +44,7 @@ or Windows:
50
44
~\metafacture\flux.bat path\to\your.flux
51
45
```
52
46
53
-
(Hint: You need to know the path to your file to run the function.)
47
+
(Hint: You need to know the path to your Flux file to run it.)
54
48
55
49
To get quick started let's revisit a Flux we toyed around with in the playground.
56
50
The playground has a nice feature to export and import Metafacture Workflows.
@@ -77,12 +71,10 @@ Windows:
77
71
78
72
To simplify the code examples we will be using unix paths for the terminal commands. Windows Powershell will change these to windows paths automatically.
79
73
80
-
The result of running the Flux-Script via CLI should be the same as with the Playground.
74
+
The result of running the Flux script via CLI should be the same as with the Playground.
81
75
82
-
The Metafacture CLI Tool expects a flux file for every workflow.
83
-
Our runned workflow only has the following flux and no additional files since it is fetching data from the web and it has no fix transformations.
84
-
85
-
The downloaded file should have the following content, defining the playground specific variables and the flux workflow that you also saw in the playground. You can delete the playground specific variables since they are not needed here.
76
+
The Metafacture CLI tool expects a Flux file for every workflow.
77
+
Our workflow only has the following Flux and no additional files since it is fetching data from the web and it has no fix transformations. The file should have the following content, defining the playground specific variables and the Flux workflow that you also saw in the playground. You can delete the playground specific variables since they are not needed so you woul end with this:
86
78
87
79
```text
88
80
"https://openlibrary.org/books/OL2838758M.json"
@@ -96,11 +88,11 @@ The downloaded file should have the following content, defining the playground s
96
88
97
89
## Use local files for transformation
98
90
99
-
If you want to load a local file instead of fetching data from the web we need to change the flux a little bit with an texteditor.
91
+
If you want to load a local file instead of fetching data from the web you need to change the Flux a little bit with an text editor.
100
92
Download the following file [11942150X.json](./sample-scripts/lesson_06/11942150X.json)
101
93
and adjust the path to your file.
102
94
103
-
Adjust your `downloads/playground.flux` script, so that it does not load data from the web, but opens a local file with `open-file` and read it `as-recrods` since the json file is pretty printed:
95
+
Adjust your `downloads/playground.flux` script, so that it does not load data from the web, but opens a local file with `open-file` and reads it using `as-records` since the json file is pretty printed (not as one record per line):
104
96
105
97
```text
106
98
"path/to/your/file/11942150X.json" // Adjust your path!
@@ -284,7 +276,7 @@ It should output:
284
276
}
285
277
```
286
278
287
-
If we want to use fix we need to refrence the fix file that in the playground we only refrenced via `| fix`
279
+
If we want to use Fix we need to reference the Fix file (in the playground we only referenced t via `| fix`):
288
280
289
281
```text
290
282
"path/to/your/file/11942150X.json"
@@ -297,20 +289,19 @@ If we want to use fix we need to refrence the fix file that in the playground we
297
289
;
298
290
```
299
291
300
-
Create a new file with the name `fixFile.fix`, files with fix scripts should have a `.fix` file suffix.
292
+
Create a new file with the name `fixFile.fix`. Files with Fix scripts should have a `.fix` file suffix to easily discriminate them later.
301
293
302
-
Add the follwoing line as content to this file:
294
+
Add the following line as content to this file:
303
295
304
296
```perl
305
297
retain("preferredName","id","type[]")
306
-
307
298
```
308
299
309
-
Save it in the same folder as the flux file. (Hint: It does not always have to be in the same folder.)
300
+
Save it in the same folder as the Flux file. (Hint: It does not always have to be in the same folder.)
310
301
311
302
## Use variables
312
303
313
-
Hint: You can use the varliable FLUX_DIR to shorten the file path if the file is in the same folder as the flux-file.
304
+
Hint: You can use the varliable FLUX_DIR to shorten the file path if the file is in the same folder as the Flux file:
314
305
315
306
```text
316
307
FLUX_DIR + "file.json"
@@ -323,7 +314,7 @@ FLUX_DIR + "file.json"
323
314
;
324
315
```
325
316
326
-
If you are using variables, that are not defined in the flux, you can pass them on with the CLI:
317
+
If you are using variables that are not defined in the flux, you can pass them on with the CLI:
Excercise: Download the following folder (TODO) with three test examples and run them. Adjust them if needed:
349
340
350
341
- Run example script locally.
351
-
- Adjust example script so that all json files but no other in the folder are read. Get inspired by https://github.com/metafacture/metafacture-core/blob/master/metafacture-runner/src/main/dist/examples/misc/reading-dirs/read-dirs.flux.
352
-
- Change the FLUX script so that you write the output in the local file instead of stoudt.
353
-
- Add a fix file and add the fix module in the flux. With `nothing()` as content.
354
-
- Add some transformations to the fix e.g. add fields.
342
+
- Adjust example script so that all json files but no other in the folder are read. Get inspired by the [reading directories example](https://github.com/metafacture/metafacture-core/blob/master/metafacture-runner/src/main/dist/examples/misc/reading-dirs/read-dirs.flux).
343
+
- Change the Flux script so that you write the output in the local file instead of stoudt.
344
+
- Add a Fix file and add the Fix module in the Flux. With `nothing()` as content.
345
+
- Add some transformations to the Fix e.g. add fields.
0 commit comments