|
2 | 2 |
|
3 | 3 | [](https://travis-ci.com/forensic-architecture/mtriage) |
4 | 4 |
|
5 | | -### NB: currently unstable, in active development, and should not be used in production |
| 5 | +##### select, download, and analyse media |
6 | 6 |
|
7 | | -##### scrape and analyse media on the web |
| 7 | +mtriage is a command-line application that can be used to scrape and analyse |
| 8 | +media. mtriage is developed by [Forensic Architecture](https://forensic-architecture.org), and is intended for use |
| 9 | +by open source research agencies, journalists, and activists. |
8 | 10 |
|
9 | | -mtriage is a command-line application that scrapes and analyses public domain media. mtriage is developed by [Forensic Architecture](https://forensic-architecture.org), and is intended for use by open source research agencies, journalists, and activists. |
10 | | - |
11 | | -mtriage is a framework that orchestrates two different kinds of components: |
| 11 | +mtriage consists of two types of components: |
12 | 12 |
|
13 | 13 | * **selectors**: to search for and download media from various platforms. |
14 | | -* **analysers**: to derive data from media that has been retrieved by |
15 | | - a selector. |
| 14 | +* **analysers**: to derive data from media that has been retrieved by a selector. |
| 15 | + |
| 16 | +Below are the following components that are supported. If you are interested in |
| 17 | +helping us to develop additional selectors and analysers, please consider |
| 18 | +joining [the conversaton on Discord](https://discord.gg/FJ4XsCg). |
16 | 19 |
|
17 | 20 | ### selectors |
18 | 21 | * youtube - search and download via the [v3 API](https://developers.google.com/youtube/v3/). |
| 22 | +* local - use media that already exists on your filesystem. |
19 | 23 |
|
20 | 24 | ### analysers |
21 | | -* frames - extract one frame for each second from a video. |
22 | | -* ocr - analyse an image using [Google Cloud Platform](https://cloud.google.com/vision/docs/ocr). |
23 | | -<!-- * pytorch - run inference with a [PyTorch](https://pytorch.org/) model on an image. --> |
| 25 | +* frames - extract frames from videos as images. |
| 26 | +* yolov3 - detect and classify objects in images using [YoloV3](https://pjreddie.com/darknet/yolo/) trained on [ImageNet](http://www.image-net.org/) classes. |
| 27 | +* keras_pretrained - classify objects in images using [Resnet50 trained on |
| 28 | + ImageNet](https://resources.wolframcloud.com/NeuralNetRepository/resources/ResNet-50-Trained-on-ImageNet-Competition-Data). |
| 29 | +* ocr - analyse an image using optical character recognition from [Google Cloud Platform](https://cloud.google.com/vision/docs/ocr). |
24 | 30 |
|
25 | | - |
26 | | -## development |
| 31 | +## setup |
27 | 32 | mtriage is currently in active development, and is not yet packaged in any way. |
28 | | -It uses [Docker](https://www.docker.com/products/docker-desktop) to manage |
29 | | -dependencies, and is written in Python. |
| 33 | +It uses [Docker](https://www.docker.com/products/docker-desktop) to manage dependencies, which you will need to download to ensure mtriage works as expected. |
30 | 34 |
|
31 | | -### dependencies |
32 | 35 | - Docker Desktop (Mac installation [here](https://docs.docker.com/v17.12/docker-for-mac/install/), Ubuntu installation [here](https://docs.docker.com/v17.12/install/linux/docker-ce/ubuntu/)). |
33 | | -- [docker](https://docs.docker.com/install/) (python library, v3.5.0) |
34 | | - |
35 | | -Follow the instructions relevant to your operating system to install Docker CE, |
36 | | -and then install the python dependency with: |
37 | | - |
38 | | -```bash |
39 | | -python -m pip install -r requirements.txt |
40 | | -``` |
41 | | - |
42 | | -(Note that mtriage was developed using Python 3, but you should be able to run it with 2.x as well.) |
43 | 36 |
|
44 | | -### configuration setup |
45 | | -Selectors and analysers often rely on private credentials such as API keys. mtriage deals with these in two ways: |
| 37 | +Follow the instructions relevant to your operating system to install Docker CE. |
46 | 38 |
|
47 | | -* **`.env` file at the top level**: contains API keys and other environment variables, which are made available when |
48 | | - mtriage is running. |
49 | | -* **`credentials` folder**: in some cases, components require JSON configs, such as for GCP service accounts. mtriage |
50 | | - currently deals with this by adding a path to the credentials file in `.env`, and adding the credential file itself |
51 | | - in the `credentials` folder. |
| 39 | +You also need to ensure that a version of [Python](https://www.python.org/downloads/) is installed on your computer. |
| 40 | +Most modern operating systems have a version installed by default. |
52 | 41 |
|
53 | | -The specific configuration steps depend on which components you intend to use. For every component you wish to use, run |
54 | | -through its setup: |
| 42 | +### additional setup |
| 43 | +Depending on what components you intend to use, there may be additional setup |
| 44 | +required. Ensure to read the documentation for each component you wish to use. |
55 | 45 |
|
56 | | -##### selectors |
57 | | -* [youtube](docs/config/youtube.md) |
58 | | -##### analysers |
59 | | -* frames |
60 | | -* ocr |
61 | | - |
62 | | -### running |
63 | | -You can run mtriage in a Docker container with: |
| 46 | +## Run |
| 47 | +Once you have Docker and Python installed, you can run mtriage using one of the |
| 48 | +examples provided. From this folder: |
64 | 49 | ```bash |
65 | | -python run.py develop |
| 50 | +./mtriage run examples/_demo/youtube.yaml |
66 | 51 | ``` |
67 | 52 |
|
68 | | -Selectors and analysers are currently specified as runtime arguments to the |
69 | | -entrypoint script, "src/run.py". In "scripts" you can find a series of example |
70 | | -bash scripts that construct appropriate arguments and execute them. |
71 | | - |
72 | | -A more robust interface for passing options is a work in progress. |
73 | | - |
74 | | - |
75 | | -### building locally |
76 | | -You can build the mtriage image locally via run.py as well: |
77 | | -```bash |
78 | | -python run.py build |
79 | | - |
80 | | - |
| 53 | +When you first run mtriage, it will download the necessary Docker images to |
| 54 | +your system. The first time you run it, it may take several minutes to get up |
| 55 | +and running. Subsequent uses will be much faster. |
0 commit comments