Update README

Author: Marcin Kardas

README.md

@@ -1,42 +1,78 @@
-# Scripts for extracting tables
+# AxCell: Automatic Extraction of Results from Machine Learning Papers
+[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/axcell-automatic-extraction-of-results-from/scientific-results-extraction-on-pwc)](https://paperswithcode.com/sota/scientific-results-extraction-on-pwc?p=axcell-automatic-extraction-of-results-from)
+[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/axcell-automatic-extraction-of-results-from/scientific-results-extraction-on-nlp-tdms-exp)](https://paperswithcode.com/sota/scientific-results-extraction-on-nlp-tdms-exp?p=axcell-automatic-extraction-of-results-from)
 
-Dependencies:
- * [jq](https://stedolan.github.io/jq/) (`sudo apt install jq`)
- * docker (run without `sudo`)
- * [conda](https://www.anaconda.com/distribution/)
+This repository is the official implementation of [AxCell: Automatic Extraction of Results from Machine Learning Papers](https://arxiv.org/abs/2004.14356).
 
-Directory structure:
-```
-.
-└── data
-    ├── annotations
-    │   └── evaluation-tables.json.gz     # current annotations
-    └── arxiv
-        ├── sources                       # gzip archives with e-prints
-        ├── unpacked\_sources             # automatically extracted latex sources
-        ├── htmls                         # automatically generated htmls
-        ├── htmls-clean                   # htmls fixed by chromium
-        └── tables                        # extracted tables
+![pipeline](https://user-images.githubusercontent.com/13535078/81287158-33e01000-905a-11ea-8573-d716373efbdd.png)
+
+```bibtex
+@inproceedings{axcell,
+    title={AxCell: Automatic Extraction of Results from Machine Learning Papers},
+    author={Marcin Kardas, Piotr Czapla, Pontus Stenetorp,
+            Sebastian Ruder, Sebastian Riedel, Ross Taylor, Robert Stojnic},
+    year={2020},
+    booktitle={2004.14356}
+}
 ```
 
+## Requirements
 
-To preprocess data and extract tables and texts, run:
-```
-make pull_images
+To create a [conda](https://www.anaconda.com/distribution/) environment named `axcell` and install requirements run:
+
+```setup
 conda env create -f environment.yml
-source activate xtables
-make -j 8 -i extract_all > stdout.log 2> stderr.log
 ```
-where `8` is number of jobs to run simultaneously. Optionally one can specify path to data directory, f.e., `make DATA_DIR=mydata ...`.
 
-## Test
-To test the whole extraction on a single file run
-```
-make test
-```
+Additionally, `axcell` requires `docker` (that can be run without `sudo`). Run `scripts/pull_docker_images.sh` to download necessary images.
 
-### Unit Tests
+## Datasets
+We publish the following datasets:
+* [ArxivPapers](https://github.com/paperswithcode/axcell/releases/download/v1.0/arxiv-papers.csv.xz)
+* [SegmentedTables & LinkedResults](https://github.com/paperswithcode/axcell/releases/download/v1.0/segmented-tables.json.xz)
+* [PWCLeaderboards](https://github.com/paperswithcode/axcell/releases/download/v1.0/pwc-leaderboards.json.xz)
 
-```
-PYTHONPATH=. py.test
+See [datasets](notebooks/datasets.ipynb) notebook for an example of how to load the datasets provided below. The [extraction](notebooks/extraction.ipynb) notebook shows how to use `axcell` to extract text and tables from papers.
+## Training
+
+
+
+## Evaluation
+
+See the [evaluation](notebooks/evaluation.ipynb) notebook for the full example on how to evaluate AxCell on the PWCLeaderboards dataset. 
+
+## Pre-trained Models
+
+You can download pretrained models here:
+
+- [axcell](https://github.com/paperswithcode/axcell/releases/download/v1.0/models.tar.xz) — an archive containing the taxonomy, abbreviations, table type classifier and table segmentation model. See the [results-extraction](notebooks/results-extraction.ipynb) notebook for an example of how to load and run the models 
+- [language model](https://github.com/paperswithcode/axcell/releases/download/v1.0/lm.xz) — [ULMFiT](https://arxiv.org/abs/1801.06146) language model pretrained on the ArxivPapers dataset
+
+## Results
+
+AxCell achieves the following performance:
+
+### 
+
+
+| Dataset | Macro F1 | Micro F1 |
+| ---------- |---------------- | -------------- |
+| [PWC Leaderboards](https://beta.paperswithcode.com/sota/scientific-results-extraction-on-pwc)     |     21.1         |      28.7       |
+| [NLP-TDMS](https://beta.paperswithcode.com/sota/scientific-results-extraction-on-nlp-tdms-exp)    |     19.7         |      25.8       |
+
+
+
+## License
+
+AxCell is released under the [Apache 2.0 license](LICENSE).
+
+### Citation
+The pipeline is described in the following paper:
+```bibtex
+@inproceedings{axcell,
+    title={AxCell: Automatic Extraction of Results from Machine Learning Papers},
+    author={Marcin Kardas and Piotr Czapla and Pontus Stenetorp and Sebastian Ruder and Sebastian Riedel and Ross Taylor and Robert Stojnic},
+    year={2020},
+    booktitle={2004.14356}
+}
 ```

notebooks/evaluation.ipynb

@@ -96,7 +96,7 @@
     }
    ],
    "source": [
-    "MODELS_URL = 'http://10.0.1.145:8001/static/axcell/models.tar.xz'\n",
+    "MODELS_URL = V1_URL + 'models.tar.xz'\n",
     "MODELS_ARCHIVE = 'models.tar.xz'\n",
     "MODELS_PATH = Path('models')\n",
     "\n",

notebooks/results-extraction.ipynb

@@ -43,7 +43,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "MODELS_URL = 'http://localhost:8001/static/axcell/models.tar.xz'\n",
+    "MODELS_URL = 'https://github.com/paperswithcode/axcell/releases/download/v1.0/models.tar.xz'\n",
     "MODELS_ARCHIVE = 'models.tar.xz'\n",
     "MODELS_PATH = Path('models')\n",
     "\n",