Latent Space 2D (LS2D)

Community | Documentation | Resources | Contributors | Release Notes

LS2D is a lightweight tool to create embeddings from complex data into two dimensions. In addition, it includes a web app that (i) facilitates performance comparison among the pipelines created, (ii) enables visualisation of the observations and the distribution of the features/outcomes and (iii) allows to query patients based on distance and displays a demographic table.

demo-v1.mp4

Table of Contents

• About the project
• Getting Started
  • Installation
  • Training
  • Running the app
• License

About the project

This work aims to facilitate the development and validation of unsupervised techniques to reduce data complexity to a 2D space so that the information can be relayed to the end user through accessible graphical representations. In addition to traditional technices such as PCA, autoencoders, a type of neural network, have been used in the included examples.

Live demo (Heroku)**: Link

When using any of this project's source code, please cite:

@article{xxx,
  title = {xxx},
  author = {xxx},
  doi = {xxx},
  journal = {xxx},
  year = {xxx}
}

** Heroku puts processes to sleep after certain period of inactivity. Thus, it might take around 5-7 seconds to load.

Getting Started

Installation

First, install all the requireed libraries

$ python -m pip install -r requirements.txt   # Install al the requirements

Then, install the LS2D library (for example in editable mode)

$ python -m pip install --editable  .         # Install in editable mode

Training

Lets create a yaml configuration file according to your needs, for more information see the example settings.iris.yaml. This file allows you to define the path where the data is stored, the path where the workbench should be saved, to select which features should be used for training, which targets should be used to compute the performance metrics and last but not least, the methods to create the embeddings and the hyperparameters to consider during the grid search.

Once the configuration is completed, run the search script.

$ python search.py --yaml <yaml_file>

This script will create a new workbench within the output folder containing (i) the generated pipelines saved as pickle (.p) files, (ii) all the metrics obtained aggregated in the 'results.csv' file and the (iii) settings configuration.

Running the app

Using the script

Run the ls2d flask app (https://localhost:5000):

$ python server.py

Using Docker

Build and run the docker container (https://localhost:8000):

$ docker-compose build # Build
$ docker-compose up    # Run

Deploy to Heroku

Build an image and push

To build an image and push it to Container Registry, make sure that your directory contains a Dockerfile. Note that this will not inspect the docker-compose.yml nor the heroku.yml files and therefore the app should be run in the Docker file:

Thus, in the Dockerfile include:

$ CMD ["python", "server.py"]

Run the container to create the image:

$ heroku container:push <process-type> 

For this example

$ heroku container:push web 

Pushing an existing image

To push an image to Heroku, such as one pulled from Docker Hub o create with your your docker-compose instructions, tag it and push it according to this naming template.

$ docker tag <image> registry.heroku.com/<app>/<process-type>
$ docker push registry.heroku.com/<app>/<process-type>

For this example

$ docker tag main_web_1 registry.heroku.com/ls2d-demo/web
$ docker push registry.heroku.com/ls2d-demo/web

By specifying the process type in the tag, you can release the image using the CLI.

$ heroku container:release web

For more information, check the heroku official page here.

Generate docs

Go to the docs folder and run

    $ make github

This will clean the folder and thengenerate the documentation and copy it into gh-pages folder

License

Distributed under the GNU v3.0 License. See LICENSE for more information.

Limitations

At the moment, only one person can access it at the same time. This is because the database is just a global variable with the loaded csv file.