Description

Codes for our article, "A Pitfall of Learning from User-generated Dataset", on a type of class noise specific to user-generated datasets (e.g. customer reviews) called Subjective Class Issue. We used datasets provided generaously by Donorschoose.org, Yelp Review, and Amazon Fine Food. By following the usage below, you can replicate the results shown in our paper.

Requirements

If you would like to run our notebooks, please follow the steps below:

Download "Project Essays" provided by Donorschoose.org
git clone this repo
cd into the repo
open python virtual environment
run pip install -r requirements.txt
in the python virtual environment, open jupyter notebook
open ipynb files, where the name represents the tasks of each notebook (*make sure you train doc2vec before you run tsne plots)

Name		Name	Last commit message	Last commit date
Latest commit History 14 Commits
.ipynb_checkpoints		.ipynb_checkpoints
nfis		nfis
preprocess		preprocess
F1-macro scores of classes in Donorschoose dataset.ipynb		F1-macro scores of classes in Donorschoose dataset.ipynb
LICENSE		LICENSE
Plot Donorschoose Data by tsne.ipynb		Plot Donorschoose Data by tsne.ipynb
Plot NFIS with Doroschoose data.ipynb		Plot NFIS with Doroschoose data.ipynb
Poverty levels of schools in school districts with no more than two schools.ipynb		Poverty levels of schools in school districts with no more than two schools.ipynb
README.md		README.md
requirements.txt		requirements.txt
train_doc2vec.py		train_doc2vec.py