Skip to content

Commit 955a3c8

Browse files
committed
add performance
1 parent d26e062 commit 955a3c8

File tree

1 file changed

+12
-0
lines changed

1 file changed

+12
-0
lines changed

README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,18 @@ python setup.py install
2929

3030
### Performance
3131

32+
- [AWS P3 2xlarge instance](https://aws.amazon.com/ec2/instance-types/p3/) is used to the experiment. (One Tesla V100 GPU with 8 vcpus)
33+
- results can be reproduced by running `examples/example_w2v.py` and `examples/example_lda.py`
34+
- To evaluate w2v model, we used `evaluate_word_pairs` function ([ref link](https://radimrehurek.com/gensim/auto_examples/tutorials/run_word2vec.html#evaluating)) in gensim, note that better performance on WS-353 test set does not mean that the model will workbetter in application as desribed on the link. However, it is good to be measured quantitively and fast training time will be at least very objective measure of performaance.
35+
- I trained W2V model on quora-duplicat-questions dataset from gensim downloader api with cusim and the performance with gensim.
36+
- To evaluate LDA model, I think there is no good way to measure the quality of traing results quantitatively. But we can check the model by looking at the top words of each topic. Also, we can compare the training time here.
37+
- W2V (CBOW, negative sampling)
38+
39+
| attr | 1 workers | 2 workers | 4 workers | 8 workers | GPU |
40+
|:--------------|------------:|------------:|------------:|------------:|---------:|
41+
| training_time | 181.009 | 102.302 | 58.9811 | 47.7482 | 9.60324 |
42+
| pearson | 0.203882 | 0.207705 | 0.221758 | 0.198408 | 0.331749 |
43+
| spearman | 0.25208 | 0.254706 | 0.275231 | 0.238611 | 0.295346 |
3244

3345
### Future tasks
3446

0 commit comments

Comments
 (0)