You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Jul 21, 2025. It is now read-only.
@@ -37,7 +40,7 @@ The following is a support matrix of LightSeq **inference** library compared wit
37
40
38
41
## Performance
39
42
40
-
### Training
43
+
### [Training](./lightseq/training)
41
44
Here we present the experimental results on WMT14 English to German translation task based on Transformer-big models. We train Transformer models of different sizes on eight NVIDIA Tesla V100/NVIDIA Ampere A100 GPUs with data parallel and fp16 mixed precision.
42
45
[Fairseq](https://github.com/pytorch/fairseq) with [Apex](https://github.com/NVIDIA/apex) is choosed as our baseline.
43
46
@@ -47,7 +50,7 @@ We compute speedup on different batch size using the WPS (real words per second)
47
50
48
51
More results is available [here](./docs/training/performance.md)
49
52
50
-
### Inference
53
+
### [Inference](./lightseq/inference)
51
54
Here we present the experimental results on neural machine translation based on Transformer-base models using beam search methods.
52
55
We choose Tensorflow and
53
56
[FasterTransformer](https://github.com/NVIDIA/DeepLearningExamples/tree/master/FasterTransformer) as a comparison.
@@ -79,6 +82,8 @@ sh examples/training/fairseq/ls_fairseq_wmt14en2de.sh
79
82
80
83
To compare lightseq with fairseq, delete the arguments with `ls_`prefix to using the original fairseq implementation
81
84
85
+
More usage is available [here](./lightseq/training/README.md).
86
+
82
87
### Fast inference from HuggingFace bart
83
88
84
89
We provide an end2end bart-base example to see how fast Lightseq is compared to HuggingFace. First you should install these requirements.
@@ -97,14 +102,15 @@ python ls_bart.py
97
102
98
103
LightSeq installation from pypi only supports python 3.6 to 3.8 on Linux for now. Consider compiling from source if you have other environments.
99
104
105
+
More usage is available [here](./lightseq/inference/README.md).
100
106
101
107
## Cite Us
102
108
103
109
If you use LightSeq in your research, please cite the following paper.
104
110
105
111
```
106
112
@InProceedings{wang2021lightseq,
107
-
title = "{L}ight{S}eq: A High Performance Inference Library for Transformers",
113
+
title = "{L}ight{S}eq: A High Performance Inference Library for Transformers",
108
114
author = "Wang, Xiaohui and Xiong, Ying and Wei, Yang and Wang, Mingxuan and Li, Lei",
109
115
booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers (NAACL-HLT)",
110
116
month = jun,
@@ -117,4 +123,4 @@ If you use LightSeq in your research, please cite the following paper.
117
123
## Contact
118
124
119
125
Any questions or suggestions, please feel free to contact us at
0 commit comments