|
| 1 | +# DIETClassifier - Pytorch |
| 2 | + |
| 3 | + |
| 4 | +[](https://travis-ci.org/joemccann/dillinger) |
| 5 | + |
| 6 | +DIETClassifier stand for Dual Intent Entity from Transformers which can be used to do intent classification and entities recognition at the same time. |
| 7 | + |
| 8 | + - Using Huggingface Transformers's BERT architect |
| 9 | + - Wrapped by python, with various implemented functions (reads dataset from .yml, builds and trains model, gives dictionary ouput) |
| 10 | + |
| 11 | +# Requirements |
| 12 | + |
| 13 | +* [transformers] - Library for using transformers models in nlp task |
| 14 | +* [pytorch] - Framework for deep learning task in python |
| 15 | +* [fastapi] - Backend building framework |
| 16 | + |
| 17 | +You can also install all requirement packages by: |
| 18 | +```sh |
| 19 | +git clone https://github.com/WeiNyn/DIETClassifier-pytorch.git |
| 20 | +cd DIETClassifier-pytorch/ |
| 21 | +pip install -r requirements.txt |
| 22 | +``` |
| 23 | + |
| 24 | +### Demo |
| 25 | + |
| 26 | +You can use demo server to create a server that receive text message and predict intent, entities: |
| 27 | + |
| 28 | +- Download pretrained model from [this link](https://drive.google.com/drive/folders/1cAucUHO0FP_I-_atSpbyRwKEiflPPN7v?usp=sharing) |
| 29 | +- extract "latest_model" to "DIETClassifier-pytorch/" |
| 30 | +- run |
| 31 | +```sh |
| 32 | +uvicorn demo.server:app |
| 33 | +``` |
| 34 | + |
| 35 | +### Configuration |
| 36 | + |
| 37 | +All project configurations stored in [config.yml] file |
| 38 | +```yaml |
| 39 | +model: |
| 40 | + model: latest_model |
| 41 | + tokenizer: latest_model |
| 42 | + dataset_folder: dataset |
| 43 | + exclude_file: null |
| 44 | + entities: |
| 45 | + - working_type |
| 46 | + - shift_type |
| 47 | + intents: |
| 48 | + - WorkTimesBreaches |
| 49 | + - WorkingTimeBreachDiscipline |
| 50 | + - HolidaysOff |
| 51 | + - AnnualLeaveApplicationProcess |
| 52 | + - SetWorkingType |
| 53 | + - TemporarySetWorkingType |
| 54 | + - WorkingHours |
| 55 | + - WorkingDay |
| 56 | + - BreakTime |
| 57 | + - Pregnant |
| 58 | + - AttendanceRecord |
| 59 | + - SelectShiftType |
| 60 | + - LaborContract |
| 61 | + - Recruitment |
| 62 | + - SickLeave |
| 63 | + - UnpaidLeave |
| 64 | + - PaidLeaveForFamilyEvent |
| 65 | + - UnusedAnnualLeave |
| 66 | + - RegulatedAnnualLeave |
| 67 | + - rating |
| 68 | + device: cuda |
| 69 | +training: |
| 70 | + train_range: 0.95 |
| 71 | + num_train_epochs: 100 |
| 72 | + per_device_train_batch_size: 4 |
| 73 | + per_device_eval_batch_size: 4 |
| 74 | + warmup_steps: 500 |
| 75 | + weight_decay: 0.01 |
| 76 | + logging_dir: logs/ |
| 77 | + early_stopping_patience: 10 |
| 78 | + early_stopping_threshold: 0.0001 |
| 79 | + output_dir: results/ |
| 80 | +util: |
| 81 | + intent_threshold: 0.7 |
| 82 | + entities_threshold: 0.5 |
| 83 | + ambiguous_threshold: 0.2 |
| 84 | +``` |
| 85 | +
|
| 86 | +| Attribute | Explain | |
| 87 | +| --------- | ------- | |
| 88 | +| model | name of transformers pretrained model or path to local model | |
| 89 | +| tokenizer | name of transformers pretrained tokenizer or path to local tokenizer | |
| 90 | +| dataset_folder | folder that container dataset files, using rasa nlu format | |
| 91 | +| exclude_file | files in folder that will not be used to train | |
| 92 | +| entities | list of entities | |
| 93 | +| intents | list of intents | |
| 94 | +| device | device to use ("cpu", "cuda", "cuda:0", etc) | |
| 95 | +| train_range | range to split dataset into train and valid set | |
| 96 | +| num_train_epochs | number of training epochs | |
| 97 | +| per_device_train/eval_batch_size | batch size when train/eval | |
| 98 | +| logging_dir | directory to save log file (tensorboard supported) | |
| 99 | +| early_stopping_patience/threshold | hyper parameters for early stopping training | |
| 100 | +| output_dir | directory to save model while training | |
| 101 | +
|
| 102 | +### Usage |
| 103 | +
|
| 104 | +You can use DIETClassifierWrapper for loading, training, predicting in python code: |
| 105 | +```python |
| 106 | +from src.models.wrapper import DIETClassifierWrapper |
| 107 | + |
| 108 | +config_file = "src/config.yml" |
| 109 | +wrapper = DIETClassifierWrapper(config=config_file) |
| 110 | + |
| 111 | +#predict |
| 112 | +wrapper.predict(["How to check attendance?"]) |
| 113 | + |
| 114 | +#train |
| 115 | +#after training, wrapper will load best model automatically |
| 116 | +wrapper.train_model(save_folder="test_model") |
| 117 | +``` |
| 118 | + |
| 119 | +You can also use DIETClassifier in src.models.classifier as huggingface transformers model |
| 120 | +```python |
| 121 | +from src.models.classifier import DIETClassifier, DIETClassifierConfig |
| 122 | + |
| 123 | +config = DIETClassifierConfig(model="BERT-base-uncased", |
| 124 | + intents=[str(i) for i in range(10)], |
| 125 | + entities=[str(i) for i in range(5)]) |
| 126 | + |
| 127 | +model = DIETClassifier(config=config) |
| 128 | + |
| 129 | +``` |
| 130 | + |
| 131 | +### Notice |
| 132 | + |
| 133 | +* This DIETClassifier using BERT base as the base architect, if you want to change to RoBerta, ALBert, etc. You need to modify the DIETClassifier Class. |
| 134 | +* You can also use any BERT base pretrained from Huggingface transformers for creating and fine tune yourself |
| 135 | +* Please read the source code to understand how the dataset be created in case that you want to make dataset in another file format. |
| 136 | + |
| 137 | +[//]: # (These are reference links used in the body of this note and get stripped out when the markdown processor does its job. There is no need to format nicely because it shouldn't be seen. Thanks SO - http://stackoverflow.com/questions/4823468/store-comments-in-markdown-syntax) |
| 138 | + |
| 139 | + |
| 140 | + [dill]: <https://github.com/joemccann/dillinger> |
| 141 | + [git-repo-url]: <https://github.com/joemccann/dillinger.git> |
| 142 | + [john gruber]: <http://daringfireball.net> |
| 143 | + [df1]: <http://daringfireball.net/projects/markdown/> |
| 144 | + [markdown-it]: <https://github.com/markdown-it/markdown-it> |
| 145 | + [Ace Editor]: <http://ace.ajax.org> |
| 146 | + [node.js]: <http://nodejs.org> |
| 147 | + [Twitter Bootstrap]: <http://twitter.github.com/bootstrap/> |
| 148 | + [jQuery]: <http://jquery.com> |
| 149 | + [@tjholowaychuk]: <http://twitter.com/tjholowaychuk> |
| 150 | + [express]: <http://expressjs.com> |
| 151 | + [AngularJS]: <http://angularjs.org> |
| 152 | + [Gulp]: <http://gulpjs.com> |
| 153 | + |
| 154 | + [PlDb]: <https://github.com/joemccann/dillinger/tree/master/plugins/dropbox/README.md> |
| 155 | + [PlGh]: <https://github.com/joemccann/dillinger/tree/master/plugins/github/README.md> |
| 156 | + [PlGd]: <https://github.com/joemccann/dillinger/tree/master/plugins/googledrive/README.md> |
| 157 | + [PlOd]: <https://github.com/joemccann/dillinger/tree/master/plugins/onedrive/README.md> |
| 158 | + [PlMe]: <https://github.com/joemccann/dillinger/tree/master/plugins/medium/README.md> |
| 159 | + [PlGa]: <https://github.com/RahulHP/dillinger/blob/master/plugins/googleanalytics/README.md> |
0 commit comments