Distill-ALBERT

Hidden-State-Aligned heterogeneous distillation of ALBERT using OpenWebText Corpus.

This implementation is currently based on huggingface/transformers.

Name		Name	Last commit message	Last commit date
Latest commit History 34 Commits
training_configs		training_configs
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
binarize.sbatch		binarize.sbatch
binarized_data.py		binarized_data.py
bookcorpus.py		bookcorpus.py
clean_example.py		clean_example.py
distiller.py		distiller.py
freeze.yml		freeze.yml
glue.py		glue.py
glue_metric.py		glue_metric.py
grouped_batch_sampler.py		grouped_batch_sampler.py
lm_seqs_dataset.py		lm_seqs_dataset.py
load_dataset_test.py		load_dataset_test.py
new_node.sh		new_node.sh
openwebtext.py		openwebtext.py
print_layer.py		print_layer.py
run_glue.py		run_glue.py
run_glue.sbatch		run_glue.sbatch
test.sh		test.sh
test_train.sh		test_train.sh
token_counts.py		token_counts.py
token_counts.sbatch		token_counts.sbatch
train.py		train.py
train.sbatch		train.sbatch
tsv_to_csv.py		tsv_to_csv.py
utils.py		utils.py
wikipedia.py		wikipedia.py

Provide feedback