Error and general question

[jbmaxwell]

First the error — I get this both when trying to run the notebook locally (ubuntu 18.04) and from Colab:

Traceback (most recent call last):
  File "main.py", line 89, in <module>
    num_predict=args.num_predict)
  File "/content/xlnet-Pytorch/data_utils.py", line 345, in make_permute
    reuse_len)
  File "/content/xlnet-Pytorch/data_utils.py", line 292, in _local_perm
    non_mask_tokens = (~is_masked) & non_func_tokens
RuntimeError: Expected object of scalar type Byte but got scalar type Bool for argument #2 'other' in call to _th_and

Any ideas?

For the general question; I really want to pretrain from scratch with my own small corpus. Any tips on how I might go about doing that?

Thanks

[cmdllx]

conda install pytorch=1.0.1 torchvision=0.2.2 cudatoolkit=10