Improve vocab decoding

[benlebrun]

This PR fixes an issue in which vocabulary decoding was failing because some tokenizers contained special tokens with characters that were not in the byte decoder. We now convert special tokens to bytes by directly encoding their string representation instead of passing it through the byte decoder.

I also got rid of some decoding functionality that wasn't being used by any of our tests. We can add it back in if we identify a tokenizer for which it is needed.

[gitstream-cm]

🥷 Code experts: sritchie

sritchie, benlebrun have most 👩‍💻 activity in the files.
benlebrun, sritchie have most 🧠 knowledge in the files.

See details

genlm_backend/tokenization/bytes.py

Activity based on git-commit:

	sritchie	benlebrun
MAR
FEB
JAN	77 additions & 350 deletions	303 additions & 10 deletions
DEC		273 additions & 0 deletions
NOV
OCT

Knowledge based on git-blame:
benlebrun: 74%
sritchie: 26%

requirements-dev.txt

Activity based on git-commit:

	sritchie	benlebrun
MAR	8 additions & 0 deletions
FEB
JAN
DEC
NOV
OCT

Knowledge based on git-blame:
sritchie: 100%

tests/test_vocabulary.py

Activity based on git-commit:

	sritchie	benlebrun
MAR
FEB
JAN	37 additions & 17 deletions	1 additions & 1 deletions
DEC		90 additions & 2 deletions
NOV
OCT

Knowledge based on git-blame:
benlebrun: 66%
sritchie: 34%

To learn more about /:\ gitStream - Visit our Docs