The LLaMA PR has been merged into transformers

https://github.com/huggingface/transformers/pull/21955

The tokenizer class has been changed from

"LLaMATokenizer"

to

"LlamaTokenizer"

It is necessary to edit this change in every tokenizer_config.json
that you had for LLaMA so far.
This commit is contained in:
oobabooga 2023-03-16 11:16:17 -03:00
parent d54f3f4a34
commit 23a5e886e1

View file

@ -9,4 +9,4 @@ rwkv==0.4.2
safetensors==0.3.0
sentencepiece
tqdm
git+https://github.com/zphang/transformers.git@68d640f7c368bcaaaecfc678f11908ebbd3d6176
git+https://github.com/huggingface/transformers