How can I solve this problem in ChatGLM-6b? AttributeError: ‘ChatGLMTokenizer’ object has no attribute ‘sp_tokenizer’
The default program from “https://github.com/THUDM/ChatGLM-6B” works out while I was running api.py.
But recently when I try to run it again it suddenly errors:”AttributeError: ‘ChatGLMTokenizer’ object has no attribute ‘sp_tokenizer'”.
I tried to solve it on my own but nothing happens. I reinstalled transformers==4.27.1, updated tokenization_chatglm.py on https://huggingface.co/THUDM/chatglm-6b/blob/main/tokenization_chatglm.py, etc.
I cannot understand how tokenization_chatglm.py or api.py works for they’re too complex for me.
I wonder if there is anything I haven’t done to solve my problem and if there is anyone else having the same problem. Please someone help me out.