[bawana]

MacBook with M1 chip here.python installed with homebrew tried to install with: pip install fschat

then tried to run it with: python3 -m fastchat.serve.cli --model -name vicuna-7b --device mps --load-8bit

got this:

traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/opt/homebrew/lib/python3.11/site-packages/fastchat/serve/cli.py", line 9, in <module> from transformers import AutoTokenizer, AutoModelForCausalLM, LlamaTokenizer ModuleNotFoundError: No module named 'transformers'

so I did this:

  pip install transformers command

tried again:

python3 -m fastchat.serve.cli --model -name vicuna-7b --device mps --load-8bit

got:

Traceback (most recent call last): File "/opt/homebrew/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1126, in _get_module return importlib.import_module("." + module_name, self.__name__) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1206, in _gcd_import File "<frozen importlib._bootstrap>", line 1178, in _find_and_load File "<frozen importlib._bootstrap>", line 1128, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1206, in _gcd_import File "<frozen importlib._bootstrap>", line 1178, in _find_and_load File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 690, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 940, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/opt/homebrew/lib/python3.11/site-packages/transformers/models/__init__.py", line 15, in <module> from . import ( File "/opt/homebrew/lib/python3.11/site-packages/transformers/models/mt5/__init__.py", line 29, in <module> from ..t5.tokenization_t5 import T5Tokenizer File "/opt/homebrew/lib/python3.11/site-packages/transformers/models/t5/tokenization_t5.py", line 26, in <module> from ...tokenization_utils import PreTrainedTokenizer File "/opt/homebrew/lib/python3.11/site-packages/transformers/tokenization_utils.py", line 26, in <module> from .tokenization_utils_base import ( File "/opt/homebrew/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 74, in <module> from tokenizers import AddedToken File "/opt/homebrew/lib/python3.11/site-packages/tokenizers/__init__.py", line 80, in <module> from .tokenizers import ( ImportError: dlopen(/opt/homebrew/lib/python3.11/site-packages/tokenizers/tokenizers.cpython-311-darwin.so, 2): no suitable image found. Did find: /opt/homebrew/lib/python3.11/site-packages/tokenizers/tokenizers.cpython-311-darwin.so: mach-o, but wrong architecture /opt/homebrew/lib/python3.11/site-packages/tokenizers/tokenizers.cpython-311-darwin.so: mach-o, but wrong architecture

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/opt/homebrew/lib/python3.11/site-packages/fastchat/serve/cli.py", line 9, in <module> from transformers import AutoTokenizer, AutoModelForCausalLM, LlamaTokenizer File "<frozen importlib._bootstrap>", line 1231, in _handle_fromlist File "/opt/homebrew/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1116, in __getattr__ module = self._get_module(self._class_to_module[name]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1128, in _get_module raise RuntimeError( RuntimeError: Failed to import transformers.models.auto because of the following error (look up to see its traceback): dlopen(/opt/homebrew/lib/python3.11/site-packages/tokenizers/tokenizers.cpython-311-darwin.so, 2): no suitable image found. Did find: /opt/homebrew/lib/python3.11/site-packages/tokenizers/tokenizers.cpython-311-darwin.so: mach-o, but wrong architecture /opt/homebrew/lib/python3.11/site-packages/tokenizers/tokenizers.cpython-311-darwin.so: mach-o, but wrong architecture

[zhwu]

You need to use the transformers from the main branch instead of the pypi version, because the llama support is recently added. According to the readme of the repo, you need to install transformers with: pip3 install git+https://github.com/huggingface/transformers

[rnosov]

It looks like you're on python 3.11 which has some issues with Pytorch. Downgrade to python 3.10 and try running it again.

[forgingahead]

This is incredible to me (not your comment per se, but what you're referencing). I really don't understand how brittle and fragile Python is with all its dependencies. It's crazy to me that a simple bump from 3.10 to 3.11 can break Pytorch. This is like bumping your Ruby version up one level and suddenly Rails doesn't work.

Why on earth is Python like this? It's so frustrating coming from other languages where the dependency management plan isn't just so YOLO and free-for-all.

[matwood]

I have despised Python ever since the 2=>3 transition for the reasons you say. Tools like pyenv help, but it's still a mess. It makes me sad that all the popular ML tooling ends up built in Python.

[jondwillis]

I wonder how much the space has been encumbered by Python’s relative weaknesses. As a bit of an outsider, I kind of assume there’s some hidden advantage of Python for AI/ML that I just don’t “get.”

[matwood]

I think Python is easy to learn/use for programming adjacent fields like data science. It's seen as easy for non-traditional programmers.

[baq]

C API changes between minor versions. It’s one of the bigger reasons to bump the minor version.

I agree though, Python packaging is consistently hell in an otherwise pleasant environment… saying that as a Python user since 1.5.