[valine]

I’ve got 4 different llama models running locally with CUDA and can freely switch between them, including LLaVA which is a multimodal LLaMA variant.

None of them are particularly difficult to get running, the trick is to search the project’s github issue tracker. 99% of the time your problem will be in there with steps to fix it.

[jiggawatts]

> the trick is to search the project’s github issue tracker.

What ever happened to the crazy notion of Dockerfiles that simply build successfully?

Isn’t half the point of containerisation that it papers over the madness of the Python module ecosystem?

[avereveard]

The problem with that is gpu access from the docker image and iirc Nvidia doesn't have a Windows passtrough cuda driver

[UncleEntity]

They require someone to take time out of their busy schedule to build?

Complaining that people won’t work for you for free is a bit much, don’t you think?

[celdon25]

I remember the days when lowering the barrier to entry was considered to be a safe investment since it would pay for itself through increased project interest (and more contributions by the community).

Now it is apparently seen as "working for free for ungrateful people"

[valine]

Most of the repos for these new LLaMA derivatives are brand new, and put together by people who specialize in ML research not software engineering. I’m just happy to have access to their code. I’m sure most of these teams would be happy to merge a PR that lowered the barrier to entry, why don’t you get on that?

[jiggawatts]

> specialize in ML research not software engineering.

That has nothing to do with Python tooling being bad. A safe assumption is that Python package managers are being developed by developers, who have no excuse.

If a C++ codebase developed by scientists had null pointer exceptions in it, then I could excuse things. But if the C++ compiler itself introduced unforced null pointer errors, then it absolutely deserves criticism.

It shouldn't be possible for a ML researcher to use Conda or whatever package manager in a way that despite using a formally specified "requirements.txt", it won't build a week later because of how loose the specification of module versions is allowed to be.

The Python attitude and more specifically Conda is at fault here, not the ML researching trying to get his job done.

[alchemist1e9]

One point you have I absolutely agree with:

> Conda is at fault here

Conda is so so bad. But trying to explain why to people who have fallen into it’s trap is difficult. People don’t realize the packages are not signed on enough information to reproduce them. The optimizer to find matching versions to make an environment that satisfies your constraints is really bad idea.

As an experienced C++ developer unfortunately and fortunately I’ve concluded the most “correct” solution is to use nixpkgs.

[qaq]

Poetry is fairly reasonable package manager

[UncleEntity]

There’s a big difference between making a project easy to use and requiring they package it all up in someone’s preferred container format which can’t even legally include all the dependencies.

Facebook seems to be pretty hands off (as is expected since the code is open source) unless you distribute the model weights and then they drop the dmca banhammer.

So, yeah, simply complaining with no effort to understand the problem is kind of ungrateful.

[MakeUsersWant]

Could you publish a set of known-good versions (pip freeze, OS version, etc)?