Hacker News new | ask | show | jobs
by CuriouslyC 808 days ago
Given you're running a 3090 24gb, go with Oobabooga/Sillytavern, and don't come here for advice on this stuff, go to https://www.reddit.com/r/LocalLLaMA/, they usually have a "best current local model" thread pinned, and you're less likely to get incorrect advice.
5 comments

Non-cancer link: https://old.reddit.com/r/LocalLLaMA/ (though that won't work on VPNs now)

(I just got a little emotional because these are things we used to say on reddit and now we say them about reddit. How the mighty have fallen)

> (though that won't work on VPNs now)

Did for me just now, although as of a week or two ago reddit has been blocking many of my attempts to access through a VPN old. or not. Usually need to reconnect about 3 or 4 times before a page will load.

VPNs were being blocked over the weekend (I checked after seeing https://news.ycombinator.com/item?id=39883747) but as of today within the last hour or so it seems like it works again for both old.redddit.com and regular reddit.com so something changed.
I miss when redditors were really nice xD That was a while ago now. (Or maybe people just make me more nervous these days lol)
RFC: Is there some straightforward way to use a Pi-hole like setup to 302 redirect `reddit.com` to `old.reddit.com`?

I'm sure there are myriad browser extensions that will do it at the DOM level, but that's such a heavy-handed solution, and also lol I'm not putting an extension on the cartesian product of all my browsers on all my machines in the service of dis-enshittifying one once-beloved social network.

I doubt you could do this at the DNS level. Reddit probably loads static assets from `reddit.com`.

You could perhaps intercept the HTTP requests, but that would require decrypting SSL, so you'd have to MITM yourself if you wanted to do it at the network level.

You'd replace the DNS request for `reddit.com` to some device that intercepts the traffic. That device would redirect the HTTP request to `old.reddit.com` if applicable; static assets would be routed to `reddit.com`. I don't know how things like HSTS and certificate pinning fit into an idea like this.

Doing this at the device level is probably easiest. I've used Redirector [0] in the past and it works well.

[0]: https://github.com/einaregilsson/Redirector

I found this extension that makes it even easier, and you can configure it to go to either old.reddit.com or new.reddit.com (instead of going to 3rd generation even worse sh.reddit.com) without having to configure Redirector: https://chromewebstore.google.com/detail/ui-changer-for-redd...
OP said they didn't want to use an extension, but if they did I would vote Reddit Enhancement Suite anyway
I guess you have a webserver there on your pihole?

Create a vhost for reddit.com, add reddit.com to your hosts file to point to the webserver (or setup a stub in the dns to the vhost webserver), and do a redirect to old.reddit.com on your webserver vhost.

This is probably the easiest (simple http server and a php file) and most reliable (https can be redirected to http to avoid any ssl headaches) way to accomplish what the OP wants.
Most straightforward way is to login once and then turn on the "Opt out of redesign" option. Done.

I can't imagine any type of Pi-hole setup which would be faster, seems like a "holding a hammer" kind of solution.

A nice thing about not being logged in is that it is less addictive as a lot of the front page is less relevant.
>Given you're running a 3090 ... "best current local model" thread pinned, and you're less likely to get incorrect advice

why do you think my info about the 3090 https://en.wikipedia.org/wiki/IBM_3090 is going to be anything less than up-to-date?

ibm 3090 makes 3 slots seem quite svelte.

on the other hand, 24gb of late 80's memory... how many acres of raised floor data center would that take?

That doesn't seem to be the case, at least right now. There's no pinned post as far as I can tell, and searching the sub for `best model` returns mostly questions.
It might be because a lot of people use chatbot arena leaderboard for that purpose now, check it out here and look for local models:

https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...

Looks interesting, but I see no way to filter local models. There are only listings of popular SaaS models. Could be because I’m on mobile?
The local models should be colored differently and have licenses/etc that should tip you off. You might need to scroll down the list to see one though.
If you are looking for a downloadable app as an alternative to Oobabooga / Sillytavern, we built https://github.com/transformerlab/transformerlab-app as a way to easily download, interact, and even train local llms.
The reason everyone loves oobabooga is that it’s made with a maximalist design paradigm.

Everyone in the VC world misunderstands why oobabooga is successful and tries to embrace not maximalism.

Your example product to benchmark yourself against is blender, if you want to serious compete against oobabooga. You need maximalism

I wouldn't say maximalism so much, as Oobabooga was designed to emulate Automatic1111, where it's a good enough tool by itself, but then it has a strong extension ecosystem, and it can be utilized through other tools via web service. It's not great at any one thing but it's "good enough" pretty much everywhere.
I agree with you re:extensions. I know something of the extension ecosystem for it - I built one - https://github.com/Hellisotherpeople/llm_steer-oobabooga

Oobabooga the closest thing we have to maximalism. It has exposure for by far the largest number of parameters/settings/backends compared to all others.

My main point is that the world yearns for a proper "Photoshop for text" - and no one has even tried to make this (closest is oobabooga). All VC backed competitors are not even close to the mark on what they should be doing here.

Just let reddit die already, the VPN blocking is just bullshit.