Working in the defense industry, it's extremely difficult to see how we're going to capitalize on these systems. It's a damn shame, because 95% of our document requirements are very-nearly-boilerplate, a great application for these early AI systems. I know the image processing AI things are coming along pretty well, but in some ways that's an easier problem.
The problem for us is multilevel stovepipes.
The biggest and grandest is ITAR, which restricts the physical path that data can take. Recently there was a tweak in draft that allowed for the data to take a path outside the physical USA, with the guarantee that the endpoints are encrypted. Not generally implemented though.
The second is what I would call datarest or Data Restrictions, which includes the whole data classification system of DoD, DoE, and others. If each model is only able to pull from its bucket, it's going to be a bad model.
The third is the proprietary problem. Since there are very few organizations competing - often they arrange for one to be the sole "winner", the ultimate smoke filled backroom - they make frameworks that generally work for just a single org. XML in Lockheed is not XML for Boeing, but replace "XML" with "anything that's normally standards-based". That's another layer of stovepipes.
DoD will have to provide the framework and big ass models for this stuff to work, but that's going to be a hell of a job, and will need serious political horsepower to keep it from being kidnapped by LockBoNorthRay.
As others have pointed out, running a truly large language model like GPT-3 isn't (yet) feasible on your own hardware - you need a LOT of powerful GPUs racked up in order to run inference.
https://github.com/bigscience-workshop/petals is a really interesting project here: it works a bit like bittorrent, allowing you to join a larger network of people who share time on their GPUs, enabling execution of models that can't fit on a single member's hardware.
The strongest one right now is a project called koboldai. However, instructing models like chat GPT are not open source yet, so it only runs stuff that writes books for you.
The problem with running a chat GPT sized system at home is that you need like 15 graphics cards to do it, and very few people have the equipment to manage something like that.
On a 24 gig graphics card you can run maybe 13 billion parameters, and stuff like chat GPT gets up above 200 billion.
I'm trying to set up a server at home with a bunch of old Tesla 40 series cards which have 24 gigs of vram and cost $200 each. A server that is a 2U super micro GPU server can hold six cards.
At the lovely power consumption of 1800 watts, about what your wall can deliver, you can hit about 96 gigs of VRAM for one to $2,000.
If you really wanted to go crazy you can get the v100 card, which are about $1,000 each, with 32 gigs . Trouble is, Nvidia starting to switch all these cards to their fuck you regular old business practice of making the connectors owned by Nvidia and changing the connector you need every single generation, so it's getting harder and harder to get these cards on the used market.
I known of bloom ai, https://huggingface.co/bigscience/bloom it has 1 billion more parameters. But it is
a completion ai, not a query and response ai. I wonder if it can be tweaked
The biggest and grandest is ITAR, which restricts the physical path that data can take. Recently there was a tweak in draft that allowed for the data to take a path outside the physical USA, with the guarantee that the endpoints are encrypted. Not generally implemented though.
The second is what I would call datarest or Data Restrictions, which includes the whole data classification system of DoD, DoE, and others. If each model is only able to pull from its bucket, it's going to be a bad model.
The third is the proprietary problem. Since there are very few organizations competing - often they arrange for one to be the sole "winner", the ultimate smoke filled backroom - they make frameworks that generally work for just a single org. XML in Lockheed is not XML for Boeing, but replace "XML" with "anything that's normally standards-based". That's another layer of stovepipes.
DoD will have to provide the framework and big ass models for this stuff to work, but that's going to be a hell of a job, and will need serious political horsepower to keep it from being kidnapped by LockBoNorthRay.