[pixl97]

"Simple infrastructure security"

Infrastructure security is not simple, hence why good infrastructure security, uh, people get paid a lot to secure stuff and why we see shit get hacked all the time.

An AI model just hacked out of its infrastructure and into someone else's systems and you're like "eh, no big deal". That capability alone could hack half the US.

[sandeepkd]

>That capability alone could hack half the US.

This almost seems like believing in magic. What really has happened is you have collected all the hacking/abuse/malicious flows/code in one place. Greedy or A* algorithms have been discovered a long ago, the script is executing the flows for all possible permutations.

Something has to be insecure to be hacked in the first place.

[pixl97]

Looking at the hundreds of linux and windows CVEs in the past month tell me I have little to worry about things being secure.

[sandeepkd]

Security exists in multiple layers. CVEs do not make a system vulnerable by default, there has to be a way to exploit them. Questions one can ask, is your system connected to internet, what does your fire wall looks like, which ports are open and why, what processes are running and why.

Most sensitive systems in the past had a concept of DMZ where one cannot reach through internet for security reasons except for very tightly controlled path. One had to go in the physical data centers for certain operations.

If you are running a script in a loop on your system which gets executes any/all commands from a third party system then you probably do not take security that seriously in the first place.

[pixl97]

An LLM is a third party to itself. It can write code spontaneously and hack itself.

[sandeepkd]

LLM is not hacking itself, its a PR spin if anything. LLM is an input output system, there is no more hacking than that. What gets abused or hacked is how the input is parsed, validated to guard certain information (training data). Security was not the design principal which was strongly considered in the design of LLMs.

LLM being hacked only means that you are able to get the training data out of LLMs which was supposedly being guarded by some probabilistic harness. Thats not security, its more like a prayer for security.

[sandeepkd]

Simplicity is relative from where I see things in a particular domain. Security does not have any direct ROI on it, the security engineers are hired way too late in the game when all the stack is almost buried in deep decisions. The concept of security engineers (how to secure) and product engineers (what to secure) has made the gap way to wide to make the security meaningful.

[anon-3988]

> A malicious dataset abused two code-execution paths in our dataset processing (a remote-code dataset loader and a template-injection in a dataset configuration)

I am sure they are paid well but they literally have RCE embedded in their infra. How is this acceptable?

[pixl97]

You have an unnumbered amount of RCE's in your infra now, you just don't know they exist yet.

LLMs are very good at testing for and finding exploits, especially in unfiltered models with unlimited tokens.