Hacker News new | ask | show | jobs
by montecarl 485 days ago
Do you think that you can use those machines for confidential workflows for enterprise use? I'm currently struggling to balance running inference workloads on expensive AWS instances where I can trust that data remains private vs using more inexpensive platforms.
3 comments
tucnak 485 days ago
Of course you cannot use these machines "for confidential workflows for enterprise use", at least with AWS you know whose computer you're working with, but also keep in mind that it's really hard to steal your data as long as your data stays in memory, and you use something like mTLS to actually get it in and out of memory via E2EE. You can figure out the rest of your security model along the way, but anything sensitive (i.e. confidential) would surely fall way out of this model.
link
mmastrac 485 days ago
I read through the FAQ and the answer is "no", but they say it basically as "nobody really cares what your data is".

I wouldn't put anything confidential through it.

link
gopher_space 485 days ago
What's your workload and timeline? I'm wondering how much of that workload could be handled in-house.
link
montecarl 485 days ago
Just currently exploring how custom AI workflows (e.g. text to sql, custom report generation using private data) can help given the current SOTA. Looking to develop tooling over the next 3-6 months. I'd like to see what we can come up with before dropping $50-100k on hardware.
link
gopher_space 480 days ago
I threw together a toy project to see if it would help me understand the basic concepts and my takeaway was that, if you can shape your input into something a dedicated classification model (e.g. YOLO for document layout analysis) can work with, you can farm each class out to the most appropriate model.

It turns out that I can run most of the appropriate models on my ancient laptop if I don't mind waiting for the complicated ones to finish. If I do mind, I can just send that part to OpenAI or similar. If your workflow can scale horizontally like my OCR pipeline crap, every box in your shop with RAM >= 16GB might be useful.

Apologies if this is all stuff you're familiar with.

link