Hacker News new | ask | show | jobs
by M0r13n 37 days ago
I don't think that LLMs are trustworthy companions in managing a complex metabolic disease like diabetes - especially if you deviate (ever so slightly) from the norm (very lean, very active, strict diet, etc.)!

I'm a T1D myself and like to experiment with ChatGPT (or Opus). My experiences are mixed

LLMs are overly cautious when it comes to correcting with insulin. They regularly advise against correcting before going to bed, even if this means that my blood glucose remains above 140 mg/dl for the whole night.

I am following a low to medium carb diet (<100g a day). ChatGPT always nudges me to consume more carbohydrates, even though I have a TIR of 90% (70-150 mg/dl). Why would I change my diet if it currently works very well for me? Still, most LLMs seem to favor carbs for some reason.

I am using Fiasp as my fast acting insulin. Typically, I inject around 1 to 4 IUs of Fiasp. Its glucose-lowering effect typically lasts for roughly 2-3 hours. Therefore, I know that it is safe to re-inject after three hours without risking insulin stacking. But ChatGPT regularly advises against that and wants me to wait another 1-2 hours.

I am not against automating diabetes management. In fact, I really appreciate projects that help with that. But I don't consider LLMs to be helpful in this regard. Their combination of training data bias, liability aversion, lack of context, and one-size-fits-all thinking disqualifies them from such tasks.

3 comments

[flagged]
(I'm a mod here) It looks like a useful project and I'm glad you submitted it, but please don't post AI-generated or AI-edited comments to HN - see https://news.ycombinator.com/newsguidelines.html#generated and https://news.ycombinator.com/item?id=47340079
Really appreciate this feedback. Every failure you named is real. I have had similar experiences with ChatGPT giving me bad information. I plan on solving for this by expanding on the RAG system that's being used. The AI gets its context based on settings you are giving it. The platform has some settings built in around insulin types but this needs to be expanded. The platform is not designed to give you dosing recommendations I really built this to help you understand your own data so you don't fly blind. Any thing the AI says to you is clearly labeled as not medical advice and the expectation is that your medical provider is in the loop before you make any medical decisions. I will take your feedback into consideration as I continue to explore new ways to shape the output you get from the platform.
I understand this instinct, but I can see the appeal of capabilities that are well within the limits of a well-designed agentic system.

Imagine asking such a system, "look at my postprandial response to dosing for the past week and make ratio suggestions for breakfast, lunch, and dinner." This is genuinely helpful, saves time, and well within the reasoning limits. You could spot check if you like.

Is it worth setting up such an assistant for the value you'd get out of it? I guess that's on the user and how many similar use cases exist.

> look at my postprandial response to dosing for the past week and make ratio suggestions for breakfast, lunch, and dinner

I'm not so sure about that. A patient absolutely must critically evaluate the LLM's suggestions. A naive user risks severe complications. A user with that kind of competence, however, doesn't need an LLM for such trivial adjustments - they're obvious