You know, I’m not sure but word on the street from folks who have used the experimental OmniPod system is that it handles unannounced meals like Chinese pretty well. Of course I don’t know what “pretty well” is – I heard this from a teenager so maybe pretty well means not spiking over 250 ever…
So my guess is that as you get smarter algorithms you can begin to overcome some of the insulin time lag.
2 Likes
How customizeable is the system you are using?
I think any out-of-the-box algorithm that is written for the masses will always fall short, unless it allows the individual to customize it.
2 Likes
I think that’s part of what makes openAPS work. I think the actual assumptions aren’t great… but you can tweak it a ton. My guess is that any system with even crummy assumptions that can be tweaked will give decent results, while all one-size-fits-all solutions need to be much more sophisticated to work decently well.
3 Likes
This is an excellent review. Thanks for posting it, @drbbennett!
According to my endocrinologist (the 670G isn’t out in Canada yet, but they are running clinical trials), auto mode will only adjust insulin doses so far and for so long before kicking you into manual mode to fix a problem. So if you’re running high, the pump will increase your basal a certain amount for a certain length of time, but if you haven’t come back down into range at a certain point, it goes back to manual mode for you to adjust basals or change sites or whatever needs to be done.
2 Likes
Can you elaborate? I am currently thinking about and testing out some modifications to Loop prediction and dosing algorithms and would love to hear your thoughts on underlying assumptions and models (both OpenAPS and Loop are based on more or less the same basic assumptions).
How much time do you have? For starters want to say I really appreciate the algorithm and find it really life-altering and helpful much of the time.
But of course there are things I would totally change.
-
The idea of taking the calculations we all already know – ISF, carbF, basal rate and then simply using these same numbers in a linear fashion to make predictions for the future is problematic. For one, DIA becomes a much more important, hidden variable that can’t really be tweaked to the exact level needed to eliminate hidden stacking. For another, we all have experienced that situation where ISF isn’t linear across BG range, so when you’re high the predictions for how much you need usually aren’t enough, aren’t enough, aren’t enough…until you’ve overshot. I know that in theory that means ISF is too weak and you need to strengthen it, but in my experience there’s no perfect ISF that will work for the entire BG range, so you’re basically picking a number and hoping it’s adequate for catching either highs or preventing lows, not both.
-
Other assumptions I’m not crazy about. The autosens function doesn’t work well for us because I believe there’s an assumption that the last 6 hours of data is required to change your settings. For us, BG numbers change more rapidly than that, so autosense is always behind the curve to a significant degree in catching trends. Then if you have 3 programmed ISFs, there are weird discontinuities in how autosense implements its recommendations at those transition points.
-
I don’t like that the values used as the jumping off point are the hard-coded values in your pump. You should be able to change variables automatically without having to go in and change the hard-coded functions. In general I think openAPS parameters should be their own thing and that in auto mode you should have one set of parameters and a different set in manual mode.
-
Another issue I have is with the way it modifies basals around mealtimes. The model for how long carbs stick around is just never perfect. Overall I’d say it is too slow to take away insulin at the beginning after a bolus, so he might go low, but then takes away insulin for too long over all…so we get this two-camel-hump thing going on with his BG trend after a meal. This doesn’t happen if we’re totally in manual mode.
-
When someone is low, there should be an option to set a zero temp basal of 0 until they rise above some threshold BG (80). We sometimes have a situation where technically Samson has negative IOB for whatever reason, and is at 55 mg/DL, and as soon as he’s +2 the program cranks on with insulin again because it predicts he will be high because of his negative IOB…which brings me to the biggie
-
Assuming your BG should be steady at the programmed in basal rate, and using that basal rate as the “net zero” point. the idea of negative IOB is nonsensical, and at a certain point (say you’ve been running in the negative IOB range for hours), what that really means is your body just needs less insulin for its baseline function… you should be throwing out the programmed basal rate for calculations at that point. The program just doesn’t do well with that. That’s why to me the idea of just having a constantly changing ISF seems to intrinsically work better, it’s much more in the moment and less based on who you were in the past.
-
openAPS was designed to be totally intelligible to people used to the normal way of doing things…but that may not be the best algorithm for a predictive algorithm, and ironically there are all these emergent properties and edge cases that crop up daily if not hourly that aren’t what you’d predict, and tweaking it is not simple because there are so many variables… the main three of course, but also ones like DIA, carb absorption, number of carbs. So it’s like a cockpit with so many knobs to turn, and yet we don’t have clear models of how dialing those knobs works.
-
The same settings we use in manual mode do very different things than when they’re used with openAPS. Not sure why.
For instance, the “basal rate” needed to get the program to work okay for our kid may make him go low constantly if it’s in manual mode. Or we find a carb F that is perfect when in manual mode, but because of the algorithm it winds up causing highs later on because it takes away too much insulin because carb absorption is weird. Maybe that means you need to be tweaking the absorption rate of the carbs… but really, ultimately, every single food has as lightly different absorption profile and the idea that you have to be constantly tweaking this doesn’t work for me.
What I’ve noticed is that, using our programmed in rates, whenever our son is unable to use openAPS and is in manual mode his overall BG profiles are flatter but that the extremes are much more pronounced. So if it’s a good day, his BG looks much flatter than on openAPS. But openAPS does do a good job of moderating how high the spikes are, or how steep the lows are.
- You should be able to simply turn the algorithm off at times without having to manually unplug your rig.
That said, it may be that we’ve just never optimized his settings quite as much as we should have; but that’s the point, to me; there are too many settings to ever optimize unless your life is painfully routine. So in the end it’s not truly transparent what the algorithm is doing.
Does that make sense?
6 Likes
Great, thank you very much for the comments! There are some significant differences between Loop algorithm (which I am very familiar with), and OpenAPS oref0/oref1 algorithm (which I understand to some extent, but not in fine details). Some of the points you are making are more OpenAPS/oref0 specific, others also apply to Loop. I’ll need some more time to digest your excellent insights - thanks again.
1 Like
Finally got some time to read through your comments in more detail, so here we go:
1.1. One should not really mess with DIA too much. Unlike ISF, CF or basal needs, which may vary by an order of magnitude from one person to another, insulin absorption is a property of insulin and tissue, which varies some but not that much. Both OpenAPS and Loop now have much better models using parametrized exponential insulin absorption curves with preset peak action time and duration. We may be able to fine tune these in the future. In any case, IOB modeling errors are probably relatively small compared to other uncertainties we are dealing with.
1.2 True. If you have any pointers to how we could model ISF(bg) nonlinearity, let me know. Even if ISF were just a contact across bg range, I think it is fair to say ISF is a significant uncertainty. On the other hand, from a control system point of view, (1/ISF) is just a gain in the system, which does affect speed of response and stability, but we can still do reasonably within some limits.
I can see how autosense can be behind the curve, and I do not think we have any really good alternatives at this time. Regarding discontinuities, maybe you could bring this up as an OpenAPS issue, I am sure Scott would look into this.
- Pump has its own ISF, CR, basal rates and DIA, and these do not need to be the same as the parameters used in the algorithm, so I am not sure what you mean.
- OpenAPS and Loop treat carb absorption differently. I am more familiar with Loop approach, which is fairly straightforward.
- Loop has exactly such “suspend threshold”. I thought OpenAPS had something similar, but I am not sure.
- I do not entirely agree here. At some point in the future, body will need a certain level of insulin action. which includes what is commonly referred to as basal need. Predicting basal needs in the future is very difficult, so in OpenAPS/Loop we assume that pre-programmed basal rates are meeting that future need. BTW, whether the calculations are done as +/- around that level or as absolute levels is irrelevant - I’d not be concerned about that. A system such as 670G is probably using an integral action in the control loop to determine the effective basal rate. This is not automatically better than Loop or OpenAPS, since it is based on past behavior, which may not be a good predictor for the future. Even if the system “learns” effective basal needs over some number of days, and uses that as a future assumption (instead of preprogrammed basal rates), there are still no guarantees that future needs will remain the same. This is a conceptually difficult problem.
- I am less familiar with OpenAPS oref0 or oref1 details, so I am unable to say much here. In comparison, Loop prediction algorithm is relatively simple and clear.
- I like Loop’s dynamic carb absorption because it allows user to enter different absorption times as a starting point, instead of letting the algorithm figure those out in some ways. On the other hand, some people prefer OpenAPS because it requires fewer user inputs. I have no idea how 670G deals with carb absorption. In the end, there will never be a perfect model for carb absorption - it just varies so much.
- That’s trivial with Loop, just slide a button and you go open loop. However, Loop does not have any remote control capabilities - does not read anything from NS or anywhere else, which is all fine as long as one can take care of own needs, but is a problem for people with little kids.
Work in progress. Again, thanks for your comments!
1 Like
1.1. One should not really mess with DIA too much. Unlike ISF, CF or basal needs, which may vary by an order of magnitude from one person to another, insulin absorption is a property of insulin and tissue, which varies some but not that much. Both OpenAPS and Loop now have much better models using parametrized exponential insulin absorption curves with preset peak action time and duration.
So, I have not done updates to the openAPS algorithm in quite some time so it’s possible we’re using an older version, but what we were using is a linear DIA curve. The reason we haven’t updated is because I’m not sure I like some of the other updates…
Pump has its own ISF, CR, basal rates and DIA, and these do not need to be the same as the parameters used in the algorithm, so I am not sure what you mean.
But these are always used as the baseline – again maybe we’re using an older version of openAPS, but you can’t have, say, an ISF of 100 programmed for manual mode but an ISF of 50 *programmed as the baseline for the algorithm. In other words, I basically think that the numbers that make the pump work in manual mode might be completely different than the ones needed in auto mode, and so you shouldn’t use the former as the baseline at all.
Also, so far as I know openAPS doesn’t have a suspend threshold – anytime our son starts rising but has negative IOB, insulin delivery turns back on even though he hasn’t recovered yet.
I do not entirely agree here. At some point in the future, body will need a certain level of insulin action. which includes what is commonly referred to as basal need.
I guess what I’m saying is that I believe another conceptual framework, which completely discards the idea of basal insulin need but instead relies on the notion of sensitivity would probably be more reactive and adaptive to long-term changes in underlying insulin usage by the body. This, of course, is a hypothesis. But my personal opinion, having seen how my son’s insulin usage changes with food, is that you cannot truly disentangle insulin need due to carbohydrates from alleged “basal” insulin need – what you eat now will influence your basal rate later. So it seems better to just somehow roll those into one variable.
*What I would love to see is some kind of bolus library… where you can essentially tag different foods, deliver a bolus (your best guess of what you need) and then record the outcome over, say, 5 hours. Then the algorithm could take everything with, say #ihopwaffle tagged, look at how much insulin was delivered, what the BG was, and make a prediction for what the ideal bolus should have been in order to keep BG constant for the average of all those #ihopwaffle events. That could then be the recommended bolus regime for the next time you eat that, and it could be the jumping off point from which the algorithm makes changes. Perhaps that wouldn’t work well but I know that at the least it would be a convenient way to remember what you bolused last time for a food and whether it worked well…
Thanks, are you working on modifications or you’re just looking into this from a purely theoretical point of view.
1 Like
It’s been a while since I used OpenAPS, but I am pretty sure you can be have entirely different ISF in the algorithm versus the pump. I am 100% sure about that for Loop.
Possibly, I do not know. I am skeptical about discarding the idea of basal insulin need entirely.
Loop meal entry interface actually looks like that, but it does not have a memory function to record the absorption and offer that as a menu item in the future. That would be cool to add and I think Pete (the main Loop developer) might have something along these lines in mind for future development.
I am now running an experimental version Loop with integral action, which should in a limited way move us away from the pre-programmed basal rates concept. I also have a Loop simulator in the works, which should allow us to test out various other algorithm options. Actual implementation is a problem - we have too few people with necessary programming skills, and I am not one of them.
2 Likes
It’s been a while since I used OpenAPS, but I am pretty sure you can be have entirely different ISF in the algorithm versus the pump. I am 100% sure about that for Loop.
I’d be curious to see if that’s the case. When we set it up, there was “autosense” which allows ISF to float…but you initially base that number on the programmed ISF in the pump and then the feedback it gets from BG is what enables it to change. And it doesn’t change for 6 hours I believe?
1 Like
In the OpenAPS settings folder, there should be insulin-sensitivities.json file or such, which is initially populated from the pump. I think you could manually edit that file and enter whatever ISF schedule you like. But, I’m no longer using OpenAPS, so I’m not sure, better ask at intend-to-bolus gitter of looped fb group.
1 Like
my whole life is a fantasy land 