Database cleanup

[psonnera]

After some time of use, my son's xDrip+ database went above 3.4GB and I decided it was time to cleanup. I thought Less common settings -> Glucose retention would do that for me and set it to 90 days. The day after, no change. I then thought Save database daily would trigger the feature above and enabled it. Next day still 3.4GB. Out of desperation I saved all settings and deleted xDrip+ data at Android apps level, then recovered settings etc.
Where did I misunderstood the glucose retention feature? What was the correct sequence?
Thanks!

[Navid200]

I am seeing something in the code that could be an issue.
Although, I cannot read the code and understand all the details because of sometimes the method names may not perfectly reflect what they do.

I am performing a test right now and need to wait 24 hours before it is applied.
If this turns out to really be an issue, I will ask you to please convert this to an issue.

[Navid200]

Thanks for reporting this.
I don't have any test results.

However, I am looking at my logs and this is what I see showing all the daily job runs.

If you focus on the dates, you can see that it actually runs every other than and not everyday.
So, I suspect that if you had waited one more day, it may have deleted the old readings.

Everything that needs to run once a day is something that can run every other day instead. I don't believe we have anything that must either run once a day or else!

Regardless, I am going to spend some more time on this to make sure this indeed is not an issue that needs to be addressed.

Thanks for reporting

[Navid200]

@jamorham Even though the title "DailyIntentService" suggests something that runs daily, it seems that in reality it runs once every 48 hours. Or am I wrong in making that conclusion from my logs?

Do we have any task in the list in DailyIntentService that must run once a day?
Should I work on this to change it so that it really runs everyday?
Or, are we OK with the jobs running once every 48 hours?

Thanks

[parapenT1sta]

After some time of use, my son's xDrip+ database went above 3.4GB and I decided it was time to cleanup. I thought Less common settings -> Glucose retention would do that for me and set it to 90 days. The day after, no change. I then thought Save database daily would trigger the feature above and enabled it. Next day still 3.4GB. Out of desperation I saved all settings and deleted xDrip+ data at Android apps level, then recovered settings etc. Where did I misunderstood the glucose retention feature? What was the correct sequence? Thanks!

I had the same issue and the only solution was in fact to delete all data and re-setup xDrip again.

Was a good idea to have a button like AndroidAPS does to cleanup database. I think is a good approach.

[Navid200]

It took a few days until the daily service ran. The database was cleaned and the daily database save was created.

My conclusion is that the daily service does not run everyday.

Either we need to change that and ensure that it runs everyday. Or, we need to document this clearly so that the users don't think that there is a malfunction.

This screenshot shows the dates the daily service ran on my phone:

It ran on December 12 and 13. Did not run on 14 or 15. It ran again on December 16.
It did not run on 17. And it ran today, 18.

I will be looking at the code to see if I can figure out what causes the inconsistency and if it is a simple change, I will open a PR to fix it.

Thanks

[Navid200]

@psonnera Would you please click on Create issue from discussion in the right pane so that an issue is created?

[PhilTU]

what would be the impact if the DailyIntentService never runs ?

[Navid200]

That's like the heartbeat of xDrip. All maintenance, all cleanups, run in there. If it is not running, I would suspect a corrupt database.

[PhilTU]

That's like the heartbeat of xDrip. All maintenance, all cleanups, run in there. If it is not running, I would suspect a corrupt database.

ok, that was the lady with the child and 69 GB database. On my test phone it last run 2 weeks ago, I'm guessing because of the charger thing - is it scheduled to run at night when on charge ? (I noticed you were looking to change the "on charge" part)

[Navid200]

It will only run if the phone is idle and is being charged and is not on metered internet.
I already have opened a PR: #3823
That PR will change the requirements to this:
It will only run if the phone is idle and the battery is not very low and internet is not metered.

Why it runs at midnight is not something I can explain from either experience or knowledge. But, I can see that it runs as close as possible to midnight.

[Navid200]

I found an old post in the facebook group with a screenshot of the daily backups.
It shows that it runs daily:

It does not any longer. So, something has clearly changed since then. I am running tests to see if I can find out when the change ocurred. Unfortunately, tests take a long time. So, it may take me a while to find out exactly when the change ocurred.

[jonprogers]

@Navid200 I'm running version aee9315-2024.09.19, subscribed to the nightly channel although am manually updating to the latest nightly version every so often.

I run with G6 on a Pixel7a although I have yet to upgrade from Android 14 to 15.

Having just looked at my logs I can see that the DailyIntent service has successfully run each day.

If it would help your testing I'm happy to update to the latest nightly to see if anything changes - let me know...

[Navid200]

@jonprogers
This is very interesting.
No, this is not related to the latest Nightly. I am not 100% sure what is causing it. Thanks a lot for sharing. Because it makes me wonder something else ...

Please let me share something else:

That shows my Google Drive automatic backups. It is also triggered by the daily intent.
You can see it does not run everyday and you can see it goes back several releases.

I don't charge my phone every night. I suspect you charge your phone every night and it is being charged at midnight.
This sounds like a private question. But, do you mind approving or denying that?
Because I suspect that could be the cause.

There is a conditional in the code that states that if the schedule is due, the condition will be overlooked and it will run at the latest in 24 hours. But, I don't think that is a valid statement.
It looks to me the job waits until the phone is being charged.

[jonprogers]

No problem, yes the phone would have been on charge every time it ran in that sample. I'll try to remember not to plug it in tonight and see what happens...

[jonprogers]

@Navid200 You appeared very clear that you understood what is going on so this update is possibly overkill, although not charging my phone over the midnight period did, indeed, result in the DailyIntent service not running.

[Navid200]

Thanks for confirming.
I am testing my PR and so far that also confirms my understanding.

[Navid200]

If my suspicion is correct and the daily intent runs when the phone is being charged eventually, I don't think we have an issue.
Whether the database is cleaned once a day or even once a month, everything will be fine.

Of course we will always have one person who waits for their phone storage to be 99.999% full before they decide to post in a forum. No matter what we do, we can never be prepared for something like that. Even if the database is cleared once a day if you wait long enough, it will not work.

[Navid200]

I am opening a PR to change the requirement for the phone to be charging to a requirement for the battery not to be too low.
I will test and if it works, I will submit. Let's see what JamOrHam thinks.

[Navid200]

I can see that the daily intent service runs everyday after my PR on my test phone. But, I don't know why it does not create a daily database backup even though I have enabled it for the sake of the test.

[Navid200]

I think I just discovered another bug.
I attempted to create a database backup and xDrip asked for permission to write in storage. But, when I enabled daily save it never asked for that permission. If it saves the daily backup tonight, it means we have one more bug to fix.

[Navid200]

After saving database and giving permission, the next time daily intent ran, the daily database save succeeded. This is weird because there should be a log if the daily database save fails and there wasn't.
Anyway, this is something we can look at as a separate possible issue.
Hopefully, the PR will be merged the daily intent will then run daily whether you charge your phone everyday or not.