[BUG] High Memory Usage Causing WeeWX to Crash When Generating Statistics Page

[skinner12]

Describe the bug
Hello,

I have been using WeeWX with a custom skin, and everything was working fine. Recently, I upgraded to version 5.0.2 and updated my database from the original version 3 started in 2018 to the extended database format. I also recalculated all values.

I configured WeeWX to update weekly and yearly reports every hour. However, I am experiencing a significant issue when generating the statistics page. I have allocated 8GB of RAM to the VM running Ubuntu with python 3.12, but within a few minutes of starting to produce the statistics page, WeeWX consumes all available RAM, causing the system to terminate the application.

If I comment out the statistics section in the skin.conf file and do not generate the statistics page, everything works fine, and there are no memory issues.

Could you please help me understand what might be causing this problem? Are there any settings or configurations that I can adjust to prevent WeeWX from consuming so much memory?

Thank you for your assistance.

To Reproduce
Steps to reproduce the behavior:
Build statistic page

[Daveiano]

Hello there,

there are currently some problems with the statistics page, see #267 and #278, so this could be a bug/problem in the skin or weewx.

How many years of data do you have? I have data since 2021 and weewx is running with 512MB of RAM without any problems. So I guess this is also related to the amount of data.

Do you have the seasons default skin running also?

Would it be possible for you to share your weewx database with me to analyze the problem?

[Daveiano]

Please also see my comment in #267 (comment)

[skinner12]

Hello there,

there are currently some problems with the statistics page, see #267 and #278, so this could be a bug/problem in the skin or weewx.

How many years of data do you have? I have data since 2021 and weewx is running with 512MB of RAM without any problems. So I guess this is also related to the amount of data.

Do you have the seasons default skin running also?

Would it be possible for you to share your weewx database with me to analyze the problem?

Hello,
sorry for late response.

I read your suggestions but I continue got errors. OS kill weewx because it eats too much resources. I try with 8Gb of RAM and it competed statistic page, so I guess it should be an error with too many data.

I run all default, I didn't change anything relevant.

I also added column:

weectl database add-column visibility
weectl database add-column cloud_cover
weectl database add-column aqi

and then run weectl database calc-missing
but got same error.

So I tried with Belchertown and it created all files within seconds.

I uploaded my archive table to drive just in case you wanna test somethings.

Thanks

[Daveiano]

Ok thank you very much for the input and feedback!

I have downloaded your archive and will have a look at it at the weekend, perhaps some tests on my local machine will reveal the bottleneck.