4.5.1: Worse memory management?

[reyaz006]

qBittorrent & operating system versions

qBittorrent: 4.5.1 x64
Operating system: Windows 8.1 6.3.9600 x64
Qt: 5.15.8
libtorrent-rasterbar: 1.2.18+gitd22612ca1b

What is the problem?

Upgraded from 4.4.5.

With 4.4.5:

• Run qbt.
• Wait ~25 minutes for GUI to start being reponsible.
• qbt consumes about 3.5 GB RAM.

With 4.5.1:

• Run qbt.
• qbt first writes a file %TEMP%\etilqs_* roughly equal to torrents.db in size.
• qbt then writes a second file %TEMP%p\etilqs_*, also roughly equal to torrents.db in size.
• Wait ~20 minutes for GUI to appear.
• qbt consumes about 5 GB RAM.

I specifically ignored 4.5.0 because there were some qt5 and performance related fixes for large torrents collections in 4.5.1.
Is this expected? Am I missing something? What's the point of huge startup files in %TEMP%?

Steps to reproduce

No response

Additional context

No response

Log(s) & preferences file(s)

No response

[reyaz006]

Restarted qbt once. Now it also consumes about 3.5 GB RAM.

[glassez]

What's the point of huge startup files in %TEMP%?

Apparently you're the first one to find this.

[glassez]

• qbt then writes a second file %TEMP%p\etilqs_*, also roughly equal to torrents.db in size.

Anyway, it doesn't look like it's something qBittorrent does directly.

[reyaz006]

What's the point of huge startup files in %TEMP%?

Apparently you're the first one to find this.

Do you mean it was always like that and I just didn't notice before?

• qbt then writes a second file %TEMP%p\etilqs_*, also roughly equal to torrents.db in size.

Anyway, it doesn't look like it's something qBittorrent does directly.

Could there be something to do about this?

[glassez]

Do you mean it was always like that and I just didn't notice before?

Perhaps so. Or maybe it appeared recently and is related to some changes in the qBittorrent code. You're just the first one who thought it was a problem and reported it.
In fact, there are no problems. These are just temporary files created by SQLite:
https://www.sqlite.org/tempfiles.html
Since SQLite creates them, it means they are required to perform some operations.

[glassez]

Perhaps it is related with a31755b. Then most likely it was a one-time phenomenon associated with the conversion of the database format.

[reyaz006]

Nope, this is my second restart of qbt at v.4.5.1 and it still creates those temp files, ~3.2 GB in size total (then automatically removes them before torrents start loading). torrents.db is ~1.87 GB.

[xavier2k6]

Firefox?

[sledgehammer999]

torrents.db is ~1.87 GB.

Kinda offtopic. Isn't this a ridiculously big file for saving torrent list/info? Even if you have thousands of torrents.
Disclaimer: I've never had above 100 torrents in my session, so I can't really know what the size of thousands should be.

[reyaz006]

Firefox?

?

torrents.db is ~1.87 GB.

Kinda offtopic. Isn't this a ridiculously big file for saving torrent list/info? Even if you have thousands of torrents.

You can check other issues here by me, most of them are related to how qbt handles huge amount of torrents.

Disclaimer: I've never had above 100 torrents in my session,

What do you mean by session? I think I'll be fine if I only downloaded/seeded 100 torrents during one session. After they are finished, qbt would not process them (switch to a new session). But no, there are only paused torrents for qbt and if there are 70000 of them then qbt thinks it needs to keep them all in memory. It would help to teach qbt what archived torrents are maybe. #11878

By the way, when it's possible to get things related to certain software go "ridiculously big", wouldn't it be a better choice to set some hard limit? This way, we could stop all those "my browser eats all my memory when I open more than 10 tabs"/"you shouldn't need more than 10 tabs in your browser" wars. You know, stop the resource hogging. Or develop better methods to handle information in order to raise that limit.

EDIT:
Also, I'm the guy whose SSD was killed by qbt. #9152

[glassez]

Isn't this a ridiculously big file for saving torrent list/info? Even if you have thousands of torrents.

SQLite database file fragments over time and begins to contain unused data areas, so it needs optimization. I have a task on my to-do list to implement periodic optimization. Or we can configure it to do this all the time, but I'm afraid it will negatively affect performance.

[alfakr0ll]

SQLite database file fragments over time and begins to contain unused data areas, so it needs optimization. I have a task on my to-do list to implement periodic optimization. Or we can configure it to do this all the time, but I'm afraid it will negatively affect performance.

Cant it be run at shutdown or start up (edit: of qBittorrent)? Or will that increase exit/startup times a lot?

[githubbapoopa]

Isn't this a ridiculously big file for saving torrent list/info? Even if you have thousands of torrents.

SQLite database file fragments over time and begins to contain unused data areas, so it needs optimization. I have a task on my to-do list to implement periodic optimization. Or we can configure it to do this all the time, but I'm afraid it will negatively affect performance.

Something similar to the former speedyfox

https://crystalidea.com/de/speedyfox

or the more modern Arctic Profile Optimizer

https://github.com/arctic-sun/apo

?

Would be nice indeed.

[nokti]

or the more modern Arctic Profile Optimizer

https://github.com/arctic-sun/apo

I tried APO on qBit's sqlite database and it seems safe. After running it, I restarted qBit and everything is seeding as normal (4K+ torrents). qBit also restarted super fast, only a couple of seconds!

[githubbapoopa]

Thanks for taking the effort.

This however indicates that the SQLite database indeed needs some love from qbittorrent.

[nokti]

This however indicates that the SQLite database indeed needs some love from qbittorrent.

I restarted my compter later in the day and qBit's starting time was back to around a minute. So no changes. But the sqlite database is now 10% smaller. I guess I just gained a "cleaner" database.

[githubbapoopa]

That's a "fragmentation over time" problem so will always be a problem unless adressed. 10% improvement on the first run is huge.

[glassez]

#18612 is intended to address the "database fragmentation" problem.

[reyaz006]

Any idea if this should also fix the huge temporary files on every qbt start?

[reyaz006]

Noticed that RAM usage actually goes lower with time, e.g. to ~1.1 GB. But if I check with Process Explorer it shows ~7.7 GB in Private Bytes.

[glassez]

Any idea if this should also fix the huge temporary files on every qbt start?

The fact that it is now creating such files is most likely a sign of some kind of flaw. I'm working on it.
But the optimization done in #18612 also requires the creation of a temporary file for its execution.

[glassez]

Any idea if this should also fix the huge temporary files on every qbt start?

The fact that it is now creating such files is most likely a sign of some kind of flaw. I'm working on it.

#18623 should probably address this issue.

[reyaz006]

Any idea if this should also fix the huge temporary files on every qbt start?

The fact that it is now creating such files is most likely a sign of some kind of flaw. I'm working on it.

#18623 should probably address this issue.

I'm guessing this is not included in 4.5.2, right?

[glassez]

I'm guessing this is not included in 4.5.2, right?

Right.