why not add an (optional) sync before exiting?

[RJVB]

I was discussing EMD with some data reliability experts, found we agreed it would be a good idea to do a single sync() before exiting and we wondered why this wasn't yet implemented.

Of course sync() will flush all filesystems making it potentially more expensive than necessary but most of the time it shouldn't cost a lot while making EMD a lot safer in theory. Evidently one could make such a final sync optional, or allow to deactivate it (via an env. variable for instance).

A library destructor function (attribute(destructor)) appears to be unreliable for this, but exit() can be overloaded exactly like the other functions.

I can make a PR if there's interest for this.

[laurynas-biveinis]

Since one of the use cases for libeatmydata is speeding up database testing, I wonder how much extra syncing this would result with PostgreSQL process-per-connection design

[RJVB]

I have no idea, but probably one sync per connection. You'd have to test how costly that is I've forked and branched the code and done an implementation of my idea, plus some unrelated changes to get the code to build and run properly on Mac (OS X 10.9). One addition I made to my initial idea: sync only when anything has actually been "swallowed", i.e. keep a counter that tracks the number of times a sync operation or mode has been avoided. I will do a PR later today or tomorrow. R.

[dermoth]

Why not run sync after the application has finished? It could also be implemented as a wrapper script, but please don't make this the default. I've seen systems take hours to complete a single sync() because of the sheer amount of dirty data and the nonstop writes - surely if I used libeatmydata for an app on them that would be to avoid impact to the block devices and I would never want to flush GB's of dirty buffers at the end!

[dermoth]

Another option that would be quite nice, allowing to disable libeatmydata using env. For fun I added libeatmydata.so /etc/ld.so.preload (including on initrd) on a very slow test system so the entire system runs libeatmydata. but I would like to be able to tell it when to sync the fs anyway (ex before a reboot in case it would spend too much time syncing and timeout before everything is flushed).