Running out of file descriptors

[harendra-kumar]

Is it possible to avoid using a pipe for communicating between C to Haskell? A pipe uses 2 fds. It will be much more scalable if we can do away with those fds.

Haskell stack runs out of fds on OSX while using filewatch via fsnotify. Please see commercialhaskell/stack#1979 .

[luite]

hmm, perhaps I can change it to use a single pipe for all watches. if many watches are created, having an thread for each of them would probably also be less than ideal.

[harendra-kumar]

Haskell threads should be relatively cheap but pthreads on the C side might not scale well. Can we make watchCallback a Haskell function and use the C wrapper of the Haskell function in the C code? GHC supports that. That way we can avoid the pipes as well as pthreads, making the library scalable irrespective of how the application designers use it.