Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add handle HandleStreamingReader (performance improvements) #160

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jostillmanns
Copy link

This commit introduces a new type 'StreamingReadResponse' which
implements the io.Writer interface. Thus users are writing into the
response rather than updating the Data property of ReadResponse. This
allows us to preallocate the space for the whole message and therefore
reduce allocations. Additionally we may now use a sync.Pool to manage
the required space needed for the read request and further minimize
allocations.

This patch intends to not break existing implementations of the
fuse.HandleReader interface by introducing a new interface that provides
the advantages described above.

I've benchmarked this using a scenario that is similar to the
environment I need for the application I am currently developing. The
benchmark reads data blobs of different sizes from the solid-state
drive. Using this patch we can achieve a 1.60x speed up as well as
reduce memory allocations by 85%.

benchmark.txt

This commit introduces a new type 'StreamingReadResponse' which
implements the io.Writer interface. Thus users are writing into the
response rather than updating the Data property of ReadResponse. This
allows us to preallocate the space for the whole message and therefore
reduce allocations. Additionally we may now use a sync.Pool to manage
the required space needed for the read request and further minimize
allocations.

This patch intends to not break existing implementations of the
fuse.HandleReader interface by introducing a new interface that provides
the advantages described above.

I've benchmarked this using a scenario that is similar to the
environment I need for the application I am currently developing. The
benchmark reads data blobs of different sizes from the solid-state
drive. Using this patch we can achieve a 1.60x speed up as well as
reduce memory allocations by 85%.

https://github.com/bazil/fuse/files/570909/bazil_patch_bench.txt
@jostillmanns jostillmanns changed the title dd handle HandleStreamingReader Add handle HandleStreamingReader (performance improvements) Nov 4, 2016
@jostillmanns
Copy link
Author

jostillmanns commented Nov 4, 2016

Full disclosure: I am not to confident with the prefix 'Streaming' for the handle interface as well as the response type.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant