Skip to content

Commit

Permalink
Lazy-load client libraries.
Browse files Browse the repository at this point in the history
Currently when loading the gem we also implicitly load the client
libraries for the bucket providers. These are quite large, and increase
the time to load the gem (and the number of files included) by orders of
magnitude.

To avoid unnecessary slowing down the bootstrap, we can instead load
these gems only if the provider is used, and memoise this on the class
to avoid repeated calls to `require` (as this returns `true` when the
library is first required).

In testing with Rails, lazy-loading these libraries reduces the time
taken for Bundler to require the gem during bootstrap from 0.25 to 0.005
seconds (yes, that's two orders of magnitude faster).
  • Loading branch information
benk-gc committed Apr 21, 2024
1 parent 643dea4 commit 1e7f761
Show file tree
Hide file tree
Showing 3 changed files with 15 additions and 5 deletions.
10 changes: 7 additions & 3 deletions lib/bucket_store/gcs.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,21 @@
require "stringio"
require "uri"

require "google/cloud/storage"

module BucketStore
class Gcs
DEFAULT_TIMEOUT_SECONDS = 30

def self.load_client_library
@load_client_library ||= require "google/cloud/storage"
end

def self.build(timeout_seconds = DEFAULT_TIMEOUT_SECONDS)
Gcs.new(timeout_seconds)
new(timeout_seconds)
end

def initialize(timeout_seconds)
self.class.load_client_library

# Ruby's GCS library does not natively support setting up a simulator, but it allows
# for a specific endpoint to be passed down which has the same effect. The simulator
# needs to be special cased as in that case we want to bypass authentication,
Expand Down
9 changes: 7 additions & 2 deletions lib/bucket_store/s3.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,23 @@

require "uri"

require "aws-sdk-s3"

module BucketStore
class S3
DEFAULT_TIMEOUT_SECONDS = 30

def self.load_client_library
@load_client_library ||= require "aws-sdk-s3"
end

def self.build(open_timeout_seconds = DEFAULT_TIMEOUT_SECONDS,
read_timeout_seconds = DEFAULT_TIMEOUT_SECONDS)
S3.new(open_timeout_seconds, read_timeout_seconds)
new(open_timeout_seconds, read_timeout_seconds)
end

def initialize(open_timeout_seconds, read_timeout_seconds)
self.class.load_client_library

@storage = Aws::S3::Client.new(
http_open_timeout: open_timeout_seconds,
http_read_timeout: read_timeout_seconds,
Expand Down
1 change: 1 addition & 0 deletions spec/spec_helper.rb
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# frozen_string_literal: true

require "logger"
require "bucket_store"

RSpec.configure do |config|
Expand Down

0 comments on commit 1e7f761

Please sign in to comment.