Implement paperclip or other way to upload to Google Cloud Storage

[icarito]

In order to migrate to Google Cloud Storage, we've had to mount our storage bucket with gcsfuse. This solution seems not stable enough as it sometimes locks the mountpoint (#9758 is an instance of this). This happened recently and we didn't notice until today, and it may have caused more problems in the past week.

Please describe the desired behavior.
Ideally, we would not have to mount this volume, but the app would upload the data directly to the Google Cloud Storage API.

[anirudhprabhakaran3]

What exactly are we storing on Google Cloud? Images and attachments? I don't understand the part about mounting the volume...

[jywarren]

I think gcsfuse is a library which allows Google Cloud Storage to act as a linux-native disk mount, and fuse in general is a class of disk-mount interfaces -- for example, i think someone for fun made a gmail-fuse library which is able to use your gmail inbox appear as a fuse-based disk on linux devices. Obviously not that efficient 😄

Here, i guess it was just too much of a beta, or something -- so agreed, let's look at how Paperclip and Google Cloud's storage options can interact!

[anirudhprabhakaran3]

It seems as though Paperclip has been depreciated. The official Paperclip GitHub repo advises us to use ActiveStorage.

The RoR docs give instructions to use ActiveStorage along with Google Cloud storage, along with lots of other services, like AWS and Azure. They can be found here: https://guides.rubyonrails.org/active_storage_overview.html#google-cloud-storage-service

We'll have to see how much time and effort it takes to use ActiveStorage to use Google Cloud

[jywarren]

Thanks @anirudhprabhakaran3 -- we had tried switching to ActiveStorage a while back but it didn't yet support very many backends. Looking at it again now, but... my preference would be to try using Paperclip for a relatively quick fix on this, then scheduling a bigger switch to ActiveStorage later. But let's see what's possible. Thanks!

[jywarren]

Google storage is supported through Fog Storage: https://github.com/thoughtbot/paperclip#storage

[jywarren]

And we're on Paperclip v6.1.0: #3782

[jywarren]

We'll probably change the storage here: https://github.com/publiclab/plots2/blob/main/config/initializers/paperclip.rb#L1-L10

[jywarren]

OK - @icarito i started the changeover here: #9807

My questions/thoughts:

• what about moving the existing files over
• changes in path config for cloud instead of file storage?
• secrets/keys storage, i chose ENV["GOOGLE_STORAGE_KEY"] and ENV["GOOGLE_STORAGE_SECRET"] but open to whatever
• Can you set up unstable with the keys following the above naming so we can try this out?
• what about testing in gitpod, what if we did this /only/ in production or something? otherwise relying on local storage?
• Do we condition the switch on what mode (production/development) or using an env variable?
• anything else you can think of before this ought to start working?

Thank you!!