Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try to download all files at once #268

Open
m-mohr opened this issue Jul 22, 2022 · 12 comments
Open

Try to download all files at once #268

m-mohr opened this issue Jul 22, 2022 · 12 comments
Labels
enhancement New feature or request Platform
Milestone

Comments

@m-mohr
Copy link
Member

m-mohr commented Jul 22, 2022

My suggestion in regard of user convenience would be the possibility to download the whole layerstack by integrating this opportunity into the action buttons in the web editor. As for now, this opens a window where it is possible to download one layer after another.

Try to add a download button to the modal where you can download all files at once. Related to #265 (don't download metadata by default)

@m-mohr m-mohr added the enhancement New feature or request label Jul 22, 2022
@m-mohr m-mohr added this to the v0.12 milestone Jul 22, 2022
@m-mohr m-mohr self-assigned this Jul 22, 2022
@m-mohr m-mohr modified the milestones: v0.12, v1.0 Jul 28, 2022
@m-mohr m-mohr removed their assignment Aug 4, 2022
@m-mohr
Copy link
Member Author

m-mohr commented Jan 26, 2023

Closing as this is restricted by the browsers (either memory or number of parallel downloads) and would basically need a server-side solution which would need to zip the files and offer them for download.

@m-mohr m-mohr closed this as not planned Won't fix, can't repro, duplicate, stale Jan 26, 2023
@m-mohr
Copy link
Member Author

m-mohr commented Oct 4, 2023

There seems to be a Chrome only API: https://github.com/WICG/file-system-access

Anyway, no solution seems to work across all browsers in a reliable way.

@Patrick1G
Copy link

@m-mohr some option for downloading potentially hundreds of files from the editor would be valuable, e.g. here: eodcgmbh/sen2like#3 (comment)

if the browser does not support this, could the save_results process be adapted or a new process be defined e.g. save_custom_results()... For example having the ability to stack a time series by spectral bands would be super valuable..
Or even simply adding all result files into a single zip file would be valuable..

@m-mohr
Copy link
Member Author

m-mohr commented Oct 9, 2023

@patrick-griffiths I understand and agree that this would be valuable and I would be very happy to implement this if there would be a technical solution available in the Browser. Unfortunately, it looks like there is non (yet). We also discussed this recently in the openEO meeting with a wide variety of developers and we agreed that there is no good solution due to Browser limitations. It needs a server side implementation if we want to provide a user-friendly solution, even for adding them to a ZIP file. So this should be discussed as part of the overall architecture. Maybe the aggregator can do this as an added functionality in openEO Platform based on normal content-negotiation mechanisms (happy to clarify this in the API).

@Patrick1G
Copy link

@m-mohr - then lets propose this (server side implementation, via the aggregator) as one of the evolution enhancements we could still do within the platform contract during the team call on Thursday... Wonder if you could come up with a cost estimate for such a task...

In the meantime, I wonder if we could create a workaround in the Editor using the UDP functionality?

@m-mohr
Copy link
Member Author

m-mohr commented Oct 9, 2023

In the meantime, I wonder if we could create a workaround in the Editor using the UDP functionality?

I'm not aware of any implemented process that would allow this.

@Patrick1G
Copy link

@m-mohr - maybe along the python client solution that @ValentinaHutter had suggested:

results = job.get_results()
metadata = results.get_metadata()
results.download_files("./data/")

@m-mohr
Copy link
Member Author

m-mohr commented Oct 9, 2023

I don't understand how that should work?! I can't run Python in the Browser.

@Patrick1G
Copy link

I was wonder if we could pack the python functionality for downloading all files at once into a User Defined Process that can then be (re)used in the editor to download all files at once. But maybe the problem with the browser would still persist?

@m-mohr
Copy link
Member Author

m-mohr commented Mar 18, 2024

Some smart people seem to have found a solution that we could try an re-use: https://github.com/jimmywarting/StreamSaver.js
-> Saving multiple files as a zip

@m-mohr m-mohr reopened this Mar 18, 2024
@Patrick1G
Copy link

@m-mohr - lets put in a dedicated task for our final SAP!

@m-mohr
Copy link
Member Author

m-mohr commented Mar 22, 2024

It has limitations though, which seem to be around 500MB - 1 GB in total. So a server-side solution is still preferrable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Platform
Projects
None yet
Development

No branches or pull requests

2 participants