You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently file attribute retrieval in the adapter interface consists of several separate functions:
fileSize()
lastModified()
mimeType()
visibility()
All of these functions return a object of type FileAttributes.
There are several downsides to this approach:
When you have a FileAttributes object you can not trust it unless you know via which function it was created (ie: is the lastModified property null because we obtained the object via mimeType() or is it null because it is not set in the underlying storage.
The filesystem cannot assume that the object is fully hydrated, so calling fileSize() and lastModified() may lead to duplicate API calls to the underlying adapter.
In practice most adapters get all meta data in one request:
AsyncAwsS3Adapter
AwsS3V3Adapter
AzureBlobStorageAdapter
GoogleCloudStorageAdapter
SftpAdapter
Some don't:
FtpAdapter,
InMemoryFilesystemAdapter, note: this should not affect performance
Local, note: for listContents it is fully hydrated so performance wise this should be fine.
ZipArchiveAdapter
WebDAVAdapter, note: the underlying propFind method does support requesting multiple properties in one call
Possible solution
Change the contract so that users may always assume that a StorageAttributes object is fully hydrated
Create lazy loading implementations of StorageAttributes that load unset attributes if needed:
Fully hydrate the object when the underlying API supports this efficiently
Fully hydrate the object when the filesystem is local (in the Local adapter we fully hydrate it for listContents but not for a single file)
Have a standard lazy implementation that passes simple closures for lazy loading attributes: new LazyFileAttributes($path, () => $this->fileSize($path), ...)
Add a function metaData() to the adapter interface.
Change FileSystem to use this new function to get the metadata and use it when specific properties are requested, since we know now the meta data is fully hydrated (lazily or greedily) it can reuse this metadata for other properties.
Add a metaData() to the FilesystemReader that returns the StorageAttributes object for the path.
Q
A
Flysystem Version
v4
Adapter Name
all
Adapter version
n/a
Scenario / Use-case
This feature would reduce API calls when they matter most, since the remote APIs for cloud storage providers all support retrieving meta data in 1 call.
Summary
Add a getMetaData(): StorageAttributes
Rework internals to be more efficient with API calls if possible
Make it explicit that a StorageAttributes method result of null means not set in the underlying storage instead of not set in the underlying storage, or not loaded by whatever method you used to create this object
The text was updated successfully, but these errors were encountered:
Yes, I second this!
Method getMetaData(): StorageAttributes to filesystem interface would be great. I wanted to implement custom metadata for my adapter and current design of interface does not support it.
Entity class FileAttributes has method extraMetadata(): array which seems similar idea was there, but it can not be used, because filesystem object can never return this full object via any of current interface methods.
Feature Request
Currently file attribute retrieval in the adapter interface consists of several separate functions:
fileSize()
lastModified()
mimeType()
visibility()
All of these functions return a object of type
FileAttributes
.There are several downsides to this approach:
FileAttributes
object you can not trust it unless you know via which function it was created (ie: is thelastModified
propertynull
because we obtained the object viamimeType()
or is itnull
because it is not set in the underlying storage.fileSize()
andlastModified()
may lead to duplicate API calls to the underlying adapter.In practice most adapters get all meta data in one request:
AsyncAwsS3Adapter
AwsS3V3Adapter
AzureBlobStorageAdapter
GoogleCloudStorageAdapter
SftpAdapter
Some don't:
FtpAdapter
,InMemoryFilesystemAdapter
, note: this should not affect performanceLocal
, note: forlistContents
it is fully hydrated so performance wise this should be fine.ZipArchiveAdapter
WebDAVAdapter
, note: the underlyingpropFind
method does support requesting multiple properties in one callPossible solution
StorageAttributes
object is fully hydratedStorageAttributes
that load unset attributes if needed:This way we can:
Local
adapter we fully hydrate it forlistContents
but not for a single file)new LazyFileAttributes($path, () => $this->fileSize($path)
, ...)metaData()
to the adapter interface.FileSystem
to use this new function to get the metadata and use it when specific properties are requested, since we know now the meta data is fully hydrated (lazily or greedily) it can reuse this metadata for other properties.metaData()
to theFilesystemReader
that returns theStorageAttributes
object for the path.Scenario / Use-case
This feature would reduce API calls when they matter most, since the remote APIs for cloud storage providers all support retrieving meta data in 1 call.
Summary
getMetaData(): StorageAttributes
StorageAttributes
method result ofnull
means not set in the underlying storage instead of not set in the underlying storage, or not loaded by whatever method you used to create this objectThe text was updated successfully, but these errors were encountered: