Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

extend aws #47

Open
wants to merge 33 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
3afd602
move aws tests to subdirectory
MAfarrag Dec 19, 2024
7c33c5b
end to end test for aws.s3 upload
MAfarrag Dec 21, 2024
e160fe6
end to end test for aws.s3 download
MAfarrag Dec 21, 2024
161e926
create `aws.Bucket` class
MAfarrag Dec 21, 2024
01d43fe
reformat aws tests
MAfarrag Dec 21, 2024
aa644b8
add `aws.Bucket.delete` and tests
MAfarrag Dec 22, 2024
e0d4a34
rename test_gcs_bucket.py to test_bucket.py
MAfarrag Dec 22, 2024
9e6d161
add `aws.Bucket.upload` and tests
MAfarrag Dec 22, 2024
f5ff156
give unique names to test files
MAfarrag Dec 22, 2024
dfd758e
add `aws.Bucket.download` and tests
MAfarrag Dec 22, 2024
f779f2e
add docstring to the `aws.Bucket` class
MAfarrag Dec 22, 2024
a3b2d8e
add `google_cloud.gcs.Bucket.name` property
MAfarrag Dec 22, 2024
73a70b0
add mock test for the aws.Bucket class
MAfarrag Dec 22, 2024
6bdba85
add test upload existing file
MAfarrag Dec 22, 2024
d7e07b8
the delete method raise an error if the given directory is empty
MAfarrag Dec 22, 2024
6ace5b3
the delete method raises an error if the given file does not exist
MAfarrag Dec 22, 2024
793b7fc
reformat delete tests
MAfarrag Dec 22, 2024
3ec1333
add docstring
MAfarrag Dec 22, 2024
de8b7d4
restucture test classes
MAfarrag Dec 22, 2024
beccd38
test the cases of the overwrite parameter in the `download` method
MAfarrag Dec 22, 2024
0c675ba
the `upload` method raises an error if the directory is empty
MAfarrag Dec 22, 2024
e0fda5f
add docstring to the upload method
MAfarrag Dec 22, 2024
a3a557a
the `download` method will raise an error if the download directory i…
MAfarrag Dec 22, 2024
47a7aef
add mock tests for the `upload` and `download` methods for empty dire…
MAfarrag Dec 23, 2024
ad888fe
the `upload` method raises an error if the directory is empty
MAfarrag Dec 23, 2024
bd60569
correct the test marker issues in pyproject.toml
MAfarrag Dec 23, 2024
8e1354a
update docstring
MAfarrag Dec 23, 2024
f30f0ac
create `AbstractBucket` class and use it as a superclass for the `aws…
MAfarrag Dec 23, 2024
18aca0c
add parameters to the `AbstractBucket` class and unify it for the sub…
MAfarrag Dec 23, 2024
6a14d75
refactor the `delete` method
MAfarrag Dec 23, 2024
1664d7b
add `__str__` and `__repr__` to both gcs and aws
MAfarrag Dec 23, 2024
fdcb32a
add `__str__` and `__repr__` to the abstract bucket class
MAfarrag Dec 23, 2024
c380a6d
add `aws.Bucket.rename` method and e2e tests
MAfarrag Dec 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
327 changes: 155 additions & 172 deletions poetry.lock

Large diffs are not rendered by default.

15 changes: 6 additions & 9 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -77,12 +77,6 @@ show_column_numbers = true
show_error_codes = true
show_error_context = true

[tool.pytest.ini_options]
minversion = "6.0"
addopts = "-ra -q"
testpaths = [
"tests",
]

[tool.flake8]
ignore = ["E501", "E203", "E266", "W503"]
Expand All @@ -94,11 +88,14 @@ count = true
max-complexity = 18
#select = B,C,E,F,W,T4


[tool.pytest.ini_options]
minversion = "6.0"
addopts = "-ra -q"
testpaths = [
"tests",
]
markers = [
"fast: marks tests as fast (deselect with '-m \"not fast\"')",
"e2e: marks tests as end-to-end (deselect with '-m \"e2e\"')",
"serial",
"mock: marks tests as mock (deselect with '-m \"mock\"')",
]

Expand Down
54 changes: 54 additions & 0 deletions src/unicloud/abstract_class.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
"""This module contains the abstract class for cloud storage factory."""

from abc import ABC, abstractmethod
from pathlib import Path
from typing import Union


class CloudStorageFactory(ABC):
Expand Down Expand Up @@ -30,3 +32,55 @@ def download(self, source, file_path):
- file_path: The path to save the downloaded file.
"""
pass


class AbstractBucket(ABC):
"""Abstract class for cloud storage bucket."""

@abstractmethod
def __str__(self):
"""Return the name of the bucket."""
pass

@abstractmethod
def __repr__(self):
"""Return the name of the bucket."""
pass

@abstractmethod
def upload(
self,
local_path: Union[str, Path],
bucket_path: Union[str, Path],
overwrite: bool = False,
):
"""Upload a file/directory to the bucket."""
pass

@abstractmethod
def download(
self, bucket_path: str, local_path: Union[str, Path], overwrite: bool = False
):
"""Download a file/directory from the bucket."""
pass

@abstractmethod
def delete(self, bucket_path: str):
"""Delete a file/directory from the bucket."""
pass

@abstractmethod
def list_files(self):
"""List the files/directory in the bucket."""
pass

@abstractmethod
def file_exists(self):
"""Check if a file/directory exists in the bucket."""
pass

@property
@abstractmethod
def name(self):
"""Get the name of the bucket."""
pass
Loading
Loading