Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Add docs for first release #35

Merged
merged 2 commits into from
Jul 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 33 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,13 @@ Python API implementation for Actual server.

# Installation

Install it via Pip using the repository url:
Install it via Pip:

```bash
pip install actualpy
```

If you want to have the latest git version, you can also install using the repository url:

```bash
pip install git+https://github.com/bvanelli/actualpy.git
Expand Down Expand Up @@ -72,7 +78,21 @@ with Actual(base_url="http://localhost:5006", password="mypass", file="My budget

Will produce:

![added-transaction](./docs/static/added-transaction.png)
![added-transaction](https://github.com/bvanelli/actualpy/blob/main/docs/static/added-transaction.png?raw=true)

## Generating backups

You can use actualpy to generate regular backups of your server files. Here is a script that will backup your server
file on the current folder:

```python
from actual import Actual
from datetime import datetime

with Actual(base_url="http://localhost:5006", password="mypass", file="My budget") as actual:
current_date = datetime.now().strftime("%Y%m%d-%H%M")
actual.export_data(f"actual_backup_{current_date}.zip")
```

# Experimental features

Expand All @@ -95,7 +115,7 @@ with Actual(base_url="http://localhost:5006", password="mypass", bootstrap=True)

You will then have a freshly created new budget to use:

![created-budget](./docs/static/new-budget.png)
![created-budget](https://github.com/bvanelli/actualpy/blob/main/docs/static/new-budget.png?raw=true)

If the `encryption_password` is set, the budget will additionally also be encrypted on the upload step to the server.

Expand Down Expand Up @@ -152,9 +172,8 @@ and can be encrypted with a local key, so that not even the server can read your

The Actual Server is a way of only hosting files and changes. Since re-uploading the full database on every single
change is too heavy, Actual only stores one state of the database and everything added by the user via frontend
or via the APIs are individual changes above the base stored database on the server, stored via separate endpoint.
This means that on every change, done locally, a SYNC request is sent to the server with a list of the following string
parameters:
or via the APIs are individual changes on top of the "base database" stored on the server. This means that on every
change, done locally, a SYNC request is sent to the server with a list of the following string parameters:

- `dataset`: the name of the table where the change happened.
- `row`: the row identifier for the entry that was added/update. This would be the primary key of the row (a uuid value)
Expand All @@ -163,13 +182,18 @@ parameters:
a numeric value and `0:` to denote a null value.

All individual column changes are computed on an insert, serialized with protobuf and sent to the server to be stored.
Null values and server defaults are not required to be present in the SYNC message, unless a column is changed to null.
If the file is encrypted, the protobuf content will also be encrypted, so that the server does not know what was changed.

New clients can use this individual changes to then sync their local copies and add the changes executed on other users.
Whenever a SYNC request is done, the response will also contain changes that might have been done in other browsers, so
that the user the retrieve the information and update its local copy.

But this also means that new users need to download a long list of changes, possibly making the initialization slow.
Thankfully, user is also allowed to reset the sync. This would make sure all changes are actually stored in the
Thankfully, user is also allowed to reset the sync. When doing a reset of the file via frontend, the browser is then
resetting the file completely and clearing the list of changes. This would make sure all changes are actually stored in the
database. This is done on the frontend under *Settings > Reset sync*, and causes the current file to be reset (removed
from the server) and re-uploaded again, with all changes already in place. In this case, the sync list of changes is
reset because the server already has the latest version of the database.
from the server) and re-uploaded again, with all changes already in place.

This means that, when using this library to operate changes on the database, you have to make sure that either:

Expand Down
35 changes: 28 additions & 7 deletions actual/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -179,16 +179,26 @@ def rename_budget(self, budget_name: str):
raise UnknownFileId("No current file loaded.")
self.update_user_file_name(self._file.file_id, budget_name)

def delete_budget(self):
if not self._file:
raise UnknownFileId("No current file loaded.")
self.delete_user_file(self._file.file_id)
# reset group id, as file cannot be synced anymore
self._file.group_id = None

def export_data(self, output_file: str | PathLike[str] | IO[bytes] = None) -> bytes:
"""Export your data as a zip file containing db.sqlite and metadata.json files. It can be imported into another
Actual instance by closing an open file (if any), then clicking the “Import file” button, then choosing
“Actual.” Even though encryption is enabled, the exported zip file will not have any encryption."""
if not output_file:
output_file = io.BytesIO()
with zipfile.ZipFile(output_file, "a", zipfile.ZIP_DEFLATED, False) as z:
“Actual.” Even when encryption is enabled, the exported zip file will not have any encryption."""
temp_file = io.BytesIO()
with zipfile.ZipFile(temp_file, "a", zipfile.ZIP_DEFLATED, False) as z:
z.write(self._data_dir / "db.sqlite", "db.sqlite")
z.write(self._data_dir / "metadata.json", "metadata.json")
return output_file.getvalue()
content = temp_file.getvalue()
if output_file:
with open(output_file, "wb") as f:
f.write(content)
return content

def encrypt(self, encryption_password: str):
"""Encrypts the local database using a new key, and re-uploads to the server.
Expand Down Expand Up @@ -221,6 +231,11 @@ def upload_budget(self):
"""Uploads the current file to the Actual server."""
if not self._data_dir:
raise UnknownFileId("No current file loaded.")
if not self._file:
file_id = str(uuid.uuid4())
metadata = self.get_metadata()
budget_name = metadata.get("budgetName", "My Finances")
self._file = RemoteFileListDTO(name=budget_name, fileId=file_id, groupId=None, deleted=0, encryptKeyId=None)
binary_data = io.BytesIO()
with zipfile.ZipFile(binary_data, "a", zipfile.ZIP_DEFLATED, False) as z:
z.write(self._data_dir / "db.sqlite", "db.sqlite")
Expand Down Expand Up @@ -267,12 +282,17 @@ def apply_changes(self, messages: list[Message]):
s.flush()
s.commit()

def get_metadata(self) -> dict:
"""Gets the content of metadata.json."""
metadata_file = self._data_dir / "metadata.json"
return json.loads(metadata_file.read_text())

def update_metadata(self, patch: dict):
"""Updates the metadata.json from the Actual file with the patch fields. The patch is a dictionary that will
then be merged on the metadata and written again to a file."""
metadata_file = self._data_dir / "metadata.json"
if metadata_file.is_file():
config = json.loads(metadata_file.read_text()) | patch
config = self.get_metadata() | patch
else:
config = patch
metadata_file.write_text(json.dumps(config, separators=(",", ":")))
Expand Down Expand Up @@ -371,7 +391,8 @@ def commit(self):
# commit to local database to clear the current flush cache
self._session.commit()
# sync all changes to the server
self.sync_sync(req)
if self._file.group_id: # only files with a group id can be synced
self.sync_sync(req)

def run_rules(self):
ruleset = get_ruleset(self.session)
Expand Down
9 changes: 9 additions & 0 deletions actual/api/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,15 @@ def update_user_file_name(self, file_id: str, file_name: str) -> StatusDTO:
response.raise_for_status()
return StatusDTO.model_validate(response.json())

def delete_user_file(self, file_id: str):
"""Deletes the user file that is loaded from the remote server."""
response = requests.post(
f"{self.api_url}/{Endpoints.DELETE_USER_FILE}",
json={"fileId": file_id, "token": self._token},
headers=self.headers(),
)
return StatusDTO.model_validate(response.json())

def user_get_key(self, file_id: str) -> UserGetKeyDTO:
"""Gets the key information associated with a user file, including the algorithm, key, salt and iv."""
response = requests.post(
Expand Down
1 change: 1 addition & 0 deletions actual/api/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ class Endpoints(enum.Enum):
DOWNLOAD_USER_FILE = "sync/download-user-file"
UPLOAD_USER_FILE = "sync/upload-user-file"
RESET_USER_FILE = "sync/reset-user-file"
DELETE_USER_FILE = "sync/delete-user-file"
# encryption related
USER_GET_KEY = "sync/user-get-key"
USER_CREATE_KEY = "sync/user-create-key"
Expand Down
11 changes: 10 additions & 1 deletion tests/test_api.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import pytest

from actual import Actual
from actual.exceptions import ActualError
from actual.exceptions import ActualError, UnknownFileId
from actual.protobuf_models import Message


Expand All @@ -16,3 +16,12 @@ def test_api_apply(mocker):
m.dataset = "accounts"
with pytest.raises(ActualError, match="column 'bar' at table 'accounts' not found"):
actual.apply_changes([m])


def test_rename_delete_budget_without_file():
actual = Actual.__new__(Actual)
actual._file = None
with pytest.raises(UnknownFileId, match="No current file loaded"):
actual.delete_budget()
with pytest.raises(UnknownFileId, match="No current file loaded"):
actual.rename_budget("foo")
30 changes: 27 additions & 3 deletions tests/test_integration.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,10 @@
)


@pytest.fixture
def actual_server():
with DockerContainer("actualbudget/actual-server:24.5.0").with_exposed_ports(5006) as container:
@pytest.fixture(params=["24.3.0", "24.4.0", "24.5.0", "24.6.0", "24.7.0"])
def actual_server(request):
# we test integration with the 5 latest versions of actual server
with DockerContainer(f"actualbudget/actual-server:{request.param}").with_exposed_ports(5006) as container:
wait_for_logs(container, "Listening on :::5006...")
yield container

Expand Down Expand Up @@ -98,3 +99,26 @@ def test_update_file_name(actual_server):
with Actual(f"http://localhost:{port}", password="mypass") as actual:
with pytest.raises(ActualError):
actual.rename_budget("Failing name")


def test_reimport_file_from_zip(actual_server, tmp_path):
port = actual_server.get_exposed_port(5006)
backup_file = f"{tmp_path}/backup.zip"
# create one file
with Actual(f"http://localhost:{port}", password="mypass", bootstrap=True) as actual:
# add some entries to the budget
actual.create_budget("My Budget")
get_or_create_account(actual.session, "Bank")
actual.commit()
actual.upload_budget()
# re-download file and save as a backup
with Actual(f"http://localhost:{port}", password="mypass", file="My Budget") as actual:
actual.export_data(backup_file)
actual.delete_budget()
# re-upload the file
with Actual(f"http://localhost:{port}", password="mypass") as actual:
actual.import_zip(backup_file)
actual.upload_budget()
# check if the account can be retrieved
with Actual(f"http://localhost:{port}", password="mypass", file="My Budget") as actual:
assert len(get_accounts(actual.session)) == 1
Loading