Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Script that uses this utility abruptly stopped working #118

Open
Etyre opened this issue Sep 1, 2024 · 1 comment
Open

Script that uses this utility abruptly stopped working #118

Etyre opened this issue Sep 1, 2024 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@Etyre
Copy link

Etyre commented Sep 1, 2024

Describe the bug
I have a script that uses roam-to-git to back up my roam database to a github repo with a github actions script. That script abruptly stopped working on July 23 and hasn't worked since then. I'm wondering if this is because something changed either with this utility or on the roam-side.

Any suggestions would be appreciated.

Thank you!

The git hub actions logs of the part that fails:

Run roam-to-git --skip-git . --formats json
  roam-to-git --skip-git . --formats json
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.8.18/x64
    ROAMRESEARCH_USER: ***
    ROAMRESEARCH_PASSWORD: ***
    ROAMRESEARCH_DATABASE: ***
[2](https://github.com/Etyre/roam-backup-JSON-only/actions/runs/10651358435/job/29523821315#step:11:2)024-09-01 05:29:32.516 | DEBUG    | roam_to_git.__main__:main:77 - No secret found at /home/runner/work/roam-backup-JSON-only/roam-backup-JSON-only/.env
2024-09-01 05:29:[3](https://github.com/Etyre/roam-backup-JSON-only/actions/runs/10651358435/job/29523821315#step:11:3)2.541 | DEBUG    | roam_to_git.scrapping:download_rr_archive:1[4](https://github.com/Etyre/roam-backup-JSON-only/actions/runs/10651358435/job/29523821315#step:11:4)7 - Creating browser
2024-09-01 0[5](https://github.com/Etyre/roam-backup-JSON-only/actions/runs/10651358435/job/29523821315#step:11:5):29:42.189 | DEBUG    | roam_to_git.scrapping:signin:251 - Opening signin page
2024-09-01 05:29:43.807 | DEBUG    | roam_to_git.scrapping:signin:254 - Waiting for  email and password fields
2024-09-01 05:29:43.85[6](https://github.com/Etyre/roam-backup-JSON-only/actions/runs/10651358435/job/29523821315#step:11:6) | DEBUG    | roam_to_git.scrapping:signin:260 - Fill email '***'
2024-09-01 05:29:44.516 | DEBUG    | roam_to_git.scrapping:signin:263 - Fill password
2024-09-01 05:29:44.666 | DEBUG    | roam_to_git.scrapping:signin:266 - Defensive check: verify that the user input field is correct
2024-09-01 05:29:46.6[7](https://github.com/Etyre/roam-backup-JSON-only/actions/runs/10651358435/job/29523821315#step:11:7)9 | DEBUG    | roam_to_git.scrapping:signin:260 - Fill email '***'
2024-09-01 05:29:46.706 | DEBUG    | roam_to_git.scrapping:signin:263 - Fill password
2024-09-01 05:29:46.72[8](https://github.com/Etyre/roam-backup-JSON-only/actions/runs/10651358435/job/29523821315#step:11:8) | DEBUG    | roam_to_git.scrapping:signin:266 - Defensive check: verify that the user input field is correct
2024-0[9](https://github.com/Etyre/roam-backup-JSON-only/actions/runs/10651358435/job/29523821315#step:11:10)-01 05:29:48.737 | DEBUG    | roam_to_git.scrapping:signin:272 - Activating sign-in
2024-09-01 05:29:48.746 | DEBUG    | roam_to_git.scrapping:go_to_database:286 - Load database from url 'https://roamresearch.com/#/app/***'
2024-09-01 05:29:48.763 | DEBUG    | roam_to_git.scrapping:_download_rr_archive:183 - Wait for interface to load
2024-09-01 05:36:29.598 | DEBUG    | roam_to_git.scrapping:download_rr_archive:163 - Closing browser json
2024-09-01 05:36:29.914 | DEBUG    | roam_to_git.scrapping:download_rr_archive:165 - Closed browser json
2024-09-01 05:36:29.915 | ERROR    | __main__:<module>:8 - An error has been caught in function '<module>', process 'MainProcess' (2404), thread 'MainThread' (140225589996416):
Traceback (most recent call last):
> File "/opt/hostedtoolcache/Python/3.8.18/x64/bin/roam-to-git", line 8, in <module>
    sys.exit(main())
    │   │    └ <function main at 0x7f88ce9dc4c0>
    │   └ <built-in function exit>
    └ <module 'sys' (built-in)>
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/roam_to_git/__main__.py", line [11](https://github.com/Etyre/roam-backup-JSON-only/actions/runs/10651358435/job/29523821315#step:11:12)1, in main
    scrap(root_zip_path, roam_formats, config)
    │     │              │             └ <roam_to_git.scrapping.Config object at 0x7f88ceb53880>
    │     │              └ ['json']
    │     └ PosixPath('/tmp/tmpqc4g64iw')
    └ <function scrap at 0x7f88ce9dc160>
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 320, in scrap
    download_rr_archive(f, format_zip_path, config=config)
    │                   │  │                       └ <roam_to_git.scrapping.Config object at 0x7f88ceb53880>
    │                   │  └ PosixPath('/tmp/tmpqc4g64iw/json')
    │                   └ 'json'
    └ <function download_rr_archive at 0x7f88cec78b80>
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 156, in download_rr_archive
    return _download_rr_archive(browser, output_type, output_directory, config)
           │                    │        │            │                 └ <roam_to_git.scrapping.Config object at 0x7f88ceb53880>
           │                    │        │            └ PosixPath('/tmp/tmpqc4g64iw/json')
           │                    │        └ 'json'
           │                    └ <roam_to_git.scrapping.Browser object at 0x7f88ce9d1ee0>
           └ <function _download_rr_archive at 0x7f88ce9d9ee0>
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 206, in _download_rr_archive
    assert dot_button is not None, "All roads leads to Roam, but that one is too long. Try " \
           └ None

AssertionError: All roads leads to Roam, but that one is too long. Try again when Roam servers are faster.
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.18/x64/bin/roam-to-git", line 8, in <module>
    sys.exit(main())
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/loguru/_logger.py", line 1[14](https://github.com/Etyre/roam-backup-JSON-only/actions/runs/10651358435/job/29523821315#step:11:15)9, in catch_wrapper
    return function(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/roam_to_git/__main__.py", line 111, in main
    scrap(root_zip_path, roam_formats, config)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 320, in scrap
    download_rr_archive(f, format_zip_path, config=config)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line [15](https://github.com/Etyre/roam-backup-JSON-only/actions/runs/10651358435/job/29523821315#step:11:16)6, in download_rr_archive
    return _download_rr_archive(browser, output_type, output_directory, config)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/roam_to_git/scrapping.py", line 206, in _download_rr_archive
    assert dot_button is not None, "All roads leads to Roam, but that one is too long. Try " \
AssertionError: All roads leads to Roam, but that one is too long. Try again when Roam servers are faster.
2024-09-01 05:36:29.924 | DEBUG    | roam_to_git.scrapping:_kill_child_process:294 - Terminate child process [psutil.Process(pid=2408, name='geckodriver', status='sleeping', started='05:29:32')]
Error: Process completed with exit code 1.

The script itself:


name: "Roam Research backup"

on:
  push:
    branches:
      - main
  schedule:
    -   cron: "5 ,11 * * *"
  workflow_dispatch:

jobs:
  backup:
    runs-on: ubuntu-latest
    name: Backup
    timeout-minutes: 15
    steps:
      -   uses: actions/checkout@v3
          with:
            lfs: true
      -   name: Set up Python 3.8
          uses: actions/setup-python@v1
          with:
            python-version: 3.8

      -   name: Setup dependencies
          run: |
            pip install git+https://github.com/MatthieuBizien/roam-to-git@master
# All the large file storage stuff to be able to work with the >100 MiB JSON.
      -   name: Install Git LFS
          run: |
            sudo apt-get update
            sudo apt-get install git-lfs
            git lfs install
      -   name: Check Git LFS Version
          run: git-lfs version

      -   name: Verify Git LFS Permissions
          run: ls -l $(which git-lfs)

      -   name: Track Large Files
          run: git lfs track

      -   name: Debugging Information
          run: |
              which git-lfs
              echo $PATH
      -   name: Pull LFS files
          run: git lfs pull
          
# End of Large File storage stuff.

      -   name: Run backup
          run: roam-to-git --skip-git . --formats json
          env:
            ROAMRESEARCH_USER: ${{ secrets.ROAMRESEARCH_USER }}
            ROAMRESEARCH_PASSWORD: ${{ secrets.ROAMRESEARCH_PASSWORD }}
            ROAMRESEARCH_DATABASE: ${{ secrets.ROAMRESEARCH_DATABASE }}

      -   name: Commit changes
          # uses: elstudio/actions-js-build/commit@v3
          # with:
          #   commitMessage: Automated snapshot
          run: |
            git config user.email "[email protected]"
            git config user.name "Etyre"
            git add .
            git commit -m "Backing up the JSON file."
            git push

@Etyre Etyre added the bug Something isn't working label Sep 1, 2024
@ErikBjare
Copy link

Seems to be yet another duplicate of #47

I've apparently also had the same issue for 4 months (last commit).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants