Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

R server #10

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

R server #10

wants to merge 12 commits into from

Conversation

sverhoeven
Copy link
Member

@sverhoeven sverhoeven commented Sep 25, 2024

TODO

  • add missing methods to https://github.com/eWaterCycle/bmi-r/blob/master/R/abstract-bmi.R
  • add route handlers for missing methods
  • rename package from R/remotebmi to /remotebmir
  • look at other usethis commands that can be used to improve the remotebmi R package
    • coverage
    • publish
    • docs site
    • styler / tidy
  • validate against openapi.yaml, aka use Python client or swagger ui to test all methods

@sverhoeven
Copy link
Member Author

Merged eWaterCycle/bmi-r#3

@sverhoeven sverhoeven marked this pull request as ready for review September 25, 2024 12:59
@BSchilperoort
Copy link
Member

Cool, I'm looking forward to trying it out! 🚀


``` r
# install.packages("pak")
pak::pak("github::eWaterCycle/remotebmi/R/remotebmi")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This command does not work,

> pak::pak("github::eWaterCycle/remotebmi/R/remotebmi")
! Using bundled GitHub PAT. Please add your own PAT using `gitcreds::gitcreds_set()`.
Error:                                                                          
! error in pak subprocess
Caused by error: 
! Could not solve package dependencies:
* github::eWaterCycle/remotebmi/R/remotebmi: ! pkgdepends resolution error for
github::eWaterCycle/remotebmi/R/remotebmi.
Caused by error: 
! Can't find R package in GitHub repo eWaterCycle/remotebmi in directory 'R/remotebmi'

Is that because of this:?

rename package from R/remotebmi to /remotebmir

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, the path is according to https://pak.r-lib.org/reference/pak_package_sources.html#github-packages-github-

I was using

cd R/remotebmi
R
devtools::load_all()
# and/or
devtools::install_local()

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah that uses main branch, try pak::pak("github::eWaterCycle/remotebmi/R/remotebmi#10")

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I git cloned the repo and installed it from a local path. That worked fine

## basic example code

pak::pak("github::ClaudiaBrauer/WALRUS")
pak::pak("github::eWaterCycle/bmi-r")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get stuck on building the stringi package. install.package("stringi") fails to build, and R binaries are only available for Windows and MacOS, not linux 🙄

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Managed to finally build it with install.packages("stringi", configure.args="--disable-pkg-config")

array([0.0044])
client.get_var_nbytes('Q')
'mm/h'
# TODO get_var_nbytes should return int not str
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting error! Seems like a mismapping of a function (get_var_units vs. get_var_nbytes)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, seems I fixed it in https://github.com/eWaterCycle/grpc4bmi-examples/pull/10/files, but never completed it.

Copy link
Member

@BSchilperoort BSchilperoort left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't fully follow the readme, but got past the errors and have WALRUS running 🥳

@BSchilperoort
Copy link
Member

BSchilperoort commented Sep 26, 2024

Something still seems to be going wrong. I can do the first update, but after the second .update() I can't use get_value anymore:

>>> client.update()
>>> client.get_value("Q", dest=np.array([0.]))
array([0.0044])
>>> client.update()
>>> client.get_value("Q", dest=np.array([0.]))
Traceback (most recent call last):
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 72, in map_httpcore_exceptions
    yield
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 236, in handle_request
    resp = self._pool.handle_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 216, in handle_request
    raise exc from None
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 196, in handle_request
    response = connection.handle_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 101, in handle_request
    return self._connection.handle_request(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 143, in handle_request
    raise exc
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 113, in handle_request
    ) = self._receive_response_headers(**kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 186, in _receive_response_headers
    event = self._receive_event(timeout=timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 220, in _receive_event
    with map_exceptions({h11.RemoteProtocolError: RemoteProtocolError}):
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.RemoteProtocolError: illegal status line: bytearray(b'14')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/bart/git/remotebmi/python/remotebmi/client/client.py", line 118, in get_value
    response = self.client.get(f"/get_value/{name}")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1066, in get
    return self.request(
           ^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_client.py", line 837, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_client.py", line 926, in send
    response = self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_client.py", line 954, in _send_handling_auth
    response = self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_client.py", line 991, in _send_handling_redirects
    response = self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1027, in _send_single_request
    response = transport.handle_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 235, in handle_request
    with map_httpcore_exceptions():
         ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 89, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.RemoteProtocolError: illegal status line: bytearray(b'14')

The running R server shows no errors

@sverhoeven
Copy link
Member Author

Something still seems to be going wrong. I can do the first update, but after the second .update() I can't use get_value anymore:

>>> client.update()
>>> client.get_value("Q", dest=np.array([0.]))
array([0.0044])
>>> client.update()
>>> client.get_value("Q", dest=np.array([0.]))
Traceback (most recent call last):
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 72, in map_httpcore_exceptions
    yield
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 236, in handle_request
    resp = self._pool.handle_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 216, in handle_request
    raise exc from None
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 196, in handle_request
    response = connection.handle_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 101, in handle_request
    return self._connection.handle_request(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 143, in handle_request
    raise exc
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 113, in handle_request
    ) = self._receive_response_headers(**kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 186, in _receive_response_headers
    event = self._receive_event(timeout=timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 220, in _receive_event
    with map_exceptions({h11.RemoteProtocolError: RemoteProtocolError}):
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.RemoteProtocolError: illegal status line: bytearray(b'14')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/bart/git/remotebmi/python/remotebmi/client/client.py", line 118, in get_value
    response = self.client.get(f"/get_value/{name}")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1066, in get
    return self.request(
           ^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_client.py", line 837, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_client.py", line 926, in send
    response = self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_client.py", line 954, in _send_handling_auth
    response = self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_client.py", line 991, in _send_handling_redirects
    response = self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1027, in _send_single_request
    response = transport.handle_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 235, in handle_request
    with map_httpcore_exceptions():
         ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/home/bart/git/remotebmi/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 89, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.RemoteProtocolError: illegal status line: bytearray(b'14')

The running R server shows no errors

What does R server say and what does curl do

curl http://localhost:50051/get_value/Q 

@BSchilperoort
Copy link
Member

What does R server say and what does curl do

curl http://localhost:50051/get_value/Q 

Before the first update it returns nothing.

After one update it returns
[0.0044]
and after the second update
[0.0044]
and after a third:
[0.0047]

So that looks OK. Running the curl command and then using get_value also works fine 😕 .

  1. start remote bmi R server
  2. connect in python
  3. initialize
  4. update
  5. get value
  6. update
  7. get value --> breaks!
  8. use curl command
  9. get value works OK.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants