🗃️ Plumb through the BLE objects to the server #963
Merged
Commits on Apr 14, 2024
-
🗃️ Plumb through the BLE objects to the server
These are the server side changes related to e-mission/e-mission-docs#1062 The changes are fairly straightforward, and consistent with https://github.com/e-mission/e-mission-docs/blob/2665b39e1335ea04896b6944a4a065e7887b6cdc/docs/dev/back/adding_a_new_data_type.md?plain=1#L4 Concretely: - we add a new `bluetoothble` wrapper - we add references to it to `entry.py` and `builtin_timeseries.py` - we add formatters for both android and iOS A new wrinkle this time is that we are modifying the FSM, so there are also new transitions. Those needed to be added to the enums in the transition wrapper, and to the maps in the formatters so that the enums could be created properly. Bonus fix: check for the `None` transition properly on android and iOS to avoid spurious errors ``` >>> broken_transition_example = {'_id': ObjectId('661b129fc271a44bb612b464'), 'metadata': {'time_zone': 'America/Los_Angeles', 'plugin': 'none', 'write_ts': 1713050268.574551, 'platform': 'ios', 'read_ts': 0, 'key': 'statemachine/transition', 'type': 'message'}, 'user_id': UUID('f1aaae55-fc42-4527-bf7f-33f84d7c8c2f'), 'data': {'currState': 'STATE_ONGOING_TRIP', 'transition': None, 'ts': 1713050268.574418}} >>> broken_transition_example_entry = ad.AttrDict(broken_transition_example) >>> enufit.format(broken_transition_example_entry) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/kshankar/Desktop/data/e-mission/gis_branch_tests/emission/net/usercache/formatters/ios/transition.py", line 64, in format data.transition = transition_map[entry.data.transition].value KeyError: None ----- fixed code ------ >>> importlib.reload(enufit) <module 'emission.net.usercache.formatters.ios.transition' from '/Users/kshankar/Desktop/data/e-mission/gis_branch_tests/emission/net/usercache/formatters/ios/transition.py'> >>> enufit.format(broken_transition_example_entry) AttrDict({'_id': ObjectId('661b129fc271a44bb612b464'), 'user_id': UUID('f1aaae55-fc42-4527-bf7f-33f84d7c8c2f'), 'metadata': AttrDict({'time_zone': 'America/Los_Angeles', 'plugin': 'none', 'write_ts': 1713050268.574551, 'platform': 'ios', 'read_ts': 0, 'key': 'statemachine/transition', 'type': 'message', 'write_local_dt': LocalDate({'year': 2024, 'month': 4, 'day': 13, 'hour': 16, 'minute': 17, 'second': 48, 'weekday': 5, 'timezone': 'America/Los_Angeles'}), 'write_fmt_time': '2024-04-13T16:17:48.574551-07:00'}), 'data': AttrDict({'curr_state': 2, 'transition': None, 'ts': 1713050268.574551, 'local_dt': AttrDict({'year': 2024, 'month': 4, 'day': 13, 'hour': 16, 'minute': 17, 'second': 48, 'weekday': 5, 'timezone': 'America/Los_Angeles'}), 'fmt_time': '2024-04-13T16:17:48.574551-07:00'})}) ``` Testing done: Used the corresponding changes in e-mission/e-mission-phone#1144 to simulate BLE as follows: - Region exit - A few range updates until the `ble_beacon_found` transition was generated - Turned on location mocking from the android and iOS simulators, and manually generated the start trip transition on android - Clicked "range update" at random times during the simulated trip - BLE beacon lost from the UI - Turn off location mocking - Force end trip transition Testing Results: Android: ``` START 2024-04-13 17:36:02.096342 POST /usercache/put END 2024-04-13 17:36:02.313529 POST /usercache/put ebc13f1b-671b-4094-bce6-fed342da7e9c 0.2171182632446289 START 2024-04-13 17:36:02.583812 POST /usercache/get END 2024-04-13 17:36:02.591868 POST /usercache/get ebc13f1b-671b-4094-bce6-fed342da7e9c 0.007989168167114258 ``` ``` >>> edb.get_usercache_db().count_documents({"metadata.key": "background/bluetooth_ble"}) 57 >>> edb.get_timeseries_db().count_documents({"metadata.key": "background/bluetooth_ble"}) 0 ``` ``` 2024-04-13 17:37:57,635:DEBUG:140704655566784:write_ts = 1713054811.255 2024-04-13 17:37:57,635:DEBUG:140704655566784:module_name = emission.net.userca che.formatters.android.bluetooth_ble 2024-04-13 17:37:57,636:DEBUG:140704655566784:write_ts = 1713054811.294 2024-04-13 17:37:57,636:DEBUG:140704655566784:module_name = emission.net.userca che.formatters.android.bluetooth_ble 2024-04-13 17:37:57,636:DEBUG:140704655566784:write_ts = 1713054811.316 2024-04-13 17:37:57,636:DEBUG:140704655566784:module_name = emission.net.userca che.formatters.android.bluetooth_ble 2024-04-13 17:37:57,637:DEBUG:140704655566784:write_ts = 1713054811.339 2024-04-13 17:37:57,637:DEBUG:140704655566784:module_name = emission.net.userca che.formatters.android.bluetooth_ble 2024-04-13 17:37:57,637:DEBUG:140704655566784:write_ts = 1713054811.369 ``` ``` >>> edb.get_usercache_db().count_documents({"metadata.key": "background/bluetooth_ble"}) 0 >>> edb.get_timeseries_db().count_documents({"metadata.key": "background/bluetooth_ble"}) 57 ``` iOS ``` START 2024-04-13 16:17:50.707151 POST /usercache/put START 2024-04-13 16:17:50.737703 POST /usercache/get END 2024-04-13 16:17:50.763880 POST /usercache/get f1aaae55-fc42-4527-bf7f-33f84d7c8c2f 0.026064157485961914 END 2024-04-13 16:17:51.340867 POST /usercache/put f1aaae55-fc42-4527-bf7f-33f84d7c8c2f 0.6329052448272705 ``` ``` >>> edb.get_usercache_db().count_documents({"metadata.key": "background/bluetooth_ble"}) 74 ``` ``` DEBUG:root:module_name = emission.net.usercache.formatters.ios.bluetooth_ble DEBUG:root:write_ts = 1713050204.075974 DEBUG:root:module_name = emission.net.usercache.formatters.ios.bluetooth_ble DEBUG:root:write_ts = 1713050204.077563 DEBUG:root:module_name = emission.net.usercache.formatters.ios.bluetooth_ble DEBUG:root:write_ts = 1713050204.078974 DEBUG:root:module_name = emission.net.usercache.formatters.ios.bluetooth_ble DEBUG:root:write_ts = 1713050207.32417 DEBUG:root:module_name = emission.net.usercache.formatters.ios.bluetooth_ble DEBUG:root:write_ts = 1713050207.326033 ``` ``` >>> edb.get_usercache_db().count_documents({"metadata.key": "background/bluetooth_ble"}) 0 >>> edb.get_timeseries_db().count_documents({"metadata.key": "background/bluetooth_ble"}) 74 >>> edb.get_timeseries_error_db().count_documents({"metadata.key": "background/bluetooth_ble"}) 0 ```
-
✅ Change the transition for the negative test to one that actually fails
Our negative test in this test suite used `transition': none` as an invalid transition. However, in e-mission@c86fcf5 we started handling `None` transitions properly So we need to change it to a transition that is actually invalid. Testing done: Before this, `testMoveWhenEmpty` failed. Now, `testMoveWhenEmpty` passes
-
⬆️ Upgrade the pymongo calls to the latest version
`save` -> `replace_one` with upsert `remove` -> `delete_one`
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.