Issue while using NRF dongle with HCI_USB firmware

[akuker]

Good afternoon!

I'm trying to run through some of the demo programs that come with Bumble. However, I'm getting an error and the commands hang indefinitely. Ex:

akuker@xxx:~/bumble/bumble_src$ bumble-controller-info usb:1
<<< connecting to HCI...
<<< connected
WARNING:bumble.transport.usb:!!! out transfer not completed: status=4

akuker@xxx:~/bumble/bumble_src$ bumble-scan usb:2fe3:000b
<<< connecting to HCI...
<<< connected
WARNING:bumble.transport.usb:!!! out transfer not completed: status=4

Using pyusb gives a slightly different error:

akuker@xxx:~/bumble/bumble_src$ bumble-scan pyusb:2FE3:000B
<<< connecting to HCI...
<<< connected
WARNING:bumble.transport.pyusb:USB write error: [Errno 32] Pipe error

My setup:

I'm running Ubuntu 20.04.05 on a reasonably high-end IBM Thinkpad.

I'm using the Nordic nRF52840 dongle with the Zephyr USB_HID demo loaded. I believe this is working properly, since lsusb gives reasonable output:

akuker@xxx:~$ lsusb
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 028: ID 2fe3:000b ZEPHYR USB-DEV
......

Bumble usb probe shows two Bumble-compatible devices:

akuker@xxx:~/bumble/bumble_src$ bumble-usb-probe 
ID 8087:0032
  Bumble Transport Names: usb:0 or usb:8087:0032
  Bus/Device:             003/007
  Class:                  Wireless Controller
  Subclass/Protocol:      1/1 [Bluetooth]

ID 2FE3:000B
  Bumble Transport Names: usb:1 or usb:2FE3:000B or usb:2FE3:000B/912A41BB46CF0B92
  Bus/Device:             003/028
  Class:                  Device
  Subclass/Protocol:      0/0
  Serial:                 912A41BB46CF0B92
  Manufacturer:           ZEPHYR
  Product:                USB-DEV

I'm assuming the host BT stack (BlueZ) is using usb:0/hci0 and not using the Nordic dongle.

akuker@xxx:~$ hciconfig 
hci0:	Type: Primary  Bus: USB
	BD Address: xx:xx:xx:xx:xx  ACL MTU: 1021:4  SCO MTU: 96:6
	DOWN 
	RX bytes:16331 acl:0 sco:0 events:2614 errors:0
	TX bytes:641217 acl:0 sco:0 commands:2612 errors:0

Do you have any suggestions of things to try to troubleshoot/get to root cause?

[akuker]

FYI: I tried this on a vanilla Ubuntu 22.04.01 desktop with the same results. (The Thinkpad I mentioned above is a corporate-owned asset with extra security stuff on it)

Are there any special settings required in the udev config?

akuker@yyy:/etc/udev/rules.d$ cat /etc/udev/rules.d/70-bumble.rules 
#Nordict nRF52840 with Zephyr USB_HID installed
SUBSYSTEMS=="usb", ATTRS{idVendor}=="2fe3", ATTRS{idProduct}=="000b", MODE:="0666"

[akuker]

Additional note: I tried the same with the HCI_UART firmware for the NRF dongle, and things appear to be working. bumble-scan serial:/dev/ttyACM0 dumped a bunch of information about real BT devices

[barbibulle]

Hi. This should work in theory (I have used the Zephyr USB HCI app on an nrf dongle in the past).
The USB descriptors exposed by Zephyr in that configuration are somewhat unusual (they don't use the "normal" endpoint numbers), but that shouldn't be a problem.
In my tests, I was using a mac as the host, I have not tried that specific case on a Linux host. I will try that shortly and follow up.

[barbibulle]

I did a bit of digging into this issue. I built the most recent version of the Zephyr HCI USB sample, and ran into the same issue on a mac. I went back to the previous version I had built, and that worked, on the same nrf52840 dongle.
Looking at the difference between the two, I noticed that the USB descriptors in the newer build included a new entry in the composite device: a CDC ACM interface was added, where none existed before.
Looking at it with bumble-usb-probe -v:
Old version:

ID 2FE3:000B
  Bumble Transport Names: usb:0 or usb:2FE3:000B or usb:2FE3:000B/47FE3BD1D1E3CB33
  Bus/Device:             020/030
  Class:                  Device
  Subclass/Protocol:      0/0
  Serial:                 47FE3BD1D1E3CB33
  Manufacturer:           ZEPHYR
  Product:                USB-DEV
  Configuration 1
      Interface: 0 (Wireless Controller, 1/1 [Bluetooth])
        Endpoint 0x81: INTERRUPT IN
        Endpoint 0x01: BULK OUT
        Endpoint 0x82: BULK IN

New version:

ID 2FE3:000B
  Bumble Transport Names: usb:0 or usb:2FE3:000B or usb:2FE3:000B/47FE3BD1D1E3CB33
  Bus/Device:             020/052
  Class:                  Device
  Subclass/Protocol:      0/0
  Serial:                 47FE3BD1D1E3CB33
  Manufacturer:           ZEPHYR
  Product:                USB-DEV
  Configuration 1
      Interface: 0 (Communications and CDC Control, 2/0)
        Endpoint 0x81: INTERRUPT IN
      Interface: 1 (CDC Data, 0/0)
        Endpoint 0x82: BULK IN
        Endpoint 0x01: BULK OUT
      Interface: 2 (Wireless Controller, 1/1 [Bluetooth])
        Endpoint 0x83: INTERRUPT IN
        Endpoint 0x02: BULK OUT
        Endpoint 0x84: BULK IN

This CDC ACM interface somehow breaks things. I removed it from the config (CONFIG_USB_CDC_ACM in KConfig) and rebuilt, everything works as expected.
I suspect that somehow when that new interface is enabled, the HCI USB sample app stops working.
I don't know enough about the internals of Zephyr, and about those specific subsystems and how they get enabled for the sample app, but it looks like something's not working as expected when both CDC ACM and HCI USB are enabled.
(and since CDC ACM is enabled by default, which is a recent change, the HCI USB sample app may be broken in the current version of the project).

NOTE: the fact that there's an extra interface isn't by itself a problem. The Bumble USB transport code does a dynamic lookup of the endpoints and will find the correct endpoint numbers (they are not hardcoded). This can be seen by running with BUMBLE_LOGLEVEL=debug. The relevant lines from the log:

DEBUG:bumble.transport.usb:USB Device: Bus 020 Device 052: ID 2fe3:000b
DEBUG:bumble.transport.usb:selected endpoints: configuration=1, interface=2, setting=0, acl_in=0x84, acl_out=0x02, events_in=0x83, 

We can see here that the USB transport did find the right endoints for ACL_IN, ACL_OUT and EVENTS_IN.