Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request: USB routing API should support priorities #158

Open
CFSworks opened this issue Dec 5, 2023 · 2 comments
Open

Request: USB routing API should support priorities #158

CFSworks opened this issue Dec 5, 2023 · 2 comments
Labels
BMCD related to BMCD

Comments

@CFSworks
Copy link
Contributor

CFSworks commented Dec 5, 2023

Is your feature request related to a problem? Please describe.
I have a workload that makes use of the USB 2.0 port on my board. As the workload may move "randomly" between nodes (depending on CPU/RAM availability, etc.) I have it claim the USB port with tpi usb ... on initialization. This works well, but can interfere with BMC<->node flashing operations: if the workload happens to reinitialize while I'm flashing a different node, bmcd will naively grant the USB reconnection request.

Describe the solution you'd like
I propose 2 different "priorities" for API-driven USB routing changes: Request and Demand.

bmcd shall track not only the USB connection that is established, but also the type of API call that made the connection. The USB connection is at all times in one of 3 states:

  • Idle: If and only if the USB connection is disconnected.
  • Requested: If the USB connection is connected, and the connection was due to a Request.
  • Demanded: If the USB connection is connected, and the connection was due to a Demand.

When a Demand comes in, bmcd will always apply the demanded change, and will transition to either the Demanded state (in case of connection demands) or Idle state (in case of disconnection demands).

When a Request comes in, bmcd will only grant the request if the connection is currently in the Idle or Requested state. If the connection is Demanded, it will return with an error code. (There is one exception to this: if the requested connectivity is the same as the demanded connectivity, the result is a "successful no-op" -- the API server returns success but no change to the USB state occurs.)

For backwards-compatibility, the default is Request for node (host mode)->USB 2.0 port connections. The default is Demand for all others (including bmcd-initiated USB connections).

Describe alternatives you've considered
A simpler alternative would be to disable(/read-only) the USB API altogether while node flashing is in progress, but I also make use of msd mode and the bmcd cannot readily know when I'm done with it.

Happy to discuss alternatives.

@CFSworks
Copy link
Contributor Author

CFSworks commented Dec 6, 2023

On second thought: This can be generalized to any number of priority levels (not just "Request"/"Demand") through the following rules:

  • Every USB change requested on the API includes a (positive integer) priority level
  • The current USB state has an associated (nonnegative integer) priority level
  • The current priority level is 0 if the USB is disconnected, else it's the priority of the (highest) API request for the current connection state
  • USB API requests may change the USB state only if the request priority >= current priority
  • If the requested state = the current state, USB API requests are a "successful no-op" (but will raise the "current priority" level if the "request priority" is higher)

@svenrademakers
Copy link
Collaborator

I agree that we should do something about mutually exclusive operations. Other areas of the API, such as power, suffer from the same problem. e.g., a flashing operation can be interrupted by powering off the nodes using the KEY_1 button. Currently, subsequent requests are behaving more or less blocking until the node install is finished. (This is more an effect than enforced behavior. Incoming requests are blocked on TCP level during a file transfer).

I like the priority system idea, But would a more simplistic approach not suffice already?
I expect that 95% of all boards are operated by a single user. Some, like you, have an additional script running, So let's say occasionally, we have at most 2/3 clients competing for exclusive access.

If we just go for non-blocking requests:

  • a request takes exclusivity over USB or Power
  • if another request comes in while exclusivity is taken, it returns immediately.
  • It would be enough for human operators to understand the problem.
  • The running script needs to have means to check when it can retry again.

@svenrademakers svenrademakers added the BMCD related to BMCD label Dec 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BMCD related to BMCD
Projects
None yet
Development

No branches or pull requests

2 participants