Skip to content

Latest commit

 

History

History
72 lines (53 loc) · 5.37 KB

actions.md

File metadata and controls

72 lines (53 loc) · 5.37 KB
Raw

Low-Level Insights on Android Input Events

What Are Input Events

Android OS uses events concept to handle signals received from different input devices. It supports a wide range of different devices, such as touch screen, light pen, mouse, keyboard, but most of them are using MotionEvent or KeyEvent APIs, which are derived from the base InputEvent class. These APIs are quite flexible and support a wide range of different settings. We are particularly interested in the part of these APIs, which are responsible for touch and keyboard events generation/emulation.

How Input Events Are Working

An event is an object, which is generated in response to a signal from an input device. These objects are then delivered to the corresponding kernel subsystem, which processes them and notifies all listening processes about taps, key presses, swipes, etc. This means that in order to emulate a signal generated by an external device, such as touch screen, it is necessary to just send event objects with the same properties and in the same sequence as they would be generated by a real device.

Lets Simulate a Single Tap

Each input device has a set of actions whose property ranges and sequences are already predefined in the operating system. These actions we call "tap", "swipe" or "double tap", etc. The properties of each action could be found either in the Android documentation or in the OS source code. In order to perform events sequence, which is recognized as single tap, it is necessary to generate the following motion events:

  • ACTION_POINTER_DOWN
  • wait 125ms (525ms or longer wait will synthesize a long tap action instead)
  • ACTION_POINTER_UP. The downTime property should be set to the same timestamp as for ACTION_POINTER_DOWN

It is also important, that coordinates and other properties of both the starting and the closing event should be equal except of the eventTime one, which is always equal to the current system timestamp in milliseconds (SystemClock.uptimeMillis()). The MotionEvent object itself could be created via obtain API, where parameters are the corresponding event properties.

After events are created they must be passed to the system for execution. Such action is not secure, so it is only possible in instrumented tests via injectInputEvent method of IUiAutomationConnection interface. This is a very low-level method and it can only be accessed via reflection in automated tests. Normally, UiAutomator APIs have wrappers over it (like touchDown, touchMove, etc.), that already simulate the stuff described above.

How About More Complicated Actions

In theory it is possible to emulate any input action using a generated events sequence. Although, some actions, like multi-finger swipe, are really complicated and require a lot of events to be generated with correct properties and timings. The OS simply ignores given events if they don't follow internal action requirements. There is also a little assistance from UiAutomator framework, because Google only has wrappers for a limited set of simple actions, like tap, drag or swipe. So, in order to generate two-finger symmetric swipe we need to supply the following events chain:

  • ACTION_POINTER_DOWN (finger1)
  • ACTION_POINTER_DOWN (finger2)
  • start a loop, that generates ACTION_POINTER_MOVE event each 20ms for both finger1 and finger2 until ACTION_POINTER_UP is performed. The downTime should be set to the same timestamp as for the corresponding ACTION_POINTER_DOWN. The coordinates of each move event should be points belonging to the path between the corresponding start and end point coordinates normalized by the current timestamp (x0 + sqrt(sqr(x0) + sqr(x1))) * k, y0 + sqrt(sqr(y0) + sqr(y1))) * k).
  • ACTION_POINTER_UP (finger1) The downTime property should be set to the same timestamp as for the corresponding ACTION_POINTER_DOWN
  • ACTION_POINTER_UP (finger2) The downTime property should be set to the same timestamp as for the corresponding ACTION_POINTER_DOWN

Google uses 5ms as interval duration between move events in UiAutomator code, but according to our observations this value is too little, which causes noticeable delays in actions execution.

Further Reading

Unfortunately, there is no so much detailed information on this topic. The only reliable source of the information are Android OS sources themselves. Consider visiting the following resources: