The primary motivation for this project is a proof of concept to demonstrate that a computer vision approach for an aimbot is possible for cloud hosted games. The secondary motivation for this project is to exercise my programming skills in a computer vision context.
- A single thread captures the active screen by taking a screenshot in a loop using the win32api for Windows or the mss screen capture library for MacOS or Linux.
This bot uses the YOLOv5 object detection model to detect humanoid figures in captured frames. This network was trained using real world photographs of various scenes containing many different types of objects, including humans. This model can be used to detect humanoid character models in video game screen captures. However this model does not functional perfectly in the context of Destiny 2 and will miss some characters. In practice, any object detection model trained on photographs of humans should work in a video game context that uses humanoid character models.
The Lucas–Kanade method is used to track detected objects between frames. This method is only used when capturing frames at a rate that is >= 60 fps. The Lucas-Kanade method is ineffective when the frames are captured at a rate < 60 fps because the change in position of objects between frames will be to large to track effectively. If the captured frame rate is < 60 fps objects will be redetected in each frame instead.
The priority target is the character model who's bounding box has the largest area. The bounding box area is a proxy for how close the character is to yourself. Character models with large bounding box areas tend to be close while character models with small bounding box areas tend to be far away. In comparison with tradisional aimbots, the distance between your character and every opposing character is calculated given the (x, y, z) coordinates of each character entity. The closest visible character is selected as the target.
- direct x
Temp
- Enemies behind cover (partial)
- Enemies peek shooting (partial or full)
- inability to detect enemies that arent on the screen (full)
- Team mates are indistinguishable from enemies (false positive)
- Non character models detected (false positive)
- Missed character models (false negative)
Temp
>= 30
Temp
- Temp