Add pattern matching

[AlexandreKempf]

Description

👋 Hello here

In many real-world scenarios, training a detection model, deploying it, and getting predictions is only one half of the job. Processing of the predictions is the other half of data scientists' work, and Supervision is the only tool to help do that in object detection.
However, thinking about my previous jobs and what I needed, I realized that there was a part of post-processing that was not covered by Supervision: looking for patterns. We can think of it as a regex for object detection: Given a set of rules, give me the set of boxes that respect these rules.

For instance in this image with predictions:

That we obtained with:

    import cv2
    import supervision as sv
    from ultralytics import YOLO

    image = cv2.imread("men_on_bicycle.jpg")
    model = YOLO('yolov8s.pt')

    classes = [person, bicycle, ...]

    result = model(image)[0]
    detections = sv.Detections.from_ultralytics(result)

We could add a pattern to find an individual with their bike. The rule to identify this pattern (the regex) would be:

• there should be a person
• there should be a bike
• the person center and the bike should be aligned vertically

    pattern = sv.MatchPattern(
       [
           (lambda id: id == classes.index("bicycle") , ["Bike.class_id"]),
           (lambda id: id == classes.index("person") , ["Rider.class_id"]),
           (lambda bike_center, rider_center: abs(bike_center[0] -  rider_center[0]) <= 50),
               ["Bike.CENTER", "Rider.CENTER"]
           ),
       ]
   )
   matches = pattern.match(detections)

matches is a list of detections (List[sv.Detections]) that match the pattern. Once displayed, the 4 first matches from the list look like this:

For each match, an additional data attribute named "match_name" is added to the sv.Detection to specify to which detection corresponds each object of the pattern ("Bike" and "Rider" in our case).

Additional notes

1 - I added a little sugar on top of the feature. It is possible to define a value instead of a function. In that case, it will ensure that the arguments match exactly the value you provide.

    pattern = sv.MatchPattern(
       [
           (1 , ["Bike.class_id"]),
           (0 , ["Rider.class_id"]),
           (lambda bike_center, rider_center: abs(bike_center[0] -  rider_center[0]) <= 50),
               ["Bike.CENTER", "Rider.CENTER"]
           ),
       ]
   )
   matches = pattern.match(detections)

2 - The arguments you can give to the function directly come from sv.Detections. The complete list is:

• xyxy, mask, class_id, confidence, or tracker_id
• one of the Position enum strings
• a field from the data attribute of your detections for more custom use cases

3 - It we want to continue this feature, it would be nice to provide a small bank of function that the user can use out of the box for bounding boxes (iou, included_in, etc.) and segmentation (next_to, area_are_touching, etc.)

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

I added functional tests.

Docs

• Docs updated? What were the changes? I wanted to update the document, but I couldn't find a way to host images in the long run. Your documentation images are hosted on https://media.roboflow.com, but I can't access them. How are you usually doing it for people who don't have access to internal resources?

[CLAassistant]

All committers have signed the CLA.

[SkalskiP]

Hi @AlexandreKempf 👋🏻 Thanks for your interest in supervision. Is there any reason you decided to close your PR?

[onuralpszr]

Hi @AlexandreKempf 👋🏻 Thanks for your interest in supervision. Is there any reason you decided to close your PR?

#982

Because of CLA issue username was not valid.

[AlexandreKempf]

Hi @SkalskiP,
Sorry it was a problem on my end. To fix it,
I created a very similar PR that is open and waiting for review ;)