Method process([class boofcv.struct.image.InterleavedU8]) does not exist

[andresilmor]

Hi, im having the error mentioned on the title, the scenary is the following:

Image is "loaded" on a client which uses gRPC to send bytes to a server, in the server (where is used PyBoof), I decode and detect QR Codes, then returns the QR Code content.

For testing i use this to simulate the client:

img = Image.open("download.png")
img_byte_arr = io.BytesIO()
img.save(img_byte_arr, format='PNG', subsampling=0, quality=100)
img_byte_arr = img_byte_arr.getvalue()

Then i send the img_byte_arr to the server, I already used this for other goals and works fine, and cant really change this process.

On the server side I normally do this on the bytes i receive:
npimg = cv2.imdecode(np.frombuffer(img_byte_arr, np.uint8), -1)
And then i do ML inferences on the image, working without problems, i can do things like cv2.rectangle()

As I could understood i need also to do this to use the PyBoof

boof_img = pb.ndarray_to_boof(npimg=npimg)
detector.detect(boof_img)

The problem now is that i get the error:

Traceback (most recent call last):
  File "[REDACTED]\server.py", line 45, in <module>
    detector.detect(boof_img)
  File "[REDACTED]\.venv\Lib\site-packages\pyboof\recognition.py", line 341, in detect
    self.java_obj.process(image)
  File "[REDACTED]\.venv\Lib\site-packages\py4j\java_gateway.py", line 1322, in __call__
    return_value = get_return_value(
                   ^^^^^^^^^^^^^^^^^
  File "[REDACTED]\.venv\Lib\site-packages\py4j\protocol.py", line 330, in get_return_value
    raise Py4JError(
py4j.protocol.Py4JError: An error occurred while calling o38.process. Trace:
py4j.Py4JException: Method process([class boofcv.struct.image.InterleavedU8]) does not exist
        at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:321)
        at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:329)
        at py4j.Gateway.invoke(Gateway.java:274)
        at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
        at py4j.commands.CallCommand.execute(CallCommand.java:79)
        at py4j.GatewayConnection.run(GatewayConnection.java:238)
        at java.base/java.lang.Thread.run(Thread.java:1623)

I have almost 0% knowledge on Java XD.
The best I could do was to compare the objects types, between my code and the ones present on the examples, but on that it seems fine, at least the final object to do the detection is the same, <class 'py4j.java_gateway.JavaObject'>, also im using Windows for development/experiment XD

I found this issue which seems to have the same/close problem but completely different scenary (AttributeError: 'numpy.ndarray' object has no attribute '_get_object_id' #21 )

Among the open-source libraries to decode QR Codes and Micro QR Codes yours seems to answer my needs very well (i tested with your app) so I wanted to resolve this problem XD

[andresilmor]

Update: Hi again, i did not solve the problem but in meantime I putted my hands on the Java library and did got it working already with gRPC and deployed to an server, still not tested with the end-user app but the test flow works without problems. But i will keep the issue opened in case someone else find the same error.

[lessthanoptimal]

I think I know what's going on. It's expecting a gray scale image not a color image. In Java this would get caught at compile time. I'm going to work on adding a runtime check that gives a more informative error message.

[lessthanoptimal]

More informative error message:

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./qrcode_detect.py", line 13, in <module>
    detector.detect(image)
  File "/home/pja/projects/PyBoof/venv/lib/python3.8/site-packages/PyBoof-0.43.1-py3.8.egg/pyboof/recognition.py", line 347, in detect
    raise TypeError("Expected image to be GrayU8 or GrayF32") from exc

[andresilmor]

Thanks,

It must be that then, since i, indeed, do not convert the image to gray scale, my bad there XD, i did notice that in the Java there was that, but since the QRCode_Detect example that i followed didn't had that and it uses Color Image, and being a "need to do", i thought that would be happening "behind the scenes".

Anyways the Java version is working already with the end-user app, it takes some tries to detect the qr codes, 3-10, not sure why, but not an issue.

[lessthanoptimal]

Could you share those challenging cases? I can add them to the test data set.

[andresilmor]

You mean the images?

Could you share those challenging cases? I can add them to the test data set.

[lessthanoptimal]

yep

[andresilmor]

Im affraid not, at least not today, they are camera frames captured from a HoloLens 2 device that are sended to the server as "streaming".

The QR Codes used till now are, one on a laptop screen containing a random JWT, almost full screen, and QR Codes containing NanoID's, just a large number basically XD, in paper 8cm x 8cm, i will probably change that last one to Micro QR Code. The QR Codes, both, dont seem to be the problem because I tested them as just image and it worked in the first try.

I did not saved the frame itself and tested it alone, true, i will do that tomorrow and put in here the "results".