动态张量的输入和输出

[sihaiwuhu]

`func run() {

ort.SetSharedLibraryPath(`/home/lixiang/workspace/gospace/smart_pas/onnx/shared_lib/libonnxruntime.so`)

err := ort.InitializeEnvironment()
if err != nil {
	panic(err)
}

ocr, _ := ort.NewDynamicAdvancedSession(
	`/home/lixiang/workspace/gospace/smart_pas/onnx/model/ch_PP-OCRv4_rec_infer.onnx`,
	[]string{"x"}, []string{"softmax_11.tmp_0"},
	nil,
)
defer ocr.Destroy()

mat := gocv.IMRead("image1.jpg", gocv.IMReadColor)

// mat := gocv.IMRead("/home/lixiang/workspace/gospace/smart_pas/screenshot-20241110-150936.png", gocv.IMReadColor)
defer mat.Close()

outputTensor1 := []ort.ArbitraryTensor{nil}

for _, value := range []int{100, 150, 200, 250, 300} {

	blob1 := gocv.BlobFromImage(mat, 1.0/255.0, image.Point{value, 48}, gocv.NewScalar(0, 0, 0, 0), true, false)
	data, _ := blob1.DataPtrFloat32()
	blob1.Close()

	Input1, err := ort.NewTensor([]int64{1, 3, 48, int64(value)}, data)
	if err != nil {
		panic(err)
	}

	err = ocr.Run(
		[]ort.ArbitraryTensor{Input1},
		outputTensor1,
	)
	if err != nil {
		fmt.Println(err)
	}
	Input1.Destroy()

	output := outputTensor1[0].(*ort.Tensor[float32])

	fmt.Println(len(output.GetData()), output.GetShape())

	time.Sleep(time.Millisecond * 20)

}

}
func main() {

run()

}
`
In the loop, I input tensors of different shapes, but an error occurs when outputting. Is there something wrong with what I wrote?

2024-11-17 09:40:13.789631626 [E:onnxruntime:, sequential_executor.cc:514 ExecuteKernel] Non-zero status code returned while running Softmax node. Name:'p2o.Softmax.2' Status Message: /onnxruntime_src/onnxruntime/core/framework/execution_frame.cc:171 onnxruntime::common::Status onnxruntime::IExecutionFrame::GetOrCreateNodeOutputMLValue(int, int, const onnxruntime::TensorShape*, OrtValue*&, const onnxruntime::Node&) shape && tensor.Shape() == *shape was false. OrtValue shape verification failed. Current shape:{1,12,6625} Requested shape:{1,19,6625}

If you put the line ‘outputTensor1 := []ort.ArbitraryTensor{nil}’ in the loop, the tensor can be output correctly, but the memory will continue to be consumed. Calling destroy() will not release the memory.

[yalue]

Yes, you'll need to Destroy() the previous tensor and replace the output tensor with nil in every loop if the dimensions may change. It's a limitation of onnxruntime itself---if it's writing output into an existing tensor, it will not be able to change the dimensions. onnxruntime only offers two options for receiving output when running a session:

1. Write output into an existing tensor that matches all types and dimensions
2. Allocate an entirely new tensor.

Calling Destroy() will not release the memory

How did you test this? Quite often, simply calling destroy will not cause the GC to be invoked, and onnxruntime may also be caching its own internal data. Have you checked that memory usage will continue to rise if you iterate the loop 100+ times? For example, see my response to a similar issue here: #48 (comment)

If you do see an issue with memory continuing to increase after ~1000 loops, please post a minimal snippet with your test code. The portion of the code for transferring onnxruntime-allocated output tensors into Go-managed Tensors is fairly complicated and I may have made a mistake there.