Implementation of Cython or pybind for better performance

[ACE07-Sev]

Is your feature request related to a problem?

This feature request would provide an objectively better performance, especially on exponentially expensive operations such as tensor contraction, which in context of quantum computing, and more specifically state preparation using MPS become massively expensive in terms of time and computation when piled up.

Whilst the package is using numpy, it's not as efficient or as fast as a C based code, so this would be a great but also difficult addition.

Describe the solution you'd like

Depending on preference, quimb should use either pybind11, or Cython to implement the computationally intensive operations, such as contraction, permutation, compression, and SVD using C code instead of python, and simply use python as the interface. This will allow for the integration of the feature without breaking the current codebase or requiring massive update for current users.

Describe alternatives you've considered

No response

Additional context

No response

[jcmgray]

It would of course be nice to make things as fast as possible. I'll just mention a different things in this regard:

1. quimb itself will probably remain pure python. There is some tension between making things as optimized and focused as possible, and making quimb as versatile and interactive as possible, but my priorities tend towards the latter.
2. Given that, its obviously still possible to use other fast libraries written in C like numpy and torch. Currently pretty much all the functions you mention already don't really involve any python - tensordot, einsum, qr, svd, eigh, etc which make up the bulk of TN operations. The question is then
   A. whether one can abstract some 'higher level' primitives (i.e. blocks of array functions) for the tensor operations, quimb actually already does this to some extent with numba and autoray.compose
   B. whether one can accelerate the 'python scaffolding' around the array functions, e.g. tags and index manipulation. One place I have recently done this is cotengrust which provides rust implementions of basic contraction path finding algorithms.
3. Because quimb can support a wide range of array backends, you can just use it trace through e.g. jax arrays and thus generate highly optimized routines if that is the goal, though of course there are various caveats here. There is some experimental support for automating this process here - https://github.com/jcmgray/quimb/blob/main/quimb/experimental/autojittn.py.

With regard to these points, useful things for you or anyone to provide would be:

1. any specific benchmarks where there is a large difference between quimb and another library, i.e where can we expect better performance?
2. Any profiling which shows which the slow parts are for you particular case, and based on that A. any suggestions for 'higher level' tensor primitives or B. python scaffolding that could be accelerated, a la cotengrust., e.g. oset comes to mind.
3. actual implementations! I'm unlikely to write any C myself for example.

[ACE07-Sev]

Greetings dear Johnny,

Hope you are doing well. Thank you very much for making this discussion possible!

I see, with that being said I think I will proceed with quimb, and also perform a case by case basis of comparing whether the functions/methods used are as fast as a pure C or Julia-based code using Fastor or Eigen. I will get back to this thread once I have a concrete finding on this and can make a PR if needed.