Alternative Allocators

[lkdvos]

This PR attempts to add some additional allocator strategies to the toolbox, focused around dense arrays of isbits types.

I added some rudimentary support for PtrArrays.jl, which provides a manual way of implementing malloc and free for the temporaries. My first tests seem to indicate however that this does not improve the performance (even making it slightly worse in most cases). This probably requires more investigation, as it seems unlikely that this should be happening.

I also added support for Bumper.jl. Here, I use their buffer types as an allocator, which means that it is quite easy to manually make use of the bumper interface as follows:

buf = Bumper.default_buffer()
@no_escape buf begin
    @tensor allocator=buf tensorexpr...
end

Nevertheless, for further automation, I also added the convenience @butensor macro, which does exactly that.

---

Some implementation notes:

In order to make this work, the current way of dispatching with StridedViews and choosing between GPU and CPU definitely does not work. Both these options require a parent type of the StridedView which is not Array (but is DenseArray!), which would now be unsupported. I could add manual select_backend procedures for this, but I am a bit scared of the ambiguities, as I don't want to have to deal with the many combinations. This should probably be reconsidered.

In principle the current implementation of the Bumper methods could be part of a package extension, as it does not even require a definition of an Allocator type. Is this something we would like?

We should probably invest some time in a proper benchmark suite, as it is quite hard to gauge the effectiveness of these methods.

[lkdvos]

Small note to myself as well: I think there are some tensoralloc calls that don't have a matching free statement in the base implementations. I'll try to fix this in this PR

[lkdvos]

I moved the Bumper implementation to a package extension and found a way to work around defining the macro in the base package, while the implementation is in the extension.
I think the main missing ingredient now is just a couple tests, after which I think this could be ready to go.

[Jutho]

Looks great. Maybe we can use the same macro in extension package technique for @cutensor?

[lkdvos]

I think that's definitely a good idea, but I would move that to a separate PR or commit ;)

[Jutho]

Ok, if tests work I think this completes this PR.

[lkdvos]

Left some minor comments, otherwise definitely looks okay to me.
I don't think we currently test the Bumper functionality at all, and @butensor seems not exported.

[lkdvos]

Technically, there is no guarantee that the lhs is the same object as the one we allocated. In that case, we created a memory leak here. In practice this probably does not really happen, but this is definitely why I used different variable names in my initial change.

[Jutho]

I am not sure, I do think permutedims! does guarantee to store the result in the first argument, i.e. this should work without the Atemp = part.

  permutedims!(dest, src, perm)

  Permute the dimensions of array src and store the result in the array dest. perm is a vector specifying a permutation of length ndims(src). The preallocated array dest should have size(dest) == size(src)[perm] and is completely
  overwritten. No in-place permutation is supported and unexpected results will happen if src and dest have overlapping memory regions.

[lkdvos]

I know the docstring guarantees this, but for example, in TensorOperations with AD, we abuse the fact that the macro automatically adds C = tensoradd!(C,...) to hack into the system and make the lhs a copy. In principle anyone could do something like this for a custom type, and then the memory management chain would be broken (doom thinking here)