Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CUDA example of tensor network contraction #154

Merged
merged 4 commits into from
Jun 19, 2024
Merged

Add CUDA example of tensor network contraction #154

Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
40d8549
Add CUDA example
Todorbsc Jun 18, 2024
266c7a2
Update CUDA example
Todorbsc Jun 18, 2024
5d02c62
Update CUDA example
Todorbsc Jun 19, 2024
4be25b7
Add packages for CUDA example
Todorbsc Jun 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
183 changes: 183 additions & 0 deletions examples/cuda.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# CUDA tensor network contraction demo"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Requirements\n",
"- The system must have a CUDA GPU available."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"┌ Warning: You are using a non-official build of Julia. This may cause issues with CUDA.jl.\n",
"│ Please consider using an official build from https://julialang.org/downloads/.\n",
"└ @ CUDA /home/bsc/bsc021386/.julia/packages/CUDA/75aiI/src/initialization.jl:180\n"
]
}
],
"source": [
"using Tenet\n",
"using EinExprs\n",
"using Adapt\n",
"using CUDA\n",
"using BenchmarkTools"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a random tensor network and find its contraction path:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"SizedEinExpr{Symbol}(EinExpr{Symbol}(Symbol[], EinExpr{Symbol}[EinExpr{Symbol}([:C, :a, :N, :I], EinExpr{Symbol}[]), EinExpr{Symbol}([:C, :a, :N, :I], EinExpr{Symbol}[EinExpr{Symbol}([:C, :N, :G, :Z, :U, :R, :E, :X, :S], EinExpr{Symbol}[]), EinExpr{Symbol}([:a, :I, :G, :Z, :U, :R, :E, :X, :S], EinExpr{Symbol}[EinExpr{Symbol}([:B, :D, :G, :Z], EinExpr{Symbol}[]), EinExpr{Symbol}([:B, :a, :I, :D, :U, :R, :E, :X, :S], EinExpr{Symbol}[EinExpr{Symbol}([:I, :D, :E, :O, :M], EinExpr{Symbol}[]), EinExpr{Symbol}([:B, :a, :U, :R, :X, :S, :O, :M], EinExpr{Symbol}[EinExpr{Symbol}([:B, :V, :P, :a, :W, :A], EinExpr{Symbol}[EinExpr{Symbol}([:B, :V, :J, :b, :c, :P], EinExpr{Symbol}[]), EinExpr{Symbol}([:J, :b, :c, :a, :W, :A], EinExpr{Symbol}[])]), EinExpr{Symbol}([:V, :P, :U, :R, :X, :S, :W, :A, :O, :M], EinExpr{Symbol}[EinExpr{Symbol}([:P, :X, :T, :L, :H], EinExpr{Symbol}[]), EinExpr{Symbol}([:V, :U, :R, :S, :W, :A, :O, :M, :T, :L, :H], EinExpr{Symbol}[EinExpr{Symbol}([:R, :S, :A, :K, :M, :d, :Y, :T], EinExpr{Symbol}[]), EinExpr{Symbol}([:V, :U, :W, :K, :O, :d, :Y, :L, :H], EinExpr{Symbol}[EinExpr{Symbol}([:W, :K, :Q, :O, :F], EinExpr{Symbol}[]), EinExpr{Symbol}([:V, :U, :Q, :F, :d, :Y, :L, :H], EinExpr{Symbol}[])])])])])])])])]), Dict(:b => 9, :F => 3, :D => 4, :B => 9, :V => 5, :c => 5, :K => 9, :S => 6, :X => 9, :E => 7…))"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Initialize random tensor network\n",
"regularity = 6\n",
"ntensors = 10\n",
"tn = rand(TensorNetwork, ntensors, regularity)\n",
"path = einexpr(tn; optimizer=Exhaustive())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Transform the tensors' data types to `CuArray`s:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"TensorNetwork (#tensors=10, #inds=30)"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"cudatn = adapt(CuArray, tn)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Benchmark CUDA tensor network contraction:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"BenchmarkTools.Trial: 3371 samples with 1 evaluation.\n",
" Range \u001b[90m(\u001b[39m\u001b[36m\u001b[1mmin\u001b[22m\u001b[39m … \u001b[35mmax\u001b[39m\u001b[90m): \u001b[39m\u001b[36m\u001b[1m1.324 ms\u001b[22m\u001b[39m … \u001b[35m 16.946 ms\u001b[39m \u001b[90m┊\u001b[39m GC \u001b[90m(\u001b[39mmin … max\u001b[90m): \u001b[39m0.00% … 55.47%\n",
" Time \u001b[90m(\u001b[39m\u001b[34m\u001b[1mmedian\u001b[22m\u001b[39m\u001b[90m): \u001b[39m\u001b[34m\u001b[1m1.389 ms \u001b[22m\u001b[39m\u001b[90m┊\u001b[39m GC \u001b[90m(\u001b[39mmedian\u001b[90m): \u001b[39m0.00%\n",
" Time \u001b[90m(\u001b[39m\u001b[32m\u001b[1mmean\u001b[22m\u001b[39m ± \u001b[32mσ\u001b[39m\u001b[90m): \u001b[39m\u001b[32m\u001b[1m1.469 ms\u001b[22m\u001b[39m ± \u001b[32m940.462 μs\u001b[39m \u001b[90m┊\u001b[39m GC \u001b[90m(\u001b[39mmean ± σ\u001b[90m): \u001b[39m3.02% ± 4.18%\n",
"\n",
" \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m▁\u001b[39m▅\u001b[39m█\u001b[34m▇\u001b[39m\u001b[39m▅\u001b[39m▃\u001b[39m▁\u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[32m \u001b[39m\u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \n",
" \u001b[39m▂\u001b[39m▃\u001b[39m▆\u001b[39m▇\u001b[39m█\u001b[39m▇\u001b[39m█\u001b[39m█\u001b[39m█\u001b[39m█\u001b[34m█\u001b[39m\u001b[39m█\u001b[39m█\u001b[39m█\u001b[39m▇\u001b[39m▅\u001b[39m▅\u001b[39m▄\u001b[39m▃\u001b[39m▃\u001b[39m▂\u001b[39m▂\u001b[32m▂\u001b[39m\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▁\u001b[39m▁\u001b[39m▂\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▂\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▂\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m \u001b[39m▃\n",
" 1.32 ms\u001b[90m Histogram: frequency by time\u001b[39m 1.71 ms \u001b[0m\u001b[1m<\u001b[22m\n",
"\n",
" Memory estimate\u001b[90m: \u001b[39m\u001b[33m258.63 KiB\u001b[39m, allocs estimate\u001b[90m: \u001b[39m\u001b[33m4418\u001b[39m."
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"@benchmark CUDA.@sync contract(cudatn; path)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Benchmark regular tensor network contraction:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"BenchmarkTools.Trial: 40 samples with 1 evaluation.\n",
" Range \u001b[90m(\u001b[39m\u001b[36m\u001b[1mmin\u001b[22m\u001b[39m … \u001b[35mmax\u001b[39m\u001b[90m): \u001b[39m\u001b[36m\u001b[1m 65.622 ms\u001b[22m\u001b[39m … \u001b[35m261.489 ms\u001b[39m \u001b[90m┊\u001b[39m GC \u001b[90m(\u001b[39mmin … max\u001b[90m): \u001b[39m 3.43% … 74.33%\n",
" Time \u001b[90m(\u001b[39m\u001b[34m\u001b[1mmedian\u001b[22m\u001b[39m\u001b[90m): \u001b[39m\u001b[34m\u001b[1m 97.514 ms \u001b[22m\u001b[39m\u001b[90m┊\u001b[39m GC \u001b[90m(\u001b[39mmedian\u001b[90m): \u001b[39m 2.04%\n",
" Time \u001b[90m(\u001b[39m\u001b[32m\u001b[1mmean\u001b[22m\u001b[39m ± \u001b[32mσ\u001b[39m\u001b[90m): \u001b[39m\u001b[32m\u001b[1m130.156 ms\u001b[22m\u001b[39m ± \u001b[32m 69.544 ms\u001b[39m \u001b[90m┊\u001b[39m GC \u001b[90m(\u001b[39mmean ± σ\u001b[90m): \u001b[39m39.14% ± 29.56%\n",
"\n",
" \u001b[39m▄\u001b[39m▄\u001b[39m▁\u001b[39m \u001b[39m \u001b[39m \u001b[39m█\u001b[39m▄\u001b[39m \u001b[39m█\u001b[34m \u001b[39m\u001b[39m█\u001b[39m \u001b[39m▁\u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m▁\u001b[39m \u001b[32m \u001b[39m\u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m▁\u001b[39m▄\u001b[39m \u001b[39m▁\u001b[39m \u001b[39m \u001b[39m \u001b[39m \n",
" \u001b[39m█\u001b[39m█\u001b[39m█\u001b[39m▁\u001b[39m▁\u001b[39m▆\u001b[39m█\u001b[39m█\u001b[39m▁\u001b[39m█\u001b[34m▆\u001b[39m\u001b[39m█\u001b[39m▁\u001b[39m█\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m█\u001b[39m▆\u001b[32m▁\u001b[39m\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▆\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▆\u001b[39m█\u001b[39m█\u001b[39m▁\u001b[39m█\u001b[39m▁\u001b[39m▆\u001b[39m \u001b[39m▁\n",
" 65.6 ms\u001b[90m Histogram: frequency by time\u001b[39m 261 ms \u001b[0m\u001b[1m<\u001b[22m\n",
"\n",
" Memory estimate\u001b[90m: \u001b[39m\u001b[33m222.25 MiB\u001b[39m, allocs estimate\u001b[90m: \u001b[39m\u001b[33m2835\u001b[39m."
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"@benchmark CUDA.@sync contract(tn; path)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Julia 1.10.0",
"language": "julia",
"name": "julia-1.10"
},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia",
"version": "1.10.0"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Loading