Upper/LowerTriangular gradients

[pshashk]

No description provided.

[MikeInnes]

Looks good to me, but can you do a quick rebase / merge master so that the tests pass?

[pshashk]

Somehow *Triangular gradient definitions overwrite each other and UpperTriangular(::TrackedArray) returns tracked LowerTriangular. Minimal example with current master:

using LinearAlgebra, Tracker
using Tracker: track, data, @grad
using LinearAlgebra: UpperTriangular, LowerTriangular

LinearAlgebra.UpperTriangular(a::TrackedMatrix) = track(UpperTriangular, a)
LinearAlgebra.LowerTriangular(a::TrackedMatrix) = track(LowerTriangular, a)
@grad UpperTriangular(a) = UpperTriangular(data(a)), Δ -> (UpperTriangular(Δ),)
@grad LowerTriangular(a) = LowerTriangular(data(a)), Δ -> (LowerTriangular(Δ),)

UpperTriangular(param(rand(4,4)))

Tracked 4×4 LowerTriangular{Float64,Array{Float64,2}}:
 0.938394   ⋅         ⋅         ⋅      
 0.755833  0.960363   ⋅         ⋅      
 0.426979  0.843864  0.238508   ⋅      
 0.101401  0.10873   0.555092  0.381779