DiffMatic.jl

Symbolic differentiation of vector/matrix/tensor expressions in Julia

Example

Create a matrix and a vector:

using DiffMatic

@matrix A
@vector x

Create an expression:

expr = x' * A * x

The variable expr now contains an internal representation of the expression x' * A * x.

Compute the gradient and the Hessian with respect to the vector x.

g = gradient(expr, x)
H = hessian(expr, x)

Convert the gradient and the Hessian to standard notation using to_std:

to_std(g) # "Aᵀx + Ax"
to_std(H) # "Aᵀ + A"

Jacobians can be computed with jacobian:

to_std(jacobian(A * x, x)) # "A"

The function derivative can be used to compute arbitrary derivatives.

to_std(derivative(tr(A), A)) # "I"

The function to_std will throw an exception when given an expression that that cannot be converted to standard notation.

Supported functions and operators

• Basic operators +, -, ', *, ^, abs, sin, cos and log
• Element-wise operators sin., cos., abs., .*, .^ and log.
• Diagonal matrix using LinearAlgebra.diagm
• Vector of a matrix diagonal using LinearAlgebra.diag
• Vector 1-norm and 2-norm using LinearAlgebra.norm(..., 1) and LinearAlgebra.norm(..., 2)
• Sums of vectors using sum
• Matrix traces using LinearAlgebra.tr
• LinearAlgebra.I for the identity matrix

Installation

Installation from the general registry:

using Pkg; Pkg.add("DiffMatic")

Acknowledgements

The implementation is based on the ideas presented in

S. Laue, M. Mitterreiter, and J. Giesen. Computing Higher Order Derivatives of Matrix and Tensor Expressions, NeurIPS 2018.