0 矩阵求导的本质

矩阵\(A\)对矩阵\(B\)求导：矩阵\(A\)中的每个元素分别对矩阵\(B\)中的每个元素求导。

1 两个概念

1.1 标量函数

\(\left [ f \right ]_{1\times 1}\)：函数\(f\)是一个具体的值，如：\(f(x_{1},x_{2})=2x_{1}+3x_{2}^{2}\)。

1.2 向量函数

\(\left [ f \right ]_{m\times n}\)：形如\(f=\left [f_{1}(x)=x_{1},f_{2}(x)=x^{2} \right]_{1\times 2}\)

2 以分母布局为例的矩阵求导基本原则

2.1 原则一：

若\(f\)为标量函数，\(f=f(x_{1},x_{2},...,x_{p})\)，\(X\)为列向量，\(X=\begin{bmatrix} x_{1}\\ x_{2}\\ \vdots \\ x_{p}\\end{bmatrix}\)，则定义：
\(\frac{\partial f}{\partial X}=\begin{bmatrix} \frac{\partial f}{\partial x_{1}}\\ \frac{\partial f}{\partial x_{2}}\\ \vdots\\ \frac{\partial f}{\partial x_{p}}\\end{bmatrix}\)

若\(f\)为标量函数，\(x\)为行向量，则定义：

\(\frac{\partial f}{\partial X}=\begin{bmatrix} \frac{\partial f}{\partial x_{1}} &\frac{\partial f}{\partial x_{2}} & \cdots & \frac{\partial f}{\partial x_{p}} \end{bmatrix}_{1\times p}\)

2.2 原则二

若\(f\)为列向量，\(f=\begin{bmatrix} f_{1}(x)\\ f_{2}(x)\\ \vdots \\ f_{m}(x) \end{bmatrix}_{m\times 1}\)，\(X\)为标量，则定义：

\(\frac{\partial f}{\partial X}=\begin{bmatrix} \frac{\partial f_{1}(x)}{\partial X} & \frac{\partial f_{2}(x)}{\partial X} & \cdots & \frac{\partial f_{m}(x)}{\partial X} \end{bmatrix}=\frac{\partial (f^{T})}{\partial X}\)

若\(\begin{bmatrix} f \end{bmatrix}_{m\times 1}\)，\(\begin{bmatrix} X \end{bmatrix}_{p\times 1}\)，则定义：

\(\frac{\partial f}{\partial X}=\begin{bmatrix} \frac{\partial f}{\partial x_{1}}\\ \frac{\partial f}{\partial x_{2}}\\ \vdots \\ \frac{\partial f}{\partial x_{p}} \end{bmatrix}=\begin{bmatrix} \frac{\partial f_{1}}{\partial x_{1}} & \frac{\partial f_{2}}{\partial x_{1}} & \cdots & \frac{\partial f_{m}}{\partial x_{1}}\\ \frac{\partial f_{1}}{\partial x_{2}} & \frac{\partial f_{2}}{\partial x_{2}} & \cdots & \frac{\partial f_{m}}{\partial x_{2}}\\ \vdots & \vdots & \vdots & \vdots \\ \frac{\partial f_{1}}{\partial x_{p}} & \frac{\partial f_{2}}{\partial x_{p}} & \cdots & \frac{\partial f_{m}}{\partial x_{p}} \end{bmatrix}\)

3 常用的公式

对于形如\(f(A,x)\)的函数，其中\(A=\begin{bmatrix} a_{1} &a_{2} &\cdots & a_{p} \end{bmatrix}\)，\(x=\begin{bmatrix} x_{1}\\ x_{2}\\ \vdots \\ x_{p} \end{bmatrix}\)，则有：

• \(\frac{\partial f}{\partial x}=\frac{\partial A^{T}x}{\partial x}=\frac{\partial x^{T}A}{\partial x}=A\)
• \(\frac{\partial f}{\partial x}=\frac{\partial x^{T}Ax}{\partial x}=(A+A^{T})x\)
• \(\frac{\partial f}{\partial x}=\frac{\partial Ax}{\partial x}=A^{T}\)
• \(\frac{\partial x^{T}x}{\partial x}=2x\)

若：\(U=\begin{bmatrix} u_{1}(x)\\ u_{2}(x)\\ \vdots \\ u_{p}(x) \end{bmatrix}\)，\(V=\begin{bmatrix} v_{1}(x)\\ v_{2}(x)\\ \vdots \\ v_{p}(x) \end{bmatrix}\)，则有：

• \(\frac{\partial U^{T}V}{\partial x}=\frac{\partial U}{\partial x}V+\frac{\partial V}{\partial x}U\)
• \(\frac{\partial (U+V)}{\partial x}=\frac{\partial U}{\partial x}+\frac{\partial V}{\partial x}\)

矩阵求导

原文：https://www.cnblogs.com/foghorn/p/14964849.html