矩阵求导

Hey

Machine Learning notes

矩阵求导

1. 向量关于标量X的求导 $$\left[ \begin{matrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{matrix} \right]$$

经过求导后

$$\left[ \begin{matrix} \frac{\partial y_1}{\partial x} \\ \frac{\partial y_2}{\partial x} \\ \vdots \\ \frac{\partial y_n}{\partial x} \end{matrix} \right]$$

1. 矩阵对标量X的求导-和对向量的求导使同理的

2. 矩阵对向量X求导

$$A=\left[ \begin{matrix} a11 & a12 & \cdots & a1n \\ a21 & a22 & \cdots & a2n \\ \vdots & \vdots & \ddots & \vdots \\ am1 & am2 & \cdots & am \\ \end{matrix} \right]$$ $$\vec{X}=\begin{Bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{Bmatrix} \tag{5}$$ $$\vec{Y} = \vec{A}\cdot\vec{X}=\left[ \begin{matrix} a11\cdot x1 + a12\cdot\ x2 \cdot + a1n \cdot xn \\ a21\cdot x1 + a22\cdot\ x2 \cdot + a2n \cdot xn \\ \vdots & \\ am1\cdot x1 + am2\cdot\ x2 \cdot + amn \cdot xn \\ \end{matrix} \right]$$ $$\frac{\partial \vec{Y}}{\partial \vec{X}}=\left[ \begin{matrix} a11 & a21 & \cdots & am1 \\ a12 & a22 & \cdots & am2 \\ \vdots & \vdots & \ddots & \vdots \\ a1n & a2n & \cdots & amn \\ \end{matrix} \right]=A^T$$