Derive the Jacobian and the Hessian of $ \phi(\boldsymbol X)=\left \lVert{\boldsymbol X \boldsymbol v}\right \rVert $
$\def\half{\frac{1}{2}} \def\bm#1{\boldsymbol{#1}} \def\R{\mathbb{R}} \def\norm#1{\left\lVert #1 \right\rVert} \def\X{\bm{X}} \def\v{\bm{v}} \def\vec{\operatorname{vec}} \def\d{{\sf d}} \def\Xv{\widehat{\X\v}} \def\D{\operatorname{D}} \def\H{\operatorname{H}} \def\It{{\bm I}_2}$$\textbf{Exercise}$: Derive the Jacobian and the Hessian of $$ \phi(\boldsymbol X)=\norm{\boldsymbol X \boldsymbol v} $$ w.r.t. $\bm{X}$ where $\bm{X} \in \R^{2 \times N}$ and $\bm{v} \in \R^{N \times 1}$. $\textit{Solution}$. Since $$\norm{\X\v}=\qty(\v^\top\X^\top\X\v)^{\half}$$ we have \begin{align*} \d\phi & = \half \qty(\v^\top\X^\top\X\v)^{-\half} \d\qty(\v^\top\X^\top\X\v) \\ & = \half \norm{\X\v}^{-1} \qty(\v^\top\qty(\d\X)^\top\X\v+\v^\top\X^\top\qty(\d\X)\v) \\ & = \half \norm{\X\v}^{-1} 2 \v^\top\X^\top\qty(\d\X)\v & {}^* \tag{I} \end{align*} ${}^*$ $\bm a^\top \bm b = \bm b^\top \bm a$ If we apply the $\vec...