Singular Value Decomposition (as an optimization)
$\def\half{\frac{1}{2}} \def\bm#1{\boldsymbol{#1}} \def\R{\mathbb{R}} \def\norm#1{\left\lVert #1 \right\rVert} \def\X{\bm{X}} \def\U{\bm{U}} \def\V{\bm{V}} \def\L{\bm{L}} \def\M{\bm{M}} \def\I{\bm{I}} \def\u{\bm{u}} \def\v{\bm{v}} \def\y{\bm{y}} \def\Q{\bm{Q}} \def\D{\bm{D}} \def\S{\bm{S}} \def\x{\bm{x}} \def\vec{\operatorname{vec}} \def\tr{\operatorname{tr}} \def\d{\operatorname{d}} \def\Xv{\widehat{\X\v}} \def\H{\operatorname{H}} \def\It{{\bm I}_2}$ The setting for the svd is the following optimization problem: $$ \sup_{\norm{u}=1,\norm{v}=1} \u^\top \X \v $$ where $\u \in \R^{m}$, $\v \in \R^{n}$ and $\X \in \R^{m \times n}$. This can be extended to the following optimization using matrices: $$ \begin{equation} \sup_{\U,\V} \qty[ \tr\qty(\U^\top \X \V ) -\frac{1}{2} \tr\qty(\L^\top\qty(\U^\top\U-\I_m)) -\frac{1}{2} \tr\qty(\M^\top\qty(\V^\top\V-\I_n)) ] \equiv \sup_{\U,\V} \phi(\U,\X,\V,\L,\M) \end{equation...