Most machine learning and AI models combine large data, intense computations and special hardware. At the intersections mathematical problems emerge, which enable systematic solutions. In this talk, I discuss three novel methods, with open-source software, that exploit certain problem properties, such as Kronecker form or low-rankness. In particular, when tensor data is a combination of continuous and discrete information (hybrid), standard decompositions either don't exploit this structure fully or become computationally very expensive. Through careful preprocessing, I decouple large factor subproblems into a sequence of smaller ones or apply effective iterative solvers. Second, for high throughput streaming data, it is typically important to process this information rapidly. For large systems, recomputing the best low-rank approximation is prohibitive. Therefore, I develop an updating method that is closely related to updating the singular value decomposition, however is more effective at tracking a low-rank subspace.

Third, nonlinear optimization is a mathematical foundation for training AI models. When only gradients are available, it is often effective to approximate 2nd order curvature information for the iterates. Thus, I propose a parametric family of 2nd derivative Hessian estimates, which applies to deterministic or stochastic optimization. I conclude with a set of future directions.