Written exam (in case of a small number of participants: oral tests)

The lecture will repeat and introduce basic notions of mathematics used in machine learning
• Calculus: multivariate calculus (gradient and Hessian), Taylor expansion etc.
• Linear Algebra: eigenvectors, eigenvalues (including variational characterization), singular value decomposition and best low rank approximation, inverse and pseudo-inverse, norms, basic algorithms and their complexity (solving linear equations, matrix inversion, eigenvectors (power method)) etc.
• Probability: discrete and continuous probability measures (and mixed ones), basic notions, generation of random variables, conditional expectation and independence, law of large numbers and concentration inequalities for rates of convergence, central limit theorem etc.
• Statistics: parametric and non-parametric tests
• Optimization: Lagrangian and dual optimization problem, popular optimization techniques and their properties
• Optional: basic functional analysis and approximation theory, curse of dimensionality

Students learn the mathematical foundations for the latter machine learning courses. In particular,
• they know multivariate calculus and linear algebra as needed in machine learning lectures
• they can apply probability and statistics and are able to prove basic properties
• they have an overview of existing optimization techniques and are able to reformulate equivalent constrained optimization problems

The literature for this lecture will be provided at the beginning of the semester.