Statistical and Probabilistic Foundations of AI

Week 1: Elementary Statistics We will begin by introducing basic statistical vocabulary, which we will then use to provide methods for summarising and displaying data. We will use the process of data binning to provide histograms as well as providing standard key characteristics for location and deviation measures (e.g., means, quantiles, empirical variance). Week 2: Elementary Statistics In the second week we will continue with methods suitable for comparing and visualising data. We will introduce box plots and linear transformations as simple tools before exploring possible relationships between variables. Finally, we will look at regression models and related methods, focusing on (multiple) linear regression. Week 3: Fundamentals of Probability In the third week we will introduce the basic principles of probability. This includes the notion of sample spaces and the calculation of probabilities. We will also study the notion of conditional probability and the independence of random events. Week 4: Random Variables We will introduce the basic idea of random variables and their distribution as a way of modelling random events. We will also consider univariate discrete probability distributions before moving on to univariate probability distributions with Riemann density functions and multivariate distributions. Week 5: Random Variables Starting with the concept of multivariate distributions, we will consider their marginal and conditional distributions. We then move on to the concept of expectation of random variables and random vectors (for both discrete and continuous distributions). In particular, we study the properties of expected values as well as those of variance and covariance, respectively. Week 6: Random Variables The sixth week marks the end of the probability part, where we look at sums of random variables and ways of gaining insight into their distribution. In particular, we consider both the convolution and the generation function techniques. We finish with limit results for sequences of random variables, covering the strong law of large numbers and the central limit theorem. Week 7: Inferential Statistics In the final week we will introduce the core concepts of inferential statistics. Using probability theory, we will be able to construct point estimators and confidence intervals in a parametric statistical model. Some non-parametric examples will also be given. We will conclude the course with a brief introduction to statistical tests and selected statistical tests for Gaussian distributions.