Artificial Intelligence
from First Principles

In Week 1, we introduced you to the artificial intelligence landscape, exploring the relationships between AI, Machine Learning (ML), Deep Learning (DL), and Generative AI. We exposed you to various DL architectures—like Feedforward Neural Networks (FFNN), RNNs, CNNs, and Transformers—alongside practical application examples. Next, we shifted our focus to the core structure of a shallow neural network, specifically diving into a single hidden layer network. Together, we thoroughly examined fundamental concepts, including the roles of neurons, channel weights, and biases at each layer.

Building on the fundamental concepts of Neural Networks introduced in our first week, the second week took a more structured and hands-on approach to understanding how neural networks actually learn. We focused on the backpropagation and parameter updating processes, with particular emphasis on applying the chain rule to compute loss gradients and understand how errors are propagated through the network. To strengthen this understanding, we also implemented a neural network entirely from scratch, without relying on any AI or deep learning libraries. This allowed us to explore the underlying computations directly and develop a solid foundation before moving on to PyTorch and other libraries in the following week.

Building on the first two weeks, where we used a simple neural network and implemented the computations from scratch to understand the mathematics behind it, Week 3 marked our transition from first principles to practical automation. In practice, frameworks such as PyTorch and TensorFlow/Keras provide standard libraries for constructing neural networks, selecting activation and loss functions, optimizers, and automating backpropagation and parameter updates. In this session, we introduced PyTorch and connected these automated processes directly to the mathematical concepts developed previously. This progression reflects the philosophy of the course: understanding the fundamentals behind the tools, enabling us to use AI technologies more effectively and ultimately develop new ideas, innovations, and research solutions.

Building on the mathematical foundations and PyTorch implementation introduced in the earlier weeks, Week 4 focused on understanding the mathematics behind Convolutional Neural Networks (CNNs) and their application to engineering problems. In this session, we explored the fundamental operations of CNNs before developing a CNN–FFNN surrogate model step by step for a simply supported beam problem based on the Finite Element Method (FEM), represented by the governing equation ([K]{D}={F}). A total of 5,000 labelled datasets were generated for training and validation, with the model trained to predict the displacement degrees of freedom ({D}) from the stiffness matrix ([K]) and load vector ({F}). This session continued the course philosophy of understanding AI mathematically through engineering arguments, while demonstrating how deep learning can be applied directly to practical engineering problems.

Building on the introduction to Convolutional Neural Networks and engineering surrogate modelling in Week 4, Week 5 shifted the attention towards Recurrent Neural Networks (RNNs) and their application to sequential and time-dependent data. The session explored the fundamental mathematical ideas behind recurrent connections, hidden states, and the propagation of information from one time step to another, including the forward-pass calculations involving weights, biases, activation functions, and prediction losses across sequential steps. The unfolded RNN representation and Backpropagation Through Time (BPTT) were also introduced, highlighting how gradients were propagated through recurrent paths using the chain rule and how temporal dependencies could contribute to vanishing gradients. These concepts were related to physical engineering problems involving structural responses, signals, sensor measurements, and inflow–outflow hydrographs. A practical Python implementation was demonstrated through a flood-routing example using generated datasets, data normalisation, model training, and performance evaluation based on MSE, RMSE, and R². The trained RNN successfully reproduced outflow hydrographs, illustrating how recurrent architectures could capture the time-dependent behaviour of physical systems while maintaining the emphasis on understanding the underlying mathematics and computational logic rather than treating AI as a black box.

Building on the neural network foundations developed in the earlier weeks, Week 6 introduced Physics-Informed Neural Networks (PINNs), where governing equations, boundary conditions, and initial conditions are incorporated directly into the learning process through the loss formulation. Unlike conventional data-driven neural networks, PINNs use the underlying physics of the problem to guide the network towards physically meaningful solutions. In Part A, we developed the formulation entirely from first principles using the dimensional one-dimensional wave equation and a single-hidden-layer neural network. The forward-pass equations, physics residuals, boundary and initial-condition losses, gradients, and parameter updates were derived explicitly before implementing the complete PINN from scratch using NumPy, without relying on PyTorch or automatic differentiation. A simple SGD-like sequential update strategy was used so that each mathematical step could be followed directly in the computational implementation. Numerical results demonstrated how the PINN solution progressively converged towards the analytical wave solution as the number of neurons and training epochs increased. This session established the mathematical and computational foundations required for Part B, where the formulation will be extended to nondimensional equations, total weighted and normalised losses, deeper neural networks, and automatic differentiation.