Hello everyone! Welcome to my blog! I’ll be sharing my journey as a PhD student at the Academy of Mathematics and Systems Sciences (AMSS), Chinese Academy of Sciences (CAS).

This blog isn’t just about mathematics – I’ll also explore topics like economics, finance, and anything else that sparks my interest.

All the cover images on this blog were taken by myself. It is strictly prohibited to reproduce any part of this website without my permission.

If you have any questions or just want to reach out, feel free to email me at wangtuo1020@outlook.com or contact me on WhatsApp at +64 20 4138 4018.

Some of the posts on this blog were originally written by me quite a long time ago in other languages. I have used AI to translate them into English before sharing them here. In addition, some of the articles were first handwritten on paper by me and then converted into Markdown code using AI tools before being uploaded. If you notice any mistakes, please feel free to contact me.

Note: If the LaTeX in the text doesn’t display properly, just refresh the page.

The cover image of this article was taken at the Sydney Opera House in Sydney, Australia.

The Stein Variational Gradient Descent (SVGD) algorithm was first introduced by Liu and Wang [LW16], whose idea is to transport a set of $N$ particles $\{x_i\}_{i=1}^N$ in $\mathbb R^d$ so that their empirical measure

$$
\mu^N:=\frac{1}{N}\sum_{i=1}^N\delta_{x_i}
$$

approximates the target probability measure

$$
\rho_\infty (x)\ dx=Z^{-1} e^{-V(x)}\ dx
$$

with an unknown normalization factor $Z$.

Optimization over the space of probability measures is not only widely applicable, but also offers a useful perspective for analyzing certain complicated finite-dimensional nonconvex optimization problems. In particular, lifting such problems to optimization problems over probability measures can lead to better structural properties, such as convexity. Mean-field Langevin dynamics provides a representative example of this idea. Its central motivation is that some highly nonconvex optimization problems arising in neural network training become better behaved when reformulated as the optimization of a functional on the space of probability measures. This viewpoint also makes it possible to build a theoretical foundation for understanding the convergence of SGD. In what follows, we briefly introduce this perspective, mainly based on the paper by [Hu, Kaitong, et al]. The main analytical framework of this theory can be illustrated by Figure 2.

This is the talk I presented in the Optimization Group Seminar, titled: An Introduction to Optimization over the Space of Probability Measures: From Sampling to Wasserstein Gradient Flow.

In generative modeling, we are given a collection of training samples $\{x_i\}_{i=1}^N$ and wish to generate new samples from the underlying target distribution $\pi$. There are already many established approaches to this problem, including likelihood-based methods, implicit generative models such as GANs, and score-based diffusion models. More recently, the flow matching framework has emerged as another powerful paradigm. In what follows, we introduce the basic ideas of flow matching and explain how works.

We consider the following interacting particle system in $\mathbb{R}^d$:

$$
\begin{aligned}
\dot{x}_i(t) &= -\frac{1}{N}\sum _{j=1}^{N}\nabla K(x_i(t)-x_j(t)) -\frac{1}{N} \sum _{j=1} ^{N}K\big(x_i(t)-x_j(t)\bigr)\nabla V (x_j(t)), \\\
x_i(0) &= x_i ^0 \in \mathbb R^d, \qquad i=1,\cdots,N.
\end{aligned} \tag{1}
$$

We refer to each of the $N$ functions $x_i(\cdot) \in \mathbb{R}^d$ as a particle. The function $K : \mathbb{R}^d \mapsto \mathbb{R}$ is a smooth, symmetric, and positive definite kernel. The function $V : \mathbb{R}^d \to \mathbb{R}$ is a smooth potential such that $e^{-V(x)}$ is integrable. More specific assumptions about $K$ and $V$ are given below.

We are interested in the macroscopic behavior of the particle system $(1)$ as $N \to \infty$ in the framework of mean field limit. Formally this mean field limit is described by the following non-local, nonlinear partial differential equation (PDE):
$$
\begin{aligned}
&\partial_t \rho = \nabla \cdot \left(\rho\left(K \ast (\nabla \rho+\nabla V\rho)\right)\right), \\\
&\rho(0,\cdot) = \rho_0(\cdot).
\end{aligned} \tag{2}
$$

In this article, we will prove the well-posedness of (1) and (2).

In this article, we will introduce the gradient flows in Wasserstein space.

In this article, we will introduce the general theory in metric spaces.

In this article, we will introduce the gradient flows in the metric setting.

In this article, we will introduce some basic results of gradient flows in the Euclidean space.