What Is Flow Matching in Generative AI?

How does a model turn random noise into an image or a piece of audio? Flow matching learns which way to move a sample, and how fast. Imagine that sample as a dot on a page, with an arrow showing its velocity. A real sample is represented by many numbers. For our illustration, we'll simplify that to two coordinates, so one dot stands for the whole sample. At each position and time, the model predicts an arrow. Together, these local arrows form what's called a velocity field.

We'll use time zero for noise and time one for data. Across many samples, the aim is to generate examples resembling the training data. To build a training example, choose a random noise point and, independently, a point from the dataset. Connect them with a straight line. Halfway through the time interval, take the halfway position. This straight interpolation is one useful choice within flow matching. Other paths are possible, including paths related to diffusion.

Because we chose this path, we know its velocity. Subtract the starting position from the endpoint, and you get an arrow that stays constant throughout this particular trip. Its length tells you the speed needed to cover that distance in one unit of time. We give the network an intermediate position and its time, then ask it to predict that target arrow.

The flow-matching loss measures how far the predicted arrow is from the target arrow. For the squared-error version we're using, compare their components, square each difference, and add the results. Direction and speed both matter. An arrow pointing the right way but moving twice as fast still makes an error. Training averages these errors across sampled pairs and times, giving the network feedback it can use to adjust its predictions.

Here's a tiny made-up example with two coordinates. Put the noise point at zero, zero, and the data point two units right and one unit up. Halfway along the path, we're one unit right and half a unit up. The target velocity is still two units right and one unit up per unit of time. It's the velocity for the whole straight path, rather than the remaining distance to the endpoint.

Suppose the model predicts one unit right and one unit up. The horizontal component is off by one; the vertical component is correct. Squaring those errors and adding gives a loss of one. Code that averages over the two coordinates reports one half instead. Matching the target exactly gives zero for this example. Either convention measures velocity error. That number alone doesn't tell us how convincing a generated image will look.

There's an important detail behind those target arrows. During this training setup, the model receives the current position and time, without being told which endpoint pair we sampled. Different pairs can contribute different target velocities at the same position and time. Learning with squared error therefore favors their conditional average: the average appropriate to that particular position and time. This follows from the mathematics of squared error. Each local average depends on which pairs could bring a sample there at that moment.

That also means a correct average field can still disagree with individual training targets. Some training loss can remain because those targets vary, even when the model predicts their average correctly. And although our paired paths are straight, following the learned average field can produce curved trajectories. The arrow we follow can change as we move to a different position and time.

Training can jump directly to randomly chosen moments on these known paths. It doesn't need to simulate a complete trip for each example. Generation works differently. We start with fresh noise at time zero and follow the learned field toward time one. A simple numerical method called Euler stepping multiplies the predicted velocity by a small time step, updates the position, then asks for another velocity at the new position and time.

Other numerical solvers can follow the field too. How many steps we need depends on the learned field and the accuracy we're aiming for; straight training paths don't guarantee generation in one step. The central idea is that known training paths supply velocity targets. Matching those arrows teaches a field we can follow from noise to generate new samples.

Let's turn that sample into something we can actually see. In an image model such as Stable Diffusion 3, an encoder compresses a training image into a grid of numbers called a latent representation. That grid takes the place of our dot. Training mixes it with noise and teaches the velocity along a known path. To make a new image, we start with a fresh noise grid, follow the predicted velocities, and decode the result into pixels. The text prompt helps guide the content of the picture.

For speech, a model called Voicebox works with a spectrogram, a map of how much energy different sound frequencies have over time. Its flow model learns to transform noise into that representation, guided by the words to say and any supplied audio context. A vocoder then turns the result into a waveform we can hear. The moving dot now stands for a whole sound representation. Each step changes the numbers describing that sound. The velocity tells us how those numbers change as generation progresses.

So the arrows on our page were a small view of a much bigger operation. An image or sound can involve thousands of coordinates changing together. The loss checks whether the model predicts the right change in those coordinates during training. Following the learned field produces a new representation, and the final decoder gives us something we can see or hear.