Lesson 19: Client Prediction & Reconciliation

🏛️ The Authoritative Server

Think of a board game played by mail with a referee. Players never move pieces themselves; they post their moves to the referee, who updates the one real board and posts a photo of it back to everyone. Online games work the same way:

• The client sends inputs ("I held right for 0.016 s"), never positions.
• The server runs the real simulation at a fixed tick rate and sends back snapshots of the world, each stamped with its tick number.
• Each snapshot also says which of your inputs it includes: the sequence number of the last input the server applied.

Because the server only accepts inputs, a modified client can't claim to be somewhere it could never have walked to. The server can also check each input: a movement longer than it allows, or an input it has already applied, is refused.

The messages are small, fixed records, so a frozen dataclass (from Intermediate Python) fits them well:

@dataclass(frozen=True)
class InputCmd:
    seq: int          # 1, 2, 3, ... in the order the client made them
    move: tuple       # (x, y), each -1, 0 or 1
    dt: float         # how long this input was held, seconds


@dataclass(frozen=True)
class Snapshot:
    tick: int         # server tick that produced it
    player: tuple     # tuples, not Vector2s: a snapshot must never change later
    last_seq: int     # newest input the server has applied
    bot: tuple

Why tuples? A snapshot is a photo of one moment. If it held the server's own Vector2, it would hold a reference to an object that keeps changing. This complete program shows the bug:

import pygame

player = pygame.Vector2(100, 50)
history = []

for tick in range(3):
    history.append(player)              # BUG: stores the same Vector2 object three times
    player += pygame.Vector2(10, 0)     # += changes that object in place
print("aliased:", [tuple(p) for p in history])

player = pygame.Vector2(100, 50)
history = []
for tick in range(3):
    history.append(tuple(player))       # a tuple is a frozen copy of this moment
    player += pygame.Vector2(10, 0)
print("copied: ", history)

Every "aliased" entry shows the final position, because all three are the same object. Store a copy: a tuple, pygame.Vector2(p), or a new dict. Frozen dataclasses of tuples can't be changed at all, which turns this bug into an error you'd see immediately.

🐢 A Network You Can Control

Real latency is hard to experiment with: it changes by the minute and you can't turn it up on demand. So the lab replaces the network with a tiny class that delivers each message a fixed time after it was sent, in order, like a TCP connection with no loss. One LaggyLink carries inputs up, another carries snapshots down.

class LaggyLink:
    """One direction of a network connection: in order, nothing lost, `delay` seconds late."""

    def __init__(self, delay):
        self.delay = delay
        self.queue = deque()                        # (deliver_at, message)

    def send(self, now, message):
        self.queue.append((now + self.delay, message))

    def receive(self, now):
        out = []
        while self.queue and self.queue[0][0] <= now:
            out.append(self.queue.popleft()[1])
        return out

now is the simulation's own clock, the sum of every frame's dt, so the whole program stays repeatable and the tests can drive it with any frame times they like. With a 50 ms delay each way, the round trip is 100 ms.

Now imagine the client simply waits for the server. You press right; the input needs 50 ms to reach the server, waits up to one tick (33 ms at 30 Hz) to be applied, and the snapshot needs another 50 ms to come back. Your square starts moving well over 100 ms after you pressed the key. You can feel that delay in the demo below, and it only gets worse on a long-distance connection.

🔮 Client-Side Prediction

The fix is to stop waiting. The client knows the movement rules, so it applies its own input immediately and shows the result, while the same input travels to the server. This works only if client and server compute exactly the same thing, so both call one shared function:

def apply_input(pos, cmd):
    """The ONE movement rule, shared by client and server. Returns a new Vector2."""
    move = pygame.Vector2(cmd.move)
    if move.length_squared() > 0:
        move = move.normalize()
    new = pos + move * SPEED * cmd.dt
    half = SIZE / 2
    new.x = max(half, min(WIDTH - half, new.x))     # the client clamps exactly like the server
    new.y = max(half, min(HEIGHT - half, new.y))
    return new

Three details matter:

• Every input carries its own dt. Frames are not all 1/60 s long. If the client moves you by one frame's dt but the server assumes 1/60 s, the two drift apart on every uneven frame.
• The clamp lives inside the shared rule. If only the server kept you inside the arena, the client's prediction would run past the wall and snap back on every snapshot.
• The server applies each input on its own. Several inputs can arrive in one tick. Applying only the newest, or treating them as one tick's worth of movement, loses movement the client already showed.

def apply(self, cmd):
    """Server: apply ONE input with its own dt, after checking it is sane."""
    if cmd.seq <= self.last_seq:
        return                                  # duplicate or out of date
    dt = max(0.0, min(MAX_INPUT_DT, cmd.dt))    # nobody gets to move for "5 seconds"
    move = (max(-1, min(1, cmd.move[0])), max(-1, min(1, cmd.move[1])))
    self.player = apply_input(self.player, InputCmd(cmd.seq, move, dt))
    self.last_seq = cmd.seq
    self.applied += 1

Capping each input's dt stops the simplest speed hack. A real server would also check that a client's inputs don't add up to more time than has actually passed.

🔁 Server Reconciliation

Prediction alone has a flaw: the server is still the referee. Sometimes it knows something the client doesn't, like another player who bumped into you or a door that closed, and its answer differs from your guess. When a snapshot arrives, the client must accept the server's position without throwing away the inputs the server hasn't seen yet.

The snapshot's last_seq makes this possible. It says: "this position includes every input up to number last_seq". So the client:

1. drops every pending input with seq <= last_seq (the server has applied them);
2. starts from the server's position;
3. replays the remaining inputs, each with its own dt, through the same apply_input.

def on_snapshot(self, snap):
    self.snapshots.append(snap)
    self.server_pos = pygame.Vector2(snap.player)
    self.pending = [c for c in self.pending if c.seq > snap.last_seq]
    if not self.predict:
        self.pos = pygame.Vector2(self.server_pos)
    elif self.reconcile:
        corrected = pygame.Vector2(self.server_pos)
        for cmd in self.pending:                      # replay what the server hasn't seen,
            corrected = apply_input(corrected, cmd)   # each with ITS OWN dt
        self.last_correction = corrected.distance_to(self.pos)
        self.pos = corrected

When the server agreed with every input, the replay lands exactly where the prediction already was, and last_correction is 0.0: the player sees nothing. When the server disagreed, the player jumps once to the corrected position, and prediction carries on from there. The pending list holds about one round trip's worth of inputs: at 60 frames per second and a 100 ms round trip, that is around six of them, plus a few more while inputs wait for the next server tick.

🎞️ Entity Interpolation on Server Ticks

Your own player is predicted, but you can't predict other players: you don't know what keys they're about to press. All you have are their snapshots, arriving 30 times a second, a little late and not quite evenly spaced. Drawing each new snapshot as it lands makes remote players jitter.

Instead, draw remote entities a little in the past, at a render time between two snapshots you already have, and blend between them with the lerp from the Interpolation & Easing lesson. Measure that time in server ticks, not in your computer's clock: snapshot tick numbers mean the same thing on every machine, while wall-clock times from two computers can't be compared.

• The client keeps a tick clock: set to the first snapshot's tick, advanced by dt * TICK_RATE every frame, and re-synced if it drifts more than a few ticks from the snapshots.
• Remote entities are drawn at render_tick = clock - INTERP_TICKS. With INTERP_TICKS = 3 at 30 Hz, that is 0.1 s behind, enough to usually have a snapshot on each side even if one arrives late.
• If the render tick is newer than every snapshot (they stopped arriving), hold the newest position rather than guessing. Guessing ahead, called extrapolation, is a separate technique with its own errors.

def interpolate(snapshots, render_tick):
    """Bot position at a (fractional) tick, blended between the two snapshots around it."""
    if render_tick >= snapshots[-1].tick:
        return pygame.Vector2(snapshots[-1].bot)    # nothing newer yet: hold, don't guess
    if render_tick <= snapshots[0].tick:
        return pygame.Vector2(snapshots[0].bot)
    for a, b in zip(snapshots, list(snapshots)[1:]):
        if a.tick <= render_tick <= b.tick:
            span = b.tick - a.tick
            t = (render_tick - a.tick) / span if span else 0.0
            return pygame.Vector2(a.bot).lerp(b.bot, t)
    return pygame.Vector2(snapshots[-1].bot)

The cost is honest and worth knowing: everything you see of other players is one-way latency plus the interpolation delay old. That gap is exactly what the next lesson, Lag Compensation, has to solve when you shoot at them.

🎮 Try It: All Three Together

The demo runs the same rules as the lab: inputs with their own dt, a server at 30 ticks per second, and a delay you can change. An autopilot holds the arrow keys for you. Turn the techniques off one at a time:

• Prediction off: the green square only moves when snapshots arrive, visibly behind the plan. Raise the latency and it gets worse.
• Server shove: the server moves you and the client didn't know. With reconciliation on, you jump once to the right place. With it off, the green square and the red outline drift apart for good.
• Interpolation off: the blue bot sits on the newest snapshot (the gray ring) and moves in small jumps instead of a smooth circle.

Each frame of the real program runs in this order: read input, predict and send it, run any server ticks that are due (applying every input that has arrived, then sending a snapshot), receive snapshots and reconcile, advance the tick clock, draw.

🏋️ Practice Exercise: Predict, Reconcile, Interpolate

"""Predict, Reconcile, Interpolate: Advanced Lesson 19 practice exercise (solution).

A client and an authoritative server run in one program, joined by two
simulated network links that deliver every message after a one-way delay.
  * Your square (green) is predicted: it moves the frame you press a key.
  * The red outline is the server's position for you, as last heard.
  * The orbiting bot exists only on the server. It shoves you when you touch it,
    so the server sometimes disagrees with your prediction; reconciliation fixes it.
  * The blue bot is interpolated between snapshots; the gray outline is the raw
    newest snapshot.
Arrows move. P = prediction, R = reconciliation, I = interpolation, L = latency.
"""
from collections import deque
from dataclasses import dataclass

import pygame


WIDTH, HEIGHT = 800, 450
TICK_RATE = 30                     # server ticks (and snapshots) per second
TICK_DT = 1 / TICK_RATE
SPEED = 220.0                      # px/s
SIZE = 28                          # player square, px
BOT_RADIUS = 18
MAX_INPUT_DT = 0.1                 # the server refuses to simulate longer inputs
INTERP_TICKS = 3                   # render remote entities 3 ticks (0.1 s) in the past
RESYNC_TICKS = 6                   # re-sync the client's tick clock if it drifts this far
LATENCIES = [0.05, 0.1, 0.2]       # one-way delays to cycle through, seconds
START = (160.0, 225.0)
ORBIT_CENTER = (430.0, 225.0)
ORBIT_RADIUS = 130.0
ORBIT_SPEED = 1.2                  # radians per second


@dataclass(frozen=True)
class InputCmd:
    seq: int                       # 1, 2, 3, ... in the order the client made them
    move: tuple                    # (x, y), each -1, 0 or 1
    dt: float                      # how long this input was held, seconds


@dataclass(frozen=True)
class Snapshot:
    tick: int                      # server tick that produced it
    player: tuple                  # tuples, not Vector2s: a snapshot must never change later
    last_seq: int                  # newest input the server has applied
    bot: tuple


def apply_input(pos, cmd):
    """The ONE movement rule, shared by client and server. Returns a new Vector2."""
    move = pygame.Vector2(cmd.move)
    if move.length_squared() > 0:
        move = move.normalize()
    new = pos + move * SPEED * cmd.dt
    half = SIZE / 2
    new.x = max(half, min(WIDTH - half, new.x))     # the client clamps exactly like the server
    new.y = max(half, min(HEIGHT - half, new.y))
    return new


class LaggyLink:
    """One direction of a network connection: in order, nothing lost, `delay` seconds late."""

    def __init__(self, delay):
        self.delay = delay
        self.queue = deque()                        # (deliver_at, message)

    def send(self, now, message):
        self.queue.append((now + self.delay, message))

    def receive(self, now):
        out = []
        while self.queue and self.queue[0][0] <= now:
            out.append(self.queue.popleft()[1])
        return out


class Server:
    def __init__(self):
        self.tick = 0
        self.player = pygame.Vector2(START)
        self.last_seq = 0
        self.applied = 0
        self.bot_angle = 0.0

    def bot_pos(self):
        offset = pygame.Vector2(ORBIT_RADIUS, 0).rotate_rad(self.bot_angle)
        return pygame.Vector2(ORBIT_CENTER) + offset

    def apply(self, cmd):
        """Apply ONE input with its own dt, after checking it is sane."""
        if cmd.seq <= self.last_seq:
            return                                  # duplicate or out of date
        dt = max(0.0, min(MAX_INPUT_DT, cmd.dt))
        move = (max(-1, min(1, cmd.move[0])), max(-1, min(1, cmd.move[1])))
        self.player = apply_input(self.player, InputCmd(cmd.seq, move, dt))
        self.last_seq = cmd.seq
        self.applied += 1

    def step(self):
        """Advance the world one tick and return a snapshot of it."""
        self.tick += 1
        self.bot_angle += ORBIT_SPEED * TICK_DT
        bot = self.bot_pos()
        gap = self.player - bot                     # the server-only rule: the bot shoves you
        reach = BOT_RADIUS + SIZE / 2
        if 0 < gap.length_squared() < reach * reach:
            self.player = bot + gap.normalize() * (reach + 25)
            self.player.x = max(SIZE / 2, min(WIDTH - SIZE / 2, self.player.x))
            self.player.y = max(SIZE / 2, min(HEIGHT - SIZE / 2, self.player.y))
        return Snapshot(self.tick, tuple(self.player), self.last_seq, tuple(bot))


class Client:
    def __init__(self):
        self.pos = pygame.Vector2(START)            # what the player sees
        self.server_pos = pygame.Vector2(START)     # the server's word, as last heard
        self.seq = 0
        self.pending = []                           # inputs the server hasn't confirmed yet
        self.snapshots = deque(maxlen=32)
        self.clock = None                           # estimated snapshot tick, float
        self.predict = True
        self.reconcile = True
        self.interpolate = True
        self.last_correction = 0.0

    def make_input(self, move, dt):
        self.seq += 1
        cmd = InputCmd(self.seq, move, dt)
        if self.predict:
            self.pos = apply_input(self.pos, cmd)   # show the result right away
        self.pending.append(cmd)
        return cmd

    def on_snapshot(self, snap):
        self.snapshots.append(snap)
        self.server_pos = pygame.Vector2(snap.player)
        if self.clock is None or abs(self.clock - snap.tick) > RESYNC_TICKS:
            self.clock = float(snap.tick)
        self.pending = [c for c in self.pending if c.seq > snap.last_seq]
        if not self.predict:
            self.pos = pygame.Vector2(self.server_pos)
        elif self.reconcile:
            corrected = pygame.Vector2(self.server_pos)
            for cmd in self.pending:                # replay what the server hasn't seen,
                corrected = apply_input(corrected, cmd)   # each with ITS OWN dt
            self.last_correction = corrected.distance_to(self.pos)
            self.pos = corrected

    def advance_clock(self, dt):
        if self.clock is not None:
            self.clock += dt * TICK_RATE

    def bot_view(self):
        """Where to draw the bot: interpolated INTERP_TICKS in the past, or the raw newest."""
        if not self.snapshots:
            return None
        if not self.interpolate:
            return pygame.Vector2(self.snapshots[-1].bot)
        return interpolate(self.snapshots, self.clock - INTERP_TICKS)


def interpolate(snapshots, render_tick):
    """Bot position at a (fractional) tick, blended between the two snapshots around it."""
    if render_tick >= snapshots[-1].tick:
        return pygame.Vector2(snapshots[-1].bot)    # nothing newer yet: hold, don't guess
    if render_tick <= snapshots[0].tick:
        return pygame.Vector2(snapshots[0].bot)
    for a, b in zip(snapshots, list(snapshots)[1:]):
        if a.tick <= render_tick <= b.tick:
            span = b.tick - a.tick
            t = (render_tick - a.tick) / span if span else 0.0
            return pygame.Vector2(a.bot).lerp(b.bot, t)
    return pygame.Vector2(snapshots[-1].bot)


def arrow_move(held, keys):
    """(x, y) from the arrow keys: held = keys seen going down, keys = get_pressed()."""
    def is_down(k):
        return k in held or keys[k]
    return (is_down(pygame.K_RIGHT) - is_down(pygame.K_LEFT),
            is_down(pygame.K_DOWN) - is_down(pygame.K_UP))


def main():
    pygame.init()
    screen = pygame.display.set_mode((WIDTH, HEIGHT))
    pygame.display.set_caption("Predict, Reconcile, Interpolate")
    clock = pygame.time.Clock()
    font = pygame.font.Font(None, 24)

    server, client = Server(), Client()
    latency_index = 1
    up = LaggyLink(LATENCIES[latency_index])        # client -> server
    down = LaggyLink(LATENCIES[latency_index])      # server -> client
    now = 0.0                                       # the simulation's clock, seconds
    tick_timer = 0.0
    held = set()                                    # arrow keys held (works with scripted input)
    sent = 0
    worst_correction = 0.0

    running = True
    while running:
        dt = min(clock.tick(60) / 1000, 0.1)
        now += dt
        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                running = False
            elif event.type == pygame.KEYDOWN:
                held.add(event.key)
                if event.key == pygame.K_p:
                    client.predict = not client.predict
                elif event.key == pygame.K_r:
                    client.reconcile = not client.reconcile
                elif event.key == pygame.K_i:
                    client.interpolate = not client.interpolate
                elif event.key == pygame.K_l:
                    latency_index = (latency_index + 1) % len(LATENCIES)
                    up.delay = down.delay = LATENCIES[latency_index]
            elif event.type == pygame.KEYUP:
                held.discard(event.key)

        move = arrow_move(held, pygame.key.get_pressed())

        # Client: predict locally, then send the input
        up.send(now, client.make_input(move, dt))
        sent += 1

        # Server: fixed ticks; every input that has arrived is applied on its own
        tick_timer += dt
        while tick_timer >= TICK_DT:
            tick_timer -= TICK_DT
            for cmd in up.receive(now):
                server.apply(cmd)
            down.send(now, server.step())

        # Client: take in snapshots, advance the tick clock
        for snap in down.receive(now):
            client.on_snapshot(snap)
            worst_correction = max(worst_correction, client.last_correction)
        client.advance_clock(dt)

        # Draw
        screen.fill((18, 22, 34))
        pygame.draw.circle(screen, (60, 66, 90), ORBIT_CENTER, ORBIT_RADIUS, 1)
        raw = client.snapshots[-1].bot if client.snapshots else None
        if raw is not None:
            pygame.draw.circle(screen, (140, 140, 150), raw, BOT_RADIUS, 2)
        view = client.bot_view()
        if view is not None:
            pygame.draw.circle(screen, (90, 160, 250), view, BOT_RADIUS)
        ghost = pygame.FRect(0, 0, SIZE, SIZE)
        ghost.center = client.server_pos
        pygame.draw.rect(screen, (240, 90, 90), ghost, 2)
        me = pygame.FRect(0, 0, SIZE, SIZE)
        me.center = client.pos
        pygame.draw.rect(screen, (90, 230, 120), me)
        one_way = LATENCIES[latency_index] * 1000
        lines = [
            f"One-way delay {one_way:.0f} ms each way, round trip {2 * one_way:.0f} ms   (L to change)",
            f"[P] prediction {'ON' if client.predict else 'off'}   [R] reconciliation "
            f"{'ON' if client.reconcile else 'off'}   [I] interpolation {'ON' if client.interpolate else 'off'}",
            f"Unconfirmed inputs: {len(client.pending)}   last correction: {client.last_correction:.1f} px",
        ]
        for i, line in enumerate(lines):
            screen.blit(font.render(line, True, (225, 228, 235)), (12, 10 + i * 22))
        pygame.display.flip()

    pygame.quit()
    in_flight = len(up.queue)
    print(f"Inputs sent: {sent}, applied by the server: {server.applied}, still in flight: {in_flight}")
    print(f"Every input accounted for: {server.applied + in_flight == sent}")
    print(f"Largest reconciliation correction: {worst_correction:.1f} px")


if __name__ == "__main__":
    main()

📓 Learning Journal

📝 Summary

An authoritative server owns the game: clients send numbered inputs that carry their own dt, and the server applies each one, checks it, and sends back snapshots stamped with the tick and the last input it applied. Waiting for those snapshots makes every key press feel slow, so the client predicts its own movement with the same rule the server uses. When a snapshot arrives, reconciliation starts from the server's answer and replays the inputs still in flight, so agreement costs nothing and disagreement costs one small jump. Other players are drawn a few ticks in the past, blended between snapshots on a tick clock.

🔭 Looking Ahead

You now see other players a little in the past, which is smooth but raises a hard question: when you shoot where you see an enemy, the server's enemy has already moved on. The next lesson, Lag Compensation, lets the server rewind time to judge your shot fairly.

❓ Common Questions

🎯 Quick Quiz

🌟 Going Further

• Smooth corrections: keep the reconciled position exact but draw the square at a point that eases toward it with drawn = drawn.lerp(pos, 1 - math.exp(-15 * dt)) (import math for it). Compare how a shove feels.
• Jitter: give LaggyLink a random.Random(seed) and add up to 30 ms to each delay while keeping messages in order. How large must INTERP_TICKS be before the bot stops stuttering?
• A second player: add another client with its own links and inputs, and draw each client's view of the other one interpolated.
• Read more: Gabriel Gambetta's illustrated series Fast-Paced Multiplayer covers these same techniques step by step.
• Coming up in Game Dev III: Advanced: the optional reading State Sync & Bandwidth shrinks these snapshots with deltas and quantization.