Understanding World Models Part 4 of 7
  1. 1 Your Time Series Is Already Training Data
  2. 2 A World Model in Forty Lines of TypeScript
  3. 3 The World Model Equation Is a WHERE Clause
  4. 4 Learning the Heater From Its Own Logs
  5. 5 Latent State Is the Fields You Never Logged
  6. 6 How a World Model Differs From an LLM, and Why You Want Both
  7. 7 Planning Against the Model
About this series

Learning the Heater From Its Own Logs

Learning the Heater From Its Own Logs

The first code post in this series had a predict function whose body I typed by hand. Heat leaks towards outside at a tenth of the difference per step, the heater adds a degree and a half, and I made both of those numbers up. That was fine for showing the shape of a world model. It is useless for a real room, because a real room does not tell you its constants. What it gives you instead is a log.

This post throws the hand-written body away and fits a new one from logged state and action pairs. The model is four numbers. The training loop is twenty lines. At the end it predicts a room it has never seen more closely than my guess did, and you can run the whole thing in your browser.

A room that keeps its constants to itself

I don't have a thermostat wired up to this blog, so a simulator stands in for the hardware. It has the same shape as the 101 predict, with two changes: the constants are different from the ones I guessed, and there is a little noise, because someone keeps opening the door.

const REAL_LEAK = 0.07;   // better insulated than I guessed
const REAL_HEAT = 1.2;    // and a weaker heater
const NOISE = 0.1;        // draughts, in °C per step

function realRoom(state: State, action: Action): State {
  const heaterOn =
    action === "turnHeaterOn" ? true :
    action === "turnHeaterOff" ? false :
    state.heaterOn;
  const leak = (state.outsideTemp - state.roomTemp) * REAL_LEAK;
  const heat = heaterOn ? REAL_HEAT : 0;
  const draught = gauss() * NOISE;
  return {
    roomTemp: state.roomTemp + leak + heat + draught,
    outsideTemp: state.outsideTemp,
    heaterOn,
  };
}

The rule for the rest of the post is that nothing below this line gets to look inside realRoom. The model only sees what a logger would see.

Recording the logs

The log comes from running a thermostat against the room. Each episode picks an outside temperature between 2 and 12 degrees, a starting room temperature between 10 and 22, a setpoint between 18 and 21, and runs sixty steps. One step in five the thermostat does something random instead, because a log with only sensible decisions in it never shows the model what a daft decision does. Eighty episodes gives 4,800 transitions, each one the Transition type from the 101 post.

function record(episodes: number, steps: number): Transition[] {
  const log: Transition[] = [];
  for (let e = 0; e < episodes; e++) {
    let state: State = {
      roomTemp: 10 + rand() * 12,
      outsideTemp: 2 + rand() * 10,
      heaterOn: rand() < 0.5,
    };
    const setpoint = 18 + rand() * 3;
    for (let t = 0; t < steps; t++) {
      const action = thermostat(state, setpoint);
      const nextState = realRoom(state, action);
      log.push({ state, action, nextState });
      state = nextState;
    }
  }
  return log;
}

const log = record(80, 60);

Here are the first two rows, which is all of the data the model ever gets, four thousand eight hundred times over.

{ state: { roomTemp: 16.51, outsideTemp: 7.62, heaterOn: false },
  action: "turnHeaterOn",
  nextState: { roomTemp: 17.09, outsideTemp: 7.62, heaterOn: true } }
{ state: { roomTemp: 17.09, outsideTemp: 7.62, heaterOn: true },
  action: "wait",
  nextState: { roomTemp: 17.75, outsideTemp: 7.62, heaterOn: true } }

If you run energy hardware, this is your telemetry table with the commands joined on, which was the whole point of the first post.

The WHERE clause runs out

The last post computed a world model with a filter and a count: keep the rows where the state and action matched, tally what happened next. It worked because temperatures were bucketed into cold, mild and warm. Try it on the real log and it falls over immediately.

const probe = log[1234];
const matches = log.filter(t =>
  t.state.roomTemp === probe.state.roomTemp &&
  t.state.outsideTemp === probe.state.outsideTemp &&
  t.state.heaterOn === probe.state.heaterOn &&
  t.action === probe.action);

matches.length;  // 1, and it's row 1234 itself

Loosening the match to within a twentieth of a degree finds two rows. A continuous state almost never recurs, so the condition on the right of the bar almost never matches, which is the same wall the LLM equation hit in that post. The fix there was a very large neural network. The fix here is a very small one, because the room is a very small world.

The model is four numbers

What I want is a function with the predict signature whose body I did not write. The simplest body that could work is a weighted sum. Take the readings the logger can see, multiply each by a weight, add them up, and call the result the change in room temperature for this step.

type Weights = { room: number; outside: number; heater: number; bias: number };

function features(state: State, action: Action) {
  const heaterOn =
    action === "turnHeaterOn" ? true :
    action === "turnHeaterOff" ? false :
    state.heaterOn;
  return { room: state.roomTemp, outside: state.outsideTemp, heater: heaterOn ? 1 : 0, bias: 1, heaterOn };
}

function predictWith(w: Weights, state: State, action: Action): State {
  const f = features(state, action);
  const delta = w.room * f.room + w.outside * f.outside + w.heater * f.heater + w.bias * f.bias;
  return { roomTemp: state.roomTemp + delta, outsideTemp: state.outsideTemp, heaterOn: f.heaterOn };
}

Predicting the change rather than the next temperature outright is the one design decision in here, and it matters more than it looks. The next temperature is dominated by the current temperature, so a model asked to predict it spends its capacity learning “next is roughly the same as now” and has little left for the interesting part. The change is the interesting part on its own.

Notice what the model is not given. It does not know that the leak depends on the difference between inside and outside. It gets the two temperatures as separate inputs and has to discover the relationship, if there is one.

The training loop

Training means finding the four weights that make predictWith agree with the log. Gradient descent does that the slow and honest way: start at zero, measure how wrong the model is across every row, nudge each weight in the direction that reduces the error, repeat.

function train(log: Transition[], epochs: number, lr: number) {
  const w: Weights = { room: 0, outside: 0, heater: 0, bias: 0 };
  const losses: number[] = [];
  for (let epoch = 0; epoch < epochs; epoch++) {
    const g: Weights = { room: 0, outside: 0, heater: 0, bias: 0 };
    let loss = 0;
    for (const t of log) {
      const f = features(t.state, t.action);
      const target = t.nextState.roomTemp - t.state.roomTemp;
      const guess = w.room * f.room + w.outside * f.outside + w.heater * f.heater + w.bias * f.bias;
      const err = guess - target;
      loss += err * err;
      g.room += err * f.room; g.outside += err * f.outside;
      g.heater += err * f.heater; g.bias += err * f.bias;
    }
    const n = log.length;
    w.room -= lr * g.room / n; w.outside -= lr * g.outside / n;
    w.heater -= lr * g.heater / n; w.bias -= lr * g.bias / n;
    losses.push(loss / n);
  }
  return { w, losses };
}

const { w, losses } = train(log, 10000, 0.003);

The err * f.room lines are the entire derivative. For a weighted sum with a squared error, the gradient of the loss with respect to each weight is the error times that weight's input, which is why no library is needed. Ten thousand passes over 4,800 rows takes about 300 milliseconds in Node 23.

Mean squared error during training, ten thousand epochsOne line falling from 0.30 at the first epoch to 0.10 by epoch one thousand and flattening at 0.01, the level of the noise in the logs, from about epoch five thousand.00.10.20.302,0004,0006,0008,00010,000epochnoise floor, 0.01epoch 1: 0.2958epoch 1001: 0.0971epoch 5001: 0.0109epoch 10000: 0.01010.300.010
Mean squared error per epoch. It flattens at 0.01, which is the variance of the draughts: the model has learned everything in the log that is not noise.Download the data (CSV)

Ten thousand epochs is a lot for four weights, and the reason is worth knowing. Room temperature and outside temperature move together in the log (warm days make warm rooms), so the loss surface is a long thin valley and plain gradient descent zigzags down it. Scaling the inputs or using a smarter optimiser would fix that in a few hundred epochs. I left it slow because the loop is easier to read this way, and because the playground is more fun when you can watch it converge.

Here are the weights it found, next to the constants realRoom was hiding.

weight learned truth
room -0.074 -0.07
outside 0.069 0.07
heater 1.19 1.2
bias 0.077 0

The room and outside weights came out equal and opposite. Nobody told the model that heat flow depends on the difference between the two temperatures. It found that from the logs, which is the small version of the thing world models are for. The bias absorbed a little of the room weight's share, because the two inputs are correlated and the noise lets them trade off. That is what a fit looks like when the data is real rather than typed in.

And predict now has a learned body with the same signature as before.

const predict = (state: State, action: Action) => predictWith(w, state, action);

A room it has never seen

The test that matters is a room outside the training range. Every episode in the log had an outside temperature between 2 and 12 degrees. The test room is at minus four, starts at 22, and follows a script the thermostat never produced: heater off for ten steps, on for ten, off for ten. The same script runs through three functions: the real room, the learned model, and the hand-written guess from 101.

const unseen: State = { roomTemp: 22, outsideTemp: -4, heaterOn: false };

// rollout from 101, with the step function passed in instead of baked in
const truth = rollout(unseen, script, realRoom);
const learned = rollout(unseen, script, predict);
const guessed = rollout(unseen, script, predictGuess);
Thirty steps in a room the model never saw: outside at minus four, starting at 22 degreesThree lines. The real room and the learned model fall together from 22 to about 9 degrees over ten steps with the heater off, climb to 11 over ten steps with it on, then fall to about 4. The hand-written guess drops much faster, reaching 5 by step ten and under 1 by step thirty.heater on0°4°8°12°16°20°24°051015202530stepReal room, step 0: 22.0°CReal room, step 1: 20.3°CReal room, step 2: 18.7°CReal room, step 3: 17.1°CReal room, step 4: 15.6°CReal room, step 5: 14.3°CReal room, step 6: 12.9°CReal room, step 7: 11.7°CReal room, step 8: 10.7°CReal room, step 9: 9.7°CReal room, step 10: 8.8°CReal room, step 11: 9.1°CReal room, step 12: 9.4°CReal room, step 13: 9.7°CReal room, step 14: 9.9°CReal room, step 15: 10.0°CReal room, step 16: 10.3°CReal room, step 17: 10.3°CReal room, step 18: 10.7°CReal room, step 19: 10.9°CReal room, step 20: 11.0°CReal room, step 21: 10.0°CReal room, step 22: 9.0°CReal room, step 23: 8.2°CReal room, step 24: 7.4°CReal room, step 25: 6.7°CReal room, step 26: 6.0°CReal room, step 27: 5.2°CReal room, step 28: 4.5°CReal room, step 29: 4.1°CReal room, step 30: 3.7°CLearned model, step 0: 22.0°CLearned model, step 1: 20.2°CLearned model, step 2: 18.5°CLearned model, step 3: 16.9°CLearned model, step 4: 15.5°CLearned model, step 5: 14.2°CLearned model, step 6: 12.9°CLearned model, step 7: 11.8°CLearned model, step 8: 10.7°CLearned model, step 9: 9.7°CLearned model, step 10: 8.8°CLearned model, step 11: 9.1°CLearned model, step 12: 9.4°CLearned model, step 13: 9.8°CLearned model, step 14: 10.0°CLearned model, step 15: 10.3°CLearned model, step 16: 10.5°CLearned model, step 17: 10.7°CLearned model, step 18: 10.9°CLearned model, step 19: 11.1°CLearned model, step 20: 11.3°CLearned model, step 21: 10.2°CLearned model, step 22: 9.3°CLearned model, step 23: 8.4°CLearned model, step 24: 7.6°CLearned model, step 25: 6.8°CLearned model, step 26: 6.1°CLearned model, step 27: 5.5°CLearned model, step 28: 4.9°CLearned model, step 29: 4.3°CLearned model, step 30: 3.8°CHand-written guess, step 0: 22.0°CHand-written guess, step 1: 19.4°CHand-written guess, step 2: 17.1°CHand-written guess, step 3: 14.9°CHand-written guess, step 4: 13.1°CHand-written guess, step 5: 11.3°CHand-written guess, step 6: 9.8°CHand-written guess, step 7: 8.4°CHand-written guess, step 8: 7.2°CHand-written guess, step 9: 6.1°CHand-written guess, step 10: 5.1°CHand-written guess, step 11: 5.7°CHand-written guess, step 12: 6.2°CHand-written guess, step 13: 6.7°CHand-written guess, step 14: 7.1°CHand-written guess, step 15: 7.5°CHand-written guess, step 16: 7.8°CHand-written guess, step 17: 8.2°CHand-written guess, step 18: 8.4°CHand-written guess, step 19: 8.7°CHand-written guess, step 20: 8.9°CHand-written guess, step 21: 7.6°CHand-written guess, step 22: 6.5°CHand-written guess, step 23: 5.4°CHand-written guess, step 24: 4.5°CHand-written guess, step 25: 3.6°CHand-written guess, step 26: 2.9°CHand-written guess, step 27: 2.2°CHand-written guess, step 28: 1.6°CHand-written guess, step 29: 1.0°CHand-written guess, step 30: 0.5°Clearned 3.8°real 3.7°guess 0.5°Real roomLearned modelHand-written guess
Thirty steps at minus four outside. The learned model tracks the real room to within a few tenths of a degree the whole way. The hand-written guess leaks too fast and heats too hard, and is three degrees adrift by step thirty.Download the data (CSV)

The numbers at the three turning points:

step real room learned hand-written
10, heater goes on 8.78 8.79 5.07
20, heater goes off 11.02 11.28 8.93
30 3.69 3.80 0.51

Root mean square error over the thirty steps is 0.20 degrees for the learned model and 2.77 for my guess. The model extrapolated to an outside temperature six degrees colder than anything in its logs, because what it learned was the physics rather than the trajectories. A lookup table could not have done that, and neither could I, since I had the constants wrong.

What four weights cannot hold

The playground at /heater/ runs all of the above. Change the room's constants, record fresh logs, press train, and watch the weights settle while the prediction on the right pulls into line with the real room. Turn the draughts up and the loss floor rises to match. Set the leak rate to 0.10 and the heater to 1.5 and the hand-written guess becomes correct, which is a nice reminder that my guess was only ever a model with its weights typed in.

It also shows where this model stops. The weighted sum can only learn relationships that are a straight line through the inputs it can see. The room I built is linear and fully observed, so four weights are enough. Give the real room a door that opens and closes on a schedule the logger never records, and the best a linear model can do is average over the door, predicting a room that is always half open. The field it needs is one that is not in the log. The 102 post called that a private field, and the next post gives the model one and watches it work out the door on its own.

Update, November 2026: that post is Latent State Is the Fields You Never Logged, and the playground now has the room with the door in it.