UEMA Specification v3.5

0Source and reading guide

Four kinds of boxes

The specification mixes design text with notes about the code that existed when it was written. This page keeps the two apart so a reader can tell a requirement from a record:

Reading guide

2Core thesis and dual objective source §1–2

2.1 Why a policy network is not enough

A policy network learns one mapping, observation → action. The source states that this collapses five distinct cognitive stages into one mapping:

"I perceive → I predict → I feel myself → I intend → I act"

Requirement: each of the five stages must be a meaningful internal process with its own computational substrate. A policy network computes f(observation) = action and has no internal state representing why an action works; the source calls this "the Zombie Problem" ("There is nothing it is LIKE to be this system").

No permanent policy exceptions. The thesis applies to the entire system. There is no permanent reflex network trained by reinforcement learning (RL), and no bootstrap scaffold: a temporary RL policy whose commands are blended in with a weight that decays over training. Every motor command flows through the prediction-wellness architecture from the start; section 8.5 gives the source's account of how learning starts without a scaffold.

2.2 The dual objective

Prediction error

Trains the forward model (the source also calls it the world model), which minimises prediction error on sensorimotor experience. The source's reason: "Prediction is compression; to predict well you must understand the underlying process."

Wellness gradient

Drives action selection. The wellness evaluator maps the current sensory state to a multi-dimensional wellness vector; its gradient with respect to the command's numbers is the motor controller's input (7.4).

The source holds that every other behaviour follows from this dual objective, with no external curriculum, no task-specific objective and no permanent reinforcement-learning objective anywhere (the field's term for that externally defined score is reward):

3The body: senses and handles source §3

The source requires that every concept maps to measurable quantities. This section lists the sensors and the motor interface of the specified avatar. The engine-specific implementation passages of the source are not reproduced (section 21); the measured quantities, units and rates are.

3.1 Tactile sensing

Biological mapping: slowly adapting type I receptors (SA-I) → normal pressure; rapidly adapting type I (RA-I) → shear magnitude and slip rate; rapidly adapting type II (RA-II) → vibration power at 200 Hz and 400 Hz. The vibration channels are frequency-domain features computed by a Fourier transform over a rolling window of the contact-force history, not over the simulation timestep.

3.2 Vision (foveated)

Each patch carries its angular position (azimuth, elevation), not its pixel position (x, y). The encoder is a Swin Transformer, a transformer that computes attention within local windows of patches that shift from layer to layer (section 5.1). One frame of vision is 48 MB.

3.3 Audio

Binaural stereo sound (PCM) recorded at the two ear locations, with the interaural time difference (ITD, in μs) and interaural level difference (ILD, in dB). Voice is synthesised with the Pink Trombone vocal synthesiser. A self-voice coherence value lies in [0, 1]; the source does not define how it is computed. Loudness: 30–60 dB is comfortable; 75–90 dB gives negative wellness.

3.4 Motor control through handles

The body is a three-layer physics stack:

1. A rigid-body skeleton: mass, inertia, joint limits.
2. A postural tone layer: baseline stiffness that holds posture against gravity.
3. Handles: the motor commands the learning program sets.

Final motion (per frame) = gravity + postural tone + handle forces + tactile contact forces

Why the scaling: per-region stiffness gives the same apparent compliance across body segments of different mass.

Effort. The resistance of the postural tone layer is the proprioceptive effort signal. Handles overcome postural tone, gravity and contact forces; that total resistance is "effort".

E_total = Σ |F · v| · dt + λ · Σ PosturalToneStrength · dt

E_total is the energy the body spends (the source's label is "Energy"). F is a handle force, v the velocity of the body part it moves, dt one frame, and λ a weight the source does not give a value for. Fatigue: capacity depletes under load, recovers at rest, and limits the effective strength of the postural tone.

The released MIMo body has different sensors and 24 or 26 handles, depending on its body layout; section 19 lists the differences that are visible in the released code.

5Encoders source §5

Each sense has its own encoder, which compresses its raw data into a token of d_model = 768 numbers. Requirements: every encoder computes at d_model width internally (768-dimension attention, 768-dimension MLPs) and is trained jointly with the forward model. There is no intermediate bottleneck dimension; an encoder discards information only at its final pooling step.

Why cross-patch attention: in the tactile encoder it captures spatial relationships between patches ("4 fingers gripping, 1 slipping").

5.7 Preventing encoder collapse

An encoder collapses when it outputs nearly the same token for different inputs, which makes prediction trivially easy and useless. The source names two signals against it: (1) prediction utility, because the forward model needs distinguishable representations; (2) an InfoNCE contrastive loss, which requires different inputs to produce different embeddings, temperature 0.07.

7Wellness source §7–8

7.1 What wellness is

Wellness is a region of the possible grounded-register states P_t where four things hold: predictions succeed (low prediction error); sensors function (tactile sensing active, fovea tracking, audio in range); the body has capacity (muscle not depleted); and the self-model is coherent (high coherence, section 10.1). The source states that wellness is a multi-dimensional state, not one number judged against a line such as 0.7; two evaluators compute it, one for each register.

7.2 Grounded wellness evaluator (execution)

It reads the raw sensor data P_t. It is the authoritative wellness signal: all learning signals are ultimately derived from it.

The evaluator returns the four components stacked into one wellness vector, and the components by name.

7.3 Approximate wellness evaluator (imagination)

Input

The five tokens other than the motor token, from the encoders or predicted by the forward model, flattened to 5 × 768 = 3,840 numbers.

Computation

Linear 3,840 → 256, ReLU, Linear 256 → 128, ReLU, Linear 128 → 4, sigmoid.

Output

Four wellness components, each 0 to 1. Trained during execution to match the grounded wellness evaluator: loss = ‖W_approx(z_t) − W_grounded(P_t)‖².

7.4 From wellness gradient to motor command

The source calls the wellness gradient "the core motor drive signal". During execution:

Input

P_t with its motor-command numbers marked for differentiation; the prediction error.

Computation

Run the grounded wellness evaluator; sum the wellness vector; take the gradient of the sum with respect to the motor-command numbers (with the graph kept, so that training can differentiate through it again). The result says, for each command number, which direction raises wellness.

Output

The gradient, passed to the motor controller, which turns it into structured handle commands.

With Many Minds active, each mind calls this computation independently with the system's wellness sum.

7.5 Pain and pleasure as trajectories source §8

9Intention as differentiable model inversion source §10

9.1 From forward model to intention

During exploration the forward model learns action → consequence. Intention inverts it: desired consequence → action.

9.2 The planner search on the structured action space

Input

The current encoded state; a desired encoded state; the current command [n_handles, 9], used as the starting point.

Computation

Copy the command's position (3), rotation (4), radius (1) and strength (1) as free variables. Run n_steps = 50 steps of the Adam optimiser (gradient descent with a step size adapted per number from running averages of its gradients), learning rate 0.1. At each step: normalise the rotation to a unit quaternion; bound radius and strength with a sigmoid; encode the action with the motor encoder; let the forward model predict the next state from the current state and the encoded action (predict_from_action); loss = mean squared error(predicted, desired) + 0.01 × Σ strength²; take a gradient step on the free variables. The forward model's own weights are not changed.

Output

The optimised command: position, unit rotation, sigmoid(radius), sigmoid(strength).

The strength term makes weaker commands cheaper, so among commands whose predicted tokens match the desired tokens equally well, the one with less strength has the lower loss.

9.3 Multi-step intention by imagination rollout

Input

Current state, goal state, max_steps = 5.

Computation

Repeat up to 5 times: stop if the mean squared error between the imagined state and the goal is below 0.01; otherwise run the planner search from the imagined state toward the goal, append the action to the plan, and replace the imagined state by the forward model's prediction for that action; stop if the approximate wellness evaluator's lowest component for the imagined state is below 0.2 (the imagined trajectory leads to unwellness).

Output

A plan: a list of up to 5 commands.

Why plans stay short: imagination drifts (section 6.3). The source draws the consequence: plan a few steps, execute, re-observe in the grounded register, plan again.

9.4 Cross-simulation attention

Input

Current state; goal; noise 0.1.

Computation

Make n_rollouts = 5 multi-step plans for the same goal; rollouts 2 to 5 use the goal plus Gaussian noise of scale 0.1, rollout 1 the exact goal. Encode each plan's actions and stack them. Use the mean over rollouts as the query of a multi-head attention (4 heads) over all rollouts' actions.

Output

A consensus action sequence and the attention weights.

Why: what all rollouts agree on is kept and divergent details average out; in the source's words, "intention crystallizes as the common element".

9.5 Visual imagination module (future)

10Self and body source §11

10.1 The self as the motor-correlation boundary

The self is not defined by skeleton or sensor layout. It is defined by what correlates with the motor commands. The boundary is dynamic and its membership values are differentiable.

Input

Each frame: the change of the motor command and the change of every sensor dimension.

Computation

Correlation score per sensor dimension, a running average outside the gradient path: score ← 0.99 × score + 0.01 × (instantaneous correlation of motor change and sensor change; the source does not define how this one-frame correlation is computed).
Membership = sigmoid(5.0 × (score − 0.5)): above 0.5 for a dimension whose score is above 0.5 (self), below for one whose score is below (world). The temperature 5.0 sets how sharp the boundary is.
Coherence = mean over dimensions of |membership − 0.5| × 2: 1 when every dimension is clearly self or clearly world, 0 when every membership is 0.5.
Incorporation strength of a set of dimensions = mean membership over that set.

Output

Per-dimension membership in [0, 1]; the coherence score in [0, 1], an input of the grounded wellness evaluator (7.2); incorporation strengths.

10.2 Object incorporation

Grasping expands the self. Before a grasp a cup is world: its visual position is independent of the commands, correlation ≈ 0. During the grasp the cup follows the commands: correlation rises and so does its membership. On release the correlation drops and the self contracts. On a slip the contraction is partial, and the wellness gradient drives grip correction.

10.3 Slip detection as self-contraction

Mechanism: a slip lowers the incorporation strength of the cup's tactile contact dimensions; lower incorporation lowers coherence; lower coherence lowers grounded wellness; the gradient of grounded wellness with respect to the motor command therefore points toward restoring coherence, which means restoring grip and incorporation. The source's function slip_driven_correction (in the listing below) returns that gradient. The source states that this needs no grip-force controller and no external objective signal.

10.4 The hierarchical self-model

The boundary of 10.1 is the Level 1 self: what responds directly to motor commands. The source extends the self to slower timescales: a grasped tool is Level 1 self, but losing a house also shrinks the set of wellness states the system can reach, so the house is self in a meaningful sense. A hierarchical self-model tracks three levels with the same correlation machinery:

Input

Level 1: motor and sensor changes. Levels 2 and 3: for each of n_entity_slots = 64 entity slots, the change of the entity's presence (current − previous; the source does not define how entities are found or how presence is measured); the wellness change over the level's window (last − first value of the wellness trajectory).

Computation

Each level is a body schema of 10.1 with its own momentum. Full membership of a sensor or entity = the maximum of its membership over the three levels. Loss-of-self signal at a level (the source's loss_at_level; not a training loss) = 1 − that level's coherence.

Output

Memberships per level and overall; a loss signal per level.

Level 2 includes entities whose presence correlates with the ability to bring about wellness through action sequences: a house does not move with the hand, but having it correlates with reachable wellness states (shelter, storage, a base of operations). Level 3 extends to multi-step causal chains (money → resource access → wellness maintenance); correlation strength is proportional to the reliability of the chain.

The source states that this produces the right behaviour without programming it: the system is not instructed to protect a house or to be possessive; certain states correlate with sustained wellness and the gradient maintains them. Weak or unreliable chains give weak incorporation: ten houses when one is needed give diminishing correlation.

Connection to Many Minds (section 12). When a mind splits, the subordinate inherits the domain-specific self at Levels 1 and 2 (the walking subordinate's Level 2 includes familiar terrain; the grasping subordinate's includes known tools). The higher mind keeps the full hierarchy including Level 3, so the self-model is distributed across minds.

class DifferentiableBodySchema(nn.Module):
    def __init__(self, n_sensor_dims, temperature=5.0, ema_momentum=0.99):
        self.correlation_scores = nn.Parameter(
            torch.zeros(n_sensor_dims), requires_grad=False)
        self.temperature = temperature
        self.momentum = ema_momentum
    def update(self, motor_delta, sensor_delta):
        with torch.no_grad():
            instant_corr = self._compute_correlation(motor_delta, sensor_delta)
            self.correlation_scores.data = (
                self.momentum * self.correlation_scores.data +
                (1 - self.momentum) * instant_corr)
    def self_membership(self, sensor_dims=None):
        scores = self.correlation_scores[sensor_dims] if sensor_dims else self.correlation_scores
        return torch.sigmoid(self.temperature * (scores - 0.5))
    def coherence_score(self):
        membership = self.self_membership()
        certainty = (membership - 0.5).abs() * 2
        return certainty.mean()  # differentiable
    def incorporation_strength(self, sensor_dims):
        return self.self_membership(sensor_dims).mean()

def slip_driven_correction(body_schema, grounded_eval, P_t, pe):
    cup_dims = P_t.tactile.cup_contact_patch_indices
    incorporation = body_schema.incorporation_strength(cup_dims)
    # Incorporation drop = self-contraction = prediction error about self
    coherence = body_schema.coherence_score()
    wellness_grad = torch.autograd.grad(
        grounded_eval(P_t, pe)[0].sum(), P_t.motor_commands, create_graph=True)[0]
    # Gradient naturally drives grip tightening because:
    # lower coherence -> lower wellness -> gradient toward restoring coherence
    # -> restoring grip -> restoring incorporation
    return wellness_grad

class HierarchicalSelfModel(nn.Module):
    '''Multi-timescale self-model. Each level tracks correlation between entities
    and wellness maintenance at different time horizons.
    Level 1: Motor-correlation (body, grasped objects). ~100 frames.
    Level 2: Control-correlation (sustained wellness maintenance). ~10K frames.
    Level 3: Abstract-correlation (long-term wellness trajectory). ~100K frames.'''
    def __init__(self, n_sensor_dims, n_entity_slots=64):
        self.levels = nn.ModuleDict({
            'motor': DifferentiableBodySchema(n_sensor_dims, ema_momentum=0.99),
            'control': DifferentiableBodySchema(n_entity_slots, ema_momentum=0.9999),
            'abstract': DifferentiableBodySchema(n_entity_slots, ema_momentum=0.99999),
        })
    def update_level1(self, motor_delta, sensor_delta):
        '''Standard motor-correlation update (every frame).'''
        self.levels['motor'].update(motor_delta, sensor_delta)
    def update_level2(self, entity_states, wellness_trajectory):
        '''Control-correlation: does entity presence correlate with
        ability to maintain wellness over minutes?'''
        wellness_delta = wellness_trajectory[-1] - wellness_trajectory[0]
        for entity_id, state in entity_states.items():
            presence_delta = state.current - state.previous
            self.levels['control'].update(presence_delta, wellness_delta)
    def update_level3(self, entity_states, long_wellness):
        '''Abstract-correlation: does entity correlate with wellness over hours?'''
        wellness_delta = long_wellness[-1] - long_wellness[0]
        for entity_id, state in entity_states.items():
            presence_delta = state.current - state.previous
            self.levels['abstract'].update(presence_delta, wellness_delta)
    def full_self_membership(self, sensor_or_entity_dims):
        '''Returns max membership across all levels.'''
        return torch.stack([
            level.self_membership(sensor_or_entity_dims)
            for level in self.levels.values()
        ]).max(dim=0).values
    def loss_at_level(self, level_name):
        '''PE pattern differs by level:
        Level 1 loss (hand injury): immediate, sharp PE
        Level 2 loss (tool taken): delayed, moderate PE
        Level 3 loss (house burns): slow, deep PE, long resolution time'''
        return 1.0 - self.levels[level_name].coherence_score()

10.5 Self-preservation

The body model is the most complex and most refined prediction set. Damage (a limb stops responding, sensors go dark) produces the largest possible prediction error. The source derives self-preservation from this: the system acts to prevent body-model degradation because that is the most costly prediction-error event.

11Grounded anticipation source §12

Anticipation is not a label or an abstract pattern. It is the actual state of the motor and sensor systems when features predict an upcoming interaction, and it is measurable:

These measures become the supervision of a learned anticipation network: the system learns to anticipate, and the measures check that anticipation shows up as observable motor preparation.

Input

The six current tokens, concatenated (6 × 768).

Computation

Anticipation network: Linear 6 × 768 → 256, ReLU, Linear 256 → 128, ReLU, Linear 128 → 4 + n_handles × 8. Confidence gate: Linear 6 × 768 → 64, ReLU, Linear 64 → 1, sigmoid. The first 4 outputs are the predicted measures; the remaining outputs, reshaped per handle and multiplied by the confidence, are the motor preparation.

Output

Motor preparation per handle, the confidence, and the 4 predicted measures.

Training

Over a trajectory, with lookahead 30: for each frame t up to the trajectory length − 30, mean squared error between the 4 predicted measures at t and the measures computed at frame t+1; summed. The measure functions: gaze dwell time; the norm of the centre-of-mass change; 1 if the tactile sensing's iteration count exceeds its baseline; 1 if any handle has strength > 0 and velocity < 0.01.

The source states the purpose: the learned network generalises to novel situations, while the fixed measures verify that anticipation appears as real motor preparation.

14Curiosity and emergence source §21–22

14.1 Curiosity as moderate prediction error

Curiosity seeks states where prediction error is moderate: high enough to learn from, low enough to make progress. It is not "seek maximum prediction error" (chaos-seeking).

Input

One prediction-error value pe; the range (pe_min, pe_max) = (0.1, 0.4).

Computation

If pe < 0.1 (too easy, bored): drive = pe / 0.1 − 1, a negative number.
If pe > 0.4 (too hard, overwhelmed): drive = 1 − pe / 0.4, negative.
Otherwise (the "goldilocks zone"): drive = 1 − |pe − 0.25| / (0.4 − 0.1).

Output

The drive. The source states that the drive changes the direction of the wellness gradient (positive drive toward states of new prediction error, negative toward familiar or simpler states) and is not an external objective signal. It gives no formula for that change. In the code of its time the drive scaled the wellness loss (status below).

14.2 Emergent phenomena

Stress as sustained prediction error. Prediction error stays high across several senses without resolution: the forward model keeps failing, the exploration drive stays high (spending energy), and the motor controller finds no command that improves wellness. The distinction from curiosity: stress is elevated prediction error that resists reduction despite effort; curiosity is elevated prediction error that is being reduced, where each action reveals learnable structure.

Grief as unresolvable prediction error. A deeply reinforced prediction (an object on a shelf for thousands of timesteps) becomes permanently false (the object is removed). Sequence: (1) prediction error spikes; (2) exploration burst; (3) repeated failure to resolve; (4) energy depletion; (5) structural model update: objects can cease to exist; (6) gradual overwriting. Validation milestone: grief about abstract losses (capabilities, resources, relationships in the hierarchical self-model of section 10.4), not only object disappearance. Grief about a Level 3 entity would indicate that perception spaces of a higher level have formed.

Conditions for emergence. The architecture must preserve them: prediction error is allowed to persist (no clamping); the coupling of energy and exploration is kept; the forward model supports structural updates, not only parameter updates.

15The path to self-awareness source §23 and Summary

15.1 What self-awareness requires

The source holds that self-awareness is not binary and arises from: (1) a model of the world (the forward model); (2) a model of oneself as part of that world (the body schema as a subset); (3) the ability to predict one's own future states (the forward model includes the self); (4) the ability to distinguish self from other (the motor-correlation boundary); (5) the ability to model one's own prediction process (meta-cognition, emergent).

15.2 Four levels of meta-prediction

The source's statement about Level 4: "The hierarchy IS the self." Identity is the whole graph of minds, their relationships and their communication; asked what it is doing, the system's answer is distributed (posture mind keeps the body balanced, grasp mind holds the object, top mind plans the next action), as a person does not consciously control breathing or posture yet identifies both as "me".

15.3 The critical test

Input

A state; the forward model with three extra outputs (own prediction error, wellness gap, intermind state shifts); both evaluators; the hierarchy.

Computation

Predict z, the own prediction error, the wellness gap and the intermind states. Execute and observe. Actual prediction error = MSE(predicted z, actual z); actual gap = MSE(grounded wellness of the actual state, approximate wellness of the predicted z); read the actual intermind states.
PE accuracy = 1 − |predicted PE − actual PE| / (actual PE + 10⁻⁶).
Gap accuracy = 1 − |predicted gap − actual gap| / (actual gap + 10⁻⁶).
Intermind accuracy = a comparison of predicted and actual intermind states (the source does not define it further).

Output

The three accuracies. 1 means exact; 0 means an error as large as the actual value.

15.4 Emergence through architecture

Per the source, self-awareness is not programmed; it emerges when (1) the forward model predicts its own failures, (2) the body schema models the prediction process, (3) the hierarchy provides distributed self-knowledge, (4) enough experience teaches the limits of reliability.

15.5 Summary: from policy network to mind

At each stage something missing becomes present; the source calls the last two stages "the closest we can get to defining self-awareness computationally". Its closing statements, quoted:

"The system is not told it has a self. It discovers that some predictions correlate with its commands. The system is not told to have intentions. It discovers that inverting its forward model achieves desired consequences. The system is not told to be curious. It discovers that moderate PE is where learning happens. The system is not told to value resources. It discovers that certain entities correlate with sustained wellness across timescales, and acts to maintain them. The system is not told to fear death. It discovers its body model is its most valuable asset. The system is not told its imagination differs from experience. It discovers its approximate wellness evaluator is less reliable than its grounded [wellness] evaluator. The system is not told it has subordinate minds. It discovers it can predict its own inter-mind communication patterns. These are not metaphors. They are the computational facts of what this architecture does."

16Implementation specifications source §24–27

This section is the source's sizing and compute plan for the full-size model. The numbers are the source's projections of early 2026; section 19 gives the sizes the released MIMo trainer uses.

16.1 Hardware: Google Cloud G4 instances

MML uses Google Cloud G4 instances with NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs (generally available October 2025), replacing the RTX 4090 the specification was originally designed around.

CPU: AMD Turin (EPYC 9005). Interconnect: Google custom peer-to-peer over PCIe Gen 5; NVLink is not available. 48 vCPUs per GPU. Development target: g4-standard-96 (2 GPUs, 96 vCPUs, 360 GB RAM), GPU 0 for rendering and the tokenizer (the program that converts the simulator's raw sensor output into the fixed-shape number arrays of the grounded register P_t), GPU 1 for model training. Upgrade to g4-standard-192 only if d_model must grow beyond 768.

16.2 Corrected parameter counts

Erratum in the source: version 3.2 stated "total_forward_model_params ~27M". That estimate assumed standard single-attention transformer blocks; the three-stage blocks of section 6.1 give 173M forward-model parameters at d_model = 768.

Memory: mixed-precision Adam with gradient checkpointing needs 4.3 GB; one 96 GB GPU holds the model with 95% of its memory unused. More GPUs are needed for computation, not memory.

16.3 The vision encoder is 96% of the computation

16.4 Split-rate senses

Decision: the vision encoder runs at 24 Hz; all other encoders and the forward model run at 60 Hz. Reasons given: human visual cortex object recognition (V4/IT) runs at 20–30 Hz; film looks smooth at 24 FPS; the Swin scene understanding corresponds to V4/IT processing, and running it at 60 Hz "was never biologically motivated" and is 96% of the cost. Proprioception, tactile and motor benefit from 60 Hz: muscle spindle afferents fire at ~100 Hz, mechanoreceptors up to 300 Hz, and the motor control loop runs at 50–100 Hz.

The forward model reuses the latest vision token between vision updates. The source expects its temporal attention to learn that vision updates are slower, and its cross-modal bridge to weight tactile and proprioception more during fast-changing situations, which it calls more biologically faithful than one rate for all senses.

16.5 GPU allocation and real-time feasibility (g4-standard-96)

The two GPUs work in parallel: GPU 0 tokenizes frame N while GPU 1 trains on frame N−1. The transfer between them is 13 KB through pinned memory (< 0.1 ms).

16.6 Model dimensions

16.7 Training compute budget

At d_model = 768, vision at 24 Hz, one training GPU on g4-standard-96 (~$9/hour): 71 FPS sustained.

Stream 3 overlaps Stream 1 from step zero. The source's projection to first goal-directed behaviour: 5–20 days, $1,000–$4,000.

Near-term plan in the source (a $1,200 cloud credit expiring mid-February 2026): day 1, code the model from the specification (encoders, forward model, motor controller, wellness, training loop) and wire it to the tokenizer outputs; deliverable: the loop runs, the loss computes, gradients flow. Days 2–7: train at d_model = 768 on g4-standard-96 with vision at 24 Hz, Streams 1 and 3 together; deliverable: ~30M training steps, body-schema validation, prediction-loss curves. $1,200 at ~$9/hour is ~133 hours, ~5.5 days continuous. Scaling path after the credit: (1) stay on g4-standard-96 (768, 24 Hz vision, 71 FPS), ~$6,500/month; (2) if prediction quality needs more capacity, g4-standard-192 with 3 training GPUs, d_model = 1024 (~480M parameters) at 30 Hz vision; (3) for parallel experiments, two g4-standard-96 instances with FedAvg, the cost of one g4-standard-192, with cleaner isolation for A/B comparison.

16.8 Loss functions

total = 1.0·prediction + 0.1·contrastive + 0.5·imagination + 0.01·latent_reg + 0.3·approx_wellness + 0.2·anticipation

No habitual distillation loss: skill consolidation is done by Many Minds splitting, a structural operation (the source writes "cloning + freezing weights"), not a loss term.

16.9 Differentiability

The four components off the gradient path (developmental constraints, body schema update, mind splitting, motor arbitration) are outside it by design: they are structural or scheduling operations, not learned transforms.

17Open questions and research program source §28

The source lists these as open:

18Deviations and priorities source §29

The source's last section reconciles the specification with the codebase of its time (branch UEMA-feature, configuration version 2.1); its stated rule is that the code is the source of truth and this section names where it departs from the specification's intent. The codebase calls the system LEM (Learning Embodiment Model) in configuration comments and log prefixes; the specification calls it UEMA; they are the same system.

The notes disagree by date. Source §29 and several in-section notes predate the notes marked "UPDATED 2026-06". The later notes record context length 8 (against CF-1's 1), VICReg enabled (against CF-4), the wellness gradient wired (against CF-3), and imagination training and the latent regulariser enabled (against CF-8 and CF-11). The configuration file in the release, lem/master_config.yaml, reads d_model: 768, context_length: 8, vicreg: enabled: true, d_internal: null (meaning "d_model"), imagination: enabled: true and latent_reg: enabled: true, which agrees with the later notes.

18.1 Code improvements the source adopts as canonical

• 23-handle full-body motor control: extends the earlier 12 hand-only handles to head, arms, legs and feet, with per-region stiffness and data-driven handle definitions, so the embodied-cognition thesis covers the whole body (3.4).
• Gaze head (6.4): active gaze output, loss weight 0.1.
• Loss weights by data origin (code: source_scales): real sensor data 1.0, synthetic placeholder data 0.0, stale data (older than 50 ms) 0.3, per-sense loss weights and ramps; prevents learning from placeholder noise; needed for synthetic-to-real transitions.
• VICReg as the collapse-prevention method (5.7).
• Real-time infrastructure: an asynchronous ring-buffer reader (ring size 3, frames older than 50 ms marked stale), torch.compile with CUDA graphs (reduce-overhead mode), a configurable skip of backward passes (backward_every_n_frames), a voice input stub (56 numbers, disabled; not one of the six senses).

18.2 Priorities and frame-rate impact

The source: four configuration changes (CF-1, CF-2, CF-4, CF-5) unlock most of the model's capability with zero code changes and should be applied before any further training run.

Frame rate: the kernel-dispatch bottleneck makes most P0 changes safe under CUDA graphs. 768 is benchmarked free (512/768/1024 identical FPS); the source calls the cost of enabling VICReg negligible and gives no number; a context of 8–16 is likely safe (to be benchmarked at 32). Removing d_internal raises encoder parameters from ~15M to ~55M, which the source expects the idle capacity of the target GPU to absorb (4.3 GB of 96 GB in use). The vision encoder at 768 and 24 Hz fits the budget: 945 GFLOPs × 24 calls/s = 22.7 TFLOP/s of the 88 TFLOP/s available. All P0 changes together fit on one training GPU.

20Glossary

21About this edition

What this edition changed relative to the source

• Added: the setting and part definitions (section 1); the reading guide; Figures 1 and 4 to 7, drawn for this edition; Figures 2 and 3, redrawn from perceptionSpace-4.docx with the originals linked; input/computation/output blocks for every component; companion notes; section 19 (released code); this glossary.
• Reordered: the source's sections keep their sequence, except that the Summary, which follows source §28, is page 15.5. Section numbers on this page differ (each heading gives the source's number). The pain and pleasure walkthroughs (source §8) are one table in 7.5.
• Merged repeated passages: the neonate limits (2% strength, 5% ROM), stated in source §1, §9.4, §9.5, §19.2 and §20.2, are given in full in 8.4 and referred to elsewhere; the d_internal paragraph of source §5.1 and §5.2 and its repetition as CF-5 and in §29.6 are given in full once, in section 18; the gaze head (§6.4 and §29.4) and the 23 handles (§3.4 and §29.4) are described once, with the adoption noted; the in-section status notes and the CF items they repeat are set side by side in section 18; the second paragraph of source §23.5 repeats the Summary's closing statements and is carried by the quotation in 15.5; the source's book-summary analogy in §4.2 is cut because the bullets of 4.2 carry each of its facts.
• Code listings: kept in collapsible blocks for sections 6, 8, 10. Short listings (sections 7, 9, 11, 12, 13, 14, 15) are given as input/computation/output text with every number.
• Removed (engine-specific): passages describing the game-engine avatar's own implementation: implementation document names, the soft-body tactile simulation's mesh and solver settings, engine C++ structures and shared-memory buffers, the engine rendering and capture lines of the GPU plan, an editor widget, and section 20.6 of the text export. Body-independent requirements those passages served are kept (sections 3, 13.4, 16.5, 18.1).
• Renamed for one name per thing: the source's IntentionGenerator is called planner search; the source's abbreviations for the postural tone layer and the handle are written out.

Points in the source a reader may stumble on

• Numbers per handle in a command: 9 in source §4.1 and §10.2; 8 in §5.5, §9.2–9.3 and §12.2.
• The sense sequence differs: "V,A,T,P,M,I" in source §4.2 and §6, vision-tactile-audio in §16.3.
• Source §5 requires no encoder bottleneck, while its tactile, audio and motor encoders list 64- and 128-wide hidden layers.
• In the adaptive motor network (source §9.3) the residual sum adds a 512-number layer output to a wider input.
• The approximate wellness evaluator reads "5 sensory modality tokens" with "no motor token" (source §7.3), while the Stream 4 listing (§20.5) passes it the first five tokens, which in the sequence of §4.2 include the motor token and exclude intermind.
• The pain walkthrough uses a thermal reading that the tactile field list of source §3.1 does not include.
• The anticipation training loss is declared with lookahead 30 but compares with frame t+1.
• Implementation-status notes of different dates disagree (section 18).
• The grounded wellness evaluator's four inputs (source §7.2) include no command number, yet §7.4 takes its gradient with respect to the command numbers. §9.5 says it assesses joint limits and posture quality, which none of the four inputs carries.
• Level 3 of the hierarchical self is described as hours, but its running-average momentum of 0.99999 spans about 100,000 frames, 28 minutes at 60 frames per second.

References in the source

• DreamZero: NVIDIA GEAR Lab, February 2026. https://dreamzero0.github.io/DreamZero.pdf
• HRM: Wang et al., June 2025. arXiv:2506.21734
• TRM: Jolicoeur-Martineau, October 2025. arXiv:2510.04871
• Compute analysis scripts named by the source and not in the release: uema_compute_analysis.py, split_rate_analysis.py, training_timeline.py
• Companion references: patent US 11,537,850 B2; Bengio, Courville and Vincent 2013.

Edition prepared October 25, 2026 from UEMA_Spec_v3_5.docx, UEMA_Spec_v3_5.txt, perceptionSpace-4.docx and fastGainOfFunction-4.docx.