Many Minds Architecture
UEMA (Unified Embodied Mind Architecture) is a learning program that controls a simulated body. At every model step (one update of the program; 50 ms in the released MIMo code) it reads the body's senses, predicts what they will read at the next model step, scores the body's state with a small set of numbers called wellness channels (for example, how comfortable the touch on the skin is, or how accurate the predictions were), and outputs a command for the body's handles, the controllable targets such as a wrist or a foot. The published body is MIMo, a simulated infant moved by the MuJoCo physics program. The single-mind UEMA specification describes one such program. This specification, Many Minds, describes how that one program grows into a hierarchy of several copies, called minds: when the program has mastered one region of the body, all of its weights are copied into a new mind that takes over that region's handles, and the original mind's capacity is freed for new learning. This page states what each mind is, what the minds share, how their commands are combined into one command for the body, when and how a mind splits, and what the source leaves open.
| Source | ManyMinds_Spec_v2.md: "Many Minds Architecture — Specification v0.2 (DRAFT)" |
| Version | 0.2, draft |
| Status | "Working draft — under active discussion" |
| Date | March 2026 |
| Amends | UEMA_Spec_v3_4, Sections 13–18 |
| Supersedes | ManyMinds_Spec_v1.md |
| Companion | "Many Minds Gap Analysis — Changes for ManyMinds_Spec_v3 and UEMA_Spec_v5", status "In progress", March 2026; the later of its two copies is used (section 14). Its findings appear on this page in boxes labelled "Gap analysis". |
| Implementation | None of the Many Minds hierarchy exists in the released code; a single mind is what runs (section 13). |
| This edition | October 25, 2026. Reordered and drawn for first-time readers; every requirement, number and formula of the source is kept, except passages specific to a body that is not part of this release (section 16). |
1Where Many Minds sits in the single-mind UEMA architecture
The single-mind UEMA specification (UEMA_Spec_v3_5, Version 3.5, header date February 2026, status "Working Specification") defines one program: encoders, a forward model, prediction heads, wellness evaluators, a motor controller, intention by model inversion, a body schema and a training pipeline. The Many Minds source states that "every mind in the hierarchy is a complete instance of the UEMA core loop", so a single mind is the hierarchy with one member, and every Many Minds statement is a statement about several copies of that loop.
The Many Minds source amends Part VIII of UEMA Spec v3.4 (Sections 13–18, "Many Minds Architecture"). UEMA Spec v3.5 already carries those amendments, and cites "ManyMinds_Spec v2" and its gap resolutions by name (v3.5 Part VIII, §5.6, §7.4, §11.4, §20.5, §23.3 and §29.3). Its §18 records the hierarchy as "NOT IMPLEMENTED".
2The words this page uses
These terms appear from the next section on. Each is given once here; the glossary in section 15 repeats them with every other term.
| Term | Meaning on this page |
|---|---|
| model step | One update of the core loop: senses read, encoded, predicted, scored, and a command produced. The source calls it a "frame" and targets 60 per second (vision at 24 per second). In the released MIMo code a model step is 50 ms. |
| handle | One controllable target on the body, such as a wrist, a finger or a foot. The source writes "PH handle" (physics handle). |
| command | For each handle, 9 numbers: target position (3), target rotation as a quaternion (4), radius (1) and strength (1). Radius, from 0 to 1, sets how far one handle's command spreads to nearby joints; strength, from 0 to 1, sets how hard it pulls. |
| sense and token | A sense (the source says "modality") is one input stream: vision, audio, tactile (skin contact), proprioception (the body's own joint positions and forces), motor (the command being executed) and intermind (a token built from the other minds' internal state, section 5.3). An encoder turns one sense's raw numbers into a token, a vector of d_model numbers (768 in the source). |
| prediction error (PE) | The difference between the tokens the forward model predicted for this model step and the tokens the encoders produced. |
| wellness channel | One term of the wellness sum. The source has five, computed from raw sensor data (section 4.4). |
| gap | The source's name for one open design question, lettered A to K. "Resolved" means the source specifies a mechanism for it, not that the mechanism is built (section 10 lists all eleven). |
| mind | One complete copy of the single-mind core loop (section 4) with its own weights, working over a set of handles, its domain. |
| split, parent, child | A split copies a mind (the parent) into a new mind (the child) whose domain is a set of handles the parent had mastered (their PE stayed below 0.05 and steady over a 50,000-step window, section 7.2). The child starts as a deepcopy: a complete copy of every weight of the parent (Python's copy.deepcopy). |
| top mind | The mind at the top of the grown hierarchy: no handles, no command, the broadest scope and the longest imagination chains (section 9). |
| attending mind | The mind directly under the top mind: it coordinates the body in the present situation and holds the handles no child has taken (section 9). |
3Design principles
The source: "The Many Minds architecture must follow the same philosophy as the rest of UEMA: capabilities emerge from prediction and wellness gradients, not from programmed protocols." Specifically:
- No hardcoded message types, alert enums or communication protocols.
- No explicit override or release commands.
- No ownership flags on motor channels. The source carries an open author's note here: "should maybe various levels of imagined wellness change help drive this? For example a sudden drastic negative change imagined. not sure."
- Inter-mind communication is learned, not prescribed.
- Motor control transitions emerge from attention and confidence, not from arbitration rules.
Why: UEMA v3.4 specified a fixed protocol (alert → urgency threshold → override → release). A fixed protocol is a rule written by an engineer, which the project's thesis excludes; v2 "replaces it with emergent dynamics grounded in prediction error and learned internal state sharing".
The gap analysis resolves every open gap of this source under three constraints: (1) "Philosophical purity: No hardcoded arbitration, override flags, or prescribed protocols." (2) "Patent alignment: Pain/pleasure model (US 11,537,850 B2) is foundational. Wellness gradient IS the implementation." (3) "Training/inference separation: Backward passes teach. Inference uses forward-pass signals only. Async training keeps weights current." It states that no existing specification or code is modified until all gaps are resolved.
Its developmental context, which "applies to all gaps": movement produces touch; touch changes sensor resolution (per the patent); richer sensors give a sharper wellness evaluation, which gives better motor learning, so a sum of all minds' wellness, used for training (section 6.2), favors minds whose domains involve active movement. "Self is homeostatic, growing with function (ability to perceive)." Higher perceptions are built from lower ones, and the self extends to entities whose interaction improves prediction and produces touch (parents, toys, tools), limited to what the model has learned to perceive.
4One mind: the single-mind core loop
Every mind runs this loop at every model step. The source restates it in its section 1.1 "so that all Many Minds reasoning accounts for every component"; the details belong to the single-mind specification.
4.1 Sensors and encoders
- Input
- Pt, the raw physical state of the body at model step t. The single-mind specification calls the place where raw state is kept the Grounded Register.
- Computation
- Six encoders, one per sense: V (vision), A (audio), T (tactile), P (proprioception), M (motor), InterMind. They are shared: they run once per model step, not once per mind.
- Output
- One token per sense, each of shape [B, 1, d_model] (B = batch size): zt = {z_vis, z_aud, z_tac, z_pro, z_mot, z_inter}.
4.2 Forward model
- Input
- zt plus the context buffer: the tokens of past model steps.
- Computation
- 8 three-stage transformer blocks. Stage 1: cross-modal attention (within one model step, across senses). Stage 2: causal temporal attention (within one sense, across time). Stage 3: cross-modal temporal bridge (across both).
- Output
latents[B, seq_len, d_model];last_step[B, n_mod, d_model] (n_mod = 6, one row per sense), which the source names "THIS IS THE MIND'S STATE" and which section 5.5 uses;context_update[B, n_mod, d_model].
4.3 Prediction heads
- Input
- The forward model's output.
- Computation
- Five heads, one per sensory sense plus intermind; motor has no head. Each head: a learned query, multi-head attention (MHA), then a multilayer perceptron (MLP).
- Output
- A predicted token for the next model step, [d_model] numbers, per head. Comparing it with the next tokens gives PE.
4.4 Wellness evaluators
Two parts compute wellness. The wellness evaluators (the source's "grounded" evaluators; code GroundedEvaluator) compute the five wellness channels from raw state; they are the authoritative wellness signal. The wellness predictor (the source's "approximate wellness evaluator"; code ApproximateEvaluator) is a learned network that works on tokens, so that it can score imagined states, which have tokens but no raw state.
| Wellness channel (code name) | Computed from raw state Pt |
|---|---|
prediction_error | 1 − clamp(PE, 0, 1) |
tactile_comfort | 1 − clamp(mean_pressure, 0, 1) |
capability | mean(body_set_capacities): the mean remaining capacity of the body sets (groups of handles such as one arm or one hand) |
coherence | the body schema's coherence score (section 4.9), or a cosine-similarity fallback |
motor_state | a range-of-motion (ROM) boundary penalty, or a preference for the center of the range |
- Input
- Wellness predictor: the sensory tokens, excluding intermind.
- Computation
- Pool the tokens, then one MLP. Trained to match the five outputs of the wellness evaluators.
- Output
- [B, 5], one estimate per channel. Used in imagination.
4.5 Wellness gradient
- Input
- The total wellness (sum of the channels) and the command.
- Computation
torch.autograd.grad(total_wellness, motor_commands, create_graph=True)- Output
- [B, n_handles, 9]: for each number of each handle's command, the direction in which changing it raises wellness. The source calls it "THE core motor drive signal".
4.6 Motor controller
- Input
- Pooled latents, the current handle state and the wellness gradient.
- Computation
- A confidence head: pooled latents → GELU → sigmoid → a number in [0, 1]. If confident: depth 1 (fast path, about 1 ms). If uncertain: depth 2 up to a maximum, using imagination rollouts. The source calls this "adaptive computation depth".
- Output
- [B, n_handles, 9]: per handle, position (3) + rotation (4) + radius (1) + strength (1), sent to the body as the command.
4.7 Imagination
Imagination is running the forward model on candidate commands without sending them to the body. It starts when the motor controller's confidence is below a threshold, and runs in this sequence:
forward_model.predict_from_action(): a single step. It replaces the motor token with the candidate command, runs the prediction heads, and detaches the non-motor tokens (gradient flows one step only).score_predicted_wellness(): the wellness predictor scores the imagined state.forward_model.imagine_step(): a multi-step chain that does not detach, so the gradient flows through the whole chain. It shifts the context buffer forward. It is used for imagination-aware training (single-mind spec §6.3).- Cross-simulation attention (future): several rollouts with noise, then attention across the rollouts to find what is common to them. The source: "Divergent details wash out, intention crystallizes", followed by the open author's note "(what does wash out mean?)".
The imagination depth is the number of chained steps. It matters for Many Minds: the top mind uses longer chains than the other minds (section 7.6).
4.8 Intention (future; requires an accurate forward model)
Intention needs a desired state: a target vector in the forward model's latent space. The source names two mechanisms that produce it, at two levels.
Level 1, pain and pleasure → direction of the wellness gradient (Patent US 11,537,850 B2; single-mind spec §8). The patent specifies that pain attenuates sensor resolution (withdrawal) and pleasure amplifies it (approach). In UEMA the wellness evaluators map the current sensory state to a wellness vector, and torch.autograd.grad(wellness, motor_commands) gives a gradient that points toward states the system has learned are pleasant and away from states it has learned are painful. The forward model learns these associations ("this visual pattern led to wellness increase → approach when detected"). "A predicted-pleasant state IS a goal state": the forward model can generate it as a target, and a search over commands (the planner search, below) then looks for a command whose predicted result matches it.
Level 2, curiosity → seeking moderate prediction error → select_goal() (single-mind spec §21). CuriositySystem.compute_drive() returns a positive drive when PE is between 0.1 and 0.4, and a negative one when PE is below that (boredom) or above it (overwhelm). The drive modulates the direction of the wellness gradient. In Stream 4 (single-mind spec §20.5: goal-directed behavior and Many Minds splitting, from about 1M training steps), curiosity.select_goal(fwd_model, z_t) uses the forward model to find reachable states where PE would be moderate; the source calls that the desired-state generator.
| Layer | Link in the chain |
|---|---|
| Patent | pain/pleasure → sensor attenuation/amplification |
| Spec §8 | pain/pleasure → wellness gradient direction (approach/avoid) |
| Spec §21 | curiosity drive → moderate-PE seeking → select_goal() |
| Spec §10 | desired_state → IntentionGenerator → model inversion → motor plan |
| Spec §10.3 | plan_multistep + cross-simulation attention → robust execution |
- Input
- The desired state (from the wellness gradient and the curiosity drive) and the current state.
- Computation
- The planner search (
IntentionGeneratorin the source and the code; single-mind spec §10.2) inverts the forward model by gradient descent on the command: start from a random command; compute MSE(forward_model.predict(current, action), desired) + a strength penalty; update the commands, not the weights; repeat for 50 optimization steps at learning rate 0.1. Thenplan_multistep()(spec §10.3) chains intentions with imagination rollouts: plan a few steps, execute them, re-observe the real state in the Grounded Register, replan. - Output
- A short plan of commands, 3 to 5 steps long. Plans stay short because each imagined step adds forward-model error ("imagination drifts"), and re-observing the real state after each short plan removes the accumulated error.
Dependency gaps the source records for this mechanism:
curiosity.select_goal()is called in the Stream 4 pseudocode (§20.5), but its body is never defined: the curiosity drive is a scalar, and the projection from "moderate PE wanted" to a specific target vector in [d_model] latent space is not specified. It is a Stream 4 feature (after 1M steps), not a Many Minds prerequisite: Many Minds needs mastery detection, splitting and motor blending, none of which require goal selection.- The planner search exists and is functional (
intention_generator.py:78-189, 11 tests); it is disabled by default (enabled: false), which the source calls correct because it requires an accurate forward model. It includesplan_multistepandCrossSimulationAttention. CuriositySystemexists and is functional (curiosity.py:12-48);compute_drive()is implemented and called in the training step. Onlyselect_goal()is unimplemented.
4.9 Body schema (future; single-mind spec §11)
- Input
- At every model step, the change in the command and the change in the sensors:
update(motor_delta, sensor_delta). - Computation
DifferentiableBodySchemakeeps an exponential moving average (EMA) of the correlation between motor changes and sensor changes.self_membership()= sigmoid(temp × (corr − 0.5));coherence_score()= mean(|membership − 0.5| × 2).- Output
- The coherence score, which feeds the coherence wellness channel. Size: about 6.6K elements (EMA buffers, about 26 KB, no trainable weights). Section 7.3 uses its correlation buffers to find split domains.
4.10 Training loss
| Weight | Loss term | What it is |
|---|---|---|
| 1.0 | prediction_loss | Per-sense MSE of the forward model |
| 0.1 | contrastive_loss | VICReg or InfoNCE, to keep encoders from collapsing to one output |
| 0.5 | imagination_loss | Multi-step chain against the real trajectory |
| 0.01 | latent_reg_loss | Keeps predictions within 2σ of the real token distribution |
| 0.3 | approx_wellness_loss | Wellness predictor matches the wellness evaluators |
| 0.2 | anticipation_loss | NOT IMPLEMENTED: future observable motor preparation |
| 0.1 | gaze_loss | Active gaze direction |
| variable | motor_loss | Mode-dependent (BASELINE / DIRECT / FORWARD) |
One AdamW optimizer, learning rate 3e-4, weight decay 0.01, gradient clipping at 1.0, bf16 precision.
4.11 Key dimensions (specification target)
| Component | Value |
|---|---|
| d_model | 768 |
| n_heads | 12 (head_dim = 64) |
| n_layers (forward model) | 8 three-stage blocks |
| n_modalities | 6 (V, A, T, P, M, InterMind) |
| n_handles | 23 (full body of the source's avatar; the released MIMo layouts have 24 or 26, section 13) |
| context_length | 8 current, target 32 (about 0.5 s at 60 model steps per second) |
| n_bodysets | 10 (fatigue and energy tracking) |
| vision_rate | 24 Hz |
| other_rates | 60 Hz |
| total_params | about 271M |
5What every mind shares, and how minds read each other
5.1 Minds are prediction engines over domains
Each mind has its own forward model, motor controller, wellness evaluators and wellness predictor, working over a subset of handles and sensors. A mind always runs inference: it predicts, computes PE and produces a command at every model step.
"Frozen" means that a mind's gradients are near zero because its domain is mastered, not that its inference stops. A mastered mind still encodes, predicts, computes PE and produces a command every model step; its gradients stay near zero until its PE rises (section 8).
5.2 The shared sensory stream is ground truth
All minds receive identical sensory tokens: vision, audio, tactile, proprioception and motor. The encoders run once and the tokens are broadcast read-only. Why: it is the same body in the same world, so every mind receives the same vision token when a road exit sign comes into view, and the same tactile token when a foot slips (the source's examples). The shared stream is how each mind observes the consequences of the other minds' commands without explicit messages.
5.3 The inter-mind channel carries readable internal state
The inter-mind channel delivers to each mind a token, z_intermind, built from the other minds' readable internal state: a learned projection of each mind's last forward-model layer (5.5), which the source says carries that mind's hidden activations, its motor intent (the command it is about to produce) and its prediction error. It is not a message channel. It skips the physics delay: through the shared sensory stream, a command's effect reaches the other minds only after the physics program has moved the body, at a later model step.
The source's analogy is the efference copy of neuroscience: when the motor cortex sends a command, a copy goes to the sensory cortex so that the consequence can be predicted before it arrives through sensation. The inter-mind channel does the same across minds.
Why it must exist, when the shared stream already shows every consequence: a human reaching for a cup pre-shapes the fingers during the reach, before contact and before tactile feedback; the hand region has the arm's motor intent through fast internal neural pathways rather than through the arm's sensory consequences. The source argues that this is what makes biological motor learning fast compared with flat reinforcement learning: a robot trained by reinforcement learning learns one monolithic policy over all joints (no decomposition, no anticipatory coordination, millions of trials), while biological systems have hierarchical decomposition and direct sharing of internal state between regions. Without z_intermind, each mind would have to wait for the physical consequences of the other minds' commands to appear in the sensory stream, which the source judges "too slow for coordinated movement".
5.4 Influence within the same model step
z_intermind lets one mind's internal state change another mind's computation in the same model step, before the command is produced. It is encoded alongside the other tokens and enters the receiving mind's forward model at the current model step. Because the motor controller reads the resulting latent state, a change in another mind's readable state can at once change the receiving mind's predictions, its wellness predictor's estimate, its confidence, its adaptive computation depth and its motor output.
This matters most between the top mind and the attending mind. The top mind sends no prescribed messages and no override commands; it changes the attending mind's computation by shifting the attending mind's latent state through z_intermind.
5.5 What the readable state contains (Gap A, resolved)
Each mind's readable internal state is a learned linear projection of the forward model's last hidden layer: the output of the final transformer block, before the prediction heads. The projection gives a fixed-size vector of [d_model] numbers.
Why the last hidden layer and not one of the other candidates:
| Candidate | Why the source rejects or accepts it |
|---|---|
| PE (a scalar) | Too compressed: it gives how wrong a mind is, not what it is predicting or what command it is about to produce. |
| Prediction head outputs (one vector per sense) | Already projected to single senses, so the cross-modal context is lost: the arm mind's predicted proprioception would arrive without its relation to vision. |
| Motor controller output (handle commands) | Gives only the command, not the context it came from or the predicted consequence. The hand mind needs the arm's intent and context, not only its raw joint targets. |
| Last hidden layer (accepted) | Holds all of the above in compressed form: it integrates cross-modal and temporal context through 8 attention blocks, and both the prediction heads and the motor controller read from it. "It is the mind's state." |
UEMA Spec v3.4 is consistent with this: its §16.1 AlertSignal.anomaly_context used forward_model.get_last_hidden(), and its §16.3 defines InterMindToken.content: Tensor[d_model]. What v2 removes is the prescribed protocol (alert types, urgency thresholds, override flags) that v3.4 wrapped around it, replacing it with learned attention over the raw internal states.
5.6 Per-mind filtering
z_intermind must be computed per mind, not shared as one vector. Each mind uses a learned identity embedding to attend over the states of the other minds and extract what is relevant to it. The source's example: the hand mind places high attention weight on the arm mind's state; the posture mind may place almost none there.
- Input
mind_id(an integer index for this mind) andall_mind_states[n_minds, d_model], the projected states of the other minds.- Computation
- query = the mind's learned identity embedding (
nn.Embedding(max_minds, d_model)); multi-head cross-attention with 4 heads, keys and values =all_mind_states. - Output
- z_intermind, [d_model] numbers, and the attention weights. The source notes that the weights are interpretable (they show which minds' states a mind draws on) and learned, not prescribed.
The source's target code; it states that this class does not yet exist and that the current implementation is a stub, SelfAttentionPoolEncoder (lem/model/encoders/self_attention.py), "receiving synthetic Gaussian noise with loss_weight=0":
# TO BE BUILT — this class does not yet exist in the codebase.
class PerMindIntermindEncoder(nn.Module):
def __init__(self, d_model, max_minds):
self.mind_embeddings = nn.Embedding(max_minds, d_model)
self.cross_attn = nn.MultiheadAttention(d_model, num_heads=4)
def forward(self, mind_id, all_mind_states):
# mind_id: int index for this mind
# all_mind_states: [n_minds, d_model] internal states from all other minds
query = self.mind_embeddings(mind_id).unsqueeze(0)
z_intermind, weights = self.cross_attn(query, all_mind_states, all_mind_states)
return z_intermind.squeeze(0) # [d_model]
6One command from many minds
Several minds may produce a command for the same handle. This section gives the source's answer to how those commands become the one command the body receives, during training and during inference.
6.1 Motor control is confidence-weighted competition
A higher mind is any mind above another in the hierarchy (its parent, or the parent's parent); its wellness evaluators cover a wider scope of body sets and sensors than the lower mind's. There are no ownership flags and no override/release protocol. Every mind produces commands for its handles with a confidence, and control transitions follow from that:
- "Override" is not a command. The higher mind's PE rises in the lower mind's domain. The higher mind's commands for those handles become stronger and its confidence higher. The lower mind's own PE is also high there, so its confidence drops. In the blend of section 6.2 the higher mind's commands then carry more weight, through competition, not a flag.
- "Release" is not a command. When the situation has passed, the higher mind's commands for those handles fade (the source: "Its attention drifts elsewhere"), and the lower mind's steady-state commands carry the blend again because nothing competes with them.
- Ceding control needs no signal. The lower mind never receives a "you're overridden" signal; it keeps producing commands throughout, and they are outweighed only while the higher mind's commands for those handles are strong.
6.2 Combining wellness gradients and commands (Gap B, resolved)
The problem: when several minds compute wellness gradients for the same handles, a rule is needed to combine them. The source's resolution has two stages that run in different execution contexts.
Stage A, training: one unified system wellness gradient. All minds' wellness evaluations are summed into one scalar, Wsys, and one gradient is taken with autograd. By linearity of differentiation this equals the sum of the individual gradients, but no separate gradient vectors ever exist that would need a combination rule.
W_sys = sum(mind.wellness_evaluator(P_t, pe_i).sum() for mind in hierarchy.all_minds())
system_grad = torch.autograd.grad(W_sys, motor_commands, create_graph=True)[0]
Each motor controller is trained on the unified gradient. What it is meant to learn is when to yield (the parent's broader wellness is threatened) and when to act (only its own domain has signal). "Scope IS the weighting": when the parent's broader-scope wellness evaluators detect a problem, more of its channels are active, so it contributes more to the gradient; no explicit priority weighting is needed.
Stage B, inference: a confidence-weighted blend, forward pass only (no backward pass). For each handle h, over the minds whose handle set contains h:
weights_h = softmax([conf_i for mind_i if h in mind_i.handles] / temperature)
final_cmd[h] = sum(weights_h[i] * cmds_i[h] for i in competing_minds)
The confidence is the output of each motor controller's confidence head (single-mind spec §9.2). The softmax temperature should be per handle and learned. Whether higher minds actually produce the higher confidence on contested handles is "an empirical consequence of the training mechanism, not independently guaranteed at runtime".
Oscillation between minds is damped by three existing mechanisms: continuous physical inputs change smoothly; the context buffer gives temporal context; and fluctuating wellness produces low confidence, which triggers deeper imagination (temporal smoothing).
Contradiction resolved: UEMA v3.4's §17 (binary arbitration with override flags) is fully incompatible with this and is replaced entirely.
Problem as the gap analysis states it. No combination mechanism was specified for several minds' wellness gradients on the same handles; UEMA v3.4's §17 used binary arbitration with override flags; v2 §2.5 rejected that in favor of emergent competition but left the mechanism unspecified.
Added reasoning not in the source. "During training, scope IS the weighting (mathematically guaranteed)": a parent detecting a problem has 3 active wellness channels (prediction_error, coherence and motor_state, written w_pred + w_coh + w_motor), and a domain child has 1 (prediction_error), and more active terms give a larger gradient contribution. "Wellness gradient is a training signal, not a runtime signal": the motor controller receives the gradient during training and learns the mapping; during inference "it executes from latents alone". Arbitration at inference needs priority (which domain is urgent), not direction, and priority is available as forward-pass quantities: wellness scores, PE and confidence heads. Its oscillation argument names a "32-frame context buffer" (the source's table gives 8 current, 32 target).
Tension this edition notes. "Executes from latents alone" at inference differs from the core loop of section 4.6, whose motor controller input includes the wellness gradient at every model step; the released motor controller also takes it as an input and substitutes zeros when it is absent (section 13).
Specification changes it lists.
| Document | Section | Current | Change |
|---|---|---|---|
| UEMA | §7.4 compute_motor_drive | Single-mind pseudocode | No change; note that it becomes a per-mind inner call when Many Minds is active. |
| UEMA | §13 Mind dataclass | override_active: bool | Remove the field. |
| UEMA | §16.2 override/release | Explicit methods with flags | Remove both methods. |
| UEMA | §17 MotorArbitrator | Binary: deepest non-overridden wins | Replace with the confidence-weighted softmax blend per handle. |
| ManyMinds v2 | §2.5 Motor control | Emergent (unspecified) | Add the two-stage mechanism. |
| ManyMinds v2 | §3 Gap B | Open gap | Mark resolved. |
| Both | New: system training function | N/A | Add the system-level wellness sum across minds, active only when there is more than 1 mind. |
Contradictions it records: UEMA §17 against v2 §2.5 is fully incompatible and §17 is replaced entirely; UEMA §7.4's signature assumes a single mind and is correct as it is, with Many Minds wrapping it.
6.3 Attending-mind engagement: escalation without override
The attending mind is the present-world motor coordinator. The top mind has no handles and produces no command, so its influence on the body acts only through the attending mind's inputs: z_intermind, and through it the engagement signal below, not by entering the per-handle blend. The attending mind computes a learned engagement signal at every model step:
g_engage is a forward-pass quantity only: not a runtime wellness gradient, not an override flag, not a programmed protocol. The source intends it to raise, at the current model step, the attending mind's computation depth and coordination gain, and to lower the share of contested handles taken by routine lower-level commands (the two boxes above), for imminent danger, route-critical events or unstable novel tasks; it gives no formula for any of the three (section 12). The result is a continuous transition between guidance and strong re-engagement, without top-mind commands and without hardcoded override logic.
6.4 Disagreement between confident minds (Gap E)
Two minds may both be confident within their scopes and still pull behavior in different directions: a lower mind may be confident locally while a higher mind's wider-scope wellness channels score a larger consequence as bad.
- Training: resolved by Gap B. The single system wellness sum and its one autograd call give every motor controller a gradient that includes the higher minds' wellness channels, so a lower mind's commands are trained to change when a higher mind's wellness would fall.
- Inference: no runtime gradients; forward-pass quantities only. The higher mind's readable state, which reflects its lower wellness, travels through z_intermind and changes the receiving mind's latents, predictions, wellness predictor's estimate, confidence and command. Between the top mind and the attending mind the concrete path is g_engage (section 6.3): a large shift in the higher mind's state raises g_engage; a small shift leaves routine lower-level control largely undisturbed.
So disagreement is resolved by (1) unified wellness during training, (2) readable inter-mind state during inference and (3) learned attending-side engagement for the apex relation, "not by override flags, ownership rules, or runtime gradient combination".
Problem as stated. When two minds both have high confidence and strong wellness gradients for the same handles, "Pure confidence weighting fails when both confidence heads output high values."
Two regimes of one mechanism. Guidance: the top mind's state weakly conditions the attending mind while routine control stays delegated. Escalation: the top mind's state drives g_engage high, raising the attending mind's computation depth and coordination gain, for imminent route decisions, sudden danger, or active acquisition of a novel motor task.
When it fails, "fighting autopilot". If the higher mind's wellness drop is small, its inter-mind state shift is small, g_engage stays low and the habitual response carries the blend. The gap analysis calls this correct: "mild discomfort should not displace confident routine control"; the threshold comes from learned signal magnitudes, not a programmed priority rule. The lower mind "is not overridden; it updates itself".
Specification changes. v2 §3 Gap E: mark resolved (training: scope-based unified wellness per Gap B; inference: forward-pass inter-mind conditioning plus g_engage; no runtime gradients, no override flags). v2 §§2.3–2.5: clarify that top-to-attending influence is same-model-step latent conditioning plus attending-side engagement, not top-mind commands. Both documents: add the learned forward-pass attending engagement mechanism, driven by z_intermind, PE and the wellness predictor's estimate.
6.5 Imagination scope is the higher mind's authority
When confidence is low the motor controller runs imagination (section 4.7): it predicts the consequences of candidate commands with the forward model, scores them with the wellness predictor, and picks the best. This is where the hierarchy matters most. Each mind imagines within its domain:
- Hand mind: candidate finger commands in, predicted tactile tokens out.
- Arm mind: candidate reach commands in, predicted proprioception tokens out.
- Top mind: a reach, grasp and lift sequence in, predicted total wellness over 30 model steps out.
The higher mind's advantage is not better motor execution; the lower mind is better at that, since it mastered the domain, which is why it was split off. The advantage is broader imagination scope combined with broader-scope wellness evaluators. The source's example: a driving mind can imagine "take exit → road curves → maintain lane", which domain-local wellness scores as fine; the higher mind can imagine "take exit → wrong destination → 30 minutes lost", which broader wellness scores as bad.
When a mind splits, the child gets domain-scoped wellness channels:
| Channel | Child's version |
|---|---|
| prediction_error | From its own, domain-scoped forward model |
| tactile_comfort | Only the tactile patches in its domain |
| capability | Only its body sets |
| motor_state | Only its handles' range-of-motion positions |
| coherence | Still needs the body schema; the source marks its scope "unclear — Gap" here, and section 7.7 gives an answer |
The parent keeps the broader-scope evaluators. The wellness predictor must also be per mind, trained to match that mind's wellness evaluators, so a lower mind's wellness predictor becomes specialized: good at scoring imagined grasps, poor at scoring imagined navigation decisions.
The higher mind does not need better commands to override. It needs a wellness gradient that points in a different direction from what the lower mind is doing. In UEMA the wellness gradient is the motor drive signal, so broader wellness scope gives a broader gradient and authority over the lower mind's narrower gradient. This is the source's §2.10; its later Gap B and Gap E resolutions (6.2, 6.4) use this gradient only in training, so at inference the higher mind's authority acts through z_intermind and the confidence blend. For the top mind this authority acts through z_intermind and attending-side engagement, not through top-mind commands.
6.6 Radius and hierarchy are complementary
Radius (section 2) and the mind hierarchy act on different dimensions. Radius is within-mind motor synergy: how far one command spreads to nearby joints, fine (radius → 0, a single fingertip) or gross (radius → 1, a whole-arm sweep), inside one mind's handle set. The hierarchy is across-mind coordination in time: the arm reaches, then the hand grasps, delegated in sequence and in parallel.
They compose: the Arm_L mind uses a high radius for gross reaching; the Hand_L mind uses a low radius for fine finger positioning. Neither has to learn the other's policy, and the inter-mind channel lets the hand pre-shape during the reach from the arm's motor intent.
7Splitting a mind
7.1 Splitting crystallizes stable weights
When a set of handles has near-zero PE and near-zero gradients over a sustained period, all of the mind's weights are copied into a new mind that takes over those handles (7.5). The parent's capacity is freed for new learning. The child's weights are already near-optimal, so "frozen" describes their state, not an imposed constraint.
7.2 What triggers a split (Gap C, resolved by a programmed trigger)
The single-mind spec's §15.1 defines a mastery detector (MasteryDetector), and MasteryTracker in curriculum.py:111-178 already implements its tracking logic with the same thresholds:
PE_THRESHOLD = 0.05 # mean PE below this
VARIANCE_THRESHOLD = 0.01 # PE variance below this
WINDOW = 50,000 steps # evaluation window
CONSISTENCY = 0.95 # fraction of window meeting criteria
The tracker keeps a rolling window of 50,000 PE records, one per curriculum step in the released code (curriculum.py line 416), and checks these criteria for advancing the curriculum phase, so the trigger itself exists. The remaining gap is per-domain PE input: MasteryTracker receives one scalar, mean_pe, not per-handle or per-domain values. To support splitting it needs per-handle PE from the loss manager, so that mastery can be detected within a body region and not only globally.
7.3 Which handles go to the child (Gap D, resolved)
Domain boundaries come from the model's own learned coordination, not from predefined groups. The _get_domain_groups() dictionary of the Implementation Plan (an earlier planning document, not in this release) is removed, because a predefined grouping is a prescribed protocol, which the foundational constraint forbids.
- Input
- Per-handle PE over the mastery window; the body schema's EMA correlation buffers,
correlation_matrixof shape [n_handles, n_handles × n_proprio_per_handle] inbody_schema.py. - Computation
- The revised
find_mastered_domains(): (1) compute per-handle PE over the evaluation window; (2) keep the handles whose PE is consistently below threshold; (3) among those, cluster by motor-to-proprioception correlation, using simple thresholding on the correlation matrix (not spectral clustering or k-means), threshold a hyperparameter with initial value 0.6. - Output
- Each correlated cluster, as a candidate domain for splitting.
The four sub-questions, answered:
- Clustering algorithm: simple thresholding on the body schema's correlation matrix. Why it suffices: the body's physics makes the right handles correlate.
- Kinematic chain constraints: emergent, not prescribed. Handles on the same kinematic chain show high motor-to-proprioception correlation because moving one moves the others through the skeleton. Thumb_L will never split away from Hand_L, because their activations are always correlated during grasping.
- Several children at once: yes. If distinct clusters are found at the same time, all can split. The source expects this to be rare, because developmental constraints stagger mastery.
- Minimum mind size: one handle. A low-PE handle whose correlation with every other handle stays below the threshold forms its own domain, for example the head handle used for gaze.
clone_for_domain(), the split function that UEMA v3.4 §15.2 calls but never defines, is resolved as a full deepcopy of the parent's weights (section 7.6). "for_domain" refers to domain-scoped wellness training, not architectural pruning; sensor scoping uses handle_set_to_sensor_scope() (section 7.7). Contradiction resolved: the Implementation Plan's §13.10 note 3 ("Domain groups are predefined, not discovered") is reversed; the plan itself had acknowledged it was a stopgap.
Problem as stated. Four unresolved sub-questions (the clustering algorithm for "consistently coordinated activation"; hard constraint or soft prior for kinematic chains; several children at once; minimum mind size), and clone_for_domain() in UEMA §15.2 called but never defined. Three documents described the same mechanism three ways: UEMA §15.1 "clustering motor channels with consistently coordinated activation"; the Implementation Plan §13.10 note 3 "Domain groups are predefined, not discovered", with a hand-authored dictionary of handle sets named posture, grasp_left, grasp_right, gaze and balance; v2 §2.7 splits that follow kinematic chains.
Added reasoning. The perception-space framework (perceptionSpace-4.docx §11 and §13, in the release's spec folder) has splitting divide the perception space and follow kinematic chains and developmental sequence, with the regions of deepest function (longest training, lowest PE) splitting first; the gap analysis keeps that, but requires the regions to be identified from the model's own PE and correlation patterns, not a lookup table. The correlation tracked is motor-to-proprioception, not pure motor-to-motor co-activation, and it captures which handles move together with coordinated proprioceptive consequences. Implementation dependency: correlation-based splitting depends on the body schema being active and accumulating meaningful correlation data during training. The threshold 0.6 means "handles whose motor commands produce correlated proprioceptive consequences more than 60% of the time belong together" (a correlation coefficient of 0.6 is not a share of time; the gap analysis gives no other reading). The body-set mapping stays the scoping mechanism, not the split trigger: the split decision comes from learned behavior, and the body structure then sets the child's perception-space scope. Robustness: for a non-human body (four arms, a tail) predefined groups would be wrong, while correlation-based discovery would still work; "The architecture should not assume human body topology."
clone_for_domain(). The Implementation Plan's §13.10 note 1 defines it as copy.deepcopy() of the parent's weights, with domain_channels set to the discovered cluster's handle indices. The child's forward-model scope is Gap H, not Gap D.
Connection to Gap F. Correlation-based discovery makes the left/right question empirical: if both arms are enabled together and used independently, the correlation pattern may produce a left/right split; if left and right are staggered, functional splits are more likely.
| Document | Section | Current | Change |
|---|---|---|---|
| UEMA | §15.1 find_mastered_domains() | Checks PE against the predefined _get_domain_groups() dictionary | Replace with the three steps above, clustering on [n_handles, n_handles × n_proprio_per_handle]; remove _get_domain_groups(). |
| UEMA | §15.2 clone_for_domain() | Called, never defined | Add the definition: copy.deepcopy(); child's domain_channels = the cluster's handle indices; wellness scoped by handle_set_to_sensor_scope(). |
| UEMA | §15.2 _infer_sensor_domains() | Called, undefined | Replace with handle_set_to_sensor_scope() of v2 §2.13. |
| Implementation Plan | §13.10 note 3 | "Domain groups are predefined, not discovered." | Replace with "Domain groups are discovered through motor-to-proprioceptive correlation among low-PE handles, not predefined." |
| ManyMinds v2 | §3 Gap D | Open gap | Mark resolved. |
Contradictions recorded: the Implementation Plan's note 3 against the design philosophy (it had itself said "In a more advanced implementation, domains could be discovered through correlation clustering"); and UEMA §15.1's wording, which the gap analysis finds already describes the correct mechanism.
7.4 Splits follow kinematic chains
Any mind can split, not only the top one: the arm mind can split off a hand mind; the posture mind can split off a balance mind. Why: this gives anatomically meaningful hierarchies rather than flat clusters of skills. The source's growth example, step by step:
- [Top] → all 23 handles.
- [Top] splits off [Posture] (legs, feet, spine).
- [Top] splits off [Arm_L] (UpperArm_L, LowerArm_L, Wrist_L and fingers).
- [Arm_L] splits off [Hand_L] (Thumb_L to Pinky_L).
- [Posture] splits off [Balance] (feet, ankles).
7.5 What is copied at a split
The child receives a complete instance of the core loop of section 4:
| Component | Copied? | Scope change |
|---|---|---|
| Forward model (8 blocks) | Cloned | Full deepcopy. All 6 sense inputs kept. Domain specialization comes from training, not architectural pruning. |
| Prediction heads (5) | Cloned | Needed for imagination |
| Motor controller | Cloned | Domain-scoped to the child's handles |
| Confidence head | Cloned (part of the motor controller) | |
| Wellness evaluators (5) | Cloned | Domain-scoped where applicable |
| Wellness predictor | Cloned | Per mind, for imagination scoring |
| Context buffer | Fresh (empty) | Begins accumulating from the split |
| PE history | Fresh | Begins tracking from the split |
| Inter-mind state projection | New (learned linear) | Projects the last hidden layer to [d_model] |
The parent keeps its full-scope versions. After the split:
- The parent's motor controller no longer produces commands for the child's handles. The source's reason: those handles' wellness gradients are near zero in the parent's broader-scope wellness evaluators, because the child keeps them in good state.
- The parent's forward model still predicts the full sensory stream, so it can imagine what the child's domain will do.
- The child's forward model is a full deepcopy of the parent's and keeps all 6 sense inputs, including global vision and audio.
7.6 The child's forward model and the top mind's imagination depth (Gap H, resolved)
The child receives a full deepcopy of the parent's forward model; no pruning is done at the split. It keeps all 6 sense inputs. Why: predicting the domain still depends on context outside the domain; a hand mind needs vision to predict contact, and a local controller may need broader body context to predict the sensory consequences of its commands.
Specialization therefore comes from training, not from cutting inputs or shrinking the model. The child's wellness channels are domain-scoped, so learning pressure falls on its own handles, body sets and sensory consequences; it uses the global senses as context while its role becomes more specialized to its domain. The parent keeps full-stream prediction. This resolves clone_for_domain(): clone means a full deepcopy; "for_domain" means functional specialization through domain-scoped learning.
Top-mind specialization is a separate question. After the apex split (section 9), the top mind diverges through longer imagination chains: the attending mind keeps the standard depth of 3–5 steps, while the top mind uses a larger per-mind depth (initial: 16+, subject to tuning). Imagination loss at those horizons is expected to bias it toward longer-timescale structure. Whether it does "remains an empirical question to validate"; the architectural definition of the child's scope is resolved.
Problem as stated. Does the child's forward model predict only its domain or the full stream? Domain-only is cheaper but cannot use vision as context; full-stream duplicates the parent; clone_for_domain() was undefined.
Added mechanism. The child's domain-scoped wellness drives gradients only through domain handles, so its prediction heads for non-domain senses receive no wellness-driven gradient and become less accurate relative to the parent's: "doesn't waste capacity being a good vision predictor". The top mind's longer chains use the existing imagination_loss over longer chains, not a new loss function; imagination depth is a per-mind hyperparameter set at the split.
Flag the source does not carry. The gap analysis calls the long-horizon claim "the strongest empirical assumption in the gap analysis" and "the highest-priority validation target for Many Minds experiments": if longer chains do not produce specialization, "an alternative training signal for the top mind would need to be designed".
Specification changes. UEMA §15.2: clarify that the clone is a full deepcopy with all prediction heads and all 6 sense inputs. Both documents: add a per-mind imagination depth (motor children and attending mind: 3–5; top mind: 16+, initial, subject to tuning), with empirical validation needed.
7.7 Mapping a handle set to sensors (source §2.13 and Gap K, resolved)
A mind's motor domain is a set of handle indices, but its wellness channels and forward model also need to know which sensors belong to that domain. The mapping chain is: handle index → the handle's body set → that body set's sensors. The source states that this mapping already exists in the codebase; in this release that mapping is lem/model/body_sets.py (section 13).
| Sense | How a mind's share of it is chosen |
|---|---|
| Tactile | Each tactile patch carries an identifier of the skeleton bone that produced it, and bones map to body sets. A mind's tactile domain = the patches whose bone belongs to one of its body sets. |
| Body-set state | Direct: capacity and accumulated energy per body set (10 entries). A mind reads only the entries of its body sets. |
| Proprioception | Each proprioception bone entry carries a bone index (the source's body has 12 entries). A mind's proprioception domain = the entries whose bone belongs to its body sets. |
| Vision | Global: every mind receives the full vision token. |
| Audio | Global: every mind receives the full audio token. |
The source's target function, which it states does not yet exist (the Python-side handle-to-body-set mapping, body_sets.py, is the foundation for it):
# TO BE BUILT — this function does not yet exist in the codebase.
def handle_set_to_sensor_scope(handle_indices: Set[int], handle_defs) -> SensorScope:
"""Given a mind's handle set, compute which sensor dimensions belong to it."""
# handle_defs: one definition per handle, giving its body_set and bone_name
body_sets = {handle_defs[i].body_set for i in handle_indices}
bones = {handle_defs[i].bone_name for i in handle_indices}
return SensorScope(
body_sets=body_sets,
bone_names=bones,
# Tactile: filter patches by bone_hash ∈ bones
# Proprio: filter entries by bone_index ∈ bones
# Body set state: filter by body_set ∈ body_sets
# Vision/Audio: always included (global)
)
It runs once, at split time, and sets the scopes of the child's wellness channels listed in 6.5: tactile_comfort over the patches of SensorScope.bone_names, capability over SensorScope.body_sets, motor_state over the handles in handle_indices, and coherence over the body schema correlations of those handles only.
7.8 Starting the inter-mind channel (Gaps G and J, resolved)
Gap G (after a split the child cannot use inter-mind signals) and Gap J (the system goes from no inter-mind content to active inter-mind content) are the same cold-start problem at two scales. Three mechanisms remove it:
- Pre-split self-reading. During single-mind training the one mind reads its own last hidden layer through the inter-mind channel instead of zeros, so the inter-mind encoder processes real activations from step 0. Its
loss_weightis nonzero from the start (initial: 0.1). The target encoder of section 5.6 uses cross-attention with a learned identity embedding as query over projected states, which is not a trivial identity mapping. The currentSelfAttentionPoolEncoderstub must be replaced or extended to accept the forward model's last hidden layer as input. - Deepcopy continuity. At a split the child is a deepcopy, so its internal state is nearly identical to what the parent produced for that domain; the token the parent reads from the child looks almost the same as what it read from itself. The distribution shift at the split is minimal.
- Gradual divergence. After the split the child specializes and the parent keeps learning; their states diverge gradually, so both inter-mind encoders adapt through normal training without an abrupt shift. This holds for every split, including the top-mind split.
Problems as stated. Gap G: before a split, coordination was implicit (one forward model); after it, two minds must coordinate through z_intermind, and neither was trained with inter-mind signals. Gap J: at training start z_intermind is zeros; after the first split the channel suddenly has content, and the inter-mind encoder (UEMA §5.6) has never processed real internal states.
Added detail. Why self-reading is not trivial: the projection through a learned linear layer means the encoder processes a transformed representation, not a raw identity copy. After the split the child's projected state uses the same (deepcopied) projection weights and the same format, with gradually different content, so the encoder has already been trained on the format.
Discrepancy between the two documents. The gap analysis describes the current stub as SelfAttentionPoolEncoder in intermind.py, "a stub returning zeros"; this source describes it as the encoder in lem/model/encoders/self_attention.py "receiving synthetic Gaussian noise". Section 13 states what the released files show.
Specification changes. UEMA §5.6 InterMind Encoder: change loss_weight from 0.0 to 0.1; enable self-reading from step 0. v2 §3 Gaps G and J: mark both resolved.
8Freezing and thawing
If freezing is natural (gradients go to zero when PE goes to zero), thawing should be natural too: gradients increase when PE increases because the domain changed. A mastered domain meeting novel conditions produces real PE and real gradients, and the mind adapts because there is something to learn again, not because a programmed trigger re-enables learning.
8.1 Optimizer mechanics (Gap I, resolved)
Ordinary gradient descent with no freeze switch (the source calls it "natural gradient dynamics"; it is not the natural-gradient method), with eviction of optimizer state to save memory. There is no is_training flag and no learning_rate = 0.0: a mastered mind has near-zero PE, produces near-zero gradients, and its weights stop changing.
- Input
- Each mind's gradient norm over the mastery detector's window (50,000 steps); each mind's PE.
- Computation
- If the gradient norm has stayed below threshold for the whole window, evict the mind's Adam optimizer state from GPU memory. The mind keeps running inference, computing wellness, producing commands and monitoring its domain; its weights do not update because the gradients are near zero. If PE rises (a novel situation), gradients become nonzero, the optimizer state is reinitialized with fresh Adam momentum, and training resumes.
- Output
- GPU memory freed while a mind is mastered; training restored when its domain changes.
The cost of the memory saving: reinitializing Adam loses its accumulated variance estimates, which can cause larger weight updates while the fresh estimates restabilize. Gradient clipping (grad_clip=1.0) bounds them, and the source considers this appropriate for a mind meeting novel conditions, where the old estimates were stale. So ordinary gradient descent is kept, and the memory optimization adds a small training discontinuity at thaw that uninterrupted dynamics would not have. Contradiction resolved: UEMA v3.4's §15.2 explicit freeze (is_training = False, learning_rate = 0.0) is replaced.
Problem as stated. v2 advocates natural freezing and thawing; UEMA used an explicit freeze; and "Adam maintains optimizer state (~240MB per 30M-param mind) even when gradients are negligible."
Specification changes. UEMA §15.2 freeze logic: remove the explicit freeze; use natural gradient dynamics; evict optimizer state when the gradient norm is below threshold for the mastery window; reinitialize it when gradients return. v2 §3 Gap I: mark resolved, consistent with the removal of the ThawDetector (section 11).
9The two-level apex: top mind and attending mind (Gap F)
The source's Gap F began as "left brain / right brain": should the first split be left/right or functional? Its answer: the architecturally important split is not between hemispheres but between attention to the present and broader thought, a present-world attending mind and a broader top mind.
- Top mind: broadest perception space; longest temporal horizon; deepest self-model (Levels 1 to 3, defined in the box below); no handles and no direct motor output; changes behavior only by shifting the attending mind's computation through z_intermind.
- Attending mind: full-body perception space of the present; its own grounded self-model; the present-world motor coordinator, responsible for navigation, situational judgment, route decisions, gaze allocation and overall coordination of the body; reads the top mind every model step through z_intermind.
Below the attending mind sits the motor hierarchy, whose shape comes from the correlation-based discovery of section 7.3. Child minds take mastered local domains, but the attending mind stays the coordinator for the present embodied situation.
The top mind can spend capacity beyond the immediate physical present only while the attending mind's PE is low enough that its engagement stays low. When the physical situation becomes urgent or unstable, the attending mind's high-PE state reaches the top mind's forward model through z_intermind, and the top mind reshapes present-world behavior at once by raising the attending mind's engagement through z_intermind. The question the source poses is "when does the system benefit from separating broad present-world coordination from broader long-horizon cognition?"
Problem as stated. Should the first split be left/right or functional (posture/manipulation)? UEMA §18 prescribed functional splits without weighing the left/right alternative.
Resolution. "The left-brain / right-brain question is dissolved. Our architecture does not assume hemispheres"; left/right is a detail of bilateral biological anatomy, and the motor topology comes from Gap D. The self-model levels are those of single-mind spec §11.4, not the intention levels of section 4.8: Level 1 tracks what moves with the command (the body, a grasped object; about 100 model steps); Level 2 tracks entities whose presence goes with keeping wellness up (about 10K model steps); Level 3 tracks entities tied to long-term wellness (about 100K model steps). Self-model levels: the top mind holds Level 1+2+3 (Level 3 entities: career, relationships, long-horizon goals, plus everything the attending mind tracks); the attending mind holds Level 1+2, grounded in "my body", "my route", "my current task". When something exceeds the attending mind's scope, its PE and state shift and the top mind reads that shift through z_intermind, for example when a road exit comes up, a deer is on the road, or a task stays unstable (the gap analysis's examples).
The top mind does not act through Gap B directly. It has no handles and does not enter per-handle arbitration. Its influence is upstream of actuation: it changes the attending mind's latent state through z_intermind, and the attending mind's engagement mechanism converts that into changes of computation depth, arbitration posture and coordination strength before the command is written. Cases it names: re-engagement at a driving exit; sudden danger (deer on the road) driving strong engagement without a hardcoded override; novel hand-eye skill acquisition, where the attending mind's persistently high PE keeps g_engage high, so g_engage stays high for as long as the PE stays high.
Top-mind split criterion, not stated in the source. The same mastery detector and the same thresholds. It triggers when the attending mind's PE for all its remaining physically coordinated responsibilities stays below threshold over the evaluation window, when broad present-world coordination is stable enough that a separate top mind can spend capacity beyond the immediate situation. "The top mind is the last split." Abstract specialization comes after the split, through longer-horizon training (Gap H), and is not a split criterion.
| Document | Section | Current | Change |
|---|---|---|---|
| UEMA | §18 Growth trajectory | Prescribes Posture first, then Grasp | Remove the prescribed sequence; split sequence comes from Gap D; add the two-level apex as a late developmental milestone. |
| UEMA | §13 Mind dataclass | Assumes a single apex; all minds have handles | Add: handle_indices can be an empty set (top mind), which joins the wellness sum and the inter-mind channel only. Implementation note: motor_controller.py currently requires n_handles > 0 and must be made optional for the zero-handle case. |
| ManyMinds v2 | §§2.3–2.5 | Inter-mind readable state; motor competition | Clarify that the top mind acts through attending-side engagement, not per-handle arbitration. |
| ManyMinds v2 | §3 Gap F | Open gap | Mark resolved. |
| Both | New: attending engagement | N/A | Add the learned attending-mind engagement mechanism. |
10Register of the source's gaps
The source numbers its open questions as Gaps A to K. Their state as the source records it:
| Gap | Question | State in the source | On this page |
|---|---|---|---|
| A | What is in the inter-mind state vector? | RESOLVED last hidden layer, learned linear projection | 5.5 |
| B | How are several minds' wellness gradients combined? | RESOLVED W_sys in training; confidence blend at inference | 6.2 |
| C | What triggers a split? | RESOLVED by a programmed trigger (mastery detector); per-domain PE input still missing | 7.2 |
| D | Which handles go to the child? | RESOLVED correlation-based discovery | 7.3 |
| E | Disagreement between confident minds | Resolution text given; heading carries no "RESOLVED" mark | 6.4 |
| F | Left/right brain and the first split | Resolution text given; heading carries no "RESOLVED" mark | 9 |
| G and J | Inter-mind cold start | RESOLVED self-reading, deepcopy continuity, gradual divergence | 7.8 |
| H | Child forward-model scope | RESOLVED full deepcopy; top-mind specialization an open empirical question | 7.6 |
| I | Optimizer mechanics of freezing and thawing | RESOLVED natural dynamics with optimizer-state eviction | 8.1 |
| K | Not stated; the entry reads only "RESOLVED (see Section 2.13)" | RESOLVED | 7.7 |
"Resolved" in this table is the source's design status: a mechanism is specified. In the released code only two inputs exist: the mastery tracker's thresholds (Gap C) and the body schema's correlation buffer (Gap D's input). No gap's mechanism has run or been measured with more than one mind (section 13).
11What v2 removed from v1, and what it keeps from UEMA v3.4
| v1 concept | Status | Reason |
|---|---|---|
| Hardcoded message types (alert, override, release, status, goal, complete) | Removed | Contradicts the UEMA philosophy: communication must be learned, not prescribed as enum types. |
| Override/release as explicit operations with flags | Removed | Override follows from the higher mind's stronger commands on the handles in question; release follows from those commands fading. |
| Motor channel ownership sets | Removed | Replaced by confidence-weighted competition; no mind owns handles. |
| GoalSignal / CompletionSignal classes | Removed | Prescribed protocol. Goal-like and completion-like behavior, if needed, must emerge from learned inter-mind state patterns. |
| ThawDetector with programmed thresholds | Removed | Thawing should be natural (gradients increase when PE increases). |
override_active flag on Mind | Removed | No flags; competition sets control. |
| MotorArbitrator with ownership rules | Removed | Replaced by confidence-weighted blending. |
Kept from UEMA v3.4 and compatible with v2:
- SensoryBus with shared broadcast of the sensory tokens (5 senses).
- Per-mind forward model over the shared tokens.
- Mastery detection (low PE and low variance over a window); trigger resolved as the programmed mastery detector of §15.1 (Gap C).
- InterMind encoder, reinterpreted as a reader of internal state, not a message encoder.
- Growth trajectory from one mind to a hierarchy, extended to allow splits below the top.
- Mind data structure core fields: mind_id, parent, children, depth, forward_model, motor_controller, wellness_evaluator, pe_history.
12Questions the source leaves open
Collected from the source and the later gap analysis; each is stated in the source at the place named.
- Whether broader-scope minds produce the higher confidence on contested handles: "an empirical consequence of the training mechanism, not independently guaranteed at runtime" (Gap B). The per-handle learned softmax temperature is "Open for empirical validation" (gap analysis).
- Whether longer imagination chains make the top mind specialize in longer timescales: "the highest-priority validation target for Many Minds experiments" (Gap H; gap analysis).
- The scope of the coherence channel in a child: marked "scope unclear — Gap" in the source's §2.10 (6.5 here), while its §2.13 (7.7 here) states "body schema correlations for handles in handle_indices only".
- Per-domain PE input to the mastery tracker (Gap C).
- The correlation threshold for domain discovery, initial 0.6, a hyperparameter (Gap D); the self-reading loss weight, initial 0.1 (Gaps G and J); the top mind's imagination depth, initial 16+ (Gap H); the gradient-norm threshold for optimizer eviction, no value given (Gap I).
- Whether imagined wellness change should help drive control transitions (the source's author's note, section 3 here).
- What "wash out" means in cross-simulation attention (the source's author's note, 4.7 here).
- The body of
select_goal()(4.8; a Stream 4 feature, not a Many Minds prerequisite). - Why rising PE raises the higher mind's confidence on a handle but lowers the lower mind's, and how a parent produces commands for handles it stopped commanding at the split (sections 2.5 and 2.11 of the source do not say).
- How g_engage acts: the source says high g_engage raises computation depth and "coordination gain" and reduces "domination by routine lower-level outputs on contested handles", but gives no formula connecting g_engage to the depth, to any gain, or to the blend weights of section 6.2; nor does it say what quantity "capacity beyond the immediate physical present" (section 9) names.
- Which mind is the copy at the apex split. The gap analysis says "The top mind is the last split", but a split gives the child a set of mastered handles (7.3) and the top mind has none; neither document says whether the top mind or the attending mind is the new copy.
- What one unit of the motor controller's depth is (4.6: "depth 1 ... depth 2 up to a maximum"), and whether it is the same quantity as the imagination depth of 4.7; the source uses "depth" for both.
13The specification and the released code
What the released Python code under Source/Python contains for each component, from files opened for this edition. A search of the whole tree for MindHierarchy, MasteryDetector, PerMindIntermindEncoder, SensoryBus, MotorArbitrator, handle_set_to_sensor_scope, find_mastered_domains, clone_for_domain and g_engage found none of them: the hierarchy, splitting, the blend and attending engagement are not implemented, which matches the single-mind spec v3.5 §18 ("NOT IMPLEMENTED").
| Component | Released module | What the file shows, and differences from the spec |
|---|---|---|
| Inter-mind encoder (5.6) | lem/model/encoders/self_attention.py | SelfAttentionPoolEncoder: input [B, n_agents, d_model+3], self-attention layers, then a learned pool query; output [B, 1, d_output]. No identity-embedding query and no projection of a last hidden layer, so it is not the source's PerMindIntermindEncoder. |
| Inter-mind input | tokenizer/modalities/intermind.py; mimo_bridge/frame_encoder.py (line 246); lem/master_config.yaml | The tokenizer is marked "STUB" and returns a zero vector of 3 numbers; the MIMo frame encoder sets intermind to zeros(3). The config has data_source: synthetic, loss_weight: 0.0, n_agents: 5. Self-reading (loss weight 0.1, Gaps G and J) is not in place. |
| Intermind slot in MIMo runs | mimo_bridge/lem_model.py (lines 752–754) | With the vestibular setting (--vestibular-sense) switched on, "intermind is switched off and a 'vestibular' modality is added, which the resolver places in intermind's slot (index 5 on every bridge run)". The source's sixth token is then the vestibular token, not intermind. |
| Mastery tracking (7.2) | lem/training/curriculum.py (class at line 133) | MasteryTracker with window 50,000, PE threshold 0.05, variance threshold 0.01, consistency 0.95, as the source says. record(mean_pe: float) takes one scalar, so there is no per-handle PE, as the source says. The source cites lines 111–178; in the released file the class starts at line 133. |
| Body schema (4.9, 7.3) | lem/model/body_schema.py | Buffer correlation_matrix of shape [n_handles, n_handles × n_proprio_per_handle], defaults n_handles 24 and n_proprio_per_handle 11: the input Gap D clusters on exists. |
| Handle-to-body-set map (7.7) | lem/model/body_sets.py; mimo_bridge/body_layout.py; mimo_bridge/proprio_adapter.py (lines 166–167) | 10 body sets and 24 handles (the source's 23 plus gaze, mapped to the Head body set). The MIMo layout trunk_v1 adds lower_trunk and chest for 26 handles; the proprioception adapter assigns those two to the Spine body set. handle_set_to_sensor_scope is not present. |
| Motor controller and confidence (4.6, 6.2) | lem/model/motor_controller.py | Output 9 numbers per handle; a confidence head with a sigmoid output of shape [B, 1], one confidence per body sample, not per handle; adaptive depth with a learned threshold (code default for its initial value: 0.7). The wellness gradient is an input, replaced by zeros when absent. The handle count comes from config["body"]["n_handles"]; whether a zero-handle top mind would run was not tested for this edition. |
| Wellness evaluators (4.4) | lem/model/wellness.py; lem/model/wellness_channels.py; lem/master_config.yaml | GroundedEvaluator and ApproximateEvaluator (config n_outputs: 5, "One per grounded evaluator"). The MIMo path adds further channels (controllability, learning progress) and a normalized ranking used to choose the desired state that the planner search works toward; these are not in this spec. |
| Intention (4.8) | lem/model/intention_generator.py | IntentionGenerator (class at line 149) with default n_steps 50 and lr 0.1; plan_multistep at line 436; CrossSimulationAttention at line 524. The source's line range 78–189 does not match the released file. |
| Curiosity (4.8) | lem/model/curiosity.py | compute_drive returns a value in [0.0, 1.0]: 0 when bored (PE at or below 0.1) or overwhelmed (PE at or above 0.4), peaking at 1.0 at the midpoint. The source's §1.1 says the drive is negative outside the band. No select_goal exists in lem or mimo_bridge. |
| Many Minds boundary contracts | lem/model/self_sharing.py | "Many-Minds boundary: share completed percepts, count physical functioning once." read_completed refuses any report whose time is not earlier than now ("inter-mind reading requires a completed earlier perception") and detaches it. This differs from section 5.4's same-model-step influence. unique_physical_function counts each physical site of a body once per time and raises an error when two minds disagree about it; the source's Wsys sums every mind's evaluation, so a site inside several minds' scope would count several times. |
| Sizes (4.11) | lem/master_config.yaml | d_model: 768, context_length: 8, forward model n_layers: 8, imagination steps: 3 and loss_weight: 0.5, n_handles: 24. |
14The two gap-analysis copies
The gap analysis exists as manyMindGapAnalysis.docx and manyMindGapAnalysis (4).docx. Both carry the same title page (status "In progress", March 2026) and the same figure. The copy named "(4)" is the later one: its internal file dates are March 30, 2026 against March 29; its wording is the wording of this source and of single-mind spec v3.5 where the two copies differ (g_engage, motor-to-proprioception correlation, the Adam thaw caveat); and it is the longer copy. This page uses the later copy.
The gap analysis names two products, ManyMinds_Spec_v3.md and UEMA_Spec_v5.docx; neither is in the released specification folder.
15Glossary
| Term | Code or source name | Definition |
|---|---|---|
| adaptive computation depth | The motor controller uses depth 1 (the fast path, about 1 ms) when confident and depth 2 up to a maximum, with imagination, when not. | |
| wellness predictor | ApproximateEvaluator (source: approximate wellness evaluator) | A learned MLP on sensory tokens, trained to match the wellness evaluators; scores imagined states. |
| attending mind | The mind under the top mind that coordinates the body in the present situation. | |
| attending engagement | g_engage | sigmoid(MLP([latents_attend, z_inter_from_top, pe_attend, approx_wellness_attend])), a number in [0, 1]: the attending mind's engagement signal (6.3). |
| body schema | DifferentiableBodySchema | EMA correlation of motor changes with sensor changes; gives self-membership and the coherence score. |
| body set | BodySet (lem/model/body_sets.py) | A named group of handles, such as Arm_L or Hand_L; 10 in the source. |
| child | The new mind made by a split. | |
| command | source: motor commands | Per handle, position (3), rotation quaternion (4), radius (1), strength (1). |
| confidence head | confidence_head | Part of the motor controller: pooled latents → GELU → sigmoid → [0, 1]. |
| confidence-weighted blend | source: MotorArbitrator (v3.5) | Per handle, softmax of the competing minds' confidences divided by a temperature, used to weight their commands. |
| context buffer | context_length | The tokens of past model steps that the forward model attends over; 8 now, 32 target. |
| cross-simulation attention | CrossSimulationAttention | Attention across several noisy imagination rollouts to find what they share (future). |
| curiosity drive | CuriositySystem.compute_drive | Positive for PE between 0.1 and 0.4 and negative outside that band (source); the released code returns 0 outside it (section 13). |
| d_model | The number of values in a token; 768 in the specification. | |
| domain | The set of handles (and, through them, sensors) a mind works over. | |
| EMA | Exponential moving average. | |
| forward model | ForwardModel | 8 three-stage transformer blocks over the tokens and the context buffer; predicts the next tokens. |
| Grounded Register | The single-mind spec's name for raw physical state, Pt. | |
| wellness evaluators | GroundedEvaluator (source: grounded wellness evaluators) | The five formulas that compute the wellness channels from raw state; the authoritative wellness signal. |
| gap | The source's name for one open design question, lettered A to K; "resolved" means a mechanism is specified, not built. | |
| handle | source: PH handle | One controllable target on the body. |
| imagination | predict_from_action, imagine_step | Running the forward model on candidate commands without sending them to the body. |
| imagination depth | The number of chained imagined steps: 3–5 for most minds, 16+ initial for the top mind. | |
| planner search | IntentionGenerator, plan_multistep | Gradient descent on the command through the forward model toward a desired state: 50 steps, learning rate 0.1. |
| inter-mind channel | The path by which each mind's projected state reaches the other minds as z_intermind. | |
| last hidden layer | last_step, get_last_hidden() | The output of the forward model's final block, before the prediction heads; the mind's state. |
| mastery detector | MasteryDetector (spec), MasteryTracker (code) | Mean PE < 0.05 and PE variance < 0.01 in at least 95% of a 50,000-step window. |
| mind | Mind | One complete copy of the core loop with its own weights and domain. |
| model step | source: frame | One update of the core loop; 60 per second in the source, 50 ms in the released MIMo code. |
| motor controller | AdaptiveMotorController (spec), MotorController (code) | Turns pooled latents, handle state and the wellness gradient into a command and a confidence. |
| optimizer-state eviction | Removing a mastered mind's Adam state from GPU memory; recreated fresh when gradients return. | |
| parent | The mind a split copies from; keeps its full-scope versions. | |
| prediction error (PE) | pe | The difference between predicted and actual tokens. |
| prediction heads | PredictionHead | Five networks (query + attention + MLP) predicting the next token of each sense except motor. |
| radius | The command number, 0 to 1, that sets how far one handle's command spreads to nearby joints. | |
| readable internal state | intermind_state | A mind's last hidden layer passed through a learned linear layer to [d_model] numbers. |
| ROM | Range of motion of a joint. | |
| self-reading | Before any split, the single mind feeds its own projected state through the inter-mind channel. | |
| sensor scope | handle_set_to_sensor_scope, SensorScope | The body sets, bones and sensor entries that belong to a handle set. |
| sense | source: modality | vision, audio, tactile, proprioception, motor, intermind. |
| split | MindHierarchy.split (v3.5) | Copying a parent into a child whose domain is a mastered set of handles. |
| strength | The command number, 0 to 1, that sets how hard a handle pulls. | |
| system gradient | system_grad | autograd of Wsys with respect to the command; trains every motor controller. |
| thaw | The return of nonzero gradients in a mastered mind when its PE rises. | |
| token | z_vis, z_aud, … | A vector of d_model numbers produced by one encoder for one sense. |
| top mind | The mind with no handles, the broadest scope and the longest imagination; the last split. | |
| VICReg, InfoNCE | Two losses that keep encoders from collapsing to identical outputs. | |
| wellness channel | prediction_error, tactile_comfort, capability, coherence, motor_state | One term of the wellness sum. |
| wellness gradient | wellness_grad | autograd of total wellness with respect to the command: [B, n_handles, 9]. |
| Wsys | W_sys | The sum of all minds' wellness evaluations. |
| z_intermind | z_inter | The token one mind extracts from all minds' readable states by cross-attention. |
16Changes in this edition
Reordered
- The source's sections 2.1–2.13 and its gap entries are regrouped by topic: what minds share (5), combining commands (6), splitting (7), freezing (8), the apex (9). Each resolved gap now sits with the established idea it completes: Gap A with 2.9 (5.5), Gap B with 2.5 (6.2), Gap E after 2.5.1 (6.4), Gaps C, D, H, G/J and K with splitting (7), Gap I with 2.8 (8), Gap F as section 9.
- Section 2.9 (what the state contains) now precedes 2.4 (per-mind filtering). The source's section 2.13, placed before 2.12, is section 7.7.
Merged
- The Gap B text and the gap analysis's Gap B text, which repeat each other, appear once (6.2), with the gap analysis's additions in its box.
- The three statements that the child is a full deepcopy keeping all 6 inputs (2.11, Gap D, Gap H) are kept once in 7.5 and once, with the reason, in 7.6.
- The source's duplicate entry "Gap J — RESOLVED (merged with Gap G)" is folded into 7.8.
Left out
Passages that describe a body which is not part of this release are left out: the last two stages of the source's core loop (its output path to that body), the type names and the 23-row handle table of section 2.13, the handle number in Gap D, and the gap analysis's Figure 2.1, which is redrawn in 6.2. The rules for each sense are kept. The released handle-to-body-set map is lem/model/body_sets.py.
Added
- The setting, the source table, section 1 (relation to single-mind UEMA v3.5), the word list of section 2, section 10 (gap register), section 12 (open questions collected), section 13 (released code), section 14 (gap-analysis copies), the glossary.
- Eight diagrams: the whole architecture, one mind's loop, the inter-mind channel, the two stages of Gap B (redrawn), attending engagement, the split pipeline, the two hierarchy shapes, freezing and thawing.
- Input / computation / output blocks for the encoders, forward model, prediction heads, wellness predictor, wellness gradient, motor controller, planner search, body schema, per-mind filtering, domain discovery and optimizer eviction, built from the source's own statements.
- The "Gap analysis" boxes, from the later copy of the gap analysis.