# Conditional entropy
## Microsim
### Live player
<div class="microsim-player">
<iframe src="https://editor.p5js.org/sciencenibber/full/0Bxx394Yu" width="100%" height="620" frameborder="0" sandbox="allow-scripts allow-same-origin"></iframe>
</div>
<div class="microsim-fallback">
<img src="Microsims/thumbs/Conditional_entropy.png" alt="Conditional_entropy microsim poster" style="width:100%;border:1px solid #4445;border-radius:6px;">
<p><em>Live microsim (desktop) · <a href="https://editor.p5js.org/sciencenibber/sketches/0Bxx394Yu">open sketch in the p5.js editor</a></em></p>
</div>
**Editor URL:** https://editor.p5js.org/sciencenibber/sketches/0Bxx394Yu
**Description (100 words):**
A 4 x 4 joint distribution p(X, Y) the reader edits live with three sliders and a [[Structure|structure]] dropdown. The rho slider continuously tunes the conditional p(Y | X) from uniform (rho = 0, channel destroys all [[Signal|signal]]) to deterministic (rho = 1, Y is a function of X), making the two corner inequalities visible: H(Y | X) <= H(Y) with equality at independence, and H(Y | X) = 0 at perfect [[Coupling|coupling]]. The dark readout panel reports H(X), H(Y), H(X, Y), H(Y | X), H(X | Y), and I(X ; Y) in bits, plus a chain-rule check that prints H(X, Y) - H(X) so the reader can verify it equals H(Y | X) numerically.
```js
// =====================================================================
// Conditional_entropy.js -- Wikitube microsim, Information room
// ---------------------------------------------------------------------
// ARTICLE Conditional_entropy
// ROOM Information
// PATTERN E -- Entropy and information measures (Information sec.10)
// AUTHORED 2026-04-30 (generative pipeline, scheduled run)
//
// PURPOSE
// Visualise the conditional entropy
//
// H(Y | X) = - sum_{x,y} p(x, y) log2 p(y | x)
// = H(X, Y) - H(X)
//
// for a 4 x 4 joint distribution p(X, Y) that the reader edits live.
// The teaching beats:
// 1. H(Y | X) <= H(Y), with equality iff X and Y are independent.
// 2. H(Y | X) = 0 iff Y is a deterministic function of X.
// 3. Chain rule: H(X, Y) = H(X) + H(Y | X) = H(Y) + H(X | Y).
// 4. Mutual information I(X ; Y) = H(Y) - H(Y | X).
//
// The reader controls only three sliders plus a structure menu, but
// between them the four corner cases (independence, determinism, two
// permutations) and everything in between are reachable.
//
// CONTROLS (DOM, top-left of canvas)
// p1 slider P(X = A) range 0.05 .. 0.95
// p2 slider P(X = B | X != A) range 0.05 .. 0.95
// rho slider coupling strength 0..1 0 = uniform Y|X,
// 1 = deterministic Y|X
// structure menu identity / shift / permute picks which Y "channel"
// each X routes to
// reset button snap defaults
//
// READOUTS (HUD bottom-left, dark panel)
// H(X), H(Y), H(X,Y) marginal & joint entropies in bits
// H(Y|X) the headline quantity (highlight)
// H(X|Y) the "other" conditional entropy
// I(X;Y) mutual information
// chain-rule check prints H(X,Y) - H(X) so the reader can
// see it equal H(Y|X) numerically
//
// EQUATION (HUD bottom-right, ASCII)
// H(Y|X) = - sum p(x,y) log2 p(y|x) = H(X,Y) - H(X)
//
// VISUAL ANATOMY
// centre : 4 x 4 joint heatmap p(X, Y) with cell numerics.
// right of : marginal P(X) horizontal bars + numbers.
// below it : marginal P(Y) vertical bars + numbers.
// far right: four mini bar-charts of P(Y | X = x), one per row.
//
// PARAMETER TABLE
// N 4 alphabet size for both X and Y.
// X_LABELS A B C D canonical alphabet for printing.
// Y_LABELS a b c d lowercase = "the receiver's view".
// STRUCT_MAP identity / shift / permute -- the permutation each
// X gets concentrated on as rho -> 1.
// EPS 1e-12 floor for log arguments.
//
// FILES
// Local archive : Articles/Information/Microsims/Conditional_entropy.js
// Editor URL : captured live from window.location.href after save.
// =====================================================================
const ARTICLE = "Conditional_entropy";
p5.disableFriendlyErrors = true;
// ---------- palette (Information room standard, sec.10) -------------
const BG = 246;
const INK = [40, 48, 60];
const BAR = [70, 130, 200];
const EDGE = [120, 130, 150];
const TOKEN = [220, 110, 60];
const ACCEPT= [80, 180, 120];
const LOWP = [200, 205, 215];
// ---------- alphabet ------------------------------------------------
const N = 4;
const X_LABELS = ["A", "B", "C", "D"];
const Y_LABELS = ["a", "b", "c", "d"];
// numerical floor for log2(p): any p < EPS contributes 0 to entropy.
const EPS = 1e-12;
// ---------- DOM controls (created in setup) -------------------------
let p1Slider, p2Slider, rhoSlider, structSel, resetBtn;
let structure = "identity";
function setup() {
createCanvas(windowWidth, windowHeight);
textFont("Helvetica");
textSize(13);
noStroke();
// p1 slider -- mass on X = A. We park it at 0.35 so the default
// joint is non-degenerate (no zero rows) and the marginal entropies
// start above 1.5 bits.
p1Slider = createSlider(0.05, 0.95, 0.35, 0.01);
p1Slider.position(20, 70);
p1Slider.style("width", "200px");
// p2 slider -- conditional mass on X = B given X != A. This is the
// "second knob" for the X marginal; the remaining 1 - p1 - p2 mass
// is split 60/40 between C and D so the marginal stays interesting
// even when the user pegs p1, p2.
p2Slider = createSlider(0.05, 0.95, 0.30, 0.01);
p2Slider.position(20, 110);
p2Slider.style("width", "200px");
// rho slider -- coupling 0..1. At rho=0 every conditional row
// P(Y|X=x) is uniform over {a,b,c,d}, so X tells us nothing about Y
// and H(Y|X) = H(Y) = log2(4) when the marginals are uniform.
// At rho=1 each row collapses onto a single Y value (the structure
// map below) and H(Y|X) = 0 exactly.
rhoSlider = createSlider(0.0, 1.0, 0.65, 0.01);
rhoSlider.position(20, 150);
rhoSlider.style("width", "200px");
// structure dropdown -- which Y each X concentrates on as rho->1.
// identity: A->a, B->b, C->c, D->d (perfect alignment)
// shift: A->b, B->c, C->d, D->a (cyclic shift; same I(X;Y))
// permute: A->c, B->a, C->d, D->b (a non-trivial permutation)
// All three give the same entropy when rho=1; structure changes the
// pictures, not the numbers (a useful teaching beat).
structSel = createSelect();
structSel.position(20, 190);
structSel.option("identity");
structSel.option("shift");
structSel.option("permute");
structSel.changed(() => { structure = structSel.value(); });
// reset button -- snap to a non-degenerate default so the chain rule
// is visibly non-trivial right after a click.
resetBtn = createButton("reset");
resetBtn.position(20, 230);
resetBtn.mousePressed(() => {
p1Slider.value(0.35);
p2Slider.value(0.30);
rhoSlider.value(0.65);
structSel.selected("identity");
structure = "identity";
});
}
// ---------- structure map: which Y does X=i concentrate on? ---------
function permFor(i) {
switch (structure) {
case "identity": return i;
case "shift": return (i + 1) % N;
case "permute": return [2, 0, 3, 1][i];
}
return i;
}
// ---------- distribution builders -----------------------------------
// Marginal P(X) from the two sliders. The "p2 / (1-p1)" framing keeps
// every slider safely inside (0, 1) regardless of the other -- no
// constraint solver, no clamping logic.
function getMarginalX() {
const p1 = p1Slider.value();
const p2 = p2Slider.value() * (1 - p1);
const rem = Math.max(0, 1 - p1 - p2);
// split remaining mass 60/40 between C and D so neither is zero.
const p3 = rem * 0.6;
const p4 = rem * 0.4;
return [p1, p2, p3, p4];
}
// Conditional P(Y | X = i): a "rho-fraction" heaped onto the target
// dictated by the structure map, the rest spread uniformly.
function getConditional(i) {
const rho = rhoSlider.value();
const target = permFor(i);
const row = new Array(N);
for (let j = 0; j < N; j++) {
row[j] = (1 - rho) / N + (j === target ? rho : 0);
}
// already sums to 1 by construction; renormalise to defang round-off.
let s = 0; for (const v of row) s += v;
for (let j = 0; j < N; j++) row[j] /= s;
return row;
}
// Joint P(X, Y) and the derived marginal P(Y).
function getJoint() {
const px = getMarginalX();
const pxy = [];
for (let i = 0; i < N; i++) {
const row = getConditional(i);
pxy.push(row.map(p => p * px[i]));
}
const py = new Array(N).fill(0);
for (let i = 0; i < N; i++) for (let j = 0; j < N; j++) py[j] += pxy[i][j];
return { px, pxy, py };
}
// ---------- entropy primitives (all in bits) ------------------------
function H(p) {
let h = 0;
for (const v of p) if (v > EPS) h -= v * Math.log2(v);
return h;
}
function jointH(pxy) {
let h = 0;
for (let i = 0; i < N; i++) for (let j = 0; j < N; j++) {
const v = pxy[i][j];
if (v > EPS) h -= v * Math.log2(v);
}
return h;
}
// Direct evaluation of H(Y|X) = sum_i p(x_i) * H(Y | X = x_i)
// We compute it the "honest" way (not as H(X,Y) - H(X)) so the chain
// rule check below is a real check, not an identity.
function condHYgX(pxy, px) {
let h = 0;
for (let i = 0; i < N; i++) {
if (px[i] < EPS) continue;
let hRow = 0;
for (let j = 0; j < N; j++) {
const cond = pxy[i][j] / px[i];
if (cond > EPS) hRow -= cond * Math.log2(cond);
}
h += px[i] * hRow;
}
return h;
}
// ---------- main draw loop ------------------------------------------
function draw() {
background(BG);
// pull the live distribution once per frame and pass it down.
const { px, pxy, py } = getJoint();
const HX = H(px);
const HY = H(py);
const HXY = jointH(pxy);
const HYgX = condHYgX(pxy, px);
const HXgY = HXY - HY;
const I = HX + HY - HXY;
// layout constants -- everything is anchored to the canvas size so
// the sketch survives a window resize.
const heatX = 280;
const heatY = 80;
const heatSize = Math.min(360, height - 280, width - 760);
const heatSafe = Math.max(220, heatSize);
drawHeatmap(heatX, heatY, heatSafe, pxy, px, py);
drawConditionalBars(heatX + heatSafe + 130, heatY, 220, heatSafe, pxy, px);
drawReadouts(20, height - 200, HX, HY, HXY, HYgX, HXgY, I);
drawControlLabels();
drawHud();
}
// ---------- the joint heatmap + marginal bars -----------------------
function drawHeatmap(x0, y0, size, pxy, px, py) {
const cell = size / N;
let pmax = 0;
for (let i = 0; i < N; i++) for (let j = 0; j < N; j++) {
pmax = Math.max(pmax, pxy[i][j]);
}
// cells: alpha encodes joint mass, numerical text inside.
for (let i = 0; i < N; i++) {
for (let j = 0; j < N; j++) {
const a = pxy[i][j] / Math.max(EPS, pmax);
noStroke();
fill(BAR[0], BAR[1], BAR[2], 30 + a * 220);
rect(x0 + j * cell, y0 + i * cell, cell - 1, cell - 1, 2);
noStroke(); fill(...INK); textAlign(CENTER, CENTER); textSize(11);
text(pxy[i][j].toFixed(3),
x0 + j * cell + cell / 2,
y0 + i * cell + cell / 2);
}
}
// axis labels: Y across the top, X down the left.
noStroke(); fill(...INK); textSize(13); textAlign(CENTER, BOTTOM);
for (let j = 0; j < N; j++) {
text(Y_LABELS[j], x0 + j * cell + cell / 2, y0 - 4);
}
textAlign(RIGHT, CENTER);
for (let i = 0; i < N; i++) {
text(X_LABELS[i], x0 - 8, y0 + i * cell + cell / 2);
}
textAlign(LEFT, BOTTOM); textSize(12); fill(...EDGE);
text("p(X, Y) joint", x0, y0 - 24);
textAlign(RIGHT, TOP);
text("Y -->", x0 + size, y0 - 24);
textAlign(LEFT, TOP);
text("X", x0 - 22, y0 - 22);
// marginal P(X) bars to the right of the heatmap.
const mxX = x0 + size + 14;
for (let i = 0; i < N; i++) {
noStroke(); fill(...BAR);
rect(mxX, y0 + i * cell + 6, 70 * px[i], cell - 12, 2);
fill(...INK); textAlign(LEFT, CENTER); textSize(11);
text(px[i].toFixed(3), mxX + 76, y0 + i * cell + cell / 2);
}
textAlign(LEFT, BOTTOM); fill(...EDGE); textSize(11);
text("p(X)", mxX, y0 - 4);
// marginal P(Y) bars below the heatmap.
const myY = y0 + size + 14;
for (let j = 0; j < N; j++) {
noStroke(); fill(...BAR);
rect(x0 + j * cell + 6, myY, cell - 12, 70 * py[j], 2);
fill(...INK); textAlign(CENTER, TOP); textSize(11);
text(py[j].toFixed(3), x0 + j * cell + cell / 2, myY + 78);
}
textAlign(LEFT, BOTTOM); fill(...EDGE); textSize(11);
text("p(Y)", x0 + size + 14, myY + 60);
}
// ---------- four mini bar-charts of P(Y | X = x), one per row -------
function drawConditionalBars(x0, y0, w, totalH, pxy, px) {
const rowH = totalH / N;
for (let i = 0; i < N; i++) {
const yTop = y0 + i * rowH;
noStroke(); fill(...INK); textAlign(LEFT, TOP); textSize(11);
text("P(Y | X = " + X_LABELS[i] + ")", x0, yTop);
const cond = [];
for (let j = 0; j < N; j++) {
cond.push(pxy[i][j] / Math.max(EPS, px[i]));
}
const barW = w / N;
const usableH = rowH - 36;
for (let j = 0; j < N; j++) {
const h = usableH * cond[j];
noStroke(); fill(...BAR);
rect(x0 + j * barW + 1,
yTop + 18 + (usableH - h),
barW - 3, h, 2);
fill(...INK); textAlign(CENTER, TOP); textSize(10);
text(Y_LABELS[j], x0 + j * barW + barW / 2, yTop + 18 + usableH + 2);
}
}
}
// ---------- entropy readouts panel ----------------------------------
function drawReadouts(x, y, HX, HY, HXY, HYgX, HXgY, I) {
noStroke(); fill(0, 170); rect(x - 8, y, 260, 196);
fill(255); textAlign(LEFT, TOP); textSize(12);
text("H(X) = " + HX.toFixed(3) + " bits", x, y + 10);
text("H(Y) = " + HY.toFixed(3) + " bits", x, y + 28);
text("H(X,Y) = " + HXY.toFixed(3) + " bits", x, y + 46);
// headline: H(Y|X) in green.
fill(...ACCEPT);
text("H(Y|X) = " + HYgX.toFixed(3) + " bits", x, y + 72);
fill(255);
text("H(X|Y) = " + HXgY.toFixed(3) + " bits", x, y + 90);
// mutual information in orange.
fill(...TOKEN);
text("I(X;Y) = " + I.toFixed(3) + " bits", x, y + 116);
// chain-rule check, computed from independent pieces.
fill(...LOWP); textSize(10);
text("chain rule: H(X,Y) - H(X) =", x, y + 144);
text(" " + (HXY - HX).toFixed(3) + " == H(Y|X)", x, y + 158);
text("(equality is the chain rule of entropy.)", x, y + 174);
}
// ---------- labels for the DOM controls -----------------------------
function drawControlLabels() {
noStroke(); fill(...INK); textAlign(LEFT, TOP); textSize(12);
text("p1 = P(X = A)", 20, 54);
text("p2' = P(X = B | X != A)", 20, 94);
text("rho (coupling 0..1)", 20, 134);
text("structure of conditional", 20, 174);
}
// ---------- HUD: title block, control hints, equation footer --------
function drawHud() {
// top-left title block.
noStroke(); fill(0, 200); rect(8, 8, 380, 26);
fill(255); textSize(13); textAlign(LEFT, TOP);
text(ARTICLE + " :: en.wikitube.io/wiki/" + ARTICLE, 16, 14);
// top-right control hint strip.
fill(0, 160); rect(width - 380, 8, 372, 26);
fill(255); textAlign(LEFT, TOP); textSize(12);
text("sliders: p1, p2', rho | menu: structure | reset",
width - 372, 14);
// bottom-right equation footer.
fill(0, 160); rect(width - 500, height - 32, 492, 26);
fill(255); textAlign(LEFT, TOP); textSize(12);
text("H(Y|X) = - sum p(x,y) log2 p(y|x) = H(X,Y) - H(X)",
width - 492, height - 26);
}
function windowResized() {
resizeCanvas(windowWidth, windowHeight);
}
```
## Links (Wikipedia order)
<!-- injected from _registry/childlinks/Conditional_entropy.json (2026-07-30T02:09:12Z) -->
`Asymptotic_equipartition_property` · `Chain_rule_(probability)` · `Channel_capacity` · `Conditional_independence` · `Conditional_mutual_information` · `Conditional_quantum_entropy` · [[Differential_entropy]] · `Directed_information` · [[Entropy_(information_theory)]] · `Entropy_power_inequality` · `Entropy_rate` · `Estimator` · [[Expected_value]] · `Hartley_(unit)` · `If_and_only_if` · `Information_content` · [[Information_theory]] · [[Joint_entropy]] · `Likelihood_function` · `Limiting_density_of_discrete_points` · `Mutual_information` · [[Nat_(unit)]] · `Noisy-channel_coding_theorem` · `Outcome_(probability)` · `Probability_mass_function` · `Probability_theory` · `Quantities_of_information` · [[Quantum_mechanics]] · `Random_variable` · `Rate–distortion_theory` · `Shannon's_source_coding_theorem` · `Shannon_(unit)` · `Shannon–Hartley_theorem` · `Support_(mathematics)` · `Thomas_M._Cover` · [[Uncertainty_principle]] · `Variation_of_information` · `Venn_diagram` · `Weight_function`
## From the Real GENERATIVE library
*Conditional entropy — placed from the Real G.E.N.E.R.A.T.I.V.E. course library (Information room). Source: Wikimedia Commons (via Wikipedia article media). [Details & license](https://commons.wikimedia.org/wiki/File:Binaryerasurechannel.png).*
> In information theory, the conditional entropy quantifies the amount of information needed to describe the outcome of a random variable Y {\displaystyle Y} given that the value of another random variable X {\displaystyle X} is known. Here, information is measured in shannons, nats, or hartleys. ([Wikipedia](https://en.wikipedia.org/wiki/Conditional_entropy))
<!-- REAL-GENERATIVE-MEDIA:END -->
<!-- LOCAL-MEDIA-PASS:START -->
## From the vault media library
!Conditional entropy thumb.png
*Conditional Entropy — from the vault's own media holdings, placed 2026-07-09. MTN / Wikitube.io original · CC BY-SA 4.0.*
<!-- LOCAL-MEDIA-PASS:END -->
> **Room:** [[Information]] · **Status:** ✅ shipped
## Overview
**Conditional entropy**, written H(Y | X), measures the average uncertainty that remains about a random variable Y once the value of another random variable X has been observed. Where Shannon's [[Entropy|entropy]] H(Y) asks "how surprised should I expect to be by Y in the dark?", conditional entropy asks the sharper question "how surprised should I expect to be by Y given that I already know X?" — and the answer is always less than or equal to H(Y), with equality precisely when X and Y are statistically independent. The defining sum is H(Y | X) = −Σ_{x,y} p(x, y) log p(y | x), equivalent to the chain-rule decomposition H(X, Y) = H(X) + H(Y | X). The complementary quantity H(Y) − H(Y | X) is the mutual information I(X; Y), the bits of Y that observing X resolves on average. Conditional entropy is the bedrock of Shannon's noisy-channel coding theorem, where H(X | Y) plays the role of the channel's equivocation — the residual uncertainty about a transmitted symbol after the receiver has seen the channel output — and the channel capacity is C = max_{p(x)} I(X; Y) = max_{p(x)} [H(Y) − H(Y | X)].
## See also
- Room hub: [[Information]]
- p5.js Editor conventions: P5 JS EDITOR
- Wiki root: MAIN
---
*Scaffolded by `generative-microsim` from row 23 of the Information sheet on 2026-04-30T14:38:02Z.*
Letters: entropy · mined_information · mined_structure · chart_glyph_dictionary · distribution · probability · measurement · kanji_radicals
<!-- REAL-GENERATIVE-MEDIA:START -->
<!-- CRAFT-LINK:START g12 -->
*Built to the [[WT!P5_js_Microsim_Master_Class|p5.js Master Class]].*
<!-- CRAFT-LINK:END -->
## Wikipedia : Wikitube
**Strict pair:** [Wikipedia](https://en.wikipedia.org/wiki/Conditional_entropy) : [Wikitube](https://en.wikitube.io/wiki/Conditional_entropy)
## Previous hub tags
Tree parent: [[Information_theory]].
Legacy hubs: `GENERATIVE`.
---
*Sources: 1 legacy note. Minted wave 1, 2026-07-30 (v1.6 order).*