From 72f0afcf7588695f887af45bb3f4e0f53a3c9bb9 Mon Sep 17 00:00:00 2001 From: John Stachurski Date: Tue, 4 Aug 2026 05:16:56 +1000 Subject: [PATCH] Caption the figures in prob_dist and observed_distributions The manual asks every code-generated figure to carry a caption via mystnb metadata. prob_dist had 3 of 20 and observed_distributions none of 20, the latter being an oversight in #811. Adds 35 captions with fig: names for numref, following the manual: sentence case, six words or fewer, descriptive names. Two figures are deliberately left bare, one in each lecture. Both sit inside solution directives, where a caption wraps the image in a LaTeX float and breaks the PDF build with "Not in outer par mode". Also fixes a bug found while captioning. The CDF figure in the lognormal section was built from scipy.stats.norm rather than scipy.stats.lognorm, so it had been drawing normal CDFs under a lognormal heading, and it looped over sigma alone with mu pinned to 1 while the density figure above it varied both. It now uses lognorm over the same (mu, sigma) pairs as that figure, so the two are a matched pair and the curves start at zero as a distribution on the positive half-line should. Co-Authored-By: Claude Opus 5 (1M context) --- lectures/observed_distributions.md | 114 +++++++++++++++++++++++++++++ lectures/prob_dist.md | 102 +++++++++++++++++++++++++- 2 files changed, 212 insertions(+), 4 deletions(-) diff --git a/lectures/observed_distributions.md b/lectures/observed_distributions.md index 0fa49e0b..bd6a1d4a 100644 --- a/lectures/observed_distributions.md +++ b/lectures/observed_distributions.md @@ -328,6 +328,12 @@ We will cover We can histogram the income distribution we just constructed as follows ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Histogram of the income data + name: fig:income-hist +--- fig, ax = plt.subplots() ax.hist(x, bins=5, density=True, histtype='bar') ax.set_xlabel('income') @@ -338,6 +344,12 @@ plt.show() Here is a histogram of the Ames house prices. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Histogram of house prices + name: fig:price-hist +--- fig, ax = plt.subplots() ax.hist(price, bins=50, density=True) ax.set_xlabel('sale price (US$)') @@ -350,6 +362,12 @@ The long right tail that the skewness told us about is clearly visible. Let's compare this with the histogram of the log prices. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Histogram of log house prices + name: fig:log-price-hist +--- fig, ax = plt.subplots() ax.hist(log_price, bins=50, density=True) ax.set_xlabel('log of sale price') @@ -362,6 +380,12 @@ The second histogram is far more symmetric, as the sample skewness led us to exp Here is the age at death data, which we found to have negative skewness. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Histogram of age at death + name: fig:age-hist +--- fig, ax = plt.subplots() ax.hist(age_at_death, bins=101, density=True) ax.set_xlabel('age at death') @@ -375,6 +399,12 @@ Let's also compare men and women, using the sex-specific counts in the data set. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Age at death by sex + name: fig:age-hist-sex +--- fig, ax = plt.subplots() for sex in ('male', 'female'): ax.hist(deaths['age'], weights=deaths[f'deaths_{sex}'], @@ -422,6 +452,12 @@ x_amazon.iloc[0] Let's turn the return observations into an array and histogram it. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Histogram of Amazon monthly returns + name: fig:amazon-hist +--- fig, ax = plt.subplots() ax.hist(x_amazon, bins=20) ax.set_xlabel('monthly return (percent change)') @@ -460,6 +496,12 @@ def plot_ecdf(sample, ax, **kwargs): Let's apply it to the house price data. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: ECDF of house prices + name: fig:price-ecdf +--- fig, ax = plt.subplots() plot_ecdf(price, ax) ax.set_xlabel('sale price (US$)') @@ -477,6 +519,12 @@ Let's compare the log prices with the CDF of the normal distribution that has the same mean and standard deviation. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Log prices against a normal CDF + name: fig:log-price-ecdf +--- u = scipy.stats.norm(log_price.mean(), log_price.std()) x_grid = np.linspace(log_price.min(), log_price.max(), 200) @@ -504,6 +552,12 @@ histogram. Let's have a look at a KDE formed from the Amazon return data. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: KDE of Amazon monthly returns + name: fig:amazon-kde +--- fig, ax = plt.subplots() sns.kdeplot(x_amazon, ax=ax) ax.set_xlabel('monthly return (percent change)') @@ -514,6 +568,12 @@ plt.show() The smoothness of the KDE is dependent on how we choose the bandwidth. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: KDE at three different bandwidths + name: fig:amazon-kde-bw +--- fig, ax = plt.subplots() sns.kdeplot(x_amazon, ax=ax, bw_adjust=0.1, alpha=0.5, label="bw=0.1") sns.kdeplot(x_amazon, ax=ax, bw_adjust=0.5, alpha=0.5, label="bw=0.5") @@ -534,6 +594,12 @@ together. Here is the log sale price data, with the histogram faded into the background. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: KDE and histogram of log prices + name: fig:log-price-kde +--- fig, ax = plt.subplots() ax.hist(log_price, bins=50, density=True, alpha=0.25, color='C0') sns.kdeplot(log_price, ax=ax, color='C0', lw=2) @@ -565,6 +631,12 @@ For example, let's compare house prices across houses with different numbers of bedrooms. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: House prices by number of bedrooms + name: fig:price-box-bedrooms +--- bedroom_counts = (1, 2, 3, 4, 5) groups = [price[houses['bedrooms'] == b] for b in bedroom_counts] @@ -638,6 +710,12 @@ Floor area is clearly the better predictor. Another way to display an observed distribution is via a violin plot. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Violin plot of Amazon returns + name: fig:amazon-violin +--- fig, ax = plt.subplots() ax.violinplot(x_amazon) ax.set_ylabel('monthly return (percent change)') @@ -658,6 +736,12 @@ x_costco = prices.pct_change()[1:] * 100 ``` ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Amazon and Costco monthly returns + name: fig:retailer-violin +--- fig, ax = plt.subplots() ax.violinplot([x_amazon['AMZN'], x_costco['COST']]) ax.set_ylabel('monthly return (percent change)') @@ -672,6 +756,12 @@ As a second comparison, let's return to the age at death data and separate men from women. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Age at death, male and female + name: fig:age-violin +--- male_deaths = np.repeat(deaths['age'], deaths['deaths_male']) female_deaths = np.repeat(deaths['age'], deaths['deaths_female']) @@ -716,6 +806,12 @@ u = scipy.stats.norm(μ, σ) ``` ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Amazon returns and a fitted normal + name: fig:amazon-normal-fit +--- x_grid = np.linspace(-50, 65, 200) fig, ax = plt.subplots() ax.plot(x_grid, u.pdf(x_grid)) @@ -737,6 +833,12 @@ Let's see this in action - then we histogram them and compare with the density. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Draws from the normal distribution + name: fig:normal-draws +--- μ, σ = 0, 1 u = scipy.stats.norm(μ, σ) N = 2000 # Number of observations @@ -769,6 +871,12 @@ We draw samples of increasing size from a fixed distribution and compare each ECDF with the CDF that generated it. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: ECDFs converge to the true CDF + name: fig:ecdf-convergence +--- u = scipy.stats.lognorm(s=0.5) x_grid = np.linspace(0, 5, 200) @@ -820,6 +928,12 @@ Judged one at a time, these are perfectly good observations. But they are useless as a sample, as the next figure shows. ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Repeated draws of a single value + name: fig:ecdf-degenerate +--- x = u.rvs(random_state=1234) # a single draw fig, ax = plt.subplots() diff --git a/lectures/prob_dist.md b/lectures/prob_dist.md index d4936fa1..83c8a42b 100644 --- a/lectures/prob_dist.md +++ b/lectures/prob_dist.md @@ -294,6 +294,12 @@ u.pmf(2) Here's a plot of the probability mass function: ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: PMF of the uniform distribution + name: fig:uniform-pmf +--- fig, ax = plt.subplots() S = np.arange(1, n+1) ax.plot(S, u.pmf(S), linestyle='', marker='o', alpha=0.8, ms=4) @@ -307,6 +313,12 @@ plt.show() Here's a plot of the CDF: ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: CDF of the uniform distribution + name: fig:uniform-cdf +--- fig, ax = plt.subplots() S = np.arange(1, n+1) ax.step(S, u.cdf(S)) @@ -410,6 +422,12 @@ u.pmf(1) ``` ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: PMF of the binomial distribution + name: fig:binomial-pmf +--- fig, ax = plt.subplots() S = np.arange(1, n+1) ax.plot(S, u.pmf(S), linestyle='', marker='o', alpha=0.8, ms=4) @@ -423,6 +441,12 @@ plt.show() Here's the CDF: ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: CDF of the binomial distribution + name: fig:binomial-cdf +--- fig, ax = plt.subplots() S = np.arange(1, n+1) ax.step(S, u.cdf(S)) @@ -491,6 +515,12 @@ u.mean(), u.var() Here's part of the PMF: ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: PMF of the geometric distribution + name: fig:geometric-pmf +--- fig, ax = plt.subplots() n = 20 S = np.arange(n) @@ -529,6 +559,12 @@ u.pmf(1) ``` ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: PMF of the Poisson distribution + name: fig:poisson-pmf +--- fig, ax = plt.subplots() S = np.arange(1, n+1) ax.plot(S, u.pmf(S), linestyle='', marker='o', alpha=0.8, ms=4) @@ -639,6 +675,12 @@ The median equals the mean because the density is symmetric. Here's a plot of the density --- the famous "bell-shaped curve": ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Densities of the normal distribution + name: fig:normal-pdf +--- μ_vals = [-1, 0, 1] σ_vals = [0.4, 1, 1.6] fig, ax = plt.subplots() @@ -658,6 +700,12 @@ plt.show() Here's a plot of the CDF: ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: CDFs of the normal distribution + name: fig:normal-cdf +--- fig, ax = plt.subplots() for μ, σ in zip(μ_vals, σ_vals): u = scipy.stats.norm(μ, σ) @@ -715,6 +763,12 @@ u.mean(), u.ppf(0.5) ``` ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Densities of the lognormal distribution + name: fig:lognormal-pdf +--- μ_vals = [-1, 0, 1] σ_vals = [0.25, 0.5, 1] x_grid = np.linspace(0, 3, 200) @@ -732,15 +786,19 @@ plt.show() ``` ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: CDFs of the lognormal distribution + name: fig:lognormal-cdf +--- fig, ax = plt.subplots() -μ = 1 -for σ in σ_vals: - u = scipy.stats.norm(μ, σ) +for μ, σ in zip(μ_vals, σ_vals): + u = scipy.stats.lognorm(σ, scale=np.exp(μ)) ax.plot(x_grid, u.cdf(x_grid), alpha=0.5, lw=2, label=rf'$\mu={μ}, \sigma={σ}$') ax.set_ylim(0, 1) - ax.set_xlim(0, 3) ax.set_xlabel('x') ax.set_ylabel('CDF') plt.legend() @@ -774,6 +832,12 @@ u.mean(), u.var() ``` ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Densities of the exponential distribution + name: fig:exponential-pdf +--- fig, ax = plt.subplots() λ_vals = [0.5, 1, 2] x_grid = np.linspace(0, 6, 200) @@ -790,6 +854,12 @@ plt.show() ``` ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: CDFs of the exponential distribution + name: fig:exponential-cdf +--- fig, ax = plt.subplots() for λ in λ_vals: u = scipy.stats.expon(scale=1/λ) @@ -834,6 +904,12 @@ u.mean(), u.var() ``` ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Densities of the beta distribution + name: fig:beta-pdf +--- α_vals = [0.5, 1, 5, 25, 3] β_vals = [3, 1, 10, 20, 0.5] x_grid = np.linspace(0, 1, 200) @@ -851,6 +927,12 @@ plt.show() ``` ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: CDFs of the beta distribution + name: fig:beta-cdf +--- fig, ax = plt.subplots() for α, β in zip(α_vals, β_vals): u = scipy.stats.beta(α, β) @@ -894,6 +976,12 @@ u.mean(), u.var() ``` ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: Densities of the gamma distribution + name: fig:gamma-pdf +--- α_vals = [1, 3, 5, 10] β_vals = [3, 5, 3, 3] x_grid = np.linspace(0, 7, 200) @@ -911,6 +999,12 @@ plt.show() ``` ```{code-cell} ipython3 +--- +mystnb: + figure: + caption: CDFs of the gamma distribution + name: fig:gamma-cdf +--- fig, ax = plt.subplots() for α, β in zip(α_vals, β_vals): u = scipy.stats.gamma(α, scale=1/β)