Bayesian Analysis Resume (emcee): LBCO, HRPT¶
This tutorial shows how to reopen the Bayesian project created previously, inspect the saved fit results and then run more sampling steps to extend the existing chain. Resuming only works with EMCEE because the current BUMPS-DREAM implementation does not support saving and resuming its state.
This workflow is useful when:
- the initial sampling run has not yet converged and more steps are needed,
- the initial sampling run has converged but more steps are desired for better posterior resolution,
- the initial sampling run has converged but the posterior plots have not yet been inspected and the user wants to see the plots before deciding whether to run more steps.
The workflow uses the same La0.5Ba0.5CoO3 powder diffraction example as the DREAM Bayesian tutorial:
- run a short local refinement,
- derive finite fit bounds for the sampled parameters,
- switch to emcee and sample the posterior,
- save the project with the emcee chain,
- resume the chain with additional steps,
- inspect posterior plots after each sampling stage.
🛠️ Import Library¶
In [2]:
Copied!
import easydiffraction as ed
import easydiffraction as ed
📂 Load Project¶
Download Project¶
The returned path points directly to the saved project directory with the completed Bayesian fit and persisted posterior samples and plot caches.
In [3]:
Copied!
project_dir = ed.download_data(id=38, destination='projects')
project_dir = ed.download_data(id=38, destination='projects')
Getting data...
Data #38: Bayesian Analysis (emcee): LBCO, HRPT
✅ Data #38 downloaded and extracted to '../../../projects/ed-38/lbco_hrpt_emcee'
Load Project¶
Loading restores the persisted fit state, posterior samples, and plot caches. No new fit is launched in this tutorial.
In [4]:
Copied!
project = ed.Project.load(project_dir)
project = ed.Project.load(project_dir)
⚠️ Switching minimizer type removes these settings: • max_iterations
⚠️ Switching minimizer type adds these settings with defaults: • burn_in_steps=1000 • initialization_method='ball' • parallel_workers=0 • population_size=32 • proposal_moves='de' • random_seed=None • sampling_steps=5000 • thinning_interval=1
📊 Inspect Results¶
Display Structure¶
Render the La0.5Ba0.5CoO3 structure restored from the saved project.
In [5]:
Copied!
project.display.structure(struct_name='lbco')
project.display.structure(struct_name='lbco')
Structure 🧩 'lbco' (Atom view type: 'covalent')
Loading plot…
drag = rotate
wheel = zoom
right-drag = pan
wheel = zoom
right-drag = pan
Display Fit Results¶
The fit summary reports the committed point estimate, sampler settings, convergence diagnostics, and posterior parameter summaries from the saved Bayesian run.
In [6]:
Copied!
project.display.fit.results()
project.display.fit.results()
⚙️ Settings used:
| Name | Value | Description | |
|---|---|---|---|
| 1 | sampling_steps | 10000 | Total sampler iterations per chain. |
| 2 | burn_in_steps | 2000 | Sampler iterations discarded as warm-up. |
| 3 | thinning_interval | 1 | Sampler thinning interval. |
| 4 | population_size | 16 | Number of chains or walkers. |
| 5 | parallel_workers | 0 | Worker count; 0 uses all available CPUs. |
| 6 | initialization_method | ball | emcee walker initialization method. |
| 7 | random_seed | None | Random seed; None uses a system-derived seed. |
| 8 | proposal_moves | de | Single emcee proposal move; move mixtures are not persisted in v1. |
📋 Bayesian fit results:
| Metric | Value | |
|---|---|---|
| 1 | 🧪 Sampler | emcee |
| 2 | ✅ Overall status | success |
| 3 | 💬 Engine message | emcee sampling completed |
| 4 | ⏱️ Fitting time (seconds) | 1147.87 |
| 5 | 📏 Goodness-of-fit (reduced χ²) | 1.29 |
| 6 | 📏 R-factor (Rf, %) | 5.65 |
| 7 | 📏 R-factor squared (Rf², %) | 4.92 |
| 8 | 📏 Weighted R-factor (wR, %) | 4.08 |
| 9 | 📉 Best log-posterior | -1157.02 |
| 10 | 📊 Convergence status | passed |
| 11 | 📊 Max r-hat | 1.002 |
| 12 | 📊 Min ess bulk | 8810.454 |
| 13 | 📊 Draws per chain | 10001 |
| 14 | 📊 Chains | 16 |
📈 Committed parameters:
| datablock | category | entry | parameter | units | start | value | s.u. | change | |
|---|---|---|---|---|---|---|---|---|---|
| 1 | lbco | cell | length_a | Å | 3.8913 | 3.8913 | 0.0001 | 0.00 % ↓ | |
| 2 | hrpt | linked_phases | lbco | scale | 9.1330 | 9.1330 | 0.0291 | 0.00 % ↑ | |
| 3 | hrpt | peak | broad_gauss_u | deg² | 0.0816 | 0.0816 | 0.0066 | 0.01 % ↑ | |
| 4 | hrpt | peak | broad_gauss_v | deg² | -0.1169 | -0.1169 | 0.0047 | 0.04 % ↑ | |
| 5 | hrpt | instrument | twotheta_offset | deg | 0.6303 | 0.6303 | 0.0017 | 0.00 % ↑ |
• start = parameter value before sampling
• value = estimate written back to the project (best posterior sample)
• s.u. = standard uncertainty (one sigma), posterior standard deviation
• change = relative change from start, in %; ↑ = increase, ↓ = decrease
• value = estimate written back to the project (best posterior sample)
• s.u. = standard uncertainty (one sigma), posterior standard deviation
• change = relative change from start, in %; ↑ = increase, ↓ = decrease
📊 Posterior distribution:
| datablock | category | entry | parameter | units | median | 95% CI | r-hat | ess bulk | |
|---|---|---|---|---|---|---|---|---|---|
| 1 | lbco | cell | length_a | Å | 3.8913 | [3.8911, 3.8915] | 1.002 | 8991.7 | |
| 2 | hrpt | linked_phases | lbco | scale | 9.1320 | [9.0753, 9.1891] | 1.002 | 8810.5 | |
| 3 | hrpt | peak | broad_gauss_u | deg² | 0.0816 | [0.0686, 0.0945] | 1.002 | 8866.8 | |
| 4 | hrpt | peak | broad_gauss_v | deg² | -0.1168 | [-0.1260, -0.1075] | 1.002 | 8888.9 | |
| 5 | hrpt | instrument | twotheta_offset | deg | 0.6303 | [0.6269, 0.6336] | 1.002 | 9044.0 |
• median = 50th percentile of the marginal posterior
• 95% CI = 95% credible interval (2.5%-97.5%, asymmetric)
• r-hat = Gelman-Rubin diagnostic (good convergence: r-hat <= 1.01)
• ess bulk = bulk effective sample size (typically >= 400)
• 95% CI = 95% credible interval (2.5%-97.5%, asymmetric)
• r-hat = Gelman-Rubin diagnostic (good convergence: r-hat <= 1.01)
• ess bulk = bulk effective sample size (typically >= 400)
Display Correlations¶
The correlation matrix is restored from the saved project state.
In [7]:
Copied!
project.display.fit.correlations()
project.display.fit.correlations()
Loading plot…
Display Posterior Densities¶
The pair plot and one-dimensional posterior distributions now load from the persisted caches generated when the Bayesian fit was saved.
In [8]:
Copied!
project.display.posterior.pairs()
project.display.posterior.pairs()
Loading plot…
In [9]:
Copied!
project.display.posterior.distribution()
project.display.posterior.distribution()
Loading plot…
Loading plot…
Loading plot…
Loading plot…
Loading plot…
Display Posterior Predictive¶
The posterior predictive view reuses the cached predictive summary stored in the project rather than recalculating it on first display. It overlays the 95% credible interval propagated from the posterior samples.
In [10]:
Copied!
project.display.posterior.predictive(expt_name='hrpt')
project.display.posterior.predictive(expt_name='hrpt')
Loading plot…
A zoomed view is useful for checking the propagated uncertainty in a narrow region of the diffraction pattern.
In [11]:
Copied!
project.display.posterior.predictive(expt_name='hrpt', x_min=92, x_max=93)
project.display.posterior.predictive(expt_name='hrpt', x_min=92, x_max=93)
Loading plot…
🎲 Resume Sampling¶
Run Sampling¶
Resume from the saved backend and append 100 more emcee steps to the existing chain. We use only 100 steps here to keep the tutorial fast, but in practice you would typically run more steps to ensure convergence and better posterior resolution.
In [12]:
Copied!
project.analysis.minimizer.random_seed = 42 # fixed seed for reproducible output
project.analysis.fit(resume=True, extra_steps=100)
project.analysis.minimizer.random_seed = 42 # fixed seed for reproducible output
project.analysis.fit(resume=True, extra_steps=100)
Standard fitting
📋 Using experiment 🔬 'hrpt' for 'single' fitting
🚀 Starting fit process with 'emcee'...
📈 Bayesian sampling progress:
| iteration | progress | time (s) | log posterior | phase | |
|---|---|---|---|---|---|
| 1 | 1/100 | 0.45 | -1157.13 | pre-processing | |
| 2 | 5/100 | 5.0% | 1.75 | -1157.31 | sampling |
| 3 | 9/100 | 9.0% | 3.05 | -1157.91 | sampling |
| 4 | 13/100 | 13.0% | 4.42 | -1157.37 | sampling |
| 5 | 18/100 | 18.0% | 6.09 | -1157.73 | sampling |
| 6 | 22/100 | 22.0% | 7.40 | -1157.89 | sampling |
| 7 | 26/100 | 26.0% | 8.73 | -1157.79 | sampling |
| 8 | 30/100 | 30.0% | 10.06 | -1157.85 | sampling |
| 9 | 34/100 | 34.0% | 11.41 | -1157.76 | sampling |
| 10 | 38/100 | 38.0% | 12.71 | -1157.76 | sampling |
| 11 | 42/100 | 42.0% | 14.08 | -1157.76 | sampling |
| 12 | 46/100 | 46.0% | 15.40 | -1157.41 | sampling |
| 13 | 50/100 | 50.0% | 16.73 | -1157.61 | sampling |
| 14 | 55/100 | 55.0% | 18.32 | -1157.40 | sampling |
| 15 | 59/100 | 59.0% | 19.70 | -1157.40 | sampling |
| 16 | 63/100 | 63.0% | 21.43 | -1157.40 | sampling |
| 17 | 67/100 | 67.0% | 23.89 | -1157.71 | sampling |
| 18 | 71/100 | 71.0% | 25.28 | -1157.44 | sampling |
| 19 | 75/100 | 75.0% | 26.61 | -1157.99 | sampling |
| 20 | 79/100 | 79.0% | 27.96 | -1157.27 | sampling |
| 21 | 84/100 | 84.0% | 29.67 | -1157.19 | sampling |
| 22 | 88/100 | 88.0% | 31.02 | -1157.19 | sampling |
| 23 | 92/100 | 92.0% | 32.38 | -1157.19 | sampling |
| 24 | 96/100 | 96.0% | 33.74 | -1157.19 | sampling |
| 25 | 100/100 | 100.0% | 35.06 | -1157.30 | sampling |
| 26 | 12101/12101 | 100.0% | 35.36 | -1157.02 | sampling |
| 27 | 97.65 | post-processing |
✅ Bayesian sampling complete.
Saving project 📦 'lbco_hrpt_emcee' to '../../../projects/ed-38/lbco_hrpt_emcee'
├── 📄 project.cif
├── 📁 structures/
│ └── 📄 lbco.cif
├── 📁 experiments/
│ └── 📄 hrpt.cif
├── 📁 analysis/
│ ├── 📄 analysis.cif
│ └── 📄 results.h5
└── 📁 reports/
└── 📄 lbco_hrpt_emcee.html
In [13]:
Copied!
project.display.fit.results()
project.display.fit.results()
⚙️ Settings used:
| Name | Value | Description | |
|---|---|---|---|
| 1 | sampling_steps | 10000 | Total sampler iterations per chain. |
| 2 | burn_in_steps | 2000 | Sampler iterations discarded as warm-up. |
| 3 | thinning_interval | 1 | Sampler thinning interval. |
| 4 | population_size | 16 | Number of chains or walkers. |
| 5 | parallel_workers | 0 | Worker count; 0 uses all available CPUs. |
| 6 | initialization_method | ball | emcee walker initialization method. |
| 7 | random_seed | 42 | Random seed; None uses a system-derived seed. |
| 8 | proposal_moves | de | Single emcee proposal move; move mixtures are not persisted in v1. |
📋 Bayesian fit results:
| Metric | Value | |
|---|---|---|
| 1 | 🧪 Sampler | emcee |
| 2 | ✅ Overall status | success |
| 3 | 💬 Engine message | emcee sampling completed |
| 4 | ⏱️ Fitting time (seconds) | 97.65 |
| 5 | 📏 Goodness-of-fit (reduced χ²) | 1.29 |
| 6 | 📏 R-factor (Rf, %) | 5.65 |
| 7 | 📏 R-factor squared (Rf², %) | 4.91 |
| 8 | 📏 Weighted R-factor (wR, %) | 4.07 |
| 9 | 📉 Best log-posterior | -1157.02 |
| 10 | 📊 Convergence status | passed |
| 11 | 📊 Max r-hat | 1.002 |
| 12 | 📊 Min ess bulk | 8980.299 |
| 13 | 📊 Draws per chain | 10101 |
| 14 | 📊 Chains | 16 |
📈 Committed parameters:
| datablock | category | entry | parameter | units | start | value | s.u. | change | |
|---|---|---|---|---|---|---|---|---|---|
| 1 | lbco | cell | length_a | Å | 3.8913 | 3.8913 | 0.0001 | 0.00 % ↑ | |
| 2 | hrpt | linked_phases | lbco | scale | 9.1330 | 9.1292 | 0.0291 | 0.04 % ↓ | |
| 3 | hrpt | peak | broad_gauss_u | deg² | 0.0816 | 0.0811 | 0.0066 | 0.57 % ↓ | |
| 4 | hrpt | peak | broad_gauss_v | deg² | -0.1169 | -0.1163 | 0.0047 | 0.47 % ↓ | |
| 5 | hrpt | instrument | twotheta_offset | deg | 0.6303 | 0.6304 | 0.0017 | 0.02 % ↑ |
• start = parameter value before sampling
• value = estimate written back to the project (best posterior sample)
• s.u. = standard uncertainty (one sigma), posterior standard deviation
• change = relative change from start, in %; ↑ = increase, ↓ = decrease
• value = estimate written back to the project (best posterior sample)
• s.u. = standard uncertainty (one sigma), posterior standard deviation
• change = relative change from start, in %; ↑ = increase, ↓ = decrease
📊 Posterior distribution:
| datablock | category | entry | parameter | units | median | 95% CI | r-hat | ess bulk | |
|---|---|---|---|---|---|---|---|---|---|
| 1 | lbco | cell | length_a | Å | 3.8913 | [3.8911, 3.8915] | 1.002 | 9192.1 | |
| 2 | hrpt | linked_phases | lbco | scale | 9.1320 | [9.0751, 9.1891] | 1.002 | 8980.3 | |
| 3 | hrpt | peak | broad_gauss_u | deg² | 0.0816 | [0.0686, 0.0945] | 1.002 | 9029.0 | |
| 4 | hrpt | peak | broad_gauss_v | deg² | -0.1168 | [-0.1260, -0.1075] | 1.002 | 9080.0 | |
| 5 | hrpt | instrument | twotheta_offset | deg | 0.6303 | [0.6269, 0.6336] | 1.002 | 9243.8 |
• median = 50th percentile of the marginal posterior
• 95% CI = 95% credible interval (2.5%-97.5%, asymmetric)
• r-hat = Gelman-Rubin diagnostic (good convergence: r-hat <= 1.01)
• ess bulk = bulk effective sample size (typically >= 400)
• 95% CI = 95% credible interval (2.5%-97.5%, asymmetric)
• r-hat = Gelman-Rubin diagnostic (good convergence: r-hat <= 1.01)
• ess bulk = bulk effective sample size (typically >= 400)
Display Resumed Posterior¶
After resume, the posterior plots use the extended chain.
In [14]:
Copied!
project.display.posterior.pairs()
project.display.posterior.pairs()
Loading plot…
In [15]:
Copied!
project.display.posterior.distribution()
project.display.posterior.distribution()
Loading plot…
Loading plot…
Loading plot…
Loading plot…
Loading plot…
In [16]:
Copied!
project.display.posterior.predictive(expt_name='hrpt', x_min=92, x_max=93)
project.display.posterior.predictive(expt_name='hrpt', x_min=92, x_max=93)
Loading plot…
💾 Save Project¶
In [17]:
Copied!
project.save_as(dir_path='projects/ed_26_lbco_hrpt_emcee')
project.save_as(dir_path='projects/ed_26_lbco_hrpt_emcee')
Saving project 📦 'lbco_hrpt_emcee' to '../../../projects/ed_26_lbco_hrpt_emcee'
├── 📄 project.cif
├── 📁 structures/
│ └── 📄 lbco.cif
├── 📁 experiments/
│ └── 📄 hrpt.cif
├── 📁 analysis/
│ ├── 📄 analysis.cif
│ └── 📄 results.h5
└── 📁 reports/
└── 📄 lbco_hrpt_emcee.html