{ "cells": [ { "cell_type": "code", "execution_count": 1, "id": "0", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:48.477262Z", "iopub.status.busy": "2026-08-18T18:19:48.477093Z", "iopub.status.idle": "2026-08-18T18:19:48.481153Z", "shell.execute_reply": "2026-08-18T18:19:48.480292Z" }, "tags": [ "hide-in-docs" ] }, "outputs": [], "source": [ "# Check whether easydiffraction is installed; install it if needed.\n", "# Required for remote environments such as Google Colab.\n", "import importlib.util\n", "\n", "if importlib.util.find_spec('easydiffraction') is None:\n", " %pip install easydiffraction==0.20.1" ] }, { "cell_type": "markdown", "id": "1", "metadata": {}, "source": [ "# Bayesian Analysis (MCMC) – LBCO and Si\n", "\n", "This notebook continues the powder diffraction refinement tutorial. In\n", "that tutorial, you built a two-phase model containing\n", "La₀.₅Ba₀.₅CoO₃ (LBCO) and a small Si impurity, refined the model against\n", "simulated time-of-flight neutron data, and saved the project.\n", "\n", "A conventional least-squares refinement gives a best-fit point and\n", "uncertainty estimates based on the local shape of the objective\n", "function. Bayesian analysis instead uses Markov chain Monte Carlo\n", "(MCMC) sampling to explore the joint posterior distribution of the\n", "free parameters. This allows us to investigate questions such as:\n", "\n", "- Where is each posterior distribution centered, and how does that\n", " compare with the deterministic best-fit value?\n", "- How broad are their credible intervals?\n", "- Do the local parameter correlations from the deterministic fit\n", " persist across the posterior, and are any relationships nonlinear or\n", " asymmetric?\n", "- How does parameter uncertainty propagate into the calculated\n", " diffraction pattern?\n", "\n", "The deterministic fit already provides a correlation matrix derived\n", "from the local covariance estimate near the best-fit point. Bayesian\n", "sampling extends this view by exploring how parameters vary together\n", "throughout the joint posterior. It can therefore reveal curved,\n", "asymmetric, bounded, or multimodal relationships that one local\n", "correlation coefficient cannot describe.\n", "\n", "This notebook follows the same teaching structure as the refinement\n", "tutorial:\n", "\n", "- **Introduction:** a complete, guided MCMC analysis of the refined\n", " LBCO+Si model.\n", "- **Exercise:** repeat the analysis after fixing one member of a\n", " strongly correlated peak-profile pair, then compare the results.\n", "\n", "We will use the DREAM sampler provided by the `bumps (dream)`\n", "minimizer. The short chains used here keep the tutorial practical. A\n", "scientific analysis requires longer chains and careful convergence\n", "checks." ] }, { "cell_type": "markdown", "id": "2", "metadata": {}, "source": [ "## 🛠️ Import Library" ] }, { "cell_type": "code", "execution_count": 2, "id": "3", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:48.483002Z", "iopub.status.busy": "2026-08-18T18:19:48.482849Z", "iopub.status.idle": "2026-08-18T18:19:51.254209Z", "shell.execute_reply": "2026-08-18T18:19:51.253614Z" } }, "outputs": [], "source": [ "import easydiffraction as edi" ] }, { "cell_type": "markdown", "id": "4", "metadata": {}, "source": [ "## 📘 Introduction: MCMC Analysis of LBCO+Si\n", "\n", "We will first work through the complete Bayesian workflow without\n", "exercises. This reference analysis samples the same seven scientific\n", "parameters that remained free at the end of the refinement tutorial:\n", "\n", "- the LBCO lattice parameter,\n", "- the LBCO and Si scale factors, and\n", "- four peak-profile parameters.\n", "\n", "The resulting project will be called `project_1`, following the\n", "convention used for the introductory Si fit in the previous tutorial." ] }, { "cell_type": "markdown", "id": "5", "metadata": {}, "source": [ "### 📂 Load the Refined Project\n", "\n", "Rather than rebuilding the experiment and structures, download the\n", "project saved at the end of the refinement tutorial. This restores the\n", "measured data, both structures, the refined values, and the\n", "free-parameter settings.\n", "\n", "The following cell downloads our pre-generated refined project from\n", "the EasyDiffraction repository. This lets you continue even if you did\n", "not complete the refinement tutorial or your saved project is missing.\n", "The `download_data` function will not overwrite an existing project\n", "unless you set `overwrite=True`, so it is safe to run even if the\n", "project is already present." ] }, { "cell_type": "markdown", "id": "6", "metadata": {}, "source": [ "📖 See\n", "[documentation](https://docs.easydiffraction.org/lib/latest/user-guide/analysis-workflow/project/#loading-a-saved-project)\n", "for more details about loading a saved project." ] }, { "cell_type": "code", "execution_count": 3, "id": "7", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:51.256603Z", "iopub.status.busy": "2026-08-18T18:19:51.256282Z", "iopub.status.idle": "2026-08-18T18:19:52.014687Z", "shell.execute_reply": "2026-08-18T18:19:52.013748Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mGetting data\u001b[0m\u001b[1;36m...\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Data \u001b[32m'proj-fitting-exercise-si-lbco-main'\u001b[0m: La0.5Ba0.5CoO3 + Si, fitted DMSC exercise project\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "✅ Data \u001b[32m'proj-fitting-exercise-si-lbco-main'\u001b[0m downloaded and extracted to \n", "\u001b[32m'../../../projects/proj-fitting-exercise-si-lbco-main-62547397a81c'\u001b[0m\n" ] } ], "source": [ "refinement_project_dir = edi.download_data(\n", " 'proj-fitting-exercise-si-lbco-main',\n", " destination='projects',\n", ")\n", "project_1 = edi.Project.load(refinement_project_dir)" ] }, { "cell_type": "markdown", "id": "8", "metadata": {}, "source": [ "Save a copy under a new name before changing the analysis. This keeps\n", "the deterministic refinement unchanged and gives the sampler its own\n", "location for posterior summaries and chain data." ] }, { "cell_type": "code", "execution_count": 4, "id": "9", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:52.016574Z", "iopub.status.busy": "2026-08-18T18:19:52.016270Z", "iopub.status.idle": "2026-08-18T18:19:52.266482Z", "shell.execute_reply": "2026-08-18T18:19:52.265745Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mSaving project 📦 \u001b[0m\u001b[32m'main'\u001b[0m\u001b[1;36m to \u001b[0m\u001b[32m'../../../projects/exercise-bayesian-si-lbco-reference'\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "├── 📄 project.edi\n", "├── 📁 structures/\n", "│ └── 📄 lbco.edi\n", "│ └── 📄 si.edi\n", "├── 📁 experiments/\n", "│ └── 📄 sim_lbco.edi\n", "├── 📁 analysis/\n", "│ └── 📄 analysis.edi\n", "└── 📁 reports/\n", " └── 📄 main.html\n" ] } ], "source": [ "project_1.metadata.title = 'Reference Bayesian Analysis of LBCO with Si Impurity'\n", "project_1.metadata.description = (\n", " 'Reference MCMC analysis of a two-phase LBCO and Si powder diffraction model.'\n", ")\n", "project_1.save_as(dir_path='projects/exercise-bayesian-si-lbco-reference')" ] }, { "cell_type": "markdown", "id": "10", "metadata": {}, "source": [ "Confirm that the restored project contains the LBCO and Si structures\n", "and the `sim_lbco` experiment." ] }, { "cell_type": "code", "execution_count": 5, "id": "11", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:52.268043Z", "iopub.status.busy": "2026-08-18T18:19:52.267881Z", "iopub.status.idle": "2026-08-18T18:19:52.274006Z", "shell.execute_reply": "2026-08-18T18:19:52.273341Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mDefined structures 🧩\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1m[\u001b[0m\u001b[32m'lbco'\u001b[0m, \u001b[32m'si'\u001b[0m\u001b[1m]\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mDefined experiments 🔬\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1m[\u001b[0m\u001b[32m'sim_lbco'\u001b[0m\u001b[1m]\u001b[0m\n" ] } ], "source": [ "project_1.structures.show_names()\n", "project_1.experiments.show_names()" ] }, { "cell_type": "code", "execution_count": 6, "id": "12", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:52.275902Z", "iopub.status.busy": "2026-08-18T18:19:52.275724Z", "iopub.status.idle": "2026-08-18T18:19:52.323884Z", "shell.execute_reply": "2026-08-18T18:19:52.323160Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mStructure 🧩 \u001b[0m\u001b[32m'lbco'\u001b[0m\u001b[1;36m \u001b[0m\u001b[1;36m(\u001b[0m\u001b[1;36mAtom view type: \u001b[0m\u001b[32m'covalent'\u001b[0m\u001b[1;36m)\u001b[0m\n" ] }, { "data": { "text/html": [ "
\n", "
\n", "
Loading plot…
\n", "
\n", "
\n", "
\n", "
drag = rotate
wheel = zoom
right-drag = pan
\n", "
\n", "
\n", "\n", "\n", "\n", "" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mStructure 🧩 \u001b[0m\u001b[32m'si'\u001b[0m\u001b[1;36m \u001b[0m\u001b[1;36m(\u001b[0m\u001b[1;36mAtom view type: \u001b[0m\u001b[32m'covalent'\u001b[0m\u001b[1;36m)\u001b[0m\n" ] }, { "data": { "text/html": [ "
\n", "
\n", "
Loading plot…
\n", "
\n", "
\n", "
\n", "
drag = rotate
wheel = zoom
right-drag = pan
\n", "
\n", "
\n", "\n", "\n", "\n", "" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.display.structure(struct_name='lbco')\n", "project_1.display.structure(struct_name='si')" ] }, { "cell_type": "code", "execution_count": 7, "id": "13", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:52.325771Z", "iopub.status.busy": "2026-08-18T18:19:52.325590Z", "iopub.status.idle": "2026-08-18T18:19:52.719369Z", "shell.execute_reply": "2026-08-18T18:19:52.718606Z" } }, "outputs": [ { "data": { "text/html": [ "
Loading plot…
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.display.pattern(expt_name='sim_lbco')" ] }, { "cell_type": "markdown", "id": "14", "metadata": {}, "source": [ "### 🎯 Choose the Sampled Parameters\n", "\n", "MCMC varies every free parameter simultaneously. Before sampling, it\n", "is worth asking which parameters are needed to answer the scientific\n", "question.\n", "\n", "First, display all free parameters in the refined project." ] }, { "cell_type": "code", "execution_count": 8, "id": "15", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:52.721042Z", "iopub.status.busy": "2026-08-18T18:19:52.720876Z", "iopub.status.idle": "2026-08-18T18:19:52.774335Z", "shell.execute_reply": "2026-08-18T18:19:52.773265Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mFree parameters for both structures \u001b[0m\u001b[1;36m(\u001b[0m\u001b[1;36m🧩 data blocks\u001b[0m\u001b[1;36m)\u001b[0m\u001b[1;36m and experiments \u001b[0m\u001b[1;36m(\u001b[0m\u001b[1;36m🔬 data blocks\u001b[0m\u001b[1;36m)\u001b[0m\n" ] }, { "data": { "text/html": [ "
datablockcategoryentryparametervalueuncertaintyminmaxunits
1lbcocelllength_a3.891470.00015-infinfÅ
2sim_lbcolinked_structurelbcoscale42.360000.24000-infinf
3sim_lbcolinked_structuresiscale0.327000.01200-infinf
4sim_lbcopeakrise_alpha_10.016550.00014-infinfμs/Å
5sim_lbcopeakdecay_beta_10.018570.00042-infinfμs/Å
6sim_lbcopeakbroad_gauss_sigma_1-68200.000001500.00000-infinfμs/Å
7sim_lbcopeakbroad_gauss_sigma_216130.00000780.00000-infinfμs²/Ų
8sim_lbcobackground1intensity0.277000.02900-infinf
9sim_lbcobackground2intensity0.251100.00590-infinf
10sim_lbcobackground3intensity0.252000.00380-infinf
11sim_lbcobackground4intensity0.241500.00300-infinf
12sim_lbcobackground5intensity0.242800.00210-infinf
13sim_lbcobackground6intensity0.221500.00250-infinf
14sim_lbcobackground7intensity0.238400.00880-infinf
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.display.parameters.free()" ] }, { "cell_type": "markdown", "id": "16", "metadata": {}, "source": [ "The seven line-segment background intensities were refined in the\n", "previous tutorial. If we left them free, this analysis would sample 14\n", "parameters instead of 7. The larger space would require more model\n", "evaluations, take longer to explore, and generally need a longer chain\n", "to mix well. Background parameters can also correlate with phase\n", "scales and broad peaks, making the posterior harder to interpret.\n", "\n", "We therefore fix the background at its refined values. This is a\n", "practical simplification for a short teaching example, not a universal\n", "rule. Fixing the background means that its uncertainty is **not**\n", "propagated into the final credible intervals. In a complete scientific\n", "analysis, you should sample relevant background parameters or otherwise\n", "account for their uncertainty when it can affect the result." ] }, { "cell_type": "code", "execution_count": 9, "id": "17", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:52.776030Z", "iopub.status.busy": "2026-08-18T18:19:52.775835Z", "iopub.status.idle": "2026-08-18T18:19:52.779694Z", "shell.execute_reply": "2026-08-18T18:19:52.778976Z" } }, "outputs": [], "source": [ "experiment_1 = project_1.experiments['sim_lbco']\n", "\n", "for line_segment in experiment_1.background:\n", " line_segment.intensity.free = False" ] }, { "cell_type": "markdown", "id": "18", "metadata": {}, "source": [ "Display the remaining free parameters to verify the sampling problem." ] }, { "cell_type": "code", "execution_count": 10, "id": "19", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:52.781178Z", "iopub.status.busy": "2026-08-18T18:19:52.780988Z", "iopub.status.idle": "2026-08-18T18:19:52.817300Z", "shell.execute_reply": "2026-08-18T18:19:52.816133Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mFree parameters for both structures \u001b[0m\u001b[1;36m(\u001b[0m\u001b[1;36m🧩 data blocks\u001b[0m\u001b[1;36m)\u001b[0m\u001b[1;36m and experiments \u001b[0m\u001b[1;36m(\u001b[0m\u001b[1;36m🔬 data blocks\u001b[0m\u001b[1;36m)\u001b[0m\n" ] }, { "data": { "text/html": [ "
datablockcategoryentryparametervalueuncertaintyminmaxunits
1lbcocelllength_a3.891470.00015-infinfÅ
2sim_lbcolinked_structurelbcoscale42.360000.24000-infinf
3sim_lbcolinked_structuresiscale0.327000.01200-infinf
4sim_lbcopeakrise_alpha_10.016550.00014-infinfμs/Å
5sim_lbcopeakdecay_beta_10.018570.00042-infinfμs/Å
6sim_lbcopeakbroad_gauss_sigma_1-68200.000001500.00000-infinfμs/Å
7sim_lbcopeakbroad_gauss_sigma_216130.00000780.00000-infinfμs²/Ų
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.display.parameters.free()" ] }, { "cell_type": "markdown", "id": "20", "metadata": {}, "source": [ "### 🚀 Run a Local Refinement\n", "\n", "Fixing the background changes the optimization problem slightly. We\n", "first run a fast local least-squares refinement to update the best-fit\n", "values and their uncertainty estimates. These values will provide the\n", "starting point and finite bounds for MCMC.\n", "\n", "We use the BUMPS Levenberg-Marquardt minimizer, `bumps (lm)`, so the\n", "local and Bayesian stages use the same minimizer library." ] }, { "cell_type": "markdown", "id": "21", "metadata": {}, "source": [ "📖 See\n", "[documentation](https://docs.easydiffraction.org/lib/latest/user-guide/analysis-workflow/analysis/#minimization-optimization)\n", "for more details about the available minimizers." ] }, { "cell_type": "code", "execution_count": 11, "id": "22", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:52.819151Z", "iopub.status.busy": "2026-08-18T18:19:52.818894Z", "iopub.status.idle": "2026-08-18T18:19:52.828752Z", "shell.execute_reply": "2026-08-18T18:19:52.827932Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mMinimizer types\u001b[0m\n" ] }, { "data": { "text/html": [ "
TypeDescription
1bumpsBUMPS library using the default Levenberg-Marquardt method
2bumps (amoeba)BUMPS library with Nelder-Mead simplex method
3bumps (de)BUMPS library with differential evolution method
4bumps (dream)BUMPS library with DREAM Bayesian sampling
5bumps (lm)BUMPS library with Levenberg-Marquardt method
6dfolsDFO-LS library for derivative-free least-squares optimization
7emceeemcee affine-invariant ensemble Bayesian sampling
8lmfitLMFIT library using the default Levenberg-Marquardt method
9lmfit (least_squares)LMFIT library with SciPy's trust region reflective algorithm
10*lmfit (leastsq)LMFIT library with Levenberg-Marquardt least squares method
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.analysis.minimizer.show_supported()" ] }, { "cell_type": "code", "execution_count": 12, "id": "23", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:52.830316Z", "iopub.status.busy": "2026-08-18T18:19:52.830148Z", "iopub.status.idle": "2026-08-18T18:19:52.835750Z", "shell.execute_reply": "2026-08-18T18:19:52.834979Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mCurrent minimizer changed to\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "bumps \u001b[1m(\u001b[0mlm\u001b[1m)\u001b[0m\n" ] } ], "source": [ "project_1.analysis.minimizer.type = 'bumps (lm)'" ] }, { "cell_type": "code", "execution_count": 13, "id": "24", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:52.837500Z", "iopub.status.busy": "2026-08-18T18:19:52.837303Z", "iopub.status.idle": "2026-08-18T18:19:54.400965Z", "shell.execute_reply": "2026-08-18T18:19:54.399992Z" } }, "outputs": [ { "data": { "text/html": [], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "\n", "(function() {\n", " const button = document.getElementById('ed-fit-stop-c98f90e62e254418b5c4767b0d811c86-button');\n", " const status = document.getElementById('ed-fit-stop-c98f90e62e254418b5c4767b0d811c86-status');\n", " const kernelId = '';\n", " if (!button) {\n", " return;\n", " }\n", "\n", " function setStatus(text) {\n", " if (status) {\n", " status.textContent = text;\n", " }\n", " }\n", "\n", " function pageConfig() {\n", " const element = document.getElementById('jupyter-config-data');\n", " if (!element || !element.textContent) {\n", " return {};\n", " }\n", " try {\n", " return JSON.parse(element.textContent);\n", " } catch (error) {\n", " return {};\n", " }\n", " }\n", "\n", " function baseUrl(config) {\n", " const configured = config.baseUrl || config.base_url ||\n", " (window.Jupyter && Jupyter.notebook && Jupyter.notebook.base_url);\n", " if (configured) {\n", " return configured.endsWith('/') ? configured : configured + '/';\n", " }\n", " const markers = ['/lab/', '/notebooks/', '/tree/'];\n", " for (const marker of markers) {\n", " const index = window.location.pathname.indexOf(marker);\n", " if (index >= 0) {\n", " return window.location.pathname.slice(0, index + 1);\n", " }\n", " }\n", " return '/';\n", " }\n", "\n", " function token(config) {\n", " return config.token || new URLSearchParams(window.location.search).get('token') || '';\n", " }\n", "\n", " function cookie(name) {\n", " const prefix = name + '=';\n", " for (const part of document.cookie.split(';')) {\n", " const trimmed = part.trim();\n", " if (trimmed.startsWith(prefix)) {\n", " return decodeURIComponent(trimmed.slice(prefix.length));\n", " }\n", " }\n", " return '';\n", " }\n", "\n", " function notebookPath() {\n", " const decoded = decodeURIComponent(window.location.pathname);\n", " const markers = ['/lab/tree/', '/notebooks/', '/tree/'];\n", " for (const marker of markers) {\n", " const index = decoded.indexOf(marker);\n", " if (index >= 0) {\n", " return decoded.slice(index + marker.length);\n", " }\n", " }\n", " return '';\n", " }\n", "\n", " async function kernelFromSessions(config) {\n", " const url = new URL(baseUrl(config) + 'api/sessions', window.location.origin);\n", " const authToken = token(config);\n", " if (authToken) {\n", " url.searchParams.set('token', authToken);\n", " }\n", " const response = await fetch(url, {credentials: 'same-origin'});\n", " if (!response.ok) {\n", " return '';\n", " }\n", " const sessions = await response.json();\n", " const path = notebookPath();\n", " const session = sessions.find((item) => item.path === path) || sessions[0];\n", " return session && session.kernel ? session.kernel.id : '';\n", " }\n", "\n", " async function interruptKernel(config, resolvedKernelId) {\n", " const url = new URL(\n", " baseUrl(config) + 'api/kernels/' + resolvedKernelId + '/interrupt',\n", " window.location.origin\n", " );\n", " const authToken = token(config);\n", " if (authToken) {\n", " url.searchParams.set('token', authToken);\n", " }\n", " const xsrfToken = cookie('_xsrf');\n", " const headers = {};\n", " if (xsrfToken) {\n", " headers['X-XSRFToken'] = xsrfToken;\n", " }\n", " const response = await fetch(url, {\n", " method: 'POST',\n", " credentials: 'same-origin',\n", " headers: headers\n", " });\n", " return response.ok;\n", " }\n", "\n", " button.addEventListener('click', async function() {\n", " button.disabled = true;\n", " setStatus('Stopping...');\n", " const config = pageConfig();\n", " try {\n", " const resolvedKernelId = kernelId || await kernelFromSessions(config);\n", " if (!resolvedKernelId) {\n", " throw new Error('Could not resolve the current kernel id.');\n", " }\n", " const interrupted = await interruptKernel(config, resolvedKernelId);\n", " if (!interrupted) {\n", " throw new Error('Jupyter Server rejected the interrupt request.');\n", " }\n", " setStatus('Interrupt sent...');\n", " } catch (error) {\n", " button.disabled = false;\n", " setStatus('Use Kernel > Interrupt to stop this fit.');\n", " }\n", " });\n", "})();\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mStandard fitting\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "📋 Using experiment 🔬 \u001b[32m'sim_lbco'\u001b[0m for \u001b[32m'single'\u001b[0m fitting\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "🚀 Starting fit process with \u001b[32m'bumps \u001b[0m\u001b[32m(\u001b[0m\u001b[32mlm\u001b[0m\u001b[32m)\u001b[0m\u001b[32m'\u001b[0m\u001b[33m...\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "📈 Goodness-of-fit progress:\n" ] }, { "data": { "text/html": [ "
iterationtime (s)χ²change / status
110.021.58
2271.151.58
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "🏆 Best goodness-of-fit \u001b[1m(\u001b[0mreduced χ²\u001b[1m)\u001b[0m is \u001b[1;36m1.58\u001b[0m at iteration \u001b[1;36m17\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "✅ Fitting complete.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "⚙️ Settings used:\n" ] }, { "data": { "text/html": [ "
NameValueDescription
1max_iterations1000Maximum solver iterations.
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "📋 Least-squares fit results:\n" ] }, { "data": { "text/html": [ "
MetricValue
1🧪 Minimizerbumps (lm)
2✅ Overall statussuccess
3⏱️ Fitting time (seconds)1.15
4📏 Goodness-of-fit (reduced χ²)1.58
5📏 R-factor (Rf, %)5.06
6📏 R-factor squared (Rf², %)4.61
7📏 Weighted R-factor (wR, %)4.05
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "📈 Refined parameters:\n" ] }, { "data": { "text/html": [ "
datablockcategoryentryparameterunitsstartvalues.u.change
1lbcocelllength_aÅ3.89153.89150.00020.00 % ↑
2sim_lbcolinked_structurelbcoscale42.360042.36070.22960.00 % ↑
3sim_lbcolinked_structuresiscale0.32700.32660.01120.14 % ↓
4sim_lbcopeakrise_alpha_1μs/Å0.01650.01660.00010.02 % ↑
5sim_lbcopeakdecay_beta_1μs/Å0.01860.01860.00040.03 % ↑
6sim_lbcopeakbroad_gauss_sigma_1μs/Å-68200.0000-68198.90421460.16730.00 % ↓
7sim_lbcopeakbroad_gauss_sigma_2μs²/Ų16130.000016127.9516772.92240.01 % ↓
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
• start = parameter value before refinement
• value = refined value from least-squares minimization
• s.u. = standard uncertainty (one sigma), from the covariance matrix
• change = relative change from start, in %; ↑ = increase, ↓ = decrease
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.analysis.fit()\n", "project_1.display.fit.results()" ] }, { "cell_type": "markdown", "id": "25", "metadata": {}, "source": [ "### 🔗 Understand the Local Correlation Chart\n", "\n", "The correlation chart summarizes how pairs of refined parameters can\n", "change together near the least-squares optimum. Each off-diagonal\n", "value is a correlation coefficient between -1 and +1:\n", "\n", "- a value near **+1** means that the two parameters tend to increase\n", " together;\n", "- a value near **-1** means that one tends to decrease when the other\n", " increases; and\n", "- a value near **0** means that there is little linear relationship.\n", "\n", "Correlation does not mean that one parameter physically causes the\n", "other. It means that the measured pattern cannot distinguish their\n", "effects independently within this model." ] }, { "cell_type": "code", "execution_count": 14, "id": "26", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:54.402760Z", "iopub.status.busy": "2026-08-18T18:19:54.402585Z", "iopub.status.idle": "2026-08-18T18:19:54.426715Z", "shell.execute_reply": "2026-08-18T18:19:54.425884Z" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "
\n", "
Loading plot…
\n", "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.display.fit.correlations(max_parameters=5)" ] }, { "cell_type": "markdown", "id": "27", "metadata": {}, "source": [ "The matrix shows only one triangular half because the other half would\n", "contain the same values in reverse order. With `max_parameters=5`,\n", "EasyDiffraction filters the chart to keep the strongest relationships\n", "readable. In an interactive Jupyter display, hover over a colored cell\n", "to see the two full parameter names and the numerical correlation\n", "coefficient.\n", "\n", "The strongest relationship is typically the negative correlation\n", "between `broad_gauss_sigma_1` and `broad_gauss_sigma_2`, close to\n", "-0.94. Both parameters contribute to the d-spacing dependence of the\n", "Gaussian TOF peak width. An increase in one can be partly compensated\n", "by a decrease in the other while producing a similar calculated peak\n", "shape.\n", "\n", "This chart comes from the local covariance estimate, so it describes\n", "only the neighborhood around the best-fit point. MCMC will show whether\n", "the relationship remains linear and approximately elliptical across a\n", "wider region of parameter space." ] }, { "cell_type": "markdown", "id": "28", "metadata": {}, "source": [ "### 🎲 Define the Sampling Region\n", "\n", "DREAM requires finite lower and upper bounds for every sampled\n", "parameter. Here, we derive them from the uncertainty estimated by the\n", "local fit. `set_fit_bounds_from_uncertainty()` places each bound four\n", "estimated standard uncertainties from the current value by default,\n", "while respecting any physical parameter limits.\n", "\n", "These bounds act as bounded prior support in this example. They must be\n", "inspected rather than accepted blindly: narrow bounds can truncate the\n", "posterior, while unnecessarily wide bounds can make sampling less\n", "efficient." ] }, { "cell_type": "markdown", "id": "29", "metadata": {}, "source": [ "📖 See\n", "[documentation](https://docs.easydiffraction.org/lib/latest/user-guide/parameters/)\n", "for more details about parameter values, uncertainties, and fit bounds." ] }, { "cell_type": "code", "execution_count": 15, "id": "30", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:54.428244Z", "iopub.status.busy": "2026-08-18T18:19:54.428063Z", "iopub.status.idle": "2026-08-18T18:19:54.435272Z", "shell.execute_reply": "2026-08-18T18:19:54.434469Z" } }, "outputs": [], "source": [ "for param in project_1.free_parameters:\n", " param.set_fit_bounds_from_uncertainty()" ] }, { "cell_type": "code", "execution_count": 16, "id": "31", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:54.436755Z", "iopub.status.busy": "2026-08-18T18:19:54.436584Z", "iopub.status.idle": "2026-08-18T18:19:54.471112Z", "shell.execute_reply": "2026-08-18T18:19:54.470255Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mFree parameters for both structures \u001b[0m\u001b[1;36m(\u001b[0m\u001b[1;36m🧩 data blocks\u001b[0m\u001b[1;36m)\u001b[0m\u001b[1;36m and experiments \u001b[0m\u001b[1;36m(\u001b[0m\u001b[1;36m🔬 data blocks\u001b[0m\u001b[1;36m)\u001b[0m\n" ] }, { "data": { "text/html": [ "
datablockcategoryentryparametervalueuncertaintyminmaxunits
1lbcocelllength_a3.891480.000153.890863.89209Å
2sim_lbcolinked_structurelbcoscale42.360690.2295841.4423543.27902
3sim_lbcolinked_structuresiscale0.326560.011190.281790.37132
4sim_lbcopeakrise_alpha_10.016550.000130.016020.01708μs/Å
5sim_lbcopeakdecay_beta_10.018580.000410.016940.02021μs/Å
6sim_lbcopeakbroad_gauss_sigma_1-68198.904161460.16730-74039.57337-62358.23495μs/Å
7sim_lbcopeakbroad_gauss_sigma_216127.95164772.9224213036.2619419219.64133μs²/Ų
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.display.parameters.free()" ] }, { "cell_type": "markdown", "id": "32", "metadata": {}, "source": [ "### 🎲 Run DREAM Sampling\n", "\n", "In Bayesian analysis, the posterior combines the likelihood of the\n", "measured data with prior assumptions. MCMC constructs chains of\n", "correlated samples whose long-run distribution approximates that\n", "posterior.\n", "\n", "DREAM uses multiple chains and differential-evolution proposals to\n", "explore correlated parameter spaces. This makes it useful for the\n", "strong peak-profile correlation observed above." ] }, { "cell_type": "markdown", "id": "33", "metadata": {}, "source": [ "📖 See\n", "[documentation](https://docs.easydiffraction.org/lib/latest/user-guide/analysis-workflow/analysis/#bayesian-analysis)\n", "for more details about Bayesian minimizers and posterior displays." ] }, { "cell_type": "code", "execution_count": 17, "id": "34", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:54.473038Z", "iopub.status.busy": "2026-08-18T18:19:54.472845Z", "iopub.status.idle": "2026-08-18T18:19:54.481442Z", "shell.execute_reply": "2026-08-18T18:19:54.480708Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "⚠️ Switching minimizer type removes these settings: \n", " • max_iterations \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "⚠️ Switching minimizer type adds these settings with defaults: \n", " • burn_in_steps=600 \n", " • initialization_method='latin_hypercube' \n", " • parallel_workers=0 \n", " • population_size=4 \n", " • random_seed=None \n", " • sampling_steps=3000 \n", " • thinning_interval=1 \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mCurrent minimizer changed to\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "bumps \u001b[1m(\u001b[0mdream\u001b[1m)\u001b[0m\n" ] } ], "source": [ "project_1.analysis.minimizer.type = 'bumps (dream)'\n", "project_1.analysis.minimizer.sampling_steps = 300 # lower than the default 3000\n", "project_1.analysis.minimizer.burn_in_steps = 60 # lower than the default 600\n", "project_1.analysis.minimizer.random_seed = 42" ] }, { "cell_type": "markdown", "id": "35", "metadata": {}, "source": [ "Burn-in samples allow the chains to move away from their initial\n", "positions before the retained posterior is summarized. The fixed seed\n", "makes the tutorial output reproducible." ] }, { "cell_type": "code", "execution_count": 18, "id": "36", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:19:54.483441Z", "iopub.status.busy": "2026-08-18T18:19:54.483249Z", "iopub.status.idle": "2026-08-18T18:22:44.639717Z", "shell.execute_reply": "2026-08-18T18:22:44.638846Z" } }, "outputs": [ { "data": { "text/html": [], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "\n", "(function() {\n", " const button = document.getElementById('ed-fit-stop-8cdad97142514602be95964261c522aa-button');\n", " const status = document.getElementById('ed-fit-stop-8cdad97142514602be95964261c522aa-status');\n", " const kernelId = '';\n", " if (!button) {\n", " return;\n", " }\n", "\n", " function setStatus(text) {\n", " if (status) {\n", " status.textContent = text;\n", " }\n", " }\n", "\n", " function pageConfig() {\n", " const element = document.getElementById('jupyter-config-data');\n", " if (!element || !element.textContent) {\n", " return {};\n", " }\n", " try {\n", " return JSON.parse(element.textContent);\n", " } catch (error) {\n", " return {};\n", " }\n", " }\n", "\n", " function baseUrl(config) {\n", " const configured = config.baseUrl || config.base_url ||\n", " (window.Jupyter && Jupyter.notebook && Jupyter.notebook.base_url);\n", " if (configured) {\n", " return configured.endsWith('/') ? configured : configured + '/';\n", " }\n", " const markers = ['/lab/', '/notebooks/', '/tree/'];\n", " for (const marker of markers) {\n", " const index = window.location.pathname.indexOf(marker);\n", " if (index >= 0) {\n", " return window.location.pathname.slice(0, index + 1);\n", " }\n", " }\n", " return '/';\n", " }\n", "\n", " function token(config) {\n", " return config.token || new URLSearchParams(window.location.search).get('token') || '';\n", " }\n", "\n", " function cookie(name) {\n", " const prefix = name + '=';\n", " for (const part of document.cookie.split(';')) {\n", " const trimmed = part.trim();\n", " if (trimmed.startsWith(prefix)) {\n", " return decodeURIComponent(trimmed.slice(prefix.length));\n", " }\n", " }\n", " return '';\n", " }\n", "\n", " function notebookPath() {\n", " const decoded = decodeURIComponent(window.location.pathname);\n", " const markers = ['/lab/tree/', '/notebooks/', '/tree/'];\n", " for (const marker of markers) {\n", " const index = decoded.indexOf(marker);\n", " if (index >= 0) {\n", " return decoded.slice(index + marker.length);\n", " }\n", " }\n", " return '';\n", " }\n", "\n", " async function kernelFromSessions(config) {\n", " const url = new URL(baseUrl(config) + 'api/sessions', window.location.origin);\n", " const authToken = token(config);\n", " if (authToken) {\n", " url.searchParams.set('token', authToken);\n", " }\n", " const response = await fetch(url, {credentials: 'same-origin'});\n", " if (!response.ok) {\n", " return '';\n", " }\n", " const sessions = await response.json();\n", " const path = notebookPath();\n", " const session = sessions.find((item) => item.path === path) || sessions[0];\n", " return session && session.kernel ? session.kernel.id : '';\n", " }\n", "\n", " async function interruptKernel(config, resolvedKernelId) {\n", " const url = new URL(\n", " baseUrl(config) + 'api/kernels/' + resolvedKernelId + '/interrupt',\n", " window.location.origin\n", " );\n", " const authToken = token(config);\n", " if (authToken) {\n", " url.searchParams.set('token', authToken);\n", " }\n", " const xsrfToken = cookie('_xsrf');\n", " const headers = {};\n", " if (xsrfToken) {\n", " headers['X-XSRFToken'] = xsrfToken;\n", " }\n", " const response = await fetch(url, {\n", " method: 'POST',\n", " credentials: 'same-origin',\n", " headers: headers\n", " });\n", " return response.ok;\n", " }\n", "\n", " button.addEventListener('click', async function() {\n", " button.disabled = true;\n", " setStatus('Stopping...');\n", " const config = pageConfig();\n", " try {\n", " const resolvedKernelId = kernelId || await kernelFromSessions(config);\n", " if (!resolvedKernelId) {\n", " throw new Error('Could not resolve the current kernel id.');\n", " }\n", " const interrupted = await interruptKernel(config, resolvedKernelId);\n", " if (!interrupted) {\n", " throw new Error('Jupyter Server rejected the interrupt request.');\n", " }\n", " setStatus('Interrupt sent...');\n", " } catch (error) {\n", " button.disabled = false;\n", " setStatus('Use Kernel > Interrupt to stop this fit.');\n", " }\n", " });\n", "})();\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mStandard fitting\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "📋 Using experiment 🔬 \u001b[32m'sim_lbco'\u001b[0m for \u001b[32m'single'\u001b[0m fitting\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "🚀 Starting fit process with \u001b[32m'bumps \u001b[0m\u001b[32m(\u001b[0m\u001b[32mdream\u001b[0m\u001b[32m)\u001b[0m\u001b[32m'\u001b[0m\u001b[33m...\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "📈 Bayesian sampling progress:\n" ] }, { "data": { "text/html": [ "
stepprogresstime (s)log posteriorphase
10.39-628.67pre-processing
221/3605.8%7.70-352.40burn-in
340/36011.1%14.84-345.10burn-in
460/36016.7%22.49-342.11burn-in
561/36016.9%22.83-341.96sampling
676/36021.1%27.97-340.55sampling
791/36025.3%33.62-339.56sampling
8106/36029.4%38.71-338.54sampling
9121/36033.6%44.18-338.50sampling
10136/36037.8%49.05-338.42sampling
11151/36041.9%54.81-338.37sampling
12166/36046.1%59.99-337.43sampling
13181/36050.3%65.79-337.80sampling
14196/36054.4%70.98-337.47sampling
15211/36058.6%76.56-338.31sampling
16226/36062.8%81.73-338.08sampling
17241/36066.9%87.73-338.21sampling
18256/36071.1%92.96-337.89sampling
19271/36075.3%98.57-337.34sampling
20286/36079.4%103.64-336.96sampling
21301/36083.6%109.31-337.48sampling
22316/36087.8%114.49-338.37sampling
23331/36091.9%120.15-338.27sampling
24346/36096.1%125.68-337.94sampling
25360/360100.0%131.49-337.66sampling
26170.10post-processing
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "✅ Bayesian sampling complete.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "⚠️ Convergence diagnostics indicate the posterior may be poorly mixed. \n" ] } ], "source": [ "project_1.analysis.fit()" ] }, { "cell_type": "markdown", "id": "37", "metadata": {}, "source": [ "### 📋 Understand the Bayesian Fit Summary\n", "\n", "The result table now reports posterior medians and 95% credible\n", "intervals in addition to fit-quality metrics. A 95% credible interval\n", "is the interval containing 95% of the retained marginal posterior\n", "samples under this model and bounded sampling setup.\n", "\n", "The table also reports convergence diagnostics:\n", "\n", "- **r-hat** compares variation within and between chains. Values close\n", " to 1 are desirable; EasyDiffraction uses `r-hat <= 1.01` as its\n", " displayed convergence criterion.\n", "- **ess bulk** is the effective sample size after accounting for\n", " autocorrelation. A larger value means that the chain contains more\n", " independent information; the display recommends at least 400." ] }, { "cell_type": "code", "execution_count": 19, "id": "38", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:44.641300Z", "iopub.status.busy": "2026-08-18T18:22:44.641140Z", "iopub.status.idle": "2026-08-18T18:22:44.672689Z", "shell.execute_reply": "2026-08-18T18:22:44.671918Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "⚙️ Settings used:\n" ] }, { "data": { "text/html": [ "
NameValueDescription
1sampling_steps300Total sampler iterations per chain.
2burn_in_steps60Sampler iterations discarded as warm-up.
3thinning_interval1Sampler thinning interval.
4population_size4Number of chains or walkers.
5parallel_workers0Worker count; 0 uses all available CPUs.
6initialization_methodlatin_hypercubeSampler initialization method.
7random_seed42Random seed; None uses a system-derived seed.
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "📋 Bayesian fit results:\n" ] }, { "data": { "text/html": [ "
MetricValue
1🧪 Samplerbumps (dream)
2❌ Overall statusfailed
3💬 Engine messageDREAM sampling completed
4⏱️ Fitting time (seconds)170.10
5📏 Goodness-of-fit (reduced χ²)1.58
6📏 R-factor (Rf, %)5.06
7📏 R-factor squared (Rf², %)4.61
8📏 Weighted R-factor (wR, %)4.05
9📉 Best log-posterior-333.99
10📊 Convergence statusfailed
11📊 Max r-hat1.162
12📊 Min ess bulk280.912
13📊 Draws per chain300
14📊 Chains28
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "📈 Committed parameters:\n" ] }, { "data": { "text/html": [ "
datablockcategoryentryparameterunitsstartvalues.u.change
1lbcocelllength_aÅ3.89153.89150.00010.00 % ↓
2sim_lbcolinked_structurelbcoscale42.360742.36070.20680.00 % ↓
3sim_lbcolinked_structuresiscale0.32660.32660.00960.00 % ↓
4sim_lbcopeakrise_alpha_1μs/Å0.01660.01660.00010.00 % ↓
5sim_lbcopeakdecay_beta_1μs/Å0.01860.01860.00040.00 % ↓
6sim_lbcopeakbroad_gauss_sigma_1μs/Å-68198.9042-68198.90421320.41830.00 % ↓
7sim_lbcopeakbroad_gauss_sigma_2μs²/Ų16127.951616127.9516716.87920.00 % ↓
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
• start = parameter value before sampling
• value = estimate written back to the project (best posterior sample)
• s.u. = standard uncertainty (one sigma), posterior standard deviation
• change = relative change from start, in %; ↑ = increase, ↓ = decrease
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "📊 Posterior distribution:\n" ] }, { "data": { "text/html": [ "
datablockcategoryentryparameterunitsmedian95% CIr-hatess bulk
1lbcocelllength_aÅ3.8915[3.8912, 3.8917]1.162307.7
2sim_lbcolinked_structurelbcoscale42.3521[41.9423, 42.7509]1.124333.6
3sim_lbcolinked_structuresiscale0.3269[0.3083, 0.3458]1.154346.3
4sim_lbcopeakrise_alpha_1μs/Å0.0166[0.0163, 0.0168]1.160280.9
5sim_lbcopeakdecay_beta_1μs/Å0.0186[0.0179, 0.0193]1.133308.0
6sim_lbcopeakbroad_gauss_sigma_1μs/Å-68217.1483[-71110.3724, -65728.8313]1.149310.6
7sim_lbcopeakbroad_gauss_sigma_2μs²/Ų16147.0046[14865.4694, 17725.2796]1.150299.3
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
• median = 50th percentile of the marginal posterior
• 95% CI = 95% credible interval (2.5%-97.5%, asymmetric)
• r-hat = Gelman-Rubin diagnostic (good convergence: r-hat <= 1.01)
• ess bulk = bulk effective sample size (typically >= 400)
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
⚠️ r-hat > 1.01: Consider longer sampling, better initialization, or reparameterization.
⚠️ ess bulk < 400: Consider longer sampling or reparameterization.
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.display.fit.results()" ] }, { "cell_type": "markdown", "id": "39", "metadata": {}, "source": [ "A short teaching run may fail these convergence criteria even when it\n", "finishes normally. That means the numerical posterior summaries are\n", "provisional. Increase the number of sampling steps and inspect the\n", "diagnostics again before drawing scientific conclusions." ] }, { "cell_type": "markdown", "id": "40", "metadata": {}, "source": [ "### 🔗 Understand Posterior Correlations\n", "\n", "Calling the same method after MCMC now builds the matrix from posterior\n", "samples rather than from the local least-squares covariance estimate.\n", "It answers: *across the sampled posterior, which parameter pairs vary\n", "together, and how strongly?*" ] }, { "cell_type": "code", "execution_count": 20, "id": "41", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:44.674671Z", "iopub.status.busy": "2026-08-18T18:22:44.674511Z", "iopub.status.idle": "2026-08-18T18:22:44.696017Z", "shell.execute_reply": "2026-08-18T18:22:44.695158Z" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "
\n", "
Loading plot…
\n", "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.display.fit.correlations(max_parameters=5)" ] }, { "cell_type": "markdown", "id": "42", "metadata": {}, "source": [ "Compare this chart with the local chart above. Similar coefficients\n", "suggest that the local approximation captured the main linear\n", "relationship. A substantial difference can indicate curvature,\n", "asymmetry, bounds, or another feature that a local covariance matrix\n", "cannot represent." ] }, { "cell_type": "markdown", "id": "43", "metadata": {}, "source": [ "### 🗺️ Understand the Posterior Pair Plot\n", "\n", "The pair plot shows more information than a matrix of single\n", "coefficients:\n", "\n", "- **Diagonal panels** show each parameter's one-dimensional marginal\n", " posterior density. A narrow peak indicates greater precision; a wide,\n", " skewed, truncated, or multimodal shape indicates more uncertainty or\n", " a more complicated posterior.\n", "- **Lower-triangle panels** show joint posterior samples for pairs of\n", " parameters. The contours summarize where the sampled density is\n", " concentrated.\n", "- A compact, nearly round cloud suggests weak correlation. An elongated\n", " upward cloud suggests positive correlation, and an elongated downward\n", " cloud suggests negative correlation. Curved or split contours cannot\n", " be summarized well by one correlation coefficient.\n", "- The upper triangle is intentionally blank because it would duplicate\n", " the lower triangle." ] }, { "cell_type": "code", "execution_count": 21, "id": "44", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:44.697695Z", "iopub.status.busy": "2026-08-18T18:22:44.697483Z", "iopub.status.idle": "2026-08-18T18:22:45.137499Z", "shell.execute_reply": "2026-08-18T18:22:45.136658Z" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "
\n", "
Loading plot…
\n", "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.display.posterior.pairs(max_parameters=5)" ] }, { "cell_type": "markdown", "id": "45", "metadata": {}, "source": [ "In Jupyter, the default plotting engine resolves to interactive Plotly.\n", "Hover over a diagonal density curve to see the full parameter name,\n", "parameter value, and probability density. Hover over a visible sample\n", "point in a lower-triangle panel to see the exact values of both\n", "parameters for that posterior draw. The shaded contour itself has no\n", "hover tooltip; it is a smoothed two-dimensional density guide. Use the\n", "Plotly toolbar to zoom, pan, and reset the view, and click legend items\n", "to hide or show samples, contours, or marginal densities.\n", "\n", "The `broad_gauss_sigma_1`–`broad_gauss_sigma_2` panel should form a\n", "long downward band. This reveals the range of compensating parameter\n", "combinations, whereas the correlation chart reduces the relationship\n", "to one number." ] }, { "cell_type": "markdown", "id": "46", "metadata": {}, "source": [ "### 📈 Understand Marginal Posterior Distributions\n", "\n", "A separate distribution plot gives a more detailed view of each\n", "diagonal panel. It includes a histogram, a smoothed marginal density,\n", "the median, the best posterior sample, and the 95% credible interval.\n", "Hover over the traces and interval markers to read their values." ] }, { "cell_type": "code", "execution_count": 22, "id": "47", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:45.143194Z", "iopub.status.busy": "2026-08-18T18:22:45.143022Z", "iopub.status.idle": "2026-08-18T18:22:45.239900Z", "shell.execute_reply": "2026-08-18T18:22:45.239211Z" } }, "outputs": [ { "data": { "text/html": [ "
Loading plot…
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Loading plot…
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Loading plot…
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Loading plot…
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Loading plot…
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Loading plot…
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Loading plot…
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.display.posterior.distribution()" ] }, { "cell_type": "markdown", "id": "48", "metadata": {}, "source": [ "A density pressed against a fit bound warns that the allowed region may\n", "be too narrow or that the parameter is poorly identified. A best\n", "sample far from the median can occur for a skewed or irregular\n", "posterior and is another reason not to summarize MCMC with only one\n", "point estimate." ] }, { "cell_type": "markdown", "id": "49", "metadata": {}, "source": [ "### 📊 Understand the Posterior-Predictive Plot\n", "\n", "Posterior prediction propagates an evenly spaced subset of retained\n", "parameter combinations through the diffraction calculation. The\n", "best-posterior-sample curve shows one calculated pattern, while the\n", "95% band shows parameter uncertainty propagated into the pattern." ] }, { "cell_type": "code", "execution_count": 23, "id": "50", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:45.241792Z", "iopub.status.busy": "2026-08-18T18:22:45.241632Z", "iopub.status.idle": "2026-08-18T18:22:45.286739Z", "shell.execute_reply": "2026-08-18T18:22:45.286089Z" } }, "outputs": [ { "data": { "text/html": [ "
Loading plot…
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.display.posterior.predictive(expt_name='sim_lbco')" ] }, { "cell_type": "code", "execution_count": 24, "id": "51", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:45.288289Z", "iopub.status.busy": "2026-08-18T18:22:45.288123Z", "iopub.status.idle": "2026-08-18T18:22:45.331330Z", "shell.execute_reply": "2026-08-18T18:22:45.330631Z" } }, "outputs": [ { "data": { "text/html": [ "
Loading plot…
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.display.posterior.predictive(\n", " expt_name='sim_lbco',\n", " x_min=80000,\n", " x_max=81500,\n", ")" ] }, { "cell_type": "markdown", "id": "52", "metadata": {}, "source": [ "Compare the width of the band with the experimental uncertainty and\n", "remaining residuals. A narrow band does not prove that the model is\n", "correct: systematic deviations can indicate model inadequacy,\n", "underestimated experimental uncertainty, or effects not included in\n", "the refinement. In the interactive view, hover over the measured and\n", "best-sample curves to inspect individual x and intensity values. The\n", "shaded 95% band is read from its upper and lower boundaries and does\n", "not itself show a hover tooltip." ] }, { "cell_type": "markdown", "id": "53", "metadata": {}, "source": [ "Save the completed reference Bayesian project. Its MCMC chain and\n", "posterior data are stored with the project and remain available for\n", "comparison with the exercise below." ] }, { "cell_type": "code", "execution_count": 25, "id": "54", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:45.332927Z", "iopub.status.busy": "2026-08-18T18:22:45.332765Z", "iopub.status.idle": "2026-08-18T18:22:45.559526Z", "shell.execute_reply": "2026-08-18T18:22:45.558421Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mSaving project 📦 \u001b[0m\u001b[32m'main'\u001b[0m\u001b[1;36m to \u001b[0m\u001b[32m'../../../projects/exercise-bayesian-si-lbco-reference'\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "├── 📄 project.edi\n", "├── 📁 structures/\n", "│ └── 📄 lbco.edi\n", "│ └── 📄 si.edi\n", "├── 📁 experiments/\n", "│ └── 📄 sim_lbco.edi\n", "├── 📁 analysis/\n", "│ ├── 📄 analysis.edi\n", "│ └── 📄 mcmc.h5\n", "└── 📁 reports/\n", " └── 📄 main.html\n" ] } ], "source": [ "project_1.save()" ] }, { "cell_type": "markdown", "id": "55", "metadata": {}, "source": [ "## 💪 Exercise: Fix One Correlated Parameter\n", "\n", "The reference analysis showed that `broad_gauss_sigma_1` and\n", "`broad_gauss_sigma_2` are strongly negatively correlated. In this\n", "exercise, you will fix `broad_gauss_sigma_2` at its refined value and\n", "repeat the Bayesian workflow.\n", "\n", "Removing one member of the pair reduces the sampling dimension and\n", "removes that pair from the correlation and pair plots. This can improve\n", "sampling efficiency, but it changes the scientific question: the new\n", "posterior is conditional on the chosen fixed value of\n", "`broad_gauss_sigma_2`. Its uncertainty is no longer propagated. The\n", "exercise demonstrates the computational and visual effect of fixing a\n", "parameter; it does not establish that fixing it is always the correct\n", "scientific choice." ] }, { "cell_type": "markdown", "id": "56", "metadata": {}, "source": [ "### 📂 Exercise 1: Create a Fresh Project\n", "\n", "Load the original deterministic refinement again as `project_2` and\n", "save it under a new name. Starting from the deterministic project\n", "ensures that the second MCMC run does not reuse the first posterior\n", "state or its uncertainty estimates." ] }, { "cell_type": "markdown", "id": "57", "metadata": {}, "source": [ "**Hint:**" ] }, { "cell_type": "markdown", "id": "58", "metadata": {}, "source": [ "Use `edi.Project.load()` with `refinement_project_dir`, which was\n", "downloaded in the introduction." ] }, { "cell_type": "markdown", "id": "59", "metadata": {}, "source": [ "**Solution:**" ] }, { "cell_type": "code", "execution_count": 26, "id": "60", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:45.561059Z", "iopub.status.busy": "2026-08-18T18:22:45.560897Z", "iopub.status.idle": "2026-08-18T18:22:46.217291Z", "shell.execute_reply": "2026-08-18T18:22:46.216545Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mSaving project 📦 \u001b[0m\u001b[32m'main'\u001b[0m\u001b[1;36m to \u001b[0m\u001b[32m'../../../projects/exercise-bayesian-si-lbco-main'\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "├── 📄 project.edi\n", "├── 📁 structures/\n", "│ └── 📄 lbco.edi\n", "│ └── 📄 si.edi\n", "├── 📁 experiments/\n", "│ └── 📄 sim_lbco.edi\n", "├── 📁 analysis/\n", "│ └── 📄 analysis.edi\n", "└── 📁 reports/\n", " └── 📄 main.html\n" ] } ], "source": [ "project_2 = edi.Project.load(refinement_project_dir)\n", "project_2.metadata.title = 'Bayesian Analysis with Fixed broad_gauss_sigma_2'\n", "project_2.metadata.description = (\n", " 'MCMC analysis of LBCO and Si with one peak-profile parameter fixed.'\n", ")\n", "project_2.save_as(dir_path='projects/exercise-bayesian-si-lbco-main')" ] }, { "cell_type": "markdown", "id": "61", "metadata": {}, "source": [ "### 🎯 Exercise 2: Reduce the Free-Parameter Set\n", "\n", "#### Exercise 2.1: Fix the Background\n", "\n", "Fix every background intensity for the same computational reason as in\n", "the reference analysis." ] }, { "cell_type": "markdown", "id": "62", "metadata": {}, "source": [ "**Hint:**" ] }, { "cell_type": "markdown", "id": "63", "metadata": {}, "source": [ "Iterate over `project_2.experiments['sim_lbco'].background` and set\n", "each point's `intensity.free` attribute to `False`." ] }, { "cell_type": "markdown", "id": "64", "metadata": {}, "source": [ "**Solution:**" ] }, { "cell_type": "code", "execution_count": 27, "id": "65", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:46.219081Z", "iopub.status.busy": "2026-08-18T18:22:46.218925Z", "iopub.status.idle": "2026-08-18T18:22:46.222025Z", "shell.execute_reply": "2026-08-18T18:22:46.221331Z" } }, "outputs": [], "source": [ "experiment_2 = project_2.experiments['sim_lbco']\n", "\n", "for line_segment in experiment_2.background:\n", " line_segment.intensity.free = False" ] }, { "cell_type": "markdown", "id": "66", "metadata": {}, "source": [ "#### Exercise 2.2: Fix `broad_gauss_sigma_2`\n", "\n", "Remove `broad_gauss_sigma_2` from the refined and sampled parameter\n", "set, then display the remaining free parameters. How many remain?" ] }, { "cell_type": "markdown", "id": "67", "metadata": {}, "source": [ "**Hint:**" ] }, { "cell_type": "markdown", "id": "68", "metadata": {}, "source": [ "Set the parameter's `free` attribute to `False`, then call\n", "`project_2.display.parameters.free()`." ] }, { "cell_type": "markdown", "id": "69", "metadata": {}, "source": [ "**Solution:**" ] }, { "cell_type": "code", "execution_count": 28, "id": "70", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:46.223646Z", "iopub.status.busy": "2026-08-18T18:22:46.223499Z", "iopub.status.idle": "2026-08-18T18:22:46.226961Z", "shell.execute_reply": "2026-08-18T18:22:46.225957Z" } }, "outputs": [], "source": [ "experiment_2.peak.broad_gauss_sigma_2.free = False" ] }, { "cell_type": "code", "execution_count": 29, "id": "71", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:46.228301Z", "iopub.status.busy": "2026-08-18T18:22:46.228155Z", "iopub.status.idle": "2026-08-18T18:22:46.266629Z", "shell.execute_reply": "2026-08-18T18:22:46.265727Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mFree parameters for both structures \u001b[0m\u001b[1;36m(\u001b[0m\u001b[1;36m🧩 data blocks\u001b[0m\u001b[1;36m)\u001b[0m\u001b[1;36m and experiments \u001b[0m\u001b[1;36m(\u001b[0m\u001b[1;36m🔬 data blocks\u001b[0m\u001b[1;36m)\u001b[0m\n" ] }, { "data": { "text/html": [ "
datablockcategoryentryparametervalueuncertaintyminmaxunits
1lbcocelllength_a3.891470.00015-infinfÅ
2sim_lbcolinked_structurelbcoscale42.360000.24000-infinf
3sim_lbcolinked_structuresiscale0.327000.01200-infinf
4sim_lbcopeakrise_alpha_10.016550.00014-infinfμs/Å
5sim_lbcopeakdecay_beta_10.018570.00042-infinfμs/Å
6sim_lbcopeakbroad_gauss_sigma_1-68200.000001500.00000-infinfμs/Å
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_2.display.parameters.free()" ] }, { "cell_type": "markdown", "id": "72", "metadata": {}, "source": [ "Six parameters remain. The background intensities and\n", "`broad_gauss_sigma_2` will stay at their refined values throughout the\n", "following local fit and MCMC run." ] }, { "cell_type": "markdown", "id": "73", "metadata": {}, "source": [ "### 🚀 Exercise 3: Repeat the Local Refinement\n", "\n", "Select `bumps (lm)`, run the fit, display the result table, and inspect\n", "the local correlation chart. Is the original strongly correlated pair\n", "still present?" ] }, { "cell_type": "markdown", "id": "74", "metadata": {}, "source": [ "**Hint:**" ] }, { "cell_type": "markdown", "id": "75", "metadata": {}, "source": [ "Repeat the local-refinement sequence from the introduction with\n", "`project_2`." ] }, { "cell_type": "markdown", "id": "76", "metadata": {}, "source": [ "**Solution:**" ] }, { "cell_type": "code", "execution_count": 30, "id": "77", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:46.268166Z", "iopub.status.busy": "2026-08-18T18:22:46.268004Z", "iopub.status.idle": "2026-08-18T18:22:48.043935Z", "shell.execute_reply": "2026-08-18T18:22:48.042828Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mCurrent minimizer changed to\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "bumps \u001b[1m(\u001b[0mlm\u001b[1m)\u001b[0m\n" ] }, { "data": { "text/html": [], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "\n", "(function() {\n", " const button = document.getElementById('ed-fit-stop-91585775dd854421a9dac73d226c990c-button');\n", " const status = document.getElementById('ed-fit-stop-91585775dd854421a9dac73d226c990c-status');\n", " const kernelId = '';\n", " if (!button) {\n", " return;\n", " }\n", "\n", " function setStatus(text) {\n", " if (status) {\n", " status.textContent = text;\n", " }\n", " }\n", "\n", " function pageConfig() {\n", " const element = document.getElementById('jupyter-config-data');\n", " if (!element || !element.textContent) {\n", " return {};\n", " }\n", " try {\n", " return JSON.parse(element.textContent);\n", " } catch (error) {\n", " return {};\n", " }\n", " }\n", "\n", " function baseUrl(config) {\n", " const configured = config.baseUrl || config.base_url ||\n", " (window.Jupyter && Jupyter.notebook && Jupyter.notebook.base_url);\n", " if (configured) {\n", " return configured.endsWith('/') ? configured : configured + '/';\n", " }\n", " const markers = ['/lab/', '/notebooks/', '/tree/'];\n", " for (const marker of markers) {\n", " const index = window.location.pathname.indexOf(marker);\n", " if (index >= 0) {\n", " return window.location.pathname.slice(0, index + 1);\n", " }\n", " }\n", " return '/';\n", " }\n", "\n", " function token(config) {\n", " return config.token || new URLSearchParams(window.location.search).get('token') || '';\n", " }\n", "\n", " function cookie(name) {\n", " const prefix = name + '=';\n", " for (const part of document.cookie.split(';')) {\n", " const trimmed = part.trim();\n", " if (trimmed.startsWith(prefix)) {\n", " return decodeURIComponent(trimmed.slice(prefix.length));\n", " }\n", " }\n", " return '';\n", " }\n", "\n", " function notebookPath() {\n", " const decoded = decodeURIComponent(window.location.pathname);\n", " const markers = ['/lab/tree/', '/notebooks/', '/tree/'];\n", " for (const marker of markers) {\n", " const index = decoded.indexOf(marker);\n", " if (index >= 0) {\n", " return decoded.slice(index + marker.length);\n", " }\n", " }\n", " return '';\n", " }\n", "\n", " async function kernelFromSessions(config) {\n", " const url = new URL(baseUrl(config) + 'api/sessions', window.location.origin);\n", " const authToken = token(config);\n", " if (authToken) {\n", " url.searchParams.set('token', authToken);\n", " }\n", " const response = await fetch(url, {credentials: 'same-origin'});\n", " if (!response.ok) {\n", " return '';\n", " }\n", " const sessions = await response.json();\n", " const path = notebookPath();\n", " const session = sessions.find((item) => item.path === path) || sessions[0];\n", " return session && session.kernel ? session.kernel.id : '';\n", " }\n", "\n", " async function interruptKernel(config, resolvedKernelId) {\n", " const url = new URL(\n", " baseUrl(config) + 'api/kernels/' + resolvedKernelId + '/interrupt',\n", " window.location.origin\n", " );\n", " const authToken = token(config);\n", " if (authToken) {\n", " url.searchParams.set('token', authToken);\n", " }\n", " const xsrfToken = cookie('_xsrf');\n", " const headers = {};\n", " if (xsrfToken) {\n", " headers['X-XSRFToken'] = xsrfToken;\n", " }\n", " const response = await fetch(url, {\n", " method: 'POST',\n", " credentials: 'same-origin',\n", " headers: headers\n", " });\n", " return response.ok;\n", " }\n", "\n", " button.addEventListener('click', async function() {\n", " button.disabled = true;\n", " setStatus('Stopping...');\n", " const config = pageConfig();\n", " try {\n", " const resolvedKernelId = kernelId || await kernelFromSessions(config);\n", " if (!resolvedKernelId) {\n", " throw new Error('Could not resolve the current kernel id.');\n", " }\n", " const interrupted = await interruptKernel(config, resolvedKernelId);\n", " if (!interrupted) {\n", " throw new Error('Jupyter Server rejected the interrupt request.');\n", " }\n", " setStatus('Interrupt sent...');\n", " } catch (error) {\n", " button.disabled = false;\n", " setStatus('Use Kernel > Interrupt to stop this fit.');\n", " }\n", " });\n", "})();\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mStandard fitting\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "📋 Using experiment 🔬 \u001b[32m'sim_lbco'\u001b[0m for \u001b[32m'single'\u001b[0m fitting\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "🚀 Starting fit process with \u001b[32m'bumps \u001b[0m\u001b[32m(\u001b[0m\u001b[32mlm\u001b[0m\u001b[32m)\u001b[0m\u001b[32m'\u001b[0m\u001b[33m...\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "📈 Goodness-of-fit progress:\n" ] }, { "data": { "text/html": [ "
iterationtime (s)χ²change / status
110.021.58
2241.161.58
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "🏆 Best goodness-of-fit \u001b[1m(\u001b[0mreduced χ²\u001b[1m)\u001b[0m is \u001b[1;36m1.58\u001b[0m at iteration \u001b[1;36m15\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "✅ Fitting complete.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "⚙️ Settings used:\n" ] }, { "data": { "text/html": [ "
NameValueDescription
1max_iterations1000Maximum solver iterations.
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "📋 Least-squares fit results:\n" ] }, { "data": { "text/html": [ "
MetricValue
1🧪 Minimizerbumps (lm)
2✅ Overall statussuccess
3⏱️ Fitting time (seconds)1.16
4📏 Goodness-of-fit (reduced χ²)1.58
5📏 R-factor (Rf, %)5.06
6📏 R-factor squared (Rf², %)4.61
7📏 Weighted R-factor (wR, %)4.04
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "📈 Refined parameters:\n" ] }, { "data": { "text/html": [ "
datablockcategoryentryparameterunitsstartvalues.u.change
1lbcocelllength_aÅ3.89153.89150.00020.00 % ↑
2sim_lbcolinked_structurelbcoscale42.360042.36070.22890.00 % ↑
3sim_lbcolinked_structuresiscale0.32700.32660.01110.14 % ↓
4sim_lbcopeakrise_alpha_1μs/Å0.01650.01660.00010.02 % ↑
5sim_lbcopeakdecay_beta_1μs/Å0.01860.01860.00040.03 % ↑
6sim_lbcopeakbroad_gauss_sigma_1μs/Å-68200.0000-68202.5586509.26450.00 % ↑
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
• start = parameter value before refinement
• value = refined value from least-squares minimization
• s.u. = standard uncertainty (one sigma), from the covariance matrix
• change = relative change from start, in %; ↑ = increase, ↓ = decrease
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_2.analysis.minimizer.type = 'bumps (lm)'\n", "project_2.analysis.fit()\n", "project_2.display.fit.results()" ] }, { "cell_type": "code", "execution_count": 31, "id": "78", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:48.045631Z", "iopub.status.busy": "2026-08-18T18:22:48.045437Z", "iopub.status.idle": "2026-08-18T18:22:48.067838Z", "shell.execute_reply": "2026-08-18T18:22:48.067063Z" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "
\n", "
Loading plot…
\n", "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_2.display.fit.correlations(max_parameters=5)" ] }, { "cell_type": "markdown", "id": "79", "metadata": {}, "source": [ "The `broad_gauss_sigma_1`–`broad_gauss_sigma_2` pair is absent because\n", "`broad_gauss_sigma_2` was not varied. This does not show that the\n", "physical ambiguity disappeared; it shows that the fixed parameter is\n", "no longer part of the estimated covariance or posterior." ] }, { "cell_type": "markdown", "id": "80", "metadata": {}, "source": [ "### 🎲 Exercise 4: Set New Sampling Bounds\n", "\n", "Derive finite bounds from the new local-fit uncertainties and verify\n", "them. Why is it better to recalculate the bounds than to copy them from\n", "`project_1`?" ] }, { "cell_type": "markdown", "id": "81", "metadata": {}, "source": [ "**Hint:**" ] }, { "cell_type": "markdown", "id": "82", "metadata": {}, "source": [ "Iterate over `project_2.free_parameters` and call\n", "`set_fit_bounds_from_uncertainty()`." ] }, { "cell_type": "markdown", "id": "83", "metadata": {}, "source": [ "**Solution:**" ] }, { "cell_type": "code", "execution_count": 32, "id": "84", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:48.069351Z", "iopub.status.busy": "2026-08-18T18:22:48.069195Z", "iopub.status.idle": "2026-08-18T18:22:48.076247Z", "shell.execute_reply": "2026-08-18T18:22:48.075401Z" } }, "outputs": [], "source": [ "for param in project_2.free_parameters:\n", " param.set_fit_bounds_from_uncertainty()" ] }, { "cell_type": "code", "execution_count": 33, "id": "85", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:48.077686Z", "iopub.status.busy": "2026-08-18T18:22:48.077533Z", "iopub.status.idle": "2026-08-18T18:22:48.108781Z", "shell.execute_reply": "2026-08-18T18:22:48.108053Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mFree parameters for both structures \u001b[0m\u001b[1;36m(\u001b[0m\u001b[1;36m🧩 data blocks\u001b[0m\u001b[1;36m)\u001b[0m\u001b[1;36m and experiments \u001b[0m\u001b[1;36m(\u001b[0m\u001b[1;36m🔬 data blocks\u001b[0m\u001b[1;36m)\u001b[0m\n" ] }, { "data": { "text/html": [ "
datablockcategoryentryparametervalueuncertaintyminmaxunits
1lbcocelllength_a3.891480.000153.890863.89209Å
2sim_lbcolinked_structurelbcoscale42.360720.2288641.4452743.27618
3sim_lbcolinked_structuresiscale0.326560.011150.281970.37115
4sim_lbcopeakrise_alpha_10.016550.000130.016030.01708μs/Å
5sim_lbcopeakdecay_beta_10.018580.000400.016960.02019μs/Å
6sim_lbcopeakbroad_gauss_sigma_1-68202.55856509.26448-70239.61649-66165.50064μs/Å
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_2.display.parameters.free()" ] }, { "cell_type": "markdown", "id": "86", "metadata": {}, "source": [ "Fixing one correlated parameter changes the local covariance matrix and\n", "therefore the uncertainty estimates of the remaining parameters. The\n", "new bounds should be based on this new local problem." ] }, { "cell_type": "markdown", "id": "87", "metadata": {}, "source": [ "### 🎲 Exercise 5: Repeat DREAM Sampling\n", "\n", "Configure DREAM with the same short-chain settings used for\n", "`project_1`, then sample the six-parameter posterior." ] }, { "cell_type": "markdown", "id": "88", "metadata": {}, "source": [ "**Hint:**" ] }, { "cell_type": "markdown", "id": "89", "metadata": {}, "source": [ "Use 300 sampling steps, 60 burn-in steps, and random seed 42 so the two\n", "runs use comparable settings." ] }, { "cell_type": "markdown", "id": "90", "metadata": {}, "source": [ "**Solution:**" ] }, { "cell_type": "code", "execution_count": 34, "id": "91", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:48.110750Z", "iopub.status.busy": "2026-08-18T18:22:48.110440Z", "iopub.status.idle": "2026-08-18T18:22:48.118306Z", "shell.execute_reply": "2026-08-18T18:22:48.117649Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "⚠️ Switching minimizer type removes these settings: \n", " • max_iterations \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "⚠️ Switching minimizer type adds these settings with defaults: \n", " • burn_in_steps=600 \n", " • initialization_method='latin_hypercube' \n", " • parallel_workers=0 \n", " • population_size=4 \n", " • random_seed=None \n", " • sampling_steps=3000 \n", " • thinning_interval=1 \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mCurrent minimizer changed to\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "bumps \u001b[1m(\u001b[0mdream\u001b[1m)\u001b[0m\n" ] } ], "source": [ "project_2.analysis.minimizer.type = 'bumps (dream)'\n", "project_2.analysis.minimizer.sampling_steps = 300\n", "project_2.analysis.minimizer.burn_in_steps = 60\n", "project_2.analysis.minimizer.random_seed = 42" ] }, { "cell_type": "code", "execution_count": 35, "id": "92", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:22:48.120095Z", "iopub.status.busy": "2026-08-18T18:22:48.119897Z", "iopub.status.idle": "2026-08-18T18:25:29.850804Z", "shell.execute_reply": "2026-08-18T18:25:29.850233Z" } }, "outputs": [ { "data": { "text/html": [], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "\n", "(function() {\n", " const button = document.getElementById('ed-fit-stop-c2cbe6df0fd74d37b2a281625a19c220-button');\n", " const status = document.getElementById('ed-fit-stop-c2cbe6df0fd74d37b2a281625a19c220-status');\n", " const kernelId = '';\n", " if (!button) {\n", " return;\n", " }\n", "\n", " function setStatus(text) {\n", " if (status) {\n", " status.textContent = text;\n", " }\n", " }\n", "\n", " function pageConfig() {\n", " const element = document.getElementById('jupyter-config-data');\n", " if (!element || !element.textContent) {\n", " return {};\n", " }\n", " try {\n", " return JSON.parse(element.textContent);\n", " } catch (error) {\n", " return {};\n", " }\n", " }\n", "\n", " function baseUrl(config) {\n", " const configured = config.baseUrl || config.base_url ||\n", " (window.Jupyter && Jupyter.notebook && Jupyter.notebook.base_url);\n", " if (configured) {\n", " return configured.endsWith('/') ? configured : configured + '/';\n", " }\n", " const markers = ['/lab/', '/notebooks/', '/tree/'];\n", " for (const marker of markers) {\n", " const index = window.location.pathname.indexOf(marker);\n", " if (index >= 0) {\n", " return window.location.pathname.slice(0, index + 1);\n", " }\n", " }\n", " return '/';\n", " }\n", "\n", " function token(config) {\n", " return config.token || new URLSearchParams(window.location.search).get('token') || '';\n", " }\n", "\n", " function cookie(name) {\n", " const prefix = name + '=';\n", " for (const part of document.cookie.split(';')) {\n", " const trimmed = part.trim();\n", " if (trimmed.startsWith(prefix)) {\n", " return decodeURIComponent(trimmed.slice(prefix.length));\n", " }\n", " }\n", " return '';\n", " }\n", "\n", " function notebookPath() {\n", " const decoded = decodeURIComponent(window.location.pathname);\n", " const markers = ['/lab/tree/', '/notebooks/', '/tree/'];\n", " for (const marker of markers) {\n", " const index = decoded.indexOf(marker);\n", " if (index >= 0) {\n", " return decoded.slice(index + marker.length);\n", " }\n", " }\n", " return '';\n", " }\n", "\n", " async function kernelFromSessions(config) {\n", " const url = new URL(baseUrl(config) + 'api/sessions', window.location.origin);\n", " const authToken = token(config);\n", " if (authToken) {\n", " url.searchParams.set('token', authToken);\n", " }\n", " const response = await fetch(url, {credentials: 'same-origin'});\n", " if (!response.ok) {\n", " return '';\n", " }\n", " const sessions = await response.json();\n", " const path = notebookPath();\n", " const session = sessions.find((item) => item.path === path) || sessions[0];\n", " return session && session.kernel ? session.kernel.id : '';\n", " }\n", "\n", " async function interruptKernel(config, resolvedKernelId) {\n", " const url = new URL(\n", " baseUrl(config) + 'api/kernels/' + resolvedKernelId + '/interrupt',\n", " window.location.origin\n", " );\n", " const authToken = token(config);\n", " if (authToken) {\n", " url.searchParams.set('token', authToken);\n", " }\n", " const xsrfToken = cookie('_xsrf');\n", " const headers = {};\n", " if (xsrfToken) {\n", " headers['X-XSRFToken'] = xsrfToken;\n", " }\n", " const response = await fetch(url, {\n", " method: 'POST',\n", " credentials: 'same-origin',\n", " headers: headers\n", " });\n", " return response.ok;\n", " }\n", "\n", " button.addEventListener('click', async function() {\n", " button.disabled = true;\n", " setStatus('Stopping...');\n", " const config = pageConfig();\n", " try {\n", " const resolvedKernelId = kernelId || await kernelFromSessions(config);\n", " if (!resolvedKernelId) {\n", " throw new Error('Could not resolve the current kernel id.');\n", " }\n", " const interrupted = await interruptKernel(config, resolvedKernelId);\n", " if (!interrupted) {\n", " throw new Error('Jupyter Server rejected the interrupt request.');\n", " }\n", " setStatus('Interrupt sent...');\n", " } catch (error) {\n", " button.disabled = false;\n", " setStatus('Use Kernel > Interrupt to stop this fit.');\n", " }\n", " });\n", "})();\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mStandard fitting\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "📋 Using experiment 🔬 \u001b[32m'sim_lbco'\u001b[0m for \u001b[32m'single'\u001b[0m fitting\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "🚀 Starting fit process with \u001b[32m'bumps \u001b[0m\u001b[32m(\u001b[0m\u001b[32mdream\u001b[0m\u001b[32m)\u001b[0m\u001b[32m'\u001b[0m\u001b[33m...\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "📈 Bayesian sampling progress:\n" ] }, { "data": { "text/html": [ "
stepprogresstime (s)log posteriorphase
10.34-416.89pre-processing
221/3605.8%6.21-344.12burn-in
340/36011.1%12.12-339.61burn-in
460/36016.7%18.86-337.61burn-in
561/36016.9%19.16-337.54sampling
676/36021.1%23.49-337.57sampling
791/36025.3%28.64-337.44sampling
8106/36029.4%32.94-337.81sampling
9121/36033.6%37.22-337.40sampling
10136/36037.8%42.49-337.05sampling
11151/36041.9%46.82-336.71sampling
12166/36046.1%51.47-336.64sampling
13181/36050.3%56.18-336.74sampling
14196/36054.4%60.71-336.34sampling
15211/36058.6%65.63-337.28sampling
16226/36062.8%70.28-336.99sampling
17241/36066.9%74.64-337.34sampling
18256/36071.1%79.68-336.46sampling
19271/36075.3%84.12-336.20sampling
20286/36079.4%88.81-336.28sampling
21301/36083.6%93.85-336.98sampling
22316/36087.8%98.21-336.51sampling
23331/36091.9%102.75-337.36sampling
24346/36096.1%107.84-337.14sampling
25360/360100.0%112.17-337.66sampling
26161.68post-processing
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "✅ Bayesian sampling complete.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "⚠️ Convergence diagnostics indicate the posterior may be poorly mixed. \n" ] } ], "source": [ "project_2.analysis.fit()" ] }, { "cell_type": "markdown", "id": "93", "metadata": {}, "source": [ "### 📊 Exercise 6: Compare the Posterior Results\n", "\n", "#### Exercise 6.1: Check Convergence and Credible Intervals\n", "\n", "Display the Bayesian result table. Compare r-hat, effective sample\n", "size, and the credible intervals with the reference run. Did fixing one\n", "parameter automatically guarantee convergence?" ] }, { "cell_type": "markdown", "id": "94", "metadata": {}, "source": [ "**Hint:**" ] }, { "cell_type": "markdown", "id": "95", "metadata": {}, "source": [ "Use `project_2.display.fit.results()`, then compare the diagnostics\n", "with those displayed for `project_1` in the introduction." ] }, { "cell_type": "markdown", "id": "96", "metadata": {}, "source": [ "**Solution:**" ] }, { "cell_type": "code", "execution_count": 36, "id": "97", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:25:29.855000Z", "iopub.status.busy": "2026-08-18T18:25:29.854783Z", "iopub.status.idle": "2026-08-18T18:25:29.900766Z", "shell.execute_reply": "2026-08-18T18:25:29.899633Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "⚙️ Settings used:\n" ] }, { "data": { "text/html": [ "
NameValueDescription
1sampling_steps300Total sampler iterations per chain.
2burn_in_steps60Sampler iterations discarded as warm-up.
3thinning_interval1Sampler thinning interval.
4population_size4Number of chains or walkers.
5parallel_workers0Worker count; 0 uses all available CPUs.
6initialization_methodlatin_hypercubeSampler initialization method.
7random_seed42Random seed; None uses a system-derived seed.
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "📋 Bayesian fit results:\n" ] }, { "data": { "text/html": [ "
MetricValue
1🧪 Samplerbumps (dream)
2❌ Overall statusfailed
3💬 Engine messageDREAM sampling completed
4⏱️ Fitting time (seconds)161.68
5📏 Goodness-of-fit (reduced χ²)1.58
6📏 R-factor (Rf, %)5.06
7📏 R-factor squared (Rf², %)4.61
8📏 Weighted R-factor (wR, %)4.04
9📉 Best log-posterior-334.00
10📊 Convergence statusfailed
11📊 Max r-hat1.112
12📊 Min ess bulk307.085
13📊 Draws per chain300
14📊 Chains24
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "📈 Committed parameters:\n" ] }, { "data": { "text/html": [ "
datablockcategoryentryparameterunitsstartvalues.u.change
1lbcocelllength_aÅ3.89153.89150.00010.00 % ↓
2sim_lbcolinked_structurelbcoscale42.360742.36070.18600.00 % ↓
3sim_lbcolinked_structuresiscale0.32660.32660.00910.00 % ↓
4sim_lbcopeakrise_alpha_1μs/Å0.01660.01660.00010.00 % ↓
5sim_lbcopeakdecay_beta_1μs/Å0.01860.01860.00030.00 % ↓
6sim_lbcopeakbroad_gauss_sigma_1μs/Å-68202.5586-68202.5586412.57310.00 % ↓
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
• start = parameter value before sampling
• value = estimate written back to the project (best posterior sample)
• s.u. = standard uncertainty (one sigma), posterior standard deviation
• change = relative change from start, in %; ↑ = increase, ↓ = decrease
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "📊 Posterior distribution:\n" ] }, { "data": { "text/html": [ "
datablockcategoryentryparameterunitsmedian95% CIr-hatess bulk
1lbcocelllength_aÅ3.8915[3.8912, 3.8917]1.107307.1
2sim_lbcolinked_structurelbcoscale42.3328[41.9703, 42.6813]1.088419.3
3sim_lbcolinked_structuresiscale0.3269[0.3084, 0.3454]1.094406.7
4sim_lbcopeakrise_alpha_1μs/Å0.0166[0.0164, 0.0168]1.091415.1
5sim_lbcopeakdecay_beta_1μs/Å0.0186[0.0179, 0.0194]1.112313.1
6sim_lbcopeakbroad_gauss_sigma_1μs/Å-68155.3315[-68965.5420, -67306.6883]1.108381.2
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
• median = 50th percentile of the marginal posterior
• 95% CI = 95% credible interval (2.5%-97.5%, asymmetric)
• r-hat = Gelman-Rubin diagnostic (good convergence: r-hat <= 1.01)
• ess bulk = bulk effective sample size (typically >= 400)
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
⚠️ r-hat > 1.01: Consider longer sampling, better initialization, or reparameterization.
⚠️ ess bulk < 400: Consider longer sampling or reparameterization.
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_2.display.fit.results()" ] }, { "cell_type": "markdown", "id": "98", "metadata": {}, "source": [ "Reducing the dimension can make sampling easier, but a 300-step chain\n", "is still short. Convergence must be judged from the diagnostics, not\n", "assumed from the number of free parameters." ] }, { "cell_type": "markdown", "id": "99", "metadata": {}, "source": [ "#### Exercise 6.2: Inspect Correlations and Pair Relationships\n", "\n", "Display the posterior correlation matrix and pair plot. Confirm that\n", "`broad_gauss_sigma_2` is absent, then inspect whether\n", "`broad_gauss_sigma_1` is correlated with any remaining parameter." ] }, { "cell_type": "markdown", "id": "100", "metadata": {}, "source": [ "**Hint:**" ] }, { "cell_type": "markdown", "id": "101", "metadata": {}, "source": [ "Use the same `display.fit.correlations(max_parameters=5)` and\n", "`display.posterior.pairs(max_parameters=5)` calls as in the\n", "introduction. Hover over cells and sample points to identify the\n", "parameter pairs and values." ] }, { "cell_type": "markdown", "id": "102", "metadata": {}, "source": [ "**Solution:**" ] }, { "cell_type": "code", "execution_count": 37, "id": "103", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:25:29.904024Z", "iopub.status.busy": "2026-08-18T18:25:29.903800Z", "iopub.status.idle": "2026-08-18T18:25:29.960165Z", "shell.execute_reply": "2026-08-18T18:25:29.959499Z" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "
\n", "
Loading plot…
\n", "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_2.display.fit.correlations(max_parameters=5)" ] }, { "cell_type": "code", "execution_count": 38, "id": "104", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:25:29.964060Z", "iopub.status.busy": "2026-08-18T18:25:29.963702Z", "iopub.status.idle": "2026-08-18T18:25:30.315701Z", "shell.execute_reply": "2026-08-18T18:25:30.314504Z" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "
\n", "
Loading plot…
\n", "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_2.display.posterior.pairs(max_parameters=5)" ] }, { "cell_type": "markdown", "id": "105", "metadata": {}, "source": [ "The original pair is gone because only sampled parameters appear in\n", "posterior charts. Look for any remaining elongated contours rather\n", "than concluding that all correlations have disappeared. Fixing one\n", "parameter can expose or strengthen relationships among the parameters\n", "that remain free." ] }, { "cell_type": "markdown", "id": "106", "metadata": {}, "source": [ "#### Exercise 6.3: Compare `broad_gauss_sigma_1`\n", "\n", "Plot the marginal posterior for `broad_gauss_sigma_1` from both runs.\n", "Compare the medians, shapes, and 95% credible intervals. Why might the\n", "second interval be narrower?" ] }, { "cell_type": "markdown", "id": "107", "metadata": {}, "source": [ "**Hint:**" ] }, { "cell_type": "markdown", "id": "108", "metadata": {}, "source": [ "Call `display.posterior.distribution()` for each project and pass the\n", "corresponding `broad_gauss_sigma_1` parameter using `param`." ] }, { "cell_type": "markdown", "id": "109", "metadata": {}, "source": [ "**Solution:**" ] }, { "cell_type": "code", "execution_count": 39, "id": "110", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:25:30.323844Z", "iopub.status.busy": "2026-08-18T18:25:30.323620Z", "iopub.status.idle": "2026-08-18T18:25:30.347711Z", "shell.execute_reply": "2026-08-18T18:25:30.346345Z" } }, "outputs": [ { "data": { "text/html": [ "
Loading plot…
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.display.posterior.distribution(\n", " param=experiment_1.peak.broad_gauss_sigma_1,\n", ")" ] }, { "cell_type": "code", "execution_count": 40, "id": "111", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:25:30.349900Z", "iopub.status.busy": "2026-08-18T18:25:30.349723Z", "iopub.status.idle": "2026-08-18T18:25:30.372780Z", "shell.execute_reply": "2026-08-18T18:25:30.372072Z" } }, "outputs": [ { "data": { "text/html": [ "
Loading plot…
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_2.display.posterior.distribution(\n", " param=experiment_2.peak.broad_gauss_sigma_1,\n", ")" ] }, { "cell_type": "markdown", "id": "112", "metadata": {}, "source": [ "The conditional posterior can be narrower because\n", "`broad_gauss_sigma_1` no longer shares uncertainty with\n", "`broad_gauss_sigma_2`. That apparent precision comes from assuming the\n", "fixed value is exact. It may underestimate the true uncertainty if\n", "`broad_gauss_sigma_2` is not independently known." ] }, { "cell_type": "markdown", "id": "113", "metadata": {}, "source": [ "#### Exercise 6.4: Compare Posterior Predictions\n", "\n", "Plot the same zoomed posterior-predictive region for both analyses.\n", "Does fixing `broad_gauss_sigma_2` noticeably change the best curve or\n", "uncertainty band?" ] }, { "cell_type": "markdown", "id": "114", "metadata": {}, "source": [ "**Hint:**" ] }, { "cell_type": "markdown", "id": "115", "metadata": {}, "source": [ "Call `display.posterior.predictive()` for both projects with the same\n", "experiment name and the same `x_min` and `x_max` values." ] }, { "cell_type": "markdown", "id": "116", "metadata": {}, "source": [ "**Solution:**" ] }, { "cell_type": "code", "execution_count": 41, "id": "117", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:25:30.378433Z", "iopub.status.busy": "2026-08-18T18:25:30.378076Z", "iopub.status.idle": "2026-08-18T18:25:30.443114Z", "shell.execute_reply": "2026-08-18T18:25:30.439605Z" } }, "outputs": [ { "data": { "text/html": [ "
Loading plot…
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_1.display.posterior.predictive(\n", " expt_name='sim_lbco',\n", " x_min=80000,\n", " x_max=81500,\n", ")" ] }, { "cell_type": "code", "execution_count": 42, "id": "118", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:25:30.446631Z", "iopub.status.busy": "2026-08-18T18:25:30.445308Z", "iopub.status.idle": "2026-08-18T18:25:30.534900Z", "shell.execute_reply": "2026-08-18T18:25:30.534047Z" } }, "outputs": [ { "data": { "text/html": [ "
Loading plot…
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "project_2.display.posterior.predictive(\n", " expt_name='sim_lbco',\n", " x_min=80000,\n", " x_max=81500,\n", ")" ] }, { "cell_type": "markdown", "id": "119", "metadata": {}, "source": [ "Two parameterizations can produce similarly good calculated patterns\n", "while assigning different uncertainties to individual parameters.\n", "This is why parameter correlations, marginal posteriors, and posterior\n", "predictions should be interpreted together." ] }, { "cell_type": "markdown", "id": "120", "metadata": {}, "source": [ "### 💾 Exercise 7: Save the Project\n", "\n", "Save the second posterior and its MCMC chain." ] }, { "cell_type": "markdown", "id": "121", "metadata": {}, "source": [ "**Hint:**" ] }, { "cell_type": "markdown", "id": "122", "metadata": {}, "source": [ "The project directory was set in Exercise 1, so use `project_2.save()`\n", "to update the existing saved project." ] }, { "cell_type": "markdown", "id": "123", "metadata": {}, "source": [ "**Solution:**" ] }, { "cell_type": "code", "execution_count": 43, "id": "124", "metadata": { "execution": { "iopub.execute_input": "2026-08-18T18:25:30.538140Z", "iopub.status.busy": "2026-08-18T18:25:30.537793Z", "iopub.status.idle": "2026-08-18T18:25:30.873735Z", "shell.execute_reply": "2026-08-18T18:25:30.872444Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1;36mSaving project 📦 \u001b[0m\u001b[32m'main'\u001b[0m\u001b[1;36m to \u001b[0m\u001b[32m'../../../projects/exercise-bayesian-si-lbco-main'\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "├── 📄 project.edi\n", "├── 📁 structures/\n", "│ └── 📄 lbco.edi\n", "│ └── 📄 si.edi\n", "├── 📁 experiments/\n", "│ └── 📄 sim_lbco.edi\n", "├── 📁 analysis/\n", "│ ├── 📄 analysis.edi\n", "│ └── 📄 mcmc.h5\n", "└── 📁 reports/\n", " └── 📄 main.html\n" ] } ], "source": [ "project_2.save()" ] }, { "cell_type": "markdown", "id": "125", "metadata": {}, "source": [ "#### Final Remarks\n", "\n", "In this part of the notebook, you learned how to:\n", "\n", "- prepare a refined EasyDiffraction project for MCMC;\n", "- reduce runtime by fixing nuisance parameters while recognizing the\n", " uncertainty tradeoff;\n", "- use a local fit to obtain starting values and finite bounds;\n", "- sample a correlated posterior with DREAM;\n", "- distinguish a local covariance correlation from a posterior-sample\n", " correlation;\n", "- read correlation matrices, pair plots, marginal distributions, and\n", " posterior-predictive plots; and\n", "- understand how fixing one member of a correlated pair changes the\n", " statistical question and the reported uncertainty.\n", "\n", "For a scientific analysis, run longer chains, verify convergence,\n", "examine sensitivity to bounds and fixed values, and reconsider the\n", "diffraction model when systematic residuals remain." ] }, { "cell_type": "markdown", "id": "126", "metadata": {}, "source": [ "## 🎁 Bonus\n", "\n", "Congratulations — you've now completed Part 2 of the diffraction data\n", "analysis exercises for the DMSC Summer School!\n", "\n", "If you'd like to keep exploring, the EasyDiffraction library offers\n", "many additional tutorials and examples on the official documentation\n", "site: 👉 https://docs.easydiffraction.org/lib/latest/tutorials\n", "\n", "Besides the Python package, EasyDiffraction also comes with a\n", "graphical user interface (GUI) for deterministic diffraction\n", "refinement workflows. Bayesian analysis is not yet available in the\n", "GUI, so MCMC workflows currently require the Python library.\n", "\n", "If you prefer a point-and-click interface over coding, the GUI\n", "provides a user-friendly way to perform deterministic refinements. You\n", "can download it as a standalone application here: 👉\n", "https://easydiffraction.org\n", "\n", "We'd love to hear your feedback on EasyDiffraction — both the library\n", "and the GUI! 💬" ] } ], "metadata": { "jupytext": { "cell_metadata_filter": "tags,-all", "main_language": "python", "notebook_metadata_filter": "-all" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.14.6" } }, "nbformat": 4, "nbformat_minor": 5 }