{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Parameter Learning in Discrete Bayesian Networks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this notebook, we demonstrate examples of learning the parameters (CPDs) of a Discrete Bayesian Network given the data and the model structure. pgmpy has three main algorithms for learning model parameters:\n", "\n", "1. **Maximum Likelihood Estimator** (`pgmpy.estimators.MaximumLikelihoodEstimator`): Simply estimates the Maximum Likelihood estimates of the parameters.\n", "2. **Bayesian Estimator** (`pgmpy.estimators.BayesianEstimator`): Allows users to specify priors. \n", "3. **Expectation Maximization** (`pgmpy.estimators.ExpectationMaximization`): Enables learning model parameters when latent variables are present in the model.\n", "\n", "Each of the parameter estimation classes have the following two methods:\n", "\n", "1. `estimate_cpd`: Estimates the CPD of the specified variable.\n", "2. `get_parameters`: Estimates the CPDs of all the variables in the model.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 0: Generate some simulated data and a model structure\n", "\n", "To do parameter estimation we require two things:\n", "1. **Data**: For the examples, we simulate some data from the alarm model (https://www.bnlearn.com/bnrepository/discrete-medium.html#alarm) and use it to learn back the model parameters.\n", "2. **Model structure**: We also need to specify the model structure to which to fit the data to. In this example, we simply use the structure to the alarm model." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "07163638093f43efa83db9a97b16347b", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/37 [00:00" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Estimate the CPD of node CVP\n", "mle_est.estimate_cpd(node=\"CVP\")" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Estimate all the CPDs for `new_model`\n", "all_cpds = mle_est.get_parameters(n_jobs=1)\n", "\n", "# Add the estimated CPDs to the model.\n", "new_model.add_cpds(*all_cpds)\n", "\n", "# Check if the CPDs are added to the model\n", "new_model.get_cpds('PCWP')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using the Bayesian Estimator" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "# Initialize the Bayesian Estimator\n", "from pgmpy.estimators import BayesianEstimator\n", "\n", "be_est = BayesianEstimator(model=new_model, data=samples)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The estimator methods in `BayesianEstimator` class allows for a few different ways to specify the priors. The prior type can be chosen by specifying the `prior_type` argument. Please refer the documentation for details on different ways these priors can be specified: https://pgmpy.org/param_estimator/bayesian_est.html#bayesian-estimator\n", "\n", "1. **Dirichlet prior** (`prior_type=\"dirichlet\"`): Requires specifying `pseudo_counts` argument. The pseudo_counts arguments specifies the priors to use for the CPD estimation.\n", "2. **BDeu prior** (`prior_type=\"BDeu\"`): Requires specifying `equivalent_sample_size` arguemnt. The equivaluent_sample_size is used to compute the priors to use for CPD estimation. \n", "3. **K2** (`prior_type=\"K2\"`): Short hand for dirichlet prior with pseudo_count=1." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "+--------------+-------+\n", "| FIO2(LOW) | 0.279 |\n", "+--------------+-------+\n", "| FIO2(NORMAL) | 0.721 |\n", "+--------------+-------+\n", "+-------------+---------------------+-----+---------------------+\n", "| LVEDVOLUME | LVEDVOLUME(HIGH) | ... | LVEDVOLUME(NORMAL) |\n", "+-------------+---------------------+-----+---------------------+\n", "| CVP(HIGH) | 0.48841698841698844 | ... | 0.10685483870967742 |\n", "+-------------+---------------------+-----+---------------------+\n", "| CVP(LOW) | 0.19884169884169883 | ... | 0.13709677419354838 |\n", "+-------------+---------------------+-----+---------------------+\n", "| CVP(NORMAL) | 0.3127413127413127 | ... | 0.7560483870967742 |\n", "+-------------+---------------------+-----+---------------------+\n" ] } ], "source": [ "print(be_est.estimate_cpd(node=\"FIO2\", prior_type=\"BDeu\", equivalent_sample_size=1000))\n", "print(be_est.estimate_cpd(node=\"CVP\", prior_type=\"dirichlet\", pseudo_counts=100))" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ]" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "be_est.get_parameters(prior_type=\"K2\", equivalent_sample_size=1000)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using Expectation Maximization " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Expectation Maximization (EM) estimator can work in the case when latent variables are present in the model. To simulate this scenario, we will specify some of the variables in our `new_model` as latents and drop those variables from `samples` to simulate missing data." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "model_latent = DiscreteBayesianNetwork(alarm_model.edges(), latents={'HISTORY', 'CVP'})\n", "samples_latent = samples.drop(['HISTORY', 'CVP'], axis=1)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "9ce6966d366a427798a81f190072b282", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/100 [00:00,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ]" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from pgmpy.estimators import ExpectationMaximization as EM\n", "em_est = EM(model=model_latent, data=samples_latent)\n", "em_est.get_parameters()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Shortcut for learning and adding CPDs to the model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `BayesianNetwork` class also provies a `fit` method that acts as a shortcut way to estimate and add CPDs to the model." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "+--------------+-------+\n", "| FIO2(LOW) | 0.058 |\n", "+--------------+-------+\n", "| FIO2(NORMAL) | 0.942 |\n", "+--------------+-------+\n", "+--------------+-------+\n", "| FIO2(LOW) | 0.279 |\n", "+--------------+-------+\n", "| FIO2(NORMAL) | 0.721 |\n", "+--------------+-------+\n" ] } ], "source": [ "# Shortcut for learning all the parameters and adding the CPDs to the model.\n", "\n", "model_struct = DiscreteBayesianNetwork(ebunch=alarm_model.edges())\n", "model_struct.fit(data=samples, estimator=MaximumLikelihoodEstimator)\n", "print(model_struct.get_cpds(\"FIO2\"))\n", "\n", "model_struct = DiscreteBayesianNetwork(ebunch=alarm_model.edges())\n", "model_struct.fit(\n", " data=samples,\n", " estimator=BayesianEstimator,\n", " prior_type=\"BDeu\",\n", " equivalent_sample_size=1000,\n", ")\n", "print(model_struct.get_cpds(\"FIO2\"))" ] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.0" } }, "nbformat": 4, "nbformat_minor": 4 }