PC (Constraint-Based Estimator)

class pgmpy.estimators.PC(data: DataFrame | None = None, independencies: Independencies | None = None, **kwargs)

Class for constraint-based estimation of DAGs using the PC algorithm from a given data set. Identifies (conditional) dependencies in data set using statistical independence tests and estimates a DAG pattern that satisfies the identified dependencies. The DAG pattern can then be completed to a faithful DAG, if possible.

When used with expert knowledge, the following flowchart can help you figure out the expected results based on different choices of parameters and the structure learned from the data.

┌──────────────────┐ No ┌─────────────┐ │ Expert Knowledge ├──────────► │ Normal PC │ │ specified? │ │ run │ └────────┬─────────┘ └─────────────┘

│

Yes │

│ ▼

┌──────────────────┐ │ Enforce expert │ │ knowledge? │ └────────┬─────────┘

│ │

Yes │ No

┌─────────────────────────┴───────────────────────┐ │ │ ▼ ▼

┌──────────────────────────────┐ ┌─────────────────────────┐ │ │ │ │ │ 1) Forbidden edges are │ │ Conflicts with learned │ │ removed from the skeleton │ │ structure (opposite │ │ │ │ edge orientations)? │ │ 2) Required edges will be │ │ │ │ present in the final │ └───────────┬─────────────┘ │ model (but direction is │ │ │ not guaranteed) │ ┌────────────────┴──────────────────┐ │ │ Yes │ │ No └──────────────────────────────┘ │ │

▼ ▼

┌───────────────────┐ ┌──────────────────┐ │ Conflicting edges │ │ Expert knowledge │ │ are ignored │ │ applied fully │ └───────────────────┘ └──────────────────┘

Parameters:

data (pandas DataFrame object) – dataframe object where each column represents one variable. (If some values in the data are missing the data cells should be set to numpy.nan. Note that pandas converts each column containing numpy.nan`s to dtype `float.)

References

[1] Koller & Friedman, Probabilistic Graphical Models - Principles and Techniques,

2009, Section 18.2

[2] Neapolitan, Learning Bayesian Networks, Section 10.1.2 for the PC algorithm (page 550),

http://www.cs.technion.ac.il/~dang/books/Learning%20Bayesian%20Networks(Neapolitan,%20Richard).pdf

estimate(variant: str = 'parallel', ci_test: str | Callable | None = None, return_type: str = 'pdag', significance_level: float = 0.01, max_cond_vars: int = 5, expert_knowledge: ExpertKnowledge | None = None, enforce_expert_knowledge: bool = False, n_jobs: int = -1, show_progress: bool = True, **kwargs) → DAG | PDAG | Tuple[Graph, Dict[Tuple[str, str], Set[str]]]

Estimates a DAG/PDAG from the given dataset using the PC algorithm which is a constraint-based structure learning algorithm[1]. The independencies in the dataset are identified by doing statistical independence test. This method returns a DAG/PDAG structure which is faithful to the independencies implied by the dataset.

Parameters:

• variant (str (one of "orig", "stable", "parallel")) –

  The variant of PC algorithm to run.

  ”orig”: The original PC algorithm. Might not give the same

  results in different runs but does less independence tests compared to stable.

  ”stable”: Gives the same result in every run but does needs to

  do more statistical independence tests.

  ”parallel”: Parallel version of PC Stable. Can run on multiple

  cores with the same result on each run.

• ci_test (str or fun) –

  The statistical test to use for testing conditional independence in the dataset. If str values should be one of:

  ”independence_match”: If using this option, an additional parameter

  independencies must be specified.

  ”chi_square”: Uses the Chi-Square independence test. This works

  only for discrete datasets.

  ”pearsonr”: Uses the partial correlation based on pearson

  correlation coefficient to test independence. This works only for continuous datasets.

  ”g_sq”: G-test. Works only for discrete datasets. “log_likelihood”: Log-likelihood test. Works only for discrete dataset. “freeman_tuckey”: Freeman Tuckey test. Works only for discrete dataset. “modified_log_likelihood”: Modified Log Likelihood test. Works only for discrete variables. “neyman”: Neyman test. Works only for discrete variables. “cressie_read”: Cressie Read test. Works only for discrete variables.

• return_type (str (one of "dag", "cpdag", "pdag", "skeleton")) –

  The type of structure to return.

  If return_type=pdag or return_type=cpdag: a partially directed structure

  is returned.

  If return_type=dag, a fully directed structure is returned if it

  is possible to orient all the edges.

  If `return_type=”skeleton”, returns an undirected graph along

  with the separating sets.

• significance_level (float (default: 0.01)) –

  The statistical tests use this value to compare with the p-value of the test to decide whether the tested variables are independent or not. Different tests can treat this parameter differently:

  1. Chi-Square: If p-value > significance_level, it assumes that the

     independence condition satisfied in the data.

  2. pearsonr: If p-value > significance_level, it assumes that the

     independence condition satisfied in the data.

• max_cond_vars (int (default: 5)) – The maximum number of variables to condition on while testing independence.

• expert_knowledge (pgmpy.estimators.ExpertKnowledge instance) – Expert knowledge to be used with the algorithm. Expert knowledge includes required/forbidden edges in the final graph, temporal information about the variables etc. Please refer pgmpy.estimators.ExpertKnowledge class for more details.

• enforce_expert_knowledge (boolean (default: False)) –

  If True, the algorithm modifies the search space according to the edges specified in expert knowledge object. This implies the following:

  1. For every edge (u, v) specified in forbidden_edges, there will

     be no edge between u and v.

  2. For every edge (u, v) specified in required_edges, one of the

     following would be present in the final model: u -> v, u <- v, or u - v (if CPDAG is returned).

  If False, the algorithm attempts to make the edge orientations as specified by expert knowledge after learning the skeleton. This implies the following:

  1. For every edge (u, v) specified in forbidden_edges, the final

     graph would have either v <- u or no edge except if u -> v is part of a collider structure in the learned skeleton.

  2. For every edge (u, v) specified in required_edges, the final graph

     would either have u -> v or no edge except if v <- u is part of a collider structure in the learned skeleton.

• n_jobs (int (default: -1)) – The number of jobs to run in parallel.

• show_progress (bool (default: True)) – If True, shows a progress bar while running the algorithm.

Returns:

Estimated model –

The estimated model structure:

1. Partially Directed Graph (PDAG) if return_type=’pdag’ or return_type=’cpdag’.

2. Directed Acyclic Graph (DAG) if return_type=’dag’.

3. (nx.Graph, separating sets) if return_type=’skeleton’.

Return type:

pgmpy.base.DAG, pgmpy.base.PDAG, or tuple(networkx.UndirectedGraph, dict)

References

[1] Original PC: P. Spirtes, C. Glymour, and R. Scheines, Causation,

Prediction, and Search, 2nd ed. Cambridge, MA: MIT Press, 2000.

[2] Stable PC: D. Colombo and M. H. Maathuis, “A modification of the PC algorithm

yielding order-independent skeletons,” ArXiv e-prints, Nov. 2012.

[3] Parallel PC: Le, Thuc, et al. “A fast PC algorithm for high dimensional causal

discovery with multi-core PCs.” IEEE/ACM transactions on computational biology and bioinformatics (2016).

[4] Expert Knowledge: Meek, Christopher. “Causal inference and causal

explanation with background knowledge.” arXiv preprint arXiv:1302.4972 (2013).

Examples

>>> from pgmpy.utils import get_example_model
>>> from pgmpy.estimators import PC
>>> model = get_example_model("alarm")
>>> data = model.simulate(n_samples=1000)
>>> est = PC(data)
>>> model_chi = est.estimate(ci_test="chi_square")
>>> print(len(model_chi.edges()))
28
>>> model_gsq, _ = est.estimate(ci_test="g_sq", return_type="skeleton")
>>> print(len(model_gsq.edges()))
33

static orient_colliders(skeleton: UndirectedGraph, separating_sets: Dict[FrozenSet, Set], temporal_ordering: Dict[Hashable, int] = {}) → PDAG

Orients the edges that form v-structures in a graph skeleton based on information from separating_sets to form a DAG pattern (PDAG).

Parameters:

• skeleton (nx.Graph) – An undirected graph skeleton as e.g. produced by the estimate_skeleton method.

• separating_sets (dict) – A dict containing for each pair of not directly connected nodes a separating set (“witnessing set”) of variables that makes them conditionally independent.

Returns:

Model after edge orientation – An estimate for the DAG pattern of the BN underlying the data. The graph might contain some nodes with both-way edges (X->Y and Y->X). Any completion by (removing one of the both-way edges for each such pair) results in a I-equivalent Bayesian network DAG.

Return type:

pgmpy.base.PDAG

References

[1] Neapolitan, Learning Bayesian Networks, Section 10.1.2, Algorithm

10.2 (page 550)

[2] http://www.cs.technion.ac.il/~dang/books/Learning%20Bayesian%20Networks(Neapolitan,%20Richard).pdf

Examples

>>> import pandas as pd
>>> import numpy as np
>>> from pgmpy.estimators import PC
>>> data = pd.DataFrame(
...     np.random.randint(0, 4, size=(5000, 3)), columns=list("ABD")
... )
>>> data["C"] = data["A"] - data["B"]
>>> data["D"] += data["A"]
>>> c = PC(data)
>>> pdag = c.orient_colliders(*c.build_skeleton())
>>> pdag.edges()  # edges: A->C, B->C, A--D (not directed)
OutEdgeView([('B', 'C'), ('A', 'C'), ('A', 'D'), ('D', 'A')])

Conditional Independence Tests for PC algorithm

class pgmpy.estimators.CITests.CITestRegistry

Registry to manage Conditional Independence (CI) Test Strategies.

Allows looking up tests by name or inferring suitable tests based on data type.

get_test(test: str | None | Callable, data: DataFrame | None = None) → Callable

Retrieves a CI test strategy.

Parameters:

• test (str, callable or None) – The name of the test, a callable function, or None.

• data (pandas.DataFrame, optional) – The dataframe used to infer the test type if test is None.

Returns:

The CI test function.

Return type:

callable

Raises:

ValueError – If test is None and data is None, or if the test name is not found.

register(name: str, data_types: List[str], is_default: bool = False)

Decorator to register a CI test strategy.

Parameters:

• name (str) – The name of the test (case-insensitive).

• data_types (list of str) – List of data types this test supports (e.g., [‘continuous’, ‘discrete’]).

• is_default (bool, default=False) – If True, sets this test as the default for the provided data types.

pgmpy.estimators.CITests.chi_square(X, Y, Z, data, boolean=True, **kwargs)

Perform Chi-square conditional independence test.

Tests the null hypothesis that X is independent from Y given Zs.

Parameters:

• X (int, string, hashable object) – A variable name contained in the data set

• Y (int, string, hashable object) – A variable name contained in the data set, different from X

• Z (list, array-like) – A list of variable names contained in the data set, different from X and Y. This is the separating set that (potentially) makes X and Y independent. Default: []

• data (pandas.DataFrame) – The dataset on which to test the independence condition.

• boolean (bool) – If boolean=True, an additional argument significance_level must be specified. If p_value of the test is greater than equal to significance_level, returns True. Otherwise returns False. If boolean=False, returns the chi2 and p_value of the test.

Returns:

result – If boolean=False, returns (chi, p_value, dof). If boolean=True, returns True if p_value > significance_level.

Return type:

bool or tuple

References

[1]

https://en.wikipedia.org/wiki/Chi-squared_test

Examples

>>> import pandas as pd
>>> import numpy as np
>>> np.random.seed(42)
>>> data = pd.DataFrame(
...     np.random.randint(0, 2, size=(50000, 4)), columns=list("ABCD")
... )
>>> data["E"] = data["A"] + data["B"] + data["C"]
>>> chi_square(X="A", Y="C", Z=[], data=data, boolean=True, significance_level=0.05)
np.True_
>>> chi_square(
...     X="A", Y="B", Z=["D"], data=data, boolean=True, significance_level=0.05
... )
np.True_
>>> chi_square(
...     X="A", Y="B", Z=["D", "E"], data=data, boolean=True, significance_level=0.05
... )
np.False_

pgmpy.estimators.CITests.g_sq(X, Y, Z, data, boolean=True, **kwargs)

G squared test for conditional independence. Also commonly known as G-test, likelihood-ratio or maximum likelihood statistical significance test. Tests the null hypothesis that X is independent of Y given Zs.

Parameters:

• X (int, string, hashable object) – A variable name contained in the data set

• Y (int, string, hashable object) – A variable name contained in the data set, different from X

• Z (list (array-like)) – A list of variable names contained in the data set, different from X and Y. This is the separating set that (potentially) makes X and Y independent. Default: []

• data (pandas.DataFrame) – The dataset on which to test the independence condition.

• boolean (bool) – If boolean=True, an additional argument significance_level must be specified. If p_value of the test is greater than equal to significance_level, returns True. Otherwise returns False. If boolean=False, returns the chi2 and p_value of the test.

Returns:

result – If boolean=False, returns (chi, p_value, dof). If boolean=True, returns True if p_value > significance_level.

Return type:

bool or tuple

References

[1]

https://en.wikipedia.org/wiki/G-test

Examples

>>> import pandas as pd
>>> import numpy as np
>>> np.random.seed(42)
>>> data = pd.DataFrame(
...     np.random.randint(0, 2, size=(50000, 4)), columns=list("ABCD")
... )
>>> data["E"] = data["A"] + data["B"] + data["C"]
>>> g_sq(X="A", Y="C", Z=[], data=data, boolean=True, significance_level=0.05)
np.True_
>>> g_sq(X="A", Y="B", Z=["D"], data=data, boolean=True, significance_level=0.05)
np.True_
>>> g_sq(
...     X="A", Y="B", Z=["D", "E"], data=data, boolean=True, significance_level=0.05
... )
np.False_

pgmpy.estimators.CITests.gcm(X, Y, Z, data, boolean=True, **kwargs)

The Generalized Covariance Measure(GCM) test for CI.

It performs linear regressions on the conditioning variable and then tests for a vanishing covariance between the resulting residuals. Details of the method can be found in [1].

Parameters:

• X (str) – The first variable for testing the independence condition X ⟂ Y | Z

• Y (str) – The second variable for testing the independence condition X ⟂ Y | Z

• Z (list/array-like) – A list of conditional variable for testing the condition X ⟂ Y | Z

• data (pandas.DataFrame) – The dataset in which to test the independence condition.

• boolean (bool) –

  If boolean=True, an additional argument significance_level must

  be specified. If p_value of the test is greater than equal to significance_level, returns True. Otherwise returns False.

  If boolean=False, returns the pearson correlation coefficient and p_value

  of the test.

Returns:

CI Test results – If boolean=True, returns True if p-value >= significance_level, else False. If boolean=False, returns a tuple of (Pearson’s correlation Coefficient, p-value)

Return type:

tuple or bool

References

[1]

Rajen D. Shah, and Jonas Peters. “The Hardness of Conditional Independence Testing and the Generalised Covariance Measure”.

pgmpy.estimators.CITests.independence_match(X, Y, Z, independencies, **kwargs)

Check if X ⟂ Y | Z is in independences.

This method is implemented to have a uniform API when the independences are provided explicitly instead of being inferred from data.

Parameters:

• X (str) – The first variable for testing the independence condition X ⟂ Y | Z.

• Y (str) – The second variable for testing the independence condition X ⟂ Y | Z.

• Z (list or array-like) – A list of conditional variables for testing the condition X ⟂ Y | Z.

• independencies (pgmpy.independencies.Independencies) – The object containing the known independences.

Returns:

True if the independence assertion is present in independences, else False.

Return type:

bool

pgmpy.estimators.CITests.log_likelihood(X, Y, Z, data, boolean=True, **kwargs)

Log likelihood ratio test for conditional independence. Also commonly known as G-test, G-squared test or maximum likelihood statistical significance test. Tests the null hypothesis that X is independent of Y given Zs.

Parameters:

• X (int, string, hashable object) – A variable name contained in the data set

• Y (int, string, hashable object) – A variable name contained in the data set, different from X

• Z (list (array-like)) – A list of variable names contained in the data set, different from X and Y. This is the separating set that (potentially) makes X and Y independent. Default: []

• data (pandas.DataFrame) – The dataset on which to test the independence condition.

• boolean (bool) – If boolean=True, an additional argument significance_level must be specified. If p_value of the test is greater than equal to significance_level, returns True. Otherwise returns False. If boolean=False, returns the chi2 and p_value of the test.

Returns:

CI Test Results – If boolean = False, Returns a tuple (chi, p_value, dof). chi is the chi-squared test statistic. The p_value for the test, i.e. the probability of observing the computed chi-square statistic (or an even higher value), given the null hypothesis that X ⟂ Y | Zs is True. If boolean = True, returns True if the p_value of the test is greater than significance_level else returns False.

Return type:

tuple or bool

References

[1] https://en.wikipedia.org/wiki/G-test

Examples

>>> import pandas as pd
>>> import numpy as np
>>> np.random.seed(42)
>>> data = pd.DataFrame(
...     np.random.randint(0, 2, size=(50000, 4)), columns=list("ABCD")
... )
>>> data["E"] = data["A"] + data["B"] + data["C"]
>>> log_likelihood(
...     X="A", Y="C", Z=[], data=data, boolean=True, significance_level=0.05
... )
np.True_
>>> log_likelihood(
...     X="A", Y="B", Z=["D"], data=data, boolean=True, significance_level=0.05
... )
np.True_
>>> log_likelihood(
...     X="A", Y="B", Z=["D", "E"], data=data, boolean=True, significance_level=0.05
... )
np.False_

pgmpy.estimators.CITests.modified_log_likelihood(X, Y, Z, data, boolean=True, **kwargs)

Modified log likelihood ratio test for conditional independence. Tests the null hypothesis that X is independent of Y given Zs.

Parameters:

• X (int, string, hashable object) – A variable name contained in the data set

• Y (int, string, hashable object) – A variable name contained in the data set, different from X

• Z (list (array-like)) – A list of variable names contained in the data set, different from X and Y. This is the separating set that (potentially) makes X and Y independent. Default: []

• data (pandas.DataFrame) – The dataset on which to test the independence condition.

• boolean (bool) – If boolean=True, an additional argument significance_level must be specified. If p_value of the test is greater than equal to significance_level, returns True. Otherwise returns False. If boolean=False, returns the chi2 and p_value of the test.

Returns:

CI Test Results – If boolean = False, Returns a tuple (chi, p_value, dof). chi is the chi-squared test statistic. The p_value for the test, i.e. the probability of observing the computed chi-square statistic (or an even higher value), given the null hypothesis that X ⟂ Y | Zs is True. If boolean = True, returns True if the p_value of the test is greater than significance_level else returns False.

Return type:

tuple or bool

Examples

>>> import pandas as pd
>>> import numpy as np
>>> np.random.seed(42)
>>> data = pd.DataFrame(
...     np.random.randint(0, 2, size=(50000, 4)), columns=list("ABCD")
... )
>>> data["E"] = data["A"] + data["B"] + data["C"]
>>> modified_log_likelihood(
...     X="A", Y="C", Z=[], data=data, boolean=True, significance_level=0.05
... )
np.True_
>>> modified_log_likelihood(
...     X="A", Y="B", Z=["D"], data=data, boolean=True, significance_level=0.05
... )
np.True_
>>> modified_log_likelihood(
...     X="A", Y="B", Z=["D", "E"], data=data, boolean=True, significance_level=0.05
... )
np.False_

pgmpy.estimators.CITests.pearsonr(X, Y, Z, data, boolean=True, **kwargs)

Compute Pearson correlation coefficient and p-value for testing non-correlation.

Should be used only on continuous data. In case when Z \neq \emptyset uses linear regression and computes pearson coefficient on residuals.

Parameters:

• X (str) – The first variable for testing the independence condition X ⟂ Y | Z.

• Y (str) – The second variable for testing the independence condition X ⟂ Y | Z.

• Z (list or array-like) – A list of conditional variables for testing the condition X ⟂ Y | Z.

• data (pandas.DataFrame) – The dataset in which to test the independence condition.

• boolean (bool, default=True) – If True, returns a boolean indicating independence (based on significance_level). If False, returns the test statistic and p-value.

• **kwargs – Additional arguments. Must contain significance_level if boolean=True.

Returns:

result – If boolean=True, returns True if p-value >= significance_level, else False. If boolean=False, returns a tuple of (Pearson’s correlation Coefficient, p-value).

Return type:

bool or tuple

References

[1]

https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

[2]

https://en.wikipedia.org/wiki/Partial_correlation#Using_linear_regression

pgmpy.estimators.CITests.pillai_trace(X, Y, Z, data, boolean=True, **kwargs)

A mixed-data residualization based conditional independence test[1].

Uses XGBoost estimator to compute LS residuals[2], and then does an association test (Pillai’s Trace) on the residuals.

Parameters:

• X (str) – The first variable for testing the independence condition X ⟂ Y | Z

• Y (str) – The second variable for testing the independence condition X ⟂ Y | Z

• Z (list/array-like) – A list of conditional variable for testing the condition X ⟂ Y | Z

• data (pandas.DataFrame) – The dataset in which to test the independence condition.

• boolean (bool) –

  If boolean=True, an additional argument significance_level must

  be specified. If p_value of the test is greater than equal to significance_level, returns True. Otherwise returns False.

  If boolean=False, returns the pearson correlation coefficient and p_value

  of the test.

Returns:

CI Test results – If boolean=True, returns True if p-value >= significance_level, else False. If boolean=False, returns a tuple of (Pearson’s correlation Coefficient, p-value)

Return type:

tuple or bool

References

[1]

Ankan, Ankur, and Johannes Textor. “A simple unified approach to testing high-dimensional” “conditional independences for categorical and ordinal data.” Proceedings of the AAAI Conference on Artificial Intelligence.

[2]

Li, C.; and Shepherd, B. E. 2010. Test of Association Between Two Ordinal Variables While Adjusting for Covariates. Journal of the American Statistical Association.

[3]

Muller, K. E. and Peterson B. L. (1984) Practical Methods for computing power in testing the multivariate general linear hypothesis. Computational Statistics & Data Analysis.

pgmpy.estimators.CITests.power_divergence(X, Y, Z, data, boolean=True, lambda_='cressie-read', **kwargs)

Computes the Cressie-Read power divergence statistic [1]. The null hypothesis for the test is X is independent of Y given Z. A lot of the frequency comparision based statistics (eg. chi-square, G-test etc) belong to power divergence family, and are special cases of this test.

Parameters:

• X (int, string, hashable object) – A variable name contained in the data set

• Y (int, string, hashable object) – A variable name contained in the data set, different from X

• Z (list, array-like) – A list of variable names contained in the data set, different from X and Y. This is the separating set that (potentially) makes X and Y independent. Default: []

• data (pandas.DataFrame) – The dataset on which to test the independence condition.

• lambda (float or string) –

  The lambda parameter for the power_divergence statistic. Some values of lambda_ results in other well known tests:

  • ”pearson” 1 “Chi-squared test”

  • ”log-likelihood” 0 “G-test or log-likelihood”

  • ”freeman-tuckey” -1/2 “Freeman-Tuckey Statistic”

  • ”mod-log-likelihood” -1 “Modified Log-likelihood”

  • ”neyman” -2 “Neyman’s statistic”

  • ”cressie-read” 2/3 “The value recommended in the paper[1]”

• boolean (bool) –

  If boolean=True, an additional argument significance_level must

  be specified. If p_value of the test is greater than equal to significance_level, returns True. Otherwise returns False.

  If boolean=False, returns the chi2 and p_value of the test.

• **kwargs – Must contain significance_level if boolean=True.

Returns:

result – If boolean=False, returns (chi, p_value, dof). If boolean=True, returns True if p_value > significance_level.

Return type:

bool or tuple

References

[1]

Cressie, Noel, and Timothy RC Read. “Multinomial goodness‐of‐fit tests.” Journal of the Royal Statistical Society: Series B (Methodological) 46.3 (1984): 440-464.

Examples

>>> import pandas as pd
>>> import numpy as np
>>> np.random.seed(42)
>>> data = pd.DataFrame(
...     np.random.randint(0, 2, size=(50000, 4)), columns=list("ABCD")
... )
>>> data["E"] = data["A"] + data["B"] + data["C"]
>>> chi_square(X="A", Y="C", Z=[], data=data, boolean=True, significance_level=0.05)
np.True_
>>> chi_square(
...     X="A", Y="B", Z=["D"], data=data, boolean=True, significance_level=0.05
... )
np.True_
>>> chi_square(
...     X="A", Y="B", Z=["D", "E"], data=data, boolean=True, significance_level=0.05
... )
np.False_