FastCIT: Fast Conditional Independence Tests#
Conditional independence tests for causal discovery
FastCIT is a library of conditional independence tests for causal discovery. It has a Rust core and bindings for Python, R and JavaScript. All tests share one API in each language. FastCIT is part of the pgmpy project.
Install#
pip install fastcit
install.packages("fastcit")
npm install fastcit
cargo add fastcit-core
Quickstart#
>>> import numpy as np
>>> import fastcit
>>> # Each language makes its own random data, so the values differ from the other tabs.
>>> rng = np.random.default_rng(0)
>>> n = 500
>>> z = rng.standard_normal(n)
>>> x = z + rng.standard_normal(n) # X and Y depend on Z only
>>> y = z + rng.standard_normal(n)
>>> data = fastcit.Dataset({
... "X": ("continuous", x),
... "Y": ("continuous", y),
... "Z": ("continuous", z),
... })
>>> test = fastcit.FisherZ(data)
>>> result = test.run_test("X", "Y", ["Z"])
>>> result.statistic
-0.5566
>>> result.p_value
0.5778
>>> print(result.dof)
None
>>> result.effect_size
0.02499
library(fastcit)
# Each language makes its own random data, so the values differ from the other tabs.
set.seed(0)
n <- 500
z <- rnorm(n)
x <- z + rnorm(n) # X and Y depend on Z only
y <- z + rnorm(n)
df <- data.frame(X = x, Y = y, Z = z)
data <- dataset(df)
test <- fisher_z(data)
res <- run_test(test, "X", "Y", "Z")
print(res$statistic)
#> [1] -0.1752
print(res$p_value)
#> [1] 0.8609
print(res$dof)
#> NULL
print(res$effect_size)
#> [1] 0.007868
const { Dataset, FisherZ } = require("fastcit");
// Each language makes its own random data, so the values differ from the other tabs.
// Seeded generator (Park-Miller), so every run gives the same data.
let seed = 12345;
const uniform = () => (seed = (seed * 16807) % 2147483647) / 2147483647;
const normal = () =>
Math.sqrt(-2 * Math.log(uniform())) * Math.cos(2 * Math.PI * uniform());
const draw = (n, f) => Array.from({ length: n }, f);
const n = 500;
const z = draw(n, normal);
const x = z.map((v) => v + normal()); // X and Y depend on Z only
const y = z.map((v) => v + normal());
const data = new Dataset({
X: { kind: "continuous", values: x },
Y: { kind: "continuous", values: y },
Z: { kind: "continuous", values: z },
});
const test = new FisherZ(data);
const result = test.runTest("X", "Y", ["Z"]);
console.log(result.statistic);
// -0.9402
console.log(result.pValue);
// 0.3471
console.log(result.dof);
// null
console.log(result.effectSize);
// 0.04219
use std::f64::consts::PI;
use fastcit_core::ci_tests::FisherZ;
use fastcit_core::dataset::{ColumnKind, Dataset};
use fastcit_core::strategy::CITest;
fn main() -> Result<(), Box<dyn std::error::Error>> {
// Each language makes its own random data, so the values differ from the other tabs.
// Seeded generator (Park-Miller), so every run gives the same data.
let mut seed = 12345.0_f64;
let mut uniform = move || {
seed = seed * 16807.0 % 2_147_483_647.0;
seed / 2_147_483_647.0
};
let mut normal = || (-2.0 * uniform().ln()).sqrt() * (2.0 * PI * uniform()).cos();
let z: Vec<f64> = (0..500).map(|_| normal()).collect();
let x: Vec<f64> = z.iter().map(|v| v + normal()).collect(); // X and Y depend on Z only
let y: Vec<f64> = z.iter().map(|v| v + normal()).collect();
let data = Dataset::from_columns(vec![
("X".into(), ColumnKind::Continuous, x),
("Y".into(), ColumnKind::Continuous, y),
("Z".into(), ColumnKind::Continuous, z),
])?;
let x = data.index_of("X").unwrap();
let y = data.index_of("Y").unwrap();
let z = data.index_of("Z").unwrap();
let result = FisherZ::new().test(&data, x, y, &[z])?;
println!("{:?}", result.statistic);
// Some(-0.9402)
println!("{:?}", result.p_value);
// 0.3471
println!("{:?}", result.dof);
// None
println!("{:?}", result.effect_size);
// Some(0.04219)
Ok(())
}
One API for all tests#
Every test has the same calls and the same result fields. To change the test, change only the constructor. HSIC takes no Z.
test = fastcit.FisherZ(data) # or fastcit.KCI(data), fastcit.GCM(data), ...
result = test.run_test("X", "Y", ["Z"])
# result.statistic, result.p_value, result.dof, result.effect_size
test.is_independent("X", "Y", ["Z"], significance_level=0.05)
test <- fisher_z(data) # or kci(data), gcm(data), ...
res <- run_test(test, "X", "Y", "Z")
# res$statistic, res$p_value, res$dof, res$effect_size
is_independent(test, "X", "Y", "Z", significance_level = 0.05)
const test = new FisherZ(data); // or new KCI(data), new GCM(data), ...
const result = test.runTest("X", "Y", ["Z"]);
// result.statistic, result.pValue, result.dof, result.effectSize
test.isIndependent("X", "Y", ["Z"], 0.05);
let test = FisherZ::new(); // or Kci::new(), Gcm::new(), ...
let result = test.test(&data, x, y, &[z])?;
// result.statistic, result.p_value, result.dof, result.effect_size
let independent = test.is_independent(&data, x, y, &[z], 0.05)?;
Runtime Comparison#
Test |
FastCIT |
pgmpy dev |
causal-learn |
gcastle |
tigramite |
pingouin |
hyppo |
dowhy |
|---|---|---|---|---|---|---|---|---|
χ² |
0.049 |
0.098ᵇ |
0.128 |
8.47ᵇ |
— |
— |
— |
— |
G² |
0.048 |
0.094ᵇ* |
0.131 |
8.46ᵇ |
1.78 |
— |
— |
— |
Cressie-Read |
0.050 |
0.098ᵇ |
— |
8.63ᵇ |
— |
— |
— |
— |
Freeman-Tukey |
0.050 |
0.094ᵇ* |
— |
3.68ᵇ* |
— |
— |
— |
— |
Modified log-likelihood |
0.048 |
0.091ᵇ* |
— |
8.67ᵇ |
— |
— |
— |
— |
Fisher-zᵈ |
0.001 |
0.945 |
0.043 |
0.143 |
— |
— |
— |
— |
Pearson tᵈ |
0.001 |
0.880 |
— |
— |
1.65 |
3.59 |
— |
— |
HSIC (n = 300) |
1.21 |
1.66 |
1.67ᶜ |
— |
— |
— |
3.67ᶜ |
— |
KCI (n = 300) |
17.3 |
16.3 |
27.8ᶜ |
— |
— |
— |
— |
— |
KCI (n = 1 000) |
503 |
258 |
449 |
— |
— |
— |
— |
— |
GCM |
0.404 |
4.03 |
— |
— |
— |
— |
— |
1.65 |
Pillai |
0.143 |
141ᵃ |
— |
— |
— |
— |
— |
— |
ᵃ pgmpy runs a random-forest Pillai test. FastCIT runs a linear Pillai test. The two tests differ.
ᵇ pgmpy and gcastle apply the Yates correction on 2 × 2 tables and cannot turn it off. FastCIT runs without it.
ᶜ Different null approximation or kernel width rule from FastCIT.
ᵈ The time does not include the one-time setup. FastCIT and causal-learn precompute the correlation matrix. FastCIT computes it in the first call and keeps it. The other packages in these rows compute it in every call. The total-time tables in summary.md include the setup.
* The engine raised an error or returned NaN on some queries. summary.md lists them. The median uses only the queries with a result. Those queries can cost less or more than the excluded ones. Compare such a cell with care.
Test |
FastCIT (R) |
pcalg |
bnlearn |
dHSIC |
Rfast |
ppcor |
comets |
|---|---|---|---|---|---|---|---|
χ² |
0.067 |
— |
0.302ᵉᶠ |
— |
0.034ᵉ* |
— |
— |
G² |
0.066 |
5.27ᵉ |
0.301ᵉᶠ |
— |
0.041ᵉ |
— |
— |
Fisher-zᵈ |
0.013 |
0.032 |
0.378ᶠ |
— |
— |
— |
— |
Pearson tᵈ |
0.013 |
— |
0.380ᶠ |
— |
— |
0.782ᵍ |
— |
HSIC (median width) |
1.41 |
— |
— |
5.79ᶜ |
— |
— |
— |
GCM |
0.479 |
— |
— |
— |
— |
— |
20.8 |
ᶜ Different null approximation or kernel width rule from FastCIT.
ᵈ The time does not include the one-time setup. FastCIT and pcalg precompute the correlation matrix. FastCIT computes it in the first call and keeps it. bnlearn and ppcor compute it in every call. The total-time tables in summary.md include the setup.
ᵉ Uses the nominal degrees of freedom. FastCIT uses the adjusted degrees of freedom.
ᶠ bnlearn ci.test() checks the whole data frame in each call. The benchmark gives it only the columns of the query.
ᵍ ppcor needs at least one conditioning variable. For a query with an empty set, the benchmark calls stats::cor.test, which runs the same t-test.
* The engine raised an error or returned NaN on some queries. summary.md lists them. The median uses only the queries with a result. Those queries can cost less or more than the excluded ones. Compare such a cell with care.
Available tests#
Test |
Data type |
|---|---|
Chi-squared ( |
Discrete |
Log-likelihood ( |
Discrete |
Cressie-Read ( |
Discrete |
Freeman-Tukey ( |
Discrete |
Modified log-likelihood ( |
Discrete |
Pearson correlation ( |
Continuous |
Fisher-z ( |
Continuous |
Pearson equivalence ( |
Continuous |
HSIC ( |
Continuous |
KCI ( |
Continuous |
GCM ( |
Continuous |
Pillai’s trace ( |
Mixed (discrete and continuous) |
API reference#
Each language has its own API reference.
The fastcit module: Dataset, the test classes and CIResult.
The fastcit package reference: dataset(), the test functions and run_test().
The fastcit WebAssembly bindings: Dataset and the test classes.
The fastcit-core crate: Dataset, the CITest trait and the test types.