phlippe / enco

Official repository of the paper "Efficient Neural Causal Discovery without Acyclicity Constraints"

Hi there 👋 I’m Phillip, a PhD student in the Qualcomm-UvA lab (QUVA) at the University of Amsterdam supervised by Efstratios Gavves and Taco Cohen. My research focuses on the intersection of causality and machine learning 🤖, but I am also interested in generative modeling 🌀, reinforcement learning 🕹, AI4Science 🧪, and natural language processing 💬. Besides that, I like teaching 👨‍🏫. A short guide of my main repositories:

Teaching/Education

• uvadlc_notebooks: Jupyter notebook tutorials for the Deep Learning course at UvA. They cover basic deep learning topics such as initialization and optimization, to more complex topics including Normalizing Flows, Vision Transformers and Meta Learning. All notebooks executed can be viewed on our RTD website, and are integrated in PyTorch Lightning's documentation.
• UvA_summaries: A collection of summaries that I wrote during my Master studies of Artificial Intelligence at the University of Amsterdam (2018-2020). Topics cover courses including Machine Learning, Reinforcement Learning, and many more.
• jax_trainer: A small library for providing a Lightning-like API for JAX with Flax. A template research repository based on jax_trainer is shown here.

Research

• BISCUIT (📚BISCUIT: Causal Representation Learning from Binary Interactions - UAI 2023): We scale causal representation learning to Robotics and Embodied AI.
• CITRIS (📚CITRIS: Causal Identifiability from Temporal Intervened Sequences - ICML 2022, 📚iCITRIS: Causal Representation Learning for Instantaneous Temporal Effects - ICLR 2023): We identify causal variables and their (instantaneous) causal graph from videos with interventions.
• ENCO (📚Efficient Neural Causal Discovery without Acyclicity Constraints - ICLR 2022): We scale neural causal structure learning to 1000 variables by replacing constrained optimization with orientation-based parameterization.
• CategoricalNF (📚Categorical Normalizing Flows via Continuous Transformations - ICLR 2021): We explore the application of normalizing flows on categorical data and propose a permutation-invariant generative model on graphs, called GraphCNF. On molecule generation, GraphCNF outperforms both one-shot and autoregressive flow-based state-of-the-art of its time.

enco's Issues

convergence_guarantees_ENCO

sorry to bother,
but did u remove the file named 'convergence_guarantees_ENCO.ipynb'?
it's mentioned in the paper but cannot be found.

Missing SID metric

Hi, Phillip

Nice Work! But I didn't find the implementation of SID metric in your code, can you add it?
I found it in another R package, but it seems that the implementation is not good.
Thank you so much!

Best,
Wei.

How to generate/create the interventions

Dear Authors,
Thank you for sharing complete and comprehensive code of the article. I have some questions.

1. I want to apply this model on the fMRI data, so I only have observational data/graph (num_time * num_var), and don't have interventional data.
2. Can I run this model without interventional data or how could I generate the interventions on this kind of data.

Thanks a lot.
Warmest Regards,

Exponentiating `log_var` twice in NLL function of `GaussianNoiseModel`

Hey,

I know you haven't applied your model to continuous data, but we have used the GaussianNoiseModel in one of our projects.

At

ENCO/causal_discovery/multivariable_flow.py

Line 258 in 3816840

nll = - log_normal(x, y[...,0], torch.exp(y[...,1]))

you exponentiate the third argument, which is already exponentiated in

ENCO/causal_discovery/multivariable_flow.py

Lines 25 to 26 in d3318cb

	def log_normal(x, mean, log_var, eps=0.00001):
	return - (x - mean) ** 2 / (2. * torch.exp(log_var) + eps) - log_var / 2. + c

This leads to y[..., 1] being exponentiated twice, which we think shouldn't.

Best,
Heiner