Describe the bug
Generating tests for a large DAG/dataset takes a lot of memory. I think this is because a new estimator is defined for each test case, where each estimator has its own copy of the dataset. I think it would be better to have tests as a generator.
To Reproduce
Steps to reproduce the behavior:
- Take a large causal DAG (50+ nodes)
- Take a large dataset (10k+ rows)
- Try to generate causal test cases
- Watch your memory fill up until the process is killed, or you computer becomes non-responsive.
Expected behavior
The CTF should handle larger scenarios smoothly.
Describe the bug
Generating tests for a large DAG/dataset takes a lot of memory. I think this is because a new estimator is defined for each test case, where each estimator has its own copy of the dataset. I think it would be better to have tests as a generator.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The CTF should handle larger scenarios smoothly.