Merge pull request #29 from grace-sng7/zero_test

Zero tested walkthrough notebook and fixed issues

Author: amit-sharma

.gitignore

@@ -157,4 +157,8 @@ cython_debug/
 #  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
 #  and can be added to the global gitignore or merged into this file.  For a more nuclear
 #  option (not recommended) you can uncomment the following to ignore the entire idea folder.
-#.idea/
+.idea/
+
+#Mac OsX
+.DS_Store
+

notebooks/walkthrough.ipynb

@@ -1,10 +1,9 @@
-from .model_suggester import ModelSuggester
-from .simple_model_suggester import SimpleModelSuggester
-from .simple_identification_suggester import SimpleIdentificationSuggester
+from .suggesters.simple_model_suggester import SimpleModelSuggester
+from .suggesters.model_suggester import ModelSuggester
+from .suggesters.validation_suggester import ValidationSuggester
 from .helpers import ModelType, RelationshipStrategy
 
-from .identification_suggester import IdentificationSuggester
-from .validation_suggester import ValidationSuggester
+from .suggesters.identification_suggester import IdentificationSuggester
 
 __all__ = [
     "ModelSuggester",

pywhyllm/__init__.py

@@ -14,7 +14,7 @@
         "expert_suggests_latent_negative_controls": """You are an expert in {{domain_expertise}} and are studying the {{analysis_context}}. You are using your domain knowledge to help understand the negative controls for a causal model that contains all the assumptions about the {{analysis_context}}. Where a causal model is a conceptual model that describes the causal mechanisms of a system. You will do this by answering questions about cause and effect using your domain knowledge in the {{domain_expertise}}. From your perspective as an expert in {{domain_expertise}}, think step by step as you consider the factors, relevant to the {{analysis_context}}, in which we should see zero treatment effect when changing the {{treatment}}? What factors, relevant to the {{analysis_context}}, are negative controls on the causal mechanisms that affect the {{outcome}} when changing {{treatment}}? Using your domain knowledge, what factors, relevant to the {{analysis_context}}, should we expect to be unaffected by any changes to {{treatment}}? What factors would be surprising to see affected by a change made to the {{treatment}}? Be concise and keep your thoughts under two paragaphs. Then provide your step by step chain of thoughts within the tags <thinking></thinking>. Once you have thought things through, wrap the name of the factor(s) that has/have a high likelihood of being negative controls on the causal mechanisms that affect {{outcome}} when changing {{treatment}}, within the tags <negative_control>factor_name</negative_control>. Wrap the name of the factor(s) that has/have a high likelihood of being unaffected when changing {{treatment}}, within the tags <negative_control>factor_name</negative_control>. If a factor does not have a high likelihood of being a negative control relevant to the {{analysis_context}}, then do not wrap the factor with any tags. Provide your step by step answer as an expert in {{domain_expertise}}: {{gen "output" temperature=temperature}}""",
         "expert_suggests_negative_controls": """You are an expert in the {{domain_expertise}} and are studying the {{analysis_context}}. You are using your domain knowledge to help understand the negative controls for a causal model that contains all the assumptions about the {{analysis_context}}. Where a causal model is a conceptual model that describes the causal mechanisms of a system. You will do this by answering questions about cause and effect using your domain knowledge in the {{domain_expertise}}. factor_names: {{factors_list}} From your perspective as an expert in the {{domain_expertise}}, what factor(s) from the list of factors, relevant to the {{analysis_context}}, should see zero treatment effect when changing the {{treatment}}? Which factor(s) from the list of factors, if any at all, relevant to the {{analysis_context}}, are negative controls on the causal mechanisms that affect the {{outcome}} when changing {{treatment}}? Using your domain knowledge, which factor(s) from the list of factors, if any at all, relevant to the {{analysis_context}}, should we expect to be unaffected by any changes in {{treatment}}? Which factor(s) from the list of factors, if any at all, would be surprising if affected by a change in {{treatment}}? Be concise and keep your thoughts under two paragaphs. Then provide your step by step chain of thoughts within the tags <thinking></thinking>. Once you have thought things through, wrap the name of the factor(s) from the list of factors, that has/have a high likelihood of being negative controls on the causal mechanisms that affect {{outcome}} when changing {{treatment}}, within the tags <negative_control>factor_name</negative_control>. Wrap the name of the factor(s) from the list of factors, that has/have a high likelihood of being unaffected when changing {{treatment}}, within the tags <negative_control>factor_name</negative_control>. Where factor_name is one of the items within the factor_names list. If a factor does not have a high likelihood of being a negative control relevant to the {{analysis_context}}, then do not wrap the factor with any tags. Provide your step by step answer as an expert in the {{domain_expertise}}: {{gen "output" temperature=temperature}}""",
         "expert_critiques_parents": """U: helpful causal assistant, expert in {{domain_expertise}}, studying {{analysis_context}}. Task: identify factors causing {{factor}}. Steps: (1) Analyze potential factors [{{potential_factors_list}}] for factors directly influencing/causing/affecting {{factor}}. Is relationship direct? Ignore feedback mechanisms/factors not in list. Keep thoughts within <thinking></thinking> tags. (2) Use prior thoughts to answer: how {{factor}} influenced/caused/affected by  [{{potential_factors_list}}]? Is relationship direct? Ignore feedback mechanisms/factors not in list. Wrap factors highly likely directly influencing/causing/affecting {{factor}} in <influencing_factor></influencing_factor> tags. No tags for low likelihood factors. Ignore feedback mechanisms/factors not in list. Answer as {{domain_expertise}} expert. {{gen "output" temperature=temperature}}""",
-        "expert_critiques_children": """U: helpful causal assistant, expert in {{domain_expertise}}, studying {{analysis_context}}. Task: identify factors caused by {{factor}}. Steps: (1) Analyze potential factors [{{potential_factors_list}}] for factors directly influenced/caused/affected by {{factor}}. Is relationship direct? Ignore feedback mechanisms/factors not in list. Keep thoughts within <thinking></thinking> tags. (2) Use prior thoughts to answer: how {{factor}} influences/causes/affects [{{potential_factors_list}}]? Is relationship direct? Ignore feedback mechanisms/factors not in list. Wrap factors highly likely directly influenced/caused/affected by {{factor}} in <influenced_factor></influenced_factor> tags. No tags for low likelihood factors. Ignore feedback mechanisms/factors not in list. Answer as {{domain_expertise}} expert. {{gen "output" temperature=temperature}}""",t
+        "expert_critiques_children": """U: helpful causal assistant, expert in {{domain_expertise}}, studying {{analysis_context}}. Task: identify factors caused by {{factor}}. Steps: (1) Analyze potential factors [{{potential_factors_list}}] for factors directly influenced/caused/affected by {{factor}}. Is relationship direct? Ignore feedback mechanisms/factors not in list. Keep thoughts within <thinking></thinking> tags. (2) Use prior thoughts to answer: how {{factor}} influences/causes/affects [{{potential_factors_list}}]? Is relationship direct? Ignore feedback mechanisms/factors not in list. Wrap factors highly likely directly influenced/caused/affected by {{factor}} in <influenced_factor></influenced_factor> tags. No tags for low likelihood factors. Ignore feedback mechanisms/factors not in list. Answer as {{domain_expertise}} expert. {{gen "output" temperature=temperature}}""",
         "expert_critiques_pairwise": """U: helpful causal assistant, expert in {{domain_expertise}}, studying {{analysis_context}}. Task: identify relationship between {{a}} and {{b}}. Steps: (1) Does {{a}} influence/cause/affect {{b}}? Is relationship direct? Does {{b}} influence/cause/affect {{a}}? Is relationship direct? Ignore feedback mechanisms/factors not in list. Keep thoughts within <thinking></thinking> tags. (2) Use prior thoughts to select likely answer: (A) {{a}} influences {{b}} (B) {{b}} influences {{a}} (C) Neither. Wrap answer in <answer></answer>. e.g. <answer>A</answer>, <answer>B</answer>, <answer>C</answer>. No tags for low likelihood factors. Ignore feedback mechanisms/factors not in list. Answer as {{domain_expertise}} expert. {{gen "output" temperature=temperature}}""",
         "expert_suggests_parents_condensed": """U: helpful causal assistant, expert in {{domain_expertise}}, studying {{analysis_context}}. Task: identify factors causing {{factor}}. Steps: (1) Analyze potential factors [{{factors_list}}] for factors directly influencing/causing/affecting {{factor}}. Is relationship direct? Ignore feedback mechanisms/factors not in list. Keep thoughts within <thinking></thinking> tags. (2) Use prior thoughts to answer: how {{factor}} influenced/caused/affected by  [{{potential_factors_list}}]? Is relationship direct? Ignore feedback mechanisms/factors not in list. Wrap factors highly likely directly influencing/causing/affecting {{factor}} in <influencing_factor></influencing_factor> tags. No tags for low likelihood factors. Ignore feedback mechanisms/factors not in list. Answer as {{domain_expertise}} expert. {{gen "output" temperature=temperature}}""",
         "expert_suggests_children_condensed": """U: helpful causal assistant, expert in {{domain_expertise}}, studying {{analysis_context}}. Task: identify factors caused by {{factor}}. Steps: (1) Analyze potential factors [{{factors_list}}] for factors directly influenced/caused/affected by {{factor}}. Is relationship direct? Ignore feedback mechanisms/factors not in list. Keep thoughts within <thinking></thinking> tags. (2) Use prior thoughts to answer: how {{factor}} influences/causes/affects [{{potential_factors_list}}]? Is relationship direct? Ignore feedback mechanisms/factors not in list. Wrap factors highly likely directly influenced/caused/affected by {{factor}} in <influenced_factor></influenced_factor> tags. No tags for low likelihood factors. Ignore feedback mechanisms/factors not in list. Answer as {{domain_expertise}} expert. {{gen "output" temperature=temperature}}""",