Devin 0

.env-example

@@ -1,3 +1,2 @@
 OPENAI_API_KEY  = ""
-OPEN_AI_MODEL_TYPE = "gpt-5"
-
+OPEN_AI_MODEL_TYPE = "gpt-4o"

.gitignore

@@ -1,5 +1,6 @@
 myprolog.csv
 myprolog.pl
+world.pl
 
 # Byte-compiled / optimized / DLL files
 __pycache__/
@@ -130,3 +131,8 @@ dmypy.json
 
 # Pyre type checker
 .pyre/
+world.pl
+
+# Added to prevent tracking of test results and virtual environment directories
+test_results.xml
+venv_test/

PROGRESS.md

@@ -0,0 +1,41 @@
+# Project Progress Documentation
+
+## Overview
+This document outlines the progress made on the "logical" repository, specifically on the 'devin-0' branch. The goal is to develop 1000 logical English examples, determine their truth values, and create corresponding Prolog statements and a test runner to validate these truth values.
+
+## Tasks Completed
+- Reviewed the "logical" repository and corrected file path definitions in `storage.py`.
+- Verified the creation of `myprolog.csv` and determined the current count of logical examples.
+- Developed the `generate_examples.py` script to automate the generation of logical English examples.
+- Fixed assertion errors in the `test_validate_logical_statement` function within `generate_examples.py`.
+- Enhanced the `generate_examples.py` script to use parsing functions from `__init__.py` for generating Prolog statements.
+- Implemented a Prolog test runner script (`test_runner.pl`) to validate the truth values of logical statements.
+- Regularly committed and pushed changes to the remote repository to ensure progress is tracked and saved.
+
+## Scripts Functionality
+- `generate_examples.py`: Automates the generation of logical English examples and their corresponding Prolog statements. It includes validation checks and ensures uniqueness of generated statements.
+- `test_runner.pl`: A Prolog script that loads Prolog statements from `myprolog.csv`, parses each statement, and executes it to validate its truth value.
+
+## Important Notes
+- The `validate_logical_statement` function in `generate_examples.py` has been refined to correctly handle simple logical statements.
+- The `parse_logic` function in `__init__.py` is used to convert English statements into Prolog format, which is then integrated into the `generate_examples.py` script.
+- The Prolog test runner script is designed to be run using the SWI-Prolog compiler with the command `swipl -s test_runner.pl -g run_tests -t halt`.
+
+## Next Steps
+- Document the process and progress to facilitate future reviews.
+- Prepare for the creation of a pull request to merge the completed work into the main branch of the repository.
+
+## How to Run Tests Manually
+To run the Prolog tests manually, navigate to the `/home/ubuntu/logical/logical` directory and execute the following command:
+```
+swipl -s test_runner.pl -g run_tests -t halt
+```
+This will run the test runner script and output the results of each test.
+
+## Commit History Reference
+- Commit messages have been descriptive to provide context for each set of changes made.
+- Regular commits have been made to ensure that the work is saved and can be reviewed at any point.
+
+## Pull Request Preparation
+- Ensure that all tests pass and the code is clean and well-documented before creating a pull request.
+- The pull request will include a summary of the changes made and the purpose of the changes for the reviewer's context.

README.md

@@ -4,26 +4,32 @@ ChatGPT logic engine using [Prolog](https://en.wikipedia.org/wiki/Prolog).
 
 First developed at the [OpenAI emergency hackathon on 3/5/2023](https://twitter.com/nonmayorpete/status/1632456433102098434).
 
- <img alt="Bertrand Russell"   src="./russell.png" />
+<img alt="Bertrand Russell" src="./russell.png" />
 
-## status 3/16/2023
+## Usage
 
-GPT-3.5 outputs prolog along with additional text sometimes breaking the automated push to the . It may refuse to generate prolog if there are no obvious logical statements. 
+To use the logical engine, follow these steps:
 
-## usage
+1. Install the package and its dependencies.
+2. Set up the required environment variables, including your OpenAI API key.
+3. Use the `interactive_logic` task to input English statements and receive Prolog queries or truth values interactively.
 
+Example interactive session:
 ```
-$ inv logic.run
-$ parse 
-$ Men are mortal. Men are human. I am human. 
-$ ask
-$ Am I mortal?
-
+$ logical interactive-logic
+Enter an English statement: All humans are mortal. Socrates is a human.
+The truth value of the statement 'All humans are mortal. Socrates is a human.' is: True
+Enter an English statement: (or type 'exit' to quit): exit
+Exiting interactive logic mode.
 ```
+This session demonstrates adding Prolog code to `world.pl` and querying its truth value. The `world.pl` file accumulates knowledge without overwriting previous facts and is not tracked in the repository.
+
+The `myprolog.csv` file contains 1000 logical English examples with truth values and corresponding Prolog statements, serving as a test and validation dataset.
+
 ## background
 
 One of the promises of logic is that it can give formal grounding for truth.
-As LLMs develop more sophisicated responses, we will be more challenged to detect truth.  
+As LLMs develop more sophisticated responses, we will be more challenged to detect truth.
 
 Via ChatGPT:
 
@@ -49,32 +55,63 @@ Via ChatGPT:
 
 
 
-## install
+## Installation
+
+To install the logical package and all necessary dependencies, use the following command:
+
+```bash
+$ poetry install
+```
+
+After installation, create a `.env` file based on the `.env-example` template and set the necessary environment variables, including your `OPENAI_API_KEY`.
+
+## Quick Start
+
+1. Clone the repository using `git clone https://github.com/your-username/logical.git` and navigate to the project directory.
+2. Run `poetry install` to install dependencies, including `folpy`.
+3. Copy `.env-example` to `.env` and configure your environment variables:
+   - `OPENAI_API_KEY`: Your OpenAI API key.
+   - `OPEN_AI_MODEL_TYPE`: Set to "gpt-4o" or another model type as required.
+4. Start an interactive logic session with `logical interactive-logic` to input English statements and receive Prolog queries or truth values interactively.
+
+## Commands
+
+- `help`: Display help information about the available commands.
+- `exit`: Exit the interactive logic session safely.
+- `parse`: Input text to extract logical statements and convert them into Prolog syntax.
+- `ask`: Pose a question to the logic engine and receive a logical answer.
+- `validate`: Check the syntax and validity of Prolog code using the Prolog interpreter.
+
+## Debugging
+
+To debug the logic engine and test the generated Prolog code, follow these steps:
+
+1. Load the Prolog file in the SWI-Prolog interpreter to test the logic engine's output:
+   ```
+   $ swipl
+   ?- ['world.pl'].
+   ```
+2. Run tests to validate the Prolog code by executing:
+   ```
+   $ poetry run pytest
+   ```
+3. If you encounter any issues, refer to the error messages and consult the `analyze_invalid_prolog.py` script to identify common error patterns.
 
-    brew install pyenv pyenv-virtualenv git
-    brew install swi-prolog --HEAD
-    pyenv install 3.11.2
-    pyenv virtualenv 3.11.2 logical
-    pip install --upgrade pip
-    chmod +x main.pl
+## updates
 
-Then copy the `.env-example` to `.env`
-
+The `parse_logic` function prompts have been refined to guide the OpenAI model more explicitly in avoiding common error patterns in Prolog code generation.
 
-# Commands:
+The `run_parser` function has been enhanced to handle a wider range of logical constructs, allowing for more complex English statements to be accurately translated into Prolog syntax.
 
-    - help
-    - exit
-    - parse: input text to extract logic from
-    - ask: : ask a logical question
+The `analyze_invalid_prolog.py` script now includes a feature to summarize common error patterns found in invalid Prolog statements.
 
+New error handling mechanisms have been implemented to provide informative messages for common issues such as authentication failures and rate limits when interfacing with the OpenAI API.
 
-## debug
+The `parse_logic` function now includes additional validation steps to ensure the semantic validity of the Prolog code generated from the OpenAI API responses.
 
-You can load the generated file in swipl to test also
+## myprolog.csv
 
-    $ swipl
-    ?- ['myprolog.pl'].
+The `myprolog.csv` file is used to store logical English examples with their truth values and corresponding Prolog statements. This file is generated by the `parse` task and is utilized for testing and validation purposes. To generate this file, use the `logical parse` command with your English statements.
 
 ## see also
 

analyze_invalid_prolog.py

@@ -0,0 +1,50 @@
+import re
+import datetime
+
+def process_file(file_path):
+    print('Processing file...')
+    invalid_statements = {}
+    with open(file_path, 'r') as file:
+        for line in file:
+            match = re.search(r'Invalid Prolog statement found: (.+)', line)
+            if match:
+                statement = match.group(1)
+                # Count occurrences of invalid statements
+                if statement in invalid_statements:
+                    invalid_statements[statement] += 1
+                else:
+                    invalid_statements[statement] = 1
+    return invalid_statements
+
+def summarize_errors(file_path):
+    error_summary = {}
+    with open(file_path, 'r') as file:
+        for line in file:
+            # Look for common error patterns and summarize them
+            if ':-' in line:
+                error_summary['Implication Error'] = error_summary.get('Implication Error', 0) + 1
+            if 'if' in line:
+                error_summary['Conditional Error'] = error_summary.get('Conditional Error', 0) + 1
+            if 'No' in line or 'All' in line or 'Some' in line or 'Most' in line or 'Few' in line:
+                error_summary['Quantifier Error'] = error_summary.get('Quantifier Error', 0) + 1
+            if re.search(r'\w+\s:-\s\w+', line):
+                error_summary['Predicate Error'] = error_summary.get('Predicate Error', 0) + 1
+            if re.search(r'\w+\s:-\s\+\w+', line):
+                error_summary['Negation Error'] = error_summary.get('Negation Error', 0) + 1
+            if re.search(r'\w+\s:-\s\w+\s:-\s\w+', line):
+                error_summary['Chained Predicate Error'] = error_summary.get('Chained Predicate Error', 0) + 1
+    return error_summary
+
+# Generate a dynamic output file path based on the current timestamp
+timestamp = datetime.datetime.now().strftime("%Y%m%d%H%M%S")
+output_file_path = f'/home/ubuntu/full_outputs/analyze_invalid_prolog_{timestamp}.txt'
+
+# Process the file and print a summary of invalid statements
+invalid_summary = process_file(output_file_path)
+for statement, count in invalid_summary.items():
+    print(f'{statement}: {count}')
+
+# Summarize and print common error patterns
+error_patterns_summary = summarize_errors(output_file_path)
+for error, count in error_patterns_summary.items():
+    print(f'{error}: {count}')

documentation/folpy_setup_and_usage.md

@@ -0,0 +1,80 @@
+# FOLPy Setup and Usage Documentation
+
+## Installation
+To install FOLPy, run the following command in your virtual environment:
+```
+pip install folpy
+```
+
+## Overview
+FOLPy is a First Order Logic Python Library designed to work with structures such as lattices and posets. It provides utilities for creating and manipulating these structures and methods for testing substructures with or without isomorphism filters.
+
+## Example Usage
+The following example demonstrates how to use FOLPy to create models and test substructures:
+
+```python
+from unittest import TestCase
+import random
+
+from folpy.examples import lattices, posets
+from folpy.utils.methods import (
+    substructures_updown,
+    substructures_downup,
+    substructures_by_maximals
+)
+
+# Define a list of models using functions from folpy.examples
+models = [
+    lattices.gen_chain(2),
+    lattices.gen_chain(3),
+    lattices.gen_chain(4),
+    lattices.gen_chain(5),
+    lattices.gen_chain(2) * lattices.gen_chain(3),
+    lattices.rhombus,
+    lattices.M3,
+    lattices.N5,
+    posets.gen_chain(2),
+    posets.gen_chain(3),
+    posets.gen_chain(4),
+    posets.gen_chain(5),
+    posets.gen_chain(2) * posets.gen_chain(3),
+    posets.rhombus,
+    posets.M3
+]
+
+# Define a test class using the TestCase class from unittest
+class SubstructuresTest(TestCase):
+    def test_always_passes(self):
+        self.assertTrue(
+            self.without_iso(),
+            msg="error in substructure without isomorphism"
+        )
+        self.assertTrue(
+            self.with_iso(),
+            msg="error in substructure with isomorphism"
+        )
+
+    # Helper method to test substructures without isomorphism filters
+    def without_iso(self):
+        result = True
+        for model in random.choices(models, k=5):
+            t = len(list(substructures_downup(model, filter_isos=False))) == \
+                len(list(substructures_updown(model, filter_isos=False)))
+            result = result and t
+        for model in random.choices(models, k=5):
+            t = len(list(substructures_updown(model, filter_isos=False))) == \
+                len(list(substructures_by_maximals(model, filter_isos=False)))
+            result = result and t
+        return result
+
+    # Helper method to test substructures with isomorphism filters
+    def with_iso(self):
+        result = True
+        for model in random.choices(models, k=5):
+            t = len(list(substructures_updown(model, filter_isos=True))) == \
+                len(list(substructures_by_maximals(model, filter_isos=True)))
+            result = result and t
+        return result
+```
+
+This example showcases the creation of models using `folpy.examples` and the application of `folpy.utils.methods` to test the substructures of these models.

fix-devin-0-branch.devin.md

@@ -0,0 +1,17 @@
+Playbook: Simplify Logical Repo
+
+## Overview
+
+We want the logical repo to be a simple invoke task that can be installed via pip. Modify as little code as possible, and do so cleanly.
+
+## Procedure
+
+1. pull in the latest changes from the devin-0 branch
+2. notice the refactored files in logical/tasks/
+3. make sure it can be instantiated and runs.
+4. make sure the desired behavior is achieved for `parse`. Which is that 1) openai converts english to prolog 2) the prolog is validated by running in prolog 3) the valid prolog is saved to world.pl 4) the file world.pl can be loaded
+5. make sure the desired behaviror for `interactive-logic` where 1) world.pl is loaded, 2) new sentences can be convereted to prolog and run as a query against the world.pl to determine their truth value
+6. ensure the user experience is one where a user can enter various english sentences, even several paragraphs and continually add it to world.py in the proper sections so that facts and other statements accumulate. Then the user should be able to query that world in interactive mode. Tests should be set up so that a 500 words of english can be parsed into prolog logic, saved to world.pl to create a fact database, then a user can ask logical questions about that world with natural language. This user experience is critical. The code should be easy to follow and understand, and the "quick start" documentation in the readme should be accurate and easy to understand.
+7. some functions in function.py may be unused. remove them if they are completly unused.
+8. make sure it can be installed from a clean environment with pip install.
+

heights_logic.pl

@@ -0,0 +1,17 @@
+% Definitions
+taller(j, x).
+taller(x, d).
+taller(d, j).
+
+% Assumption to handle potential logical inconsistency. In practice, this set of statements
+% results in a contradiction because if J is taller than X, X taller than D and D taller than J,
+% then it cannot satisfy the circular taller relation in a consistent way.
+
+% Circular contradiction resolution can be handled by additional clauses such as:
+% detection of inconsistency, or enforcement of acyclicity in the taller relationships.
+% Therefore, let's add a preventive rule to check inconsistency:
+
+inconsistent :- taller(A, B), taller(B, C), taller(C, A).
+
+% This rule can be used to detect inconsistency:
+% ?- inconsistent. would return true in this case, indicating a logical inconsistency.

invoke.yaml

@@ -0,0 +1,2 @@
+tasks:
+  search_root: './logical/tasks'