Skip to content

Latest commit

 

History

History
1618 lines (1299 loc) · 82 KB

ReferenceManual.md

File metadata and controls

1618 lines (1299 loc) · 82 KB

Table of Contents

  1. Technical Overview
    1. THIS REFERENCE MANUAL IS UNDER CONSTRUCTION
  2. Links
  3. You are at the ReferenceManual.md
  4. Reference Manual Contents
  5. Introduction
    1. Purpose
    2. User Base
    3. Sample Use Cases
    4. Status
  6. Features
    1. Voice Assistant
    2. Workflow Manager
    3. Health
      1. Evidence-Based Wellness
      2. Exercise
      3. Nutrition
      4. Doctor's Visits and Orders
      5. Medications
      6. Mental Health
    4. Time Management
      1. Recurrences
      2. Calendaring
      3. Planning, Scheduling and Execution
    5. Organization
      1. Inventory Management
      2. Adulting
      3. Communication Management
      4. Chore Charting
      5. Maintenance
      6. Smart Home
      7. Research and Development
    6. Self-Discipline
      1. To-Do
      2. Checklists
      3. Note-Taking
      4. Scheduling
      5. Self-Discipline State Machine
      6. Gamification
      7. Movement Discipline
      8. List-Manager
    7. Transportation/Shopping/Errands
      1. Transportation
      2. Shopping/Errands
    8. Document Management
    9. Financial Planner
    10. Emergency Preparedness
    11. Employment
    12. Executive Function
    13. Emacs Integration
  7. Future Work
    1. Integrations
  8. Special Use Cases
    1. Homelessness
    2. Illness
    3. Insolvency
    4. Abuse
    5. Climate Change and Disaster Management
  9. Major Technologies Used
  10. More Info
    1. More Use Cases
    2. Recognition
    3. Links
      1. Screenshots
      2. Subsystems
      3. More Links

Technical Overview

THIS REFERENCE MANUAL IS UNDER CONSTRUCTION

Links

You are at the ReferenceManual.md

Reference Manual Contents

Introduction

Purpose

  • A free/libre program for helping people with the logistics of daily life, helping to plan and secure things like food, medicine, finances, transportation, health and so on.
  • A short, medium and long-term life planner
  • Cognitive prosthesis for executive function
    • Including for people w/ ADHD, autism, schizophrenia, dementia, etc
  • A "life manual" / "skills for living life"

User Base

  • This software is intended to be of general use
  • However, people facing challenges are particularly likely to benefit more
    • With disabilities such as pervasive developmental disorders
      • "Folks do not realize that they are one fall, auto accident, or vascular event away from being a person with a disability." - Mary Hart
    • Of limited means who are one misstep from disaster
    • With disease, illness or medical conditions
    • Who are unhoused
    • Returning citizens
    • Victims of:
      • Relationship violence
      • Persecution and hate
      • War or political oppression
      • Natural or man-made disasters and climate change
    • Immigrants and refugees
    • Other demographics not here accounted for

Sample Use Cases

Status

  • Currently working to release the latest version onto GitHub
  • Interoperability between all systems

Features

Voice Assistant

  • ✅ Interactive Voice Assistant
    • ✅ Using Alexa
      • ✅ Networked/online access only
      • ✅ "Alexa, tell David, Andrew drank another glass of water"
    • ✅ Using Rhasspy
      • ✅ Both Air-gapped/offline access and networked/online access
      • ✅ "Porcupine, How many glasses of water did Andrew drink today?"
      • ✅ ASR/STT (Automatic Speech Recognition, Speech To Text)
      • ✅ TTS
        • ✅ Using coqui-ai/TTS server (Text To Speech)
        • [50%] Using Bark (Text To Speech)
  • [60%] Dialog manager
  • ✅ Dialog manager
    • ✅ For single-step dialogues
  • ✅ Latest LLM models for Question Answering (Large Language Models)
hasDialogEntry('tell  ').
dialogInterfaceQuery(Entry)  –> ([tell];[talk,to]),grabber(Person,person),([about];[]),oneOrMore(token,Tokens),
        {getCurrentDateTime(Now),
         currentAgent(Agent),
         Entry = assert(atTime(Now,tell(Agent,Person,Tokens)))}.

curGaeilgeArSeo(tell(Agent,Person,TokenizedStatement),Gaeilge) :-
        getGloss(Agent,AgentGloss),
        getGloss(Person,PersonGloss),
        atomiclistconcat(TokenizedStatement,' ',Statement),
        atomiclistconcat([AgentGloss,told,PersonGloss,Statement],' ',Gaeilge).

Workflow Manager

  • An overarching tool/wizard which pages through and helps complete all necessary tasks
    • ✅ PDDL based planner, which creates a (for now, linear) path through FLP to accomplish all objectives (Planning Domain Description Language)
      • [30%] Generating constraints for Workflow Manager's PDDL planner
'move-to-page'(begin,'user-agenda').
'complete-task'('mark-off-all-completed-from-agenda').
'complete-task'('sort-agenda').  'finish-page'('user-agenda').
  • [55%] AgentSpeak integration
    • [30%] SPAMI (SWI-Prolog AgentSpeak(L) Meta-Interpreter)
      • [20%] Prolog Jason-like event/control-loop
    • [65%] Contingency planning
      • ✅ Original Perl-based WOPR NL-based Contingency Planner (War Operation Plan Response, Natural Language)
      • ✅ Newer Prolog-based WOPR
        • ✅ Using NL for KRR and generating contingency plans via BFS (Knowledge Representation and Reasoning, Breadth First Search)
          • ✅ Exports current fluents from WSM (World State Monitor) as world state, push onto queue
          • ✅ Prompts Llama3 Instruct 70B to generate:
            • ✅ Shift queue to obtain world state
            • ✅ Threats for the initial world state
            • ✅ Potential contingency plans for those threats
            • ✅ Effects of the contingency plans
            • ✅ New world state by applying the effects to the parent world state
            • ✅ Push new world state onto queue,and repeat above steps
        • [50%] Draft paper on contingency planning with WOPR
      • ✅ WebUI to select ground or unground contingency triggers
        • ✅ Latest LLM models for generation of NL contingency plans for triggers (Natural Language)
    • [30%] Dynamically add/remove goals and replan
    • Deprecated: 
+!quarantine(Person) <-
        ?hasRoom(Person,Room);
        !atLocation(Person,Room);
        +prohibited((move(Person,Location),Location \\= Room));
        …

Health

Evidence-Based Wellness

  1. General

    • Medical Q&A using DoctorGPT or a successor LLM

  2. Data Collection and User Modeling

    • [50%] Record / create an inventory of all user's medical symptoms and side effects
      • List conditions and their symptoms
        • Akahige should know what your current conditions
          • Rotate around roundrobin always checking for:
            • Known side effects
            • Symptoms of other conditions which are associated with the known conditions
      • [75%] List medications and their side effects
      • Inventory medication effects, side-effects and risks
    • SNA of providers (Social Network Analysis)
      • Who they they are, what they do, what they advise and how often we need to visit
    • Empirically determine effectiveness of treatments
      • Opt-in anonymized data sharing
        • With users' fine-grained consent
        • With secure multiparty computation
        • For use with data mining, ML

  3. Prevention / Early Detection

    • Achieve and maintain medical situational awareness
      • Take users' known conditions/symptoms, estimate conditional probabilities of additional conditions/symptoms
        • Similar to the probabilistic techniques behind SLAM algorithms from autonomous robotics (Simultaneous Localization and Mapping)
          • Propagating conditional densities
        • Sorted by a norm on or MCDA for likelihood and impact/severity (multi-criteria decision analysis)
        • Observations substantiated by FLP data collection
        • [12%] Medical KBS integrations: SIDER 4.1, possibly: MedDRA, LEX, OpenGALEN, RxNorm, UMLS, etc
        • [33%] Medical KG integrations: Hetionet, possibly: PDT (Personal Digital Twin)
      • Help with prophylaxis for known risks
      • Help to administer treatments (preventative, ongoing and restorative)

  4. Diagnostics and Treatments

    • Expert system for diagnosis of medical problems and treatment
      • Application of otherwise unavailable medical knowledge (in the absence of a medical professional)
      • Argumentation-based synchronization and reconciliation when able to see medical professional
        • Either in person
        • Telemedicine / remote doctor
      • WOPR contingency plans, represented in Prolog, fulfilled by Code Llama or GPT-4
        • Library of COAs/SOPs (Courses Of Action, Standard Operating Procedures)
          • Instructions for CPR, Heimlich, etc
            • Just in time
            • Ahead of time, studying administered by an ITS (Intelligent Tutoring System)
        • Upon activation of contingency trigger:
          • Executed using interactive execution system
            • SPAMI or JASWIPL
            • IEM2 (Interactive Execution Monitor verson 2)
            • Plan-Monitor
        • [30%] Flowcharts / workflows for determining best professional treatment options
          • ER, Urgent care, phone on-call doctor/nurse, telemedicine, etc (Emergency Room)
            • [20%] Hospital packing list including all necessary supplies for ER visit
      • Medical fact checking
        • Visual library of images for diagnosis and explanation
          • CV-based algorithms trained to identify conditions (such as skin conditions or lesions) (Computer Vision)
      • Integration with financial planner and inventory manager for ensuring all (potentially) necessary medical supplies kept in stock and not expired
      • Argumentation-based deliberation regarding the validation of planned treatments

Exercise

  • ✅ Fitness Manager
    • [33%] Interactive plan monitoring of exercises
      • ✅ exercise.bt
    • [66%] Track daily progress
      • ✅ Using Rhasspy Voice Assistant
        • ✅ "Porcupine, Andrew finished daily exercises"
      • ✅ Using legacy Alexa interface
        • ✅ "Alexa, tell David - Andrew did his morning exercises"

Nutrition

  1. Meal Planner

    • Improves nutrition and taste
    • [40%] Seamless integration with FLP's generalized action planning systems
    • Many meal planning resources
    • [85%] Nutrition lookup
      • ✅ Using FDC csvs converted to Prolog KB (Food Data Central)
      • ✅ Using Nutritionix
        • Working (but offline due to air-gapping development server)
    • Inventory Management
      • User modeling
        • [40%] Self-discipline coach software
          • For hitting macros
          • Understanding psychology of users' relationships to food
        • ✅ Helping w/ Portion control
          • ✅ Door sensor alerts on fridge and freezer
          • [20%] Follow up dialog regarding intent when fridge and freezer accessed
          • Gamification
        • Specifically checking for common symptoms known to affect the user, like gerd, lactose intolerance, etc
          • Inference existence of food sensitivities
        • Nutritional temporal records
          • Privacy preserving
          • Integration with health temporal records
            • Empirical analysis of nutritional factors associated with medical conditions
            • Detection and allowance for genetic factors affecting diet and heatlh
    • Ensuring food safety
      • Semi-automatic RKF of textual knowledge bases regarding food safety (Rapid Knowledge Formation)
    • [20%] Logging and planning of macros
    • Specialty Diets
      • [33%] Food ontology
        • Food properties encoded into Prolog factbase
          • Medical properties such as allergies
          • Food storage duration (in various conditions, such as freezer) knowledge
          • Using word embeddings for food similarity
        • Food substitutions (changing, adding or canceling)
        • Normalization / mapping NL descriptions to entities in a KB (Natural Language)
          • Parsers
            • NL recipes to planning problems
            • Normalizing parsers
              • Ingredients to food
              • Food data to food
              • Branded foods to ingredients
        • Integration with knowledge sources
          • ✅ WordNet
          • ✅ OpenCyc
      • Help with ethical diets like vegan, pescatarian, etc
      • Help with medical diets like diabetic, missing gall bladder, etc
    • Recipe manager
      • [95%] Using MM recipe archive of 150,000 recipes from SOAR, (Meal Master, Searchable Online Archive of Recipes,)
      • Integration of a separate list of recipe archives containing an additional 150,000 recipes
      • Recommender system / collaborative filtering of recipes
    • Freezer cooking
      • Reduces by >= 4X both cost and prep time
    • [50%] Cooking Assistant
      • [50%] Automatic conversion of recipes to BTs (Behavior Trees)
      • ✅ Walking the user through resultant BTs using Plan-Monitor
    • Food style and preference learning
      • Estimate when the user is likely to get tired of some recipe or ingredient(s)
    • Comprehensive meal contingent plan generation
    • Planning tool to avoid food spoilage and expiration in various storage conditions
      • ✅ Door sensor alerts on fridge and freezer
        • [20%] Follow up dialog regarding intent when fridge and freezer accessed
          • ✅ Ask user why they triggered door sensors
      • Calculate remaining time before spoilage based on storage condition temporal history
    • Rotating emergency food and water provisions
      • Stock enough provisions to survive n-months
      • Consuming the oldest staples first to prevent victuals from expiring
    • Food pantries
    • [50%] Receipt tracker
      • [25%] Bill splitter

Doctor's Visits and Orders

  • Symptom and condition tracker
  • ✅ Appointment calendar
    • Track appointments and avoid double-booking and overbooking
    • ✅ Verbal and User-Agenda reminders
    • ✅ Q&A system for asking things like when is my next appointment
  • [25%] System for note taking during visits and telehealth sessions
  • Medical diagnostics
  • [40%] Following doctors' orders
    • ✅ Scan into Document Management System for managing (among other things) medical records
    • [90%] Ensuring compliance via Machine Reading Comprehension
      • ✅ Using WizardLM to extract instructions from documents
      • ✅ Converting orders into an BT (Behavior Tree)
      • Converting orders into an SPSE2 planning context
    • Medical question answering in absence of medical professionals using software such as DoctorGPT
  • Medical adherence tracking
    • Ensuring follow-through with doctor's orders
      • LLM-based and E2C representation (PrologCyc) (Large Language Model, English to CycL)
  • Medical record management
    • Tracking nutrition, exercise, medications, supplements, doctor's visits, immunizations, hygiene, etc
  • Sanity and triple checking for potentially valid health concerns that were missed
    • Patient advocate to detect potential confusion or foul-ups due to communication breakdowns between patients, doctors and other doctors
  • For when a physician is unavailable

Medications

  • ✅ Medication reminders and recording
  • Planning, tracking and monitoring of medication and supplement usage
    • Complete CLP-based medication manager that understands time management (Constraint Logic Programming)
      • Predict when we will run out of various medications
        • Logistically ensure stable access to necessary medications
          • Accounting for factors such as medical professional turnover, weather phenomenon, etc
          • Securing medications that are logistically difficult to access
            • Hours of operation of pharmacy, clinics
    • Domain specific knowledge regarding factors like whether a med cannot be skipped, or if necessary, for how long
    • [30%] PDDL+ Medication Management
      • [30%] Manage time-dependent effects of medication dosages and half-lives
    • [33%] Medication refill tracking
      • Ensure cannot be removed from the user's agenda, cannot be postponed
      • Store relevant contact information
      • Recurrent sanity checks
      • Infer when user forgot to refill or track medications
    • Note start of medication run
      • Close monitoring of potential symptoms, associated conditions, side-effects, etc
        • ✅ Listing medication side effects using SIDER 4.1 database loaded into Akahige-Formalog
        • [50%] Recent medical history and significant data automatically added to report for doctor
          • Talking points for next associated visit with a medical professional
          • [20%] Interactive dialog manager (similar to but much improved over Audience Dialog subsystem)
            • Recording of doctor's responses, answers and orders
            • [75%] Dialog execution monitoring using cell-phone

Mental Health

  • Techniques
  • Effectively managing interpersonal boundaries
    • [25%] SHIELD system
    • [25%] Task delegation system
  • Emotional security
    • Managing triggers
    • [25%] Reframing self-talk
    • [50%] Meme-archive emotional intelligence + RAG search
  • Psychometric reporting
    • Identifying factors
  • Systems for tracking symptoms
    • Executive function
    • Negative self talk
      • [50%] Rewrites negative self talk using LLM prompts (Large Language Models)
    • Planning to achieve and maintain human flourishing
      • Planning for optimal productivity by meeting needs
        • Pre/pro-active planning
        • Root cause analysis for low-productivity
        • [75%] Responses to low mood and low productivity
          • ✅ Elicits how the user is feeling
          • [75%] Uses NLI/RTE to fetch from treatment DB (Natural Language Inference / Recognizing Textual Entailment)
        • Task tracking and MTTC (Mean Time To Completion)
  • Systems for conditions
    • ADHD (Attention-Deficit/Hyperactivity Disorder)
      • Distraction ontology
        • [40%] (Pre/re)active planning using WOPR to minimize or mitigate distractions (War Operation Plan Response)
      • [30%] different FLP modes using environmental anchors
        • ✅ Automatic priming and transitions
        • ✅ Change smart lighting colors for different modes, like working, relaxing, etc
          • ✅ Auto detection of working, relaxing, etc
        • ✅ Managing music for different modes (using playlists)
    • Autism
    • Schizophrenia
    • etc

Time Management

Recurrences

  • ✅ Dates
    • ✅ On a given date
    • ✅ On a given day of week
    • ✅ Through given days of week
    • ✅ Every n-th week/month/year
    • [33%] Every n-th day of week every m-th month
      • E.g. Second tuesday of each month
  • Times
    • ✅ Multiple specific time points
    • Durations

How to schedule something for the last day of every month:

hasRecurrenceSpec(critical(closeTheBooks),onDate(YMD),[9:0:0,13:0:0,17:0:0]) :-
        deltaTime([Year-Month-1],days(-1),YMD).

Calendaring

  1. Capabilities

    • ✅ Scheduling appointments
    • ✅ Scheduling of chores
    • Detect double-booking

  2. Integrations

    • With different agenda, calendaring, and planning / scheduling systems
      • ✅ Import from
        • ✅ Org-agenda
        • ✅ SPSE2 (Shared Priority System Editor v2)
        • ✅ Google Calendar
        • ✅ ICS (Internet Calendar Scheduling)
      • Export to
        • ✅ Org-agenda
        • [85%] SPSE2
        • [33%] Google Calendar
        • ✅ ICS

  3. Financial Calendar

    • ✅ Show on calendar possible date and time ranges for recurring transactions

Planning, Scheduling and Execution

  1. Planning

    • [30%] Workflow Manager
    • [55%] Metaplanning
      • [50%] Jason/AgentSpeak(L)
      • [60%] SPAMI
    • ✅ Temporal planning
    • ✅ Contingent planning
    • [05%] Temporally-contingent (metric) planning
    • [15%] OSP (Oversubscription planning)
    • ✅ Behavior tree reactive planning
    • ✅ BDI reactive planning (Belief-Desire-Intention)
    • [15%] Planning Knowledge Engineering
      • Nested<->Nonnested fluents
    • [80%] SPSE2 Graph-based Planning Domain/Problem/Plan Editor/Visualizer
      • Represent a multitude of relationships, logics and "domains", roundtripped with PrologCYC
      • Decision making in complex domains via arguing and critiquing of plans/COAs
      • Generate and/or import (for editing) linear and/or contingent plans
      • VAL-idation of plans
      • Task Duration Estimation
      • Interactive Execution Monitoring
      • PSEx3 integration for SPSE2 contexts with other PDDL capsules (<domain,problem> pairs)

  2. Planners

  3. Additional Planning System

    • Factored planning
    • ✅ Planning systems
      • ✅ Verber
      • ✅ SPSE2
      • ✅ SPSE2-Formalog
    • Digital twin
      • ✅ WSM (World State Monitor)
      • World State Comparison
        • Using LLMs to compare possible worlds for desirability
      • ✅ Fitness Manager
      • [55%] Nested checklists
    • [33%] Kanban
    • [33%] Goalban
    • ✅ User Agenda (for recurrences)
    • ✅ Habit tracker
      • [50%] Allow installation and uninstallation of habits
      • ✅ Tracks from a growing list of elicited habits, records when engaging in / avoiding behaviors
      • [50%] Habit chaining
        • [50%] Initiated by daily-check-in / daily metronome
    • [60%] Resource Manager (plans over inventory/etc)
      • [80%] Productivity Requirements
      • [33%] Shopping list manager
    • [25%] Subsystem monitoring
      • [25%] Mission Control
      • [40%] Normal Form
    • Gamification

  4. Some domains

    • [50%] Plan cycles (cycle)
      • ✅ Toy plan cycle
      • Real-world plan cycle
    • [33%] PDDL Standard domain library (basekb)
    • [33%] Behavior Tree standard library
      • ✅ deliveries, electrical, errands, exercise, financial, food-pantries, groceries, hospitalization, hygiene, meal-planning, medication, movement-discipline, paperwork, plumbing, security, sickness, sleep, trip-planning
    • [10%] Household emergency preparedness
      • Planning ahead for bad weather
        • e.g. Stock up on groceries before a major storm hits
        • [50%] Weather control rules
    • [15%] 71 Prolog-Agent/SPAMI/Autonomous LLM agents
      • [20%] academician - research topics completely on its own.
      • [20%] academician-remote - research topics completely on its own, via remote control.
      • [20%] agent-smith - install FRDCSA on a remote system.
      • [20%] aisa - autonomous IEM2 SPSE2 agent, it can try to accomplish tasks/goals from SPSE2.
      • [20%] api-learner - maps APIs by reading API documentation and examples.
      • [20%] architect - curate software collections and matchmake capabilities with feature requests.
      • [20%] argument - develop arguments for an against a position, using LLMs, and reason them out.
      • [20%] asset-manager - tool to navigate our networks and automatically inventory hardware, software and data assets.
      • [20%] audience - proxy all communications and make sure they are appropriate and act accordingly if not.
      • [20%] autodoc - runs around documenting all FRDCSA code and weaves it in - so that we do not forget to use it.
      • [20%] blackarch - learn to control a BlackArch instance.
      • [20%] contingent-dialog - plan conversations, possibly in a persuasive or defensive context.
      • [20%] crisis-tracker - monitoring, prioritizing and coordinating responses to multiple ongoing crises.
      • [20%] crl - constantly reprioritize goals between and during execution of tasks to achieve them.
      • [20%] cyc-navigator - interact with Cyc for exploration purposes, subagent for NL2Logic.
      • [20%] do-ooda - using our do-todo-list data, perform OODA cycle.
      • [20%] executive-function-assistant - complex OODA + learning system.
      • [20%] executive-function-c3s - executive function assistsant written by Claude 3 Sonnet.
      • [20%] executive-function - take our large to-do list systems and help us to achieve them, and set new goals.
      • ✅ fibonacci - simple proof of concept fibonacci series calculation using LLMs and templating.
      • [20%] flp-intake2 - contingent dialog system for intaking users into FLP v2.
      • [20%] flp-intake - contingent dialog system for intaking users into FLP.
      • [20%] fweb2 - tools for enhancing FRDCSAs web presence.
      • [20%] getting-things-done - implements GTD methodology.
      • [20%] iaec2 - analyze files and other digital objects and figure out what they are and act accordingly with them.
      • [20%] iaec - analyze files and other digital objects and figure out what they are and act accordingly with them.
      • [20%] manager - track me down if I am not available, and ensure I received given messages.
      • [20%] metaplanner - metaplan with the financial planner, moving things around like expected dates of certain payments.
      • [20%] movement-discipline-agent - tool for pre-computing real-life movements and actions before we make them.
      • [20%] mush - analyze error messages and research how to resolve them.
      • [20%] neo - control all subagents properly.
      • [20%] network-mapper - inventories network hardware and software assets.
      • [20%] nl2logic - reactive/agentic conversion of text to logics like FOL or SubL/CycL deliberatively and interactively.
      • [20%] org-agenda - calendaring agent.
      • [20%] packager - figure out what software we have and need, and go about packaging it for Debian.
      • [20%] pdss - tool for budget and oversubscription planning.
      • [20%] praise-bot - generate compliments and otherwise reward the user for productivity, encouraging them to stay motivated and productive.
      • [20%] redactor - prepares various FRDCSA or other files for public release, by removing personally identifying information.
      • [20%] sentinel - monitor and housekeep the computer, making sure that all files get moved to their intended locations, and extract information useful to FRDCSA/FLP/Prolog-Agent.
      • [20%] setanta-agent - administer machines, map networks, read manpages and write code, etc.
      • [20%] setup-debian - bring a vanilla Debian machine up to speed, ready to have FRDCSA installed on it.
      • [20%] sieve - work with what it knows about software systems, and use previously mentioned agents to help it create large repositories of packaged software.
      • [20%] softbot1 - act as a software robot, moving around on a system and doing intelligent things with it.
      • [20%] suasion - persuade people of the truth of some conclusion.
      • [20%] svrs - monitor all sensors and behave accordingly.
      • [20%] system-implementor - take research papers and attempt to reimplement when necessary.
      • [20%] systemize - tool for cleaning up / housekeeping FRDCSA/Linux systems.
      • [20%] vger2-claude - a tool for amassing software collections.
      • [20%] vger2 - tool to create a massive software collection, using various techniques to build an ontology of systems, and determine their availability.
      • [20%] web-agent2 - improved tool for navigating the internet.
      • [20%] web-agent - read, understand and navigate websites, helping with research.
      • [20%] wopr2 - improvements to agent for develop continency plans for a given situation.
      • [20%] wopr - develop continency plans for a given situation.
      • [20%] workflow-manager - help the user to make use of FLP by walking them throw what they need to do.
    • ✅ 10+ custom FLUX planning domains (Fluent Executor)

  5. Plan Monitoring

  6. Theorem Proving for Planning

    • ✅ Vampire-KIF (Knowledge Interchange Format) ATP (Automated Theorem Proving) system
    • General Game Playing
      • Sancho, cadiaplayer
    • ✅ FLUX integration (Fluent Executor)
      • ✅ Homemade BFS planner (but no FLUXPlayer integration as of yet)
        • ✅ 10+ custom FLUX planning domains
    • Research Directions
      • QBF solvers
      • TPTPSolver containerization and integration (Thousands of Problems for Theorem Provers)

Organization

Inventory Management

  • [10%] Supplier tracking
    • [25%] Integration with cashflow tracking
    • [10%] Semiautomatic reordering
      • Helping to reorder from preferred vendor types, like organic, local, etc
  • [66%] Pantry management
    • [80%] Nutrition lookup
    • [75%] A Mermaid diagram for inventory addition and removal control flow
      • [15%] The Platinum dynamic WebUI for walking through the Mermaid graph, and properly processing food inventory addition and removal
      • [75%] Barcoded and un-barcoded food items
        • [50%] Barcode scanning of food products
          • ✅ Wireless USB barcode scanner hidraw agent
    • [10%] Manage expected expiration dates
    • Physical layout assistant
    • Automatic shopping list generation and reordering
    • Tracking food recalls or warnings
      • Crypto signatures and source-monitoring
    • Kitchen and appliance cleaning and maintenance tips
  • [15%] Executable product manuals
    • [60%] Interactively use products
      • ✅ Use coffeemaker using plan monitor
    • [90%] Automatically schedule maintenance
    • [10%] Interactively perform maintenance
  • Executable product warranties
    • Integration with planning and scheduling

Adulting

  • ADLs
    • Daily Self Care
  • IADLs
    • Chore charting
      • [35%] Track who/what/where/when regarding chores
      • ✅ Implement rewards via Gamification

Communication Management

  • Proxy all communications
    • ✅ Dossier system
      • Relationship management
        • [25%] ACLs (Access Control Lists)
        • ✅ Talking points
      • ✅ Memcons (Memorandum of Conversation)
      • [33%] Commitments extraction
      • [33%] Goal/Interest/Ability extraction
    • Persuasion
    • Relationship management
      • Friendships
      • Family
      • [12%] Partners
        • [33%] Ways to show affection
      • Groups
        • Social advocacy/interest groups
      • ✅ Track birthdays, anniversaries, etc
    • Team building

Chore Charting

  • [60%] Chore ontology
    • ✅ Comprehensive chore lists
    • [50%] Household chores
    • [80%] Caregiving chores
  • [66%] Chore logic
    • [66%] Scheduling, reminders, tracking and gamification
    • [66%] Interactive plan execution using library of chore BTs (behavior trees)

Maintenance

  • [33%] Household maintenance
    • [50%] Complete list of scheduled home maintenance tasks
      • [10%] Test fire alarms
      • ✅ Furnace maintenance
      • etc
  • ✅ Automotive maintenance schedule

Smart Home

  • SVRS Integrated shelter management
    • Sensor network
      • ✅ Security and surveillance
        • Cyberphysical Security
          • Network Monitoring
            • Setanta, Setanta-Agent, network-monitor, ping-chime, jnettop
          • Remote Execution
            • Prolog-Agent
              • SPAMI
          • Tools for updating Air Gapped systems
          • HIPAA rules implemented in a logic program
          • [10%] Vigilance and strategic intelligence
        • ✅ SDR sensor monitoring (Software Defined Radio)
        • ✅ Home defense preparations and contingency planning
        • ✅ Video cameras and perimeter defense sensors
          • [20%] Video understanding (similar to VSAM)
            • ✅ Object recognition
            • Activity detection
              • [25%] Suspicious/anomalous activity detection
        • koo5's Emergency Alert System
      • [50%] Smart lighting control
        • [50%] For managing moods (such as, red - during work) for productivity, relaxation, etc
        • ✅ Indicating availability to housemates (red - do not disturb, etc)
      • ✅ Event logging and inference
        • ✅ Action triggers
          • [60%] Changing smart lighting based on detected work activity
          • [50%] Penalization for lack of movement discipline (e.g. left room without plan)
          • Reactive systems
            • [33%] Climate/temperature control
              • [33%] Open windows, Turn the AC/heater on/off, etc
            • [20%] Weather monitoring
              • Warnings to prepare for storms, extreme weather
            • [20%] Electrical usage
        • [10%] Complex event detection
          • [20%] RTEC (Run Time Event Calculus)
        • [25%] FLP-Log
          • Manual event logging from Emacs using and into FLP/Cyclone
    • Home maintenance
    • Emergency preparedness
    • SVRE
      • [45%] IPSVRE
        • [90%] Management of many aspects of individual environments including: lighting, climate, distractions, productivity, etc.
      • [25%] ASVRE
    • [66%] PDDL+ based pest control planners

  1. Maintenance

    • Equipment
    • Computer systems

  2. Home Maintenance

    • ✅ Home

  3. Equipment Maintenance

    • ✅ Automotive

Research and Development

  • sto0pkid's in-progress Agda mechanization of the mathematical motivation behind FRDCSA
  • ✅ CLEAR book reader (Computer LEarning ARchitecture)
  • ✅ Academician system
    • Record all reading using KBFS (Knowledge-Based File System)
    • [66%] Math OCR to LaTeX inside Markdown
      • Math understanding and code generation via LLM
    • Reference extraction
      • ✅ ParsCit
      • [20%] Grobid
    • ✅ Academician-CM (Continuous Mode)
      • View MxN pages simultaneously, with synchronized paging
  • ✅ Study system
  • ITS system (Intelligent Tutoring System)
  • ✅ Seeker system
    • KSAs (Knowledge Seeking Agents)
  • ✅ Text processing
    • Text2KG system (Text to Knowledge Graph conversion)
    • RAG QnA system with UniLang integration (Retrieval Augmented Generation)
    • NL2Logic system (converts text to CycL)
    • Text to timeline conversion
  • Automatic Argument Mining and Construction

Self-Discipline

To-Do

  • [50%] Org and Org-agenda
  • [75%] Do system (to-do lists)
    • [50%] Do-convert
      • ✅ Track millions of goals
        • [60%] Including their interdependencies
        • ✅ In a logical language
      • [90%] Convert sexps to Prolog w/ QLF-persistence
    • [50%] Do-convert-logic
    • [33%] Task operations
      • Classification
      • Commitment extraction
      • Dependency elicitation
      • [75%] Duration estimation
        • ✅ Using LLMs to infer duration, and add timing information to SPSE2/Verber/IEM2 through SPSE2-Formalog
      • [16%] Ontology
      • [33%] Prioritization
        • ✅ Using LLMs to sort tasks by priority
      • Tagging
      • Inferring completed tasks from changelogs using LangPro
  • Miscellaneous other partially completed to-do systems

Checklists

  • Disciple
    • [70%] Checklists for daily activities
    • [70%] Interfaces for marking off items from checklist
      • [80%] Alexa/Rhasspy voice interface
      • [80%] FCMS WebUI log

Note-Taking

  • ✅ Org-mode
  • [10%] Cyc-ZK (Zettelkasten)
  • ✅ [33%] MICS (Mixed Initiative Conversational System) LLM-invented and LLM-based autonomy solution
    • A graph-based conversational system that enables users to engage in mixed initiative conversations with an AI assistant
    • Utilizes a graph database to store conversation history and context
    • Employs word embeddings and graph embeddings to represent the semantic meaning of entities and relationships
    • Features a UniLang client that interacts with the conversational log and generates responses to user input
    • Includes a set of tools and techniques to rapidly annotate and query the conversational log
    • Future plans include developing and integrating the AI response algorithm and further exploring the graph representation of conversation

Scheduling

  • ✅ PDDL-2.2 (Planning Domain Definition Language)
  • [20%] PDDL+ (Planning Domain Definition Language Plus)
  • [05%] Optaplanner
  • [50%] Automatically schedule tasks using (temporal) planning algorithms
    • [35%] A CycL ontology of tasks and task properties
    • [35%] PERT Networks in CycL
  • ✅ Schedule important tasks
  • ✅ Set deadlines

Self-Discipline State Machine

  • Make sure the user is up to any task
  • Assist with follow-through on things like dieting, exercise, etc
  • [33%] Use the digital twin and plan to choose from possible things to do

Gamification

  • ✅ Score
  • [75%] Rewards/Penalties
  • ✅ Manager
  • ✅ Rewards for completing recurrent tasks
  • ✅ Daily penalty for leaving any recurrent tasks incomplete
  • ✅ Penalties for lack of movement discipline
  • [50%] Rewards for staying productive
  • [60%] Adherence tracker
    • ✅ Color coded labelled rectangles indicating the last accessed time of various FLP systems
      • ✅ Determine which systems are not being used correctly
      • ✅ Determine which sensors have stopped working (e.g. dead battery)

Movement Discipline

  • Ensure we think before we move
    • [33%] Optional mode where one must premeditate before moving to a different location

List-Manager

  • [50%] UniLang agent to manage lists
    • ✅ Using MySQL-persistent Perl linked list implementation

Transportation/Shopping/Errands

Transportation

  • [25%] Develop proper transportation plans
  • [33%] API look-up mashups (Application Programming Interface)
    • [90%] Hours of operation
      • [90%] Stores/offices/etc
    • [30%] Weather
    • [30%] Route planning
      • Point to point
      • "Traveling Salesman"
    • [90%] Reverse geocoding
  • [66%] Location logic
    • ✅ Owntracks integration
    • [75%] Use "Location Logic" to provide reminders
      • [50%] Locational rules
        • [75%] e.g. Don't forget to bring this with you when you leave
        • [75%] e.g. Silence one's cell phone in certain types of locations
performAction(addToPendingTasks(Agent,unsilenceCellPhone(Agent,Phone))) :-
        currentAgent(Agent),
        hasMobilePhone(Agent,Phone),
        leaving(Agent,Location),
        isa(Location,movieTheater),
        hasRecentlyPerformedAction(Agent,silenceCellPhone(Agent,Phone)).

Shopping/Errands

  • ✅ Shopping list management
    • ✅ Integration with financial management
      • ✅ OFX cross-referencing (Open Financial eXchange)
    • [33%] Integration with inventory and pantry management
  • Buy/sell things as necessary
  • Buying
    • Broker - Purchase/Financial Decision Support Systems
      • [10%] Argumentation (Possibly using MARGO or similar (Multiattribute ARGumentation framework for Opinion explanation))
        • [10%] Decide what to purchase
        • [10%] Decide whether to purchase
    • Ethical consumerism
    • ✅ Online Shopping
      • ✅ Receipts
        • [25%] Track all types of receipts
        • ✅ Parsers and storage for online store receipts
      • ✅ Online order tracker
      • ✅ Online delivery tracker
    • Brick and Mortar Shopping
  • Selling
    • Broker - personal selling system
      • Argumentation
        • Decide what to sell
        • Decide whether to sell

Document Management

  • Document Management System
    • ✅ Scan and OCR all paperwork (Optical Character Recognition)
      • ✅ Cross reference with schedule
      • ✅ Ensure confidentiality of information
    • Integration of document-related tasks into Workflow Manager
    • [90%] Backup w/ Git
    • ✅ Export selected documents and folders
    • ✅ Digital library system
      • ✅ Equipment manuals
    • ✅ Book reader
    • ✅ Information search management

Financial Planner

  • [50%] Financial Planning
  • ✅ Quickly forecast finances almost indefinitely into the future
    • ✅ Specify recurring transactions
    • ✅ Automatically detect recurring transactions from OFX exports and plan for them
      • e.g.
promiseToPayForSpec('ELEC',andrewDougherty,'',dollars(93.00),[can([1]),not(onTime([3])))).

  • ✅ View expected financial transactions along with primary agenda on a calendar

    • ✅ Calenderical recurrences using CLP (Constraint Logic Programming)
      • [25%] WebUI for editing financial recurrences
    • ✅ Predicted transactions' date, description, debit or credit and running balance
      • ✅ As a list
      • ✅ As a chart
    • ✅ Tell us when we're going to run out of money
      • ✅ Tell us how much we need to raise by when

  • ✅ Temporal metric planning for finances

  • Purchase Decision Support System / Broker Buy/Sell System

    • Oversubscription planning
      • Planning for maximizing utility given a cost bound, when can't accomplish all goals

  • ✅ Financial reasoning integrated with general planning

    • ✅ Adds financial reasoning to life-planning problems
    • ✅ Express expected date ranges of transactions using can, neg(onTime), months, date ranges, etc.

  • Metaplanners help develop contingency plans for different financial problems

    • TraCE temporally-contingent (metric) planner integration
    • ✅ Unexpected delays
    • Overcharges and unapproved charges
    • Cash flow problems and hardships
      • ✅ Help manage cash flow in tight situations

  • ✅ Abduction of recurrences and predicted events from bank statements in OFX or CSV (Open Financial Exchange, Comma-Separated Values)

    • Retrospectively analyze prior predictions
      • To update or correct financial recurrences

  • ✅ IEM2 for walking user through plans

    • [50%] Verify preconditions
    • ✅ Update world state with all effects
      • ✅ Extraction of world state deltas
    • [25%] WebUI for IEM2 (using Mock Perl/Tk objects)

  • [75%] SPSE2->PSEx3 export (Planning, Scheduling and Execution System (Extended), version 3)

    • [75%] Integrating financial reasoning and other domains, with SPSE's to-do dependency graph

  • ✅ Modification of PDDL/Verb domain/problem and temporal world state

  • ✅ Financial alerts

    • ✅ Text-to-speech alerts for recurrent transactions

  • Purchase Management

    • PDSS (Purchase Decision Support System)
      • [05%] Oversubscription Planning using Sym-Osp
      • Argumentation-based PDSS (Purchase Deciscion Support System)
        • Reasoning over user needs
        • Critical questions:
          • Is this purchase ethical/necessary/within-budget?
            • The assumption is "do not purchase," which must be soundly defeated to gain permission
            • Product and seller comparison
            • Payment plan management
    • ✅ Tracking purchases and deliveries
    • Inventory/pantry Inventory Management integration
    • Resource manager (automatic reordering and stock management)
    • Receipt tracker
      • [50%] Online
      • [50%] Brick and mortar
      • ✅ Matching bank records to receipts

  • Alerts for anticipated upcoming overdrafts

  • [10%] Displaying live/real-time bank information

    • [20%] Check current account balance automatically
    • Account history

  • (Personal) Accounting

    • XBRL bookkeeping
      • Loan and debt tracking/management
      • Budgeting (monthly)
        • Automatic creation of fundraising alerts and deadlines to maintain positive balance
      • Classifying (OFX) transactions
      • Financial reporting
        • Regulatory compliance (say, to Social Security rules)

  • Bill payment subsystem

Emergency Preparedness

  • Personal Emergency Management
    • [10%] Help the user stay on top of emergency preparedness
    • Analyze which kinds of emergencies are particular to the users situation
    • Prepare a "bug-out bag," emergency food and water supply, etc
    • Help the user maintain a ready posture
    • Safety plans and drills
    • [50%] Recurrences for reminders to maintain equipment, supplies, plans and drills
    • Safety plans
      • [05%] Fire
      • [80%] Pandemic
      • [10%] Food and water

Employment

Executive Function

  • Intelligence Augmentation
    • Learning
      • Education
      • Empirical
    • Cognitive Prosthesis
      • Prolog-Agent Intelligent Situated Agent / Softbot
    • Nootropics
  • Problem-Solving
    • Goal setting

Emacs Integration

  • ✅ 4680 custom ELisp functions as of [2024-04-10 Wed]
  • [25%] Org-mode integration
    • ✅ Org-agenda integration
  • [25%] Custom User Software integration
    • ✅ CLEAR book reader integration
      • ✅ Synchronized page turning for graphical emacs in Academician mode (derived from doc-view-mode)
  • [25%] Custom middleware
    • ✅ UniLang Emacs Client
  • [45%] FLP Developer's Console
    • ✅ Formalog-Agent
      • ✅ Emacs-based launching and control of Formalog agents such as FLP
      • ✅ Formalog-REPL
        • ✅ forward, backward CamelKebabCase minor mode movement
        • ✅ TAB, SHIFT-TAB code completion, apropos, etc
        • ✅ definitions, listings, scrying, source lookups and jumps, etc
      • ✅ ripgrep searching of code
      • ✅ AGENT–>GOAL
        • e.g. spse2–>member(X,[a,b,c]) nondet syntactic sugar
    • ✅ Verber planning domain/problem editor and runner
    • [25%] Prolog-Agent
      • [25%] Prolog-agent based control of Emacs over UniLang
    • [50%] LLM completion support
  • [25%] ELisp-based KBS/ATP system (Automated Theorem Prover)

Future Work

Integrations

  • FLP-2.0 automatic refactoring/rewrite using MetaGPT-like program synthesis
  • [15%] Eurisko for IAEC
  • [20%] am-utexas for IAEC
  • TraCE-lite
  • [15%] Symbolic-OSP
  • s(CASP)<->FLP(SWIProlog) integration for Event Calculus/commonsense reasoning
  • FreeKBS2/Prolog context/microtheory support
  • Privacy
    • Automatic data scrubbing and redaction
      • [20%] Using multi-stage LLM based filtering
      • [65%] Screen / video redactor
        • Ollama Llama3.2:latest 1.5B GPU-based redaction engine
    • HIPAA compliance
    • Encryption
      • Data-at-rest encryption
      • In-memory encryption
      • Full-disk encryption
    • Various security methodologies:
      • Principle of least privilege
      • Additional security partitioning

Special Use Cases

Homelessness

Illness

Insolvency

Abuse

Climate Change and Disaster Management

Major Technologies Used

  • SWI-Prolog (Sociaal-Wetenschappelijke Informatica)
    • ✅ Julian time library
    • ✅ CLP(fd) (Constraint Logic Programming)
    • [50%] QLF (Quick Load Format)
  • ✅ Perl
  • ✅ Java
  • ✅ Python
  • ✅ Bash scripting
  • ✅ Emacs integration
  • ✅ PDDL 2.2 temporal metric planning
    • ✅ LPG-td-1.0
    • ✅ OPTICCLP (Optimizing Preferences and Time-Dependent Costs) (Coin-or Linear Programming)
  • ✅ Vampire-KIF (Knowledge Interchange Format) ATP (Automated Theorem Proving) system
  • ✅ SUMO
  • ✅ OpenCyc
  • [30%] PrologCyc / Cyclone KBS system
    • Based on Prolog/FreeKBS2/CycAPI/etc
      • ✅ Prolog<->Cyc direct interface
      • [25%] Various Cyc API-like but in PrologCyc / KBS predicates
        • [25%] isa, genls, allTermAssertions, constantApropos, minGenls, maxSpecs, etc.
        • [15%] Populated subsumption hierarchy
      • [45%] Various Prolog Microtheory implementations
    • [65%] SubL/CycL parsers/cyclifiers in Prolog and Perl
  • MariaDB/MySQL persistence
  • [33%] Inform7
  • ✅ Local LLMs (Large Language Models)
  • ✅ Remote LLMs (Large Language Models)
  • ✅ Rhasspy Voice Assistant integration
  • ✅ Prolog-Agent, Autonomous-AI-Agent
    • ✅ [33%] SPAMI (SWI-Prolog AgentSpeak(L) Meta-Interpreter)
      • Reimplementation of Jason/Agentspeak(L)

More Info

More Use Cases

  • Remembering to take the trash out before the trash truck arrives
  • Getting an automatic notification to stock up on groceries before a predicted storm hits, including the list of the groceries you need
  • Remembering that when you finish filling out some particular paperwork, to send a copy to one's doctor
  • Remember what the doctors instructions were, in a paperless office with Machine Comprehension software, and helping you to conform to them
  • Reminding you to work on one's taxes, and to change one's air filter
  • Telling you that a particular plan of one's violates various moral and ethical constraints
  • Generating a complex monthly financial plan with contingencies for unexpected expenses and if certain income doesn't come through

Recognition

Links

Screenshots

Subsystems

More Links