Deceptively Simple Technologies Inc's Adaptív Application Foundation (AAF) Data Layer (AAF Data)

Locally-Sourced, Artisanal Data™

Delivering Foundational Business Agility

Copyright © 2003-2024 Deceptively Simple Technologies Inc. Some rights reserved. Please see the aafdata/LICENSE.txt file for details.

TL;DR

Today, AAF Data is:

• A standalone set of
  • Generalized business entity models, e.g. Person, Organization, OrganizationalUnit, Employee, etc,
  • Database creation scripts for roles, schemas, tables, constraints, indexes, functions, and
  • Scripted lookup/reference data, e.g. EntityType (Person, Organization, OrganizationalUnit, Employee, etc), EntitySubtype (Organization - C Corp, S Corp, LLC, etc, OrganizationalUnit - Region, Division, Department, etc), etc,
• A Dockerized, RESTful entity data microservice (EDM) providing
  • Create,
  • Read,
  • Update, and
  • Delete (CRUD) operations for each of these business entity models,
• A Dockerized, RESTful system data service (SDS) providing the capability to
  • Create New business entity definitions,
  • Clone Existing business entity definitions,
  • Create New business entity attributes,
  • Clone Existing business entity attributes,
  • Associate business entity attributes with business entity definition(s),
  • Publish entities, attributes, and associations, thereby creating corresponding new database schemas, tables, constraints, indexes, etc, and
  • Update the published status of entities, attributes, and associations,
• A lightweight, modern Progressive Web Application (PWA) business entity modeling service (EMS) for extending or modifying these models using the EDM and SDS services,
• Our first custom HTML5 Web Component (HtmlElement) EntityTable demonstrating how to quickly and easily display business entity data on a web page,
• And SwaggerDoc OpenAPI application programming interface (API) documentation.

Project Website: Deceptively Simple Technologies Inc

To Provide Product Feedback & Be Notified of Updates: 2-Minute Feedback Form

Note: Cygnus Technology Services, who is collecting product feedback, is a Deceptively Simple Technologies (DST) service partner, providing AAF Data implementation, customization, training, and support services. If you are interested in becoming a DST partner, please email [email protected].

Features

• Containerized. Services run in Docker containers, available from DockerHub.

• The entity data microservice (EDM):
  • Will validate each HTTP request (near future) using a JSON Web Token (JWT) mechanism with optional cryptographic signing and encryption capabilities,
  • Persists and retrieves business entity data via its API, which calls low-level POST, GET, PATCH, and “soft” DELETE database functions to ensure data integrity and performance -- NO direct table access,
  • Consistently follows RESTful style and best practices,
  • Returns standard and appropriate HTTP response codes, e.g. 200 OK, 201 Created, 400 Bad Request, 401 Unauthorized, 404 Not Found, 422 Unprocessable Entity, 500 Internal Server Error, etc
  • Accepts and returns structured JSON resource/entity data in HTTP request/response bodies as specified for each business entity, and

• The system data service (SDS):
  • Will validate each HTTP request (near future) using a JSON Web Token (JWT) mechanism with optional cryptographic signing and encryption capabilities,
  • Enables authorized entity Modelers and Publishers to define new EntityTypeDefinitions, EntityTypeAttribtues, and EntityTypeDefinitionEntityTypeAttributeAssociations via its API, which calls high-level process POST and low-level entity POST, PATCH, and “soft” DELETE database functions to add (or "publish") these new entity definitions, attributes, and associations to the database, resulting in new model-driven data structures, e.g. SCHEMAs, TABLEs, COLUMNs, CONSTRAINTs, INDEXs, etc,
  • Consistently follows RESTful style and best practices,
  • Returns standard and appropriate HTTP response codes, e.g. 200 OK, 201 Created, 400 Bad Request, 401 Unauthenticated, 404 Not Found, 422 Unprocessable Entity, 500 Internal Server Error`, etc
  • Accepts and returns structured JSON resource/entity data in HTTP request/response bodies as specified for each business entity, and

• The entity modeling service (EMS):
  • Will validate each HTTP request (near future) using a JSON Web Token (JWT) mechanism with optional cryptographic signing and encryption capabilities,
  • Helps authorized entity Modelers and Publishers to define new AAF Data-compliant EntityTypeDefinitions and EntityTypeAttribtues, associate them, and publish them via its API, which results in new model-driven data structures, e.g. SCHEMAs, TABLEs, COLUMNs, CONSTRAINTs, INDEXs, etc,
  • Consistently follows RESTful style and best practices,
  • Returns standard and appropriate HTTP response codes, e.g. 200 OK, 201 Created, 400 Bad Request, 401 Unauthenticated, 404 Not Found, 422 Unprocessable Entity, 500 Internal Server Error`, etc
  • Accepts and returns structured JSON resource/entity data in HTTP request/response bodies as specified for each business entity, and

Rules

• The custom database roles, e.g. AafCoreOwner, AafCoreDataReadWrite, etc, are to be used appropriately (as designed and demonstrated in this project) to improve data security.
• The current set of business entities and their descriptions can be retrieved using the GetEntityTypeDefinitions database function and includes:
  • *EntityTypeDefinition
  • *EntityTypeAttribute
  • *EntityTypeDefinitionEntityTypeAttributeAssociation
  • EntitySubtype
  • EntitySubtypeHierarchy
  • EntityType
  • Language
  • Locale
  • GeographicUnit
  • GeographicUnitHierarchy
  • Currency
  • Periodicity
  • PostalAddress
  • Organization
  • OrganizationalUnit
  • OrganizationalUnitHierarchy
  • Person
  • Employee
  • LegalEntity
• The set of "origin" (*) business entities can produce other business entities and their corresponding database structures
• The current set of business entity attributes and their descriptions can be retrieved using the GetEntityTypeAttributes database function and includes:
  • *Id bigint
  • *Uuid uuid
  • *EntitySubtypeId bigint
  • *TextKey varchar
  • LocalizedName varchar
  • LocalizedDescription varchar
  • LocalizedAbbreviation varchar
  • *ResourceName varchar
  • *Ordinal bigint
  • *IsActive boolean
  • *CorrelationUuid uuid
  • *Digest varchar
  • *CreatedAtDateTimeUtc timestamp
  • *CreatedByInformationSystemUserId bigint
  • *UpdatedAtDateTimeUtc timestamp
  • *UpdatedByInformationSystemUserId bigint
  • *DeletedAtDateTimeUtc timestamp
  • *DeletedByInformationSystemUserId bigint
• The set of common (*) business entity attributes are to be included in each business entity definition and corresponding database table, etc.
• These business entity attribute names, data types, Pascal case, etc are intentional, carefully chosen, and NOT to be altered.
• When naming entities and attributes, we avoid acronyms and abbreviations, and we strive to think carefully about, to "call things what they really are", and to be specific, e.g. AtDateTimeUtc.
• Each entity’s database table is partitioned using a corresponding Postgres schema, e.g. "Organization"."Organization".
• EntitySubtype, GeographicUnit, GeographicUnitHierarchy, etc represent scripted (canonical) lists of options like an entity’s subtypes, etc.
• Unknown (Id = -1) and None (Id = 0) are standard EntitySubtype and EntitySubtype values indicating a currently unknown/to be determined or an inapplicable entity or entity subtype value, respectively.

Soon AAF Data will also include:

• A complete, basic set of sixty (60) business entity models, built with the AAF Data Modeler, including InformationSystem and ProductAndService domains:
  • InformationSystemUser, InformationSystemRole, DigitalFile, DigitalFileFolder, Product, Service, Subscription, ContentItem, etc
• Terraform scripts for creating AWS infrastruture and deploying the AAF Data services to the cloud,
• JWT helper functions (encode/decode),
• A structure service for conforming the entity data structures to the model, and
• CRUD operation event publishing.

Getting Started

This is the third AAF Data release, and it supports very simple local (LOC) and minimal (MIN) environments, primarily to demonstrate its purpose and basic capabilities.

Please bear in mind that this is a work in progress. There are many more features and capabilities to come, and features are often "layered", meaning that they have dependencies on other features that must be implemented first. A good example of this is data validation, which is implemented with database column constraints in the data layer but which is enhanced with code-based validation in the service layer and which will be further enhanced in the presentation layer, using the same data types and constraints defined in the business entity attribute model and used to generate the corresponding database tables when published.

Prerequisites

You will need:

1. Root (default postgres role) access a PostgreSQL database server
2. Either a local copy of the aafdata/EntityDataMicroservice code (and an integrated development environment like JetBrains IntelliJ IDEA), by cloning the code repository, or a DockerHub account to pull and run the Docker image

Installation

Before running the EntityDataMicroservice, you must:

1. Change directory (cd) to aafdata/src/main/data/, make executable (chmod +x <fileName>), and run the eight (8) BASH command line scripts in aafdata/src/main/data/ in your PostgreSQL database server in the specified order and with the specified roles (changing the postgres role's password if necessary) to create the necessary roles, schemas, tables, functions, and data:
   1. ./001-CreateDatabaseServer.sh PostgreSQL 14 localhost 5432 postgres postgres postgres
   2. ./002-CreateDatabaseRoleAafCoreOwnerAsPostgresRole.sh PostgreSQL 14 localhost 5432 postgres postgres postgres
   3. ./003-CreateDatabaseRolesAndDatabaseAsAafCoreOwnerRole.sh PostgreSQL 14 localhost 5432 AafCoreOwner postgres 0wn3rCl13nt!
   4. ./004-CreateDatabaseExtensionsAsAafCoreOwnerRole.sh PostgreSQL 14 localhost 5432 AafCoreOwner AafCore 0wn3rCl13nt!
   5. ./005-CreateDatabaseSchemasAsAafCoreModeler.sh PostgreSQL 14 localhost 5432 AafCoreModeler AafCore M0d3l3rCl13nt!
   6. ./006-CreateDatabaseTablesAsAafCoreModeler.sh PostgreSQL 14 localhost 5432 AafCoreModeler AafCore M0d3l3rCl13nt!
   7. ./007-CreateDatabaseFunctionsAsAafCoreModeler.sh PostgreSQL 14 localhost 5432 AafCoreModeler AafCore M0d3l3rCl13nt!
   8. ./008-CreateDatabaseDataAsAafCoreModeler.sh PostgreSQL 14 localhost 5432 AafCoreModeler AafCore M0d3l3rCl13nt!

Please note that this third AAF Data release is primarily for demonstration and evaluation purposes. It supports very simple local (LOC) and minimal (MIN) remote/cloud environments and is not intended to be used in a production environment. Later releases will support shared, remote/cloud development (DEV), staging (STG), and production (PRD) environments and will utilize secrets management and other security best practices. Crawl, walk, run.

1. Run the Java/Springboot-based EntityDataMicroservice:

   1. Locally in IDE:
      1. Open the aafdata project in IntelliJ IDEA.
      2. Run the EntityDataMicroservice class.
   2. Locally from the command line:
      1. Set the profile environment variable with export spring_profiles_active=loc (NOTE: Remove it with unset spring_profiles_active)
      2. Change directory (cd) to aafdata/EntityDataMicroservice/.
      3. Build the project:

         mvn clean package

      4. Run the project:

         java -jar target/EntityDataMicroservice-0.0.1-SNAPSHOT.jar

   3. In Docker:
      1. Set the profile environment variable with export spring_profiles_active=min (NOTE: Remove it with unset spring_profiles_active)
      2. Build a Docker image with docker build -t deceptivelysimpletechnologies/aafdata-entitydatamicroservice:$(date +%Y%m%d_%H%M%S) .
      3. -OR-
      4. Pull the Docker image from DockerHub:

         docker pull deceptivelysimpletechnologies/aafdata-entitydatamicroservice:20250618_164418

      5. Run the Docker container:

         docker run -d --name aafdata-edm-min -e spring_profiles_active=min -p 8080:8080 -t deceptivelysimpletechnologies/aafdata-entitydatamicroservice:latest

2. Run the Java/Springboot-based SystemDataService:

   1. Locally in IDE:
      1. Open the aafdata project in IntelliJ IDEA.
      2. Run the SystemDataService class.
   2. Locally from the command line:
      1. Set the profile environment variable with export spring_profiles_active=loc (NOTE: Remove it with unset spring_profiles_active)
      2. Change directory (cd) to aafdata/SystemDataService/.
      3. Build the project:

         mvn clean package

      4. Run the project:

         java -jar target/SystemDataService-0.0.1-SNAPSHOT.jar

   3. In Docker:
      1. Set the profile environment variable with export spring_profiles_active=min (NOTE: Remove it with unset spring_profiles_active)
      2. Build a Docker image with docker build -t deceptivelysimpletechnologies/aafdata-systemdataservice:$(date +%Y%m%d_%H%M%S) .
      3. -OR-
      4. Pull the Docker image from DockerHub:

         docker pull deceptivelysimpletechnologies/aafdata-systemdataservice:20250618_165337

      5. Run the Docker container:

         docker run -d --name aafdata-sds-min -e spring_profiles_active=min -p 8081:8081 -t deceptivelysimpletechnologies/aafdata-systemdataservice:latest

3. Run the Progessive Web Application (PWA) EntityModelingService:

   1. In Docker:
      1. Change directory (cd) to aafdata/EntityModelingService/.
      2. Build a Docker image with docker build -t deceptivelysimpletechnologies/aafdata-entitymodelingservice:$(date +%Y%m%d_%H%M%S) .
      3. Run the Docker container:

         docker run -d --name EntityModelingService -e spring_profiles_active=min -p 8082:80 -v ~/Documents/Project/dst/aafdata/EntityModelingService:/usr/share/nginx/html:ro deceptivelysimpletechnologies/aafdata-entitymodelingservice:20250618_170134

      4. Open a web browser and navigate to http://localhost:8082/ to view the EntityModelingService.

(back to top)

Usage

A complete set of Postman requests for both EDM and SDS services is included with the source code at aafdata/client/DST AAF Data.postman_collection.json.

Note: Each AAF Data API request requires a JWT-encoded cookie, which must be added to the Postman client manually since it will not export with the request collection for security reasons. The cookie must be name "Authentication" and is for the moment hard-coded with the following structure and content:

Authentication=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6ImxvYyJ9.eyJpc3MiOiJBQUZEYXRhLUNsaWVudCIsInN1YiI6IkF1dGhlbnRpY2F0aW9uIiwiYXVkIjoiQUFGRGF0YS1FbnRpdHlEYXRhTWljcm9zZXJ2aWNlIiwiZXhwIjoxNzIzODE2OTIwLCJpYXQiOjE3MjM4MTY4MDAsIm5iZiI6MTcyMzgxNjc4OSwianRpIjoiZWY0YWY0ZTMtZTczNi00MjVhLWFhZmYtZWNhMDNiN2I5YjI4IiwiYm9keSI6eyJFbWFpbEFkZHJlc3MiOiJhbXkuYW5kZXJzb25AYW15c2FjY291bnRpbmcuY29tIn19.a5UdLPgo4CzMyK1_JuWbQQQEMYz-rBcLu5uH0sZElqw; Path=/; Domain=localhost; HttpOnly; Expires=Mon, 18 Aug 2025 12:07:49 GMT;

(back to top)

Table of Contents

Expand for Details

1. About The Project
2. History
3. The AAF Vision
4. What Is AAF Today?
   • How Are the AAF Models Represented in the Data Layer?
     • Business Entity Microservice
     • Data Structure Service
   • How Will the AAF Models Be Represented in the Service Layer?
   • How Will the AAF Models Be Represented in the Presentation Layer?
5. Version History
6. Built With
7. Roadmap
8. Contributing
9. License
10. Contact
11. Acknowledgments

About the Project

AAF Data is the crucial first layer -- business entity data modeling and persistence -- of the Adaptív Application Foundation (AAF).

(back to top)

History

In the 30 years or so that I've been designing, developing, and delivering software, I've seen anti-patterns (things that one should never do) like object-relational mappers (ORMs) and data transfer objects (DTOs) become not only acceptable but standard operating procedure.

Code size, dependencies, and complexity have skyrocketed while fundamentals, performance, and maintainability have plummeted to the point where many developers no longer know the basics of their craft or use the right tool for the job.

I've written about this in my Look, Fuckers publication on Medium, and I've led several teams to implement my ideas -- the best practice ideas now "baked into" this AAF Data layer product -- in custom, commercial applications, but this process is time-consuming and expensive.

So, I have worked slowly but steadily over the years, creating this free, Open Source implementation in order to to remove the "costs too much/takes too long" risks and objections that have complicated custom implementations, and I am very excited to offer these assets and capabilities for your consideration and use.

The Adaptív Application Foundation (AAF) is the result of a number of related, reinforcing, and enabling ideas for software application technical design and implementation I've had and worked to realize and refine starting in 2003. These ideas have come to illustrate and embody my Metis approach to system design and architecture (my definition here).

In 2006 I founded Deceptively Simple Technologies Inc to take up the further development and promotion of technology product ideas like the AAF. Cygnus Techology Services is my services company, which focuses on consulting, customization, and commercialization of these ideas and products, guided by my own 25+ years of experience with technology leadership & strategy, people, product & process, and technology & tactics.

The AAF has gone through many successful incarnations over the years, enabling several organizations to quickly and efficiently create and maintain business technology solutions after investing the resources needed to implement the infrastructure that makes up the foundation. I've learned and refined my ideas in each case, and in a few places I've been fortunate enough to collaborate with other professionals who truly believed in the potential of this approach and helped to improve it and to push it forward.

But most of the time I've encountered ineptitude and laziness and fear and resistance bordering on hostility, misunderstanding of the process and timeline, inability to stay the course

I myself was also fearful of making these ideas public (hey, I'm old).

Today, I believe that I have clarified and resolved most of the inconsistent ideas in the AAF, and it is light, fast, adaptable, scalable, and ready to rock. In making it available under open source licensing, I intend to remove the "costs too much/takes too long" objections to custom implementations of the foundation as well as to "bake in" my process and attention to detail, which most developers don't take the time to understand or value.

The AAF Vision

The AAF will be an integrated, standards-based collection of business entity definitions and relationships, data structures, and data, executable business process models, and business rule (or "decision") models that together provide basic out-of-the-box business functionality that can be utilized as is, modified, extended, etc.

The foundation recognizes and implements common business entity, process, and decision patterns, leading to efficiency, re-use, and happy stakeholders and end-users. Business entities and their relationships represent the persons, places, and things that our information systems are intended to manage (i.e. the "nouns"). Business processes (i.e. the "verbs") orchestrate the interactions and operations involving these entities. And business decisions (i.e. the "rules") inform and constrain these processes.

What Is AAF Today?

In an effort to deliver significant value in the shortest possible timeframe, the AAF data layer (AAF Data) will be released first.

Today, AAF Data is:

• A standalone set of
  • Generalized business entity models, e.g. Person, Organization, OrganizationalUnit, Employee, etc,
  • Database creation scripts for roles, schemas, tables, constraints, indexes, functions, and
  • Scripted lookup/reference data, e.g. EntityType (Person, Organization, OrganizationalUnit, Employee, etc), EntitySubtype (Organization - C Corp, S Corp, LLC, etc, OrganizationalUnit - Region, Division, Department, etc), etc,
• A Dockerized, RESTful entity data microservice (EDM) providing
  • Create,
  • Read,
  • Update, and
  • Delete (CRUD) operations for each of these business entity models,
• A Dockerized, RESTful system data service (SDS) providing the capability to
  • Create New business entity definitions,
  • Clone Existing business entity definitions,
  • Create New business entity attributes,
  • Clone Existing business entity attributes,
  • Associate business entity attributes with business entity definition(s),
  • Publish entities, attributes, and associations, thereby creating corresponding new database schemas, tables, constraints, indexes, etc, and
  • Update the published status of entities, attributes, and associations,
• A lightweight, modern Progressive Web Application (PWA) business entity modeling service (EMS) for extending or modifying these models using the EDM and SDS services,
• Our first custom HTML5 Web Component (HtmlElement) EntityTable demonstrating how to quickly and easily display business entity data on a web page,
• And SwaggerDoc OpenAPI application programming interface (API) documentation.

How Are the AAF Models Represented in the Data Layer?

Business Entity Microservice

Scripted database, role, schema, table, and data

Business entity type definitions, entity attributes, type definition/attribute association

EntityType structure and data

EntitySubtype structure and data

System Data Service

Create new or clone existing EntityTypeDefinitions and EntityTypeAttributes.

Associate or dissociate EntityTypeAttributes with EntityTypeDefinitions.

Publish all unpublished EntityTypeDefinitions, EntityTypeAttributes, and EntityTypeDefinitionEntityTypeAttributeAssociations, resulting in corresponding new database schemas, tables, constraints, indexes, etc.

Entity Modeling Service (EMS)

How many total, published, and unpublished EntityTypeDefinitions, EntityTypeAttributes, EntityTypeDefinitionEntityTypeAttributeAssociations, and scripted EntitySubtypes exist in the database?

List all EntityTypeDefinitions and the EntityTypeAttributes associated with each EntityTypeDefinition.

Create new or clone existing EntityTypeDefinitions and EntityTypeAttributes.

Associate or dissociate EntityTypeAttributes with EntityTypeDefinitions.

List all EntitySubtypes

Publish all unpublished EntityTypeDefinitions, EntityTypeAttributes, and EntityTypeDefinitionEntityTypeAttributeAssociations, resulting in corresponding new database schemas, tables, constraints, indexes, etc.

Features our first custom HTML5 Web Component (HtmlElement) EntityTable demonstrating how to quickly and easily display business entity data on a web page

How Will the AAF Models Be Represented in the Service Layer?

Coming soon

Out-of-the-box, ready-to-run BPMN process and DMN decision definitions

How Will the AAF Models Be Represented in the Presentation Layer?

Please see our first custom HTML5 Web Component (HtmlElement) EntityTable (above).

Coming soon: Other model-based web user interface (UI) components

Version History

• 0.0
  • Various bug fixes and optimizations
  • Initial Release of EDM and basic business entities
  • See commit change or See release history
• 1.0
  • Initial Release of SDS and related EDM changes
  • Various bug fixes and optimizations
  • See commit change or See release history
• 2.0
  • Initial Release of EMS and related SDS changes
  • Various bug fixes and optimizations
  • See [commit change](

Built With

• OpenJDK
• Spring Boot
• Webflux
• PostgreSQL
• IntelliJ IDEA
• Postman
• FasterXML/jackson
• springdoc-openapi
• SLF4J

(back to top)

Roadmap

• Set up JWT infrastructure with request body in token
• Add JWT helper functions (encode/decode)
• Finish README.md
• LOC environment setup (with profiles, etc)
• Dockerfile with AWS CloudFront logging driver, etc
• Add EDM image to DockerHub
• Add SDS image to DockerHub
• Add EMS image to DockerHub
• [.] Unit testing
• Security testing (database table direct access, etc)
• Integration testingSet up Terraform Remote Backend With AWS Using A Bash Script
• Enhance Swagger/OpenAPI documentation with cached EntityType, etc data
• Implement a global error handler for all exceptions with JSON return
• Implement localization support
  • U.S. English (system default)
  • Spanish

See the open issues for a full list of proposed features (and known issues).

(back to top)

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please check the roadmap (above) for planned enhancements, etc, clone the repo, create a new feature branch, implement your idea there, and create a pull request. Don't forget to give the project a star! Thanks again!

1. Check the Roadmap (above)
2. Clone the Project
3. Create your Feature Branch (git checkout -b feature/AmazingFeature)
4. Commit your Changes (git commit -m 'Add the AmazingFeature')
5. Push to the Branch (git push origin feature/AmazingFeature)
6. Open a Pull Request

(back to top)

License

Distributed under the GNU GPLv3. Please see the aafdata/LICENSE.txt file for more information.

(back to top)

Contact

Doug Wilson, President & CEO, Deceptively Simple Technologies Inc Project Link: DeceptivelySimpleTechnologies/aafdata

(back to top)

Acknowledgments

Here are some of the resources I've found helpful during the development of this project and would like thank.

• Choose an Open Source License
• Img Shields
• GitHub Pages
• Font Awesome
• Toptal gitignore.io
• jwt.io
• Online UUID Generator
• SHA-256 Hash Calculator
• JSON Formatter
• Browserling All Hash Generator

(back to top)