Análise de qualidade da construção de microsserviços, com foco em resiliência.
Build-quality analysis for microservices, focused on resilience.
CRAFT scores a service's resilience as built — the presence and calibration of protection mechanisms at its functional boundary — rather than as observed in production. It produces a 0–10 Nota de Resiliência (CRAFT Score), a four-tier classification, a per-item evidence ledger, and an ordered remediation plan with ready-to-execute fix prompts.
CRAFT is built around a falsifiable hypothesis:
Structural resilience conformance is a property worth measuring before runtime evidence exists.
The project does not assume that a resilience score is valuable simply because it can be computed. CRAFT becomes useful only if the evidence it produces changes or improves engineering decisions.
Three questions therefore matter more than the score itself:
- Does CRAFT surface structural risks that experienced engineers would want to know before production?
- Can independent evaluators inspect the same repository and profile and derive materially consistent evidence?
- Is the resulting evidence useful enough to influence design reviews, remediation priorities, delivery gates, or other engineering decisions?
The current public corpus demonstrates that the method can be executed against real repositories, that every awarded point can be tied to inspectable evidence, and that the positive control is distinguished from the sampled public services.
It does not yet establish that CRAFT predicts production reliability, that different evaluators will always reach the same judgement, or that organizations should use the score as a delivery gate.
Those are validation questions, not assumptions of the framework.
A useful test for CRAFT is deliberately simple:
Would an experienced engineer receive a CRAFT finding and say: “I would want to know this before putting the service into production”?
If the answer is repeatedly no, then CRAFT is only another scanner.
If the answer is repeatedly yes, structural resilience conformance may be a useful engineering control.
Looking for the agent skill? Use
skills/craft-audit/.
That directory is the complete portable skill. Copy it as a unit;SKILL.mddepends on both files under its localreferences/directory.
craft-audit/
│
├── README.md
├── LICENSE
│
├── skills/
│ └── craft-audit/
│ ├── SKILL.md
│ └── references/
│ ├── profile.md
│ └── output-schema.md
│
├── docs/
│ ├── CRAFT-v1.0.pdf
│ ├── craft-badge-howto.md
│ └── craft-score-badge.svg
│
└── corpus/
├── scorecard.md
└── ...
The separation is intentional:
skills/craft-audit/is the reusable agent skill.docs/contains human-facing methodology and presentation material.corpus/contains published audit evidence and benchmark data.README.mdexplains how the pieces fit together.
A future packaged distribution may live under dist/, but no generated package is committed today. The source of truth is the directory under skills/.
The smallest portable CRAFT unit is this directory:
skills/craft-audit/
├── SKILL.md
└── references/
├── profile.md
└── output-schema.md
The three files have different responsibilities and should always be versioned and copied together:
| File | Role | When the agent must read it |
|---|---|---|
skills/craft-audit/SKILL.md |
Audit procedure — trigger, repository inventory, boundary discovery, evidence resolution, anti-pattern checks, scoring workflow, findings and output steps | First; this is the entry point |
skills/craft-audit/references/profile.md |
Scoring policy — CRAFT/MS-1.1.1 weights, maxima, conditional rules, penalties, normalization and classification | Before assigning any score |
skills/craft-audit/references/output-schema.md |
Output contract — JSON fields, evidence ledger shape, findings/remediation structure and CSV schemas | Before writing audit artifacts |
Inside the skill directory the dependency remains intentionally relative:
SKILL.md
├── requires before scoring ──> references/profile.md
└── requires before output ──> references/output-schema.md
Do not copy SKILL.md alone. Preserve the directory layout so those relative references remain valid in any agent, repository, or harness.
The destination directory is agent/harness-specific. Copy the whole skill directory rather than selecting files individually:
git clone --depth 1 https://github.com/RudsonCarvalho/craft-audit.git
cp -R craft-audit/skills/craft-audit <agent-skill-dir>/craft-auditAfter the copy, the destination should still look like:
<agent-skill-dir>/craft-audit/
├── SKILL.md
└── references/
├── profile.md
└── output-schema.md
If an agent can read this GitHub repository directly, point it to skills/craft-audit/SKILL.md; the required references are available beside it at their expected relative paths.
CRAFT does not require a proprietary skill loader. A generic coding agent can be given the contract explicitly:
Use skills/craft-audit/SKILL.md as the CRAFT audit procedure.
Before scoring, read skills/craft-audit/references/profile.md.
Before writing outputs, read skills/craft-audit/references/output-schema.md.
Run the audit against <repository-or-service-path>.
Do not award points without file/line evidence and a resolved configuration value.
A shorter invocation is enough when the skill is already installed:
Run the CRAFT resilience audit on ./my-service
The skill creates <repo-name>-craft/ and emits, per service:
<service>-audit.json machine-readable evidence ledger and score
<service>-report.md human-readable findings and remediation plan
For multi-service/repository audits it also emits:
summary.csv corpus-level summary
scorecard.md portfolio view when 2+ services are audited
The exact machine-readable contract is defined by skills/craft-audit/references/output-schema.md.
Treat skills/craft-audit/ as one release unit:
SKILL.mddefines how to audit;references/profile.mddefines how to score;references/output-schema.mddefines how to serialize the result.
The profile_version emitted by an audit should match the profile actually loaded by the agent. The active reference profile in this repository is CRAFT/MS-1.1.1.
Human-facing material is intentionally separate from the executable skill:
docs/CRAFT-v1.0.pdf— methodology paper.docs/craft-badge-howto.md— how to display a traceable CRAFT score badge.docs/craft-score-badge.svg— versioned badge asset.
None of these files is required to execute the audit skill.
CRAFT is not observability (it does not watch production), not chaos engineering (it does not inject failure), and not SLO measurement (it does not measure outcome). It asks a different question: where is this service structurally fragile, and how would we know before the incident?
Outcome metrics — MTBF, SLIs, chaos experiments, load tests — are conditioned on what the environment happened to exercise. A service that was never stressed and a service engineered to withstand stress can produce indistinguishable readings. CRAFT instead evaluates the construction of the artifact and the protections visible at its functional boundary.
CRAFT was first published as REOF (Resilience Evaluation and Optimization Framework) on 10 April 2024: https://medium.com/@rudsonkiyoshicarvalho/resilience-evaluation-and-optimization-framework-reof-541d23018460
That article contains the original specification, including the passage recording that a precise automated index would require a code-comprehending evaluator that "does not exist on the market" — the requirement this work's agent-based auditor satisfies. It is cited in the paper as the public, third-party-timestamped provenance of that claim.
Core rule — provenance or reject: no evidence, no score. An item whose effective value cannot be resolved is scored zero and logged UNVERIFIED; the auditor must not credit an assumed framework default or a mechanism that merely appears likely to exist.
The workflow inventories services and interaction points, resolves each protection mechanism to its configured value, checks indirect/injected protection, applies anti-pattern penalties and the domain-coherence degradation factor, then normalizes the result to 0–10.
| Nota de Resiliência | Classificação | Meaning |
|---|---|---|
| 8.0–10.0 | Excelente | High reliability and resilience |
| 5.0–7.9 | Bom | Reliable, but with meaningful structural gaps |
| 3.0–4.9 | Aceitável | Corrective measures required; low robustness |
| 0.0–2.9 | Insatisfatório | Requires revision; risks damage to adjacent services |
The repository contains nine service audit entries across five public repositories. Every published score is tied to a recorded commit so the evidence can be re-derived.
| Service | Repository | Score | Classificação |
|---|---|---|---|
| customers-service | spring-petclinic-microservices | 0.00 | Insatisfatório |
| visits-service | spring-petclinic-microservices | 0.00 | Insatisfatório |
| vets-service | spring-petclinic-microservices | 0.00 | Insatisfatório |
| ts-preserve-service | train-ticket | 0.04 | Insatisfatório |
| checkoutservice | microservices-demo | 0.16 | Insatisfatório |
| api-gateway | spring-petclinic-microservices | 1.05 | Insatisfatório |
| circuitbreaker-demo | spring-circuitbreaker-demo | 2.17 | Insatisfatório |
| cartservice | microservices-demo | 2.97 | Insatisfatório |
| resilience-golden-demo | resilience-golden-demo | 10.00 | Excelente |
Scores are under profile CRAFT/MS-1.1.1. Earlier published values (api-gateway 1.13, checkoutservice 1.72,
ts-preserve-service 0.05) were scored under the uncorrected MS-1.1 maxima and are superseded —
see corpus/RECALCULATION.md for what changed and why.
See corpus/scorecard.md for the portfolio view and the structurally distinct failure shapes found in the corpus.
resilience-golden-demo is the positive-control fixture: a microscopic Spring Boot service that deliberately exposes every mechanism needed by the active profile across HTTP entry, synchronous consultation, outbound integration, Redis, Kafka, and container/deployment configuration.
Its published audit is stored under corpus/resilience-golden-demo/. The positive control also exposed an arithmetic inconsistency in the previous profile maxima, which was corrected in MS-1.1.1.
Preprint v1.0. The framework was first published in April 2024 under the name REOF; this work renames it CRAFT and adds an agent-based evaluator and an empirical corpus. The most important open validation item is an agent-versus-human auditor agreement study. The corpus also cannot currently support a meaningful weight-sensitivity analysis, because its score distribution is bimodal, and audits do not yet record evaluator versioning (model, harness, skill commit) — so this work claims auditability of evidence rather than reproducibility of judgement. All three limitations are stated in the paper rather than hidden.
@misc{carvalho2026craft,
author = {Carvalho, Rudson Kiyoshi Souza},
title = {{CRAFT}: A Structural Conformance Index for System Resilience,
and the Evaluator That Did Not Exist},
year = {2026},
note = {Preprint v1.0; supersedes REOF (2024)},
url = {https://github.com/RudsonCarvalho/craft-audit}
}Apache-2.0