A bridge isn't competent because the blueprint is elegant, but because the bridge holds — and continues to hold when trucks cross, winds rise, and inspectors check the bolts.
Tronto insists that "assuming responsibility is not yet the same as doing the actual work of care." Competence is about execution: working code that does what it promised, audited, explainable, and safe-to-fail. And crucially: "to be competent to care is not simply a technical issue, but a moral one." A system that ships broken care with good intentions has failed morally, not just technically.
The illustration's framing: We check the process — not "just trust us," but with transparency and fast operational feedback on how care is delivered.
Definition
- Safety is a property of practice. Competence is demonstrated in operation, not assumed from design.
- Proof before promotion. Features graduate only after shadowing → canary → general with guardrails.
- Observability over opaqueness. A "show your work" approach with traces, datasets, and explainable summaries tied to decisions promotes observability. (Observability means the system's reasoning is inspectable, not that the operator sees individual private interactions.)
- Least power. The simplest mechanism is used to meet the need; complexity grows attack surface.
- Fail safely. When evidence is weak or a component drifts, the system narrows scope, hands off, or pauses instead of bluffing.
Why it matters
Competence matters because public promises fail unless execution is visible, testable, and reversible. Pack 2 binds commitments; Pack 3 asks whether the system actually delivers on them. For Civic AI, safety is something people should be able to inspect in operation, not infer from vendor intent. A system that ships broken care with good intentions has failed morally, not just technically.
What it looks like in practice
- Graduated release. New policies run in shadow mode, then canary for a random representative slice, then general rollout with rollback primed.
- Decision traces. Every denial, recommendation, or escalation has a trace: which rule, which sources, uncertainty score, and a receipt link.
- Guardrails as code. Rights and red lines expressed as machine-checkable rules (deny-by-default when ambiguous).
- Security as competence. An agent with filesystem or network access runs in a strict sandbox with least-privilege permissions, validated inputs, and no implicit trust of upstream content.
- Prompt injection, privilege escalation, and lateral movement are competence failures — moral responsibilities of those who build and deploy these systems, not mere technical oversights.
- Working fallbacks. If confidence drops or a dependency fails, the system uses a reversible default, routes to a human, or pauses within the promised window.
- Data minimalism. Only what the remedy needs is collected; delete on handoff; consent honoured at every stage.
- Reproducible builds. Configs are versioned; one-click replays re-create results.
From ideas to practice
- Derive specs from contracts. Convert Pack 2 engagement contracts into acceptance tests.
- Instrument for observability. Emit decision traces with links to sources and receipts (from Pack 1).
- Run shadow mode. New policy sees inputs and proposes actions but doesn't act. Compare to human/previous system.
- Canary safely. Release to a small, representative group with automatic rollback if drift exceeds bounds.
- Audit before general. Conduct independent audit of evals, logs, and guardrails; publish attested report.
- Generalize & monitor. Enable for all; watch drift monitors; keep pause wired.
- Post-incident learning. Maintain blameless reviews; fixes become tests.
Buildable tools
- Shadow/canary orchestrator with rollback switches.
- Decision trace schema. Inputs, rules fired, sources, uncertainties.
- Guardrail engine. Policy-as-code for rights/consents.
- Drift monitors. Data, performance, fairness.
- Eval registry. Versioned tests, provenance, and localized test suites.
- Replay tooling. One-click re-runs for audits, incidents, and appeals.
- Fallback router. Confidence thresholds that trigger human handoff or pause.
One case: the flood-bot
- Shadow → canary. A new "medical receipts waiver" runs in shadow for a week; then canaries to 10% of livelihood claims; rollback bound: appeals >15%.
- Observability. Every denial has a trace: which rule, which sources, uncertainty score, and a receipt link for the claimant.
- Safe fallback. When uploaded documents are unreadable or confidence drops, the bot uses a reversible default and routes the claim to a human caseworker rather than guessing.
What could go wrong
- Unsafe confidence. The system acts despite weak evidence. Fix: Confidence thresholds, fallback routing, and pause on ambiguity.
- Train/test leakage. Evals look good; reality fails. Fix: Hold-out datasets, randomized spot checks, live A/Bs with rollback.
- Opaque "black box." "Trust us" explanations. Fix: Traceable summaries + public examples; auditors can reconstruct decisions.
- Canary bias. Canary slice is unrepresentative. Fix: Stratify sampling; publish canary demographics.
Interfaces
- From Responsibility (Pack 2): specs, SLAs, brakes.
- To Responsiveness (Pack 4): competence delivers; responsiveness checks whether it worked. Incident loops and eval results feed Pack 4.
- To Solidarity (Pack 5): dependable, well-instrumented systems make cooperation and public audits credible.
- To Symbiosis (Pack 6): competence proves an agent is ready to stay local.
Public measure
Verified execution rate is the headline public measure for competence. The public question is what share of audited decisions or releases pass guardrails, include a usable trace, and stay inside release bounds. Supporting diagnostics include trace completeness, guardrail integrity, canary health, and audit overturn rate. See Measures.
A closing image: the bridge with inspection tags
Imagine a well-kept bridge with inspection tags — date, load test, next check — visible to anyone crossing. Competence is not the absence of failure; it is the presence of proof that someone checked, and will check again.