Rule Storage Formats: JSON vs YAML vs Database

Where you keep zoning rules quietly determines how easily planners edit them, how safely engineers version them, and how fast the engine evaluates them across a jurisdiction. This module of Rule Engine Design for Zoning & Setback Automation compares three durable options for persisting setback, height, and overlay constraints: flat JSON files, human-friendly YAML files, and a relational or PostGIS-backed database. The goal is a storage layer that keeps rule definitions decoupled from execution logic while matching your team’s editing habits, review process, and dataset scale.

Prerequisites #

Before committing to a storage layer, make sure the surrounding pipeline can support any of the three options:

• A normalized rule schema with stable field names such as rule_id, zone_class, setback_m, precedence, and effective_date.
• A validation library in place: jsonschema for JSON, pydantic or cerberus for YAML-derived dicts, or database constraints for the relational path.
• Python 3.10+ with pyyaml (use yaml.safe_load), plus sqlalchemy and a PostGIS-enabled PostgreSQL instance if you intend to test the database route.
• A version-control workflow (Git) for file-based formats, or migration tooling such as Alembic for the database.
• A rule count estimate and expected edit frequency, because scale and concurrency are the two criteria that most often decide the outcome.

Comparison Criteria #

The three formats are not ranked absolutely; each optimizes a different constraint. The table below scores them across the six criteria that matter most when rules must survive council amendments, staff turnover, and growing parcel volumes.

The pattern is consistent: file formats win on transparency and review ergonomics, while the database wins on query power, concurrency, and scale. YAML edges out JSON for anything a non-developer will touch because comments let you annotate the statutory citation next to each threshold.

Core Workflow #

Whichever format you choose, the engine should never evaluate rules straight off disk or the wire. Load once, validate, and convert to a single in-memory intermediate representation so downstream code stays storage-agnostic. The following loader normalizes YAML and JSON into identical dictionaries and enforces a schema before any rule reaches the evaluator.

import json
from pathlib import Path
import yaml  # PyYAML
from pydantic import BaseModel, Field, ValidationError

class ZoningRule(BaseModel):
    rule_id: str
    zone_class: str
    setback_m: float = Field(ge=0)   # linear metres, projected CRS assumed
    precedence: int = 0
    effective_date: str

def load_rules(path: str) -> list[ZoningRule]:
    text = Path(path).read_text(encoding="utf-8")
    # safe_load blocks arbitrary object construction; json.loads for .json
    raw = yaml.safe_load(text) if path.endswith((".yaml", ".yml")) else json.loads(text)
    rules = []
    for entry in raw["rules"]:
        try:
            rules.append(ZoningRule(**entry))   # validation happens here
        except ValidationError as exc:
            raise ValueError(f"Invalid rule {entry.get('rule_id')}: {exc}") from exc
    return rules

Because both branches emit the same ZoningRule objects, you can migrate from files to a database later without rewriting the evaluator. The database path simply swaps the file read for a query and passes each row through the same model.

Implementation Patterns #

For file-based storage, keep rules in one document per jurisdiction rather than one giant file, so diffs stay scoped and merge conflicts stay rare. Store distances in metric units to match the projected CRS your spatial engine uses, and never rely on geographic degrees for setback thresholds. The database pattern below reads active rules for a zone and joins them directly to parcel geometry, which is where relational storage decisively outperforms flat files.

import geopandas as gpd
from sqlalchemy import create_engine

engine = create_engine("postgresql://gis:secret@localhost:5432/compliance")

def load_active_rules(zone_class: str, as_of: str) -> gpd.GeoDataFrame:
    # Indexed filter on zone_class and effective_date runs in the database,
    # so only relevant rows cross the wire even for millions of rules.
    sql = """
        SELECT rule_id, zone_class, setback_m, precedence, geom
        FROM zoning_rules
        WHERE zone_class = %(zone)s AND effective_date <= %(as_of)s
        ORDER BY precedence DESC
    """
    gdf = gpd.read_postgis(sql, engine, geom_col="geom",
                           params={"zone": zone_class, "as_of": as_of})
    return gdf.to_crs("EPSG:26910")  # project to a metric CRS before any distance op

Relational storage also enforces referential integrity: a foreign key from zoning_rules.overlay_id to an overlay table prevents dangling references a hand-edited YAML file would accept. The transactional guarantees pair naturally with Async Rule Execution Patterns, where workers read a consistent rule snapshot while edits land safely in another transaction.

Edge Cases & Data Repair #

Storage format changes the failure modes you guard against. YAML silently coerces unquoted values, so a zone code like NO parses as boolean False and a bare 1.10 loses its trailing zero — quote categorical fields and validate types on load. JSON has no comment channel, so provenance notes must live in a dedicated _meta field. Databases invert the risk: a rule can be deleted without an obvious diff, so add an append-only history table to preserve prior versions.

def normalize_types(entry: dict) -> dict:
    # Guard against YAML's implicit typing of codes and numbers.
    entry["zone_class"] = str(entry["zone_class"])          # 'NO' must stay a string
    entry["setback_m"] = round(float(entry["setback_m"]), 3)  # stable numeric precision
    if entry.get("precedence") is None:
        entry["precedence"] = 0                             # explicit default beats null
    return entry

For irregular parcel geometries feeding the buffer stage, the same discipline that governs Dynamic Setback Buffer Generation applies here: repair geometry before evaluation and never let a malformed rule row reach the buffer routine.

Audit Logging & Provenance #

Regardless of format, every evaluation must be reproducible from recorded provenance. Log the storage source or database snapshot identifier, the rule version or commit hash, the projected CRS in use, and the schema version the loader validated against. File formats make this nearly free: pin the Git commit that supplied the rules. Databases need explicit effort — capture the transaction timestamp and the effective_date filter so a later audit can reconstruct exactly which rows were active. Store these fields alongside the compliance result, not in a separate log that can drift out of sync.

Troubleshooting #

• Distorted setbacks after a format switch: the new source stored distances in feet while the engine assumed metres. Standardize units in the schema and validate ranges on load.
• YAML edit silently ignored: indentation placed the key at the wrong nesting level. Run yaml.safe_load in a pre-commit check and assert the expected top-level keys exist.
• Database rules stale in cache: workers cached a rule snapshot and missed an amendment. Version the snapshot and invalidate on effective_date changes rather than caching indefinitely.
• Noisy JSON diffs on review: key reordering by a formatter obscures the real change. Sort keys deterministically on write so diffs show only substantive edits.

Recommendation #

Choose by scenario rather than preference. If planners edit rules directly and the set stays in the low thousands, use YAML files in Git — readable diffs and inline citations make council amendments reviewable, and the migration guide for moving zoning rules from spreadsheets to YAML shows how to get there cleanly. If rules are consumed primarily by machines and you want the strongest off-the-shelf validation, use JSON with JSON Schema. If you must join rules to hundreds of thousands of parcels, support concurrent editors, or query by spatial predicate, use a relational or PostGIS database with an append-only history. Many mature teams run a hybrid: author in YAML for reviewability, then compile validated rules into the database as the runtime source of truth.

Conclusion #

Rule storage is an architectural decision, not a file-format detail. YAML optimizes for the humans who amend ordinances, JSON optimizes for schema-driven interchange, and a database optimizes for query power, concurrency, and scale. By loading every format through one validating loader into a shared intermediate representation, you keep the evaluation core stable while retaining the freedom to migrate as your jurisdiction and parcel counts grow.