001 Notes

Clean Code Tax, Measured

0. Abstract

Clean code is not the enemy of performance. The real problem is less dramatic: some clean-code idioms are free in cold code, some are free after the compiler inlines them, and some are expensive when they stay inside a hot loop.

This benchmark suite turns that claim into paired measurements. Every row compares a clean or idiomatic shape against a lower-abstraction control that does the same logical work.

The final run used:

SettingValue
Working sets4K, 64K, 1M, 16M, 64M
Iterations9
Threads4
Pinningcore

I also kept a small smoke run as a validation path, but the results discussed below are from the full ladder.

One implementation correction happened during the final run. A partial attempt exposed pass-count accounting that was wrong for nonlinear rows, especially linear scan and ORM-style N+1 shapes. I fixed that accounting, reduced the high-rate exception throw row sizes, rebuilt, probed the corrected large cases, and reran the suite from a clean result set.

1. Why Measure This

Low-level performance work usually starts with cache lines, branch prediction, SIMD, TLBs, allocation behavior, and data layout. Those are the physics of performance.

Production code often loses time one layer above that:

  • a helper function stops inlining,
  • a virtual call hides the target,
  • a shared_ptr copy touches an atomic reference count,
  • an exception is used as a normal parse failure,
  • a builder allocates small strings on every request,
  • an ORM-style lazy lookup turns one pass into many scans,
  • tracing spans are placed around every helper.

The useful question is not whether clean code is good or bad. The useful question is narrower:

Is this clean shape still cheap in the hot path I am measuring?

The decision rule should be simple:

  1. Keep the clean version by default.
  2. Measure the hot path.
  3. If the clean version is under budget, stop.
  4. If it misses the target, replace only the measured bottleneck.

2. Benchmark Coverage

The suite currently registers 84 benchmarks under the clean_tax_ prefix.

PatternClean or idiomatic shapeControl shape
Small functionshelper per elementinline expression, noinline, function pointer
Virtual dispatchvector<unique_ptr<Shape>>tagged record, variant, SoA
Smart pointerscopied shared_ptr<T>const&, raw stable pointer, index handle
Exceptionsthrow/catch parse failureserror code, optional
std::functiontype-erased callbackdirect lambda, function pointer
RAII hot loopper-iteration string or unique_ptrreused string, stack buffer, stack object
Validationrevalidate internallyboundary-only trusted type
Builderfluent request builderaggregate init, template request, reserved fields
DTO mappingrow to entity to service DTO to API DTOdirect row to API DTO
Loggingeager formatting, sync sinklazy guard, async queue, no log
Tracingspan around every helperboundary span, no span
Reactive pipelinemap/filter/reduce intermediatesfused imperative loop
Service boundaryqueue/serialization stagedirect function call
Connection poolsetup object per requestpooled reusable object
Cache layerhit/miss/single-flight simulationdirect compute
String concatrepeated +=reserve, join buffer
Map choicestd::map, unordered_mapflat vector, linear scan
Iterator/indexrange-forindex loop, raw pointer loop
Stream I/Oostringstreamsnprintf, manual append
Reflectionstring field lookupdirect field access
JSON parseDOM-like allocated nodesstreaming fixed-schema scan
Allocation policyheap object per itemarena, pool reuse
Atomicsglobal atomic per itemlocal accumulation and flush
ORM simulationlazy N+1 scaneager join, raw joined scan
SerializationJSON textpacked binary

3. Harness Shape

Each benchmark prepares data outside the timed region, then returns a measured function that does the hot work and produces a checksum. The checksum prevents the optimizer from deleting the loop.

BenchmarkInstance make_instance(const std::size_t elements,
                                const std::size_t actual_size_bytes,
                                const std::uint64_t target_ops,
                                std::function<std::uint64_t(std::size_t)> run,
                                const std::size_t threads = 1) {
    BenchmarkInstance instance;
    instance.actual_size_bytes = actual_size_bytes;
    instance.elements = elements;
    instance.threads_used = threads;
    instance.bytes_per_pass = std::max<std::size_t>(1, elements);
    instance.target_bytes_per_trial = target_ops;
    instance.run = std::move(run);
    return instance;
}

Heap-allocation rows also need a small compiler barrier. Without it, an optimizing compiler can sometimes prove that a temporary allocation has no observable identity and erase more work than intended.

CACHEBENCH_CLEAN_NOINLINE void escape_pointer(const void* pointer) {
#if defined(__GNUC__) || defined(__clang__)
    asm volatile("" : : "g"(pointer) : "memory");
#else
    const volatile void* escaped = pointer;
    static_cast<void>(escaped);
#endif
}

4. Verdict Rule

The benchmark runner compares named clean rows against named control rows. A tax is only confirmed when the clean row is at least 10 percent slower and both rows have coefficient of variation at or below 5 percent.

def verdict(clean: Point, control: Point) -> tuple[str, str, float]:
    clean_ns = clean.ns_per_op
    control_ns = control.ns_per_op

    if clean_ns > 0 and control_ns > 0 and math.isfinite(clean_ns) and math.isfinite(control_ns):
        ratio = clean_ns / control_ns
        timing_ratio = max(ratio, 1.0 / ratio)
    else:
        ratio = math.nan
        timing_ratio = 0.0

    if clean.cv > 0.05 or control.cv > 0.05:
        return "INCONCLUSIVE", "noisy", timing_ratio

    if clean_ns <= 0 or control_ns <= 0 or not math.isfinite(clean_ns) or not math.isfinite(control_ns):
        return "INCONCLUSIVE", "invalid", 0.0

    if ratio >= 1.10:
        return "CONFIRMED_TAX", "control", ratio
    if ratio <= 0.90:
        return "CLEAN_WINS", "clean", 1.0 / ratio
    if 0.95 <= ratio <= 1.05:
        return "NO_MEASURABLE_TAX", "tie", max(ratio, 1.0 / ratio)
    return "INCONCLUSIVE", "close", max(ratio, 1.0 / ratio)

Noisy rows are still retained in the report, but they are not promoted as evidence. That matters because a benchmark can show an interesting ratio and still be too unstable for a causal claim.

5. Code Shapes Tested

5.1 Small Functions

Expectation: helper functions should be free when inlined, but costly when the compiler cannot inline them.

inline std::uint64_t inline_transform(const std::uint64_t value) {
    return (value * 3ULL + 1ULL) ^ (value >> 7U);
}

CACHEBENCH_CLEAN_NOINLINE std::uint64_t noinline_transform(const std::uint64_t value) {
    return inline_transform(value);
}

std::uint64_t function_pointer_transform(const std::uint64_t value) {
    return inline_transform(value);
}

The same loop body is run with four call shapes: inline expression, inline helper, noinline helper, and function pointer.

for (const std::uint64_t value : *data) {
    if (mode == "inline_expr") {
        sum += (value * 3ULL + 1ULL) ^ (value >> 7U);
    } else if (mode == "inline_helper") {
        sum += inline_transform(value);
    } else if (mode == "noinline_helper") {
        sum += noinline_transform(value);
    } else {
        sum += fn_ptr(value);
    }
}

5.2 Virtual Dispatch

Expectation: per-object polymorphism should lose when a hot loop calls through a base pointer. The cost is not only the virtual call. It is also the heap object layout and the lost chance to run tight type-specific loops.

struct ShapeBase {
    virtual ~ShapeBase() = default;
    virtual std::uint64_t area() const = 0;
};

struct CircleShape final : ShapeBase {
    explicit CircleShape(std::uint64_t radius_value) : radius(radius_value) {}
    std::uint64_t area() const override { return radius * radius * 3ULL; }
    std::uint64_t radius;
};

struct SquareShape final : ShapeBase {
    explicit SquareShape(std::uint64_t side_value) : side(side_value) {}
    std::uint64_t area() const override { return side * side; }
    std::uint64_t side;
};
for (const auto& shape : *shapes) {
    sum += shape->area();
}

The SoA control buckets by type, then runs tight loops.

for (const std::uint64_t radius : *circles) {
    sum += radius * radius * 3ULL;
}
for (const std::uint64_t side : *squares) {
    sum += side * side;
}

5.3 Smart Pointers

Expectation: unique_ptr ownership is usually free when optimized, but copying shared_ptr in a hot loop should cost because each copy touches the reference count.

if (mode == "shared_ptr_copy") {
    std::shared_ptr<Config> copy = shared_config;
    sum += use_config_ref(*copy, value);
}

Control rows avoid ownership changes inside the loop.

if (mode == "const_ref") {
    sum += use_config_ref(config, value);
} else if (mode == "raw_pointer") {
    sum += raw->a * value + raw->b;
} else {
    const std::size_t handle = 0;
    const Config& selected = (*pool)[handle];
    sum += selected.a * value + selected.b;
}

5.4 Exceptions

Expectation: no-throw exception-shaped code may be fine, but throwing as a normal failure path should become very expensive as failure rate rises.

CACHEBENCH_CLEAN_NOINLINE std::uint64_t parse_or_throw(
    const std::uint64_t value,
    const std::size_t index,
    const std::size_t period) {
    if (period != 0U && index % period == 0U) {
        throw std::runtime_error("clean_tax");
    }
    return value + 11ULL;
}
TinyResult parse_result(const std::uint64_t value,
                        const std::size_t index,
                        const std::size_t period) {
    if (period != 0U && index % period == 0U) {
        return TinyResult{false, 0};
    }
    return TinyResult{true, value + 11ULL};
}

The benchmark sweeps no throw, 0.001 percent, 1 percent, and 50 percent failure rates.

if (mode.find("exception") == 0U) {
    try {
        sum += parse_or_throw(value, index, period);
    } catch (const std::exception&) {
        sum += 7ULL;
    }
} else {
    const TinyResult result = parse_result(value, index, period);
    sum += result.ok ? result.value : 7ULL;
}

5.5 std::function

Expectation: std::function should cost in a tight callback loop because it type-erases the callable and blocks direct inlining.

std::function<std::uint64_t(std::uint64_t)> erased = [](std::uint64_t value) {
    return inline_transform(value);
};
if (mode == "std_function") {
    sum += erased(value);
} else if (mode == "function_pointer") {
    sum += fn_ptr(value);
} else {
    sum += direct_lambda(value);
}

5.6 RAII In A Hot Loop

Expectation: RAII is not the problem. Repeated hidden allocation is the problem. Creating a std::string or unique_ptr inside every iteration should lose to reused storage or stack storage.

if (mode == "string_per_iter") {
    std::string key = std::string("prefix_") + std::to_string(index + pass);
    sum += key.size();
}
else if (mode == "string_reuse") {
    reused.clear();
    reused.append("prefix_");
    reused.append(std::to_string(index + pass));
    sum += reused.size();
}
else if (mode == "unique_ptr_per_iter") {
    auto value = std::make_unique<std::uint64_t>(index + pass);
    escape_pointer(value.get());
    sum += *value;
}

else if (mode == "stack_object") {
    std::uint64_t value = index + pass;
    escape_pointer(&value);
    sum += value;
}

5.7 Defensive Validation

Expectation: validation at boundaries is good. Revalidating the same invariant inside the inner loop adds branches and instructions without adding safety.

if (mode == "revalidate") {
    if (!valid_trade(trade)) {
        throw std::runtime_error("invalid trade");
    }
    if (trade.quantity == 0 || trade.price == 0) {
        throw std::runtime_error("invalid trade");
    }
}
sum += trade.quantity * trade.price + pass;

5.8 Builder Pattern

Expectation: fluent builders can be good API design, but they are not free when used at high construction rates.

Request request = RequestBuilder()
                      .url("https://svc/items/" + std::to_string(index))
                      .method("GET")
                      .header("Accept", "application/json")
                      .header("Trace", "on")
                      .timeout(30)
                      .build();

Aggregate initialization and reserved construction are the controls.

Request request{"https://svc/items/" + std::to_string(index),
                "GET",
                {{"Accept", "application/json"}, {"Trace", "on"}},
                30};
Request request;
request.headers.reserve(2);
request.url = "https://svc/items/";
request.url += std::to_string(index);
request.method = "GET";
request.headers.emplace_back("Accept", "application/json");
request.headers.emplace_back("Trace", "on");
request.timeout_ms = 30;

5.9 DTO Mapping

Expectation: one mapping layer is often acceptable. Repeated mapping layers mean repeated copies and allocations.

Entity entity{row.id, row.quantity, row.price, row.symbol};
ServiceDto service{entity.id, entity.quantity * entity.price, entity.symbol};
ApiDto api{service.id, service.notional, service.symbol};
sum += api.id + api.notional + api.symbol.size();
ApiDto api{row.id, row.quantity * row.price, row.symbol};
sum += api.id + api.notional + api.symbol.size();

5.10 Logging And Tracing

Expectation: logging is cheap only when formatting is avoided and I/O is not on the hot path. Tracing at boundaries is usually fine; tracing every helper can dominate small operations.

if (mode == "eager_disabled") {
    const std::string line = format_log_line(index, pass);
    if (debug_enabled) {
        sum += line.size();
    }
}
else if (mode == "lazy_guard_disabled") {
    if (debug_enabled) {
        sum += format_log_line(index, pass).size();
    }
}
if (mode == "boundary_span") {
    sum ^= span_marker(pass);
}

for (std::size_t index = 0; index < data->size(); ++index) {
    if (mode == "span_every_helper") {
        sum ^= span_marker(index);
    }
    sum += inline_transform((*data)[index]);
}

5.11 Reactive Pipeline

Expectation: composing map/filter/reduce through intermediate containers should cost more than a fused loop when the work is CPU-bound.

std::vector<std::uint64_t> mapped;
mapped.reserve(data->size());
for (const std::uint64_t value : *data) {
    mapped.push_back(value * 3ULL + 1ULL);
}

std::vector<std::uint64_t> filtered;
filtered.reserve(mapped.size() / 2U);
for (const std::uint64_t value : mapped) {
    if ((value & 1ULL) == 0ULL) {
        filtered.push_back(value);
    }
}

for (const std::uint64_t value : filtered) {
    sum += value;
}
for (const std::uint64_t value : *data) {
    const std::uint64_t mapped = value * 3ULL + 1ULL;
    if ((mapped & 1ULL) == 0ULL) {
        sum += mapped;
    }
}

5.12 Service Boundary

Expectation: splitting a direct call into staged work should add overhead. If serialization is included, the cost should be much larger.

if (mode == "direct") {
    sum += direct_service_call(value);
}

else if (mode == "queue_stage") {
    queue.push_back(value);
    const std::uint64_t staged = queue.front();
    queue.pop_front();
    sum += direct_service_call(staged);
}
else {
    const int written = std::snprintf(buffer.data(), buffer.size(), "%llu",
                                      static_cast<unsigned long long>(value));
    const std::uint64_t parsed =
        static_cast<std::uint64_t>(std::strtoull(buffer.data(), nullptr, 10));
    sum += direct_service_call(parsed) + static_cast<std::uint64_t>(std::max(written, 0));
}

5.13 Connection Pooling

Expectation: setup per request should lose to pooled reuse.

if (mode == "new_per_request") {
    SimConnection connection;
    sum += connection.query((*data)[index]);
}

else {
    const SimConnection& connection = (*pool)[index & (pool->size() - 1U)];
    sum += connection.query((*data)[index]);
}

5.14 Cache Layer

Expectation: cache hits can beat recompute, but miss paths are still on the critical path. A cache should not be judged only by average hit rate.

if (mode == "direct_compute") {
    sum += expensive_compute(key);
}

else {
    const auto found = cache->find(key);
    if (found != cache->end()) {
        sum += found->second;
    } else {
        const std::uint64_t computed = expensive_compute(key);
        (*cache)[key] = computed;
        sum += computed;
    }
}

5.15 String Concatenation

Expectation: repeated += can be fine for small strings, but reserve/join should win when reallocation pressure grows.

std::string result;
if (mode == "reserve" || mode == "join_buffer") {
    result.reserve(pieces->size() * 6U);
}

if (mode == "join_buffer") {
    std::vector<std::string_view> views;
    views.reserve(pieces->size());
    for (const std::string& piece : *pieces) {
        views.push_back(piece);
    }
    for (std::string_view piece : views) {
        result.append(piece.data(), piece.size());
    }
} else {
    for (const std::string& piece : *pieces) {
        result += piece;
    }
}

5.16 Map Choice

Expectation: the clean default container may not be the fastest. The winner depends on key count and access shape.

if (mode == "std_map") {
    sum += tree->find(key)->second;
} else if (mode == "unordered_map") {
    sum += hash->find(key)->second;
} else if (mode == "flat_vector") {
    const auto found = std::lower_bound(
        shared_items->begin(), shared_items->end(), key,
        [](const auto& item, const std::uint64_t query) { return item.first < query; });
    sum += found->second;
} else {
    for (const auto& item : *shared_items) {
        if (item.first == key) {
            sum += item.second;
            break;
        }
    }
}

5.17 Iterator, Index, And Raw Pointer Loops

Expectation: range-for over a vector should usually compile as well as an index loop. This is a negative control: clean code should often be free.

if (mode == "range_for") {
    for (const std::uint64_t value : *data) {
        sum += inline_transform(value);
    }
} else if (mode == "index_loop") {
    for (std::size_t index = 0; index < data->size(); ++index) {
        sum += inline_transform((*data)[index]);
    }
} else {
    const std::uint64_t* ptr = data->data();
    const std::uint64_t* end = ptr + data->size();
    while (ptr != end) {
        sum += inline_transform(*ptr);
        ++ptr;
    }
}

5.18 Stream I/O

Expectation: ostringstream is convenient, but it should be slower than simpler formatting for small repeated formatting tasks.

if (mode == "ostringstream") {
    std::ostringstream stream;
    stream << "id=" << index << ",value=" << (index + pass);
    sum += stream.str().size();
} else if (mode == "snprintf") {
    const int written =
        std::snprintf(buffer.data(), buffer.size(), "id=%zu,value=%zu", index, index + pass);
    sum += static_cast<std::uint64_t>(std::max(written, 0));
} else {
    std::string out;
    out.reserve(32);
    out.append("id=");
    out.append(std::to_string(index));
    out.append(",value=");
    out.append(std::to_string(index + pass));
    sum += out.size();
}

5.19 Reflection-Style Access

Expectation: string-based field lookup should lose to direct field access in a hot loop.

std::uint64_t read_reflected_field(const ReflectRecord& record, const std::string_view name) {
    if (name == "a") return record.a;
    if (name == "b") return record.b;
    if (name == "c") return record.c;
    return record.d;
}
if (mode == "string_lookup") {
    sum += read_reflected_field(record, kNames[index & 3U]);
} else if (mode == "table_dispatch") {
    switch (index & 3U) {
        case 0: sum += record.a; break;
        case 1: sum += record.b; break;
        case 2: sum += record.c; break;
        default: sum += record.d; break;
    }
} else {
    sum += record.a + record.b + record.c + record.d;
}

5.20 JSON-Shaped Parsing

Expectation: DOM-like allocation should lose to streaming extraction when the schema is known.

std::uint64_t parse_streaming_digits(const std::string& text) {
    std::uint64_t sum = 0;
    std::uint64_t current = 0;
    bool in_number = false;
    for (const char ch : text) {
        if (ch >= '0' && ch <= '9') {
            current = current * 10ULL + static_cast<std::uint64_t>(ch - '0');
            in_number = true;
        } else if (in_number) {
            sum += current;
            current = 0;
            in_number = false;
        }
    }
    return in_number ? sum + current : sum;
}
if (mode == "dom_allocated") {
    std::vector<JsonPair> pairs;
    pairs.reserve(2);
    pairs.push_back(JsonPair{"id", parse_streaming_digits(doc)});
    pairs.push_back(JsonPair{"qty", doc.size()});
    for (const JsonPair& pair : pairs) {
        sum += pair.value + pair.key.size();
    }
} else {
    sum += parse_streaming_digits(doc);
}

5.21 Allocation Policy

Expectation: heap allocation per item should lose to arena or pool reuse.

if (mode == "heap_temp") {
    auto object = std::make_unique<TinyObject>(TinyObject{index, pass});
    escape_pointer(object.get());
    sum += object->a + object->b;
} else {
    TinyObject& object = (*pool)[index];
    object.a = index;
    object.b = pass;
    sum += object.a + object.b;
}

5.22 Atomics

Expectation: a global atomic per item should lose to local accumulation with one flush, especially with more threads and larger workloads.

if (mode == "global_atomic") {
    for (std::size_t index = begin; index < end; ++index) {
        counter.fetch_add(index + 1U, std::memory_order_relaxed);
    }
} else {
    std::uint64_t local = 0;
    for (std::size_t index = begin; index < end; ++index) {
        local += index + 1U;
    }
    counter.fetch_add(local, std::memory_order_relaxed);
}

5.23 ORM-Style Lazy Loading

Expectation: lazy N+1 shape should be much worse than eager joined traversal.

if (mode == "lazy_n_plus_one") {
    for (const ParentRow& parent : *parents) {
        sum += parent.id;
        for (const ChildRow& child : *children) {
            if (child.parent_id == parent.id) {
                sum += child.value;
            }
        }
    }
}
else if (mode == "eager_join") {
    std::size_t child_index = 0;
    for (const ParentRow& parent : *parents) {
        sum += parent.id;
        for (std::uint32_t count = 0; count < 5U; ++count) {
            sum += (*children)[child_index++].value;
        }
    }
}

5.24 Serialization

Expectation: JSON text should lose to packed binary for repeated encode/decode work.

if (mode == "json_text") {
    const int written =
        std::snprintf(buffer.data(), buffer.size(), "{\"id\":%u,\"qty\":%u,\"price\":%llu}",
                      payload.id,
                      payload.qty,
                      static_cast<unsigned long long>(payload.price));
    sum += parse_streaming_digits(std::string(buffer.data(),
        static_cast<std::size_t>(std::max(written, 0))));
} else {
    std::array<unsigned char, sizeof(BinaryPayload)> bytes{};
    std::memcpy(bytes.data(), &payload, sizeof(payload));
    BinaryPayload decoded{};
    std::memcpy(&decoded, bytes.data(), sizeof(decoded));
    sum += decoded.id + decoded.qty + decoded.price;
}

6. Expectations

I expected three classes of results.

First, I expected obvious hot-path abstraction costs:

  • virtual dispatch should lose to SoA,
  • copied shared_ptr should lose to references or handles,
  • throwing exceptions frequently should lose to result values,
  • eager logging should lose to guarded logging,
  • per-item heap allocation should lose to arena or pool reuse,
  • DOM-like parsing should lose to streaming parsing,
  • lazy ORM-style scans should lose to eager joined scans.

Second, I expected some clean code to be free:

  • range-for over vector should be near an index or raw-pointer loop,
  • boundary-only tracing should be close to no tracing,
  • a no-throw exception-shaped success path should not automatically be bad.

Third, I expected some rows to remain sensitive or inconclusive even in the full run:

  • atomics where thread scheduling noise competes with the measured operation,
  • cache rows where the miss/hit setup dominates differently by size,
  • service/serialization simulations with high constant overhead,
  • map choices where the answer depends heavily on key count.

7. Final Results

The canonical run completed after the nonlinear accounting fix and produced this top-level result:

VerdictRows
CONFIRMED_TAX95
NO_MEASURABLE_TAX4
CLEAN_WINS12
INCONCLUSIVE154

The high number of inconclusive rows is not a failure of the suite. It is a consequence of the conservative rule: any row with CV above 5 percent is kept in the data but not promoted as causal evidence. The final result has 265 contrast rows across five sizes; noisy rows are visible, but they do not get counted as confirmed.

Several expectations were supported by stable final rows:

ContrastSizeFinal resultInterpretation
small_noinline_helper64Mcontrol faster by 2.586xforced calls inside a hot loop were not free
virtual_vs_soa1Mcontrol faster by 7.916xvirtual object layout lost badly to type-bucketed storage
shared_ptr_vs_ref64Mcontrol faster by 4.371xcopying shared_ptr in the loop was expensive
exception_1pct_vs_error64Mcontrol faster by 2.197xroutine failure should not throw
exception_50pct_vs_error64Mcontrol faster by 80.602xfrequent exceptions were catastrophic
std_function_vs_lambda64Mcontrol faster by 1.928xtype-erased callbacks blocked the cheap direct path
raii_string_vs_reuse64Kcontrol faster by 1.123xper-iteration string construction had a visible tax
validation_recheck64Mcontrol faster by 1.376xrepeated internal validation added hot-path cost
builder_vs_template64Mcontrol faster by 3.040xa prebuilt template avoided much of the builder construction cost
dto_layered_vs_direct1Mcontrol faster by 3.713xlayered DTO copies mattered
logging_eager_vs_none64Mcontrol faster by 48.696xformatting logs that are not emitted is very expensive
tracing_every_vs_boundary64Mcontrol faster by 30.099xtracing every helper dominated the tiny operation
reactive_pipeline_vs_fused1Mcontrol faster by 27.659xintermediate pipeline containers were costly
service_serialized_vs_direct16Mcontrol faster by 126.064xserialization-shaped service boundaries dominated direct work
connection_new_vs_pool64Mcontrol faster by 9.237xsetup per request lost to pooled reuse
json_dom_vs_stream64Mcontrol faster by 2.534xDOM-like allocation lost to streaming extraction
alloc_heap_vs_pool64Mcontrol faster by 6.263xheap allocation per item lost to pool reuse
orm_lazy_vs_raw64Mcontrol faster by 5713.252xthe N+1 shape was predictably bad

Some evidence was contrary or at least more interesting than expected:

ContrastSizeFinal resultWhat it means
exception_no_throw_vs_error4K, 64KNO_MEASURABLE_TAXsupports the zero-cost model when exceptions are not thrown
exception_no_throw_vs_error64Mclean row faster by 1.403xno-throw exception-shaped code was not the slow path here
variant_vs_tagged64Mclean row faster by 1.112xvariant did not behave as a simple tax in this shape
unordered_map_vs_flat64Mclean row faster by 7.619xthis query shape favored hash lookup over sorted-vector binary search
linear_scan_vs_hash64Mclean row faster by 2263.460xtiny-N linear scan does not generalize to large-N lookup
alloc_heap_vs_arena64Mclean row faster by 1.196xthis arena row allocates a fresh arena per pass, so pool reuse is the cleaner control

Those contrary rows are useful because they stop the result from becoming an ideology. Clean-code tax is not universal. The actual code shape and input size matter.

Several rows showed large timing ratios but remained INCONCLUSIVE because one side was noisy. Examples include:

  • serialization_json_vs_binary, with 274x to 426x timing ratios but noisy binary-control rows,
  • atomic_global_vs_local at 64M, with a 16.963x ratio but noisy local-flush timing,
  • reflection_string_vs_direct, where ratios were visible but string-lookup timing stayed noisy,
  • some cache rows where setup and lookup costs had high variance.

The runner keeps those ratios visible, but it does not treat them as confirmed evidence.

8. What The Evidence Supports

The evidence strongly supports the main point: there are real costs hidden inside clean-looking code when those shapes sit in the hot path. Virtual calls, shared_ptr copies, frequent exceptions, eager logging, tracing every helper, layered DTO mapping, builders, and ORM-style lazy scans all showed the expected tax in the final run.

It also supports the more important engineering point: clean code is not automatically slow. The no-throw exception row was a tie at smaller sizes and faster at 64M; variant beat the tagged control in several sizes; and string reserve was a no-measurable-tax row at 4K and 64K. Some negative controls, such as iterator comparisons, still need cleaner timing because the ratios were near 1 but the CV gate marked them inconclusive.

So the evidence supports a decision framework, not a slogan:

  1. Keep clean code by default.
  2. Measure the hot path.
  3. If the clean version is under budget, stop.
  4. If it misses the target, replace only the measured bottleneck.

9. What Not To Claim Yet

The final run is the right artifact for this suite, but it is still a benchmark suite, not a production trace. I would not claim that every large timing ratio is a final truth unless the row passed the CV gate. The report deliberately keeps large but noisy ratios as INCONCLUSIVE.

I also would not claim that the ORM, service, connection, cache, JSON, and serialization rows are replacements for full application benchmarks. They isolate the shape, not a real database, real RPC stack, real TLS path, real garbage collector, or real web framework.

The next useful step is not to add more rows. It is to rerun selected noisy contrasts with higher iteration counts, fixed-frequency controls, or perf sidecars. That is especially true for serialization, atomics, reflection, and cache miss paths.

10. Practical Rule

This benchmark set adds a way to answer questions application developers usually argue about:

  • Is this abstraction actually expensive?
  • Does the compiler erase this helper?
  • Is the builder cost visible at high request rates?
  • Is the clean OOP layout losing to data-oriented storage?
  • Is this logging or tracing pattern still expensive when disabled?

The result is not a ban on clean code. It is a way to protect clean code from superstition. If a clean pattern is free, keep it. If it is expensive only in the hot 5 percent, isolate that hot path and leave the rest of the program readable.

Comments