001 Notes
Clean Code Tax, Measured
0. Abstract
| Setting | Value |
|---|---|
| Working sets | 4K, 64K, 1M, 16M, 64M |
| Iterations | 9 |
| Threads | 4 |
| Pinning | core |
1. Why Measure This
Is this clean shape still cheap in the hot path I am measuring?
2. Benchmark Coverage
| Pattern | Clean or idiomatic shape | Control shape |
|---|---|---|
| Small functions | helper per element | inline expression, noinline, function pointer |
| Virtual dispatch | vector<unique_ptr<Shape>> | tagged record, variant, SoA |
| Smart pointers | copied shared_ptr<T> | const&, raw stable pointer, index handle |
| Exceptions | throw/catch parse failures | error code, optional |
std::function | type-erased callback | direct lambda, function pointer |
| RAII hot loop | per-iteration string or unique_ptr | reused string, stack buffer, stack object |
| Validation | revalidate internally | boundary-only trusted type |
| Builder | fluent request builder | aggregate init, template request, reserved fields |
| DTO mapping | row to entity to service DTO to API DTO | direct row to API DTO |
| Logging | eager formatting, sync sink | lazy guard, async queue, no log |
| Tracing | span around every helper | boundary span, no span |
| Reactive pipeline | map/filter/reduce intermediates | fused imperative loop |
| Service boundary | queue/serialization stage | direct function call |
| Connection pool | setup object per request | pooled reusable object |
| Cache layer | hit/miss/single-flight simulation | direct compute |
| String concat | repeated += | reserve, join buffer |
| Map choice | std::map, unordered_map | flat vector, linear scan |
| Iterator/index | range-for | index loop, raw pointer loop |
| Stream I/O | ostringstream | snprintf, manual append |
| Reflection | string field lookup | direct field access |
| JSON parse | DOM-like allocated nodes | streaming fixed-schema scan |
| Allocation policy | heap object per item | arena, pool reuse |
| Atomics | global atomic per item | local accumulation and flush |
| ORM simulation | lazy N+1 scan | eager join, raw joined scan |
| Serialization | JSON text | packed binary |
3. Harness Shape
BenchmarkInstance make_instance(const std::size_t elements,
const std::size_t actual_size_bytes,
const std::uint64_t target_ops,
std::function<std::uint64_t(std::size_t)> run,
const std::size_t threads = 1) {
BenchmarkInstance instance;
instance.actual_size_bytes = actual_size_bytes;
instance.elements = elements;
instance.threads_used = threads;
instance.bytes_per_pass = std::max<std::size_t>(1, elements);
instance.target_bytes_per_trial = target_ops;
instance.run = std::move(run);
return instance;
}
CACHEBENCH_CLEAN_NOINLINE void escape_pointer(const void* pointer) {
#if defined(__GNUC__) || defined(__clang__)
asm volatile("" : : "g"(pointer) : "memory");
#else
const volatile void* escaped = pointer;
static_cast<void>(escaped);
#endif
}
4. Verdict Rule
def verdict(clean: Point, control: Point) -> tuple[str, str, float]:
clean_ns = clean.ns_per_op
control_ns = control.ns_per_op
if clean_ns > 0 and control_ns > 0 and math.isfinite(clean_ns) and math.isfinite(control_ns):
ratio = clean_ns / control_ns
timing_ratio = max(ratio, 1.0 / ratio)
else:
ratio = math.nan
timing_ratio = 0.0
if clean.cv > 0.05 or control.cv > 0.05:
return "INCONCLUSIVE", "noisy", timing_ratio
if clean_ns <= 0 or control_ns <= 0 or not math.isfinite(clean_ns) or not math.isfinite(control_ns):
return "INCONCLUSIVE", "invalid", 0.0
if ratio >= 1.10:
return "CONFIRMED_TAX", "control", ratio
if ratio <= 0.90:
return "CLEAN_WINS", "clean", 1.0 / ratio
if 0.95 <= ratio <= 1.05:
return "NO_MEASURABLE_TAX", "tie", max(ratio, 1.0 / ratio)
return "INCONCLUSIVE", "close", max(ratio, 1.0 / ratio)
5. Code Shapes Tested
5.1 Small Functions
inline std::uint64_t inline_transform(const std::uint64_t value) {
return (value * 3ULL + 1ULL) ^ (value >> 7U);
}
CACHEBENCH_CLEAN_NOINLINE std::uint64_t noinline_transform(const std::uint64_t value) {
return inline_transform(value);
}
std::uint64_t function_pointer_transform(const std::uint64_t value) {
return inline_transform(value);
}
for (const std::uint64_t value : *data) {
if (mode == "inline_expr") {
sum += (value * 3ULL + 1ULL) ^ (value >> 7U);
} else if (mode == "inline_helper") {
sum += inline_transform(value);
} else if (mode == "noinline_helper") {
sum += noinline_transform(value);
} else {
sum += fn_ptr(value);
}
}
5.2 Virtual Dispatch
struct ShapeBase {
virtual ~ShapeBase() = default;
virtual std::uint64_t area() const = 0;
};
struct CircleShape final : ShapeBase {
explicit CircleShape(std::uint64_t radius_value) : radius(radius_value) {}
std::uint64_t area() const override { return radius * radius * 3ULL; }
std::uint64_t radius;
};
struct SquareShape final : ShapeBase {
explicit SquareShape(std::uint64_t side_value) : side(side_value) {}
std::uint64_t area() const override { return side * side; }
std::uint64_t side;
};
for (const auto& shape : *shapes) {
sum += shape->area();
}
for (const std::uint64_t radius : *circles) {
sum += radius * radius * 3ULL;
}
for (const std::uint64_t side : *squares) {
sum += side * side;
}
5.3 Smart Pointers
if (mode == "shared_ptr_copy") {
std::shared_ptr<Config> copy = shared_config;
sum += use_config_ref(*copy, value);
}
if (mode == "const_ref") {
sum += use_config_ref(config, value);
} else if (mode == "raw_pointer") {
sum += raw->a * value + raw->b;
} else {
const std::size_t handle = 0;
const Config& selected = (*pool)[handle];
sum += selected.a * value + selected.b;
}
5.4 Exceptions
CACHEBENCH_CLEAN_NOINLINE std::uint64_t parse_or_throw(
const std::uint64_t value,
const std::size_t index,
const std::size_t period) {
if (period != 0U && index % period == 0U) {
throw std::runtime_error("clean_tax");
}
return value + 11ULL;
}
TinyResult parse_result(const std::uint64_t value,
const std::size_t index,
const std::size_t period) {
if (period != 0U && index % period == 0U) {
return TinyResult{false, 0};
}
return TinyResult{true, value + 11ULL};
}
if (mode.find("exception") == 0U) {
try {
sum += parse_or_throw(value, index, period);
} catch (const std::exception&) {
sum += 7ULL;
}
} else {
const TinyResult result = parse_result(value, index, period);
sum += result.ok ? result.value : 7ULL;
}
5.5 std::function
std::functionstd::function<std::uint64_t(std::uint64_t)> erased = [](std::uint64_t value) {
return inline_transform(value);
};
if (mode == "std_function") {
sum += erased(value);
} else if (mode == "function_pointer") {
sum += fn_ptr(value);
} else {
sum += direct_lambda(value);
}
5.6 RAII In A Hot Loop
if (mode == "string_per_iter") {
std::string key = std::string("prefix_") + std::to_string(index + pass);
sum += key.size();
}
else if (mode == "string_reuse") {
reused.clear();
reused.append("prefix_");
reused.append(std::to_string(index + pass));
sum += reused.size();
}
else if (mode == "unique_ptr_per_iter") {
auto value = std::make_unique<std::uint64_t>(index + pass);
escape_pointer(value.get());
sum += *value;
}
else if (mode == "stack_object") {
std::uint64_t value = index + pass;
escape_pointer(&value);
sum += value;
}
5.7 Defensive Validation
if (mode == "revalidate") {
if (!valid_trade(trade)) {
throw std::runtime_error("invalid trade");
}
if (trade.quantity == 0 || trade.price == 0) {
throw std::runtime_error("invalid trade");
}
}
sum += trade.quantity * trade.price + pass;
5.8 Builder Pattern
Request request = RequestBuilder()
.url("https://svc/items/" + std::to_string(index))
.method("GET")
.header("Accept", "application/json")
.header("Trace", "on")
.timeout(30)
.build();
Request request{"https://svc/items/" + std::to_string(index),
"GET",
{{"Accept", "application/json"}, {"Trace", "on"}},
30};
Request request;
request.headers.reserve(2);
request.url = "https://svc/items/";
request.url += std::to_string(index);
request.method = "GET";
request.headers.emplace_back("Accept", "application/json");
request.headers.emplace_back("Trace", "on");
request.timeout_ms = 30;
5.9 DTO Mapping
Entity entity{row.id, row.quantity, row.price, row.symbol};
ServiceDto service{entity.id, entity.quantity * entity.price, entity.symbol};
ApiDto api{service.id, service.notional, service.symbol};
sum += api.id + api.notional + api.symbol.size();
ApiDto api{row.id, row.quantity * row.price, row.symbol};
sum += api.id + api.notional + api.symbol.size();
5.10 Logging And Tracing
if (mode == "eager_disabled") {
const std::string line = format_log_line(index, pass);
if (debug_enabled) {
sum += line.size();
}
}
else if (mode == "lazy_guard_disabled") {
if (debug_enabled) {
sum += format_log_line(index, pass).size();
}
}
if (mode == "boundary_span") {
sum ^= span_marker(pass);
}
for (std::size_t index = 0; index < data->size(); ++index) {
if (mode == "span_every_helper") {
sum ^= span_marker(index);
}
sum += inline_transform((*data)[index]);
}
5.11 Reactive Pipeline
std::vector<std::uint64_t> mapped;
mapped.reserve(data->size());
for (const std::uint64_t value : *data) {
mapped.push_back(value * 3ULL + 1ULL);
}
std::vector<std::uint64_t> filtered;
filtered.reserve(mapped.size() / 2U);
for (const std::uint64_t value : mapped) {
if ((value & 1ULL) == 0ULL) {
filtered.push_back(value);
}
}
for (const std::uint64_t value : filtered) {
sum += value;
}
for (const std::uint64_t value : *data) {
const std::uint64_t mapped = value * 3ULL + 1ULL;
if ((mapped & 1ULL) == 0ULL) {
sum += mapped;
}
}
5.12 Service Boundary
if (mode == "direct") {
sum += direct_service_call(value);
}
else if (mode == "queue_stage") {
queue.push_back(value);
const std::uint64_t staged = queue.front();
queue.pop_front();
sum += direct_service_call(staged);
}
else {
const int written = std::snprintf(buffer.data(), buffer.size(), "%llu",
static_cast<unsigned long long>(value));
const std::uint64_t parsed =
static_cast<std::uint64_t>(std::strtoull(buffer.data(), nullptr, 10));
sum += direct_service_call(parsed) + static_cast<std::uint64_t>(std::max(written, 0));
}
5.13 Connection Pooling
if (mode == "new_per_request") {
SimConnection connection;
sum += connection.query((*data)[index]);
}
else {
const SimConnection& connection = (*pool)[index & (pool->size() - 1U)];
sum += connection.query((*data)[index]);
}
5.14 Cache Layer
if (mode == "direct_compute") {
sum += expensive_compute(key);
}
else {
const auto found = cache->find(key);
if (found != cache->end()) {
sum += found->second;
} else {
const std::uint64_t computed = expensive_compute(key);
(*cache)[key] = computed;
sum += computed;
}
}
5.15 String Concatenation
std::string result;
if (mode == "reserve" || mode == "join_buffer") {
result.reserve(pieces->size() * 6U);
}
if (mode == "join_buffer") {
std::vector<std::string_view> views;
views.reserve(pieces->size());
for (const std::string& piece : *pieces) {
views.push_back(piece);
}
for (std::string_view piece : views) {
result.append(piece.data(), piece.size());
}
} else {
for (const std::string& piece : *pieces) {
result += piece;
}
}
5.16 Map Choice
if (mode == "std_map") {
sum += tree->find(key)->second;
} else if (mode == "unordered_map") {
sum += hash->find(key)->second;
} else if (mode == "flat_vector") {
const auto found = std::lower_bound(
shared_items->begin(), shared_items->end(), key,
[](const auto& item, const std::uint64_t query) { return item.first < query; });
sum += found->second;
} else {
for (const auto& item : *shared_items) {
if (item.first == key) {
sum += item.second;
break;
}
}
}
5.17 Iterator, Index, And Raw Pointer Loops
if (mode == "range_for") {
for (const std::uint64_t value : *data) {
sum += inline_transform(value);
}
} else if (mode == "index_loop") {
for (std::size_t index = 0; index < data->size(); ++index) {
sum += inline_transform((*data)[index]);
}
} else {
const std::uint64_t* ptr = data->data();
const std::uint64_t* end = ptr + data->size();
while (ptr != end) {
sum += inline_transform(*ptr);
++ptr;
}
}
5.18 Stream I/O
if (mode == "ostringstream") {
std::ostringstream stream;
stream << "id=" << index << ",value=" << (index + pass);
sum += stream.str().size();
} else if (mode == "snprintf") {
const int written =
std::snprintf(buffer.data(), buffer.size(), "id=%zu,value=%zu", index, index + pass);
sum += static_cast<std::uint64_t>(std::max(written, 0));
} else {
std::string out;
out.reserve(32);
out.append("id=");
out.append(std::to_string(index));
out.append(",value=");
out.append(std::to_string(index + pass));
sum += out.size();
}
5.19 Reflection-Style Access
std::uint64_t read_reflected_field(const ReflectRecord& record, const std::string_view name) {
if (name == "a") return record.a;
if (name == "b") return record.b;
if (name == "c") return record.c;
return record.d;
}
if (mode == "string_lookup") {
sum += read_reflected_field(record, kNames[index & 3U]);
} else if (mode == "table_dispatch") {
switch (index & 3U) {
case 0: sum += record.a; break;
case 1: sum += record.b; break;
case 2: sum += record.c; break;
default: sum += record.d; break;
}
} else {
sum += record.a + record.b + record.c + record.d;
}
5.20 JSON-Shaped Parsing
std::uint64_t parse_streaming_digits(const std::string& text) {
std::uint64_t sum = 0;
std::uint64_t current = 0;
bool in_number = false;
for (const char ch : text) {
if (ch >= '0' && ch <= '9') {
current = current * 10ULL + static_cast<std::uint64_t>(ch - '0');
in_number = true;
} else if (in_number) {
sum += current;
current = 0;
in_number = false;
}
}
return in_number ? sum + current : sum;
}
if (mode == "dom_allocated") {
std::vector<JsonPair> pairs;
pairs.reserve(2);
pairs.push_back(JsonPair{"id", parse_streaming_digits(doc)});
pairs.push_back(JsonPair{"qty", doc.size()});
for (const JsonPair& pair : pairs) {
sum += pair.value + pair.key.size();
}
} else {
sum += parse_streaming_digits(doc);
}
5.21 Allocation Policy
if (mode == "heap_temp") {
auto object = std::make_unique<TinyObject>(TinyObject{index, pass});
escape_pointer(object.get());
sum += object->a + object->b;
} else {
TinyObject& object = (*pool)[index];
object.a = index;
object.b = pass;
sum += object.a + object.b;
}
5.22 Atomics
if (mode == "global_atomic") {
for (std::size_t index = begin; index < end; ++index) {
counter.fetch_add(index + 1U, std::memory_order_relaxed);
}
} else {
std::uint64_t local = 0;
for (std::size_t index = begin; index < end; ++index) {
local += index + 1U;
}
counter.fetch_add(local, std::memory_order_relaxed);
}
5.23 ORM-Style Lazy Loading
if (mode == "lazy_n_plus_one") {
for (const ParentRow& parent : *parents) {
sum += parent.id;
for (const ChildRow& child : *children) {
if (child.parent_id == parent.id) {
sum += child.value;
}
}
}
}
else if (mode == "eager_join") {
std::size_t child_index = 0;
for (const ParentRow& parent : *parents) {
sum += parent.id;
for (std::uint32_t count = 0; count < 5U; ++count) {
sum += (*children)[child_index++].value;
}
}
}
5.24 Serialization
if (mode == "json_text") {
const int written =
std::snprintf(buffer.data(), buffer.size(), "{\"id\":%u,\"qty\":%u,\"price\":%llu}",
payload.id,
payload.qty,
static_cast<unsigned long long>(payload.price));
sum += parse_streaming_digits(std::string(buffer.data(),
static_cast<std::size_t>(std::max(written, 0))));
} else {
std::array<unsigned char, sizeof(BinaryPayload)> bytes{};
std::memcpy(bytes.data(), &payload, sizeof(payload));
BinaryPayload decoded{};
std::memcpy(&decoded, bytes.data(), sizeof(decoded));
sum += decoded.id + decoded.qty + decoded.price;
}
6. Expectations
7. Final Results
| Verdict | Rows |
|---|---|
CONFIRMED_TAX | 95 |
NO_MEASURABLE_TAX | 4 |
CLEAN_WINS | 12 |
INCONCLUSIVE | 154 |
| Contrast | Size | Final result | Interpretation |
|---|---|---|---|
small_noinline_helper | 64M | control faster by 2.586x | forced calls inside a hot loop were not free |
virtual_vs_soa | 1M | control faster by 7.916x | virtual object layout lost badly to type-bucketed storage |
shared_ptr_vs_ref | 64M | control faster by 4.371x | copying shared_ptr in the loop was expensive |
exception_1pct_vs_error | 64M | control faster by 2.197x | routine failure should not throw |
exception_50pct_vs_error | 64M | control faster by 80.602x | frequent exceptions were catastrophic |
std_function_vs_lambda | 64M | control faster by 1.928x | type-erased callbacks blocked the cheap direct path |
raii_string_vs_reuse | 64K | control faster by 1.123x | per-iteration string construction had a visible tax |
validation_recheck | 64M | control faster by 1.376x | repeated internal validation added hot-path cost |
builder_vs_template | 64M | control faster by 3.040x | a prebuilt template avoided much of the builder construction cost |
dto_layered_vs_direct | 1M | control faster by 3.713x | layered DTO copies mattered |
logging_eager_vs_none | 64M | control faster by 48.696x | formatting logs that are not emitted is very expensive |
tracing_every_vs_boundary | 64M | control faster by 30.099x | tracing every helper dominated the tiny operation |
reactive_pipeline_vs_fused | 1M | control faster by 27.659x | intermediate pipeline containers were costly |
service_serialized_vs_direct | 16M | control faster by 126.064x | serialization-shaped service boundaries dominated direct work |
connection_new_vs_pool | 64M | control faster by 9.237x | setup per request lost to pooled reuse |
json_dom_vs_stream | 64M | control faster by 2.534x | DOM-like allocation lost to streaming extraction |
alloc_heap_vs_pool | 64M | control faster by 6.263x | heap allocation per item lost to pool reuse |
orm_lazy_vs_raw | 64M | control faster by 5713.252x | the N+1 shape was predictably bad |
| Contrast | Size | Final result | What it means |
|---|---|---|---|
exception_no_throw_vs_error | 4K, 64K | NO_MEASURABLE_TAX | supports the zero-cost model when exceptions are not thrown |
exception_no_throw_vs_error | 64M | clean row faster by 1.403x | no-throw exception-shaped code was not the slow path here |
variant_vs_tagged | 64M | clean row faster by 1.112x | variant did not behave as a simple tax in this shape |
unordered_map_vs_flat | 64M | clean row faster by 7.619x | this query shape favored hash lookup over sorted-vector binary search |
linear_scan_vs_hash | 64M | clean row faster by 2263.460x | tiny-N linear scan does not generalize to large-N lookup |
alloc_heap_vs_arena | 64M | clean row faster by 1.196x | this arena row allocates a fresh arena per pass, so pool reuse is the cleaner control |
Comments