001 Notes

inih Optimization: Simpler Scans Beat Fancy Tricks

0. Abstract

This note looks at a small parser with a useful optimization lesson:

the best speedups did not come from adding a lot more machinery
they came from removing avoidable work that the baseline kept paying for

The measured inih changes fall into four buckets:

Area	Main change	Best role in the final result
String input	`memchr()` plus `memcpy()` instead of byte-at-a-time copying	removes avoidable front-end work
Parser scan	explicit ASCII whitespace and delimiter tests	biggest parser win
SIMD	AVX2 only for a narrow delimiter/comment scan	useful on long lines, not the main story
C++ wrapper	sorted flat storage instead of tree storage	huge enumeration win

On the measured Ryzen 9 5900HX system, the combined result versus the baseline was:

C parser, string mode: 2.572x
C parser, file mode: 2.068x
C++ parse: 1.331x
C++ lookup: 1.176x
C++ enumerate: 7.414x

The strongest conclusion is narrower than “SIMD made it fast”:

bulk copy the line input
use the exact character tests the format needs
store parsed key/value data in the shape readers actually use

1. Baseline And Choke Points

The baseline parser had three avoidable costs in hot paths:

string parsing copied input one byte at a time until \n
leading and trailing whitespace cleanup repeatedly called isspace()
delimiter and comment scans repeatedly called strchr() on tiny fixed sets

The C++ wrapper had a different kind of cost. It stored all values in a std::map<std::string, std::string>, then paid tree-walk and rescanning costs again for lookups, section enumeration, and key enumeration.

That led to the right optimization order:

remove work before vectorizing anything;
keep the parser scalar when scalar is already enough;
fix the C++ data layout instead of micro-tuning tree access.

2. Bulk Line Reads Beat Byte-At-A-Time Copying

Old Shape

The string reader copied one character at a time until it found a newline:

while (num > 1 && ctx_num_left != 0) {
    c = *ctx_ptr++;
    ctx_num_left--;
    *strp++ = c;
    if (c == '\n')
        break;
    num--;
}

That loop is correct, but it forces every line to pay for per-byte control flow before the real parsing even begins.

New Shape

The faster path uses one bounded newline search and one bulk copy:

max_copy = (size_t)num - 1;
if (max_copy > ctx_num_left)
    max_copy = ctx_num_left;
newline = (const char*)memchr(ctx_ptr, '\n', max_copy);
copy_len = newline ? (size_t)(newline - ctx_ptr) + 1 : max_copy;

memcpy(str, ctx_ptr, copy_len);
str[copy_len] = '\0';

Measured Result

Isolated on comments-16m.ini, warm cache, 15 iterations:

string parse: 48.965 ms -> 37.973 ms, 1.289x
throughput: 326.77 -> 421.36 MiB/s

Broader string-mode stability results:

Dataset	Before	After	Speedup
`comments-1m`	`3.047 ms`	`2.403 ms`	`1.268x`
`comments-16m`	`50.175 ms`	`39.473 ms`	`1.271x`
`long-1m`	`5.127 ms`	`4.366 ms`	`1.174x`
`long-16m`	`83.372 ms`	`71.810 ms`	`1.161x`

The instruction count dropped from 15.67B to 12.33B, and branches dropped from 3.60B to 2.85B.

That is exactly the kind of gain worth keeping: simpler code, less work, and a clear speedup on the workload the function actually serves.

3. Simpler Character Tests Were The Biggest Parser Win

Old Shape

The baseline parser leaned on general libc helpers:

while (end > s && isspace((unsigned char)(*--end)))
    *end = '\0';

while (*s && (!chars || !strchr(chars, *s)) &&
       !(was_space && strchr(INI_INLINE_COMMENT_PREFIXES, *s))) {
    was_space = isspace((unsigned char)(*s));
    s++;
}

For an INI parser, those helpers are broader than the problem. The parser only needs a small ASCII whitespace set and very short delimiter/comment-prefix sets.

New Shape

The faster version uses explicit format-specific checks:

static int ini_isspace(unsigned char c)
{
    return c == ' ' || c == '\t' || c == '\n' || c == '\r' ||
           c == '\f' || c == '\v';
}

static int ini_char_in(const char* chars, char c)
{
    if (!chars)
        return 0;
    while (*chars) {
        if (*chars++ == c)
            return 1;
    }
    return 0;
}

Compatibility Boundary

This changes whitespace classification from “whatever the active C locale says” to explicit ASCII whitespace. That is usually the right tradeoff for config files, but it is still the one behavior boundary worth checking before any upstream proposal.

Measured Result

On comments-16m.ini, warm cache, 15 iterations:

string parse: 39.262 ms -> 31.847 ms, 1.233x
throughput: 407.52 -> 502.40 MiB/s

Broader stability results versus the already-improved bulk-copy step:

Dataset	Mode	Before	After	Speedup
`comments-1m`	string warm	`2.403 ms`	`1.106 ms`	`2.173x`
`comments-1m`	file warm	`2.869 ms`	`1.433 ms`	`2.002x`
`comments-16m`	string warm	`39.473 ms`	`17.932 ms`	`2.201x`
`comments-16m`	file warm	`48.753 ms`	`23.546 ms`	`2.071x`
`long-1m`	string warm	`4.366 ms`	`2.409 ms`	`1.812x`
`long-16m`	file warm	`75.565 ms`	`41.526 ms`	`1.820x`

The most important counters moved in the intended direction:

instructions: 12.30B -> 8.87B
branches: 2.86B -> 1.68B
cycles: 5.37B -> 4.30B

Branch misses rose slightly in absolute terms, but the parser still won hard on wall time because it executed far fewer instructions and branches overall.

4. AVX2 Helped, But It Was Not The Main Story

The AVX2 path was deliberately narrow. It only accelerates the parser’s “find the first interesting byte” work for simple delimiter sets such as:

] on section lines
= or : on name/value lines
inline ; comments after whitespace

That is the right scope. A full “vectorize the parser” rewrite would add much more complexity than the workload justifies.

Measured Result

SIMD-only geomean versus baseline:

parser string: 1.407x
parser file: 1.414x

Best 128 MiB string-mode rows:

Dataset	Baseline	SIMD-only	Speedup
`long`	`186.49 MiB/s`	`349.71 MiB/s`	`1.88x`
`comments`	`312.95 MiB/s`	`539.66 MiB/s`	`1.72x`

Those are real wins, but the broader parser story stayed scalar-first:

scalar-only geomean: 2.750x string, 2.138x file
final combined geomean: 2.572x string, 2.068x file

So AVX2 belongs in the final build as a selective fast path, not as the main explanation for the speedup.

5. Flat Storage Made The C++ Wrapper Behave Like A Reader

The biggest C++ problem was data layout. The baseline wrapper used a tree for everything, even though the dominant usage pattern is:

parse once;
look up values many times;
enumerate sections and keys.

That pattern wants sorted contiguous storage, not one heap node per key.

New Shape

The faster wrapper stores Entry { key, value } objects in a vector, sorts once, merges duplicates, then serves reads with binary search and sequential range walks.

It also adds a GetView() path so a lookup can return std::string_view instead of always allocating a new string copy.

Measured Result

Flat-reader geomean versus baseline:

C++ parse: 1.037x
C++ lookup: 1.171x
C++ enumerate: 7.203x

Representative 1 MiB enumeration rows:

Dataset	Baseline	Flat storage	Speedup
`compact`	`138.66 ms`	`1.32 ms`	`105.05x`
`multiline`	`104.47 ms`	`1.12 ms`	`93.28x`
`duplicate`	`78.08 ms`	`0.68 ms`	`114.82x`

This is a good example of a cache-friendly optimization that earns its keep. It did not add a framework. It just stored the data in the shape the API was already asking for.

6. Combined Results

Measured geomean versus the baseline:

Variant	Parser string	Parser file	C++ parse	C++ lookup	C++ enumerate
baseline	`1.000x`	`1.000x`	`1.000x`	`1.000x`	`1.000x`
scalar-only	`2.750x`	`2.138x`	`0.909x`	`0.702x`	`0.666x`
SIMD-only	`1.407x`	`1.414x`	`1.117x`	`1.012x`	`1.061x`
flat reader only	`1.304x`	`1.342x`	`1.037x`	`1.171x`	`7.203x`
final combined	`2.572x`	`2.068x`	`1.331x`	`1.176x`	`7.414x`

Representative final parser throughput at 128 MiB:

Dataset	Baseline	Final
`comments`	`312.95 MiB/s`	`767.08 MiB/s`
`long`	`186.49 MiB/s`	`563.58 MiB/s`
`compact`	`128.80 MiB/s`	`238.47 MiB/s`

Representative final C++ rows at 1 MiB:

Metric	Baseline	Final
`compact` parse	`49.95 MiB/s`	`51.13 MiB/s`
`compact` lookup	`7,760,030/s`	`7,714,430/s`
`compact` enumerate	`138.66 ms`	`1.36 ms`
`duplicate` lookup	`5,351,000/s`	`7,231,130/s`
`duplicate` enumerate	`78.08 ms`	`0.70 ms`

7. Takeaway

The most valuable inih optimizations were not the fanciest ones.

They were:

bulk-copy line input instead of copying byte-by-byte;
use exact ASCII tests instead of repeated general-purpose libc helpers;
store parsed C++ data as sorted contiguous entries instead of a tree.

The SIMD path still helped where long lines made the search wide enough to justify vectorization. But the durable performance story was simpler:

the parser got faster because it stopped doing unnecessary work
the wrapper got faster because it stopped storing data in a reader-hostile shape

inih Optimization: Simpler Scans Beat Fancy Tricks

0. Abstract

1. Baseline And Choke Points

2. Bulk Line Reads Beat Byte-At-A-Time Copying

Old Shape

New Shape

Measured Result

3. Simpler Character Tests Were The Biggest Parser Win

Old Shape

New Shape

Compatibility Boundary

Measured Result

4. AVX2 Helped, But It Was Not The Main Story

Measured Result

5. Flat Storage Made The C++ Wrapper Behave Like A Reader

New Shape

Measured Result

6. Combined Results

7. Takeaway

Comments