SIMD Mapping: From Normal Code to Vector Code

A scalar float add operates on one value:

float add(float a, float b) {
    return a + b;
}

On x86, this kind of operation commonly maps to a scalar single-precision instruction:

addss

The suffix matters:

ss = scalar single-precision
ps = packed single-precision

Packed single-precision means multiple float values in one register.

scalar:
    addss    one float + one float

packed:
    addps    four floats + four floats
    vaddps   eight floats + eight floats with AVX2

So the SIMD question is not:

How do I make one add faster?

The SIMD question is:

Where can I line up many independent adds so one instruction can do all of them?

SIMD is often won or lost before the first intrinsic.

Bad shape:

struct Order {
    float price;
    float quantity;
    float fee;
    int customer_id;
    char symbol[16];
};

Order orders[N];

The hot loop only needs price, quantity, and fee, but memory is laid out like this:

price0 quantity0 fee0 cold0 cold0 ...
price1 quantity1 fee1 cold1 cold1 ...
price2 quantity2 fee2 cold2 cold2 ...

SIMD wants this:

price0 price1 price2 price3 price4 price5 price6 price7
qty0   qty1   qty2   qty3   qty4   qty5   qty6   qty7
fee0   fee1   fee2   fee3   fee4   fee5   fee6   fee7

That usually means Structure of Arrays:

struct OrdersSoA {
    float* price;
    float* quantity;
    float* fee;
    int count;
};

or separate arrays from the start:

const float* price;
const float* quantity;
const float* fee;

The important rule:

SIMD wants contiguous same-type values.

You can use gathers and shuffles to fight a bad layout, but those are costs. A clean layout lets the SIMD loop become load, compute, store or accumulate.

Most SIMD loops follow this structure:

load chunk
compute chunk
store or accumulate chunk
handle tail

For the running example, the core chunk loop is:

for (; i + 8 <= n; i += 8) {
    __m256 vprice = _mm256_loadu_ps(price + i);
    __m256 vqty = _mm256_loadu_ps(quantity + i);
    __m256 vfee = _mm256_loadu_ps(fee + i);

    __m256 vscore = _mm256_add_ps(_mm256_mul_ps(vprice, vqty), vfee);

    // mask/count/sum comes next
}

Why i += 8?

Because AVX2 holds eight 32-bit floats in one __m256.

If n is not a multiple of 8, the SIMD loop handles the largest clean prefix and the scalar tail handles the rest:

for (; i < n; ++i) {
    float score = price[i] * quantity[i] + fee[i];

    if (score > threshold) {
        ++result.count;
        result.total += score;
    }
}

The tail loop is not optional unless you arrange padding or masked tail handling. The simple default is scalar tail.

Here is the complete mapped version.

#include <immintrin.h>

struct ScoreSummary {
    int count;
    float total;
};

static float hsum_avx2(__m256 v) {
    __m128 lo = _mm256_castps256_ps128(v);
    __m128 hi = _mm256_extractf128_ps(v, 1);
    __m128 sum = _mm_add_ps(lo, hi);

    sum = _mm_hadd_ps(sum, sum);
    sum = _mm_hadd_ps(sum, sum);

    return _mm_cvtss_f32(sum);
}

ScoreSummary summarize_avx2(
    const float* price,
    const float* quantity,
    const float* fee,
    int n,
    float threshold
) {
    int i = 0;
    int count = 0;

    __m256 vthreshold = _mm256_set1_ps(threshold);
    __m256 total_vec = _mm256_setzero_ps();

    for (; i + 8 <= n; i += 8) {
        __m256 vprice = _mm256_loadu_ps(price + i);
        __m256 vqty = _mm256_loadu_ps(quantity + i);
        __m256 vfee = _mm256_loadu_ps(fee + i);

        __m256 vscore = _mm256_add_ps(_mm256_mul_ps(vprice, vqty), vfee);
        __m256 mask = _mm256_cmp_ps(vscore, vthreshold, _CMP_GT_OQ);

        int bits = _mm256_movemask_ps(mask);
        count += __builtin_popcount((unsigned)bits);

        __m256 selected = _mm256_and_ps(vscore, mask);
        total_vec = _mm256_add_ps(total_vec, selected);
    }

    float total = hsum_avx2(total_vec);

    for (; i < n; ++i) {
        float score = price[i] * quantity[i] + fee[i];

        if (score > threshold) {
            ++count;
            total += score;
        }
    }

    return ScoreSummary{count, total};
}

The hot loop does not look like the scalar loop anymore, but the mapping is direct:

This is the core SIMD translation pattern.

This scalar loop reduces every element into one value:

float total = 0.0f;

for (int i = 0; i < n; ++i) {
    total += a[i];
}

Bad SIMD shape:

for (int i = 0; i + 8 <= n; i += 8) {
    __m256 v = _mm256_loadu_ps(a + i);
    total += hsum_avx2(v);
}

This collapses SIMD into scalar every iteration.

Better SIMD shape:

load vector
add into vector accumulator
repeat
reduce accumulator once at the end

This is what the running example does:

total_vec = _mm256_add_ps(total_vec, selected);

It keeps eight partial sums alive:

total_vec:
[sum0 sum1 sum2 sum3 sum4 sum5 sum6 sum7]

Only after the loop does it call:

float total = hsum_avx2(total_vec);

Rule:

Vertical work in the hot loop.
Horizontal work at the end.

Strings are different.

To SIMD, a string is not words or characters. It is bytes.

"hello,world\n"

is:

h e l l o , w o r l d \n

The common mapping is:

load 32 bytes
compare all bytes against target
turn byte comparison into bitmask
use scalar bit operations on the mask
handle tail

Scalar search:

const char* find_char_scalar(const char* s, int n, char target) {
    for (int i = 0; i < n; ++i) {
        if (s[i] == target) {
            return s + i;
        }
    }

    return nullptr;
}

AVX2 byte scan:

#include <immintrin.h>

const char* find_char_avx2(const char* s, int n, char target) {
    int i = 0;
    __m256i vt = _mm256_set1_epi8(target);

    for (; i + 32 <= n; i += 32) {
        __m256i chunk = _mm256_loadu_si256((const __m256i*)(s + i));
        __m256i cmp = _mm256_cmpeq_epi8(chunk, vt);

        int mask = _mm256_movemask_epi8(cmp);

        if (mask != 0) {
            int offset = __builtin_ctz(mask);
            return s + i + offset;
        }
    }

    for (; i < n; ++i) {
        if (s[i] == target) {
            return s + i;
        }
    }

    return nullptr;
}

_mm256_movemask_epi8 is the key instruction:

byte comparison:
[00 00 FF 00 FF 00 00 FF ...]

movemask:
00101001...

Now normal integer bit operations can find the matching byte positions.

To find comma or newline:

__m256i comma = _mm256_set1_epi8(',');
__m256i newline = _mm256_set1_epi8('\n');
__m256i chunk = _mm256_loadu_si256((const __m256i*)ptr);

__m256i is_comma = _mm256_cmpeq_epi8(chunk, comma);
__m256i is_newline = _mm256_cmpeq_epi8(chunk, newline);
__m256i either = _mm256_or_si256(is_comma, is_newline);

int mask = _mm256_movemask_epi8(either);

This pattern is the base of fast CSV, JSON, HTTP, and log scanning.

It also works inside UTF-8 text when looking for ASCII delimiters. But byte scanning is not the same as Unicode character processing. Counting characters, slicing by character index, or changing case is a different problem.

Do not start with intrinsics.

Use this workflow:

write clear scalar code
benchmark it
check compiler auto-vectorization
fix memory layout
remove unpredictable branches if needed
write intrinsics only for hot loops
benchmark again

Useful commands:

g++ -O3 -march=native main.cpp -o main
g++ -O3 -fopt-info-vec main.cpp -o main
clang++ -O3 -Rpass=loop-vectorize main.cpp -o main
perf stat ./bench

Useful counters:

cycles
instructions
branches
branch-misses
cache-misses
L1-dcache-load-misses

Before writing SIMD code, ask:

Is the data contiguous?
Is the same operation repeated?
Are lanes independent?
Can branches become masks?
Can reductions be delayed?
Can the tail be handled safely?
Is this loop actually hot?

The final mapping model:

Scalar value      -> SIMD lane
Scalar loop       -> chunk loop
Scalar constant   -> broadcast vector
Scalar branch     -> mask or blend
Scalar byte scan  -> compare plus movemask
Scalar reduction  -> vector accumulators plus final horizontal reduction
Object layout     -> usually fix layout before writing intrinsics

SIMD works when code has this shape:

same operation
many contiguous elements
minimal branching
minimal lane communication

If the code has that shape, SIMD maps naturally.

If it does not, do not force it. Change the data shape first, or leave the code scalar.