Why does this crash?

[SMcNeill]

Why's it crash?

Simple answer:  You're running out of memory to store it all in, and getting a seg fault.  (Linux is fairly good at popping up a message that says, "Error -- segment fault at 123456", while Windows tends to just close and pretend it never ran such an uncouth program on its OS in its entire history!)

You should read my comment, that's actually not his problem When I applied a workaround for the issue the program works and "only" uses around 400MB of memory (a lot, but not enough to run out).

Try that with a 256 color image, where you also have to store 256 color values in each palette...  It might work in this one particular instance, but I wouldn't call it something reliable that you'd be able to say would fix the problem with any certainty.