I have a bit of code that steps through a binary file made up of blocks that are of a known size, in an arbitrary order, and come in N different flavors and save them into N output files, one for each flavor. It actually does much more too, but lets pretend this is the only operation. The simple way to do this is to step sequentially through the file and load everything into memory in N different structures and then save your files. But the problem is that sometimes you don't have enough memory to do that, so you have to loop through the file N times, loading a type, saving it, clearing memory and repeating. Since that seems very inefficient, so I thought I'd be clever and on the first pass through the file, load one of the flavors and also save file pointers to all the other blocks so that each subsequent pass could be done more quickly by seeking to the exact black that I needed to load.
Long story short, on regular hard drives, thanks to optimized sequential reads it can actually be faster to just do the dumb thing and loop through the file N times than seek though it randomly.
I have a bit of code that steps through a binary file made up of blocks that are of a known size, in an arbitrary order, and come in N different flavors and save them into N output files, one for each flavor. It actually does much more too, but lets pretend this is the only operation. The simple way to do this is to step sequentially through the file and load everything into memory in N different structures and then save your files. But the problem is that sometimes you don't have enough memory to do that, so you have to loop through the file N times, loading a type, saving it, clearing memory and repeating. Since that seems very inefficient, so I thought I'd be clever and on the first pass through the file, load one of the flavors and also save file pointers to all the other blocks so that each subsequent pass could be done more quickly by seeking to the exact black that I needed to load.
Long story short, on regular hard drives, thanks to optimized sequential reads it can actually be faster to just do the dumb thing and loop through the file N times than seek though it randomly.