It will also depend on what cutting fluid is used while in the machine, we use a water based solution that never seemed to give a lick of trouble from our Aluminum. I have taken one that was still wet from the machine, added the necessary hardware, and cast with it after it was dried off. (We didn't know the fluid we used would be that conducive to casting, it was just a happy coincidence)
I don't know what cutting fluid Al uses, but it may have a trace element like silicon that works it's way into the metal finish and takes a bit to remove. His alloy shouldn't be an issue, 6064 is a good aluminum and it machines very well. I do believe it is the design as much as anything, just like some guns have quarks, so do designs/blocks.