For a good programmer, a 3 hour take home test is like a $300+ application fee. Even if the puzzles are reasonable, filters work both ways. For a recent graduate or a mediocre programmer, it's probably worth the time for a shot at the job, but talented people will have plenty of options at good companies that don't make them jump through extra hoops. Perhaps it's fine if you pay a lot higher than average or offer significant perks, but I could see a strategy like this unwittingly driving away the top tier of candidates if you aren't careful.
That is a reasonable theory, and I was always personally very concerned about that, but in practice it didn't seem to be the case. It seemed that the opposite occurred: people who didn't even want jobs did the puzzles just for fun, and asked us if they got the right answer.
Of course, we couldn't easily do A/B experiments, so we'll never really know how the true attract:alienate ratio.
Trying to look at it objectively, I would probably be somewhat put off by being required to do a puzzle to "prove myself", but would then find the specific puzzles offered to be pretty interesting, and worth playing around with. Perhaps it's not tantamount to a $300 tax on ones life it is seemed fun and valuable in its own right?
I have less disdain for pre-interview development filters and have seen some companies do it pretty well. I think your assumption about people who would find it fun could hold some merit. But I do believe the parent poster is right, those really good developers are rarely going to bother. They are usually to busy in the first place. They may see your company and think oh cool I like these guys and then hit the pre-interview filter and decide it to much investment for just a chance and that's the core of it, on both sides it's a numbers game, we can say well they did not really want it or we are only getting the ones that really want it, but you just may be getting the ones that are not working the odds out in their head. Or you could be getting the ones that are desperate and will invest the time for a chance. There are a lot of assumptions in these type of tests.
I think in the end, most people just want to have their abilities evaluated fairly. And even if I'm not a total puzzle freak, I do love to code, and given time I can figure most of these puzzles out. So if I were looking for a job at your (former) company, this would not constitute a barrier for me. On the other hand, if I were verbally asked a question and told to work it out on a whiteboard, this would not be an accurate indicator of my abilities or my potential. Add to that the fact that my handwriting is poor, and I would be nervous, and I'm quite sure a verbal description of pascal's triangle would almost certainly elude me, whereas if I were shown a document with the written output, I think I could come up with a naive solution in 5-10 mins. I'm good at sitting down by myself with a written problem and working it out. As a result, I've always done well on standardized tests. In fact, I suspect this is the reason for some of the indignation at the whiteboard testing. People who have always gotten good grades, coded well, done well on standardized tests are suddenly told that they're not smart enough. I can understand how it would rankle.
It isn't like that because paying money to someone in exchange for merely being evaluated is an easy way to fall for a scam: the evaluator has a significant incentive to offer this deal to people who will fail.
It is much more like having to travel 90 minutes each way to get to where they want you to interview. That could be very offputting if you do enough interviews that three hours each adds up to a big number, but at least you don't need to worry about them being tempted to rip you off.
