This is the part I fundamentally disagree with and the reason that I am not a proponent of testing. Because it is loaded with so many assumptions on the part of the interviewer. I once failed to get a JavaScript gig because I did not use the arguments array in a method. The interviewer though that the question given a arbitrary amount of parameters passed into a JavaScript function how would you sum all of the parameters. I content that passing an infinite/arbitrary amount of parameters into a JavaScript function is something that I will never do. Not only that I would have a talk to any developer on my team that did, if it was not some very special need for such magic code. The interviewer thought that this was a perfectly reasonable filter and said so, it was stated as the reason that I did not get the position.
You see that's the crux of the issue, I can be a JavaScript expert and deliver massive JavaScript applications without having to resort to the arguments array to figure out how many parameters where passed into a function. Also, if I ever had a use case for it, I would look up the API and probably mentally discard it shortly afterwards. The interviewer thought that it was a perfectly reasonable filter, but yet I am one of the top JavaScript talents in my market. It did not work for them, because they assumed that knowledge of the arguments array was basic knowledge and any developer worth their salt would know it. Yet in my time working with some great JavaScript developers not a one of us ever used it. Further it harmed them because I now believe they are as incompetent as they believe me to be, as such if someone in my peer group asked about a position with them, I would relay my story. Not to be indignant but to be truthful, as such they lost access to valuable resources in a constrained market.
That being said, one can be an expert in say Java and not know the JDBC API, maybe all of their work revolved around JPA. Many in Java would argue that JDBC is fundamental and should be known to be an expert but a new crop of developers have never know anything but ORM. But we older developers assume that JDBC is a fundamental prerequisite because it was in our time. They cant see anything but the progression from JDBC to JPA because it is the life experience that they had. As such it's flawed and irrelevant. You have to be very careful to not bring interviewer bias into an interview and test are loaded with them. Even the simplest of test.
Does it not strike you that quite a few HN'ers openly admit that they would not pass the filter? Some of them long time contributors and members, some of them very respected? To me, I would question the viability of my assumptions if HN'ers where openly saying no your filter would eliminate me. I mean we have some of the best of the best lurking here. 37 has been complaining about it for a long time, I have had my own threads about it, and everyone agrees that hiring is flawed. Maybe it's time we reevaluate our assumptions based on what guys like the crew at 37 are doing since they seem to be improving on candidate selection over the rest of us.
The point I am making is what you are trying to garner, can be identified by asking a candidate to show you something they built and walk you through the routine they are most proud of. There are going to be functions, loops, if statement etc. in that code, if they can't explain what they built, then you should be suspect. Giving them a test just introduces bias of what the interviewer thinks they should know, and does nothing to tell the interviewer what they know.
"Does it not strike you that quite a few HN'ers openly admit that they would not pass the filter?"
To be quite honest, this thread is having the opposite effect. I'm questioning the quality of HN contributors based on their inability to either comprehend the question or code a simple algorithm.
I passed similar coding questions in interviews before I'd ever written a line of code in my life (in a Microsoft PM internship interview). No one cared if I wrote a semicolon or not - the interview was asking if I understood basic concepts like loops and logic.
I don't think that anyone is arguing that they should not be able to code a simple algorithm. What the original article and other respectable individuals in the industry have been arguing for, is the best way to deduce ones ability to code a simple algorithm. Some people are just so uncomfortable and out of there element coding in an interview situation, that the assumption that it is a negative filter does not hold true. May good candidates get missed over this assumption. What many hiring managers like those at 37 Signal and pen.io have found out, is you can actually identify more quality individuals while deducing the same abilities without placing people in that unnatural coding environment. They are writing about it, because they have seen evidence that it is an unnecessary filter. It is not that one does not need to deduce whether the applicant can code but rather they believe they have found a better, simpler way to deduce it. Their observations coincide with what I have seen in the industry, so I am an advocate for such practices.
"Does it not strike you that quite a few HN'ers openly admit that they would not pass the filter?"
I have yet to see someone who says that they would not pass the filter. Every single example I've seen in this thread is someone making up a different filter and saying that they would not pass that. I see people saying that they took a long time to debug subtle bugs in it, that they don't know the definition of the problem, or that in general they need documentation to code, all of which is completely irrelevant to the filter as given. Subtle bugs are allowed, the definition is given to you, the solution doesn't need any documentation (and if it does, you can just make up whatever API call you expect would be present in a sane standard library)
I'd be shocked to find any good programmer who, given the definition of Pascal's triangle, couldn't produce some decent, possibly buggy, incomplete, and syntactically incorrect, code on a whiteboard to print out a row from it.
I've yet to see anyone give an argument for how a good programmer could fail this test that doesn't rest on some feature of the test that doesn't actually exist.
Subtle bugs are allowed, the definition is given to you, the solution doesn't need any documentation (and if it does, you can just make up whatever API call you expect would be present in a sane standard library)
You are now hitting at the core of the issue, to most developers bugs, bad code, correct API's matter, you are asking them to look past something that is engrained in their work ethic, personality, and quality as a developer to measure their worth as a developer. It does not matter what the filter truly is, their is a perception on the developers part of what they are being tested for, because they attribute what makes them of value to the test. As such the correct solution is what makes them valuable. You can't tell them to give you what they perceive to be the incorrect solution and expect it to accurately reflect what they value in themselves as a developer, that the interviewers bias. The fact that the filter is being misinterpreted is irrelevant or very relevant depending on your perspective. To the interviewee, they are going to perceive the task just as many on HN have and no amount of prodding to give up their nature is going to make them perform well, conversely they are going to perform worse. I have neglected the fact that the filter is being misunderstood for the very fact that, it will be misunderstood and for that alone it sets people up to fail.
Is there such a thing as a good developer who can't write pseudocode because he is unable to look past bugs, bad code, and correct APIs? I've never heard of such a beast.
Mike that is not my position and I apologize if I am not conveying that properly. My position is that white-boarding in front of people you just met for some people will not convey that they can do it. Because they can become distracted by perceiving that they are a spectacle and their every move is under a microscope. Many developers have been in such a situation and walk away from the situation permanently affected by it. It only take one bad interviewer to criticize your every move to make a developer analyze their every move in future interviews. But that does not mean in their natural and comfortable environment they would have such an issue. It is why I prefer to look at code they produced in that natural and comfortable environment.
Stage fright is an excellent point. So excellent that I wonder why nobody else made it before, and instead went for BS reasons like "I need a debugger". Maybe due to stage fright...?
I think the two are linked, some developers feel secure that a compiler / debugger will catch the inevitable simple mistakes that we all make. Not having one in front of an audience makes it that much worse. A lot of those that protest would feel a lot more comfortable in front of people if they knew they had a net to catch the simple errors they make before they show off their code and that's the core of my position on the subject, code test guarantee that these otherwise competent individuals will look incompetent and once they do, it's hard to minimize that experience in the interviewers mind, Steve Jobs called it the bozo bit and once it is flipped it's pretty much flipped for good. The fact that they look incompetent on the whiteboard is such a strong reaction, that it outweighs compensation factors like looking at code already written, in a more conducive environment.
You're in a room with these people having a conversation. A simple "Hey what level of engineering are you looking for in this solution?" would do nicely. Maybe some people will say that it has to be unit tested, rigorously documented, and proven correct, all on a whiteboard, and that would be really dumb. But nobody is going to say that, they're going to say "Give us a sketch - we're just looking for your thought process". The whole thing about misunderstandings and misinterpretations is just really silly when you're sitting there in a room with them. It also isn't just your code that is being judged - if you can't pose a simple question to some people sitting across a desk from you, or if you're too stodgy to ever under any circumstances write pseudocode or a piece of code that you aren't sure works yet, or make up a reasonable API that you don't remember the exact incantation for, maybe because you're too honorable or something, then I don't think you're a culture fit.
It also isn't just your code that is being judged - if you can't pose a simple question to some people sitting across a desk from you, or if you're too stodgy to ever under any circumstances write pseudocode or a piece of code that you aren't sure works yet, or make up a reasonable API that you don't remember the exact incantation for, maybe because you're too honorable or something, then I don't think you're a culture fit.
This is one of the point that I am getting at, you are right it's not just the code that is being judged, candidates know that too, so when they are standing at the whiteboard a million things about being judge may be going through their mind. It has nothing to do with being stodgy or honorable or above the effort and everything to do, with some people just cannot perform in such an environment. People hold out that it is a negative filter, that if you can't pass it, you are a definite negative and observations on a lot of peoples parts are not confirming that assumption. I side with the author of the article that there is an all inclusive way to identify good candidates that fall into this category as well as candidates that would pass the whiteboard test without resorting to the whiteboard. I am not arguing that people should not be able to code, rather I side with the author that there is a better way to identify the people that can and receive more positive hits in the process. Thus reducing the interviewing cycle and identify more talent in a constrained market.
Ah, apologies, I didn't mean to advocate for coding tests as an all-or-nothing negative filter, though I see now that the thread I'm in may suggest that. It can be a good negative filter, but nobody should be disqualified by a single poor answer. Good questions are designed to reveal problem solving skills, so the path taken even to a very poor answer should be illuminating and lead the interviewer to another question.
You see that's the crux of the issue, I can be a JavaScript expert and deliver massive JavaScript applications without having to resort to the arguments array to figure out how many parameters where passed into a function. Also, if I ever had a use case for it, I would look up the API and probably mentally discard it shortly afterwards. The interviewer thought that it was a perfectly reasonable filter, but yet I am one of the top JavaScript talents in my market. It did not work for them, because they assumed that knowledge of the arguments array was basic knowledge and any developer worth their salt would know it. Yet in my time working with some great JavaScript developers not a one of us ever used it. Further it harmed them because I now believe they are as incompetent as they believe me to be, as such if someone in my peer group asked about a position with them, I would relay my story. Not to be indignant but to be truthful, as such they lost access to valuable resources in a constrained market.
That being said, one can be an expert in say Java and not know the JDBC API, maybe all of their work revolved around JPA. Many in Java would argue that JDBC is fundamental and should be known to be an expert but a new crop of developers have never know anything but ORM. But we older developers assume that JDBC is a fundamental prerequisite because it was in our time. They cant see anything but the progression from JDBC to JPA because it is the life experience that they had. As such it's flawed and irrelevant. You have to be very careful to not bring interviewer bias into an interview and test are loaded with them. Even the simplest of test.
Does it not strike you that quite a few HN'ers openly admit that they would not pass the filter? Some of them long time contributors and members, some of them very respected? To me, I would question the viability of my assumptions if HN'ers where openly saying no your filter would eliminate me. I mean we have some of the best of the best lurking here. 37 has been complaining about it for a long time, I have had my own threads about it, and everyone agrees that hiring is flawed. Maybe it's time we reevaluate our assumptions based on what guys like the crew at 37 are doing since they seem to be improving on candidate selection over the rest of us.
The point I am making is what you are trying to garner, can be identified by asking a candidate to show you something they built and walk you through the routine they are most proud of. There are going to be functions, loops, if statement etc. in that code, if they can't explain what they built, then you should be suspect. Giving them a test just introduces bias of what the interviewer thinks they should know, and does nothing to tell the interviewer what they know.