Somewhat tangential question- when you have people who conduct interviews so regularly, have companies ever tried measuring the performance of said interviewers via the ratings of the people they brought on board?
I am not personally aware of anything like that happening frequently, although I do know that Facebook has previously invested (amongst other things) in evaluating the use of certain types of questions before, with some interesting results.
It sounds like it could be a fairly unreliable indicator. I'm not sure what the interview-to-hire rate is, but there are only a pretty small number of people that are offered positions - and even then sometimes you don't win against a competing offer, or a sudden realisation that they want to stay geographically where they are. So even for the top few percent of interviewers who interview ~100 a year, you might only have 2-6 hires, which isn't really a large enough population to evaluate.
In an interview, the best you can do is try probe talents, skills, and knowledge, but you'll also get a point-in-time view into their current attitudes and beliefs, as well as the possibility for swagger and misdirection (even self-misdirection).
We do look carefully at situations where one or sometimes two people get strongly negative or positive signals while the remainder get strongly in the other direction, in case there is a problem. Most times it has turned out to be valid concerns, and it also helps us give feedback to interviewers on how to write better feedback, how strongly to weight certain traits ("not knowing the name of a particular useful function/method under pressure is reasonable, so let them just use it even if they got the name wrong"), or how to evaluate a particular type of candidate (newly graduated candidates, for example).
At least in our environment the "next step" are loops with multiple people. The goal is to help get more people a chance to see the candidate before they join etc.
It might be interesting to track the number of screens -vs- cut-offs on-site.. :)
Also I am measuring the number of screens that fail as a quality indicator for the recruiting team.
Think of it as a pipeline from the first contact all the way through new-hire and 1yr review. How are we doing at each step and what can we do to improve each step for everyone involved including the candidate.
