Earlier this year we decided to hire summer interns. We wanted to know how well someone would perform on the job, so we gave our top candidates a real paid task from our backlog. Introducing a payment allowed us to ask for something challenging and lengthy, while also respecting the candidates’ time. We also found that the strongest candidates self-selected into this paid screening challenge, unlike the traditional and mundane coding puzzles. You can find our challenge and the rewarded pull requests [1], [2], [3] publicly on Algora.

From our experience, hiring by building product predicts on-the-job performance really well.

Why?

It's sensible to hire above average engineers as opposed to below average. Over time this can amount to very sizable economic gains for an organization. So the question becomes, how does one identify the best? Over the years there's been a lot of discussion on HackerNews about the developer interview process [1], [2], [3].

What we suggest is simple: to make developer interviews more valid & inclusive, introduce payments.

Economic sense of paying developers during interviews

The mean developer productivity, when measured in dollar output terms, equals the mean market salary.

If we assume the top & bottom 16% of engineers (corresponding to ±1SD) are at least 30% more/less productive, then the difference amounts to 60% of the salary.

If a company stands to gain $30k in output, and accordingly not lose $30k in unrealized output, by simply introducing payments in the last-mile of the hiring process, it's a rational choice to do so.

Conclusion

To accurately predict job performance, just give engineers a paid job to complete.