These are good data points. I think the amount of time you should spend often depends on where you are in your career.
I know many companies don't explicitly require tests, but for them if a candidate submits a take-home challenge without them, its an automatic rejection. Do you think that if you had included tests in that first challenge that you would have gotten an offer?
Wow, so sorry to hear this! This seems to test nothing other than what your interpretation of "clever" is. I've definitely learned that it never hurts to ask questions in these situations.
This is really cool, thanks for sharing! I don't necessarily disagree with your point, though I do think an unrealistic time limit is even worse than no time limit. I also emphasize the importance of not taking on a challenge that is outside the scope of your ability here: https://www.fullstackinterviewing.com/2018/02/02/the-ultimat...
What are your thoughts on timed coding problems like Codility or Hackerrank?
My contention is that it doesn't matter if the time limit is unrealistic (within reason) so long as it's consistent.
ie: the challenge is not "Show me the best solution to this problem" but rather "Show me the best solution to this problem that you can achieve within an hour".
As long as everyone is under the same conditions, the playing field is level.
It's part of a holistic approach, too. The coding challenge is not the only part of the interview process. I suggest that there be an initial interview to assess experience, fit, explain the job, etc. Then if everyone is still interested, the coding challenge. The candidate's response to the challenge then forms the basis for a discussion in another, technically focused interview.
It's not about saying "You gave the wrong answer" but "Talk me through why you solved it this way" and "If this criteria in the challenge changed, tell me about how that might change your response"
The Codility and Hackerrank style problems are okay, but I don't like how constrained they are. I also think it's pointless having someone outside your organisation grade the results. What matters is not having a candidate who is "the best hacker" but having someone who fits what you need. If your team values robustness and debuggability above all else, you _don't_ want someone who can punch out 3 kinda-right prototypes before lunch. If you're an ideation agency, the inverse is true.
That's why takehome encourages companies to create and administer their own challenges rather than the samples. They should fit the kinds of problems your organisation faces, not contrived ones.
Good point about consistency, which is hard to achieve with the open-ended take-home challenges. I still feel like there is a catch 22 for new developers though - if you're competing with more experienced candidates, your best solution that you can achieve within one hour will rarely be one of the better ones, and the only way to really get better is to have real experience working on a production app. And you still have to get hired to do that, hence the catch 22. So, that's why I advocate for spending more time on these challenges up front, especially for people first making the switch, because they can be used as a learning opportunity. So many companies don't have the time or just don't want to invest the time to train newer developers, which automatically eliminates hiring someone straight out of a bootcamp. Maybe someone with a CS degree and no work experience would have the same problem though.
I like what you said about the importance of companies administering their own challenges as well. I've encountered some like that and they do tend to be a better reflection of the types of challenges and problems the company is facing.
Wow, I don't know, I would say 5 challenges / week.
That's our third step though, first our recruiters filter applications (I have no idea how many people are funneled out at this point)
Then we have a 30 minutes phone call to discuss experience and presell the company. We also try to answer all their questions
at this point and explain the challenge. Something along the lines of : "so the next step will be a take home test. We just want to see how you code, don't put too much pressure on you, just do your best in the time given"
Historically we have only been looking for pretty senior engineers, although we are relaxing this a bit.
The company has also been sponsoring a lot of visas (even with very interesting salaries, senior mobile engineers are hard to recruit .. most of them are already happy where they are), I think that take home are a good way to handle this case.
The applicant can take the exercise whenever he wants (the only limitation is that we might not be available to answer their questions in the middle of the night) and shows us what they can do.
Part of what I like about take-home challenges is that you can do them on your own time without anyone watching over your shoulder. Because in-person technical interviews tend to last for such a long time, they can be incredibly draining. The candidate's experience really depends on the interviewer's ability as well.
For a well established developer, this may be true, but for newer developers, they often have to jump through a lot of hoops to prove their skills first, especially if they come from a non-traditional background and do not have a CS degree.
Do you think this issue is less pronounced with the move toward companies providing compensation guidelines up front?
The ideal hiring process imo is actually a training process that is less focused on 'finding the best' but rather focused on helping their local community become great engineers. Companies fund their own bootcamps and pay students $15/hr (with no engineering experience). Students learn and help each other how to code (following a structured curriculum) with mentorship by engineers from the company. When students have finished the basics, they work on open source projects or internal projects following proper engineering process / tools that the company use. When a student has proved that he/she learned how to work well with other engineers and has built a few features, they get hired by the company as a junior engineer and this junior dev would be immediately productive.
This might seem like a crazy idea, but its actually not. The previous companies that I have worked for all offer paid volunteer hours which could go directly to this initiative. The cost of hiring is around 30k per engineer (conservatively), which could help train someone living paycheck to paycheck for an entire year.
I spent the past year and a half starting a non-profit experimenting with this idea and invested all 200k of my annual salary to paying students. The coding tools and environment is set up like a baby version of Google's engineering infrastructure. Students progress are tracked by their merge requests and the complexity of the features that they have built. Students build tools that they use on a daily basis so they are heavily eating their own dogfood. Hopefully, one day I'll be able to build my engineering team by directly hiring from this non-profit. https://garagescript.org
This is a really cool idea - because working with code written by other people is usually a large part of any job, this type of assignment could also test for that. For newer devs, I've seen anything from simple Tic Tac Toe to being given a starter app with a general set of requirements to fulfill.
Sure - as someone who transitioned into development from another industry, I could only rely on my past experience for transferrable skills, and not my actual technical skills. While Github is great, my Github is not always an accurate reflection of where I am technically as I continue learning.
I agree that time suggestions are usually way off, which can be a positive and a negative. On one hand, you can spend a ton of time to go above and beyond, while on the other hand, you may have other priorities. I think if you're really excited about a job or company, you can show that by putting a lot of effort in.
That inherently biases the interview process away from people that have additional commitments in life. You're going to end up preferentially treating those that are currently unemployed versus people that have jobs, for example.
Also, you're likely to be homogenizing your applicant base. Expect to see many more young people and recent grads versus people with kids.
I agree with your point. I'll add that for new developers and those without the traditional CS background (both of whom are my target audience for this guide) spending the extra time is a learning experience for them that can make them better developers.
Ultimately, it's not the applicant's choice what the parameters of the interview process are (I commend you for working to make it better) - given that, why shouldn't an applicant try to give themselves an edge in the process?
You're completely right, Jane. I was viewing this pretty myopically from the hiring side as that's where I usually sit. Very true that candidates usually need to play somebody else's game, regardless of whether it's rigged.
I know many companies don't explicitly require tests, but for them if a candidate submits a take-home challenge without them, its an automatic rejection. Do you think that if you had included tests in that first challenge that you would have gotten an offer?