HackerRank Open Sourced Its ATS – My Resume Scored 90, Then 74, Then 88
HackerRank has open sourced its Applicant Tracking System (ATS), sparking widespread discussion in the developer community. A writer put the system to the test with their own resume and found surprisingly inconsistent results, with scores fluctuating between 74 and 90 across different runs. The experiment raises serious questions about the reliability of AI-powered hiring tools.
HackerRank, the well-known platform for coding challenges and technical interviews, has taken a bold step by open sourcing its Applicant Tracking System (ATS). This move gives developers, recruiters, and curious observers a rare opportunity to peek under the hood of a modern hiring system and understand exactly how it scores and ranks job candidates.
One writer decided to put the system through its paces by feeding it their own resume. The results were eye-opening – not necessarily because of the score itself, but because of how wildly inconsistent it was. Running the same resume through the system multiple times yielded dramatically different scores: 90 on one run, then 74, then 88. This raises a fundamental question about the integrity of AI-driven hiring tools: how can a system claim to objectively evaluate candidates when it cannot even evaluate the same candidate consistently?
This kind of variance is a known characteristic of systems built on large language models (LLMs), which are inherently non-deterministic. Unlike traditional rule-based software, LLMs can produce different outputs for identical inputs depending on temperature settings and other parameters. When such systems are used to make high-stakes hiring decisions, this unpredictability becomes a serious concern. The Hacker News thread, which gathered 195 points and 39 comments, reflects a broad community recognition of this issue.
The open sourcing of HackerRank's ATS is a commendable gesture toward transparency in the often opaque world of recruitment technology. However, this experiment illustrates that transparency alone is not enough. The industry needs to grapple with deeper questions about consistency, fairness, and accountability in AI-powered hiring – especially as these tools become more widespread and influential in determining who gets a job interview.