Common pitfalls for AI sourcing automation

Using AI to automate the sourcing process isn’t as simple as pasting a LinkedIn profile or resume into ChatGPT. One of our early product iterations at Stardex was an AI copilot that automated sourcing on top of LinkedIn Recruiter. Here's the 3 main gaps between expectations and reality.

The biggest challenge in automating the sourcing process is the “context gap” between the human recruiter and the AI system. There’s lots of industry specific context that needs to be specified for an AI model to truly perform at a high-level for sourcing. For example, we’d often have customers looking for folks with startup experience. This seems easy enough, but AI would often categorize someone at, say, Stripe as having startup experience. Most folks in the industry wouldn’t necessarily view a candidate that joined Stripe a couple years ago as having truly worked at a startup, so you have to specify what specific funding stages are relevant explicitly to the AI. We’d see similar gaps in context with industry-specific job titles as well. Another context gap involves implicit criteria—knowledge you’ve learned intuitively through experience but you find hard to fully articulate or enumerate all the possible knowledge. For example, you might place extra weight on a designer’s background if they spent a meaningful portion of their early career at Airbnb, particularly if you’ve previously worked with top design leaders from that environment. If you coach a junior recruiter, they'll remember this. As Stardex has evolved, we’ve begun storing these bits of context as “learnings” in the platform so the system keeps getting closer to your specific definition of the perfect candidate.

Next, we also have to solve for AI’s “data gap” in the industries and verticals we care about. This became especially apparent with granular company data (for example, stage or revenue). We’d have customers that cared about VP of Sales that joined a series A startup and led it to over $50M in revenue or a Head of Talent that scaled a company from 50 to 200 people, for example. No off-the-shelf AI models will reliably be able to reason over granular data like this, because the data is often not publicly available and therefore won’t be in the model’s training set. Instead, we began using data providers to create an enriched version of the candidate’s profile that goes way deeper than their LinkedIn profile or resume. All of Stardex’s automated workflows, from auto-tagging to auto-sourcing, use this enriched profile to bridge the knowledge gap.

Once you’ve solved for the context and data gap, you still have to contend with the “relevancy gap” of actually presenting the best candidates to the recruiter. It’s great if a candidate truly meets all your requirements, but it’s not always quite that simple when dealing with an AI system. A candidate may only meet some requirements, but are those the most important ones? What if they meet most criteria but meet a disqualifying criteria, like job hopping? What if they have 4.5 years of experience instead of 5? To account for this, the system has to view requirements being on a spectrum of importance and having some gray area in their definition. Due to AI models being probabilistic and not always following a long list of instructions, we’ve designed our system to evaluate criteria individually and then report a “yes”, “no” or “almost” for a given criteria. All of this gets fed into a scorecard for the candidate.

We cut our teeth on the problems above to build the most nuanced, knowledgable, and relevant AI sourcer on the market as part of Stardex. At the end of the day, these candidates need to be tracked in a pipeline and actioned upon as well, which is where the rest of our autonomous system shines. A lot of the above problems will be solved over time as the system that stores the data learns from your feedback, and the models get better. Tribal knowledge will still be not replaceable by any AI, so we believe that's where recruiters will differentiate as these systems evolve. If you find yourself wishing AI could speed up sourcing, we’d love to chat and get your feedback on our approach!