Tackling Subjectivity in Product Manager Interviews
Have you ever considered whether those interviewing candidates are adequately trained? Even if they are experienced, there's always the possibility of bias.
Watch Podcast Episode 12: Design That Drives Results: Insights from Olha Bahaieva, Designer Unicorn
With over 10 years of experience crafting user-centric experiences, Olha shares her wisdom on the core principles of user-centered design and why it's crucial to avoid trends that hinder usability. She'll outline strategies for fostering clear communication with designers to achieve your business goals, along with practical ways to incorporate user research into your product development, regardless of your budget…
Listen now on Spotify, Apple Podcast, X and YouTube.
Tackling Subjectivity in Product Manager Interviews
Product management interviews, particularly in BigTech companies like Meta, Microsoft, and Google, are known for their rigor and complexity. These interviews often encompass three core areas: ‘Product Execution’, ‘Product Sense’, and ‘Product Leadership’. Each area is designed to thoroughly assess a candidate's skills in managing and leading product development. For example, Amazon employs the S.T.A.R. Method, focusing on candidates' experiences, leadership abilities, communication skills, and depth of understanding. This methodology is indicative of the industry's high standards, emphasizing not only technical acumen but also strategic thinking and leadership qualities. Such approaches are critical in identifying candidates who can navigate the multifaceted challenges of product management in these innovative and fast-paced environments.
Before embarking on this essay, I had reservations about the relevance of its content. However, after some contemplation, I realized the importance of addressing the intense preparation that aspiring product managers undergo for their interviews. In today's competitive job market, especially in the tech industry, candidates often engage in rigorous training, including participating in over a hundred mock interviews. This extreme level of dedication is not just about mastering the interview but also about adapting their responses to fit various interview scenarios or prompt given by the interviewers. Their goal is clear: to not only pass the interview process but to excel in it, thereby securing a coveted position in a prestigious company. This preparation process reflects the high stakes and the demanding nature of product management roles in leading tech firms.
Assessing the Assessors
Have you ever considered whether those interviewing candidates are adequately trained? Even if they are experienced, there's always the possibility of bias. Subjectivity can also play a role. For instance, consider a scenario where an interviewer asks a candidate, "How would you go about designing a gardening product for Meta?" Imagine the candidate ticks all the boxes, follows the entire framework (mission, goal, segmentation, problem, solution, metrics, etc.), and applies highly creative and unconventional thinking, yet still faces rejection. The most disheartening aspect is that the candidate often does not receive feedback on how to improve for future opportunities. The lack of constructive feedback is especially disheartening. After all, interviews don't always predict on-the-job performance, just as an individual employee's contributions don't guarantee a tech giant's success (unless they hold a leadership role).
Is Underlying Affinity Bias Present in Hiring?
Here's a perspective based on my interactions with aspiring and seasoned product managers through LinkedIn connections and mock interviews. From my experience, these candidates often perform well but still end up not securing the job. They are typically dropped during the 'Product Sense' or the final round, perhaps due to not answering an analytics question 'correctly'. But what does 'correctly' really mean? Even if a candidate takes the right approach, challenges the interviewer, and seemingly does everything right (a common sentiment among product managers), they still might not be selected.
This leads me to theorize the presence of underlying Affinity Bias in some form during the hiring process. When a candidate is shortlisted for an interview, several factors come into play:
Merit – Overall work experience and industry knowledge.
Focus on Prestige – Favoring past experience at big-name companies.
Hiring Manager Match – "Do they think the way I think"
Subjective Culture Fit – Assessing how a candidate's values, work ethics, and style align with the company's culture and ethos.
Consider this: a candidate with past experience at Meta and a candidate with equally relevant experience at a startup or public company (less well-known) are both interviewing at Google. It often seems the Meta candidate receives the offer.
Another prevalent bias is the Halo and Horn effects, where one positive or negative trait unduly influences the overall assessment of a candidate. This can lead to overlooking more qualified candidates or unfairly dismissing competent ones based on a single perceived flaw. The Halo and Horn effects demonstrate how subjective impressions can skew the hiring process, leading to suboptimal team composition.
Beyond Self-Improvement: Tackling Bias in Product Manager Hiring
Everyone does this already – deal with the issues mentioned above by focusing on self-improvement, practicing mock interviews, signing up for product management courses, etc. These efforts are commendable, but how do we truly transform the hiring process for product managers? In the world of AI, there must be a better way to hire without ghosting candidates, providing meaningful feedback for growth, and minimizing subjective assessments. I don't have all the answers, but I believe our tech industry – or a bold startup – must develop a solution that revolutionizes this process.
This lack of clarity raises serious questions about the fairness of the hiring process. While a complete overhaul might be necessary in the long term, there are tangible steps companies could take today to mitigate these issues. Here are a few examples:
Structured Feedback Rubrics: Creating detailed, standardized feedback checklists for interviewers would promote consistent evaluation across candidates. This limits subjective interpretations and ensures candidates receive focused, actionable feedback.
Mandatory Bias Training: Raising awareness of unconscious biases, like Affinity Bias or the Halo and Horn effects, is crucial. Mandatory training programs for hiring teams could help interviewers recognize and mitigate their own biases.
These interventions could significantly improve the current process. However, if the underlying issues of subjectivity and bias persist, a more drastic change might be needed. This leads me to propose; What If We Had Standardize Interview Product Management Organization?
Standardize Interview Product Management Organization
Imagine a system where all product management candidates undergo a standardized assessment based on their experience level. Candidates would pay a fee for this service and participate in interviews with multiple interviewers from various companies. These interviewers would be randomly selected from a pool of qualified individuals who choose to participate in the program.
The assessment would evaluate candidates across key areas critical to product management success: analytics, execution, design sense, and leadership. Candidates would receive a score that would be valid for a predetermined period, say four months. This score could then be used when applying to Big Tech companies, potentially reducing the interview process to just one or two focused discussions on cultural fit and industry-specific details.
Benefits for Both Sides:
Reduced Redundancy: This system would significantly streamline the hiring process for both companies and candidates. Candidates wouldn't have to repeat the same interview questions for every company, saving them significant time and effort. Companies would benefit from a pre-vetted pool of qualified candidates, reducing their need to invest extensively in initial interviews.
Objectivity: Standardized assessments offer a more objective measure of a candidate's skills compared to potentially subjective, company-specific interviews. This could help level the playing field for candidates from non-traditional backgrounds or smaller companies.
Rethinking the Process
Let's be honest, discussing potential flaws in the hiring process can be a bit of a downer, especially in an industry built on disruption. It's easy to assume merit wins the day. But when talented people keep getting the same vague rejections, it makes you wonder... what are we missing? Could there be better ways to find the right product managers? I'm not saying the system is completely broken, but maybe it's time to examine it more closely and see if innovation is needed here too.