A few years of navigating the complex terrain of corporate America led me to a startling find: Underneath it all, organizations that deceptively seem worlds apart are eerily similar. More often than not, it all boils down to the leaders’ ideologies and the culture that executives and managers foster.
In organizations where honesty and fairness aren’t just buzzwords, you’ll find people in their element, producing their best work. Here, the stars who are bound for promotion are as easy to spot – their impact and dedication tell the tale.
In stark contrast, organizations missing a clear appraisal process can feel like a tricky maze. In these places, employees are grappling with a culture rife with politics, not unlike developing your best model on garbage data. Here, success isn’t about your technical acumen or your measurable impact. It’s about how well you can navigate the murky waters of interpersonal relationships and network effectively.
Data science teams are not immune from the ills that plague larger organizations in this regard. If anything, the complexity and diversity of data science tasks, projects, skills, and expectations make it even more critical to have a transparent and objectively measurable performance appraisal process.
So, let’s dive into what makes a solid performance review system for your data science team. First, we start with different levels in data science and the baseline expectations at each. Then we review the general appraisal process and team dynamics related to appraisal. Finally, we will wrap it up with common pitfalls of the performance review and ways to dodge the bullets.
Data Scientist Levels & Expectations
Like other technical disciplines, data science levels are usually arranged into 1) Individual Contributor (IC) and 2) management path. Let’s dive into the nitty-gritty of the IC roles first, and we’ll circle back to the management roles later. Onwards we go!
Individual Contributor Levels
A basic definition of IC roles is those without management responsibility. This reciprocal definition might be a bit naive because high-level technical leaders and architects might have more influence on technical teams than their direct supervisors; however, we could accept it as a working definition.
Generally, we can divide data science IC roles into three top levels. Junior, Senior or Lead, and Principal or Staff. The description of each level is presented below.
Junior Data Scientist
This level is usually called a data scientist, with the junior title typically omitted unless multiple stages are defined. Here, I use the term junior data scientist to emphasize the distinction. A junior data scientist can understand a fairly well-defined modeling project or an analytical question and use machine learning and statistics to answer it. They are familiar with most classical machine learning methods (e.g., supervised vs. unsupervised, classification vs. regression, linear and non-linear methods, test vs. train error, etc.) and can write decent code.
Usually, junior data scientists are fresh out of college or are transitioning between career paths. For instance, individuals with a background in academia, software engineering, or analytics may seek to break into data science. For this reason, a junior data scientist typically lacks extensive business background and may require guidance to understand business requirements fully.
Senior Data Scientist
Senior data scientists can translate typical business problems into well-defined data science problems and solve them. In addition to the skills of the previous level, senior data scientists typically have more than 2 to 3 years of industry experience. They are familiar with common statistical pitfalls (such as sampling biases and data quality issues) and know how to perform exploratory data analysis beyond what is commonly taught in academic or competitive settings (like Kaggle).
Staff / Principal Data Scientist
Being a staff (or principal) data scientist is the pinnacle of achievement in the technical track. A staff data scientist can identify projects with significant value to the business and spot gaps in the current data science infrastructure. They can design and deliver large-scale, multi-project initiatives with long-term impacts on the business.
In addition to having several years of industry experience (usually >5), they can independently communicate with various business counterparts such as product managers, engineering managers, and executives to understand the strategic business direction and propose technical solutions.
Data Science Management Levels
While the terminology may vary across organizations, management levels typically correspond to IC levels. The common roles and responsibilities for each level are described below.
Lead Data Scientist
The lowest management level, which is sometimes informal, is that of the lead data scientist. This role is often filled by a senior data scientist who has a couple of interns or junior data scientists reporting to them, either directly or indirectly.
In addition to the skills of a senior data scientist, lead data scientists possess soft skills such as the ability to extract business requirements from non-technical stakeholders, manage meetings, and deliver project presentations and training sessions.
Data Science Manager
A data science manager has direct responsibility for managing a team. They work closely with an engineering manager and a product manager to define and refine project descriptions, divide work among team members, and ensure each team member can deliver the expected results. They provide corrective feedback, help remove obstacles, and communicate challenges and achievements with stakeholders.
In smaller companies, a data science manager also assists the organization with strategic directions, similar to a director of data science. We’ll discuss these responsibilities in the next section.
Director/VP of Data Science
Titles such as Senior Data Science Manager, Director of Data Science, or even VP of Data Science are commonly used for higher levels of data science management. Typically, a Director of Data Science oversees multiple teams and works with various data science teams, machine learning engineering (MLE) teams, software development engineers (SDE), etc., to ensure that larger business objectives are met.
A director should understand the overall business strategy and align the direction of the data science teams accordingly. They communicate the team’s direction with external stakeholders, including customers. In mergers and acquisitions, the Director of Data Science may conduct technical due diligence to assess the maturity of machine learning models.
A VP of Data Science is a high-level management role that does not usually involve day-to-day tasks. However, it’s worth noting that in the banking and financial industry, the VP title is often used for the senior or lead level and should not be confused with high-level managers in other industries.
Performance Review Process
Even though performance reviews might seem like a chore to some managers, it is an ongoing chance to give your team members valuable feedback. We’re going to break down this process into three manageable steps: 1) Planning, 2) Progress Tracking, and 3) The Review Meeting. Let’s take a deeper dive into each of these steps.
Planning
Having a clear plan about roles and responsibilities provides the foundation for the performance review. This should start with the Job Description to gain a precise and shared understanding of the role. In small companies without a mature data science practice, it’s easy to overlook this step. In large companies, there’s usually a high-level strategic description of the role and levels. However, inexperienced managers sometimes struggle to translate it to the specific needs of the team and the role.
The goals of a job description are to describe: 1) the required skills that establish the current and next-level performance and 2) to make measurements as objective as possible. With that in mind, here are elements of a good role description:
- How the role is divided between projects, ad-hoc analysis, and maintenance,
- What the most critical projects are, their relative importance, expected objectives, timeline, and risks,
- Establishing baseline expectations for analysis and sharing examples of well-executed ones,
- Describing maintenance and on-call duties and their relative importance in performance reviews.
The dynamic nature of projects and analysis, especially in data science, limits the ability to predict the details of all projects and analyses. However, managers should strive to make order out of chaos and provide clarity on skills and expectations. These documents can also be updated as projects and roles evolve.
Progress Tracking & Frequent Feedback
The next important step in performance reviews is ongoing and frequent assessments of performance. This could be a part of one-on-one meetings with a weekly to biweekly frequency. Managers could utilize these meetings to communicate
- Expected project deliveries, general timeline, and technical and qualitative expectations,
- Collect evidence and analyze the performance of each team member using corresponding period activities and tasks, and
- Provide feedback on the current performance, the strengths, and achievements that should be celebrated and utilized for future success, as well as the weaknesses and skills to be learned.
Managers can ask each team member to share a short conversation summary to ensure expectations are aligned.
Review Meeting
If managers have done an excellent job providing ongoing feedback, the actual review meeting should just be a summary of previous notes. Nothing should come as a surprise in this meeting, as all important issues should have already been discussed.
Many companies use the 360-degree method, where peers, subordinates, and managers provide comprehensive feedback on each individual’s performance. When conducting these reviews, it’s essential to have clear and objective standards.
Team Dynamics, Culture, and Impact on Performance
Believe it or not, the way promotions are handed out can dramatically shape the vibe of your team, its culture, and, in turn, each member’s performance. Some companies play the lone wolf game – each person’s performance is evaluated individually, and promotions depend solely on their work.
On the flip side, other companies run a version of corporate Hunger Games. Promotions at each level are measured against everyone else in the company, and only a select few emerge victorious. This approach can ignite a fierce competitive spirit, but it might also damper team collaboration and knowledge sharing.
The path executes choose for performance reviews is based on their beliefs and dreams. Usually, the company’s HR team lays out the groundwork for how these reviews are conducted. But, as data science managers, it’s crucial for us to:
- Get a grip on how each review mechanism impacts our team,
- Brainstorm ways to lessen any negative effects,
- Strive for fairness and objectivity in our assessments, and
- Keep a clear, open line of communication about the entire process.
Let’s say promotions are limited at each level. In that case, we can set expectations accordingly, create knowledge-sharing spaces, and even design specific metrics that encourage sharing knowledge. Most importantly, we must ace the execution of performance reviews and sidestep common pitfalls, which we’ll get into in the next section.
Performance Review Pitfalls
Similar to any other human subjective judgment, performance reviews are influenced by our cognitive biases. It’s critical for managers to:
- Learn and become familiar with the cognitive biases that affect our judgment,
- Take corrective actions to mitigate the impact of these biases. In dimensions such as promoting diversity, these corrective actions can significantly improve the dynamics of the team,
- Establish objective measures as well as routinely collect evidence on performance. This can lead to reinforcing positive behavior and providing negative feedback when necessary,
- Be transparent and have the candor to discuss issues freely. Communicating negative feedback requires a certain amount of courage, but in a culture of growth and over the long run, everyone will benefit from it.
Moreover, receiving monthly or quarterly feedback on the entire process and your performance in the context of informal meetings (such as lunch meetings, bowling nights, etc.), can improve your abilities.
Summary
Performance reviews aren’t just a check-the-box task; they’re an ongoing journey that can help managers to cultivate a thriving team culture. When done right, fair and transparent performance reviews can dial down office politics and spark better personal relationships. Considering the complexity and diversity of tasks in a data scientist’s role, these reviews become even more important.
To nail a performance review, we need to first lay down the skills and expectations for each level of data scientist. This isn’t a one-size-fits-all kind of job! As managers, we have to take those company-wide guidelines and tailor them into detailed plans for our own team and the unique individuals in them. Armed with this plan, we can then offer frequent feedback and keep track of everyone’s strengths and areas for improvement. The final stage is the performance review meeting, which serves as a formal summary of all the discussions we’ve been having along the way.
Performance reviews can either nourish the team spirit or spark individual competition. They can also instill a sense of fairness and objectivity. As managers, we need to be aware of our own biases and personal preferences and devise a plan to limit their influence. By keeping these points in mind, we can make performance reviews a valuable tool for growth and improvement.
Be First to Comment