The AI Playbook

Chapter 2: People

Summary

  • In AI, job titles vary and can be difficult to interpret. We describe characteristics and salaries for six key roles: Data/Machine Learning Engineer; Data scientist; Machine Learning Researcher; Head of Data; Head of Research/AI; and Chief Scientist/Chief Science Officer. For each, individuals’ capabilities vary across competencies in research, engineering, production and strategy.
  • The composition of your team should depend upon the problem being solved and your approach to doing so. It is advisable, however, to avoid hiring solo talent. Begin with a small team, and ensure you have a robust AI strategy in place before expanding your AI personnel.
  • We suggest team structures, first hires and next steps for six scenarios: “I want insights into internal data”; “I want to implement third party AI APIs”; “I want to outsource AI development”; I want to create bespoke AI models”; “I want to use a combination of bespoke and third party AI”; and “I have an idea that’s cutting edge.”
  • Recruiters, conferences and universities are primary sources of talent. Traditional recruitment agents find it difficult to screen AI candidates, so engage with specialist recruiters. Conferences and meetups are powerful vehicles for talent acquisition; be active in the AI community, attend and speak at conferences, and grow your network to discover capable candidates. Engage with universities; post on their job boards, establish partnerships and pay for projects to engage students who may seek future opportunities with you.
  • Diversity delivers economic value and competitive advantage. Review the culture in your company, AI team and hiring practices to ensure diversity, representation and inclusion.
  • An effective job description should emphasise projects (the nature of the engagements on which the successful candidate will work), skills and impact. Most data scientists seek work that will ‘make a difference’. To attract talent, demonstrate how the successful candidate’s work will do so.
  • When hiring, prioritise adaptable problem-solvers. In addition to having role-specific and technical skills, a strong AI candidate will: understand available tools to enable rapid research and development; appreciate when to release an imperfect solution and when to wait; and communicate and collaborate well.
  • Optimise every stage of your recruitment funnel. We provide best practices for: CV screening; phone screening; technical testing; face-to-face interviews and post-interview follow-up.
  • AI talent is in short supply. Challenge, culture and company are key for retention. In addition to an attractive financial package, consider: offering flexible working hours; offering challenging problems and minimising drudgery through automation; creating a culture in which diverse ideas are shared; avoiding ‘lone workers’; ensuring your AI team receives recognition for its work; and supporting team members’ publishing and presentation of work.

People: The Checklist

Structure your team effectively

  • Clarify the problem to be solved
  • Identify a development strategy and associated hiring needs
  • Understand the six core roles in AI teams
  • Shape your team to reflect the competencies required
  • Structure your team to avoid lone workers

Optimise your hiring process

  • Understand the role, seniority and requirements for which you are hiring
  • Develop a clear job description
  • Leverage recruiters, conferences, meetups, universities and investors
  • Embed best practices for screening, testing, interviews and follow-up
  • Identify adaptable problem-solvers
  • Recruit from diverse backgrounds

Invest in retention

  • Maintain an inclusive and diverse culture
  • Automate menial work
  • Offer intellectually challenging problems
  • Ensure the AI team receives recognition
  • Support team members’ publishing efforts

If you are building an in-house AI team, whether directly or via recruiters, it will be important to understand the roles you require and how to attract, deploy and maintain talent. Below, we provide a blueprint for structuring, building and retaining your AI team.

Hire for required competencies in engineering, production, research and strategy

Because AI is an emerging field, job titles vary and can be difficult to interpret. Further, people may describe themselves in different ways to market themselves for roles they want. There are at least six core roles in AI. In each, individuals’ capabilities vary across competencies of research, engineering, production and strategy (Fig. 5).

Data Engineer/Machine Learning Engineer:
  • understands data and can code AI models that are derivatives of systems already created
  • focuses on engineering (creating code for applications and solutions to go live), not research
  • create models but may lack finer understanding to push the boundaries of research.
Data Scientist:
  • focuses on obtaining insight from data using scripts and mathematical techniques; manipulates data in a variety of programming languages for solutions to specific problems;
  • typically has an academic background to PhD level
  • stays current on contemporary research and be capable of implementing ideas from academic papers
  • may lack wider development and AI skills, including understanding of the needs of live systems; typically produces reports, not applications.
Deep Learning Researcher/Machine Learning Researcher:
  • focuses on research, not building business applications
  • highly academic, typically with post-doctoral academic experience
  • seeks to push the boundaries of technical solutions
  • will have limited or no exposure of taking their work to the level of a live application.
Head of Data:
  • understands the nuances of varying data sets and is sufficiently experienced to lead a team
  • technically hands-on; works with her team to produce reports and applications
  • may have data strategy responsibilities (responsibility for acquiring, managing and deriving value from data).
Head of Research/Head of AI:
  • research-focused and with enough experience to to lead a team
  • technically hands-on; may be sufficiently experienced to support the conversion of team output into live applications
Chief Scientist/Chief Science Officer/VP of AI:
  • extensive experience in business as well as AI
  • determine AI strategy and production pipelines; work with the Chief Technology Officer (CTO) to ensure the company’s AI strategy can be executed
  • typically report directly to the CEO
  • experienced as a board level strategist.
Fig. 5. Roles vary across competencies of research, engineering, production and strategy
Role Research Engineering Production Strategy
Data Engineer Low High High Low
Data Scientist Medium Medium Low Low
Researcher High Low Low Medium
Head of Data Medium Medium Medium Medium
Head of AI High Medium Medium Medium
Chief Scientist High High Medium High

Source: MMC Ventures

Structure your team according to the problem and your approach

Salary range and job expectations will vary according to an individual’s role (Fig. 6).

The composition of your AI team should depend on the problem being solved, your team’s approach to doing so, and the level of integration required with your development team to support production. Bear in mind, however, the following principles:

  • Do not hire solo AI talent, beyond an initial ‘Head of…’ role.
  • AI professionals rely on collaboration for ideas and can feel isolated if they are a sole member of a larger team.
  • Begin with a small team to validate the data and your team’s ideas, regardless of the domain.
  • Ensure you have a robust AI strategy in place (Chapter 1) before you expand your team.

“Structure your team to avoid lone workers. AI professionals rely on collaboration.”

Fig. 6. The expectations and costs of AI professionals differ
Job title Salary range (£ thousands) Job expectations
Data Engineer 45 – 90
  • Directed problems
  • Full access to data
  • Create solutions that are deployed live
Machine Learning Engineer 60 – 90
  • Challenging problems
  • Mix of APIs and models
  • Create solutions that are deployed live
Data Scientist 45 – 90
  • Generate insights from data
  • Create models
Machine Learning Researcher 60 – 100+
  • Find new methods
  • Solve complex AI problems
  • Create new models
Head of Data 80 – 120
  • Own data strategy
  • Run the data team
  • Manage projects
Head of AI 80 – 120
  • Define approaches
  • Run the AI team
  • Manage projects
Chief Scientist 110 – 180+
  • Develop and deliver AI strategy
  • Board membership
  • Autonomy

Source: MMC Ventures

“I want insights into internal data”

  • Strategy: Build an in-house data science team. Sensitive data will not leave your company and you can control the focus and outputs of your team.
    • First hire: A Head of Data reporting to your CTO.
    • Next step: Hire two or three data engineers or data scientists, or a combination, depending on the needs of the project.
    • Success factors: These individuals will need time to understand your data and how it’s gathered. Ensure they have access to all the data they need.

“I want to implement third-party AI APIs”

  • Strategy: You will need individuals who understand your data and have the knowledge to implement and test third party APIs. If you have no budget to hire, an alternative approach is to find existing developers within your organisation who understand data well enough to manage the API integrations.
    • First hire: Two machine learning engineers.
    • Next step: For smaller projects, your engineers could report into the CTO or Head of Development. For larger projects, you may wish to hire a hands-on Head of AI as a team lead, to support the expanding team.
    • Success factors: Review the available APIs and validate that they will address your use cases. Ensure you plan for changes to the APIs in future.

“I want to outsource AI development”

  • Strategy: You need an individual who understands your project well to manage the outsourced relationship.
    • First hire: A Head of AI, if you don’t already have a suitable person within your team. A Head of AI can also enable you to bring the solution in-house, over time, if you choose to do so.
    • Next step: Empower your Head of AI to manage project costs and timelines. Ensure you receive regular status reports for clarity on each delivery cycle.
    • Success factors: Successful AI requires continuous feedback and iteration. Develop a good relationship with your AI provider and agree costs for updates up-front

“I want to create bespoke AI models”

  • Strategy: You are undertaking something unique with your AI solution and wish to keep your data and system knowledge in-house. This is the most common scenario for companies.
    • First hire: A Head of AI, or Chief Scientist, reporting to the CTO.
    • Next step: Let the Head of AI, or Chief Scientist, determine your strategy based on the problems you wish to address.
    • Success factors: Allow budget for at least four further hires – more for larger projects. These hires will be a combination of Data Scientists and Machine Learning Engineers. You may need a Machine Learning researcher as part of this team – but ensure they are challenged enough.

“I’m going to use a combination of bespoke and third party AI”

  • Strategy: You seek a fast start and third party APIs deliver enough for your minimum viable product. However, you want to develop bespoke AI in parallel, to deliver a unique value proposition.
    • First hire: A Head of AI or Chief Scientist, plus at least two Data Engineers or Machine Learning Engineers to undertake the API work.
    • Next step: Hire two or more Data Scientists.
    • Success factors: Unless your hybrid approach involves examining research, avoid Deep Learning Researchers.

“I have an idea that’s cutting edge”

  • Strategy: Validate that your idea is feasible – as well as the problems and timelines associated with it.
    • First hire: A Head of Research or Chief Scientist; two to three Deep Learning Researchers; plus potentially a Data Engineer to support them.
    • Next step: If required, expand the team with Data Scientists to balance the team’s skill-set.
    • Success factors: Manage timelines closely and be prepared to assess when research isn’t progressing to plan and alternative solutions should be explored. Maintain focus on the goal for the research and avoid research for its own stake.

Recruiters, conferences and universities are primary sources of talent

Unless you are a known company, advertising on your own website is unlikely to be effective. Alternative sources include:

1. Recruiters: Specialist recruiters exist for AI and data science. Poll your network to identify them. Unlike traditional development roles, recruitment agents find it challenging to screen AI candidates due to the research-intensive, data- specific problems they tackle. High quality AI recruiters have extensive networks of candidates, can identify candidates that fit your needs, and can save you more in the cost of your time than they charge in fees (typically 10%-45% of annual salary, plus bonuses).

2. Conferences and Meetups: Conferences and meetups are a powerful vehicle for talent acquisition. Be active in the AI community and grow your network. Every major city has an AI network you can join and there are conferences throughout Europe almost every week of the year. Many conferences offer job boards, in addition to which you can meet individuals at the event and undertake an initial screen. Even if timing does not align between you and potential candidates, making connections is valuable and you will become known as a potential employer.

Consider speaking at events – it’s easy to include a “we’re hiring” closing slide and you may receive extensive interest. Some conferences require sponsorship for a speaking slot. This may be appropriate if you are discussing your general solution in a talk close to a sales pitch. You should never have to pay, however, if you have an advancement that will benefit the community. Submit your paper to an academic conference or one of the many high quality events that avoid sponsored talks and seek excellent speakers and and topics.

Academic conferences – including NeurIPS, ICML, ICLR and AAAI – are busy events at which larger employers are highly active. These conferences are expensive and smaller companies can be overlooked. They are, however, valuable events if you seek exceptional researchers.

Other conferences, including RE•WORK and M3, focus on the intersection of academic and business applications

These can be excellent environments in which to meet individuals who wish to move from academia to industry, as well as being stimulating environments for general conversation and ideation.

Local meetups are even less formal and many offer events specifically for hiring. These are an excellent option if your team has not spoken before or if you wish to receive feedback on your approach. If there is not a meetup near you, start one.

3. Universities: If you are geographically close to a university with a strong AI department, or have alumni connections to one, the university may allow you to post on its job boards. While roles will be focused on students soon to graduate, alumni can also see the university’s digital job boards and your role may be passed around networks
of AI practitioners.

You may also be able to work in partnership with a university by paying for projects. Typically, the head of a laboratory will accept a project for a fixed cost and the project will serve as a task for graduate students. Exercise care with this approach; the university may be motivated by gaining IP and publications, so ensure your agreement is appropriate. Students who have worked with you on a part-time basis in this manner are more likely to seek future opportunities with you. Similarly, there is a growing trend for Masters and PhD students to work part-time alongside their studies. If you can offer flexibility in this regard, you may attract exceptional candidates who are not seeking full-time work.

Many universities offer excellent AI programmes and candidates. Examine their research pages and identify labs working on problems similar to yours. A small sub-set of universities with high quality AI programs includes:

  • UK: Bristol, Cambridge, Edinburgh, Imperial, Manchester, Oxford, Sheffield, Sussex, UCL.
  • USA: Carnegie Mellon, Harvard, MIT, Stanford, Yale.
  • Canada: Montreal, Toronto.
  • Worldwide: EPFL (Switzerland), Nanyang (Singapore), Politecnico de Milano (Italy), Technical University of Munich (Germany), Chinese University of Hong Kong.

4. Investors: If you have secured investment, leverage your investors. Ask them to introduce you to other AI- led companies in their portfolio so you can share ideas and recruitment opportunities. There may be excellent candidates who are no longer a fit for other companies – for example, due to a relocation – who would be ideal for yours.

5. Job Boards: Job boards can be effective at attracting applications. However, with a public posting on a generic job board you are likely to receive numerous applications that are poorer in quality or fit. The time, and therefore cost, required for an individual in your company to review them can be considerable. Specialist AI job boards, including those on Kaggle (www.kaggle.com/jobs) and StackOverflow (https://stackoverflow.com/jobs?q=AI) are typically searched only by people already part of these communities and typically offer higher-quality candidates.

Diversity delivers economic value and competitive advantage

Many problematic and embarrassing AI systems have been developed because the teams that created them lacked diversity. Without different perspectives on data and results, you may create systems that offer narrow appeal and broad offence. Review the culture in your company, AI team and hiring practices to ensure diversity, representation and inclusion.

Further, in a competitive market for AI talent, leadership in diversity will enable you to attract exceptional candidates who may otherwise have overlooked your position – or been minded to accept an alternative.

Job descriptions should emphasise projects, skills and impact

When hiring, ensure you understand the role, seniority and the minimum requirements for which you are hiring. If your team uses Python exclusively, for example, do not hire someone who only works in R or Matlab. Missing skills will impact your costs directly, as individuals take time to address gaps. Describe the projects on which the successful candidate will work. Do they relate to computer vision, language, prediction, generation or other? Use the industry-standard terms of classification, regression, generative AI, sequencing and clustering for easier comprehension. Describe the expectation for the role as well as the difficulty of the problem.

You will also need to sell your company and the domain. Most data scientists seek work that will ‘make a difference’. To attract talent, frame your problem to demonstrate how the successful candidate’s work will do so.

Fig. 7. Example job description
  • Title: DeepLearningResearcher
  • Role: Working on exclusive medical imagery, you will be solving classification and generative problems beyond the current state of the art.
  • Team structure: As part of a dedicated AI team reporting to the Chief Scientist, you will work closely with Machine Learning Engineers who manage the AI production pipeline.
  • Skills: Python 3.6, including numerical libraries; Tensorflow; Keras.

Source: MMC Ventures

When hiring, prioritise adaptable problem-solvers

Hiring exceptional talent is challenging. While it can be tempting to prioritise mathematics candidates with first class degrees, people with the largest number of academic papers, or those with the most appointments, consider your company’s needs. Exclude individuals with poor communication or collaboration skills, and people who cannot adapt to the fast pace and fluid nature of industry. In addition to role-specific skills, a strong AI candidate will:

  • have sufficient technical skills to solve AI problems
  • understand available tools to enable rapid research and development processes
  • appreciate when to release a solution, even if it is imperfect, and when to hold back a release
  • communicate and collaborate well.

Seek adaptable, intelligent problem solvers – not individuals limited to following TensorFlow tutorials. Are you recruiting an individual to undertake research, or do you need someone to obtain insights from data? Depending upon the problems to be solved, limited academic experience may be unproblematic if the individual possesses required skills. Individuals may have gained experience through alternative initiatives, including hackathons and competitions.

While the interview process for AI roles is similar to the process for other technical roles, there are differences. A conventional developer, for example, would undertake a technical test at a face-to-face meeting. AI candidates cannot demonstrate their ability to build an AI model at an interview given the time constraints. Involve existing members of your AI team throughout the process. The best candidates will complement existing ideas while bringing something new to your team.

Optimise each stage of the recruitment funnel

1. CV screen
  • Maintain your specification for minimum skill-set. While no candidate will be perfect, identify your red lines and do not waste candidates’ time, or yours, taking the wrong individuals forward.
  • Prioritise candidates who have stayed over a year in past roles; it can take months for a new team member to understand the data specific to your business.
  • Evaluate candidates’ ability to contribute to your business above their academic experience, even for research posts. An individual with a decade of experience researching an obscure problem may not adapt to your natural language challenge.
2. Phone screen
  • Identify candidates’ passions and motivations; evaluate if they will be a good fit for the projects you have planned.
  • Ask candidates about their contributions to previous projects and seek individuals who can explain their contributions clearly.
  • Let the candidates ask you questions for half the available time; good candidates will want to know as much about the company, team and projects as you will about them.
  • Progress only the candidates who can demonstrate their passions and exhibit good collaboration and communication skills.
3. Technical test
  • A technical test (Fig. 8) will be expected but it is important to be reasonable. Do not set a task that requires more than four hours to complete; candidates’ time is limited and you should not take advantage of them.
  • Offer a problem representative of the work they would undertake in your team, with (subject to privacy constraints) real data. Ideally the problem should have a trivial solution, which will highlight individuals who do not consider data complexity.
  • Technical tests should be specific to the candidate; individuals can upload information about your tests to websites, such as Glassdoor, which can give candidates an unfair advantage. Similarly, recruitment agents may prime their favoured candidates with the problem upfront. Kaggle.com can be an appropriate environment in which to run technical tests.
  • For research-based roles, implementing code from an academic paper may be a suitable test.
  • If a candidate offers a solution which demonstrates that the candidate thinks beyond the trivial, invite them for a face-to- face interview.
Fig. 8. Example technical test

”Write a script to identify and remove duplicates in the following data set.”

The candidate is given a set of 50 frames from a video. Some are identical; some close; some have the same composition but different subjects; and some are unique.

  • A strong candidate will understand that this is a data preparation problem and will consider the impact of this data on training or testing a model.
  • A trivial solution, indicative of a lack of experience, would be to identify identical images.
  • A better solution would group the images, identify a dominant image from each group as an output, and discard the rest.
  • An excellent solution would understand that the object of the images may be important and provide a script in which the important characteristics could be selected.

Source: MMC Ventures

4. Face-to-face interview
  • By this stage you should have a small number of exciting candidates.
  • Discuss each candidate’s technical test. Can they critique their own solutions? What would they do given more time? These questions will provide insight into how candidates think and plan their time.
  • Add a thought experiment with extension challenges: how would the candidate solve a problem in sub-optimal circumstances? What if there are large gaps in available data, or if data quality varies? What if the business required a 50% improvement in predictive speed? Thought experiments will enable you to understand a candidate’s creativity and how they will perform in a dynamic environment. If a candidate can only follow steps described in AI tutorials, their impact on your business will be limited. Similarly, exercise caution with candidates who express annoyance when faced with changing business requirements, or who advocate long timelines for any change. They may not have the skills and temperament to thrive in an early stage company.
  • For research-led roles, or where you seek a candidate who will digest state-of-the-art academic papers, ask the candidate to bring and present a recent paper written by someone else. Evaluate whether the candidate can explain another person’s concepts in simple terms. Invite an interested non-technical person to join this part of the interview and ask simple questions. Screen out candidates who cannot communicate the work to the wider business, or become frustrated with simple questions.
  • Discuss, in as much detail as possible, upcoming projects. See how excited the candidate becomes and prioritise candidates that are eager and propose solutions.
5. Post-interview
  • Provide immediate feedback and, if you still have other candidates to consider, manage expectations regarding a decision.
  • AI practitioners are data-led and do not appreciate uncertainty. Come to a decision promptly and make an offer quickly.

Challenge, culture and company are key for retention

AI talent is in short supply. When you have attracted high quality professionals to your team, it is important to retain them. While an attractive financial package and benefits are necessary, large companies can – and will – offer higher salaries. Retain AI talent by catering to team members’ other needs:

  • Offer flexible working hours. AI models can take a long time to run; if they finish at night or during weekends, good candidates will want to re-engage and alter parameters.
  • Offer challenging problems and minimise drudgery. Steps that can be automated should be. Ensure your team has support from other parts of the business to do so.
  • Ensure your team has appropriate hardware. Crippling your team to save £1,000 is a false economy.
  • Create a culture in which intellectual debate is encouraged and diverse ideas are shared. Advances in AI are the result of multiple scientific fields bringing different perspectives to the same problem. Individuals with different backgrounds and education see things differently; combining their ideas will present novel solutions. An environment in which all opinions can be voiced and debated will enable your AI team to solve problems faster and motivate your team.
  • Regardless of level, ensure your AI team comprises more than one person. The ‘lone AI worker’ is a frequent challenge for early stage companies. The scientific nature of AI instils a need for collaboration and the testing of ideas. While it can be exciting for an AI team member to be the first in your company, and to develop your company’s prototype, months of solo work on your company’s AI initiative, even amidst collaboration with your broader team, can be intellectually isolating and drive attrition.
  • Ensure your AI team receives recognition for its work. If individuals have worked for months to develop an effective AI model, it will be dispiriting for the team that adds the front-end to receive sole credit.
  • Decide early your approach to intellectual property that projects will produce. Ensure your team understands whether there is a patent strategy, if team members may publish results, and if they can present them at conferences. Many AI practitioners have academic careers they wish to sustain. If your company can provide support to their efforts to publish and present, it will be deemed a benefit. Balance this, however, with an understanding that developing research to the standard of an academic paper may require further work that will not benefit your company.