Will data & AI practitioners thrive at your organisation?
If you think the answer might be no, here are a few ideas to set you up for success
Long before LLMs were white hot, organisations of all shapes and sizes had a problem. They would hire ‘data people’ into the org, often after a fairly hefty amount of effort and expense, only to find those people less impactful than they had hoped and in many cases to see them churn out again within 6-24 months, oftentimes with bad feeling on all sides.
There is no silver bullet solution to this problem, and I’m not suggesting that this is only a challenge in the data space. But, data/ML/AI is a very young profession and has, for the past decade at least, had a fair amount of hype surrounding it, so I believe there are some specifics that are worth digging into. It has also been the through line of my commercial career to date (that and hard problems!) so it is an area where I have a lot of learnings (forged in the unforgiving crucible of production deployment and team creation) and hence, something to add.
So what steps can you take to make your organisation more likely to succeed in the data/ML/AI space in terms of retaining great practitioners and attracting more?
✅ Genuinely embrace the fact that without high quality data and an organisational wide focus on capturing and generating consistent, high quality data, your prospects are dim in the medium to long term. Generative AI has not changed this. Hero teams with good data engineers can turn any crappy data into an impressive PoC or even a seemingly fabulous first product launch but the effort to maintain all the data ‘fixes’ that were needed to get there will gum up the works and break the team. This is the biggest reason for brittle data & AI products that take longer than anticipated to launch and are far slower than expected to iterate and change. This cannot be made ‘the data/AI team’s’ problem - unless you want a data/AI team that grows rapidly and ends up at logger heads with the rest of your product and engineering crew. Everyone who touches data has to care about data. And in many organisations that will be the vast majority of the workforce, factoring in the data capture that occurs across marketing, sales, finance, front line workers and operations and the form design and database structuring that occurs in product and engineering teams.
✅ Uplift your legal capability (internally or by engage with external council) to include at least one senior practitioner with a strong working understanding of the data, privacy and AI regulation that is applicable to your industry in all the jurisdictions in which you operate. You may want patent protection of your AI products, you will definitely want to have robust data use coverage in your T&Cs and, if you are negotiating contracts or partnerships with big, data-savvy corporations, a legal-data team pairing who are on the look out for clauses that tilt the playing field to an unplayable angle.
✅ Make sure someone with decision making authority and sway over the distribution of money in your organisation has a genuine, hopefully practitioner level understanding of how data and AI products are built. If change has to percolate up through layers things will go slllllloooooooowwwwwwly and your great folk will either get very demotivated or leave. I mean, your CFO can read a balance sheet and your legal counsel was originally trained as a lawyer right?
✅ Be pragmatic about what your company, with the investment you can make, can achieve with the technology and data available today. There are few things more annoying to great practitioners than unrealistic expectations coupled with an unwillingness to understand why those expectations are unrealistic! Perhaps you need a good story for the investors (‘cause everybody talks smack about AI right now?!’) but keep it real with your team.
✅ Apply AI where it makes sense. That means taking seriously the idea that AI will never be 100% correct (see above point about being pragmatic about working the AI actually available, not the glamour vision of what the yet-to-be released frontier foundational model or agentic AI will provide). So choose and support the use cases where your business model and workflows can afford for the AI to be WRONG a certain amount of the time.
✅ Turn your great practitioners into talent attractors. Great people like to work with great people so don’t hide your folks under a bushel. Coax and support them into writing, speaking and contributing to the community, so you can become known as a place that treats data seriously and treats people well.
Lean into the above and you will put yourself in the top tier of workplaces. It won’t be easy but if data & AI are vital to your future thriving, it will be worth it.
I’m going to argue that what is important is not for “Data & AI” practitioners to thrive, but for their work to be well integrated and for them to collaborate well within the organisation. Over the course of my career “AI” has drastically changed. As a newbie out of university people that understood neural nets were regarded as distinctly “odd”, by the mid 2000s that had changed to “we need more data scientists”. Today I think we are well on the way to seeing ML skills as just another skill set to integrate into your team.
Just as we have evolved from having teams of front-end and back-end engineers to integrating all skills, including UX, into our product teams. So we are also evolving to including ML, AI skills into our teams. It is just the way you need to develop compelling products.
Sure we might need a team dedicated to building a new model (a complicated subsystem team in TeamTopologies language) that serves the main product. But I think our thinking has evolved. I dont see “data & AI” skills as anything special. They are equal members of the team.