Edward Kim on data & AI in 2024 and beyond

"Nothing beats looking at the data directly. I don’t expect that this aspect of AI will ever go away: to a first-order approximation, the model is the data."

I first met Edward back in mid 2020 when we were lucky enough to bring him into Xero to anchor our machine learning team in Toronto. A first class brain inside of a first class human, he is someone who lifts any team, any discussion with his curiosity, humanity and his inherent orientation on being personally better at stuff tomorrow than he was today, based on everything he will have the chance learn over the next 24 hours. Edward is currently building things at Cohere, when he isn’t at the University of Toronto where he serves as an adjunct research professor at the intersection of materials informatics and LLMs!

Edward, 2024 has been another busy year for data & AI. What’s one development / milestone / news story that really caught your eye?

For many years, “AI hardware” was synonymous with GPUs (and TPUs for some). To me, 2024 feels like the year where the momentum on changing this view has been picking up. Nvidia is still dominant, naturally, but players like Groq, Etched, Cerebras, and many others are coming into the space with hardware that is extremely tailored for AI and tons of funding to back them up.

I’m excited to see how this plays out, not just because of the market dynamics – where naively speaking, competition is usually healthy – but also because many of the newer players are specifically building for energy efficiency, which could yield meaningful environmental outcomes in the long haul.

You’ve been working in and around data & AI for a while now. Many things have changed! But tell us about something that was true when you started out in this space and is still important today.

Nothing beats looking at the data directly. There is this notion that LLMs must be trained on extraordinary amounts of data, and while this is usually true, the volume of the data does not diminish the impact of manual inspection on handfuls of data points.

Some of the models I work on today are trained on billions or even trillions of data points, and yet I often find myself staring at literal text files to debug some issue. I don’t expect that this aspect of AI will ever go away: to a first-order approximation, the model is the data.

It’s been a heady couple of years with 2024 almost as frothy as 2023. What's one common misconception about AI that you wish would go away?

LLMs in 2022-2023 were heavily influenced by text scraped from the internet, and so there was this sense that LLMs simply regurgitate what is found online, and/or have an effective “IQ” that roughly maps to the average internet comment or blog post.

This is very far from the truth in 2024: LLMs, or at least the ones that everyday end-users interact with, are heavily fine-tuned with reinforcement learning and related techniques, using data that is usually not publicly available. This shifts the distribution of the LLMs’ outputs quite far from what is found on the average website!

An easy way to prove this to yourself is to ask any modern LLM for a how-to on any topic related to a hobby that you have a lot of knowledge about – you’ll generally find that LLMs are much more verbose than any human writer.

The festive season is almost upon us, so many readers will have a bit of extra time to read / learn / reflect. Who do you follow to stay up to date with what’s changing in the world of data & AI?

It’s not strictly just AI, but I often keep tabs on what YCombinator is publishing, as they often have an excellent read of what is trending in the market (NB: these trends tend to be very speculative).

I also love to watch videos from 3Blue1Brown – it’s like the most relaxing version of the mathematics lecture, and really emphasizes visual intuition, vs cranking through tons of equations.

Leaning into your dystopian side for a moment, what’s your biggest fear for/with/from AI in 2025?

Realistically the biggest near-term risk to everyday users are probably immersive game / fantasy characters. LLMs are so compelling these days, that vulnerable folks can be drawn in at an unhealthy level – similarly to any other addictive online app or game.

I think the unique dangers are twofold here: the first is that it is very easy for anyone to build an LLM chatbot application (e.g., vs online gambling apps or multiplayer games which are quite nontrivial), and secondly there are not many market incentives to protect vulnerable populations from these kinds of applications. For example: there are many market & operational factors that help prevent AI from being too rapidly deployed in a corporate setting and causing brand damage, although admittedly it still happens. On the other hand, easy-to-make addictive shovelware has far fewer counterbalancing market forces.

And now channeling your inner optimist, what’s one thing you hope to see for/with/from AI in 2025?

I’m eager to see AI deployed in more hardware/robotics settings – e.g., in manufacturing scenarios – where increasingly greater agency is given to AI models for (hopefully) more useful results, like better or cheaper products. We are already seeing that AI can improve single machines… but what if an AI model helped to orchestrate an entire factory or plant?

You can follow Edward on LinkedIn, read his research or if you happen to live in the Toronto area, you might get very lucky and find an opportunity to hear him speak.