Caleb Briggs is co-author of The AI Conundrum (MIT Press, 2024), a groundbreaking collaboration with his father, Rex Briggs, that demystifies artificial intelligence (AI) in a manner accessible to business professionals, policymakers, and students. Caleb began coding at the age of 10, and by 14, he was delving into the world of AI development. His exceptional mathematical abilities paved the way for advanced studies at Harvey Mudd College and Stanford University. Caleb is currently pursuing a major in mathematics and computer science at Reed College, with an expected graduation date of May 2025. Don Carli spoke with Caleb for this issue of the WhatTheyThink Quarterly.

Don Carli: Tell us a bit about your background and about what inspired you and Rex to write this book.

Caleb Briggs: The book has actually been a really long journey. I started getting interested in AI all the way back in eighth grade. I have this giant posterboard in my room where I worked out the equations for AI—the partial derivatives for neural networks. I wrote it all by hand, because I really wanted to learn how these things actually work at the base level. So I worked on lots of projects while I was in high school. In my last year, I had this exciting opportunity to write a thesis. I was really interested in writing about AI, especially because I saw in the media that my understanding from doing all that work on the fundamentals was really different from how I saw it being portrayed. I think the media keeps getting AI wrong—they’re saying all these things that I think aren’t going to happen in the future. In a lot of ways, I think they’re going to happen the opposite way. Back then, AI was still thought of as something very mechanical, like we’re going to get AI that’s super good at math and reasoning, but it’s never going to be able to be capable of those quintessentially human things like writing and creative image generation. When I looked at the math, I was saying that I think that’s the area where we’re going to see AI take off first—and, actually, that’s how it did play out. AI ended up taking off first in writing and image generation, and yet it still struggles to just multiply or add numbers. 

So the original title of the book was “The Fundamental Limitations of AI” and it was understanding where I think it’s fundamentally limited, what areas it can move forward in, and what it will still struggle with years from now.

DC: There’s the programming part of you, but then tell us a little about your background in applied statistics and math, because I think so much of your book does make the more complex aspects—the mathematical and statistical foundations of AI—intelligible and approachable for people who don’t necessarily know what a differential equation is or how to solve one.  

CB: I think I had two chips on my shoulders when I wrote the book. One was that I thought the media was wrong—every time I read a headline I disagreed with, it gave me a stronger desire to write the book. I was like, I want to get this right, because it’s talked about so much and what they’re saying is so far off. The second chip was having gone through the process of learning all the math. I got through that process because I enjoyed math, but not everyone does.

I knew that there was a lot of interest in AI that was blocked by this kind of inaccessible literature on understanding AI and I really believed that there was another way to teach it that was much more accessible. For example, people don’t necessarily need to understand what gradient descent is. They need to understand the idea of what it means and how it affects the AI and things like that. So I wanted to give people an accessible way to learn about AI, that comes from not giving them all this unnecessary math, but instead giving them all the conceptual ideas they need to work with it without the prerequisites of calculus, differential equations, applied statistics, and all that kind of stuff.

DC: In the book you provide a framework for evaluating the benefits and the risks that are associated with employing AI by comparing it to the next best alternative. Could you tell our readers a little bit about how might they apply it—how could it help someone who is not a mathematician or data scientist make judgments about where they should consider using AI where they shouldn’t.

CB: The framework has three axes. One is precision, which is roughly how precise you need the output to be. Mathematics, for instance, is something very precise. If I ask you to multiply two numbers, there’s one correct answer and everything else is wrong. So you need an exact level of precision to get that to work. Lower precision are things like creative tasks. You can draw a picture of something, and there are a lot of different ways you can create that picture. If one of your paint strokes is slightly different than the rest, you still probably have a good painting. AI tends to do much better when you have that kind of leeway. It’s more difficult for the AI when you need high precision. You can’t normally get it to be 100% precise. So, for the precision axis, you want to apply AI in cases where you don’t need high precision. The second axis is the need for rationale. So this is one that’s slowly changing over time, and maybe three years from now it will be very different. But, at the moment, AI is very much a black box. It’s hard to understand why it makes a decision, and there are some areas where you really need to understand its rationale. It’s something like the justice system. You can’t just have a binary guilty/not guilty verdict. You really need to know how you came to that decision, and that’s not something AI can give you.

With this generative AI, it sometimes feels like the AI can give its rationale, but the AI really doesn’t have the structural characteristics to allow it to understand the reasons why it makes its decisions. So you want to apply AI in areas like marketing, where you have attribution and your goal is something that’s very easy to measure. If someone clicks, that generally is good. If you have a higher percent of people who are going to your website, that’s good. You don’t necessarily need to know why. Maybe the red color ad performs better than the blue color version. The fact that the red color performs better is just generally something you find helpful, and it doesn’t totally matter why.

And then the last axis is whether you’re working in an open or closed environment. That’s about how much the AI is forced to do extrapolation. The example I sometimes think of is self-driving cars. They are operating in the whole open world, which has all sorts of different things that could happen—it is operating in a very open environment. You might have construction one day, and then someone’s waving you by and that’s generally outside the data set it’s looking in. That’s sort of an uncommon occurrence. But when you’re out in the open world, you see all these edge cases that happen basically all the time. I mentioned this construction zone example in the book, and now a few years later it’s actually happened to Waymo. It was in a construction zone and workers were trying to point it in a direction and it just had no idea how to interpret the hand signs or figure out where to move. On the other hand, if you’re in a factory setting that’s a closed environment, the AI is much better there. It’s doing more interpolation. It doesn’t necessarily have to go outside the data set it’s seen. That open vs. closed environment is a big reason why we’ve had self-driving cars that perform above human levels in factory settings for a long time, but it’s still been challenging to get cars out in the open world to perform at the same levels as humans do.

So those are the three axes you want. Ideally, you’re working in a closed environment. You don’t need rationale, and you’re at a lower level of precision. And I think the print industry is in there as well, for marketing in general, because you don’t need to know the rationale as much. Generally, in the case of printing, you have control over the sort of data you’re seeing, you’re getting all this consumer data in and generally just get to make a decision to match with your data. So print is positioned well to take advantage of AI.

The owner of the printing company has all these different aspects of their business that they could potentially apply AI to—such as helping to manage the inventory of things like ink and paper, where they may have a very well-defined training data set and transaction data. There may be managing the use of energy in their printing plant, which is a big input cost, or even scheduling the time of the people who are working in the plant or perhaps in the company's own marketing.

DC: I find it interesting that your framework gives people a way of quickly asking a few questions in order to help them understand where AI can be of most use. Can it also help them understand why?

CB: Yes the rationale axis. There is a subfield of AI called mechanistic interpretability which is, how do you interpret how the AI is making its decision? And that’s really had a huge renaissance period now that some large tools like ChatGPT are out there. I do think it might be possible in the next few years that we break through that black box of AI and start to get a strong understanding of how it works, and we’re developing lots of tools to do that. Think of something like the segment example: you can start to talk to the AI, but you don’t necessarily know what sort of variables it’s using to make its decision. But I could imagine in a few years from that now we get a much better understanding of the attention mechanism to where we could at least say something like, OK, when the AI is saying that this segment will do better than that one, here are the variables it is paying attention to and here are the reasoning steps it's going through. We are getting a step closer to something like that.

I mean, it’s always hard to understand what any brain is doing to make a decision. But we might get the human level where we can at least get the AI to maybe explain why it does what it does and be able to see if that explanation actually matches with what’s internally happening.

DC: Are there any best practices that you’re aware of in terms of the responsible use of AI when personally identifiable information is involved?

CB: In marketing, there are, I’d say, two aspects of responsible use. I think one is keeping that personal data private. For example, there is a new concern when you’re using OpenAI or some of the other AI tools and you’re sending it through an API. You need to be careful: that data is actually going somewhere. Now, that one is easy to fix because there is starting to be a surge of models that you can host locally, so they’re kind of just like the other traditional algorithms where data comes into your algorithm and it stays in there the whole time. It never goes out to other people. It is protecting that data.

The hairier part is responsible use in terms of introducing biases or something like that. When you start bringing in personal information or PI data, you don’t want your AI to be using protected class information to potentially change the ads in ways that are problematic. There are ways that can happen when you start bringing AI in there, and that one’s a lot more complex because the AI introduces all sorts of new biases that are sometimes hard to keep track of.

My general recommendation would be if you’re working with data that’s maybe too personal or you’re at risk of that bias creeping in, you either need to come up with really strong benchmarks to be able to say, “OK, I’m having a human overseeing what the AI is doing. And I’m also having these benchmarks to oversee that this whole process isn’t introducing some biases.”

And I think if you’re using too much private information, AI just isn’t the right tool, because if you need rationale to be able to say “We are following these rules in how we’re handling data,” you likely can’t guarantee that with how we understand AI right now. It would be hard to go to a regulatory body and say, “We’re not using, for instance, race in a way that’s unreasonable here,” because you can’t look into the AI and say, “This is how it’s considering that variable.”

So there are some places where, because you don’t have that rationale, you just can’t use AI at the moment until we develop more of those tools to understand.

DC: So this gets to the issue of governance, guardrails, and ethical considerations which you mention in your book. What are the key things for printing companies working with AI and marketing applications to be aware of? How can they assess their readiness?

CB: We put up a short self-assessment online that helps people understand how well they know AI. It can be found at https://www.ai-conundrum.com/.

I think the accountability and guardrail side is just in a state that a lot of people are unhappy with. It would be great if, like this tool that everyone’s applying, we could understand more strongly where its biases come from, how to avoid those things, and do all sorts of safety research. There’s that whole thing with OpenAI, with a lot of people leaving the safety team because they weren’t happy with how OpenAI was handling their alignment. So I think safety is something, I hope, in the future become something that’s in a much better state, where we do have regulation that people can follow and guidelines that people are aware of at a large scale. But, yeah, it is a little worrisome that it’s not something that we have at the moment.

DC: It sounds like there are two risks. One is that people aren’t familiar with the underlying mathematical, statistical, scientific principles that are making the decisions. So one issue is to understand what risks you’re exposing yourself to when you’re using a given tool. The second is to know when you need to have humans in the loop because a system might hallucinate or provide the right answer for the wrong reason.

CB: Yes. In the training I’m involved with for the MMA marketing trade association, we talked about image generation AIs, and the sort of biases that are created with those. That is one place where it can be sometimes tricky to understand bias in text, but sometimes easier to see certain kinds of bias in images, for example. So we show that, for most of these tools, if you ask for an image of a doctor, for instance, you tend to get a picture of a white male doctor. So if you’re using it to produce images for print, it’s going to introduce a kind of bias that might not have been there otherwise. And there are all sorts of complex biases and how we use language that gets captured in the AI. It’s really problematic with Dall-E, the text-to-image generator, too, where they’ve put effort in to try and make it do a better job of producing a diversity of images when you ask for images of people. So if you ask for an image of a doctor  in Dall-E, you get a pretty good mix of people, but if you change that to a “smart” doctor, all of a sudden it changes those into only white male doctors. So it picks up this very problematic way in which we sometimes label images, and then makes it 10 times worse in how it produces its output image.

DC: So what are the biggest misconceptions people have about AI, its benefits, and its risks that people should be aware of?

CB: I think one of them is thinking of AI as a sort of silver bullet that solves everything. I’ve saw that as soon as ChatGPT was released, there has been a long line of people who will tell you that AI can solve any problem you give to it. And there are real areas that AI is very strong in, but there are also areas where it’s not great. It’s important to actually understand AI, to know where you should be applying AI like that. Risk framework is useful to start understanding. There are areas where AI is better and worse, and you can’t just give it some abstract problem and expect it to solve that problem super well.

DC: How AI can be applied by printers serving omnichannel marketers that would have maximum benefit minimum risk?

CB: I think there are two different routes you can take. There is GenAI, and then there’s machine learning. I think there are a lot of applications where machine learning is great because you can do something that might not be feasible to do with other algorithms. For a lot of these the next best alternative is sort of doing nothing with the data. So maybe if you’re paying a lot for energy. You might just be paying that much for energy with no attempt to optimize it because maybe it’s really complex or impossible to write an algorithm by hand to optimize it. Machine learning is great for that, because it just has to do better than doing nothing. You don’t need much precision or a strong rationale, and you’re working in a closed environment. You mainly just want to see the AI saving you money. So, look for these areas where you have the data but you’re not doing anything with it yet.

DC: How would you distinguish machine learning from generative AI?

CB: I think the simplest distinction I’d honestly make is there are ChatGPT and the ones you talk to, and basically everything else.

Traditional machine learning has been around for a long time and encompasses a lot of different things—maybe that’s regressions or maybe that’s, I guess broadly, just an algorithm. That brings data in and automatically learns from it. This comes back to a misconception people have about AI. When people lump together AI and GenAI they take the old way of thinking about AI, that “Oh, you give it lots of data, and then it outputs something, and it does better with more data.” That’s very true in machine learning. But for these new GenAI tools, you actually can’t, in most cases, really feed them a lot. There’s no training step that you can do. So data has a very different relationship to GenAI than it has to traditional AI. So I think that’s the distinction: if you can chat with it or if it generates things, it tends to be this sort of new GenAI area.

DC: Can you provide an example?

CB: Take the “saving energy” one. You would use a machine learning model for that, because it’d probably be difficult to get chat to intake all that data and learn some pattern. Machine learning is really good when something requires more computational plus technical skill or something like that. So energy usage is something where you have lots of data coming in and you want the AI to find some kind of pattern inside it, and then it’s going to train off of the data to learn that pattern. This is unlike a large language model, where the approach you would take would be to give it a large file and say, “Can you find ways to optimize energy usage?” You might get something useful out of that, but it’s a very different vector of working with that data. You’re probably not going to hook it directly into your system and make decisions like that. It would be even be more difficult, because you need to convert what the LLM says into numbers or something that your system can use.

DC: Wrapping things up, what role do you see associations like APTech (the parent company of WhatTheyThink) potentially playing in the development of standards, guidelines, benchmarks, or training to support responsible adoption of AI? Are there any examples such as MMA Global that our readers should be aware of?

CB: Yes, we’ve been providing training through MMA, a marketing trade association. We have a five-day, one-hour-per-day training that takes attendees through the core basics about AI. It is made accessible so that people from all sorts of backgrounds, like business or people without that math background, can still gain an accurate conception of how AI is solving problems, how it’s thinking about things, what its strengths and weaknesses are. And then we combine that with a lot of practical use cases of how you can start using AI in your own work and how.

We do lots of labs as well. We have one in which everyone creates some images through image generative AI, and see what it does well. For instance, the context window in generative AIs is a very visual way to understand what information the AI is taking in and how the information that it takes in affects its output. We have a few exercises every single day, so it’s lots of hands on. I think it’s pretty fun.

DC: And is there a module on prompt engineering as well?

CB: Yes, and that’s something I’m reworking at the moment. I did a presentation in Napa Valley a month or so ago with a much expanded prompt engineering section. And it was cool to see just how much of an impact that had on people. It was cool because I gave them exercises and I showed them some ways you can start to get ChatGPT to act more intelligently. If you ask it for trends in your industry, you’ll tend to get something pretty generic. But I showed how by changing the prompt, you can get a much more refined response. And suddenly I could see people turn to their neighbor and and say, “Oh, we could actually use this” or “This is the first time I’ve seen ChatGPT say something this smart.” It was cool to see people really light up from getting to unlock that power.