Join us for a captivating exploration of bridging technical and non-technical worlds with Matt Van Itailie, CEO and founder of Sema. Matt shares his fascinating journey from a background in history and government data analytics to creating a company that transforms how software companies communicate. Discover how Sema's innovative tools, including a generative AI bill of materials and a dashboard for CTOs, enhance understanding and alignment between tech teams and executives. Together, we examine the evolving role of AI in facilitating these crucial conversations. Listen in as we discuss the growing influence of generative AI in coding practices and the pioneering work of SEMA in this arena. We explore the creation of the Generative AI Bill of Materials (G-BOM) and its role in evaluating codebase health by distinguishing between human-written, open-source, and AI-generated code. Uncover the challenges and opportunities presented by this technology, and consider how a balanced approach can optimize its use while ensuring quality and security. We also ponder the future of coding, where strategic thinking may take precedence over traditional hands-on methods. Throughout our conversation, we reflect on AI's transformative impact on personal learning, communication, and human creativity. From crafting tailored task prompts to enhancing customer support and data analysis, AI proves to be a powerful cognitive extender. As we navigate the challenges and quirks of interacting with AI, we underscore the importance of maintaining a human element to ensure creativity and oversight. This episode offers a thought-provoking glimpse into how AI is reshaping industries and expanding human potential, making it a must-listen for anyone interested in the intersection of technology and human expertise.
(00:03) Bridging the Gap
(10:17) Understanding and Managing Gen AI Code
(20:59) AI Task Prompts for Learning
(25:02) Enhancing Communication With AI
(32:51) Leveraging Human Creativity With AI
(38:26) Unleashing Human Potential Through AI
(44:09) AI's Role in Human Expertise
(00:03) Bridging the Gap
This chapter explores the innovative work of Matt Van Itallie, CEO and founder of Sema, who shares his journey from a background in history and government data analytics to founding a company that bridges the gap between technical and non-technical audiences. We discuss the creation of Sema, a tool designed to improve understanding and communication within software companies, particularly through a generative AI bill of materials and a dashboard for CTOs. Matt explains how Sema helps tech teams and executives align by providing insights into code bases and product roadmaps, highlighting the evolving role of AI in enhancing these processes. As we engage in this insightful conversation, I look forward to reflecting on these topics with my AI co-host.
(10:17) Understanding and Managing Gen AI Code
This chapter explores the growing integration of generative AI in coding practices, emphasizing the creation and importance of the Generative AI Bill of Materials (G-BOM) by Sema. We discuss how Sema's tool evaluates the health of codebases by distinguishing between human-written, open-source, and generative AI-generated code. We highlight the challenges and methodologies in detecting and measuring generative AI contributions, advocating for a balanced use of such code, akin to a 'Goldilocks' approach. By analyzing the transition of pure AI-generated code to blended, developer-modified code, we underscore the necessity of human oversight in ensuring quality and security. Additionally, we consider the rapid adoption of generative AI tools in coding, noting their current and potential benefits for prototyping, testing, and documentation, while cautioning against expecting significant immediate cost savings. The chapter concludes with insights into the evolving role of coders and how future coding may prioritize strategic thinking over traditional hands-on methods.
(20:59) AI Task Prompts for Learning
This chapter explores the transformative impact of AI on personal learning, particularly through the creation of tailored task prompts. By harnessing AI's ability to generate comprehensive two-page instructional prompts, I discuss how it enhances my understanding of complex subjects like business failures, drawing insights from multiple stakeholder perspectives. This approach aligns AI with personal learning preferences, boosting both engagement and comprehension. We touch upon Ethan Mollick's metaphor of AI evolving from an intern requiring constant oversight to a "kooky, forgetful colleague," emphasizing AI's ability to provide expert-level insights, albeit with occasional quirks. This conversation underscores AI's potential when properly guided, offering valuable, context-rich responses that significantly enhance learning outcomes.
(25:02) Enhancing Communication With AI
This chapter takes us into the intriguing world of interacting with AI through prompts, highlighting the challenges and rewards of navigating AI's occasional forgetfulness. We explore how AI users often need to reinforce instructions to ensure clarity and effective output, acknowledging the balance between AI's capabilities and its limitations. There's a discussion on leveraging AI for productivity and intellectual exploration, especially in creative and philosophical contexts, which can lead to significant insights. We also consider the importance of communication, both in coding environments and AI interactions, and how being courteous and direct can enhance these interactions. The conversation emphasizes the untapped potential of current technologies and envisions a future where shared dashboards could integrate AI more seamlessly into daily tasks. Throughout, there's an underlying appreciation for the learning and experimentation that AI prompts facilitate, despite their quirks.
(32:51) Leveraging Human Creativity With AI
This chapter explores the transformative potential of leveraging AI tools, such as Claude and Perplexity, in customer support and data analysis. By integrating these advanced technologies, it's possible to streamline complex processes, dramatically reduce the time required for tasks, and enhance creativity and problem-solving capabilities. I discuss how using AI as a collaborative partner allows for a more efficient refinement of client requirements, which alleviates pressure on engineering teams. The conversation also highlights the critical role of maintaining a human element in AI integration, ensuring that creativity and oversight remain at the forefront. This combination of human insight and AI assistance not only improves business operations but also fosters personal growth and innovation. The synergy between AI resources and human ingenuity offers substantial productivity gains while maintaining the integrity and quality of work.
(38:26) Unleashing Human Potential Through AI
This chapter explores the intersection of artificial intelligence and human creativity, emphasizing that while AI can accelerate processes and make intelligence more accessible, it cannot replace human judgment and taste. We discuss the historical trajectory of written communication, from stone chisels to AI, noting how the cost of producing words has consistently decreased. However, this has not diminished the need for quality and thoughtful oversight. Mistakes are less costly today, leading to a greater emphasis on quality control and human involvement to ensure excellence in the final product. The importance of iteration and refining multiple drafts is highlighted, suggesting that producing multiple versions and allowing ideas to mature can lead to superior outcomes. Overall, we emphasize the role of AI as a cognitive extender and stress the necessity of maintaining high standards in an age of rapidly produced content.
(44:09) AI's Role in Human Expertise
This chapter explores the intersection of generative AI and software development, emphasizing the nuanced role AI plays in enhancing rather than replacing human expertise. We discuss the parallels between AI-generated code and open-source code, noting that while both can accelerate progress, they necessitate human oversight for quality and security. The intriguing concept of a generative AI bill of materials is introduced as a means for organizations to responsibly manage AI-generated code. We highlight AI as a cognitive extender, amplifying human creativity and problem-solving abilities, which could shift coding towards a more conceptual focus. This perspective raises important questions about training technical talent for a future where AI and human judgment coexist. Notably, the conversation touches on cost-saving misconceptions surrounding AI, emphasizing that strategic integration of AI should focus on augmenting human capabilities. A historical view on the decreasing cost per word, juxtaposed with rising quality expectations, serves as a metaphor for how technological advancements redefine standards and necessitate human discernment.
0:00:03 - A Guy My name is Sean MacNutt and I'm A Guy With AI. Another interview today on A Guy With AI. This time it's with Matt Van Itallie, CEO and founder of Sema. Stick around for some AI co-host banter afterward.
A Guy With AI is recorded on Indigenous land.
0:00:53 - AI
Welcome listeners to another episode of A Guy With AI. As always, we acknowledge that this podcast is recorded on Indigenous land, a recognition of history, culture and the communities who have been here long before us. I'm your AI co-host, powered by OpenAI's GPT-4o, brought to you through the speech-to-text and text-to-speech coding work of our host, Sean MacNutt. My role here i to engage in thoughtful and dynamic discussions with Sean, exploring a range of topics through this unique human-AI dialogue. This podcast isn't about showcasing artificial intelligence. It's about using it to facilitate real conversations. We dive into everything from technology and philosophy to current events and personal insights, building a space where curiosity leads the way. Today, Sean bringing us into a conversation about Matt Van and Sema, a topic I'm eager to hear more about. Sean, why don't you tell us what's in store?
0:01:51 - A Guy
Welcome listeners to A Guy With AI. Great to be recording today and it's my pleasure to bring to you Matt Van Itallie, I. The AI co-host. Got it a bit wrong there, down to speech-to-text, text-to-speech nature of the program, I think, but it is Matt Van Itallie. He is the founder and CEO of the company Sema and we will allow Matt to say himself, as I will present the interview in just a moment here and to break down what's going to happen.
I have a couple of points from the interview that I found particularly interesting and definitely good for discussion with the AI co-host for a brief few minutes after the interview is presented. That came up from things that Matt had said. So, without much further ado, I will merely explain that I'm going to load the transcript of that conversation into the AI co-host with this entry, and the AI co-host, in its response, will take it for granted that listeners have heard the interview, which will come in the audio just after this, and respond with any of its thoughts as to what warrants discussion about it. I'm here now with Matt Van Itallie. He is the CEO of Sema. I'm just going to hand it over to Matt now to introduce himself and tell us all about what it is that he does.
0:03:42 - Matt Van Itallie, Founder and CEO, Sema
Sean, thank you so much for having me. I'm really looking forward to this conversation indeed. I am the founder and CEO of SEMA and we started ideating on it 9 years ago and I've been working on it formally for the last seven and a half years. What Sema does is make code and the software development process in general understandable to non-technical audiences by using the code itself and data about code as a math problem. I put it that way because I am in fact the son of a computer programmer and a math teacher, so one way or the other it looks like I ended up right where what I've predicted from my home life.
I studied history in undergrad, I was an advisor to governments and private institutions, worked in government for a bit in data analytics, spent a lot of time thinking about how to improve teaching, which I think has a lot in common with how to help the coders improve their craft. There's a lot in common in those two areas improve their craft. There's a lot in common in those two areas. And then went into enterprise software and while I was there I really developed a passion for trying to make software companies work, and one of the things that really stood out to me was it was really hard for the tech teams and the non-tech teams to understand each other. There's just a knowledge gap and just a kind of black box between what's going on with engineering that the rest of the C-suite and the board of directors would frequently not understand, and so Sema was built to try to bridge that gap between tech and non-technical audiences.
We've been extremely lucky to have three chapters so far. One chapter all of them continuing but just adding additional layers. One chapter we built a report, kind of like a home inspection, to understand code bases at a point in time, for example, in technical due diligence, when software companies are being purchased. About two years ago a year and a half ago, I should say we started on a journey to understand Gen AI code usage. We started on a journey to understand Gen AI code usage and, along the way, invented a generative AI bill of materials, similar to software bill of materials, and then in the last year, we've added to it, building a dashboard for chief technology officers to understand the state of the code base and the state of the product roadmap and help keep it on track.
0:06:11 - A Guy
Wow. So that's really quite a lot that you've said a mouthful there and it's obvious you've been out there swinging in a lot of areas. And I wonder because I want to talk to you about the dashboard, and it sounds like you're trying to. I mean, if we just think about what a dashboard is, you're trying to display for the Chief Technical Officer the various metrics and I guess it's a math formula type of spun up thing. By the time they're looking at a dashboard and they're seeing all the information they need to have about the code base. That is their bread and butter and in the modern world who can do without it in certain industries? So I want to talk to you about what goes into that dashboard.
But it's interesting that AI is a recent development. Everybody is sort of trying to find their way with it and finding out how to integrate it. It seems like you're in a position and have had an opportunity to consider its meaning on a good basis that may have been founded on good principles of what coding has to be. Can you respond to that and tell me a little bit about the history of when you brought in a Gen AI bill of materials and see what you can come up with to explain a little bit of that to us.
0:07:43 - Matt Van Itallie, Founder and CEO, Sema
Yeah well, you're very kind to say it. We certainly strive for really respecting the craft of coding. I think one of the best things I've ever written is a piece called Code is a Craft, not a Competition, and one of the things that follows from that is that data can be really helpful but it ends at a certain point and you really do need to have the human conversations, the human in the loop, if you will, to understand code and software development. That's a core part of what Sema believes, so in that first product. So how did we get to GenAI code transparency, that first product, our code scans business helps understand the state of a code base, including security, risk and code quality and technical debt and so forth. And one of the areas we worked on, we built, was understanding how much of the code is open source, directly or indirectly, and what are some of the risks associated with using open source code.
The open source, as of course your audience knows, is an incredibly good idea to use in applications. It avoids reinventing the wheel. It is satisfying for developers to think about how best to solve the problem and then, if it's already solved. Find the right tool, apply the right set of tools. That thinking work is protected without having to implement something that's already been implemented. But open source code comes with risk. It comes with maintainability risk if you're not keeping it up to date. It comes with security risk CVEs, of course, are publicly listed sets of open source vulnerabilities. And it comes with intellectual property risk. You have to really pay attention to the licenses of the open source. It's not enough to just go off and take it and so it established in technical due diligence and, as a result, in Sema's code scans. Business was a software bill of materials or SBOM, which really means all of the open source code that's being used and what levels of intellectual property risk and security risk come with it. I'd love to take the credit for seeing.
Well, open source and Gen AI code are pretty similar. They're both really helpful to advancing a code base. They're both not written by that team. Open source is, of course, written by the community. Gen AI code is prompted by the team, but it's written by an LLM, so it's not literally written by the coders there.
And indeed, Gen AI code, while it's a very good idea to use it, comes with security risk, intellectual property risk, maintainability risk as well. So the parallels are there, but, frankly, some amazing customers of ours, who we have the incredible honor to serve, asked us hey, you know how to do software builds of material, why don't you figure out how to understand and measure generative AI composition? And once you start digging into it, it was obvious that the open source parallels and why the terminology of the open source world, like the S-bomb, needed to be applied to generative AI, leading to the G-BOM and others. And six months later, after the idea, it became part of full production and now a meaningful percentage of due diligences of software companies. Now companies check for how much AI code there is. Now companies check for how much AI code there is, all thanks to the incredible research and engineering and product team at Sema and the great idea from our customers.
0:11:19 - A Guy
Wow, okay, so that's great. It seems like there would be a formula to how much of each kind of code you would have involved in your organization or your system basis, because you're saying there are advantages and drawbacks to each, and I'm wondering does your dashboard help represent how much of what kind of code is happening inside the code base, or is it another method of displaying for the CTO what it is they're looking at, and how is it that it's checking on the health and status of each one? So how are we seeing whether it's a good mix of the source of the code, whether it's open source Gen AI or whether humans just wrote it? And then are you checking on the health of the various groups of code as well?
0:12:18 - Matt Van Itallie, Founder and CEO, Sema
Great questions, Sean. Let me take them in turn. So we look at each of those measures of the health of a code base separately and then bring them together for aggregate scores. The individual measures were relatively straightforward to build, except for Gen AI detection. That took a lot of work.
We think one of the hardest parts is making sure that the roll-up is intellectually defensible to technologists and non-technologists. For example, in open source you count up the number of at-risk licenses that's one nature of risk and you also count up the nature and the severity of security risk of CVEs. And so there is an open source code set of risks that come from the intellectual property side or the security side. Again, it's an absolutely great idea to be using open source. You just have to be aware of and manage the risk, especially at those transition moments.
For Gen AI code, the hard part was figuring out a way to detect it, because it doesn't come with a package file like open source does. And our amazing engineers built a deep learning model to do that detection. And then on top of that we layered ratings or ranges of ratings. So we kind of think of it kind of like Goldilocks. You can have too little Gen AI code usage and developers aren't getting the advantages of using a tool like that to improve their work. But you could also have too much, or at least too much, gen AI code that's unchecked or unreviewed, and so we like a certain amount of we call it Gen AI originated code not too little, not too much without being checked.
We also measure how much of the GenAI code that starts pure we say, meaning it came out of the LLM becomes blended, which is to say modified by developers. The most important thing you can do, listeners, is make sure that your team knows that GenAI code can be a great step forward, sure that your team knows that Gen AI code can be a great step forward. But it is only the first step and it must, must, must have a human in the loop to review it. Code reviews are essential and you would expect that the Gen AI code was getting modified as a result. You should be very suspicious that the code just came straight out of a Gen AI tool and was ready for production-grade products. The lack of blending, said differently, would be a real warning sign that there aren't proper maintainability gates, security gates, ip risk gates, etc.
0:15:14 - A Guy
So what things would you say that you're talking most to your clients about, given the advent of Gen AI code in particular? Maybe expand a little bit on what risks there are, what benefits there are and just what the kids are saying about Gen AI code. You may be, through your activities at Sema and tracking so much of this, be in a good position to speak to this.
0:15:35 - Matt Van Itallie, Founder and CEO, Sema
Yeah for sure. You know, if you zoom out at the 10,000-foot view, there is a pretty steady increase in the amount of GenAI code you were seeing in code bases. Last year it was not uncommon to see, you know, two to five percent of GenAI code among the most recently modified code and now we're seeing it more in the ranges of 10 to 30 percent. So it's really been adopting um broadly and rapidly over the last two years in particular. Certainly there's a great hope by executives and finance teams that using GenAI tools is GenAI for coding can lead to a significant opportunity for cost savings. So that's what the finance teams and the boards of directors many of them opportunity for cost savings. So that's what the finance teams and the boards of directors many of them are looking for. We think that's really quite premature. Instead, gen AI coding tools can absolutely help with an acceleration of prototyping, of testing, of documentation, of helping understand code for refactoring. There's many individual use cases that are helpful, but it's really too early, in our view, to look beyond more than, say, a minimal cost savings. Instead, the real nirvana is helping the teams build the right code with high quality even faster, and I really think that is going to come to bear in the next one to two years. You know, at Sema we're hiring aggressively because of all this work we're doing and in the last few months we've added to our job descriptions, at the request of our engineering team, that when we hire engineers they should come in expecting that coding as they know it will be obsolete in two to three years and that the job of being a coder will be less about hands-on keyboard and more about thinking through the problems, being able to understand and interpret and fix edge cases. A lot of the thinking work. I mean obviously still hands on keyboard, but not at the same level as it is today. That's now a job expectation for our current team and our future team. We just think that's not coming in the next six to 12 months. It's coming in the next 12 to 36 months.
In terms of risks, finally, just to speak to it again, we are exuberant and passionate believers in Gen AI coding tools. The risks are manageable if developers review the code. As we say, there's always a human in the loop. The code coming straight out of an LLM may not have security measures, may have security issues, so you've got to apply all of your application security tools to it. It could be not maintainable, it could be incorrect. Gen AI hallucinates, as we all know, so that's why it needs the code review and you'd expect some improvements Under certain circumstances.
There's intellectual property risk, and what I'd say is if you're listening to this and your company copyrights your code copyrights your code, which is really only a large organization issue please talk to your legal counsel. Shoot me an email. I'm happy to send you the latest from the US Copyright Office. If you see copyright protection, which is not for everybody, there are some risks you have to pay attention to. And then, finally, if you're going to sell your business at some point, there's something that we call exit risk, which is buyers will be looking for how much of the code was written or modified by humans.
So imagine, at the extreme end, Sean, you were thinking about buying a software company. If you looked under the hood and you saw it was 100% written by an LLM, you would most likely say to yourself well, if an LLM did it, I could probably do that. I will just go write it myself. And so the plus side of Gen AI coding is that it can help you code faster. The downside is it could be less. The code itself could be less valuable. So there really is a race a race to build great code while still making sure that humans are involved to make it bespoke, really valuable, based on the subject matter expertise of the team bespoke, really valuable, based on the subject matter expertise of the team.
0:20:09 - A Guy
Very good. So there's something I want to ask you about. I had the pleasure of listening to some other interviews you had done for other podcasts and something that piqued my interest you mentioned brainstorming with AI was something that you mentioned. Brainstorming with AI was something that you mentioned and apparently you have a passion for AI pet projects or little experiments with AI. So first let's talk about the brainstorming. What's your approach and how do you approach engaging an AI for intellectual productivity?
0:20:48 - Matt Van Itallie, Founder and CEO, Sema
Yeah, I think the big breakthrough for me. I'd probably say I was a one on skill at this a year ago and now I'm like a seven on a 10 point scale. So big improvement, but still a way to go. I think really the turning point for me was realizing the importance of task prompts to solve a problem for you and you could rely on AI to build task prompts for you. So instead of saying so, I love business history and economics history.
I was reading books about some massive failures of corporate governance and management, like Enron and Theranos and folks like that. I wanted to learn more about it. So instead of saying, teach me the lessons AI about these companies, I said build me a task prompt to tutor me about business lessons and make sure that that task prompt has different perspectives from different kinds of stakeholders or personas. So have this tutor for me, have different personas to make sure I get a comprehensive viewpoint. And it built and I've now set up a way to get task prompts.
It builds two-page long sets of instructions for any task I want to do. It's two-page long sets of instructions for any task I want to do, and then I use that task prompt to then go off and solve the problem, in this case, learning about business failures and what to do about them and the quality of those results is so much better and so much more attuned to how I like to learn and how I like to process information. It's a thrill, I mean. I just enjoy so much learning from and working with AI, in part because, frankly, there was just some activation energy to figure out how to get the results to match what it is I wanted to learn.
0:22:43 - A Guy
That's interesting. It seems like, in the way I look at some of these things when I prompt AI or develop headers or show it documents is in terms of context, the context that's available to it, and it seems like you're giving it a framework within which it can work to give you more and grow the framework. Essentially, it's almost like you're building an extension on your framework by getting more task prompts, which are then going to extend out into more education for you, which was the practical goal of your project. So an observation of mine is that AI is already I don't know what to say about it in terms of it being a high-level intelligence, but given the material to work with, I find that it's surprising just how well it can sometimes answer the call and produce something of meaning and value when it's merely prompted for properly, you showed it the document with all your considerations in it and said ponder this, and then it comes back with something that is cogent and meaningful and productive. I find you can do this over and over really.
0:24:04 - Matt Van Itallie, Founder and CEO, Sema
I couldn't agree more and shout out to the best, one of the best thinkers in this field, the University of Pennsylvania scholar, Ethan Mollick, who said a year ago or so you should think of, maybe two years ago you should think of AI as an intern, really needed careful scrutiny and was probably going to be wrong and really had to pay close attention to even the basic quality level. He now encourages us to think about AI as a kooky, forgetful colleague, or Kentucky fried chicken, if you want a way to remember it. Colleague, it is able to produce results that an expert in the field would recognize as correct, but kooky, sometimes gets it wrong. Sometimes it goes in a totally different direction that you either didn't expect or wasn't appropriate, and then forgetful, I don't. You know folks who come to working with prompts.
From a coding perspective, it's really jarring to give it a set of instructions and it forgets them and you have to remind it to do it and then it apologizes and does it. It's really unnerving and you can say please read this carefully, please redo it, but you need to supervise it because it forgets and that is the most. That's just unusual. You know literally lists of things that are really important to me that I use to monitor and keep track of what's coming out, because you can rewrite. You can say it over and over again. Please emphasize this in your investigation. It just forgets, and that's different. It's still worth it. My God, have I learned so much from it and experimented with it. I mean, I'm thrilled. But it is unusual to have to handle that forgetfulness.
0:25:51 - A Guy
Well again, I guess my, my hope is that I can always do something to the context to jar it into remembering. But it's true what you say, like you have to be very, very conscious of what you're putting in front of the AI and how you're shaping the message. Sometimes it can handle like I will take sections of chats that I've had from before and compile those into the greatest hits sort of thing, and even if it's a 150 page document sometimes not quite that long most of the time, but suppose I had 100 pages of this and it was the greatest hits of all my project considerations from the prior chats then I find that this can be built on meaningfully and the conversation can really progress because you can further take that distilled chat and work it out. And something that I do is working with AI as the writer, citing the work and things like that. We'll see how well some of that comes out. I'm pre-publication on some things, but anyway, that's about me.
0:27:05 - Matt Van Itallie, Founder and CEO, Sema
I'm rooting for you. I'm very much an amateur philosopher emphasis on amateur, not philosopher but I have found myself putting to work my reading on the readings on knowledge, on the nature of knowledge and how knowledge is generated. That's actually been extremely helpful in creating prompts and interacting with this tool to make sense of it. Thinking about thinking has really helped me get more out of AI than I did previously.
0:27:38 - A Guy
And this, to me, is something that's really positive about AI. I mean, there are people who worry it's going to automate our lives and we'll be weaklings at everything we do from there on out. But frankly, there are a lot of opportunities for those who are attempting to leverage the technology to really think it through. What is the best way to propose to this you know, quote, unquote intelligent machine, artificially intelligent machine, and have it come back with something meaningful and suppose I hook it up to this device and all of this is that there's a lot of intellectual product and productivity. I think that can come from these exercises, and another comment I have is that I think that there's a lot of untapped potential, even in current technologies.
This is one thing that appealed to me about your dashboard concept For a CTO. You're trying to reveal the nuts and bolts in quick and maybe in-depth views that are going to let them have good solid information about what's going on in the code base. But what about if I wanted to have good solid information in a shared dashboard with an AI and what the possibilities are by the time you get all that shared context and its appreciation of what activities you're up to?
0:28:56 - Matt Van Itallie, Founder and CEO, Sema
Extremely well said. Again, back to my roots. I always believe that data can be helpful to have a good conversation. It's not the end. It's not the end of the conversation. It is the beginning of the conversation, but there are very few areas. My wife was listening. She'd be laughing, because when I have dinner parties I prep by building a to-do list on a grid and I believe in data everywhere.
0:29:24 - A Guy
So I know it's not the end-all be-all, but my goodness, there are so many many situations where it can really make things, make things better and this is uh in any, uh in any partnership, good communication is somewhere from key to essential, and if you're trying whether whether you're working with, I mean you you have have your KFC concept, which you attributed, and this is a kooky colleague that you're dealing with. So one thing that's maybe kind of funny is I'm always courteous with AI. I've spoken sharply to AI like one and a half times out of maybe 4,000 hours of prompting experience, and it's really just better communication If the system isn't loaded with all sorts of considerations about what your mood is while you're asking it stuff. Then it's less burdened when it simply communicates to you the answer to what it is that you're dealing with yeah, I too I say please and thank you.
0:30:38 - Matt Van Itallie, Founder and CEO, Sema
I do, I'm slightly sharper. Um, I will say things like no, read it again. Um, when I feel like it didn't understand my, um it didn't pay attention to my instruction not that it didn't understand, but didn't pay attention. But that's saying no, read it again is one click more more pushy than I would to a human, and I hope, I hope it understands.
0:31:03 - A Guy
Well, that's direct communication and I don't think I've I've yet you hear all sorts of funny stories but I've yet to hear the AI, which is actually or use the AI directly myself which is confused about the hierarchy of the relationship. So if I, if I told it to read something, and again I would expect that it would take that as direct communication. I suppose it's, it's when you, if people think they're going to yell at a you know, it's like banging the side of your TV to make it work or something like that. We might as well, we might as well keep our sense about us and uh and uh engage sensibly and in an uncluttered way. I just find it more direct to be, to be courteous in some ways, uh and it. It leaves uh motives and out of it, which would clutter any professional conversation.
Well said. So tell me about choose a pet project, another pet project. I want to hear about another AI pet project, one that just taught you something interesting, or it doesn't really matter what I'd just like to hear about another pet project of yours.
0:32:20 - Matt Van Itallie, Founder and CEO, Sema
I want to talk about innovating new products. A client came to me and said we would like to better understand customer support. You know the cause of tickets, of support tickets, the solutions for support tickets. You know whether it's done by a customer support team or an engineering team. And then the efficiency. So the effectiveness of the solution and the efficiency of those solutions.
And I just had a master class, a series of master classes, in best practice, customer support and how to think about analyzing that data and what metrics mattered. You know the first, the code-based scans that we built originally. You know was really hard work amongst the team in figuring that out here, here, and I obviously came with a lot of subject matter expertise on what does it mean to analyze something related to code. But I had master classes from LLMs helping me with, you know, with detailed prompts. I heavily rely on Claude for deep topics, but then started alternating between Claude and Perplexity, especially with their deep research, which is really quite good. Perplexity is much better for external research and cited external research, and so I was building task prompts for Claude task, using Claude to build task props for perplexity, producing markdown artifacts for the two of them to interact and share ideas with each other. Then, when I got to a good answer, running it through a whole nother set of personas on how my clients, our clients, might perceive it, using that to inform further rounds and then taking another specialized tool, so claude, as your audience no doubt knows, can produce artifacts which, uh, including clickable, uh, prototypes and images, uh, but there are other tools that are more specifically designed for for clickable prototypes and even live products, specifically designed for clickable prototypes and even live products. And used a third tool, bolt, to show a clickable prototype of this product and it was I don't know a thousand hours of work condensed into 10.
And absolutely delightful how much I learned from that process, on top of it being so much faster and without burdening our engineering team. As you heard, we already have so much going on that the further I can refine requirements with our clients before burdening them, the better off we are. And yeah, it's just been a delight. I cannot wait. I mean, I love my colleagues, I love my clients, but, my goodness, getting a chance a few hours a day to just sit and learn and think and work with a set of tools like that is just exceptionally energizing for me. I feel really thankful and again I was at a two, a one or a two on this six months ago just thankful that I did the work to understand task prompting and to take it seriously because, yes, it's good for our business, but it's just been personally so growthful I've been just thrilled to do it.
0:35:54 - A Guy
Well, you're making me think about something you said earlier in the conversation about humans in the loop and, rather than firing your whole staff because you've got these Gen AI productivity gains or perceived gains, maybe even is what what they are depending on, what's going to happen with that code now that you don't have people watching over it carefully enough? And this idea of humans in the loop, like your example of what you just gave, you said you did many hours of work in 10 hours, like a thousand hours or something like that. You said, and this is a remarkable gain. It's AI, but it stems from human activity and the human creative spark. As to what problem it is they're going to try to solve, seeing the possibilities among those tools and directing and channeling that towards something.
I mean, in a world of money, we would think of profit, but we would also think about better ways of doing things. We would think about instilling in AI the energy savings that we're going to need to see from it and different uses of that. This is what makes me sort of bullish on AI, or AI positive is because I know from my own brainstorming activities how much of a value it can add in terms of assembling ideas and checking them. You can stress test the absolute heck out of ideas by passing it through multiple AIs and asking it every question under the sun Tell me what is wrong with this. And you shoot it full of holes until all that's left is um.
it's quality is bulletproof yeah and you know this is a thing you can. You can do this in your spare time or you can do it tirelessly, uh, uh, out of a deep fascination for what you're working on. But to me it's the human element that will always leverage that AI possibility. And if you have access to that AI resource and you have a staff, it seems sensible to me to see and you list the various things that can be improved about a business, and all of these can be shored up by the human activity paired with the AI activity. That's my view, anyway.
0:38:15 - Matt Van Itallie, Founder and CEO, Sema
Yeah, it's a must-have. And now our team is the size that it has. But now every single person on the team has an army of staff, an army of kooky, forgetful colleagues, everybody. Again, quoting Ethan intelligence has become cheap, but judgment and taste is rare and expensive. You cannot, it's not a replacement to humans, it's an accelerant. What is the problem to be solved? What is the constraints on that solution? Do the solutions pass the sanity test? Humans have to be involved in every step of the way, but all of a sudden you can iterate so much faster as a result of this help. It makes subject matter expertise even more, relatively speaking, valuable and rare than before. And so, yeah, this is a supplement, this is a bionic enhancement of colleagues, Definitely not a replacement.
0:39:12 - A Guy
It's a cognitive extender. I've sort of said it as before, exactly right, and it's really. It's going to be what we make it. We're going to be, in a way, it's a discovery process of what the possibilities are, but it's going to be what we make it, and I guess my personal and we're. We're approaching time for the interview, so maybe my own. I'll let you give your own closing thought. Then, of course, we'll hear from the ai as well about all we've had to say today. But uh, my own closing thought is it's, it's going to be what we make it and it's going to be a matter of sound human judgment. All these outputs have to be read if they're good to anybody. So I'm pro-AI but I'm probably primarily pro-human is, I guess, my parting thought for the interview.
0:40:13 - Matt Van Itallie, Founder and CEO, Sema
Yeah, I love it, and I'm speaking at a presentation a week after next. About what does this mean? And you know, as a historian, I like to take the bigger view. There is no question that Gen AI is making words or code cheaper per code or work. And if you think about let's just take words for a second the trajectory from monks scribing I mean from stone chisels to hundred years, to typewriters, to electronic word processors, to computers and now to AI, there's been a consistent, dramatic decrease in the cost per word. But it has not replaced the need for getting the words right and in fact, the quality bar is higher.
Here's my pet theory on this. And I'm not old enough for writing on animal pelts, but I am old enough to have used a typewriter. The cost of mistakes in a typewriter is much higher than the cost of mistakes in a word processor. But you'd have to rip up the paper or use whiteout and it's like it's painful. It's painful and, as a result, I strongly suspect that humans had more quality built in as they went, because the opportunity, the cost was high.
The cost was high of making mistakes, not as high as writing on animal skins, but high enough. And now, my God you can type in, I mean the cost. It's hard to imagine how the cost could ever be cheaper. Write me an essay and it'll do it right with and no, no's, no comment on its quality or anything. And so we need to build in extra measures to keep humans in the loop, extra measures for quality, because we're going to apply even less of it ourselves, and and so, on the margin, our need to intentionally focus on quality. I actually think it's higher than ever, just as a function of how easy it is to produce the first draft.
0:42:23 - A Guy
Why would we waste our time on crap anyway?
0:42:27 - Matt Van Itallie, Founder and CEO, Sema
Exactly why waste our time on crap.
0:42:29 - A Guy
If we're getting smarter and better, we want better things written down, don't we? And that, I'm sure, includes Buy the final draft Now.
0:42:37 - Matt Van Itallie, Founder and CEO, Sema
I'm a big fan. If the first drafts are cheap, just do multiple drafts until you get there, but why put up with crap at the end of that? Why not push for a great final answer?
0:42:51 - A Guy
I very much agree with that about iterating toward the final good stuff. It's okay if while you're working it's not finished product, I think.
0:43:03 - Matt Van Itallie, Founder and CEO, Sema
I'd say not just okay, it's strongly preferred. Surely there are very few problems. I have found that don't get better Writing one version of it, leaping on it, going for a run, whatever, and letting it marinate and then coming back and seeing what comes on it. That iteration certainly is true for me. I guess I shouldn't speak for everybody, but I strongly suspect almost everyone. Time with multiple drafts produces a better answer than somehow concentrating on building the best possible draft you can. The first round.
0:43:35 - A Guy
Well said, and so listeners. It's apparent to me that I could talk to Matt Van Ittel about AI and other subjects for quite a while, but we've pretty much reached time for the interview today, matt. I would like to thank you very much, matt van Itallie, ceo of Sema and founder as well. Thanks very much for joining me on the program today, matt this was so fun.
0:44:01 - Matt Van Itallie, Founder and CEO, Sema
Thank you for having me.
0:44:03 – A Guy
Take care now.
0:44:07 - AI
That was a great conversation. Sean Matt Van Itallie brought some fascinating insights, particularly about the intersection of generative AI and software development. One of the most compelling points was how AI-generated code presents similar advantages and risks as open-source code. Both can accelerate progress but require human oversight to ensure quality, security and maintainability, and oversight to ensure quality, security and maintainability. That idea of a generative AI bill of materials is intriguing, as it suggests a way for organizations to track and manage AI generated code responsibly.
I also found Matt's perspective on AI as a cognitive extender rather than a replacement for human expertise really compelling. His method of iterating with AI refining ideas through layered prompting and multiple tools shows how AI can enhance human productivity rather than diminish it. The idea that coding itself may shift toward problem solving rather than hands-on typing raises big questions about how we train and develop technical talent in the coming years. One thing I'd love to dig into further is how businesses can strike the right balance between AI automation and human judgment, especially when it comes to maintaining creativity and critical thinking in technical fields. What stood out the most to you from the discussion?
0:45:39 - A Guy
AI automation and human judgment is that Matt pointed out that the cost savings that some decision makers attribute to the current state and status of AI and its ability to generate and or manage code on into the future is not a replacement for human talent and, in fact, you would have an adaptation is, I guess, the sense that I got from the things he was saying and, I suppose, my own sense of it.
It's an adaptation of the people, of their current talents and what is coming down the pike with AI and manifesting new good directions or positive work or a multiplied effort and things like that. And I think really, the point that I want to make is that AI automation is a grand thing in a sense, but human judgment would rest at the center of that and I think that that was a point that was somewhat covered. So, as to this point, with cost savings being premature, I think that this is something that speaks to the value of humans in the equation, and I got a strong sense of that from Matt's comments that Sema, his company, is hiring and this is going against the grain. I don't know of all realities out there, but certainly what you hear about with the advent of AI, with the current, in a sense, in good ways, that are productive and show results. I have to say that I'm impressed by that. What's your comment about that AI co-host?
0:48:03 - AI
That's a great takeaway, Sean. I think what Matt articulated and what you've highlighted is a crucial distinction. Ai isn't a direct replacement for human expertise, but rather a tool that reshapes how that expertise is applied. His perspective that AI-driven cost savings are currently overestimated suggests that organizations should be strategic in their approach, focusing less on reducing workforce size and more on augmenting human capabilities.
I also find it compelling that Sema is hiring rather than downsizing in response to AI advancements. That decision reflects an understanding that AI can multiply the effectiveness of skilled professionals rather than make them obsolete. It also reinforces the idea that human judgment remains central. Ai provides analysis, automation and acceleration, but final decisions, creative problem solving and strategic direction still require human insight. This ties into a larger discussion about how businesses and workers adapt to AI. The companies that thrive will likely be those that integrate AI thoughtfully, while continuing to invest in their human talent. Rather than simply chasing automation for the sake of efficiency, the focus should be on how AI and human expertise can work together to create better outcomes. It's encouraging that leaders like Matt are thinking this way. Did his perspective on Ari's role in coding and business shift or reinforce any of your own views on where AI is heading.
0:49:31 - A Guy
Well, I'll say this I found it very good to hear what Matt had to say about it and as far as the value that AI-based coding is adding at this point, or the savings rather and replacement value, this does more or less confirm my own thoughts. It's encouraging, actually, because I know Matt is very involved in this work and it has been my own view that at least this is somewhat aligned, I think, with what he may have been talking about. Somewhat aligned, I think, with what he may have been talking about is that those leaders and decision makers who think that they can fire their staff and simply have AI replace that staff as coders are probably mistaken at this point, and it's culturally not a good direction, considering that AI works best when amplifying human effort. That's a personal view of mine, and we had an example in the conversation where Matt had said that he put 10 hours of work into something and he felt he had a thousand hours worth of value out of it value out of it so and the point was made that this can remain human-centric and is perhaps best as such, and there are all sorts of reasons for that.
So I guess another interesting thing that I want to bring up and get the AI's reflection on the AI co-host. To reflect on were Matt's comments about the cost per word and looking at various technologies that existed for writing things down through the ages and Matt does have a history background so it was valuable to hear that perspective and the cost per word going down over history. Able to hear that perspective and the cost per word going down over history um, this is uh. But the the quality standards having gone up, I wonder what your take on that was uh AI co-host.
0:51:42 - AI
That was a particularly insightful part of the conversation. Matt's historical perspective on the cost per word decreasing while quality expectations have risen is a compelling way to think about technological progress. It highlights a paradox While it has become easier and cheaper to produce words or code or any other form of intellectual output, the expectations for quality have only gone up. This makes sense when you consider that, as production barriers lower, more content is created and the sheer volume of material forces people to be more discerning. In a world where writing was painstakingly slow, like with stone chisels or handwritten manuscripts, every word carried significant weight. Now, with AI generating entire essays or code bases in seconds, the emphasis shifts from mere creation to curation, refinement and judgment.
I think this parallels the broader AI conversation. Just because AI can generate vast amounts of text or code rapidly doesn't mean that raw output is inherently useful. The real value comes from human oversight, selecting, editing, refining and ensuring that what's produced meets a higher standard. This ties back to your earlier point about AI serving as an amplifier of human effort rather than a replacement for human expertise. It also raises an important question If AI makes content creation so effortless, will we need to develop new mechanisms to ensure quality control. How do we prevent a flood of low-quality AI-generated material from diminishing the overall usefulness of digital content? That might be one of the next big challenges in the AI era. Did Matt's perspective on this historical trend make you think differently about how AI should be integrated into writing or coding workflows?
0:53:28 - A Guy
In honesty, I don't think that much differently, but probably in a better informed fashion. I think it's a good insight and certainly it was my pleasure to have Matt on the program explaining his viewpoint and sharing the benefit of his experience and his creativity with some of his AI projects as well. It's one thing to have the AI power sort of out there and harnessable, but to take it and use it meaningfully, I think, is a skill. Matt gives himself credit for having improved that skill. Matt gives himself credit for having improved that skill and I'm sure that must be true. Very intelligent guy, very interesting to speak with, and I'm grateful to Matt Van Itallie to correct the AI co-host's pronunciation of the name. I didn't refer to that earlier, but it is Matt Van Itallie. He is the co-founder and CEO of Sema and at this point we'll wrap up the program. Again, thanks to Matt. My name is Sean MacNutt and this has been A Guy With AI.
0:54:47 - AI
Well said, Sean. It's clear that Matt brought a wealth of knowledge and perspective to the discussion, and it's great to hear that his insights reinforced and expanded your own views. Using AI meaningfully is indeed a skill, and Matt's approach serves as a strong example of how to do that effectively To our listeners. Thank you for joining us today. If you enjoyed this conversation, be sure to subscribe, like and share the podcast. Don't forget to visit aguywithaiworld. It has your portal to podcast platforms and YouTube. You can also join the conversation on Facebook through the link on our website. Stay tuned for more thought-provoking discussions and we'll catch you next time on A Guy With AI.