Transcript: How to be an AI first company, Enterprise AI adoption, Future of Developers, Cost of AI, Unlocking value from Enterprise Data with AI & more with Ben Kus | CTO of Box

But there's this huge untapped potential in unstructured data. And so at Box, uh, with all of these new models that coming out from all these great providers, are is kind of a gift to to to to companies and to people who think like the way that we, we, we do, which is like sort of, how can you get more out of your unstructured data? And now AI can basically understand unstructured data. So for the first time you've had this automated ability to sort of have computers be able to understand, watch, read, look at these things and then be able to not only generate for you new content, but also to understand and help you with the content that you already have, which is not, in many companies, it's just this massive, um, petabytes, uh, hundreds of millions, hundreds of billions of these kind of pieces of content. And in some cases, it's some of their most critical stuff they have in these organizations. I, I think there's, there's a couple layers that, I think what you said is true, is that you, you kind of, um, when you add a junior developer, um, in any company, like oftentimes you expect like a certain relatively small level of output compared to maybe some more senior people. Um, but now, like a person who's really good at using the latest tools is actually quite productive. Um, and that is a big value. And so, so you expect to get more out of it. So like at Box, like we are approaching, like, like I think we are right now, the most developers we've ever had. And I think that's continuing to projecting to continue. Hello everyone. Uh, my guest today is Ben. Uh, Ben is the Chief Technology Officer of Box. Uh, he was previously VP of Product, uh, at Box, and also co-founder of Subspace. Uh, in this conversation we'll talk about, you know, the role of unstructured data in AI. Uh, what are the cost structures of adopting AI and building with AI, and how AI is impacting enterprise business. Uh, if it, if this is first time listening to Start a Project, don't forget to subscribe to us wherever you're listening to us. Uh, you can also follow us on Substack, at startaproject.substack.com. Uh, uh, Ben, welcome to the show. Thanks for having me on. Uh, so I was really excited because, uh, I, um, work in unstructured data as well. And I realized how important unstructured data is. Uh, but I think let's set a little bit of context with the audience, you know, we, I think in an, in the storage industry, it's a common phrase to use unstructured data. But I think it would be good to set the context of, okay, what is unstructured data, and you know, why Box is in the center of all things AI. Yeah, so, um, it's kind of interesting. Oftentimes if you just said the word "data" uh, to anyone, especially computer scientists or people who are sort of, um, come from program backgrounds, you naturally think of data, structured data, right? And like we're going to become more data oriented. We're going to become, you know, we need to use data. And it's, and it's, um, partially because like there's been, I think for those of us who've experienced it, this like massive data revolution over the last like 10 or 20 years. Uh, it used to be, you know, like, oh, I got my data in a MySQL database somewhere. And then it became like more and more tools available for you, that you would use these terms like data lake and data warehouse, more advanced analytics tools. Uh, you see companies like Databricks and you see companies like Snowflake, that just become like, like these, these very powerful platforms of structured data. And, um, and, and that's just naturally what you think of. Um, now the world of unstructured data, interestingly, and, and I would define unstructured data as, uh, almost as data that's not in a database, um, that doesn't have a schema to it, which is like, um, uh, things like emails and messages and like, uh, webpages. Um, and then in our world at Box, is the world of what we call content or files or the stuff that goes into, uh, documents or PowerPoints, or, um, uh, uh, markdown files, or videos, or images. Like all of this is, is like, um, unstructured data. And interestingly, almost, uh, every company that you talk to, um, uh, and, and, and most of everything I'll talk about today is a very business-to-business is a very, enterprise-oriented, uh, sort of thought process, uh, is, uh, they, 90% or more of their data is, is actually unstructured. And so, um, and, and at Box, you know, we have 120,000 enterprise customers, you know, we have over an exabyte of, of, of data. And so, um, uh, this is what we've always kind of lived by. And so when you had data, some people will call it storage, some people will call it, uh, you need to collaborate on it, you need to sync it, so that you can get, you know, the different places. But then with Generative AI comes around. Generative AI is kind of born on unstructured data. And so it naturally immediately, like every company I've ever talked to, they, they're, if you say, why are you interested in, in, in, um, in, in Generative AI, one of their top three things they'll say will be, well, I've got all this internal stuff in my company, um, that is unstructured data, and I don't think I'm taking advantage of it enough. And it takes a million different forms. And it's partly why it's been hard to kind of really, automate or sort of, like, make these specialized applications to deal with these certain types of data. But there's this huge untapped potential in unstructured data. And so at Box, uh, with, uh, all of these new models that coming out from all these great providers, are is kind of a gift to to to to companies and to people who think like the way that, uh, we, we, we do, which is like sort of, how can you get more out of your unstructured data? And now AI can basically understand unstructured data. So for the first time you've had this automated ability to sort of have computers be able to understand, watch, read, look at these things and then be able to not only generate for you new content, but also to understand and help you with the content that you already have, which is not, in many companies, it's just this massive, um, petabytes, uh, hundreds of millions, hundreds of billions of these kind of pieces of content. And in some cases, it's some of their most critical stuff they have in these organizations. Um, you know, unstructured data includes, you know, Box, you know, Amazon S3, Files has, you know, um, Azure has a Blob. You know, all these places and any, any given enterprise has, you know, multiple places where they're storing data. I'm sure like, you might have experienced this. So, um, I guess in terms of strategy of building products, uh, how much of it, are you thinking as, okay, we are going to extend the Box ecosystem into all these surface areas versus like, we're going to build some tools or products within the Box ecosystem? Like talk to me a little bit about like, what is your strategic approach on building these new tools or features, uh, for Box customers? Yeah, I mean, I, I think it's, um, uh, like if you, you go back to the analogy of like, where do people, where do people store their, their, uh, structured data? It's in many places, for many different reasons. And similarly, there's, the very generically large term of unstructured data, like you would store it in, in, you know, many, you use it in many different ways. Um, but for Box, um, one of our sort of, uh, things we're typically known for is to make it very easy to use, make it very easy to, um, extend, make it very easy to secure, and to be compliant, your, uh, for all of your data. So for that, we typically would need to sort of manage it. And so we have a million ways to, like, and, and in our world, like, when you're thinking about, oh, I have data in this repository and data in this repository, uh, we could sync them, you know, like, that we could sync the local computers, you can sync them across the systems. Uh, we recently just announced a, a big partnership with Snowflake, where the, the, the structured data, the metadata about a file, goes, um, into Box, and then it automatically syncs into Snowflake tables. Um, and so that kind of thing is definitely part of, of, of what we think about. But, but it, but in general, for Box, it's key that, uh, one of the reasons we offer so much, you know, AI and many, in, in many cases we offer the AI just for free on top of the data that you have, even though it's quite expensive. And, um, it is because we want people to be able to bring their data, and then get all of the benefits of, of, of security, of collaboration, of AI on their data. And then, and then, but then, um, we don't at all believe that, like, we're going to be the only, um, people who are just part of this AI and Agentic ecosystem, which is why we partner with basically everyone. We believe that there's going to be these major enterprise platforms, uh, that every company will be looking at, and there's some different options available to them, uh, whether it's their CM system, and whether it's their, their, uh, ERP system, whether it's their enterprise, uh, unstructured data system, or their enterprise data system. And, and our job is to give the best option for them for unstructured data, and then integrate with everybody else along this, this, this way, so that, um, you, uh, can have our AI Agents and the things that we're building, working with other te, other companies, in addition to custom AI Agents you build yourself. Because we're unstructured data and a lot of people need to use it, we integrate with, um, uh, other platforms just non-AI, in addition to AI integrations that let other companies call into our AI capabilities, to ask about data, to search, to do deep research, to do data extraction and so on. Oh, I think, you know, Aaron has been pretty vocal about, uh, you know, Box, you know, different ways you're thinking about, uh, internally at Box, uh, in adopting, you know, V-Rag or, uh, Agents or, uh, co-pilots, or, um, how it's impacting the broader AI, uh, ecosystem, or enterprise ecosystem. Was there a moment within side the company where you guys realized that this is a big shift? I mean, Box has been around for what, 20 years now. Started in 2000, 2005. You've stayed through the router. What's that an internal moment where you said, okay, this is really big for us, um, and can you talk a little bit about that? Sure. Yeah. So, um, uh, if, if, if you look back like, um, search like, uh, five or six years ago, um, for like the term, like we would use the term ML at the time, like ML and unstructured data, you find, um, like, uh, we had a lot, big announcements around how Box uses ML for, for your data, to structure your data, to take unstructured data, structure it, is a big thing that we've done for many years. And so we've always had this kind of, um, sort of, uh, trying to be on the bleeding edge of what is available in the world first for structured, uh, data. Um, and, and, and, uh, but there was this kind of challenge is that, like, if you, if you had something like, like imagine a company, they have like, uh, you know, these, these, these forms that, that, that people are filling out, or they have these documents, so that is contracts, or these leases, or these research proposals, or are, are these images? Like anything that, like a company does day-to-day. Like what are people right now in the world when they're working? What are they working on? And a lot of the is unstructured data. Um, but if you're going to have like AI help, or ML, old ML help you, it would be like, uh, you'd train the model. You get the data science team together, or you go buy a company. We like, we would see like, like originally when we started got into this, this, this world, we were like, okay, we're going to get, um, an ML, uh, model that would be able to, uh, handle contracts, to be able to like kind of structure it. Turned out that like that's too complicated. What you need is an ML model to not just contracts, or not just like, uh, leases, but like, at some point you've got one commercial leases in the UK, uh, in the last three years. And you'd have a model for that and it kind of was, didn't really work that well. Like you'd have to train and customize it a lot. So that was just the nature of how it used to be, like a long time ago, being like, you know, four years ago. Um, and so, um, when Generative AI came out, and we, we were watching in the early days of like some of the GPT two style of, um, then we were like, oh, that's okay. It's, you know. But then somewhere around the time the ChatGPT came out, GPT 3.5 style of, of models, suddenly you started to see this amazing moment where a general, um, uh, purpose model could actually start to outperform the specialized models. And it could do things you never even have bothered ever tried, to get like, what is the risk, um, assessment of this, this contract? Where like, what, um, you know, things like, can you describe whether you think this image is, uh, is production ready for a catalog? Like, those used to not really be things that anybody would bother to ask an ML model. You couldn't even imagine the feature set that you would give into an the traditional ML model. But Generative AI could kind of do it is kind of one of the emergent properties of this. And, and as they got better, GPT 4 was just was like sort of, uh, this like big, like kind of like, oh, wow, it's like, you know, some of the challenges of, of the older systems are, are the older models are being fixed. And so, so GPT 3.5 is, is, I think the moment where we said, let's just go back and retrofit everything about Box to be able to apply AI models on top of it, so that you can do things like chatting through documents and extracting data. And, and then it was, um, it was amazing how fast you could actually get things working and get them working better than you ever had before, even though you'd spent, you know, like a ton of engineering and, and, uh, and, uh, in development resources on trying to get something working, then it just like an hour and a half of using one of the new models actually gave me better performance. So that was a big, "aha" moment. And then of course, you realize that, like, uh, you, you've got 90% of the problem and the last 10% problem is going to take all your time going forward. But really, since then, all of our efforts have been around preparing, um, Box to be an AI, uh, first platform where AI would be a big part of everything that people would do. We often talk internally like, what if we were building Box tomorrow? Remember, Aaron's our founder, and he still thinks in that way like, uh, even after a long time. And, uh, he, and it just comes up all the time. Like if we were building Box today, clearly it would be an AI first experience. So, why don't we do that? Like, and then that's just part of our mentality. So, uh, talk to me a little bit, and I completely relate with the whole, you know, machine learning era of things where, you know, one of the problems when I first started like doing deep learning courses was, you know, Netflix trying to pay a million dollars for the competition where, you know, improve the recommendation systems accuracy for like 2%. Right? That was the, that was the era of like solving very specific problems of recommendation systems and machine learning systems. But what are the some of the earliest use cases that you launched at Box, and how is the enterprise customer adoption been like? Because in enterprise, you know, we often see the cycle of adoption is a little bit slower, a little more complicated. But what has your perception been in terms of like, what are those features and then, how's the adoption been and what is the pace of the adoption? Yeah. So, um, the, that's, uh, so some of the first features that we, we, we, uh, uh, launched around AI were this idea of, let's say you have a, you're looking at a document. Like from our viewer, we're like, obviously, going forward forever from now on, you need to have like an AI, uh, next to you to help you chat. Like, I've got a long document. I've got a long contract. I've got this, this proposal, I've got this. Like like, help me understand it. Like it's almost like an advanced find if you will. And so for us it was, um, it was like a simple feature. Um, and but it was like it ex, it was this, um, sort of new paradigm of how people would do it. So chat with your document, right? Um, and, and then, and then the most common, people some people call it summarization, although most it's actually way more, you know, it's just like anything you want to know about it. Almost like you had a an AI reading the document and in there for you. Um, and so that was almost, like, that was very simple, uh, from the sort of conceptually about the, the, the kind of feature. Um, and then we added the concept of, of RAG. So not just a single document, but across documents, where you have to go through, and then, you know, implement the chunking, the vector databases, the ability to go through and, and be able to, you know, find the, not just, um, a document, like a search, but actually finding the answer to your question. So I've got 100,000 documents here and this like, let's say lemonade portal of all my product documentation, that I'm going to salesperson. I need to find answers this question, ask it. And the AI will, you know, quote-unquote read through all of it, using RAG and then, and then go through and provide you the answer. And then, and then a key to all this, to your question is, um, for enterprises, they were scared. I mean, and some of them still are, about AI, um, because it's so different. It's such a big change. And data security is critical to every single thing that they do. Like nobody will, even no matter what benefit of AI, if, if, if you're going to leak data, if it's going to be a source of data security challenges, remember, this is some of their most critical data they have, no one's going to use it. And so many cases, like some of, like, um, even still today, like cus, big, a lot of, like, you know, bigger organizations, like the first AI that they're actually using on their production data is, is Box, partially because it's very hard for them to trust AI, um, companies. Because not, you know, not only you need to trust the model, you need to trust the, uh, the person calling the model, you need to trust the, the, the person who has your data. And since Box is sort of that whole stack for them, they were able to say, well, I'm going to, I trust that your AI principles and AI approach that you have will be secure. And then they're able to then start with some of the simple, uh, capabilities. Um, now one of the more exciting ones I think many are looking at are, are in the world of, of data extraction, where again you have these like, these, you know, not just any document you have on your desktop or whatever, but like, these are my contracts. These are my project proposals. These are my, um, press releases. Like just this whole list of like named, like content, you know, that have this like special purpose. And there's like the implicit structure to them. So you not only want to see your unstructured data, like the contract, but you want to see the fields. Like who signed it? Uh, like what time? What are the clauses and so on? And so then you can then basically search and filter and do things on that data. So this is, these are kind the kind of things that people, uh, ent, enterprises look at that. And then they say, these are very practical benefits either from a productivity perspective or from a, they help them sort of upgrade their internal applications that they have, like on some critical data. And they start to go to adopt that and then they have that internal assessment of, we get through your AI governance committees? Can you get through the AI security screenings? Can you get through making sure that the people, you know, no, no enterprise will accept you training on their data. It's the scariest thing to them in the world. So you have to like, so we have to go in and meet with the teams and, and, and, you know, explain to them every step. Go with them the architecture diagrams, show them the audit reviews and so on, so that they know that their data is safe when you're doing AI on it. That's typically their number one concern. Uh, I want to talk a little bit about the cost of, you know, leveraging AI and I think it has dramatically gone down in the past few years. I think in storage, you know that you know, cost per gigabyte, you know, has gone down, you know, probably 90, 100% over the last 40 years. It used to cost like some $600 per GB to now it costs like $1,1 cent per GB depending on provision. We're sort of in the similar trajectory. Uh, in terms of like for you as a business, uh, are you seeing improvement in your margins by creating AI products? Because you, you've launched V-RAG, you, you launched co-pilots. And you can also talk a little bit about how you're pricing these things. But is it directly impacting your, uh, profitability as a company, or is it still like, um, it's breaking even, but, you know, over time we see that changing? Yeah. This is, this is a particularly hard problem for, uh, for, for me and for, for, for our company because, like, um, depending on, like we're a public company, right? And, and like, you know, we, we, we publish our, like, you know, here's our gross margin, here's our expenses, here's what it's going to be going forward. And so, you know, it's just not practical for us to do something that would like double our expenses, you know? Like, um, and, and so like, uh, like, nobody has $100 million laying around to just apply to whatever cool ideas. Um, at the same time, um, it's very clear that if you're too, you know, worried or stingy about your AI bills, then you will lose, uh, to somebody who is just, you know, uh, like, like trying harder, uh, to, to, to, to find the new, the new future. And so there's been a really nice sort of, um, byproduct of all the innovation going into, to, to, the chips, into the models, into the efficiency is that they're much cheaper than it used to be. Uh, although, um, uh, I'll, I'll quote Sam Altman from a long time ago when he, he, he had come by and ne, done some, some conferences with us and he made the, and, and somebody asked him, this is like three or four years ago, like, uh, like do you think models are going to get cheaper, right? Because they were so expensive at the time. And his answer, um, that, that he told us, written, uh, is, which I think is still true, yeah, absolutely. It's going to get way, way, way cheaper, dramatically cheaper, or something that cheaper. And you're also going to find that you're going to use it more and more and more and more and it's going to slightly offset that. So, and that's exactly what we found, is that we are doing way more tokens than we did previously, orders of magnitude. However, we're now, um, utilizing, uh, the cheaper models and they're just offsetting. Like so, um, so something, especially when you get to Agentic capabilities, like where you, you take, like let's say that you want to do deep research on your data, right? So that's way different than RAG. If you do RAG, you go through, you go, you know, take, um, you know, uh, hundreds of gigabytes, go through, find the appropriate level of, you know, like I don't know, 10,000 tokens, 20,000 tokens, go through, pick out the thing, give the answer. Great, you did, and your AI bill is your infra and database capabilities plus 20,000 tokens. But if you want to do deep research on that, then you're going to be like, okay, well why don't you go through look a lot of different documents, uh, for, you know, 10,000 tokens at a time, 50,000, 100,000. Then go through and re, reprocess that. You might spend hundreds of thousands of tokens or more. And, uh, and then that is just a, a like a massive exponential growth on the concept of your, of your, um, uh, of your AI spend. And so you typically get a great result. Like doing deep research on your own data is just like kind of revolutionary. You think, oh, we no way we would even consider that a couple years ago. But it's, it's, it's very expensive. So what, what, uh, the way we approach it is to say, we're going to try as much as possible to give AI for free, because that's what an AI first platform would do. Um, now sometimes if there's like a very high scale because we offer the Box as a platform, you build on. We need, we then say that, like, um, uh, you can pay. Like you can, you can like, if you wanted to do 100 million different, like data extraction jobs or some, some people do on, on, on those kind of documents. Then then there's a platform fee to it. But whenever possible, we're going to eat the costs ourselves and handle that kind of risk, because that's what you want out of your best, like products and platforms. Nobody wants to sit there and worry when they're clicking on things it's going to cost them. And so, uh, uh, that's, that's just I think what many companies are doing is they're trying to protect themselves with some sort of resource-based pricing, but then also try to just say, AI is part of the product. And that's our philosophy. And that's what's really interesting because I was looking at your, um, you know, net income, and it sort of doubled in the last two years. And then I also, I, I thought it because of AI. But you're, what you're giving me information is, uh, we are not really, you know, incrementally adding costs because you're using AI. It's just pretty interesting to me. Uh, and that's, that brings up the next question, right? Like what do you think about pricing based on usage? And there's also the state of pricing based on outcomes. So once you're getting into the Agent, uh, and you are just, I'm assuming following the regular per seat, per subscription model. Yeah. Um, uh, we, we've been through every single possible flavor of this ourselves, in addition to with our customers. And so it's, I mean, I, I, I hope that right now if you're a business school at Harvard or somewhere else, that they're doing like these case studies on what it, what like how everybody had to rethink all technology pricing and how it's continuing to still happen now. Um, so like, I mean, at the end of the day, you, um, for something like pricing a product, you, you, you, it's, it's not just about the supply side of like how much does it cost and so on. It's about like what people are willing to pay and how they're willing to pay for it. Um, so, uh, like interestingly, because when we originally launched our AI, uh, we had seen that like some people who launched AI, they were charging too much and people were like, hey, it's still early, I'm not ready for that. And then there was this like massive, like you, you may have seen some of the early, like, uh, uh, companies that were like, oh, that's $20 a month, uh, for, for this, this thing like, like enterprise companies kind of style. And then, and then the adoption was terrible because nobody quite knew what to do with it, and they wouldn't pay that amount across their company. And then, and then so we were like, okay, well we're going to, um, offer it as free as part of our product. But we put a limit on it. We said, if you, um, do too much and, uh, it, it, it's, uh, it, it'll stop. And then this was like a, you know, like, a, a big decision internally because it was like, well, if we're not careful and everybody uses death, we're going to have a giant bill. But then, uh, people would actually not turn it on because they were enterprises, because they were worried they, they would hit those limits and then everybody would ban them. So it was funny that the, the concept of the limit, became an adoption barrier. Um, and so then we had said, okay, like we got a lot of feedback from our customers and then again, in, in big enterprise customers, and, and we were like, okay, we turned that off. Um, so there was, there was a no limit. Now, there was this idea of abuse that, that we could say, like if somebody was clearly, you know, like you can't just like buy a seat to Box and then use a hundred, you know, like, like use API to like, you know, power another system. But, but through normal usage that for people we just said, we're going to handle that risk. And we do stuff like this today. Like it's incredibly expensive, uh, if you just like look on the online rates for many public cloud about like transferring data, uh, storing data, like this is, like we're used to infrastructure expenses. So we were then saying, we're going to eat the cost of it, as a way to just deliver better services to our customers. And that is continues very our philosophy. In the, in the cloud world this is pretty common scenario when you're setting storage services or compute services with, your transactions are going beyond the limits. And sometimes some products implement some kind of throttling and the, and some products don't and, you know, they just, they just make sure to adjust provisioning over time and uh, distribute it across customers. And so different products have different strategies in terms of how to control that throttling. But the interesting thing was like I think, uh, code, uh, or someone, uh, from Cloud was tweeting like some, some of the developers are calling it 24/7 multitude tenants. Yeah, right? So, have you got any such scenarios so within Box? Uh, yeah, I mean, like so people are clever. Um, and so if you have a a pricing scheme that is, um, can be abused, you're almost certainly, uh, uh, hit the, like, um, the abuse, uh, uh, some, somebody, somebody's going to figure it out, right? Like so, so, so Box, you get unlimited storage. Now and if you read the fine print, you have unlimited storage as a user and if you're a person. And that doesn't mean you can power, you know, like a, a hundred, you know, petabyte system with by buying one user on Box. That's like, that's a platform use case, that's different. But, but people, especially developers are quite clever, and they will find a way to do it. So we don't, but the the the like I think if you're overly worried about that, uh, if, if, uh, as a startup, then you're going to miss out on the people who just want to use it and, and, and, and get more value out of it. And, and so you, um, so for us, we, we typically, we do have abuse prevention, that's important to everything we do. Some people will try to like, you know, completely abuse the bandwidth and the storage on Box. Um, but, but if it's a customer, a real customer who just wants to use it, we don't stop them. Like, people can, can actually overwhelm our, like, like they buy this amount and then they, um, uh, use this amount. And, and, and it's the same with our users. We don't stop you if, if, if you want to use more users. We don't stop you if you want to use more API calls. We don't stop you if you want to use more AI, because that's great. That's what we want. That's our job is to let you use valuable things. And to the extent that it becomes a problem from a like a contractual or commercial point of view, we'll contact the customer, we'll talk to them. Um, and, and that's, that's, uh, the benefit we have as an enterprise company to be able to have like, uh, sales people and account teams that talk to them. But, but, but I would, um, we typically don't try to stop real usage. That's, that's great for us. So, uh, one of the interesting thing is, uh, you know, storage is such a horizontal sort of use case and, you know, maintaining content on Box is also a horizontal way of thinking it. But then, uh, initially all AI use cases were horizontal. Like we created a co-pilot, Microsoft has a co-pilot, and lots of co-pilots are, farm factory scheme in, which makes sense. Uh, but then, uh, you also, I mean, any companies is selling solutions for different industries. And Box also sells a solutions for different industries and you prioritize those kind of features. Um, did it come to you like, uh, just purely in terms of product strategy, you know, uh, we started the split in terms of new products coming in AI, which are specifically targeted for a particular, you know, knowledge worker or persona. Uh, like Cursor is targeted for the developer. Uh, Harvey is, targeted towards, you know, legal assistance or, you know, um, junior counsels or junior counsels in a law firm. And most of the data is in Box. Like, is, is, is, is this something you evaluated in terms of, okay, hey, we have an opportunity here where you can