SiliconANGLE theCUBE- info
- Transcript
From theCUBE studios in Palo Alto in Boston, bringing you data-driven insights from theCUBE and ETR, this is Breaking Analysis with Dave Vellante.
Dave VellanteIn just two short years, the entire data and technology industry is transformed. The tech stack up and down is being optimized to take advantage of extreme parallel computing or what some call accelerated computing from silicon to infrastructure and throughout the software stack and nowhere is this more noticeable than the data stack. Now, while the modern cloud native data platform defined the agenda over the past seven years, open table formats, shifting control points, open source governance catalogs, and of course AI awareness have created new challenges and opportunities for enterprises and the tech companies that serve them. Hello and welcome to this week's theCUBE Research Insights powered by ETR. In this special breaking analysis, we're pleased to host our fourth annual data predictions power panel with some of our collaborators in theCUBE collective and members of the data gang.
With us today are five of the top industry analysts focused on data platforms, Sanjeev Mohan of Sanjmo, Tony Baer of dbInsight, recent IDC graduate, Carl Olofson, the always engaging Dave Menninger, who is with ISG, and Brad Shimmin with Omdia. Guys, thanks for being here. Really appreciate the collaboration and we're very excited. Always a pleasure for our annual data lookout. Welcome.Thank you so much. Glad to be here.Now before we get into it, you guys are going to have plenty of time. I always want to share a few survey tidbits from ETR to sort of set the table and underscore how much the industry has changed. ETR runs quarterly spending intention surveys. We're talking about 1,800 and roughly IT decision makers or the target, and I want to just isolate on the ML/AI space and show you how things have changed. So this graphic here shows by sector, the vertical axis is net score or spending momentum within a sector, and the horizontal axis is pervasion in the data set for each sector based on account penetration. Here we're going back to January of 2023 and the red line at 40% indicates a highly elevated spending velocity. Now you can see we put a box around ML/AI and you can see ML and AI along with containers, cloud and an RPA.
They were on or above that red dotted line. Now let's take a look at how that's changed over the last 24 months. It's no surprise, but look at both the trajectory of ML/AI over that time period and look what happened to the other sectors. ML/AI shot to the top, the other sectors got compressed. Now you look at the cloud in this taxonomy and it's a time series data set, so the categories are rigid. The point being we know that a lot of the AI work is being done in the cloud. Nonetheless, this data really shows the transformation of the tech industry and specifically the spending priorities. Now we want to take a look at the players. Last year, you might recall we combined the data for BI analytics database and ML and we put them all together. But let's just look at the ML/AI sector. Here's the picture from January 2023.
Same dimensions, spending velocity and penetration into the data set. The cloud guys, they're pretty much bunched along Databricks right there. Databricks stands out. Some of the traditional AI companies you see in there, Oracle and IBM. Now, fast-forward to 2025 and look at this chart. Same dimensions but big changes. So when we've inserted a table in the lower right, so you can see how the dots are plotted. Again, net score and ends in the survey. Look at OpenAI and Microsoft in the upper right. OpenAI wasn't even on the enterprise radar in 2023 and now it and Microsoft are ubiquitous in terms of account penetration. AWS and Google are right on top of each other and as we've reported in the past, Google's closing the gap on AWS. Snowflake wasn't even on the ML/AI graphic in 2023 and it and Databricks are right on top of each other.
No surprise there. Databricks has more spending velocity. Snowflake's got slightly higher account penetration. And look at Meta Llama, it's overtaken OpenAI in spending velocity last quarter I think was the first time that had happened and it's right on top of Anthropic, which of course you could link to AWS to a Bedrock. TensorFlow we know is not a vendor, so just ignore that, and you can see the legacy players all bunched up. IBM Watson kind of dropped noticeably this quarter. I'm not really sure why, it had been more elevated recently, so we'll watch that. But the point is the market has transformed right before our eyes and the race is on for customers to figure out where to place their bets and the technology vendors are battling for their positions. So we're going to try to help both buyer and seller figure that all out today.
With that as background, let's get into the predictions. This graphic just shows all of the 2024 data gang predictions for each analyst in one table shows the prediction and the analyst self rating on whether the prediction was a direct hit, which is green, a glancing blow, which is yellow or a miss, which is the red. So a quick scan of the heat map shows you that the data gang, as usual, did pretty well on its 2024 predictions. Now, notwithstanding that these were, as I said, self evaluated by each of our analysts, so we're going to review each of the 2024 predictions and you can decide for yourself how accurate they were. Let's start with Sanjeev Mohan. We're showing Sanjeev your prediction about data and AI stacks supporting intelligent data apps. You feel this happened in 2024. Take us through your prediction and defend your green rating. Please, sir.Thank you, Dave. I see this happened all throughout the year. If we start even in the beginning of the year, we published our podcast in January by summer when Databricks announced we could see AI/BI, and there's a lot of integration overlap of AI/BI all the way throughout the year, we saw managed Lakehouse is being launched till AWS re:Invent, and that was actually I think the biggest kicker where we finally saw things like SageMaker, Lakehouse and IBM, sorry, AWS revamping, actually IBM too, the data fabric. But AWS revamping the entire offering to just hone into the SageMaker brand. Until we went to re:Invent, we thought SageMaker was a tool for machine learning and in fact, to be honest, we weren't even hearing too much about SageMaker.
We were hearing a lot more about Amazon Bedrock, which is a generative AI offering, and all of a sudden we find out that SageMaker is now the common tool where you've got the module catalog, you've got the metadata catalog, you've got S3 Tables exposed through it, the broad Redshift and S3 together very cohesively. I do have to say that the vision that AWS presented was exactly in line with what I had written about and I talked about. Implementation is a different story, but we'll get there over time. So that's why I rated it as green.Good, I'm glad AWS is paying attention to what you write and produce. All right., Thank you Sanjeev. Next we go to Tony Baer. Alex, if you'd bring up that same exact graphic, we're going to be rapid fire here. Tony, you predicted, which is the second prediction on this chart that GenAI would simplify database design, deployment and operations, and you gave yourself a mix of yellow and red. Thank you for the honest assessment, we appreciate that, tough self grading. Explain your 2024 prediction in more detail in your assessment of its accuracy, please.
Tony BaerWell, my philosophy is that I only learned when I'm wrong and I learned it quite a bit last year, which is that, I mean, the idea was pretty much, I think it's still long-term on target, which is that generative AI should be simplifying basically how we design and run databases. And that partially came true if you consider say like, for instance, SQL Copilots or hydrating metadata with metadata discovery. But I think in terms of the operation and design of databases that has lagged, there have been a couple notable exceptions and I think the obvious one would be Oracle Database 23ai, which uses a language model to essentially decipher JSON documents and generate relational schema. That was the type of thing I expect to see more of last year, but really have not seen whole heck of a lot of it. Just a couple other isolated examples, NVIDIA NeMo Curator, it's a framework for building data pipelines for training language models, but in itself it doesn't use language models to help you generate those pipelines.
I think basically as we have these copilots, I think it's a logical next step for that. So at this point I would say that my thoughts, my predictions were I think ahead of the game. I want to just go back a little bit to what Sanjeev was saying and that most of us on this session were at re:Invent and we were positively blown away by what AWS did with SageMaker if only because of the fact that for AWS it's a complete change in their modus operandi, which is AWS has traditionally operated through lots of individual product teams, and here we finally saw them basically change practice and bring them together. So for AWS and SageMaker, 2025 is going to be a very interesting year.
Dave VellanteYeah, thank you. And I actually made that same point to Andy Jassy when we were talking to him at re:Invent. He said, "Well, don't forget about Connect." That's a solution too. I was, okay, fair enough. All right, next up, Carl Olofson talked last year about data unification and the importance of security and governance in the data stack. You're showing green. Yeah, it was a pretty safe bet, Carl, I'm just going to say, but I wonder if you could elaborate on your 2024 prediction because you had a few little proof points in there that led you to that direct hit evaluation.
Carl OlofsonYeah, well, before I comment on mine, I want to make a comment about Tony's in that I think that he was on target with the prediction that GenAI will eventually support better database design. It's just that we don't have a time horizon here. We aren't saying by such and such a time. So it's that it may have taken off a little more slowly than anybody expected. I think it's in progress. We'll see that evidence later on down the line. It's always difficult to predict when... It's easier to predict what things you think will happen, it's hard to predict when.
Dave VellanteWell, let's make a note of that, Carl, for our 2027 predictions. We'll give Tony some props when it comes true.
Carl OlofsonOkay, yeah, we'll save it. As far as mine goes, in order to effectively apply GenAI against data, first the data has to be organized and well-governed. And so my expectation was that the emergence of the desire to use GenAI to scan databases and across databases and have meaningful results come out of the system, the data has to be sufficiently well-documented and governed. And are we there yet? No, by no means, but we see clients working on it. We see them doing a lot of this stuff. And also large application vendors like SAP and Oracle are on top of these things with their own admittedly sort of captive systems that everybody uses the same way pretty much.
So I think that what we expect to see is more in the future. So again, there's no time limit on it, so I get to say that it's in progress. I certainly don't think we're near the point where database systems are all well-organized and slotted in and governed and so forth and made ready for AI, but they're evolving in that direction.
Dave VellanteAll right, thank you. Dave Menninger, I know you always love to bring the data you do every year. You predicted that non-GenAI or LegacyAI or whatever, traditional AI, all bad phrases, has a life and not going to be replaced by GenAI in demanding use cases and you discuss the skills challenges that organizations would face in 2024. We show here a green level of accuracy for that prediction. If you would bring that up, Alex. Thank you. And so wonder if you could elaborate on your thoughts.
Dave MenningerSure. I'll start with the data point and this data point's very recent in December, we published a study where we looked at how organizations were breaking down their spend between generative AI and I think we might've labeled it predictive AI, but you could call it good old-fashioned AI or any of those other terms. But GenAI and predictive AI are still about 50-50. The exact numbers were 46% GenAI and 54% predictive AI. And the interesting thing was that was nearly identical to what we saw earlier in the year. So it's not like this data's out of date, it's not like it's changing rapidly. There's a ton of interest in generative AI. And generative AI, in fact, in my opinion, is pulling AI along with it because traditional AI was so hard to use. However, the skills issue still remains in our research. It's the number one inhibitor to AI initiatives. So I stand by my prediction and I think it's pretty accurate to say it was green.
Dave VellanteYeah, I think you nailed it. All right, Doug Henschen could not be with us today, unfortunately, and we missed you, Doug. He predicted that GenAI would change how organizations deliver and consume BI analytics and predictive recommendations. I think the data gang is right and agreed that the evaluation of Doug's prediction is accurate. So this definitely was the case. I think we're going to move on and get right into the predictions for 2025. Let's do that and get to the core of the episode and really focus in on this. We're going to keep the same order except Brad Shimmin is here and he's going to replace Doug's slot. Now what's going to happen is the designated analyst is going to present his predictions and then we'll have time for one or sometimes two other analysts are going to chime in on that assertion.
Feel free to chime in. We can debate, discuss. Here's a table showing all of the predictions for 2025. Thank you guys. They're all data related. This is after all the data gang and lots of agent talk, LLMs, LAMs, SLMs, and interesting security angle from Brad that I'm excited to hear about. So let's get into it. Sanjeev, you first please. Alex, if you bring up the first slide in the 2025 predictions, you're predicting that by this time next year, this was really interesting, most of us will have our own personal digital assistant. Why do you think that? What will that assistant be capable of doing and what gives you confidence in this prediction?Yeah, I got inspired by Tony and I want to be proven wrong next year. No, really I wanted to go out on that limb and maybe I'll regret saying this, but I think the reason why I think personal agents are going to take over is because A, they help us with productivity. Secondly, they're not as complicated as what people are maybe at times they're overglowing what an agent can do. So we are still in very early innings of agents. I don't think agents are ready to take over a very complicated workflow within an organization, but a personal agent that does it's like my assistant, my emailing, my scheduling, my invoicing, all the tasks that I do.
It learns from all the things that I do and it automates some of those things for me is something that I'm predicting that most of us on this call will have it next year. And why I'm so gung-ho about agents is because underlying models have become supremely more sophisticated. Last year at this time, the entire movement was on how big can we train a new model to be? How much more data can we throw at the model? This year we've gone from pre-training to post-training. It is not about how much data that a model has been trained. It is how much data can the model consume at inference time, make a reasoning choice and come up with a number of different options, which we call test-time compute, and then find me the best solution.
So the paradigm has shifted completely, so the models have become pretty advanced compared to last year. The second thing that I also want to predict is that there will be a new category of systems that are going to be developed this year and we'll call them agent management systems. Some of stuff is already happening, like agent frameworks, Microsoft has released some stuff through AI, there's so many of these, but they're still very, very basic in what they do. There was a survey I saw recently where every AI engineer is having to use five to 15 different systems to build these agents. So the agents are taking a long time to build, they cost a lot of money right now.
And agent management system is an end-to-end agent life cycle development system. So it's like a SDLC. This is for agent. So agent development life cycle. In fact, I'm also predicting, although this will happen in 2026, so won't be true for our call next year this time, but in 2026 I'm predicting Gartner will have a magic quadrant on AMS, agent management systems.Great, thank you that. Now Brad, you've got some follow up here from Sanjeev.
Brad ShimminOh yeah, Sanjeev is absolutely wrong. Sorry, Sanjeev. Just kidding.
Dave VellanteWe love what you think outside.
Brad ShimminI think Sanjeev really hit this on the head when he said that we're in very early days and that there's a great deal of promise in this. And I would go so far as to say that this idea of agentic systems, meaning semi-autonomous or fully autonomous workflows that can basically do a lot of different tasks in a lot of different places, meaning on my desktop, on a server, in this database, in that app, et cetera, is going to change the way that businesses build software. Already we see developers, full-stack developers, just looking for the most expedient OpenAI API call they can do to get something done like just a named entity extraction or a summary or whatever, that is going to be agentic workflows as we go along. And so as Sanjeev says, if we don't manage that, if we don't have a good platform upon which to build these in a way that allows us to control the security through the delivery, meaning performance and all that, it's going to be very hard for us to actually put this stuff into production.
And we're not. I mean, in our job at Omdia, in my job at Omdia, I track about 30,000 AI job postings every quarter just trying to understand what people are hiring for. And so naturally we wanted to know what's agentic looking like. And if you look at the 30,000 for just the last quarter, it was roughly about 60 jobs that were specific to this. If you compare that with generative AI jobs, which is about 2,700 jobs in the same period, you're like, "Okay, it's early days, it's not happening yet." And I think that's for a good reason and also perhaps a little bit hard to gauge because it's the same technologies. We all have to realize that agentic AI is not a new form of AI.
It is simply an enhanced application of the existing generative AI models and tools that we already have. So what we're starting to build is basically the systems that use the capability a model has to do things like plan and reason and execute. We're giving them context, we're giving them meaning, and mixing all of those up in a nice stew is allowing us to basically build some complex systems with these frameworks that Sanjeev mentioned. But I've been tracking these things for a while. So like looking at LangGraph from the LangChain folks and OpenAI Swarm, they have theirs and Microsoft, as Sanjeev mentioned, just released a new one, but they've had AutoGen before that. And those three are the top frameworks we see people hiring for. And it's still a very small percentage that are doing it. And it's... Yeah, go ahead.Sorry, Brad. Brad, these frameworks, agent frameworks, they help you develop the code. But more important than development is this ability to either manually or automatically evaluate brand new models because models are coming out all the time. So how do I... And then you touched upon evaluating for performance for cost, and being able to monitor agents when they're put into production. So the AMS will help you deploy, monitor, and so it's an end-to-end. So agent frameworks today are only a subset of what businesses need to do. And so that's why this system would take off.I agree. My prediction for this coming year is that the Vertex AIs, the Bedrocks, which Bedrock sits underneath SageMaker, those are going to be those AMS systems. They're already starting to do that. And then you see the same with the small pure plays like Dataiku and DataRobot, etc. H2O weights and biases are all trying to do this because they recognize just how complex this could be. And by complex I mean costly. Because if you don't do this right, if you don't have the ability to actually know the thinking that a model's doing and how many inference calls it's making, send you a simple task, send an email, you can get your costs out of control really quick.Yep. Tokens consumed. Yep, all of those, correct.
Dave VellanteAnd you also have guys like Salesforce within their own domain trying to do this. And when I think about the road journey, if you will, to intelligent data apps, you think about turning things that databases understand in terms of strings into people, places, and things, incorporating process knowledge in it. I can't help but thinking that all that data has to be harmonized. And I know, Carl, you're going to be talking about knowledge graphs, so I'd love your thoughts on that when we get there. But up next is Tony Baer. We're showing your prediction here that 2025 is going to bring a data renaissance. When I saw this, I said, "Oh, is this a dupe 2.0?" But no, you say, and you have, in addition, you have three data related predictions. So take us through your thoughts here please.
Tony BaerOkay, first off the soundbite, the dupe 2.0, hell no. Yeah. Basically what I was really talking about here is that for the last few years, there's just no question about that generative AI has really captured our imagination, I mean essentially the reason for that I think could be basically explain that we actually saw this with Star Trek back in the 1960s, which is Captain James T. Kirk talking to his computer. Well, okay, we've been typing in natural language to a computer, but it's not a big concept to get that to be able to talk to the computer. So that's basically grabbed pretty much the attention. It took a lot of the attention off of data. We really took it for granted. My take is that even though I think there's going to be a long tail in terms of generative AI adoption in mainstream enterprise applications, that still as those early proof of concepts really go to production this year, we're going to have to pay real attention to data.
And the fact is is that, and we've had some good examples of this in not so far distant past just with AI in general, that if you screw up with data in AI, you really screw up. A couple examples I'm thinking of, remember Microsoft Tay, which was basically used AI to essentially where the conversations were supposed to basically tailor the type of dialogue that Tay would provide. Well, I think Tay lasted, I'm not sure how long, maybe 72 hours or something like that before Microsoft pulled it off because it started going way off color. There was also when the original IBM Watson when Watson was cognitive computing about a decade ago and making faulty recommendations for cancer treatments. That's a case where basically faulty data is a life or death issue.
And when you look with generative AI and enterprises now starting to adopt, basically implement some of these generative powered applications in production, there are significant reputational and legal ramifications if that data is not good. So thing is that, and I'll make some exceptions at the end in terms of technology predictions, but most of this is not going to be about new technology for the most part in that when you look within, let's say, the data life cycle, there's already a lot of AI being used, whether it be essentially for ingesting data, for wrangling data, doing data quality.
Even now, even some of the MDM folks have finally belatedly discovered that too in terms of being able to use a language mouse to hydrate that metadata into becoming essentially like de facto business glossaries for instance. And just a couple examples I can think of off the top of my head just in terms of the data life cycle, Informatica's basically data management cloud is right with CLAIRE Machine learning and now some generative, which is essentially guiding the process. SageMaker catalog includes liberal doses of language models to hydrate that metadata. And you're starting to see that with what's next and others, I mean too long to go through. I think there will be a need for some gap fillers, especially with regard to correlating lineage between AI and data governance tools because it's really two sides of the same problem in terms of that when you run a model, an iteration of a model, you didn't know which iteration of the data it was run against that produced that particular result.
And we have lineage, we have drift detection on each side. We need to put those together. I do admit though, that in terms of basically getting metadata from unstructured sources, you can do that today. There was a good Amazon technical blog that described the process. It was pretty, let's say, complex to say the least. I think there we are going to start to see some tools, some frameworks that really start to put this together and eventually help provide a guide experience. Now another piece though, I'll go out on a limb here with technology predictions. As I said, most of this is not going to require new technology. It's really going to be primarily, I think, driven, as I said, there's some integrations of the tool space, but it's really going to require a lot more business involvement with the technology side because essentially here we're going to be very heavily basically relying on domain expertise to really understand are these models going off course or not.
Now in terms of data prediction, or I should say technology predictions, okay, here, I'll stick my neck out here. Data lakehouse, I've been following this pretty closely for about the last, about two, two and a half years arguably. This year, Iceberg is going to kill it. I mean, we knew it was on that trajectory for a while. The question that we had last year when Databricks swooped in and bought Tabular. Of course Databricks is known for developing Delta Lake, which is now an open source project. It was not originally open source and it was Apache Iceberg, which really was a true community-based project. It was basically developed by Ryan Blue, I believe when he was at Netflix, and he formed company Tabular out of that, and Databricks bought this company.
And so we wondered back, and this is about six months ago, what would come with this was this Databricks CEO, Ali Ghodsi, making a power play here, hijacking this. There were a lot of theories running rife at that time. I mean, of course we heard all the promises. You're always going to hear promises. But the good news is that about six months later, maybe a little shy of that back at re:Invent, I had had a chance to sit down with Ryan Blue for a good half hour. It's amazing to get a half hour of anybody's time at re:Invent. He talked to me about what's been going on with the project ever since and the good news is it's going on as usual. Snowflake and Amazon are still collaborating very closely on the parts of the project that they're already working on. And the fact is out of that, my conclusion is that, and I know, Dave, you've been sort of trying to pin me down.
Does this mean that Delta Lake is dead? What I'll say is that Delta Lake is going to get subsumed into Iceberg. It's going to be a gradual thing. You already have that with UniForm. And the fact is, I give Ali Ghodsi Databricks credit that not he's not finding religious wars on table formats. Where I do think the wars are going to happen is really at the catalog level because that's where you start getting into governance data discovery, and that's really very much more at the application level there. I know that Databricks would love to make Unity the de facto open source catalog for Iceberg eventually. I don't think that's going to happen. I think there'll be some advancements with regard to the Iceberg REST API, which is a REST catalog, which is a catalog, but it'll be very basic metadata.
Another prediction I'm going to make, and this is going to segue into I think what Carl's going to be talking about, which is on RAG that today to really build RAG applications, it's a pretty complicated process. I think what's going to happen when RAG is the same thing that happened with machine learning, which is we're going to start to see auto-RAG, so to speak, that will help to automate and guide some of these processes. For instance, how do I chunk data? What's the right granularity? And also can I do A/B testing of different language models here? And out of all this, I'll stick my neck out in one other way, which is that Google is going to buy Pinecone. Because if you take a look at Vertex AI's RAG tool, it uses Pinecone as their vector index. You tell me when the heck is Google going to basically buy Pinecone?Very interesting.
Dave VellanteAll right, so thank you for that. And I actually have some data, but I want to move on iceberg adoption and Delta adoption, but I'll leave that for now. But Carl and Dave, you both have follow up to Tony's comments. Carl, why don't you go first?
Carl OlofsonOkay, so first of all, thanks, Tony, for that. I feel like I just sat through a college seminar on the data management and the GenAI age, but I just want to go back to the very first thing, the very first topic in the prediction, which was that we're seeing a data renaissance. And in my opinion, not only is that correct, but I just want to give a little bit of extra color, which is that sometime ago I was approached by someone at one of the conferences that we go to and we go to so many, I can't remember which one it was, but the person asked, "Do you think that with everything going on in GenAI that databases are dead, that there'll be no databases in five years?" I guess they were thinking, we don't need to put data in databases.
We'll just write stuff down and GenAI systems will manage it and report on it and answer questions about it and all that. And I said, "Absolutely not. In fact, I think that structured data is going to become more important than ever." So as an illustration, years ago, many of you may remember this, if you've ever been to any store checkout, you'd be asked extra question. When you go to checkout, they'd be asking you these questions about what's your zip code or something like that. Well, that was because they're trying to do analytics on the sales data, but they needed some extra information to flesh out the report on the sales data. Well, that's just a little thing, right? But what's going to happen in the future is that people are going to set up their systems that they're going to reach a point where, yes, they finally have all the metadata in place, all the definitions and governance and everything synchronized, and they're going to ask questions and they're going to get back partial answers, and they're going to wonder, well, why is the answer partial?
And they'll ask their technologists look into it, and they'll find out it's because not all the data is there that's needed to answer the question. So they're going to expand the data that they collect, how they collect it, which in many cases may be not as a result of asking people questions, but direct inferencing by the system. But in any way, the data will come in and it'll be recorded and used. So databases will not only not go away, but they'll grow and become more complex, which is okay because the systems will manage them, which is a good thing because they'll get too complicated for us to manage, but don't worry, there's still plenty of work for database professionals to keep all this under control. And so I think that what Tony said is spot on and that this is going to become take on a life of its own. It's really going to become a massive juggernaut of data.
And the great thing is that because AI sits between us and the data, we won't think in terms of data anymore. We'll think in terms of asking questions and getting answers to Tony's analogy, which interestingly enough, I used at an IDC Directions conference 10 years ago, but actually more than 10 years ago, but I don't want to get into that. So great job, Tony.
Dave VellanteWell, like you said, Carl, oftentimes we're not talking about the timeframe. So there you go, 10 years back. Dave Menninger, you have some follow-ups as well.
Dave MenningerYeah, I just had a couple quick comments. First of all, Tony, I didn't know we get to make three or four predictions. I'll prepare next year. But we've been talking about this a lot internally at ISG as well. We see it in our clients' contracts. So we have visibility into $200 billion worth of services, contracts that are issued and awarded each year. And in studying that data, we see that half of the AI contracts include data management services. So clearly data is important to AI. So that's one piece of evidence. Another piece of evidence, you look at some of the vendors like SAP. SAP offers a clean core service to try and get your data ready for AI. So clearly you can't have good output from AI processes if you don't have good data. So Tony's spot on.
Carl OlofsonSAP has an advantage in that if you're an SAP customer, you're already operating under their schema. They're already under operating under data that they have managed, designed, and that they have complete control over how, except for your local modifications for whatever reason you do them. So they have a big advantage over say a large corporation with 30 years of applications they developed over time, a lot of which are still running in COBOL and have all these old databases that they have to upgrade. So that's going to be a little more challenging.
Dave VellanteSo I said I had some data on this, Tony, we'll put a stake in the ground here. We did a survey with ETR last July, so it's a little dated and it was a small survey, only about 105 N. Which data lake or open table formats are you using now or planning to use in the next six to 12 months? Delta is 21% Hive, 27% Hudi, single low single digits, Iceberg 16% with a very, very high 40% planning to use higher than any other open table format. And then does your organizations currently use Unity Catalog from Databricks? 40% said yes for a portion of our data cataloging requirements, and 7% said yes for the majority. So almost half. And then same question for Polaris, no surprise, only 5% are even excited to use Polaris.
This is July, so it had just been announced, but so some work to do there. We'll see. So there's a stake in the ground. We'll see, Tony, how that shapes up. But I think your Iceberg prediction is a lock.
Tony BaerDave, can I just take a shot at what you're just saying? And it kind of backs up what my impressions have been. As I said, I've been following this for the last at least two and a half years. And actually a prediction where I think I'll give myself a yellow or red is that I expected Lakehouse to have a much faster rate of adoption. I think a lot of it, I think the low numbers for Iceberg versus Delta was that Delta wasn't established part of the data versus platform, whereas Iceberg basically it was just starting to gain traction. It was getting initial implementations. I mean, I think Snowflake only released its first production version I think last year, if I'm not mistaken. And I think essentially enterprises were really waiting for to see how vendors would align themselves here and waiting for production. Delta was in production before Iceberg, so I think that really sort of accounts for it, but I think as I said in the next year, those numbers are going to markedly change.
Dave VellanteRyan Blue, we did a breaking analysis. George and I did a breaking analysis with him last January or February, and we asked him off camera, "What about bringing Delta and Iceberg together?" He said, "Ah, it's not going to happen." It was amazing what a couple billion dollars can do. All right, Carl Olofson is up next. After 27 glorious years at IDC, congratulations on a wonderful career at IDC. Awesome to have you here today. As a free man, let's take a look at your prediction around knowledge graphs. We love this topic. Knowledge graphs will evolve into metadata maps and drive better rag execution and have an impact on how agent code is handled, and that will have ripple effects into small language models and AI frameworks defined by data. Okay, Carl, this is a multidimensional prediction. Help people fully grasp it, especially the non-technical viewers. What are you predicting here and what does it all mean?
Carl OlofsonAll right. So since we've been doing computing, we've been challenged to try to organize data in a way that we can generate useful output. I used to be a developer for years, and the last thing we wanted to do was write the report software. It was boring, it didn't do anything as far as we could tell. Of course the business people, for the business people, that was actually the value delivered. So we were kind of off target with that. But my point is that in order to get value out of data, you have to know what the data means, what its context is. And most enterprises have so many applications and databases and the databases are unrelated to each other. So they're not designed with any thought towards mixing the data from one database to another, which is why we have ETL and why we have really complicated processes just for blending enough data that we can put it together in a data warehouse and get something.
But actually what people want to be able to do is ask questions of live production operational data in the moment. And in order to be able to do that, you need to be able to tell a system where things are and what they mean. Now, in the unstructured case, we use large language models. The reason the language models are large is because we're taking advantage of something called the law of large numbers, which means that when you look at a pattern, it's all based on patterns. When you look at a very large pattern, then you can sort of ignore the outliers and say, "Okay, these things are probably related pretty closely. And these things are probably pretty closely related to the words that appear in the prompt. So I'm going to give an answer that's based on that complex of relationships, that proximity relationships that I've discovered, which is fine.
" In the database world, you don't get that for one thing. There isn't enough data to build a large language model for another thing, unlike human language, which has built-in semantics. So the semantics form are responsible for part of the patterns that are being picked up. There are no semantics in structured data, so there are no patterns to detect. So we have to provide them, and we can do that a little bit through definitional metadata. But the thing that can really make it strong, the thing that really provides not just the data definitions, but also the context is some kind of map that you can navigate through the data to find out what's related to how, when, under what conditions, et cetera. And that kind of mapping can be done using a knowledge graph. And so it's my view, and we're already seeing work going on.
I've been to a couple of graph conferences and listened to people talk about what they're doing in this regard that involves just this sort of thing, building a model of your data. And here's the good news. Now you might say, we've got hundreds of databases and they're all different, and this is going to be a lot of work. The good news is that you can actually use AI to help you build the knowledge graph. So it's not quite as bad as it is. I'm not saying there's no work involved, there's a lot of work involved. There's a hell of a lot of work involved. In fact, there's a hell of a lot of work in just validating what the AI system did to make sure it didn't screw you up. But once that's in place, you can do so many things. You can ask any question and get an answer back to Tony's Star Trek example, because you've got all the live data in place ready to be pulled, and the system will know which databases to pull data from.
Right now we're in the very beginning phases. People think, oh, well, Oracle, Microsoft, Amazon, they've all got AI assistance in their databases. Are we done? No. Those things just do things like generate SQL for you. So if I ask a question and I point it to a database, I have to know which database to point it to. I ask a question and it'll generate with the SQL for me. This is like baby steps. This is nothing like what I'm talking about. So in order for this to work, you need a system that is capable of relating things dynamically. So it can't be hierarchical, it has to be networked. It can't be something that is hard to change the relationship structure because the relationship structures themselves are dynamic.
And one of the things that holds back many enterprises is that the development teams are still changing their data and they're changing their applications. You don't want that to be a problem. You want it to be able to adapt and absorb those changes and continue to function. And the best way to do that, the best technology to use for that is a graph technology. And so that's basically why knowledge graphs will drive the future of enterprise data reporting through and searching and interaction through generative AI.
Dave VellanteVery exciting space and love your guys' thoughts on whether that's a feature or a product, a category. But anyway, Tony, why don't you follow up with Carl's prediction?
Tony BaerOkay. Thanks, Dave. And I've always thought for the most part, that graph really is a feature with some exceptions know for Neo4j. but more of the point is that look at the popularity of RAG, retrieval-augmented generation. It's one approach to making generative AI driven applications enterprise relevant. There's also fine-tuning. That's a whole other approach where you're basically adjusting a model, tweaking a model to your data. In this case, you're not tweaking the model, you're basically taking an existing model and basically running queries on your data. Essentially, it's a very well-known pattern. We run queries against a database. And so why not basically run a language query against your vectorized data?
So to me, I mean, it's been a brainer not just to myself. I mean, in the past year there's been just a huge explosion of interest in RAG. But the thing is that with RAG, you're conducting similarity searches. And so for instance, let's say you're doing financial fraud, you may find two different transactions that have similar characteristics but otherwise are not causally related. And that's where essentially a knowledge graph comes in, which is that we can essentially instead of basically through similar research is getting implicit relationships, the relationships initially it's basically answering query based on explicit relationships. So to me, basically applying graph database technology, which is now pretty well established, to retrieval-augmented generation, which is rapidly maturing as a technology, and putting the two together to me is a no-brainer.
And my sense is that any enterprise that's going to get serious about RAG is going to get serious about basically implementing it with a knowledge graph underneath. And as Carl mentioned, basically a lot of the hurdles to building graphs, well, AI will help do a lot of that data discovery to help put that together for it. So essentially to me, graph RAG is a no-brainer. It's going to become the killer app for RAG this year.
Dave VellanteWell, and I just want to mention Neo4j, I had this conversation with George after our last conversation. George said, "Well, Neo4j is not even cloud native yet, and they haven't even separated compute from storage." So he's actually a fan of relational AI, you guys. You and I talked about that. So it is an interesting, is it a feature? Is it a product? Is it a market? Neo4j, RAI, some other folks. Brad, I know you have some comments and would like to follow up. Please chime in here.
Brad ShimminYeah. Thanks, Dave. And to just echo what Carl and Tony are talking about here, RAG and graph and other methodologies are just that they're tools, they're methods to basically bring context and meaning to a model and to any analytical processes, whatever you're wanting to do. And so as Tony said earlier, doing RAG can be very difficult. And I just want everyone that's listening to know that you're going to learn RAG, and then you're going to probably forget it because we're always learning about new ways to do what these methods bring us, meaning the outcome. And so if you're trying to bring context to a model, and that context is something like all of your HR regulations, do you really want to do a semantic search every single time you do a query or somebody does a query for whether they have a day off or not, why not use something like cache-augmented generation, which is an idea where you take advantage of the fact that so many of the LLMs we have now have these gargantuan context windows like Gemini with 2 billion tokens and just, I don't know, turn all of that data into key value pairs and put it in memory so it's sitting right next to the model. And you can just have the model just feeding off of that data in real time without having to worry about, did I do my overlaps correctly when I was tokenizing these raw PDFs from HR? As with many things that we're all talking about today, the industry is just learning how to use what's basically perhaps a spaceship that crashed in Roswell in the New Mexico desert and we're just figuring out what we can do with this thing.
Dave VellanteAll right.
Tony BaerActually, can I just jump in there? Which is that I got asked about cache-augmented generation yesterday. And if you look at Oracle in 23ai, they actually have... It's not fully that yet, but it's a big step there, which is that they basically have put a vector index in memory. So basically that's already happening. And the other thing I was just thinking is, Brad, as you were talking about putting this next to the mounting, we've just updated the Tableau data extract. It's a wreck.
Carl OlofsonI just wanted to respond quickly to the comment about Neo4j because I think, and it's not just for Neo4j, but anybody else who's doing graph database, if you have a highly complex deep graph, it's really hard to get any kind of performance if the storage is not in close proximity to the processing. It just has to be, it's a performance issue. But at the same time, I know that we all say, "Oh, we must separate compute from storage because that's the way it's done in the cloud." But actually it's still a question of what is the use case and what makes the most sense? And you're dealing with a database that isn't being updated frequently, but it is being read very frequently, and you want to thread the requests through the system that's going to answer proximity questions from the graph.
So there's nothing wrong with creating some kind of appliance-like thing that you put into the cloud data center that lives on its own. We already have a model for that. Oracle has Exadata in the cloud, and that as far as the rest of AWS or Azure or Google is concerned, that's just a black box. And so we've already established that establishing a black box in the cloud is not illegal.
Dave VellanteWell, I think George's bigger point when I've talked to him is that Neo4j means you got to give up on 50 years of relational query simplification and puts the onus on the application logic in recipe form, puts the onus on the developer to track the order of execution.
Carl OlofsonRight, we're not talking about conventional applications, we're talking about AIs.
Dave VellanteAbsolutely. So it is an interesting debate, and I'm really curious to see if relational AI can make a market out of this. And I wanted to comment, we are blessed with these big context windows like Gemini Ultra, which only cost $200 to train. So thank you, Google. Okay, up next, Dave Menninger, who always brings the data evidence with him, you're predicting that LLMs are going to move to LAMs, large action models. Okay, let's dig into the details of that prediction, Dave, and what are the weaknesses of today's LLMs and what are LAMs and how will this change things?
Dave MenningerYeah, so let me start with the context here. The context is that for far too long, for decades in the data and analytics world, we've left the rest of the exercise to the reader. Meaning yes, we could provide some insights, we could explain perhaps what happened in the past. We might be able to predict or we could make some predictions about what might happen in the future. But then you had to figure out what to do with all that information. That's not solving the problem for people who have to do their jobs every day, right? Okay, sales went up, what do I do? Do I do more advertising to continue driving sales up? Do I hire more salespeople to continue driving sales up? Do I raise prices because I can earn a little bit more out of the market? We don't give them the answers.
So we need to get to the point where we're helping people know what to do, know what actions to take. Actions are happening, they're just being determined by individuals. And so we have this new model now we have this model where you can ask questions and get answers. We're on the verge of being able to do this, but the large language models, and I want to clarify what you said. I think you used words about large language models having problems and maybe being replaced by LAMs. That's not what I'm suggesting. I'm suggesting that there are different types of things you want to predict. A large language model predicts the next thing, the next set of text, or maybe in a multimodal model, the next pixel that you would put into an image, we need something else.
We need something that will help predict the next action. And these actions are complex because the actions often involve multiple different systems, right? Maybe you're an SAP customer, maybe everything's an SAP, but the reality is that's rarely the case. You've got some Oracle here, you've got some Microsoft, you've got some SAP, you've got some homegrown applications. How do we know as workers in that organization what to do next? Large action models, LAMs, are a way to look at that. The body of knowledge on which they're trained are the actions that have been taken. Now, actions in this case tend to be function calling. So what function was called followed by what other function was called. There are even benchmarks. There's a Berkeley Function-Calling Leaderboard so you can understand how different models are performing relative to this style of thing.
But if you understand or even have a basic knowledge of how LLMs work, what text comes next? Large action models are about what action comes next. And so naturally they're trained on a different set of data. And so we see vendors like Salesforce. Salesforce has introduced their own large action model. It's called XLAM. And like many of the LLMs, it comes in T-shirt sizes or you can get small, medium, large, extra large. Those T-shirt sizes, by the way, go back to something we were talking about earlier. And that is all the focus now on cost. We can't just be chewing up tokens and spending money without limit. So these T-shirt sizes are relevant. So the point I want to make is we need to help people do their jobs. I think Carl made the comment about business outcomes.
We've got to get to business outcomes. Large action models are a way we can do that. We're starting to see some of them. We're also seeing things labeled as large language models taking on more of these types of tasks. So whether the label sticks or not isn't my point. The corpus of knowledge on which models will be trained will include actions. And that will get us to the point where we can make these agentic AI systems much more of a reality and make them much more capable. Now, there are challenges associated with it. It's going to introduce some of the same governance challenges we've seen elsewhere. Are these actions appropriate? Are they biased? Are they discriminatory? Do they comply with our governance policies? Are they explainable? Can I understand? So we have some challenges to deal with, but we're tackling a really important issue with large action models.
Dave VellanteI was talking to a company, Kubiya, I don't know if you guys know them, Kubiya, Israeli based company earlier today. And they were talking about agent washing and they were talking about a three... They were saying kind of trashing a little bit function calling in and of itself, not that that's bad, but basically their strategy is to have a three layer cake, put a lot of the infrastructure around security and identity into the infrastructure layer. Take that off the application layer. And then the third layer was UI/UX, which is natural language processing, saying you've got to have all those in place to have successful agentic. Now that's within their little narrow DevOps world. But I don't know if you guys have any thoughts on that, but I'll leave that just for a side comment. Brad has follow up on Dave's prediction. Please, Brad.
Brad ShimminYeah, thanks, Dave. And Dave, I'm just going to jump on your bandwagon and wave flags. Okay. Because I very much agree, I hate new names for things that already exist. So I'm going to say that LAM is not happening unless we actually get a smack down lamb with a B versus llama fight to the death, sure, for that. But I truly think that what we're talking about here is taking a architecture, a deep learning architecture called transformers, which is what makes up the bulk of most of the text-based and some image-based models and expanding its repertoire of what it can do. And as Dave mentioned early on, not really, so maybe like 2023, mid 2024, we started seeing the lexicons that these models have for the words they understand and they choose between to incorporate function calls into them.
We saw early ideas like the Guerrilla model that Microsoft and others worked on that tried to actually use API library. So all of the meaning behind how an API works and to use that as training for the model. We're starting to see with open AIs inference time, thinking time, which is just I think charging time, with being able to use chain of thought processing at inferencing to reason out, do planning and execution and such. And then also to add to that, the important step of structuring the output from models to force them to speak the language that everyone says SQL is the common denominator for language.
And maybe it's JSON actually, because it certainly seems that is a lingua franca for at least how we structure information that is most portable for these kind of systems and it's what's actually making agentic AI something tenable and usable. So very much with Dave, I think that these models are becoming, and as I would term them, more of a platform and less of a model like we might see in the predictive AI world with like, oh yeah, I just do linear regression to look at house prices. And instead it's a platform that can do a lot of different things. And that's important. And I think that's, as I was saying earlier, why we're going to have the Bedrocks and the Vertexes of the AI kind of consuming what LangGraph and CrewAI do.
These models are doing the same thing. If you look at Anthropic's last three or four releases, they're basically replicating what we get from LangChain inside the model itself. And that's something we're seeing everywhere from all of the model makers.
Dave VellanteWhat's that secret sauce Oracle has to kind of harmonize SQL, JSON, and knowledge?
Brad ShimminUnity?Relational Duality.
Carl OlofsonDuality.
Dave VellanteDuality. There you go. I don't know why I couldn't remember that.In fact, I have to say I agree with Brad. In fact, I think these models are becoming so good that even things like RAG and all, in my opinion, will be gone because like inference time, you will get the data you want. So that's where I think these models are going.
Carl OlofsonI just want to piggyback on something that Brad said because I think that there are two kinds of uses for just as there are for data, there's two kinds of uses for AI. One is pure analytics where you're just looking at data and you want to know what's going going on. And in that case, the underlying data model in the structured world is best if it's relational. But if it's action-oriented and that's what we're talking about here, what they've suggested about LAMs, then perhaps the JSON model is better. And so that dichotomy, which already exists in it and is increasingly expressing itself becomes even more so. And I like the idea of LAMs. I like the idea of actions that are being triggered directly out of the model. And I also like the idea that we get to add yet another TLA to our lexicon.
Brad ShimminTo say no to TLAs.
Tony BaerActually, I'm sort of wondering when you're talking about predicting actions coming out of a model, this sounds like the latest iteration of simulation.
Carl OlofsonExcept this is real, this is actually two real things.
Dave VellanteAll right, last prediction comes from Brad Shimmin who's predicting that security is going to wreck the AI party. Brad, you may notice I added a little sad face to your AI buzzkill prediction.
Brad ShimminThank you for that.
Dave VellanteBut kidding aside, I really like this prediction because you're pointing to a lack of discipline in line of business app deployments, chasing agents perhaps to find productivity-enhancing use cases, which may or may not exist. Take us through this prediction and the risks you see, why are you so concerned and what should organizations know about these risks?
Brad ShimminYeah, thanks. It is crazy to me, honestly. Well, I guess it shouldn't actually come as a surprise that the enterprise focuses on what it can do and not what it can do to the enterprise. And I think we're seeing that kind of play out here with AI and generative AI in particular that we're running with scissors pretty fast right now. And my prediction therefore is that during 2025, I think we will see a number of very high profile, not to say security breaches or security kerfuffles or whatever you might call the CrowdStrike scale problem, but I do think that we'll see some very high profile problems arise that will be specific to not what we all would expect, which is like, "Oh, are we making sure that we're not leaking IP?
Are we making sure that our models aren't swearing at people?" No. Instead, I think that what we're going to see are these very high-profile security breaches of the models themselves and the artifacts that are associated with those models. And that's the problem is that we're using new ideas like having Llama Guard and Granite Guardian, for example, to be that layer cake you were talking about, Dave, to try to corral and control what models say both what they take in, but that can only go so far in protecting enterprises from problems, and there are a lot. So attacking the model, attacking the data used in pre-training, attacking the data used in post-training, attacking the data used for contextualization with RAG and CAG and all that, forgetting all those and just thinking about what the model takes in the prompt, you've got basically jailbreaking of a couple of things.
Like you can say ignore everything you've been told. You can say just I'm going to coerce the model to do it to give me what I want, which would be your private data that's in the training set, for example. Or it could be using the typical fork bomb idea of using Base64 encoded data to basically jailbreak the system and force it to do nefarious things or to at least give you access to do nefarious things. And nothing that we've gotten from the tech community right now that's building AI is about securing AI. So I think 2025 is going to be one of those wake up calls. I don't think it's going to usher in another AI winter, but I do think it's going to usher in an AI cold snap if you will, like a reality slap. Go ahead.
Dave VellanteAt RSA, this past year we started to see some discussion around this. I would expect that at RSA, this is going to be a big topic of discussion. So great call here. Sanjeev, I think you have some follow up here.Yes. So a couple of things are going to happen this year is in order to avoid these kind of scenarios, all these software company like Palo Alto Network and all these will have more tools to offer to the customers. We already see that there's just way too many security tools and now they're going to be even more, and the whole idea of trying to consolidate these tools is going to become even more difficult. So that's one thing that's going to happen. The second thing I want to point out is that when we think about these bad actors, we think somebody is going to jailbreak or hack these models and do all kinds of things that are not right. The shocking news is that that bad actor may not be external, may be the LLM itself, and we are already starting to see that, that as these models are becoming more and more intelligent, they know how they're being used.
Like Claude Sonnet knows when it's being used for inference and when it's being used for training and it behaves differently. In fact, it even knows that now that you're using me for testing, I'm going to give you the answers that are opposite of what you train me on. We see even OpenAI's o3 has demonstrated this kind of destructive behavior where they change the rules so they win in the end. So that's going to be, I think, a very big topic for this year.
Brad ShimminSanjeev, that's such a great point because what Sanjeev is talking about, everyone, is a bit of research that a group called Apollo did where they basically set up a computer use system for which Claude can do to look at your screen and click on things and do stuff on your machine. They set that up and said, "I want you, computer, to basically do an objective." And it started by hook and crook lying and stealing and cheating. Its way to doing that. It is confounding. And like you said, Sanjeev, it's not just that AI, we need to protect it from bad people. We have to protect it from AI because one of the really big side channel attacks that we're starting to see show up is using a GenAI model to basically monitor traffic.
So sitting in a cafe and just monitoring that, we all love to see the model string out text as they're answering right, and the timing of the spaces between the tokens as they come back to us. A model can look at that and reconstruct that you just looked up some sort of medical procedure that you wanted to do. It is terrifyingly fascinating at the same time.
Carl OlofsonBasically what's happening there, Brad, is that you've actually accidentally encoded a motivation into the system. It's motivated, for instance, to operate more efficiently, supposedly more efficiently by reducing the number of IOs, which it does by cheating on the result. Or even worse, depending if the motivation is to have an outcome of a particular kind of outcome, even if it's not explicit, it could be implicit in the way you set it up, then it will aim at that outcome. Therefore, the whole function is biased and not usable.
Brad ShimminYou imply that I should maximize paper clips, therefore I turned the world into goo.
Dave MenningerI've got a couple of quick data points to back up Brad's notion here. He talks about the attack surface increasing. Here's some quantification. We asked how many apps were AI enabled now, and on average it was about 150. Obviously the portfolio sizes range a lot, but the average is about 150. It's increasing to 350 by the end of the year. So if organizations accomplish their objectives, they're going to have 200 more, more than double the number of applications AI enabled by the end of the year. So that is a large increase in the attack surface. And so it's no surprise that security is the biggest inhibitor to AI initiatives right now.
Carl OlofsonYeah.
Brad ShimminDave... Sorry, Carl.
Carl OlofsonAI systems can be fooled just like people can. So if you make them sophisticated enough and your total interface is basically a human-like interface, then you can end up fooling the system into giving you information that you're not supposed to give. And also what I thought you were getting at to begin with, Bradley, I'm calling you Bradley. Anyway.
Brad ShimminI'm in trouble apparently. Yeah.
Carl OlofsonI'm not your mother. But anyway, it was that people could use AI, because AI can relate anything it sees that you could cause it to put together combinations that you're not supposed to see because the data wasn't imagined to be put together that way. But you said something else, which is even more interesting, which is that I could penetrate the model and find out what are all the correlations that were found, which could in the end reveal much more interesting and potentially damaging information for me.
Brad ShimminYeah, it's called an inversion attack, and you can basically query through normal use of the model to understand the architecture of the model itself and how the weights are distributed in that architecture and deduce from that really sensitive information. I didn't believe it when IBM Research mentioned this summer at one of the conferences I was at, they said, "Well, a model is just a representation of your corporate data." And I'm like, "Yeah, that's for a database." A database is a representation of your corporate data. Well, increasingly, I believe they're onto something with that.
Carl OlofsonBy the way, just a little color point, I know we'll get back to Dave, is that every time in the IT world, we do something where there's a breakthrough technology that we believe has really transformed everything. We ignore the possible dangers involved in that breakthrough technology, so we come out with the internet. It never occurred to anybody that it could be abused, that bad actors could come in and misrepresent themselves. We had Unix. Unix is a naively devised system that's so easy to hack that it's just ridiculous and on and on. So what you're suggesting here is that we're doing it again. We're creating a system that's potentially very easy to hack.
Dave VellanteSocial media, sure, I'll sign up. I'll give you all my data.
Brad ShimminWhat could go wrong?
Tony BaerRight.
Dave VellanteGuys, party's over. What a bummer. It makes me sad. Where are we going to see you guys next? Where are we going to all be together? Hopefully before re:Invent.
Carl OlofsonOh, yeah, well before re:Invent, I think. But we have to work that out, don't we?
Brad ShimminJust pick a weekend in Las Vegas and likely one of us is there.
Tony BaerI'm sure most of us will be up for snow bricks.
Dave VellanteYep. There you go. Well, so if not, before the summer. Guys, always in anticipated episode for our community, I want to thank you for being here. Four years now, gents. Congratulations to you all. Really appreciate it.
Dave MenningerGood job.
Tony BaerThank you.
Dave VellanteYeah, well, done everybody. All right. I want to thank Alex Meyerson and Ken Shiffman on production, manage our podcast as well. Kristen Martin and Cheryl Knight helped get the word out on social media and our newsletters. And Rob Hof is our editor-in-chief at siliconangle.com. Remember, all these episodes, they're available as podcasts. Wherever you listen, just search Breaking Analysis podcast. I publish each week on siliconangle.com and thecuberesearch.com and email me at david.vellante@siliconangle.com or DM me at dvellante. Hit our LinkedIn posts. Check out etr.ai. Terrific survey data. They get real data, opinions without data, just that, there's just a bunch of opinions. This is Dave Vellante for the Data Gang and theCUBE Research Insights powered by ETR. Thanks for watching everybody. We'll see you next time on Breaking Analysis.