- info
- Transcript

Savannah PetersonGood afternoon, AI fans, and welcome back to beautiful Atlanta, Georgia. We are here midway through day two of our three days of coverage on theCUBE. My name's Savannah Peterson here with the ineffable and brilliant Dave Vellante. Dave, I'm super excited.
Dave VellanteThat's too kind.
Savannah PetersonHey, I'm too honest. You know me by now. You know what an honest person I am, Dave, perhaps to a fault. And so you know that we're honestly excited about our next guest, no cowboy hat for me this time, but David, thank you so much for coming to hang out with us and talk PowerEdge.
David SchmidtYeah, my pleasure. Thanks for having me back. It's good to see everybody again, so exciting.
Savannah PetersonYeah, so you're a regular, lots of new announcements coming out, lots of advancements, not even just announcements. What's new on your side? PowerEdge has been a conversation on the table, but...
David SchmidtYeah, absolutely. I think this is my third Supercompute. I think a couple years ago, we were talking about a product that, the XE9680, went on to become the most successful product in PowerEdge history. So we were excited and then we followed that up last year and now we're here talking about some great stuff. I think what has been interesting for me as we've talked to customers in the past few years is that they're seeking AI solutions, but they're seeking AI solutions of all shapes and sizes, so they really need things that fit their existing data centers. There are certainly use cases where customers are looking to build new data centers, what I would call greenfield, but we have to meet customers where they are today and provide them with powerful AI solutions. Now, you can do that in some of the compute-based solutions that we've been building with PowerEdge. We unveiled a few things a few weeks ago at OCP Summit. We are talking about some other solutions right now, and really, what we're talking about here is the breadth of solutions where we can deliver AI on compute as well as purpose-built, large-scale AI solutions, and we're doing that with systems built on fifth-generation AMD, Turin. And so we're really excited to have new servers, new rack-based servers that are supporting latest-generation Turin processors.
Savannah PetersonWhat I'm hearing from you in that, we hear a lot of hype about the big, huge systems, that's not always the solution for everyone. It seems like, and you've had some intel, you've talked to a bunch of different customers, there's a variety of solutions that the market needs. Tell me a little more about that.
David SchmidtYou know, one of the simplest ways to break it down is, I mean, we're walking around the conference and there's a lot of liquid cooling or alternative cooling stations-
Savannah PetersonLike you said, it looks like a plumbing show.
David SchmidtA little bit like a plumbing show. And that's great, but at the same time, we have customers that aren't ready for that in their data centers and we talk to those customers. And so we are really focused on providing compelling air-cooled designs that they can fit inside their existing power and thermal footprint inside their racks, inside their data centers. And then it's a matter of crafting the right AI solution and use cases and the right sizing that's going to work for that type of environment. And so I'll give you an example of maybe some of the design points that we had to focus on with our latest rack servers, our PowerEdge rack servers. We knew that providing the highest core count possible in an air-cooled package would be really, really important because you need that high core count if you're going to run, say, an inferencing model, like a small language model, on that platform. And so we're supporting 500-watt processors in an air-cooled package, very high thermal inlet air temperature, and we're doing that because we've designed the system. We've redesigned the system from our previous generation and we've added that type of thermal improvement, so we have the capacity to support that 500-watt processor, 192 core.
Savannah PetersonWow.
Dave VellanteSo how do you do that? Is that software IP that optimizes the cooling? You don't just throw big fans at it, right?
Savannah PetersonYeah, great question.
David SchmidtIt's a yes and a yes, right? It's really intelligent design, heat sink designs, just focusing on fan performance and making sure we're using high-quality fins. And then there is a software element because you have to have the right embedded baseboard management, and we get that with iDRAC. We have world-class management with our iDRAC controller, and you have software there that helps direct airflow in an intelligent way through the system.
Dave VellanteYeah, <inaudible> inside there.
David SchmidtAnd a lot of times, you'll look at the specs on a system and you'll say, "Well, it can support this," but there's a bunch of fine print. It's like, well, maybe you can support a high TDP, but it needs to be at 25 C or maybe 17 C, which is what it feels like in here right now, it's pretty cold in here, but you got to have a really cold inlet temperature to support that. We're looking more like 30 C and 35 C. I always have the opinion that we've got the best thermal engineers in the business. And so when we're publishing our thermal testing results, we're really excited to have that to see that we can provide robust configurations that use that intelligence that we're talking about to provide the highest configuration in an air-cooled package without requiring liquid.
Dave VellanteSo what's the spectrum of AI systems look like? You know?
Savannah PetersonYeah.
Dave VellanteIt used to be big, medium, small. Boom. That was the old server days, and it's much more granular now. So what's it look like? Paint a picture for us.
David SchmidtIt's from, simply put, I would say it's from eight cores to 27,000 cores, maybe.
Savannah PetersonCasual, casual differential there.
David SchmidtJust a casual, casual difference. Yeah.
Savannah PetersonYeah, it's quite the spread. We'll call it a spectrum.
David SchmidtAnd I'll explain why. If you go to our booth and you look at the systems, we have our large-scale, purpose-built AI servers, and we have really compelling compute inside those. We do have Turin built into those platforms, like the XE7745 or the 9685L, and we're obviously supporting large GPU deployments there. At the same time, you walk left to right down that table, I'm putting in a plug for the booth, by the way, if you walk left to right on that table and you'll see our rack servers, well, you can support anywhere from eight to 192 cores in rack-based server. If you want to run, say, video analysis or video inferencing on the CPU, you can do that. And we have some customers that are already looking at how they do video analysis for a smart city deployment. So that's a really compelling use.
Savannah PetersonOh, cool use case.
David SchmidtOh, yeah. And you're running that directly on the CPU. You don't need any GPU. Now, if you want to graduate to, say, running RAG-type solutions where you're augmenting your results, you can do that on high-frequency CPUs. You can do it on a 5 gigahertz CPU. You don't need to go deploy GPUs to do that. So we have support for that in our rack servers. So you do have a largest-to-smallest type of config. And then if you want to standardize on that rack server, you can do eight core count at the lowest end, and that's what we really partnered well with AMD. When we looked at all the core counts, we wanted to provide, the type of systems we wanted to provide, we wanted to give enterprise IT customers that kind of large-scale, highly adaptable system that can support a variety of use cases. And so that's what you get in the R7725.
Savannah PetersonYou know, David, you just brought up something that I think is really important. I think there is so much emphasis on GPUs. I mean, I've even got a run GPU sticker on my laptop, but it's going to take a variety of different types of compute to solve the problems that we are trying to solve.
David SchmidtExactly, yeah.
Savannah PetersonAnd I think that that's one of the unique things, I bet Dave would agree, I'll turn it to you here for a second, is that Dell is uniquely poised to support a variety of different companies. I mean, beyond your booth, which you just so lovingly plugged, you've also got your AI Factory behind us. I got to go hang out in there, see a whole bunch of different demos. I got to talk to Andy, your virtual AI friend, and actually, I need to put up the video for that, but I think that this is just a pulse-check question. Do you think the conversation around GPUs is over-hyped or overshadowing alternative compute solutions?
David SchmidtIt's nothing like that, in my opinion. It's really helping spur the conversation. It's forcing the hard-
Savannah PetersonOoh, I love that. Yes.
David SchmidtIt's forcing the hard conversations.
Savannah PetersonYeah, yeah, yeah.
David SchmidtAnd it's forcing enterprises large and small, it's forcing the customers that we see here today to really think about what they want their next generation of data center to look like. We have some really compelling solutions. That rack that's in our booth, the IR, integrated rack scale solutions, the IR7000 rack, that is us helping design data centers of the future that can accommodate both AI workloads as well as traditional core compute workloads. So I love it. It's good tension and it's the kind of tension we need. I think we all have been to this conference for many, many years now, and we've seen it grow and I think that's evidence of the tension that's being put on these different design elements.
Dave VellanteWell, and it strikes me, David, are customers asking you or are they telling you? And I would imagine some of the customers are telling you, "This is what we need," big LLM guys. A lot of the enterprises, they're probably a little nervous right now, "Wow, we have to refactor our data centers? We have to... Liquid cooling, air cooling? What do we do?" They must be asking you as well.
David SchmidtAbsolutely. And they're saying, the question is more like, "Hey, how do I deploy and demystify AI solutions for my environment and please don't try to sell me a new data center. I've got my traditional footprint. I've got an investment that I need to continue utilizing," and so that's where... The AI factor is no surprise. I love it. I think we do really, really great things. We have reference architecture, we have validated designs that run the entire spectrum. And so if we have, say, like a Llama 3.2, it's considered a small language model, I'm just giving you an example, a small language model, it's 3 billion parameters. We can support like 100 concurrent users on just a standard two-socket rack server running a 128-core Turins. And that's the kind of footprint because that's the exact same type of system a customer would deploy to run just their standard IT infrastructure as well. So it gives them a common install base, and they love that.
Savannah PetersonYeah, that user experience has got to be so much more pleasant.
David SchmidtExactly.
Dave VellanteThis is a really important point because the CIO mindset is, "I'm at Point A, I want to get to Point B, I don't want to break the bank, I don't want to rip everything out of there."
David Schmidt"How do I start?"
Dave VellanteAnd I think it's actually the workloads are, from an infrastructure standpoint, what I've just inferred from what you said, David, it's a Venn. It's not two separate silos.
David SchmidtRight. It is not mutually exclusive.
Dave VellanteIt's, "Hey, there's some overlap from an infrastructure standpoint that we can support the traditional general purpose workloads and the AI workloads."
David SchmidtThat's right.
Dave VellanteNow, the hardcore stuff, we got you covered there too, but it's a gradual, maybe not gradual, but it doesn't break the bank necessarily going from Point A to Point B, and you don't have to hire a million consultants.
David SchmidtThat's right. The best validation of that is the questions we've been getting asked this week. When they stop and they look at our platforms on the desk and they look at the 2U rack server, they look at the 1U rack server and their question is, "How many GPUs can I support in this system?" So say they do want to build something that's more GPU-based, it's, "What can I support inside this system?" And I do think that is validation of what we've been thinking, that they want to standardize on a platform, they want to use CPU where they can, they'll use GPU where it makes sense, and they'll build it into that platform. And that's how they'll get started, to your point.
Savannah PetersonI think that's a great conversation. It's about that plug and play and that usability, but also being able to adapt and evolve depending on the workloads or whatever it is that they're-
David SchmidtThat's right. And grow.
Savannah PetersonYeah. Oh, yeah, and whatever they're trying to create. You mentioned smart cities and video a second ago, which is an awesome example. Can you give a couple other customer examples of how you're seeing your system's deployed?
David SchmidtSure. We have some customers that are actually building large... They're actually doing some large language model development on top of just standard rack servers. So they're doing that. I talked earlier about, say, and we're actually, we have this video in the booth as well, we have the RAG solutions where they're using a high core count CPU to perform just retrieval augmentation on just standard results from a large language model. And we're able to just package that up as a... I think the demo that we did is analyzing a legislative bill, which sounds kind of... But probably a very appropriate thing right now. But it's a good demo because it shows what you can do just on a standard rack server and so that has resonated. That is built out of a real-world example and we're able to just document that as a white paper. We've got that coming soon and customers can just pick that up and start using it.
Savannah PetersonYou don't hear that phrase a lot, "Just pick it up and start using it" when we're having a conversation about high-performance computing.
David SchmidtI know. That's the goal. That's the goal.
Savannah PetersonIt's that usability. Whew, it gets me excited. You mentioned that your thermal engineers are some of the best in the world. What are some of the challenges or processes they have to continually adapt to the new systems that are coming out? What's that iteration cycle like?
David SchmidtThe iteration cycle is all about the richness of configurations. And so earlier, I talked about you might see that you can support a really high core count, high TDP processor, but it comes with a lot of caveats. Well, another one of those caveats is you can't put any drives up front or maybe you can't use a lot of memory because those are, especially drives, that can impede airflow through the system. And so I would say that the twist and turn has been about the richness of the config and it's not enough to just provide a really compelling compute solution. It's got to involve the storage solution, it's got to involve I/O out of the back. We've got to provide, in some cases where we're helping customers design like large-scale object storage frameworks, you've got to have like 400 gigs of I/O coming out of the back. You have to be able to support that and your thermal solution as well. And so we have these great matrices when we finish our testing and it's all, right now, it's all greens, it's looking really good. Sometimes it comes up yellow and red, saying, "You can't have that config with that particular processor," and we're doing a great job testing and putting the right requirements in place so that we can give customers a really robust solution with the thermals that they have in their environment today.
Dave VellanteDavid, what do you think happens with server life cycles? I mean, we talk about these things. You and I were talking. I got an old phone, you have an old phone, probably need a new phone. I have, I think, four PCs. I got three Dells and a Mac, I'll say at-
Savannah PetersonI have three as well. Yeah, yeah, yeah.
Dave VellanteAnd you've seen the hyperscalers, they depreciate their asset now over, I think it's six years now, so they get a longer useful life. I presume enterprises are as well. Will AI compress that, in your view, or do you think it's pretty much the status quo in terms of that life cycle, that useful life cycle? There's a power angle too here.
David SchmidtYeah. This is a great conference to have that conversation because what do HPC customers seek? They're always seeking top-end performance and when there's something next-generation that comes along, they want to take advantage of it. And of course, AI being a close cousin to HPC, you see the same thing happening.
Dave VellanteSo a good proxy.
David SchmidtRight. And so now, you've got this AI use case or AI environment that looks and feels like HPC where you're chasing latest, greatest. So to answer your question, I think there's a part of the data center that will be chasing that latest and greatest. As enterprises adopt more AI, I don't know whether we can press those cycles or they just transform in a different way, and then there's going to be more stable parts of IT environments that maybe don't have or have similar life cycles to what we have today. But I do think that the similarities we see here are going to trickle down into enterprise environments and there'll be faster turnarounds because it has tangible benefits. If the models get better, if the results get better, if the benefit to the business gets better, it's going to be very tangible and very easy to prove when it comes time to make an additional investment.
Dave VellanteIt's not like there's a, correct me if I'm wrong, it's not like a big aftermarket for, but maybe there is, for servers. You guys pretty much take them off the market, but maybe not. Maybe it's like the old mainframe days where they cycle through. What's that dynamic?
David SchmidtYou know, it goes back to the have a stable platform that is adaptable to other parts of the environment. Maybe today's AI systems are tomorrow's... Well, using the two-socket rack example, maybe that system today is running inside a data center or maybe it's running in a carpeted edge location later on because there's plenty of remote, what we used to call remote office environments before we called it edge, but there's going to be a need for bringing those use cases out into those carpeted edge environments as well. So it's a potential example we use.
Dave VellanteMaybe today's training system becomes tomorrow's inference system at the edge or something like that.
David SchmidtQuite possibly.
Savannah PetersonYeah.
David SchmidtBut if you have a standard platform, if you're building around a known compute platform, in my opinion, PowerEdge is the best known in the world, and you have a really stable lifecycle for that server, then you can embrace all sorts of different models.
Savannah PetersonAll sorts of different models, different solutions, a whole bunch of stuff. I got two more questions for you.
David SchmidtAll right.
Savannah PetersonYou get to talk to some of the coolest companies in the world doing cutting-edge stuff. Taking off your PowerEdge hat just for a second, what excites you most about our AI future personally?
Dave VellanteI will show <inaudible>.
David SchmidtMy PowerEdge hat is sometimes glued to my head.
Savannah PetersonYeah, it's all right.
David SchmidtWhat excites me most about just AI use cases or...
Savannah PetersonYeah, sure.
David SchmidtI think anything that makes our life simpler, we instantly embrace it and we self-select, and we've all figured out that smartphones make our life easier. We learn how to give the right level of feedback of where it's maybe difficult for us. And now we have different ways to manage screen time, for example. But we embrace smartphones because it gave us a tangible benefit. It made our life easier. I think the things that will make it easier for us to perform our jobs, to live our lives are going to be the most compelling and exciting things, and I can't wait. I think it's already having tangible benefits to me right now. I think it's only going to get better and better. Like I said, it has tangible benefits. As the performance improves, as the hardware infrastructure improves, and the technology improves overall, I think it's going to be very, very exciting. And I think we are making some really good cases. I think somebody said it the other day, he's a very well-known individual, his name's on the building, but he said, "Hey, this is the worst it's going to be and it's only getting better from here."
Dave VellanteIt's a great quote.
Savannah PetersonYeah.
David SchmidtAnd it's just going to explode. And I think that's what excites me the most is it's only getting better. It's only going to make things easier for us over time.
Savannah PetersonWhat a moment of inspiration. And I do love that quote. Building on that, I may have asked you this back in Dallas, but I'm not 100% sure that I did, when we're hanging out at the next Supercomputing, this is our annual reunion, what do you hope to be able to say then that you can't yet say today?
David SchmidtI think the outcomes we're going to deliver with our rack-scale solutions with our next-generation systems, we will be talking about some outcomes that will just be mind-blowing. And by outcomes, I mean we are going to enable customers to be successful in a variety of industries, a variety of sizes, like I've been saying, and we are going to be talking about some things that it feels like it's only a year away. It is only a year away, but I don't think it's a stretch to say that it's going to be very surprising. And I'm alluding to things perhaps, but I think we have some really compelling conversations going on right now. We have the capability to deliver at scale in a way that is just going to accelerate customers time to value. And it's just going to be awesome to sit here a year from now and have those types of conversations.
Dave VellanteSo I'm hearing supply chain, I'm hearing ecosystem, I'm hearing innovation and hardware. I'm hearing customer outcomes, adoption. It sounds like a package of all that stuff.
David SchmidtWe have strengths in all those areas and we're putting that all together in such a great way that I can't wait to see it. It's going to be awesome.
Savannah PetersonI can't wait to see it either.
David SchmidtAll right.
Savannah PetersonI look forward to having our minds blown when you're able to reveal some of the awesomeness that happens over the next 12 months. And frankly, I'm excited we got to have another mind-blowing and educational conversation together, David. Thank you so much for taking the time.
David SchmidtThanks for having me. All right.
Savannah PetersonAnd thank you, Dave. I'm in a David sandwich right now. I feel very lucky. I hope your mind's just as blown as ours here during our three days of coverage on theCUBE. This is the middle of day two of Supercomputing. My name's Savannah Peterson. You're watching theCUBE, the leading source for enterprise tech news.