For as long as humans have used computers, we have sought to make them faster, smarter, and more capable. So what new potential can harnessing the power of AI in PCs promise? What if every computer could use AI technology to unleash new capabilities that will benefit anyone and everyone who uses a computer. Plus, imagine data security that AI PCs can offer that Cloud AI cannot. Intel Vice President Robert Hallock helps us explore how computers equipped with artificial intelligence–like the new Intel Core Ultra Processor–are transforming productivity and IT operations.
Learn more about how Intel is leading the charge in the AI Revolution at intel.com/AIeverywhere
For as long as humans have used computers, we have sought to make them faster, smarter, and more capable. But what happens when we harness the power of artificial intelligence with new PC hardware and software? And what if every computer could use AI technology to unleash new capabilities that will benefit anyone and everyone who uses a computer.
What can I help you with?
This is not a distant dream. Each day it becomes more and more of a reality. AI is no longer exclusively running in the cloud. It's increasingly finding its way into all computers, enabling business users to do more and be more with PC technology from Intel. In this episode, we'll be focusing on defining the AIPC and what represents for the future of this transformative technology. We'll also explore what the growth of AI running directly on the PC versus the cloud means and how this AIPC revolution changes how we work, live and create. Join us as we take a journey into the future of AI computing, where machines are not just tools but partners in our endeavors. Welcome to Technically Speaking, an Intel podcast produced by iHeartMedia's Ruby Studio in partnership with Intel. Hey the I'm gram class. Joining us today is Robert Hallock, the VP and General manager of Client AI and Technical Marketing at Intel. Here has a long history with product development and PCs. Welcome to the show, Robert.
Thanks for having me.
Good to be here. Yeah, I just want to the start off by saying, there seems to be a lot of talk around AI, and it seems to always revolve around cloud based or software as a service type business models. How Intel is taking a different route. Well, but can you give us an overview of what an AIPC means at Intel and can you help us understand some of the benefits of running AI directly on a PC versus in the cloud.
Yeah, a lot of people have been exposed to AI through the cloud, and these are services like chat, GPT or Dolly three where you can create a piece of content from a text description. And that's one kind of AI that's called generative AI is a category pros and cons to that doing it in the cloud. The pro is the models are huge, right. They essentially have the Internet's collective knowledge of data to produce an answer or a picture, and that's highly detailed. But one of the cons is that they're not very specific to you what you are doing on your PC, the information on your computer, or the images that you care about. So that's sort of motivating to bring AI capabilities locally to the PC where you don't need an Internet connection to use these capabilities. And that's sort of the tip of the iceberg, because when you move AI to the PC without a cloud connection, you can also do new types of AI workloads that would be too big or simply can't run in the cloud. So a good example of that is in teleconferencing, lots of people use background blurring or background pictures, and we can actually make that a lot more energy efficient. We can actually give you hours of battery life back by using AI on that use case. So the whole point of AI is that there's a wave of software coming that uses AI to improve performance or to reduce power consumption. So you'll get better performance from your system and longer battery life as well. And this will continue to grow over the next five years.
And Robert I was wondering if you could please describe to the audience what the current state of the art is with PC architecture. You know, we've got CPUs, which are central processing units that make computing possible, and you've got GPUs, which are graphical processing units that are instrumental for machine learning, gaming applications, video editing. So how does that actually differ from the concept of an AIPC.
That's a great question. So, yeah, users may have heard about an AIPC at this point, and at the basic level, this is just a new generation computer with hardware inside that is capable of accelerating an AI based workload and previous hardware, let's say early twenty twenty three computers, you can run AI workloads, but they will fall back to the CPU cores, which are not as fast and not as energy efficient as running that same workload on the new accelerators in an AIPC. And the way we're doing it at Intel is actually the CPU cores themselves. We've added AI accelerating capabilities to those cores. The GPU that's built into our processor also has accelerators, and then we've also added an entirely new component called the NPU or neural processing unit, which sits next to the CPU and the GPU and is now its own third category of acceleration on the device. And different workloads in AI, or different features in AI run best on one of those three engines, so you kind of need all three to do this well. And our new product, Intel Core Ultra is our first AIPC processor, and so if you see that name Core Ultra, you know that it has the AI accelerators to run these features well. But at its route, it is a PC that has AI specific hardware.
And I'm quite interested in I guess the thinking behind at Intel of this shift towards this new architecture. Was it something that was internally driven or did you see some conversations and discussions with your key customers and clients driving that sort of shift.
This is very much a software industry driven transition, and CPU vendors like Intel were innovating to keep pace with what's going on in the software environment. And that's kind of a key thing I'd want to stress. You know, maybe you're on the fence about AIPC or you wonder, you know, how long is this thing going to stick around. This is one of those cases where it's both the hardware industry and the software industry agreeing that this is the right thing to do for performance and features and power. Because I truly believe that over the next three to five years we will reach this point of general acceptance in AI, where it's whether you know it or not, widely diffused. In most of the applications you're working on or working with, many of the features that you value will use AI again transparently in the background. But this is very much a collaborative, industry wide effort from all the hardware makers, all the big software vendors. This is here to stay for sure. Let's pause for second here to reiterate Robert's point. Right of adoption is always a crucial aspect of any new technology, and that's certainly true of what we're discussing today with AIPCS. It's important to understand that the leaders in the field of computing, both in the hardware and software domain, have already begun down this revolutionary path and it doesn't seem like there's any roadblocks in site. With that in mind, I asked Robert about the benefits of moving AI workloads from the cloud to the PC, especially when it comes to issues of data privacy, latency, and connectivity. You just touched couple that are really important to local AI. It's that data privacy or data security. We've all read about information going up to a cloud resource of some kind, and you don't really know what's going to happen with your data or your request after that. And that's not to say there's anything malicious implied. You just don't know. So I'm sure people in corporation, certainly at Intel, when you go to a generative AI website, it says, hey, be careful what you enter into this textbox. And so moving this stuff offline gives you a couple of things. Yes, you get chain of custody over the data. It's private, right, it's working on your information offline, right, so it doesn't have to go to the cloud. That's a big one. Cost is another component. A lot of the most powerful AI services online, you know, ten to twenty bucks a month, and it's not cheap to have several of those. The last that is interesting is cloud servers are intrinsically pricey. I'm not saying they're expensive, but for AI as a genre of software to truly take off and thrive in all the way the software vendors want it to, it has to reach the local PC. It has to reach you know, tens of millions of users, hundreds of millions of users on a local device and that's sort of that critical mass install base that takes this effort to the next level. Cloud was sort of one point zero of AI, and now we're trying to, you know, for one of a better term, create the two point zero where lots of people have access to this and it's widely available and in a couple of years Intel alone we want to get one hundred million accelerators for AI into the hands of people.
And do you have any examples of at this early stage of actually pushing the boundaries of using this sort of power in a local PC to do some really interesting work.
One for an enterprise that we've been tinkering with is a technology called rag rag, and it's the idea where you have a language model running offline on the user's PC. But this RAG component can scan your documents, your corporate information, and then specialize the LM to be for you, your work, your knowledge. So just as an example, we scanned one state's DMV manual, which hundreds of pages long for the Department of Motor Vehicle manual and now you can ask very specific procedural and legal questions about the subject of that manual and it'll spit back highly accurate answers for you. And if you extend out out words, protecting and promoting institutional knowledge is hard. You might have that employ that's been there for twenty years and has all that institutional knowledge in their head and if they leave, it goes with them. But a RAG model could synthesize that knowledge for people, so a new employee could just ask a question in a text box and get an accurate answer about what that company's working on or what this feature does. And that's just huge for business.
Yeah, you know, I'm from a small business sort of background. My dad has a small business. And the fact that you can bring this power to the small and micro businesses as well without having to pay these cloud based prices and also in conjunction with using some of the open source type software that I know Intel is very supportive of. I'm just really excited to see the little guys, you know, be able to compete with some of the technology that the big boys have.
Absolutely, AI is, at the end of the day, a force multiplier for a person. Right like at the root, AI is designed to save time writing meeting minutes or email summaries or drafting in outline. These are all just like time consuming tasks for people, and they don't require skill per se, but it's it's time consuming, and so being able to offload that to a digital assistant that can just sort of ninety percent or ninety five percent do that for you allows you to refocus your efforts back to something else that is more productive and more worthy of your time. And especially for a small business, administrative overhead, bureaucratic overhead is hard, it's time consuming. I myself own a single member LLC and administrative stuff takes up a ton of my time. Like I'd love to outsource that to AI.
That's right. In terms of Intel's history with past, you know, technological revolutions, I'm reminded of Intel's initiative to get Wi Fi into every your laptop and that was code named Centrino, and it's interesting to hear that again. Intel are trying to push new technology so that it's ubiquitous, and that's exactly what they're doing with these aipcs. And in ten twenty years time, I think that aipowered PCs will be so ubiquitous that we won't think anything of it. I'd like Robert your thoughts on that and more insights into the way Intel is evolving their strategy and Intel's role in this. Yeah.
Actually, Centrino is a really nice analogy because that's sort of what we're trying to do with these AI accelerators. It's not a huge tweak to the configuration of a system design because most of the work happens inside the CPU, and there are very few external requirements that would change a system designed to make this possible. Right, So it's a system vendor could theoretically update last year's chassis to have a new CPU and that would confer the benefits from Core Ultra and an AI workloads and Centrino not much different, right. You're adding a Wi Fi chip and an antenna to the system.
Yes, But for those.
Of you who weren't around during the Centrino days, it used to be very common that a laptop would not have Wi Fi, which seems sort of unimaginable now, but that's because Intel made this massive effort to make it common and we all sort of take it for granted now. Another analogous moment for me is the addition of graphics in the processor. I was a hardware reviewer when that started happening, and I remember the conversations back then like why are we doing this? You can't even play a game on it, what's the point? This is just going to make CPUs more expensive, YadA, YadA, YadA. Today web pages are rendered by your graphics card orphics accelerator in the processor, your browser uses it. It's everywhere. So both of those are very foundational examples for me, because I see AI as being quite analogous in both respects. But I think history will bear out that this was a pivotal moment and AI will be very, very widespread, just like graphics and just like Wi Fi.
Coming up next on Technically Speaking and Intel podcast.
Being familiar with how prompts work, it's going to be a key business skill, and I think there'll be a real advantage for employees who know how to engineer a good prompt to get a great result quickly.
We'll be right back after a brief message from our partners that Intel welcome back to Technically Speaking. I'm here now with Robert Hallett getting back to I guess more on the business side of things. Maybe if you can talk a little bit about our friends in the IT department having to manage the security and all these sort of things. Yeah, what's some of the benefits they could look forward to in terms of managing these types of aipcs in their enterprise.
I think it'll make every ITDM happy that at the end of the day, these AI applications are nothing more than endpoint applications. You download them, you install them, and they're from software vendors, and I'm sure they'll be breakout ISVs that come under the scene as a result of this AI transformation, but largely speaking, trusted vendors doing new work. And because it's entirely offline, your user has custody of the data and the information, which is no different from any other application today. So it's not like this transformation comes part and parcel with like a radical transformation and endpoint management, which would make it way harder. From a security point of view, AI has some very interesting benefits for security models. So now this is the ITDM point of view. We recently with Dell and CrowdStrike, and if people don't know what CrowdStrike is, it's, amongst many things, an endpoint security solution which is specifically designed to help prevent threats that don't attack files. Many of the tax that go into a system now aren't like a virus that infects a file, it's actually resident in memory. They're fileless attacks, and these are way harder to detect and prevent. So CrowdStrike uses convolutional neural networks, which is a simpler form of neural network or AI, to monitor the real time conditions of the system and see if something unusual is happening. And this experiment moves these convolutional neural network models or CNNs to that neural processing unit or the NPU in coraltrum. It did a couple things when we did that. First, it gave the processor cores twenty percent performance back. The second thing it did it made the security model slightly smaller, so it had a smaller memory footprint now so user gets some RAM back. And it also improved the accuracy of the model because they can make the model computationally bigger, right because it has its own dedicated accelerator now to run on. And so security got better and crowdstreg is a very popular solution, and there are other solutions in the pipe for phishing detection, which is notoriously hard pattern matching problem. And that's just two examples of the way security can be enhanced by offloading to an AI specific accelerator.
Yeah, and we talked a little bit about you know, businesses and enterprises adopting these new aipcs. Is anything special that IT departments and organizations need to do to prepare for this.
I'll say there's probably three things that an ITDM would want to think about if they intend to use large language models or just generative AI. Those workloads are pretty sensitive to memory bandwidth, so you wouldn't normally think about memory bandwidth in a system purchase, but making sure that it has two memory sticks over one. For example, right, if you want to get sixteen gigs a RAM, make sure that's two by eight instead of one by sixteen, Just as an example, that will dramatically improve the performance of the LLM, because these language models are fundamentally limited or enhanced by the performance of the memory subsystem. Outside of that, for other forms of AI in the year ahead, there is a calculation called TOPS or terra operations per second, which is sort of a ballpark for how much AI performance a device can give you. Software makes or breaks the acquisition of that TOPS figure. So I can give you a billion TOPS, and if I had a very poor software stack under that you would never see the billion tops. So there's a new level of knowledge the ITDM needs to develop on sort of software stack robustness underneath that rating. Okay, so you have to be familiar with what Intel's doing in the software space, what its competitors are doing in the software space to really understand whether or not you're going to get a good experience out of the device you're purchasing. So that's number two, and number three I think would be to say that itdms will probably be faced with a lot of advocacy for the NPU as an AIX, but it's important to understand that the software industry broadly also wants to use GPU and CPU as well. So if you make an upgrade decision that is all in on NPU performance and you didn't check on the GPU or the CPU, you may be out in the cold on performance or power efficiency for these Frankly a large number of workloads that use graphics and CPU for AI acceleration. Those are the three things that I would say are new or different in this era, But overall it is something that you can integrate into your upgrade cycle piecemeal. YEP, Newer devices will be faster. I mean, they always are, but it's not like you're missing out on a new feature, right, We're going to make them faster, But the features are delivered by the software, are not the hardware. You can kind of get in at any point and get the goodness of AI, which is pretty cool as well.
Yeah, yeah, And are you helping the software vendors, I guess compile their code so that it will help them really utilize that hardware underneath.
Yeah, that's the secret of AI. And I'll start with an analogy of PC gaming, which I think is a lot more familiar to people. So at the bottom, you have a piece of hardware, a graphics card, and they tell you, you know, it's certain terra flops or gigaflops of performance, but everybody knows that really depends on how well optimized the game engine is, how well optimized the game you're running is, how good the graphics drivers are. AI is no different. AI accelerators are actually exposed in direct X in Windows as a GPU without display outputs, so the system sees them as essentially graphics cards, and instead of game engines you have AI models will have features and apps just like games you even have run times or an environment where the code is running in there's a DirectX run time for graphics, there is equivalent run times for AI. So in many respects, the AI software stack looks and works a lot like the gaming software stack. And so if people think about all the times that a GPU that's supposed to be faster on paper didn't live up to that number because of one software reason or another, that is a possible reality for AI as well. And that's why it's so important to make your decision not just on tops or what the accelerator is, but on the robustness of the software underneath, because that's where it really happens.
Regular listeners to the show Mike remember back in season one, we divided a whole episode on the skills that workers will need in order to take advantage of the kinds of AI tools we discussed today. AI technology can only expand our expectations of what's possible if we understand the most effective ways to use it. So I asked Robert for his thoughts and what workers should focus on to take advantage of this new wave of technology, and he began his answer with something just about everyone does on the Internet, every day.
Ooh, that's a good one. I actually want to start briefly at web searching. You know, web searching, a good, well composed query is a learned skill. We have all encountered people that haven't quite learned that skill, and they get bad search results and they're frustrated with a search engine. And not a lot of people teach that skill because it's its own language of sorts. You know, you want to freeze things to a search box differently than how I would ask it out loud. So that takes me to AI, where you are still engineering a search of sorts. It's called a prompt now, but it's a search and the skill is in like, how do you craft that prompt to get the result that you're looking for? You have to be somewhat specific, and you have to understand the limitations of the AI model you're working with. So let's take an image creation model. Some of them only support what's called positive prompts. You have to describe the things that you want and it will kind of omit anything else that you're not asking for explicitly. Others support negative prompts, and if you're working with a positive only image service, you could easily have users saying I want this, this and this, but not that, that and that. Yeah, the not operator the AI engine's not going to understand it, so it's going to give you the things in the not category. In the picture.
Yeah, it's like saying I want a picture of a zoo, but no elephants.
Right, and it doesn't nderstand the word no, So in.
A positive one, it'll just have elephants.
Yeah, you get all elephants exactly right. So understanding this is a real skill. Understanding like what the AI model or by distant extension, what the game engine will let you do is really important, and how to be specific and understanding how to provide follow up commentary to tweak the result to get what you're looking for.
This is a.
Whole category of skill that is not taught, not widely known in business or even amongst users because this is so new. But being familiar with how prompts work is going to be a key business skill, and I think there'll be a real advantage for employees who know how to engineer a good prompt to get a great result quickly.
So going to take a bit of a step back and look far in the horizon in the sort of the three to five seven year time range. What's Intel's plan for this journey into AI.
I think at top level, i'd want to acknowledge that I understand the skepticism or the I don't understand of AI. But where we're going is like a complete industry transformation. Intel believes by twenty twenty seven or twenty twenty eight that eighty percent of all the computer sold will have AI accelerators inside. And by the end of this year, we want at least one hundred different software developers partnered with Intel. We want to deliver three hundred or so different AI features into the marketplace through all of those companies, help them optimize it, deliver it, market it. We want to bring one hundred million accelerators into the space by the end of twenty twenty five, and so just between now and twenty twenty seven twenty eight, we're talking about zero to eighty percent of the market in under four years, which is an extraordinary velocity. And as of right now, Intel has the largest number of accelerators, the most number of applications in the market, and sort of below the scenes, all the enabling tools and softwares and frameworks, we also have the most of those as well, So we want to be the scale provider for AI, you know, the biggest install based and we want to help to succeed because that's what software vendors are asking us to do. So that's the longest short of it.
That's great. I think I'll leave it there. Thanks Robert. Thanks my deepest thanks to Robert Hallock for his invaluable contributions to today's episode of Technically Speaking. Robert's passion and enthusiasm has convinced me that we are on the brink of a revolutionary technology that could transform humanity. We've all used PCs both in our personal and professional lives, but the leap to aipowered processes represents a once in a generation advancement. Imagine having your own personal AIS system that continually adapts to your needs and priorities. Admittedly, this is a prospect I'm still trying to fully grasp, but I especially appreciate that this technology will operate locally on my PC, keeping my data private and secure. I'm also particularly interested in how technology can empower the underdog to do amazing things. The garage based tech entrepreneur building the next world beating app the corner cafe owner getting that extra hour per day to spend with their family, and the first year design student creating beautiful and innovative art. With AI enhanced pieces readily accessible, I believe we can greatly advance human prosperity. The future looks bright. Ground. Next episode will continue to unpact the involvement of AI to create a more accessible and livable city for everyone. Join us again on Tuesday, April twenty third for another enlightening discussion here on Technically Speaking and Intel podcast. Technically Speaking was produced by Ruby Studio from iHeartRadio in partnership with Intel and hosted by me Graham Class. Our executive producer is Molly Sosher, our EP of Post Production is James Foster, and our supervising producer is Nikia Swinton. This episode was edited by Sierra Spreen and was written by Molly Sosher and Nick Ferschall.