Welcome to tech Stuff, a production from iHeartRadio. Hey there, and welcome to tech Stuff. I'm your host, Jonathan Strickland. I'm an executive producer with iHeart Podcasts. And how the tech are you? Now? There's a pretty good chance that you've heard or read something about AI chips. But what the heck is an AI chip? Is it a microchip that actually has artificial intelligence incorporated directly into the semiconductor material somehow? And if so, what does that mean? I figured it would be a good idea to talk about microchips and processors and AI enabled chips in particular to help demystify everything because part of the problem, I think is that AI chips are are kind of becoming a marketing term. It's not just a way to describe technology. It's a way to try and set aside a product to try and you know, pose it as the new hotness. And yes, I know I'm ancient and I use outdated slang. So first, when I talk about microchips, I'm talking about integrated circuits. Jack Kilby invented the first integrated circuit back in nineteen fifty eight at Texas Instruments, and an integrated circuit is a collection of interconnected electronic components that happens to be built on top of a semiconductor material. Semiconductors, as the name suggests, our materials that under certain conditions will conduct electricity and under other conditions will insulate or block the flow of electricity. The invention of the transistor, in addition to the integrated circuit is what allowed for miniaturization. That's why computers no longer have to be huge, you know. I'm talking about like those old computers, those mainframes that would fill up entire rooms or even an entire floor of a building. Miniaturization would eventually allow for the production of powerful personal computers that were a fraction of the size of their predecessors, but just as powerful or even more so. The development of arithmetic logic units or ALUs, which, as the name suggests, are circuits designed to perform arithmetic or mathematical functions on inputs and then produce the relevant outputs. Those, in turn served as a building block for the development of central processing units or CPUs. The first CPU microprocessor, arguably was Intel's for zero zero four computer on a CHEP. So this was a fairly limited processor, particularly if we judge it by today's standards. I think it had like a four bitwidth bus that would allow for processing data, which means it could not handle very large values. But you have to start somewhere, and the four zero zero four was a stepping stone to Intel's eight zero zero eight processor. That was the processor that was found in a lot of the first commercial personal computers. They used the eight zero zero eight processor as their CPU. Now, a central processing unit's job is more complex than that of an ALU. In fact, ALUs are part of a CPU. They are a component that make up part of a CPU. The CPU's job is to accept incoming instructions from programs, to retrieve those instructions, to execute those instructions on data, and to produce the relevant outcomes. And they carry out logic operations. They send results to the appropriate destination. That destination might be feeding back into software to continue that process, or it might mean that you're feeding output to some sort of output device like a display or a printer or something along those lines. CPUs have two very broad categories of operations. Again, this is super super high level. I mean, we could get far more complicated than this, but they have logic functions and memory functions. Memory being that's where you store information so that you can reference it quickly in order to carry out these operations, and logic being the actual logic gates that end up defining how data gets processed. Those are the two big components, and there are a few different ways that we measure CPU performance. One is we measure it by clock speed. You can think of this as the number of instructions the CPU is able to handle every second. So the higher the clock speed, the more instructions the CPU is able to handle per second. That like three point six gigahertz would mean three point six billion operations or instructions I should say per second. You can also have operations that have multiple sets of instructions, so it's a little more complicated than just saying, oh, it can handle this number of operations per second. Now, you can also have CPUs that have multiple cores, and a core is essentially all the little individual components of a CPU compartmentalized so that you have almost like multiple CPUs on a single chip. A single core processor is like a really fast processor. A multicore processor is one that divides the processor capabilities into these individual cores, and you might wonder, well, why would you want to do that, Why would you want to take something that is typically very powerful and very fast and then divide that up into smaller units. Well, that's because some computational processes are able to be performed in parallel. This means you can divide up a task into smaller jobs and then assign those smaller jobs to individual cores. So for these kinds of processes, a multi core processor can sometimes be faster than a more powerful single core processor would, And that means it's time to use an analogy. I bust out every time I talk about parallel processing. Fans of tech stuff who have been around for years, you all know what's going to happen. Go ahead and make yourself a cup of coffee or something. So imagine you have a math class. You're a teacher. You've got a math class, and your math class has five students in it. It's very small focus group. One of those five students is like a super math genius. They are leagues ahead of the other students. The other four students are good at math, they're great students, but they take a little more time than the genius does to work out Your typical math problem. So you decide you're going to give a pop quiz to your class. But this pop quiz is a race. It's a race that is pitting the super genius against the other four students. Now, if that pop quiz consisted of just one mathematical problem, or if it had a series of math problems, but those math problems were sequential, which means like the information you need to solve question two can be found in the answer of question one. If that were the case, your super genius is gonna win, right because they would be able to solve the problem or series of problems much more quickly than anyone in the rest of the class. And you can't divide that problem up. If it's a series that you know question two depends on the outcome of question one, you can't divide that up because you wouldn't have the information you need to work on the problem until the first part was solved. However, let's say instead you make a pop quiz that has four math problems on it. Each of these four math problems is self contained. They do not depend upon the outcomes of any of the other questions. So the super genius needs to finish all four problems. But for your other four students, they're given the option that they can each tackle a different problem on the quiz, and if all four of them finish whatever respective problem they've chosen first, then as a group they win. Now, in that case, the four students are far more likely to come out on top. The super genius could be as far as like question three or four, But each of the other students only has to solve a single problem in order to complete the pop quiz. That's like a multi core processor working on a parallel processing problem. For some subsets of computational operations, having multiple cores to work on things all at the same time is a big advantage, all right. So that's a super high level look at CPUs. Now let's turn to GPUs. These are graphics processing units. The name actually comes from the g Force two fifty six graphics card from Nvidia. So in the nineteen nineties we saw the introduction of new graphics intensive applications, particularly in things like video editing or in video games, and the CPUs of that era were not necessarily optimized to get the job done, like it was more work than the CPU could typically handle. So the performance of these kinds of programs would be substandards. Sometimes the programs wouldn't even run on a computer that just had a CPU, even a good CPU. So then you had companies like in Video that began to introduce graphics cards, and these graphics cards had integrated circuits that were better designed. They were optimized to handle graphics processing specifically, so that would let the CPU offload the graphics processing jobs to the graphics card. The CPU could then focus on other operations. The g Force two fifty six introduced a ton of new capabilities and features. And while the graphics processing unit name might have just started off as kind of a marketing strategy, you know, Nvidia gave the g Force two fifty six this designation of graphics processing unit to kind of set it apart from other graphics cards that were on the market. Well, it would turn out that the GPU name would have staying power, and today any self respecting gamer has a powerful GPU in their gaming rig. The GPUs would grow to be more important than CPUs, at least for some people. Though it would be reductive to say that gamers only need a powerful GPU and they don't have to worry about the CPU at all. It honestly depends a lot on the types of games they play. That is a big component. Sometimes having a really fast GPU isn't going to help you out that much. It all depends on the types of processing you're doing. If you're not doing a lot of parallel processing, then a really fast GPU isn't likely to boost your performance that much. But the real purpose of a GPU is to perform certain types of computational operations very quickly and efficiently, in order to do stuff like speed up image creation, video and animation. As it would turn out, GPUs would also be handy for other things. So your typical GPU consists of many specialized processor cores. These cores are not designed to do everything you know. They do a subset of operations really well, but if you ask a GPU to do something outside of that, it's not going to perform at you know, at the same level as your typical CPU would. But this does mean a GPU is a fantastic tool for specific operations and then less useful for others. Apart from processing graphics, GPUs have been found to be really useful in applications ranging from machine learning projects to proof of work cryptocurrency mining operations. Now to be clear, GPUs, at least until recently, occupied a kind of a sweet space in crypto minds. They are not the top of the heap when it comes to crypto mining integrated circuits. We'll get to the kind that are used in high end crypto mining in just a little bit. So for stuff like Bitcoin, which as I record this episode, is trading at around fifty eight thousand dollars per coin. In fact, I think it's like fifty eight point five thousand. That's a lot of money. Well, if you're using GPUs, you're not going to cut it. You're not going to compete in that space. GPUs just can't operate at a level that would make it feasible for you to use them for your mining operations. That's because the value of bitcoin is so high that it drives cryptocurrency miners to seek out the absolute top tier processors, and GPUs, while they're great, they're really more mid tier. Now it helps if you know what proof of work crypto mining is all about. So with proof of work systems, you have a network of machines that make up this cryptocurrency network, such as bitcoin. We'll use Bitcoin as the main example because that was sort of the progenitor of this space. So every so often the network issues a challenge, which is to solve a mathematical problem, and if you do solve it, if you're the first one to solve it, you will receive some crypto coins as a reward. The act of solving typically is tied to validating a block of crypto transactions, So the problem's complexity will depend upon a couple of different things. Typically, there's an ideal amount of time that it should take to solve this mathematical problem. For bitcoin, that time is ten minutes. The other thing that determines the complexity of the problem is how much computing power is being thrown at solving the problem in the first place. So let's go back to our classroom analogy. Let's say that you're creating a test, and for whatever reason, you have decided this test should take the students ten minutes to complete, so you're not really focused on any other outcome other than trying to make a test that's going to take ten minutes to complete. However, you've misjudged the difficulty. Maybe one of your students hands in their test six minutes in. Now you know you need to make the next test harder in order to hit this seemingly arbitrary goal of ten minutes. On the flip side, let's say the first student to solve the test took fifteen minutes to complete it. Then you know your test is too hard and you need to ease up a little bit for the next test. When the value of cryptocurrency goes up, there's a greater incentive to be the first to solve the mathematical problem because the reward is larger. That drives miners to buy more processors and to network them together, and these are processors that are particularly good at solving the types of problems that you get when you're crypto mining. For a while, that meant GPUs they were the best. But the value of bitcoin went up and up and up, and there were other options besides GPUs. There were options that were more expensive than GPUs, so it require a bigger investment, but then on top of that, you were looking at bigger rewards, so it made that investment worthwhile. So the integrated circuit that typically replaces GPUs for high end cryptocurrency mining, those would be application specific integrated circuits or AASAC ASIC. We'll get to those in just a bit, so you could if you wanted to still run mining rigs using GPUs, nothing would stop you from doing that, but you'd be going up against people with networks and machines running much more streamlined optimized processors, so you would be unlikely to beat them. Okay, we got a lot more to cover, but let's take a quick break to thank our sponsors. Okay, we're coming back to talk a little bit more about crypt currency mining in GPUs. So for a while, people who were crypto mining ethereum would stick with GPUs. The reason for this is ethereum had a lower value, much lower than bitcoin. All right, We're talking about a few thousand dollars as opposed to tens of thousands of dollars per coin, and this meant that it would be impractical to use high end integrade circuits like AASC circuits for mining ethereum because the cost of doing so would be such that you wouldn't be making up that cost in the profit you gained from mining the cryptocurrency. So sticking with GPUs made more sense, right because from an economic standpoint, that was the sweet spot. However, then Ethereum switched to proof of steak instead of proof of work. Proof of steak doesn't do that whole math problem thing at all, and the demand for GPUs and crypto mining plummeted as a result. There are other cryptocurrencies out there, some of which that still do use proof of work, but they're not as sought after as Bitcoin or ethereum are. So this meant that the demand for GPUs in the crypto space began to diminish, and that became really good news for people who wanted a GPU for something else, like for a gaming rig for example. Now, I would say for the majority of people out there, like your average consumers, CPUs and GPUs are the beginning and end of it when it comes to processors or microchips that are meant to act like processors, But there are a couple of other varieties out there that we use for special purposes. And the special purpose thing is the important part to keep in mind. A CPU, by necessity, has to be able to do a bit of everything right. Because a CPU is the control center of your typical computer. It needs to be able to handle operations from a variety of different programs and that kind of thing. It is a jack of all trades master of none. It needs to be able to handle whatever you throw at it, but that means it cannot be optimized for any specific task. So what it lacks in efficiency, it makes up for inversatility. GPUs are more specialized and so they can handle certain processes better than a CPU typically can, But a GPU might not be so good at executing all the different tasks that a CPU has to handle, So while it is faster with some stuff, it's slower with other stuff. Now, the next two types of semiconductor devices I want to mention are even more specialized than GPUs, and then we'll end with one that is specialized specifically for the AI field. So next up is the field programmable gait arrays or FPGA's now a definition from XLinks dot com because x links is what introduced this technology back in the mid nineteen eighties. So x links defines this as FPGA's quote. Are based around a matrix of configurable logic blocks CLBs connected via programmable interconnects. End quote that sounds like gibberish to some folks. It's definitely got some barriers there from easy understanding, but the idea is pretty simple. When you boil it down to what's basically happening. So imagine that you have a microchip and you're able to reconfigure the individual components on that microchip so that they're optimized for whatever it is you need to do. So you can reprogram this chip, in other words, so that it is better aligned with the task you have at hand. As I said, x links first introduced this type of integrated circuit back in nineteen eighty five, and the aim us to make an integrated circuit that could potentially fit the needs of different specific use cases, not by being a jack of all trades that could do anything, but do so at a kind of a mediocre level, but rather by being configured to work best for that specific application. Moreover, you can at least sometimes reconfigure without having to change the actual physical architecture of the chip itself. This is important because not everyone has access to a clean room with incredibly precise and computer operated tools. That's exactly what you would need if you wanted to perform surgery on a microchip. Instead, an FPGA has these CLBs that x links talked about, the configurable logic blocks. These can be programmed to act like simple logic gates, and these gates follow specific rules. Essentially, they either allow electrical current to flow through or they block it from flowing through, and this, when you look at it a macro level, is what allows operations on a processor. The field and field programmable gate array means you can actually do this reprogramming after the FPGA has shipped from its manufacturer. So instead of working with a manufacturer to specialize a chip from the design phase and then go all the way through to manufacturing, the manufacturer makes this FPGA that can potentially be one of thousands of different configurations, and then you program it once you receive it. Now, some of these FPGAs are limited to kind of a one time only configuration, so you can program them once you get them, but then they're set in that particular configuration from that point forward. But others are designed so that they can be reprogrammed multiple times, which obviously makes them very useful. If you wanted to prototype a technology and you aren't really sure which configuration is going to be best for whatever it is you're trying to do, it's great to have a chip you can reprogram so you can try different configurations to find the one that makes the most sense for whatever it is you're trying to achieve. One issue with FPGA's is that they are not cost efficient when you're looking at mass production. They're great if you are prototyping, but if you plan to make a whole bunch of them, it gets time consuming and expensive because not only do you have to have them made, then you have to have them programmed. Plus sometimes you may have an application in mind that an FPGA cannot accommodate even with all the reconfiguring. So think of an FPGA as having a limited number of configurations and it turns out that what you need is outside of this range. That would mean you would need to add additional integrated circuits to your system to accommodate these limitations of the FPGA itself, which means you're adding more complexity to your system, and that in turn also means you're adding more costs to your system. Next up, we have the one I mentioned earlier, the Application specific integrated circuit or AZC ASIC, as the name indicates, These chips are made to operate for specific applications, and as such, they are highly optimized from the hardware level up for that purpose. They are not meant to be general purpose processors like a CPU. So if you put an ASK to work on a task that it was not designed to handle, you are not going to get a good result. In fact, it may not work at all. But when it's integrated into a system that's meant to do that one thing it was designed to do, it does it really well. And AAK can be a speed demon and operate at an efficiency that's much more desirable than your typical CPU or even GPU. So unlike an FPGA and ASK cannot be reconfigured. It is a high tech, highly specialized chip. There are a few different approaches to create that specialization during the manufacturing process, but I feel like that's beyond the scope of this episode. I'll save it for a time when I do a full episode about AZK chips. Now, the design process for AZIK is complicated. So imagine you're building a chip intended to do one thing extremely well. You would have to do a lot of work to make sure that the chip you were designing met that purpose. So that means there's a lot of R and D and there's a lot of testing. However, once you do arrive at this final design, one big advantage of AZK over FPGA is that it can then go into large volume production. So while the development process of an AZK is typically longer and more expensive than using an FPGA, once you do get to the production stage, the AZIC chips become more cost effective. So if you're doing a one off, FPGA makes the most sense financially. If your goal is to make something that you're going to mass produce, AZIC makes far more sense. ASIC chips also tend to be more power efficient than FPGA's, so by their nature, an FPGA needs to have components that aren't necessary for all applications because the whole point of an FPGA is that you can reprogram them to do specific tasks, but not every task is going to need every component that's on that circuit. So that means there's going to be some extra stuff on that integrated circuit that ends up being superfluous for certain operations. With AZC, you can leave off anything that would be superfluous, right, You can leave that out of the design because you know ahead of time what you're putting this chip to work for, so you can only focus on the things that are absolutely necessary for the operation of that chip. That means you don't have to supply power to components that aren't actually doing anything. That keeps your power consumption costs lower in the long run. Thus, ASIC chips are more efficient. Now, most of us are not going to be shopping around for ASK chips. Your average consumer has no need for them. But for folks like cryptomners, AZK might make sense once you reach a certain level of profit. Right, once you reach a certain level at least potential profit if you mine a block of the cryptocurrency. Because again, Bitcoin created the perfect storm for this back when it was awarding six point two five coins per block, so that meant in an average day the system would release, or rather miners would mine around nine hundred bitcoins total per day, and with bitcoins trading at fifty grand each, that would mean around forty five million dollars worth of bitcoin were up for grabs every single day. That's what justified spending the huge amount of money it costs to develop and deploy ACC chips for the specific task of mining bitcoin. Yes, that design process is incredibly expensive, but if you could create a system that could grab a significant number of bitcoins every day, then it would pay for itself pretty darn quickly. You might not get all the bitcoins, you might not even get most of them, but as long as you were grabbing a decent number every single day, you would quickly accumulate wealth and justify the cost of using ACAC technology. That's what left GPU miners in the dust, because once acc systems joined the party, the GPUs just could not compete. It would be kind of like if you put me in the one hundred meter dash in the Olympics, the lead runner would be crossing the finish line before I managed to get a quarter of the way there. Now I should add that this year, in twenty twenty four, the number of bitcoins awarded per block dropped by half. This was all part of the plan. This wasn't a mistake or something. Now, if you mine a block, instead of getting six point twenty five coins, you end up getting three point one two five. So again, this was this was planned, and every four years or so the system cuts the number of coins awarded per block mind by fifty percent. When bitcoin first hit the scene back in early two thousand and nine, if you mined a block successfully you would net yourself fifty bitcoins per pop. But of course, back in two thousand and nine, the value of bitcoin was fractions of a cent. You wouldn't apply AASAC technology to bitcoin mining back in those days because the coins weren't really worth anything. In fact, on May twenty second, twenty ten, this is a famous date in crypto history. This was more than a year after bitcoin had launched. A cryptocurrency minor named Laslow's spent ten thousand bitcoins in order it to order ap pizza. So today that pizza would be worth more than five hundred and eighty five million dollars. And in fact, another interesting point, Bitcoin is a lot of volatility. When I started work on this episode, it was trading at fifty seven thousand dollars and now it's at more than fifty eight thousand, so the value changes pretty drastically. Anyway, getting back to the having, part of the bitcoin strategy is that there's a finite number of bitcoin that will ever be released into circulation, and once the last one is in circulation, no more new bitcoin will be minted. So specifically, that makes up twenty one million bitcoin to control the release of bitcoin into circulation. The system does this having business every four years, so today mining a block on the Bitcoin network will earn you three point one two five bitcoins, or around one hundred and eighty one thousand dollars worth of bitcoin crypto per block. Mind, these kinds of changes affect mining operations because if the magic number dips too much, then it would cost more to mine bitcoin. Then you would get out of mining it, so you would have to adjust your strategy. Right, you'd say, all right, well, now it doesn't make sense for me to operate this massive computer network of AASC machines that's drawing power directly from a formerly decommissioned power plant because the cost of operations is sky high and the amount that I'm able to actually mine is much lower. Now I have one more initialism to throw your way, but before we get to that, let's take another quick break to thank our sponsors. Okay, we're back, and we've talked about CPUs, and we've talked about GPUs, and we've talked about FPGAs talked about ASICs. Now it's time to talk about NPUs. And as a nancy, the initialism stands for neural processing unit. These have technically been around for a few years now, but the term is still fairly new. For mainstream audiences. I think you started to see them pop up in mainstream tech journals last year, but they've been around for a few years. And NPU is a chip with a specialized design meant for AI applications and artificial neural networks in particular. Now, just in case artificial neural networks, if that term is new to you, it is a network of processors that collectively mimics the way our neurons interconnect with one another in our brain meet. That's a very high level and oversimplified explanation, but it kind of gets the idea across. Artificial neural networks are often used in the field of machine learning, in which researchers train computer system to produce specific results given specific input. Now, that could be as simple as indicating which of a million different photographs are the ones that happen to have cats in them versus ones that don't have cats in them, or it could be something far more complicated, like learning which environmental factors impact the development of weather systems so that you can have a more accurate weather forecast. And NPU is tuned to work in this discipline, and often it could produce much better results than a GPU. Both an NPU and a GPU tend to be made with parallel processing in mind, and NPUs are typically incorporated onto integrated circuits that also have a CPU. They don't necessarily replace a CPU, they are in addition to one. So let's wrestle this all back to artificial intelligence. When you hear the phrase AI chip, chances are the chip question is one of four types. It's an FPGA, an ASIC, an NPU, or a GPU. Now you can have AI enabled CPUs that don't have these other components. But the problem with CPUs is that due to their unspecialized design, they have limited usefulness when it comes to AI applications, particularly as the AI field becomes more sophisticated and has greater data processing needs. It's kind of like giving a really good third year math student a challenging quiz meant for fifth year students. Our little test subject might do a decent job at the end of the day, but it will likely take them longer and cause more exertion than it would for someone who is more attuned to the task. So with CPUs, that means that you have to have longer processing times and you have to use more energy in order to be able to complete the task, and that means also generating more heat. It's less efficient, it's less money efficient as well, not just power efficient, but financially efficient. So two of the components you find on these integrated circuits are logic gates and we could just call them transistors for simplicity, and then memory. So while a CPU depends on both of these quite a bit in order to do its job, specialized chips like ASICs AASEYS can be made to emphasize the logic components more than the memory components, and they can be packed with more transistors with less space reserved for memory. That's typically what AI needs needs access to large capacity for data processing, so the goal is to allow for more data processing per unit of energy than you would get out of a typical microchip. AI is a power hungry technology. I mean that literally. Maybe one day AI will be power hungry in the figurative sense, like in like the super villain sense. Maybe that will happen one day, but right now, it's just it needs a lot of juice. So making the processing as efficient as possible is absolutely vital, Otherwise the costs of operations spiral out of control. Your energy needs as well as things like cooling needs and everything else that goes along with using a bucket load of power would make it harder for you to cover costs. This is part of the reason why you'll hear about companies spending billions of dollars on AI. It's not just that they have to spend that money for the research and development of AI, although that takes up a big part of it. It's that actually operating these data centers that are running these specialized machines takes a lot of energy, and so the cost of operation is in the billions of dollars. Now, these AI chips typically can handle parallel processing tasks in a much greater capacity than even your most powerful multi thread into multi core CPUs can. Which type of chip you use often depends upon the application you want, so, for example, Google's tensor processing Unit is an ASK chip. Google has spent a lot of time and money developing these processors and fine tuning them to handle intense data processing at incredible speed for the purposes of machine learning applications. Primarily, a lot of AI companies will use off the shelf GPUs and they will wire them together in order to train AI models, which has led to Nvidia, which for years was thought of as just a graphics processing unit design company, to now become a leading AI chip company. The boom in AI development has catapulted Nvidia to become a three trillion dollar company in recent years, so it has joined the likes of Microsoft and Apple. That's not to say Nvidia was always like an underdog or anything. It was always a company that was doing pretty darn well, but in recent years it has entered the stratospheric level evaluation. It was not a trillion dollar company that long ago. When we talk about consumer products, CPUs and NPUs are typically what will handle AI needs because they are the more cost efficient approach. Intel has developed NPUs under the code name metior Lake. Actually, to be more precise, the metior Lake chips include CPU cores. They also include a small GPU portion as well as the NPU unit, all on this same integrated circuit. And the idea is that these chips will be incorporated into machines that can run AI workloads locally. So let's say you've got a company and that company wants to host a language model, but it wants it locally. It doesn't want to be tapping into a cloud based language model, they want to run it on premises, while they might use computers with meteor Lake chips in them in order to do that processing, which would be more cost effective than building out a whole AI data center just to service this specific company. Okay, so when people talk about AI chips, they don't mean that somehow the chips are imbued with artificial intelligence. Instead, these chips are optimized to run AI applications, and those applications run the entire gamut of AI. There are AI chips used in robotics, There are AI chips used in autonomous cars. There are AI chips for large language models. Smaller chips and NPUs can be incorporated into smart devices, which allow some AI processing to happen at the device level rather than remotely through a network connection. That's really important for speeding up those processes and to eliminate latency, because for some implementations speed might not be that big a deal, but for others, like the autonomous cars I mentioned, being able to process information and produce results is critical to operate the technology safely. You cannot have latency in those systems or disaster can occur. You wouldn't want an autonomous car that constantly has to beam information up to the cloud and wait for a response, because real world driving conditions are constantly changing. They are dynamic, and they change at a very fast rate. Depending on how quickly you're driving, it could be an incredibly fast rate. So any latency would lead to catastrophic outcomes. So AI chips are important components in what you might call EDGEAI. This not only cuts down on processing time, but it can also help things remain more secure. Right, you're not beaming data to a different location all the time, you're processing it locally. That makes it less susceptible to being hacked. Not immune, but it's less susceptible. There's fewer links in the chain, you could say. So now we have our overview of what AI chips are all about. And I think it's good to remember that a processor's utility depends entirely upon what you plan to actually use it for. If you're doing standard computing stuff like you're working with documents or playing games or browsing the web, and AI chip isn't really going to mean much to you at all. AI chips tend to be really geared toward parallel processing. So it's possible that a computer with a good AI chip could be useful as a gaming rig. But honestly, I think, at least for now, going with a good GPU and a decent CPU matters more for gamers, And like I said, some cases, you might not need a really good GPU. You could have a decent GPU and a really good CPU. It all kind of depends on the types of games you want to play. I think it's important for regular old folks like me and at least some of y'all out there, to know about this stuff so that when we're shopping around for our next device, we have an understanding of the terminology. Right, we know what an AI chip is and what it's supposed to do, and whether or not it matches what we need. We aren't just pooled by marketing terms. You know. It doesn't mean that an AI chip labels slapped on something is going to mean that that's the best thing for us. So having this understanding is important. Being an informed consumer is important. It means you're going to get the best out of your money that meets your needs. Right, we only have so much money. We should make sure that when we're spending it. We're doing it on stuff that actually solves the problems we have, as opposed to just stuff that's shiny and new. I say this because a lot of tech enthusiasts tend to fall into the trap of I want the new thing because the new thing is somehow better than the old thing. That's not always the case. It often is in tech, but it's not always the case, and it certainly doesn't always justify spending the amount of money it takes to be part of that bleeding edge. It is important that we have a bleeding edge, but it's not important that we're all in it. We can hang back a bit if we need to. So I just wanted to take this chance to kind of break down this AI chip terminology and what it actually means, because goodness knows, Like when I started first seeing the terminology myself, I was confused. I was thinking, what makes an AI chip and aichip? And does it have some sort of AI capability built into it? Because how would that work? And obviously I was overthinking it. So hopefully this was useful for y'all, and I hope you're all doing well, and I'll talk to you again really soon. Tech Stuff is an iHeartRadio production. For more podcasts from iHeartRadio, visit the iHeartRadio app, Apple Podcasts, or wherever you listen to your favorite shows.

Welcome to tech Stuff, a production from iHeartRadio. Hey there, and welcome to tech Stuff. I'm your host, Jonathan Strickland. I'm an executive producer with iHeart Podcasts. And how the tech are you? Now? There's a pretty good chance that you've heard or read something about AI chips. But what the heck is an AI chip? Is it a microchip that actually has artificial intelligence incorporated directly into the semiconductor material somehow? And if so, what does that mean? I figured it would be a good idea to talk about microchips and processors and AI enabled chips in particular to help demystify everything because part of the problem, I think is that AI chips are are kind of becoming a marketing term. It's not just a way to describe technology. It's a way to try and set aside a product to try and you know, pose it as the new hotness. And yes, I know I'm ancient and I use outdated slang. So first, when I talk about microchips, I'm talking about integrated circuits. Jack Kilby invented the first integrated circuit back in nineteen fifty eight at Texas Instruments, and an integrated circuit is a collection of interconnected electronic components that happens to be built on top of a semiconductor material. Semiconductors, as the name suggests, our materials that under certain conditions will conduct electricity and under other conditions will insulate or block the flow of electricity. The invention of the transistor, in addition to the integrated circuit is what allowed for miniaturization. That's why computers no longer have to be huge, you know. I'm talking about like those old computers, those mainframes that would fill up entire rooms or even an entire floor of a building. Miniaturization would eventually allow for the production of powerful personal computers that were a fraction of the size of their predecessors, but just as powerful or even more so. The development of arithmetic logic units or ALUs, which, as the name suggests, are circuits designed to perform arithmetic or mathematical functions on inputs and then produce the relevant outputs. Those, in turn served as a building block for the development of central processing units or CPUs. The first CPU microprocessor, arguably was Intel's for zero zero four computer on a CHEP. So this was a fairly limited processor, particularly if we judge it by today's standards. I think it had like a four bitwidth bus that would allow for processing data, which means it could not handle very large values. But you have to start somewhere, and the four zero zero four was a stepping stone to Intel's eight zero zero eight processor. That was the processor that was found in a lot of the first commercial personal computers. They used the eight zero zero eight processor as their CPU. Now, a central processing unit's job is more complex than that of an ALU. In fact, ALUs are part of a CPU. They are a component that make up part of a CPU. The CPU's job is to accept incoming instructions from programs, to retrieve those instructions, to execute those instructions on data, and to produce the relevant outcomes. And they carry out logic operations. They send results to the appropriate destination. That destination might be feeding back into software to continue that process, or it might mean that you're feeding output to some sort of output device like a display or a printer or something along those lines. CPUs have two very broad categories of operations. Again, this is super super high level. I mean, we could get far more complicated than this, but they have logic functions and memory functions. Memory being that's where you store information so that you can reference it quickly in order to carry out these operations, and logic being the actual logic gates that end up defining how data gets processed. Those are the two big components, and there are a few different ways that we measure CPU performance. One is we measure it by clock speed. You can think of this as the number of instructions the CPU is able to handle every second. So the higher the clock speed, the more instructions the CPU is able to handle per second. That like three point six gigahertz would mean three point six billion operations or instructions I should say per second. You can also have operations that have multiple sets of instructions, so it's a little more complicated than just saying, oh, it can handle this number of operations per second. Now, you can also have CPUs that have multiple cores, and a core is essentially all the little individual components of a CPU compartmentalized so that you have almost like multiple CPUs on a single chip. A single core processor is like a really fast processor. A multicore processor is one that divides the processor capabilities into these individual cores, and you might wonder, well, why would you want to do that, Why would you want to take something that is typically very powerful and very fast and then divide that up into smaller units. Well, that's because some computational processes are able to be performed in parallel. This means you can divide up a task into smaller jobs and then assign those smaller jobs to individual cores. So for these kinds of processes, a multi core processor can sometimes be faster than a more powerful single core processor would, And that means it's time to use an analogy. I bust out every time I talk about parallel processing. Fans of tech stuff who have been around for years, you all know what's going to happen. Go ahead and make yourself a cup of coffee or something. So imagine you have a math class. You're a teacher. You've got a math class, and your math class has five students in it. It's very small focus group. One of those five students is like a super math genius. They are leagues ahead of the other students. The other four students are good at math, they're great students, but they take a little more time than the genius does to work out Your typical math problem. So you decide you're going to give a pop quiz to your class. But this pop quiz is a race. It's a race that is pitting the super genius against the other four students. Now, if that pop quiz consisted of just one mathematical problem, or if it had a series of math problems, but those math problems were sequential, which means like the information you need to solve question two can be found in the answer of question one. If that were the case, your super genius is gonna win, right because they would be able to solve the problem or series of problems much more quickly than anyone in the rest of the class. And you can't divide that problem up. If it's a series that you know question two depends on the outcome of question one, you can't divide that up because you wouldn't have the information you need to work on the problem until the first part was solved. However, let's say instead you make a pop quiz that has four math problems on it. Each of these four math problems is self contained. They do not depend upon the outcomes of any of the other questions. So the super genius needs to finish all four problems. But for your other four students, they're given the option that they can each tackle a different problem on the quiz, and if all four of them finish whatever respective problem they've chosen first, then as a group they win. Now, in that case, the four students are far more likely to come out on top. The super genius could be as far as like question three or four, But each of the other students only has to solve a single problem in order to complete the pop quiz. That's like a multi core processor working on a parallel processing problem. For some subsets of computational operations, having multiple cores to work on things all at the same time is a big advantage, all right. So that's a super high level look at CPUs. Now let's turn to GPUs. These are graphics processing units. The name actually comes from the g Force two fifty six graphics card from Nvidia. So in the nineteen nineties we saw the introduction of new graphics intensive applications, particularly in things like video editing or in video games, and the CPUs of that era were not necessarily optimized to get the job done, like it was more work than the CPU could typically handle. So the performance of these kinds of programs would be substandards. Sometimes the programs wouldn't even run on a computer that just had a CPU, even a good CPU. So then you had companies like in Video that began to introduce graphics cards, and these graphics cards had integrated circuits that were better designed. They were optimized to handle graphics processing specifically, so that would let the CPU offload the graphics processing jobs to the graphics card. The CPU could then focus on other operations. The g Force two fifty six introduced a ton of new capabilities and features. And while the graphics processing unit name might have just started off as kind of a marketing strategy, you know, Nvidia gave the g Force two fifty six this designation of graphics processing unit to kind of set it apart from other graphics cards that were on the market. Well, it would turn out that the GPU name would have staying power, and today any self respecting gamer has a powerful GPU in their gaming rig. The GPUs would grow to be more important than CPUs, at least for some people. Though it would be reductive to say that gamers only need a powerful GPU and they don't have to worry about the CPU at all. It honestly depends a lot on the types of games they play. That is a big component. Sometimes having a really fast GPU isn't going to help you out that much. It all depends on the types of processing you're doing. If you're not doing a lot of parallel processing, then a really fast GPU isn't likely to boost your performance that much. But the real purpose of a GPU is to perform certain types of computational operations very quickly and efficiently, in order to do stuff like speed up image creation, video and animation. As it would turn out, GPUs would also be handy for other things. So your typical GPU consists of many specialized processor cores. These cores are not designed to do everything you know. They do a subset of operations really well, but if you ask a GPU to do something outside of that, it's not going to perform at you know, at the same level as your typical CPU would. But this does mean a GPU is a fantastic tool for specific operations and then less useful for others. Apart from processing graphics, GPUs have been found to be really useful in applications ranging from machine learning projects to proof of work cryptocurrency mining operations. Now to be clear, GPUs, at least until recently, occupied a kind of a sweet space in crypto minds. They are not the top of the heap when it comes to crypto mining integrated circuits. We'll get to the kind that are used in high end crypto mining in just a little bit. So for stuff like Bitcoin, which as I record this episode, is trading at around fifty eight thousand dollars per coin. In fact, I think it's like fifty eight point five thousand. That's a lot of money. Well, if you're using GPUs, you're not going to cut it. You're not going to compete in that space. GPUs just can't operate at a level that would make it feasible for you to use them for your mining operations. That's because the value of bitcoin is so high that it drives cryptocurrency miners to seek out the absolute top tier processors, and GPUs, while they're great, they're really more mid tier. Now it helps if you know what proof of work crypto mining is all about. So with proof of work systems, you have a network of machines that make up this cryptocurrency network, such as bitcoin. We'll use Bitcoin as the main example because that was sort of the progenitor of this space. So every so often the network issues a challenge, which is to solve a mathematical problem, and if you do solve it, if you're the first one to solve it, you will receive some crypto coins as a reward. The act of solving typically is tied to validating a block of crypto transactions, So the problem's complexity will depend upon a couple of different things. Typically, there's an ideal amount of time that it should take to solve this mathematical problem. For bitcoin, that time is ten minutes. The other thing that determines the complexity of the problem is how much computing power is being thrown at solving the problem in the first place. So let's go back to our classroom analogy. Let's say that you're creating a test, and for whatever reason, you have decided this test should take the students ten minutes to complete, so you're not really focused on any other outcome other than trying to make a test that's going to take ten minutes to complete. However, you've misjudged the difficulty. Maybe one of your students hands in their test six minutes in. Now you know you need to make the next test harder in order to hit this seemingly arbitrary goal of ten minutes. On the flip side, let's say the first student to solve the test took fifteen minutes to complete it. Then you know your test is too hard and you need to ease up a little bit for the next test. When the value of cryptocurrency goes up, there's a greater incentive to be the first to solve the mathematical problem because the reward is larger. That drives miners to buy more processors and to network them together, and these are processors that are particularly good at solving the types of problems that you get when you're crypto mining. For a while, that meant GPUs they were the best. But the value of bitcoin went up and up and up, and there were other options besides GPUs. There were options that were more expensive than GPUs, so it require a bigger investment, but then on top of that, you were looking at bigger rewards, so it made that investment worthwhile. So the integrated circuit that typically replaces GPUs for high end cryptocurrency mining, those would be application specific integrated circuits or AASAC ASIC. We'll get to those in just a bit, so you could if you wanted to still run mining rigs using GPUs, nothing would stop you from doing that, but you'd be going up against people with networks and machines running much more streamlined optimized processors, so you would be unlikely to beat them. Okay, we got a lot more to cover, but let's take a quick break to thank our sponsors. Okay, we're coming back to talk a little bit more about crypt currency mining in GPUs. So for a while, people who were crypto mining ethereum would stick with GPUs. The reason for this is ethereum had a lower value, much lower than bitcoin. All right, We're talking about a few thousand dollars as opposed to tens of thousands of dollars per coin, and this meant that it would be impractical to use high end integrade circuits like AASC circuits for mining ethereum because the cost of doing so would be such that you wouldn't be making up that cost in the profit you gained from mining the cryptocurrency. So sticking with GPUs made more sense, right because from an economic standpoint, that was the sweet spot. However, then Ethereum switched to proof of steak instead of proof of work. Proof of steak doesn't do that whole math problem thing at all, and the demand for GPUs and crypto mining plummeted as a result. There are other cryptocurrencies out there, some of which that still do use proof of work, but they're not as sought after as Bitcoin or ethereum are. So this meant that the demand for GPUs in the crypto space began to diminish, and that became really good news for people who wanted a GPU for something else, like for a gaming rig for example. Now, I would say for the majority of people out there, like your average consumers, CPUs and GPUs are the beginning and end of it when it comes to processors or microchips that are meant to act like processors, But there are a couple of other varieties out there that we use for special purposes. And the special purpose thing is the important part to keep in mind. A CPU, by necessity, has to be able to do a bit of everything right. Because a CPU is the control center of your typical computer. It needs to be able to handle operations from a variety of different programs and that kind of thing. It is a jack of all trades master of none. It needs to be able to handle whatever you throw at it, but that means it cannot be optimized for any specific task. So what it lacks in efficiency, it makes up for inversatility. GPUs are more specialized and so they can handle certain processes better than a CPU typically can, But a GPU might not be so good at executing all the different tasks that a CPU has to handle, So while it is faster with some stuff, it's slower with other stuff. Now, the next two types of semiconductor devices I want to mention are even more specialized than GPUs, and then we'll end with one that is specialized specifically for the AI field. So next up is the field programmable gait arrays or FPGA's now a definition from XLinks dot com because x links is what introduced this technology back in the mid nineteen eighties. So x links defines this as FPGA's quote. Are based around a matrix of configurable logic blocks CLBs connected via programmable interconnects. End quote that sounds like gibberish to some folks. It's definitely got some barriers there from easy understanding, but the idea is pretty simple. When you boil it down to what's basically happening. So imagine that you have a microchip and you're able to reconfigure the individual components on that microchip so that they're optimized for whatever it is you need to do. So you can reprogram this chip, in other words, so that it is better aligned with the task you have at hand. As I said, x links first introduced this type of integrated circuit back in nineteen eighty five, and the aim us to make an integrated circuit that could potentially fit the needs of different specific use cases, not by being a jack of all trades that could do anything, but do so at a kind of a mediocre level, but rather by being configured to work best for that specific application. Moreover, you can at least sometimes reconfigure without having to change the actual physical architecture of the chip itself. This is important because not everyone has access to a clean room with incredibly precise and computer operated tools. That's exactly what you would need if you wanted to perform surgery on a microchip. Instead, an FPGA has these CLBs that x links talked about, the configurable logic blocks. These can be programmed to act like simple logic gates, and these gates follow specific rules. Essentially, they either allow electrical current to flow through or they block it from flowing through, and this, when you look at it a macro level, is what allows operations on a processor. The field and field programmable gate array means you can actually do this reprogramming after the FPGA has shipped from its manufacturer. So instead of working with a manufacturer to specialize a chip from the design phase and then go all the way through to manufacturing, the manufacturer makes this FPGA that can potentially be one of thousands of different configurations, and then you program it once you receive it. Now, some of these FPGAs are limited to kind of a one time only configuration, so you can program them once you get them, but then they're set in that particular configuration from that point forward. But others are designed so that they can be reprogrammed multiple times, which obviously makes them very useful. If you wanted to prototype a technology and you aren't really sure which configuration is going to be best for whatever it is you're trying to do, it's great to have a chip you can reprogram so you can try different configurations to find the one that makes the most sense for whatever it is you're trying to achieve. One issue with FPGA's is that they are not cost efficient when you're looking at mass production. They're great if you are prototyping, but if you plan to make a whole bunch of them, it gets time consuming and expensive because not only do you have to have them made, then you have to have them programmed. Plus sometimes you may have an application in mind that an FPGA cannot accommodate even with all the reconfiguring. So think of an FPGA as having a limited number of configurations and it turns out that what you need is outside of this range. That would mean you would need to add additional integrated circuits to your system to accommodate these limitations of the FPGA itself, which means you're adding more complexity to your system, and that in turn also means you're adding more costs to your system. Next up, we have the one I mentioned earlier, the Application specific integrated circuit or AZC ASIC, as the name indicates, These chips are made to operate for specific applications, and as such, they are highly optimized from the hardware level up for that purpose. They are not meant to be general purpose processors like a CPU. So if you put an ASK to work on a task that it was not designed to handle, you are not going to get a good result. In fact, it may not work at all. But when it's integrated into a system that's meant to do that one thing it was designed to do, it does it really well. And AAK can be a speed demon and operate at an efficiency that's much more desirable than your typical CPU or even GPU. So unlike an FPGA and ASK cannot be reconfigured. It is a high tech, highly specialized chip. There are a few different approaches to create that specialization during the manufacturing process, but I feel like that's beyond the scope of this episode. I'll save it for a time when I do a full episode about AZK chips. Now, the design process for AZIK is complicated. So imagine you're building a chip intended to do one thing extremely well. You would have to do a lot of work to make sure that the chip you were designing met that purpose. So that means there's a lot of R and D and there's a lot of testing. However, once you do arrive at this final design, one big advantage of AZK over FPGA is that it can then go into large volume production. So while the development process of an AZK is typically longer and more expensive than using an FPGA, once you do get to the production stage, the AZIC chips become more cost effective. So if you're doing a one off, FPGA makes the most sense financially. If your goal is to make something that you're going to mass produce, AZIC makes far more sense. ASIC chips also tend to be more power efficient than FPGA's, so by their nature, an FPGA needs to have components that aren't necessary for all applications because the whole point of an FPGA is that you can reprogram them to do specific tasks, but not every task is going to need every component that's on that circuit. So that means there's going to be some extra stuff on that integrated circuit that ends up being superfluous for certain operations. With AZC, you can leave off anything that would be superfluous, right, You can leave that out of the design because you know ahead of time what you're putting this chip to work for, so you can only focus on the things that are absolutely necessary for the operation of that chip. That means you don't have to supply power to components that aren't actually doing anything. That keeps your power consumption costs lower in the long run. Thus, ASIC chips are more efficient. Now, most of us are not going to be shopping around for ASK chips. Your average consumer has no need for them. But for folks like cryptomners, AZK might make sense once you reach a certain level of profit. Right, once you reach a certain level at least potential profit if you mine a block of the cryptocurrency. Because again, Bitcoin created the perfect storm for this back when it was awarding six point two five coins per block, so that meant in an average day the system would release, or rather miners would mine around nine hundred bitcoins total per day, and with bitcoins trading at fifty grand each, that would mean around forty five million dollars worth of bitcoin were up for grabs every single day. That's what justified spending the huge amount of money it costs to develop and deploy ACC chips for the specific task of mining bitcoin. Yes, that design process is incredibly expensive, but if you could create a system that could grab a significant number of bitcoins every day, then it would pay for itself pretty darn quickly. You might not get all the bitcoins, you might not even get most of them, but as long as you were grabbing a decent number every single day, you would quickly accumulate wealth and justify the cost of using ACAC technology. That's what left GPU miners in the dust, because once acc systems joined the party, the GPUs just could not compete. It would be kind of like if you put me in the one hundred meter dash in the Olympics, the lead runner would be crossing the finish line before I managed to get a quarter of the way there. Now I should add that this year, in twenty twenty four, the number of bitcoins awarded per block dropped by half. This was all part of the plan. This wasn't a mistake or something. Now, if you mine a block, instead of getting six point twenty five coins, you end up getting three point one two five. So again, this was this was planned, and every four years or so the system cuts the number of coins awarded per block mind by fifty percent. When bitcoin first hit the scene back in early two thousand and nine, if you mined a block successfully you would net yourself fifty bitcoins per pop. But of course, back in two thousand and nine, the value of bitcoin was fractions of a cent. You wouldn't apply AASAC technology to bitcoin mining back in those days because the coins weren't really worth anything. In fact, on May twenty second, twenty ten, this is a famous date in crypto history. This was more than a year after bitcoin had launched. A cryptocurrency minor named Laslow's spent ten thousand bitcoins in order it to order ap pizza. So today that pizza would be worth more than five hundred and eighty five million dollars. And in fact, another interesting point, Bitcoin is a lot of volatility. When I started work on this episode, it was trading at fifty seven thousand dollars and now it's at more than fifty eight thousand, so the value changes pretty drastically. Anyway, getting back to the having, part of the bitcoin strategy is that there's a finite number of bitcoin that will ever be released into circulation, and once the last one is in circulation, no more new bitcoin will be minted. So specifically, that makes up twenty one million bitcoin to control the release of bitcoin into circulation. The system does this having business every four years, so today mining a block on the Bitcoin network will earn you three point one two five bitcoins, or around one hundred and eighty one thousand dollars worth of bitcoin crypto per block. Mind, these kinds of changes affect mining operations because if the magic number dips too much, then it would cost more to mine bitcoin. Then you would get out of mining it, so you would have to adjust your strategy. Right, you'd say, all right, well, now it doesn't make sense for me to operate this massive computer network of AASC machines that's drawing power directly from a formerly decommissioned power plant because the cost of operations is sky high and the amount that I'm able to actually mine is much lower. Now I have one more initialism to throw your way, but before we get to that, let's take another quick break to thank our sponsors. Okay, we're back, and we've talked about CPUs, and we've talked about GPUs, and we've talked about FPGAs talked about ASICs. Now it's time to talk about NPUs. And as a nancy, the initialism stands for neural processing unit. These have technically been around for a few years now, but the term is still fairly new. For mainstream audiences. I think you started to see them pop up in mainstream tech journals last year, but they've been around for a few years. And NPU is a chip with a specialized design meant for AI applications and artificial neural networks in particular. Now, just in case artificial neural networks, if that term is new to you, it is a network of processors that collectively mimics the way our neurons interconnect with one another in our brain meet. That's a very high level and oversimplified explanation, but it kind of gets the idea across. Artificial neural networks are often used in the field of machine learning, in which researchers train computer system to produce specific results given specific input. Now, that could be as simple as indicating which of a million different photographs are the ones that happen to have cats in them versus ones that don't have cats in them, or it could be something far more complicated, like learning which environmental factors impact the development of weather systems so that you can have a more accurate weather forecast. And NPU is tuned to work in this discipline, and often it could produce much better results than a GPU. Both an NPU and a GPU tend to be made with parallel processing in mind, and NPUs are typically incorporated onto integrated circuits that also have a CPU. They don't necessarily replace a CPU, they are in addition to one. So let's wrestle this all back to artificial intelligence. When you hear the phrase AI chip, chances are the chip question is one of four types. It's an FPGA, an ASIC, an NPU, or a GPU. Now you can have AI enabled CPUs that don't have these other components. But the problem with CPUs is that due to their unspecialized design, they have limited usefulness when it comes to AI applications, particularly as the AI field becomes more sophisticated and has greater data processing needs. It's kind of like giving a really good third year math student a challenging quiz meant for fifth year students. Our little test subject might do a decent job at the end of the day, but it will likely take them longer and cause more exertion than it would for someone who is more attuned to the task. So with CPUs, that means that you have to have longer processing times and you have to use more energy in order to be able to complete the task, and that means also generating more heat. It's less efficient, it's less money efficient as well, not just power efficient, but financially efficient. So two of the components you find on these integrated circuits are logic gates and we could just call them transistors for simplicity, and then memory. So while a CPU depends on both of these quite a bit in order to do its job, specialized chips like ASICs AASEYS can be made to emphasize the logic components more than the memory components, and they can be packed with more transistors with less space reserved for memory. That's typically what AI needs needs access to large capacity for data processing, so the goal is to allow for more data processing per unit of energy than you would get out of a typical microchip. AI is a power hungry technology. I mean that literally. Maybe one day AI will be power hungry in the figurative sense, like in like the super villain sense. Maybe that will happen one day, but right now, it's just it needs a lot of juice. So making the processing as efficient as possible is absolutely vital, Otherwise the costs of operations spiral out of control. Your energy needs as well as things like cooling needs and everything else that goes along with using a bucket load of power would make it harder for you to cover costs. This is part of the reason why you'll hear about companies spending billions of dollars on AI. It's not just that they have to spend that money for the research and development of AI, although that takes up a big part of it. It's that actually operating these data centers that are running these specialized machines takes a lot of energy, and so the cost of operation is in the billions of dollars. Now, these AI chips typically can handle parallel processing tasks in a much greater capacity than even your most powerful multi thread into multi core CPUs can. Which type of chip you use often depends upon the application you want, so, for example, Google's tensor processing Unit is an ASK chip. Google has spent a lot of time and money developing these processors and fine tuning them to handle intense data processing at incredible speed for the purposes of machine learning applications. Primarily, a lot of AI companies will use off the shelf GPUs and they will wire them together in order to train AI models, which has led to Nvidia, which for years was thought of as just a graphics processing unit design company, to now become a leading AI chip company. The boom in AI development has catapulted Nvidia to become a three trillion dollar company in recent years, so it has joined the likes of Microsoft and Apple. That's not to say Nvidia was always like an underdog or anything. It was always a company that was doing pretty darn well, but in recent years it has entered the stratospheric level evaluation. It was not a trillion dollar company that long ago. When we talk about consumer products, CPUs and NPUs are typically what will handle AI needs because they are the more cost efficient approach. Intel has developed NPUs under the code name metior Lake. Actually, to be more precise, the metior Lake chips include CPU cores. They also include a small GPU portion as well as the NPU unit, all on this same integrated circuit. And the idea is that these chips will be incorporated into machines that can run AI workloads locally. So let's say you've got a company and that company wants to host a language model, but it wants it locally. It doesn't want to be tapping into a cloud based language model, they want to run it on premises, while they might use computers with meteor Lake chips in them in order to do that processing, which would be more cost effective than building out a whole AI data center just to service this specific company. Okay, so when people talk about AI chips, they don't mean that somehow the chips are imbued with artificial intelligence. Instead, these chips are optimized to run AI applications, and those applications run the entire gamut of AI. There are AI chips used in robotics, There are AI chips used in autonomous cars. There are AI chips for large language models. Smaller chips and NPUs can be incorporated into smart devices, which allow some AI processing to happen at the device level rather than remotely through a network connection. That's really important for speeding up those processes and to eliminate latency, because for some implementations speed might not be that big a deal, but for others, like the autonomous cars I mentioned, being able to process information and produce results is critical to operate the technology safely. You cannot have latency in those systems or disaster can occur. You wouldn't want an autonomous car that constantly has to beam information up to the cloud and wait for a response, because real world driving conditions are constantly changing. They are dynamic, and they change at a very fast rate. Depending on how quickly you're driving, it could be an incredibly fast rate. So any latency would lead to catastrophic outcomes. So AI chips are important components in what you might call EDGEAI. This not only cuts down on processing time, but it can also help things remain more secure. Right, you're not beaming data to a different location all the time, you're processing it locally. That makes it less susceptible to being hacked. Not immune, but it's less susceptible. There's fewer links in the chain, you could say. So now we have our overview of what AI chips are all about. And I think it's good to remember that a processor's utility depends entirely upon what you plan to actually use it for. If you're doing standard computing stuff like you're working with documents or playing games or browsing the web, and AI chip isn't really going to mean much to you at all. AI chips tend to be really geared toward parallel processing. So it's possible that a computer with a good AI chip could be useful as a gaming rig. But honestly, I think, at least for now, going with a good GPU and a decent CPU matters more for gamers, And like I said, some cases, you might not need a really good GPU. You could have a decent GPU and a really good CPU. It all kind of depends on the types of games you want to play. I think it's important for regular old folks like me and at least some of y'all out there, to know about this stuff so that when we're shopping around for our next device, we have an understanding of the terminology. Right, we know what an AI chip is and what it's supposed to do, and whether or not it matches what we need. We aren't just pooled by marketing terms. You know. It doesn't mean that an AI chip labels slapped on something is going to mean that that's the best thing for us. So having this understanding is important. Being an informed consumer is important. It means you're going to get the best out of your money that meets your needs. Right, we only have so much money. We should make sure that when we're spending it. We're doing it on stuff that actually solves the problems we have, as opposed to just stuff that's shiny and new. I say this because a lot of tech enthusiasts tend to fall into the trap of I want the new thing because the new thing is somehow better than the old thing. That's not always the case. It often is in tech, but it's not always the case, and it certainly doesn't always justify spending the amount of money it takes to be part of that bleeding edge. It is important that we have a bleeding edge, but it's not important that we're all in it. We can hang back a bit if we need to. So I just wanted to take this chance to kind of break down this AI chip terminology and what it actually means, because goodness knows, Like when I started first seeing the terminology myself, I was confused. I was thinking, what makes an AI chip and aichip? And does it have some sort of AI capability built into it? Because how would that work? And obviously I was overthinking it. So hopefully this was useful for y'all, and I hope you're all doing well, and I'll talk to you again really soon. Tech Stuff is an iHeartRadio production. For more podcasts from iHeartRadio, visit the iHeartRadio app, Apple Podcasts, or wherever you listen to your favorite shows.