Episode 8: Mag Computer
Saul and Mike climb the mag scale of computing, using RAM as a proxy for compute power, from the Apollo guidance computer and the Apple II up to today’s machines, and what each mag of memory made possible.
Listen to full episode:
[This episode sponsored by the Recurse Center.]
A Mag History of RAM (1960–2025)
RAM is a good proxy for computing power. Here’s every mag of memory from the Apollo Guidance Computer up to today’s machines — Nintendo, PlayStation, iPhone, and more. Remember, this is a log scale. (full write-up)
Transcript
[0.8s] Saul: Hello and welcome to Mag World, where we like to ask big questions, make wild guesstimates, and quantify literally everything on a magnitude scale. Just remember that every order of magnitude is a big deal, because as we say at Mag World, quantity has a quality all of its own. If you wanna learn more about Mag World, come visit our website at magworld.pw. I’m Saul Pwanson, your guide to Mag World, and I’m here with my friend Mike, who’s used a computer.
[27.5s] Mike: Yes, and I’m curious, are we going to use this one here during the episode? Should I turn it on?
[31.8s] Saul: Yes, please do.
[32.5s] Mike: All right.
[47.8s] Saul: So today’s episode is Mag Computer. The good old Apple II booting up. Um, actually, we’re gonna need to be online for this. Could you log us onto the internet, Mike?
[01:01.1] Mike: Uh, yes, yeah. All right, give it a couple minutes, but then we’ll be there.
[01:17.1] Saul: It’s not that long. Okay, here we are. We’re online now. So computers are one of the topics that inspired Mag World in the first place. I mean, I’ve loved computers basically since I was born, and each magnitude of computer that I’ve experienced has had a quality all of its own.
[01:35.2] Mike: Will you kind of explain to me what these different mag levels of computer are?
[01:42.0] Saul: Yes, great question. So I use RAM as a proxy for compute power. Gordon Moore, you’ve heard of Moore’s Law?
[01:51.1] Mike: Absolutely. It’s something about the doubling time for processing power, something like that.
[01:57.0] Saul: It’s actually about the density of transistors, that you’ll have m- twice as many transistors on an integrated circuit, and th- th- that doubling time being, I think, 18 months initially. And this was initially specified in 1965 and then revised in 1975 to be every two years instead. But it’s held true, or it did hold true, for some 40 years, and it might be flattening now, I guess we’ll still see. But the fact that it held true for so long is this exponential rise of compute. And because it’s about transistors, you need transistors both for computing power and for memory, and they kind of go hand in hand. So as your compute power goes up, so does your memory, and it turns out that you, you need more memory also to do more complicated things. Like, even if you’ve got the, the fastest possible computer but a very little amount of RAM, you can’t do very much with it, no matter how powerful it is. And so, yeah, I use RAM as a proxy for compute power, and order of magnitude RAM. So a mag 4 computer would be an Apple II that had 32 or 64 kilobytes of RAM, and then a mag 5 computer would have some hundreds of kilobytes, up to the classic 640K of RAM thing, and et cetera.
[03:17.8] Mike: Okay, great. So we’re using RAM as this proxy, uh, for computing.
[03:23.7] Saul: Mm-hmm.
[03:24.4] Mike: And RAM is an active memory rather than a storage memory, is-
[03:30.1] Saul: Yes
[03:30.3] Mike: … my understanding.
[03:31.4] Saul: Good point, yeah. So the iPhone is a mag 9 computer in my, in this nomenclature. It has a gigabyte of RAM, maybe four or eight gigabytes of RAM. It turns out the most recent ones, uh, are, like, now, I think, 12 gigabytes of RAM. But they have, uh, not only that spec of RAM, the working memory like you’re talking about, they also have the storage, which is like your, your hard disk or your USB drive or something, right? And that’s more like the, the paper that you’re writing down as opposed to the, the brain memory that you have, the working memory that you have in your-
[04:03.2] Mike: Mm-hmm
[04:03.8] Saul: … mind.
[04:04.9] Mike: Okay, great.
[04:06.5] Saul: So I went to the Recurse Center a couple weeks ago.
[04:08.9] Mike: Oh, what’s that?
[04:10.0] Saul: The Recurse Center is a retreat for programmers. People go there who are interested in computing and computing technology, and they spend somewhere from, like, six to 12 weeks being in this community of like-minded people who are curious and interested about computers and becoming a better programmer. And I went through this program actually back in 2017, and it’s really a remarkable community, and I wanted to say that this is why we met.
[04:38.0] Mike: Tell me.
[04:38.6] Saul: I did this program in 2017, and I was so excited about having a community. I was so inspired that when I came back to Seattle, I intentionally sought out a community-oriented co-working space, which was Office Nomads.
[04:55.0] Mike: Office Nomads. That was great.
[04:55.9] Saul: Yeah.
[04:56.2] Mike: This was-
[04:56.3] Saul: Where w- I, and you were the first person I think I even met there. You gave me a tour of the whole space.
[05:00.5] Mike: I remember when you came in, you were super excited to find a place.
[05:05.2] Saul: I mean, there were other co-working spaces, but Office Nomads was kind of special. It really had the community flavor, and that’s the same thing about the Recurse Center. There are coding boot camps, but the Recurse Center is very community-focused.
[05:18.5] Mike: So more like a summer camp for adults than a summer school for adults.
[05:23.9] Saul: That’s a great way of putting it. Exactly. Like, very much in their ethos is that you should drive your own education throughout, but not only at the Recurse Center and in school, but throughout life. And so they don’t have any curriculum. They don’t have teachers there.
[05:39.4] Mike: Mm-hmm.
[05:39.5] Saul: You need to be figuring out what you want to learn and then seek out the resources to help you learn that thing, and that’s one of the foundational, like, meta sk- uh, skills of learning, and they tout that very actively. So I’m bringing them up because they are our sponsor this episode.
[05:57.4] Mike: That’s wonderful.
[05:58.5] Saul: I will give them a very heartfelt endorsement. I mean, they’ve kind of changed my life. And so any of our listeners, if you’re interested in becoming a better programmer, I think the Recurse Center is actually a pretty fantastic place to spend a few weeks in New York City trying to become that. So I went to this alumni week at the Recurse Center, and I met some very interesting people there, as I always do, and I asked them about their first computer experiences. And so this is a little bit of a montage of what we got.
[06:30.8] Recurser: I was working for AT&T in 1986. I was programming firmware on an 8051 microcontroller in assembly language that had 128 bytes of RAM, which I guess is a mag 2 computer. We burned EEPROMs. We burned PROMs. And, like, so the whole process of software development was very different. It was, like, very design doc heavy, um, testing, like, really intense code reviews, and, like, a very formal QA because once you put it out there, there was no way to fix a problem. Uh, my first computer was an Apple IIe, which was a mag 4 computer, and downloading these ROMs, uh, that people would distribute over the internet, um, which meant connecting to our dial-up modem. Uh, you could only get, like, a few minutes on the internet at a time, of course, ’cause it took up the phone line, downloading these ROMs and then playing them in emulators, uh, with friends. When I went to college, this was in 1997, I was a computer science major, but I didn’t have my own computer. Like, none of us had computers in our dorm rooms. And so in order to do all of our assignments, we would go to a computer cluster on campus, and the computer clusters came in three different flavors. The most sort of formative experiences were, like, the nights before big assignments would be due, when you would find yourself in the computer lab at, you know, 3:00, 4:00 in the morning, and you knew that everybody who was in there with you was working on those same assignments, and that was, like, a real feeling of like, “Oh, we’re really in this together.” So as a child, uh, there I saw this button that said turbo on my computer, and I always, I never knew what it was, so I, I just assumed that it made the computer go fast when you pushed the turbo button. Uh, and surely if you could make your computer go fast, you’d want to do it all the time, so there must be a reason why the button exists. And I assumed that it was because it would wear the computer out, that, like, the computer couldn’t handle running at turbo speed, uh, at all times. Um, and I later learned it was because many of the games that were made for earlier computers depended on a fixed, uh, computer clock cycle. My first computer was a mag 6 computer, an IBM 386 SX 20.
[08:52.7] Saul: Awesome.
[08:53.0] Recurser: With two megs of RAM. There, there was actually a really, uh, very active, uh, BBS scene in the area at the time. They would meet at a, a local Chinese restaurant every Saturday. There was people of all ages of life in there, too. Like, somebody figured out that I liked to program, and then I became the guy who would, like, add mods for the different BBSes. My first computer was a Windows 95, which was a mag 7 computer. I used it to play point-and-click adventure games like, uh, Disney’s Aladdin and Hercules. What ended up requiring us to get to the next computer level was that I bought World of Warcraft, and it required Windows XP as the minimum performance. My first computer was a mag 8 computer. I think that my parents were very against social media and Facebook and smartphones initially, and even gaming consoles, so the first console I even had must have been when I was 16, was a GameCube.
[09:51.5] Saul: So we managed to run the gamut from a mag 4 computer, um, the first person was talking about the Apple IIe that they had, and then as younger generations use more advanced computers, and the most recent person here said they were on a mag 8 computer with, uh, hundreds of megabytes of RAM. And now, of course, we are at the modern computers are mag 10. They have a dozen, well, 16 gigabytes, 32 gigabytes. That’s mag 10 computing. And so we have seen over the c- well, over the course of my life anyway, eight orders of magnitude of increase in computing power. So I wanted to talk about my first computer. My father had a computer back in the ni- 1980. He got one of the very first, uh, computers. It was a TRS-80. I think that his TRS-80 was Model 1. It had four kilobytes of RAM. That’s mag 3.
[10:49.7] Mike: That sounds very small.
[10:51.2] Saul: Very small, and you couldn’t do very much with it, and everything you could do with it was, uh, pretty slow. So one of the memories that I actually have is playing an old, I can’t even call it a video game. It’s a text adventure game, you know-
[11:05.1] Mike: Mm-hmm
[11:05.1] Saul: … where you, um, you read the text, and it’s like a choose your own adventure, but you can type the actual, you know, open door kind of stuff.
[11:10.7] Mike: Oh, yeah. L- like Myst or Hitchhiker’s Guide.
[11:14.1] Saul: Hitchhiker’s Guide for sure. Myst is, uh, famously all graphical.
[11:17.4] Mike: Okay.
[11:17.6] Saul: We would load this text adventure game off of a cassette tape. Yeah, we’d put the, a literal, a cassette tape player, and you’d press play, and hopefully it wasn’t, like the volume was turned down so you didn’t hear all the goosh, goosh, goosh noises, whatever, and then it would load the entire thing into memory, and that’s how you loaded software in there. If it wasn’t for that, you’d be typing stuff in, and then when you turn, turn the computer off, everything you typed in was gone. So looking at the whole scale of mag computer, we’ve got my first computer was a mag 3. That was that TRS-80, and there, there are computers that are lower than that, but it’s basically they’re calculators, if that. And then your first computer, the Mac, was a mag 6 computer. And then mag 9 is an iPhone, and mag 12 we haven’t hit yet. That would be a terabyte of RAM. So yeah, Moore’s Law, we’ve been We’ve seen this exponential rise, and it has been relentless. In the early ’80s, and even in the later ’80s, it was every five years, we basically got a 10X improvement, and then in the ’90s, it started to slow down a little bit. I think it was every six years, and then by the 2000s, it was every seven years. And now, I, I bought a computer 10 years ago, and I’m still kind of using it, this laptop. It has s- 16 gigs of RAM, and it’s kinda fine.
[12:32.3] Mike: Mm-hmm.
[12:32.4] Saul: And so it’s been 10 years, and, you know, it’s starting to feel like I do want a little bit more memory, but the idea of having 128 gigs of memory in my main computer is actually… I mean, that’s, that’s a lot of memory that I don’t need.
[12:44.4] Mike: Because I feel like when I was younger, it was always getting up to the edge of usability.
[12:50.1] Saul: Yeah. You could feel it. You knew it. Your computer was kinda getting kinda slow. It couldn’t do the things it needed to do.
[12:55.9] Mike: Mm-hmm.
[12:56.0] Saul: And if only you got that, spent this extra 1,000 bucks or whatever now, now you could get the, the better stuff, right?
[13:01.4] Mike: Exactly. But, you know, that was as music first, as audio-
[13:07.7] Saul: Mm
[13:08.1] Mike: … became-
[13:08.9] Saul: Mm-hmm
[13:09.0] Mike: … a thing that you would store and play through your computer.
[13:12.0] Saul: Mm-hmm.
[13:12.1] Mike: Or, and then it was video, and then streaming, and now, for a decade, it’s been fine.
[13:19.6] Saul: Mm-hmm.
[13:20.3] Mike: Like, I don’t need anything more than what we had 10 years ago in terms of being able to c- consume media.
[13:28.6] Saul: And now that’s interesting, though. A very interesting point because as consumers, we don’t need more, but the p- the companies that are doing this, whether, whether they’re Netflix actually providing the streaming, or of course, these days, AI, those additional computing requirements are in the cloud. You know, they have to be somewhere else. They are still being developed, and there are still m- bigger things going on, but as a consumer, you don’t need that stuff.
[13:54.7] Mike: Mm-hmm.
[13:55.4] Saul: So you mentioned the audio being a, a prime thing that you were doing early on with computers.
[14:01.7] Mike: Yeah, late, when I was in my late teens, that’s when-
[14:04.6] Saul: Mm-hmm
[14:05.0] Mike: … I started being able to trade music.
[14:09.2] Saul: A mag 6 computer has a CD in it, and you can play the songs off of the CD, and then mag 7 computers allow you to encode the songs into MP3 format, which is, like, a 10X reduction in the space. And because it’s so much smaller, you can trade those, and they become these individual files that are actually kind of reasonable to store somewhere else and trade.
[14:29.9] Mike: Mm-hmm.
[14:30.9] Saul: And then you mentioned video, and video, you can’t do much video at mag 7, but starting around mag 8, you start getting some tradable video. And then around mag 9, which is the later 2000s, you start getting YouTube, which is at least short-form online streaming video. And then by the time we get to mag 10, now it’s streaming video kind of all the time, and not just streaming video, but high-quality streaming video.
[14:55.5] Mike: So can you give me a little bit of a perspective on what your first computer, as a mag 3, what is that much information?
[15:07.7] Saul: So that’s a great question. A mag 3 computer has, what, kilobytes? So that’s, like, a page of text, a couple pages of text maybe.
[15:20.7] Mike: Wow, so a double-sided print would be maxing out.
[15:24.5] Saul: Yeah. And in fact, the very first personal computer, the, the kit that was developed, the Altair, was actually a mag 2 computer. It had, uh, hundreds of bytes of memory, but it could be expanded to 2K or 4K of RAM, and Microsoft famously developed software. It’s Microsoft BASIC for that computer, but it, that, BASIC couldn’t run on the mag 2-
[15:52.3] Mike: Mm-hmm
[15:52.5] Saul: … versions of that. You had to buy the expansion kit in order to run Microsoft BASIC, and then even then, you had, like, some, like, I think 700 bytes left over for your own BASIC programs that you could use, and that was it.
[16:04.7] Mike: So going up from that, how far do we go before we get images on a computer?
[16:11.4] Saul: It’s about mag 4 when you start getting any kind of graphics that resemble a picture.
[16:16.4] Mike: Mm-hmm.
[16:16.7] Saul: But usually, the, the, of course, at mag 4, you have a handful of colors, like black and white or cyan, magenta. Some pretty loud colors.
[16:28.3] Mike: Mm-hmm.
[16:28.7] Saul: And then at mag 5, you are starting to get a little bit richer images, and then at mag 6, you are starting to go more true color. Like, it’s, “Oh, okay, now we have at least 256 colors.” Doom was famously a mag 6 game in the early ’90s-
[16:43.9] Mike: Hmm. Mm-hmm
[16:44.8] Saul: … and, and Doom is, I mean, you know, it’s, it’s very pixel art, but it’s very representational. So images, I think it’s around mag 6 where they start to become lifelike, and it’s the same thing with music, actually, where they had music at mag 4, but it’s the chiptune kinda stuff, right, where it’s definitely very generated.
[16:59.7] Mike: Mm-hmm. Beep, beep, beep, beep, beep, beep, beep.
[17:02.9] Saul: Yeah, nice rendition. Exactly, yeah.
[17:05.2] Mike: Many hours.
[17:05.8] Saul: And then you get MIDI in mag 5, so you’ve got at least different instruments. It’s kinda like now it sounds like a synth, right? But then mag 6, you start getting CD-quality music, where you can actually play samples, and it’s legitimate music.
[17:19.3] Mike: So it’s been 30 years since we’ve had that.
[17:23.2] Saul: Yeah. Yep.
[17:24.7] Mike: Going from chiptunes to 4K streaming video.
[17:30.3] Saul: Yeah, 30 years. That’s a long time, and not that long at all.
[17:35.9] Mike: Yeah, exactly, ’cause 30 years before that, it was barely color TV.
[17:41.6] Saul: So if you look at 30 years before 1995, or 1996, I guess, it was 1966, and this is the mainframe era, where they had computers. They had plenty of big computers, but they had no small computers. All of the computers were, they took up whole rooms. They had hundreds of kilobytes of memory, but, um, they were all kind of individually wired. Like, they didn’t have the integrated circuit yet. The Gordon Moore quote is from 1965, so they had just invented that transistor concept.
[18:09.8] Mike: ’Cause I remember seeing pictures of the Apollo-
[18:14.4] Saul: Mm-hmm
[18:15.0] Mike: … computer-
[18:16.0] Saul: Mm-hmm
[18:16.2] Mike: … and them saying, like, “These are hand-wound memory units.”
[18:20.2] Saul: Yeah.
[18:21.0] Mike: We’re seeing, you know, this museum piece
[18:23.5] Saul: Mm-hmm
[18:23.8] Mike: I didn’t even know that that was-
[18:25.4] Saul: It’s possible
[18:26.1] Mike: … a thing. Like I thought it went straight from cards to silicon, but
[18:31.5] Saul: Yeah, no, there was this whole period where, well, first it was vacuum tubes in the-
[18:35.2] Mike: Mm-hmm
[18:35.4] Saul: … ’40s, right? And then they did s- a little bit more solid state stuff, but it was individual transistors, like you had, you made a little transistor and then you’d wire it up. And the idea behind the chip, the, it’s called an integrated circuit. It’s not just a single transistor, but a bunch of transistors all laid out on a single chip that interact. And then once you do that, now of course, it’s off to the races. And, you know, if you have hundreds, you can do some interesting things like in a calculator. But if you have 10,000, now you’ve got an actual CPU like you can use in a, in a computer, and now we’ve got billions of transistors on a single chip. And I gotta say, it’s unfathomable how complex these modern chips are. Like you can look at an- one of those early computer processors from the late ’70s, and you can… I mean, they literally did understand everything. They taped them all out by hand. They made the little photo mask that they did to make the etching, but they did it large scale and then it got focused down. And so there’s thousands of transistors on there, and like they were all meticulously placed. And these days, first of all, the layout is done by computers themselves, so a very, people are involved, and, I mean, you, you can’t actually place a billion transistors on a chip. For that matter, a lot of the things are cut and paste, where it’s like this thing, you know, 16 times so you have 16 cores. But more importantly, you don’t get a thing that makes a simple image on a thing. The process of making these chips is several miracles stacked on each other. There is a company called ASML that makes the fabs, and they are the only company in the world that can make these fabs that make modern chips, and they cost about $400 million a piece. There are several videos online that you can find that are about this process. It is nuts what they have to do in order to get things this small. They have the, the node size for the, the transistor. It, you know, started off as so many micrometers.
[20:38.9] Mike: Mm-hmm.
[20:39.5] Saul: And now it’s down to-
[20:41.1] Mike: Sorry
[20:41.1] Saul: … micrometers.
[20:42.1] Mike: I’m sorry, no, I’m in mag world. I don’t think I understand.
[20:44.5] Saul: Well, thanks for calling me out on that, Mike. So yeah, a micrometer is mag negative six meters.
[20:50.4] Mike: It’s very small.
[20:51.1] Saul: Very small. And that was where they started, and actually I think it was tens of micrometers, so mag negative five. And now the leading edge is down to mag negative nine, three nanometers. And just for context, individual atoms are mag negative 10.
[21:10.5] Mike: So currently, the designs on an integrated circuit are only an order of magnitude larger than the atoms-
[21:20.7] Saul: Yes
[21:20.8] Mike: … that make up-
[21:22.7] Saul: Yep. Yeah
[21:23.6] Mike: … that design?
[21:24.4] Saul: Yep.
[21:25.0] Mike: That’s incredible.
[21:26.0] Saul: And so what they have to do in order to get that on there, because the wavelengths of light even it turns out are bigger than that, so they can’t actually even do the stuff with the traditional… Yeah.
[21:35.9] Mike: Well, because a wavelength of light is-
[21:38.1] Saul: What defines how the stuff is etched even
[21:39.9] Mike: … negative seven?
[21:42.0] Saul: It’s on the order-
[21:42.6] Mike: Mm-hmm
[21:42.6] Saul: … of hundreds of nanometers, so mag negative seven, right. Totally. And so they wind up like using these complicated mirrors, which have to be in a very specific position, that then wind up shooting laser beams off of them, hitting off droplets of tin to scatter in certain directions, and it, everything is so precisely timed and con- and controlled and so microscopic. Like it really is not just one, but several miraculous feats of engineering that go into these things, and no single person understands the entirety of the process, not the everything that’s on a chip, nor how to make a chip. Like it is, it is impossible for one person to know. It’s just that complicated.
[22:24.2] Mike: But as someone who wrote software, at what point did you stop understanding the entirety of what you were making?
[22:32.6] Saul: That’s a great question. So in my experience, a single person can hold some tens of thousands of lines of code in their head at one time. I’ve written personally at least tens of thousands, probably hundreds of thousands of lines of code, but I don’t remember all of them, and if you pointed me at something I wrote 15 years ago, I’d be like, “Oh, I don’t, I don’t know. I have to re- revisit this,” right?
[22:54.4] Mike: So what does tens of thousands of lines of code represent to someone like me who doesn’t interact with the code but only sees what’s on front? What about if I’m using my word processing?
[23:07.0] Saul: Mm-hmm.
[23:07.9] Mike: How big is that?
[23:09.2] Saul: So, uh, it depends which word processor. There were word processors back in the day, in the mag 5 computer era, that were necessarily only tens of thousands of lines of code.
[23:19.3] Mike: Mm-hmm.
[23:20.1] Saul: But, uh, Microsoft Word is millions these days. It’s basically any large application that has hundreds of people working on it is millions of lines of code.
[23:30.7] Mike: Mm-hmm.
[23:31.2] Saul: And, uh, an operating system these days anyway is gonna be tens of millions of lines of code. And like if you look at your car, for instance-
[23:38.5] Mike: Mm-hmm
[23:38.8] Saul: … actually has hundreds of millions of lines of code in it, but most of the, that code is Linux and the media center. And there’s actually a huge amount of code that is actually running on your, uh, car in order to make it go, to do the things it has to do, fuel injection, everything else, and those, that’s an embedded software. Then that is also a huge amount, but like I said, most of the software in there, it’s not like the car manufacturer is writing hundreds of millions of lines of code. They can’t. They suck in this code from somewhere else-
[24:04.5] Mike: Mm-hmm
[24:04.5] Saul: … and use that.
[24:05.2] Mike: So the more specialized the code, the smaller it’s going to be, but the more flexible it has to be, the larger it would?
[24:12.7] Saul: Yeah.
[24:12.8] Mike: Like the fuel injector does one thing, on a variable basis perhaps-
[24:18.0] Saul: Mm-hmm
[24:18.0] Mike: … but it’s, it’s got one job, so they can make it Small, but my media center’s gotta play-
[24:24.4] Saul: Mm-hmm
[24:24.9] Mike: … a whole bunch of things at different-
[24:26.5] Saul: Different formats.
[24:27.4] Mike: Mm-hmm.
[24:27.9] Saul: 100%. And actually, one of my personal heroes of software is a guy named Chuck Moore, who wrote a computer language-
[24:36.3] Mike: Different Moore?
[24:37.3] Saul: A different Moore, yes. So we have Gordon Moore-
[24:40.5] Mike: Give me Moore
[24:41.2] Saul: … of Moore’s Law, and we have Chuck Moore. And Chuck Moore invented a programming language called Forth in the ’60s, and it is a remarkable programming language. We’ve already mentioned BASIC, and of course, people have heard of Python and C. But, um, Forth is special because it was written at a time when computers had almost no memory, and in fact, it was written to drive telescopes that were run by a very small microprocessor. And so he wrote a higher level language, and nobody else was doing this, on a very small platform, and it could do amazing things. He could climb this ladder of abstraction very quickly in order to get things done. And he’s, he has written later in life, and he’s actually still alive, I think he’s 85 or so now, but he wrote something that was quite profound. It was that, “I can make the amount of software required to do a particular thing 10 times or even 100 times less, but the general principle eludes me.” Basically, he himself-
[25:42.5] Mike: Mm
[25:42.5] Saul: … could say that if you given a problem, he could whittle it down and be like, “This is the little bit of code that can do this thing.”
[25:49.8] Mike: Mm-hmm.
[25:50.2] Saul: But, and he can’t codify this and say, “This is how you do it, everybody.” But the way that, and when I’ve looked at what he’s done, and I see the same kind of thing in myself, how you do it is you strip away all that flexibility. You make it flexible for yourself and your own use case. You don’t need to support millions of people doing the same, well, same and slightly different things. You support just what you’re doing, and then yeah, you don’t need to do a lot of things. You do the fuel injection, and then you’re done. And it’s a very, like, old school engineering approach, but it’s actually very reasonable, right? And it’s because software is so actually expensive to make that we’ve made this thing, well, it’s better to take this one thing that already works and stitch a little bit more on it, a little bit more, until finally now it’s millions of lines of code, which is how we get here. So I’ve been playing with Claude Code recently, and other AI tools to write code, and it’s remarkable what’s happened in the last, I don’t know, six months or something like that, in terms of them being able to diagnose problems and suggest fixes.
[26:49.0] Mike: How big is Claude?
[26:50.9] Saul: Uh, well, okay, this is, this is, uh, there’s multiple answers to this. So Claude, the, when you talk about the actual AI itself, right?
[27:01.4] Mike: Mm-hmm.
[27:01.4] Saul: AIs aren’t in lines of code anymore. I mean, they’re, they’re not. They’re neural networks. It’s giant matrix, matrices.
[27:08.7] Mike: Mm-hmm.
[27:09.1] Saul: So just huge stacks of numbers that have been trained to act certain ways based on certain inputs. I mean, you can’t compare lines of code to numbers in a matrix, but they are on the order of tens of billions or hundreds of billions, so mag 10 parameters-
[27:26.6] Mike: Mm-hmm
[27:27.1] Saul: … or mag 11 parameters, and they’re talking about a trillion parameters now, that’d be mag 12, and that’s, and the, the idea is that if you give these AI neural networks more parameters, then you’ll get more intelligence.
[27:42.8] Mike: Mm-hmm.
[27:42.9] Saul: So that’s the actual AI brain. And then there’s a thing that I’ve been using, it’s, you know, Claude Code. It’s like there’s the harness around the brain. The brain itself just kind of is like the oracle, uh, where it’s just kind of like speaking in tongues, and sometimes it’ll actually will even get into speaking in tongues. It’s kind of weird. But then you have a harness around it that says, “Oh, well, if it s- if it says this, that means I’m gonna take this action, and then if it says this, then it’s like, I’ll take this action.” And it turns out that Claude Code is at least hundreds of thousands of lines of code itself. The, the, the harness is hundreds of thousands of lines of code, and I think one estimate was up almost to a million lines of code.
[28:20.1] Mike: Mm-hmm.
[28:20.7] Saul: Now, what’s interesting about this is that that’s not actually a million lines of code that humans have written. They famously have vibe coded Claude Code itself. So that’s just hundreds of thousands of AI-generated code that people have scarcely looked at.
[28:36.2] Mike: Wow.
[28:36.7] Saul: Yeah. It’s kind of a trash heap, and it kinda works.
[28:40.1] Mike: Mm-hmm. Well, so now, now that we’re talking about these, these small programs that you can make for your friends to use on their what, Mag, mag 8 computers? Mag-
[28:50.4] Saul: Mag, no, mag 10 computers.
[28:52.2] Mike: Mag, mag 10 computers.
[28:53.0] Saul: Mm-hmm.
[28:53.3] Mike: Okay. Is my phone a mag 10?
[28:54.8] Saul: An iPhone is mag 9.
[28:56.5] Mike: Okay.
[28:56.9] Saul: Most of them.
[28:57.9] Mike: I understand the use of a mag 9.
[28:59.5] Saul: Mm-hmm.
[29:00.6] Mike: I understand the use, you know, all the way down to when I was a kid. So what is industrial computing? Like, how many orders of magnitude is Claude running on compared to what I’m running at home?
[29:14.0] Saul: So back in the day, in the ’70s, ’80s, even into the ’90s, supercomputers were specialized devices-
[29:21.5] Mike: Mm-hmm
[29:21.8] Saul: … where they had their… I mean, everything was custom. They were trying to go as big and as different and interesting as possible. And then around the late ’90s, it turned out that the most efficient thing to do was to take commodity hardware and scale it up. And so since then, supercomputers have been, like I said, those, and you have some connectivity between the individual blades in the thing, but they’re all, it’s all commodity hardware from there. The Toy Story render farm was several hundred computers-
[29:50.9] Mike: Mm-hmm
[29:51.0] Saul: … that were running all at once, and you can do maybe 1,000 or maybe a couple of thousand computers in a huge cluster, but that’s about the size. So mag 3 of commodity hardware. And so if our commodity hardware now is mag 11, and conceivably you could get up to mag 12, you can buy an instance on Amazon that is, or rent an, an instance for an hour or whatever, that is mag 12. That’s a terabyte of RAM.
[30:18.3] Mike: Mm-hmm.
[30:18.9] Saul: And then you scale that up mag 3 to-
[30:22.0] Mike: 12 plus three is 15. Mag 15.
[30:24.4] Saul: Mag 15-
[30:25.6] Mike: For A modern supercomputer.
[30:28.8] Saul: Yes.
[30:30.0] Mike: And so it sounds like supercomputer compared to home computing has, has it always been this mag 3 about 1,000 times more powerful?
[30:39.1] Saul: Mag 3 to mag 4 more powerful, yes.
[30:41.1] Mike: Mm-hmm.
[30:41.9] Saul: That’s for a supercomputer. That’s when you’re doing these really complicated nuclear blast simulations, weather simulations.
[30:48.3] Mike: This is where we get genetic-
[30:50.8] Saul: Yes
[30:51.4] Mike: … decoding from.
[30:52.3] Saul: A lot of that from, stuff that, I think a lot of that stuff is coming down, but yes, exactly. That stuff’s happening there, too. And so you asked about Claude. What’s Claude running on? It’s not consumer hardware, but they, it is still commodity hardware, so it’s on the order of, I think, the individual GPUs, graphics processing units, they used to play video games, now they’re doing AI, are 50 to $100,000 a piece, so not within any consumer budget, but they’ve got maybe a terabyte of RAM on them. And then Claude runs on that or a few of those stitched together in a single chassis, and so we’re basically on the order of mag 12, possibly into mag 13, just barely, for what Claude runs on. And then when we talk about data centers that AI is running on, it’s that but times 10,000 computers in the data center. And so now it’s not that the individual, that the Claude is running on-
[31:49.8] Mike: All a data center isn’t working on a single problem-
[31:52.3] Saul: Correct
[31:52.5] Mike: … like a supercomputer might.
[31:54.2] Saul: Exactly. Exactly. So you’ve got this, but magnified out, but it’s about sharing the resources, the, uh, energy and water cooling and stuff like that.
[32:04.7] Mike: So a data center represents a mag 16, a mag 17 level-
[32:10.0] Saul: Yep
[32:10.4] Mike: … computing-
[32:11.3] Saul: Mm-hmm
[32:12.5] Mike: … in one location.
[32:13.6] Saul: In a single location, exactly. And again, that’s not tracking a single problem.
[32:18.4] Mike: Mm-hmm.
[32:18.8] Saul: That is perhaps millions of people even using that in a timeshare kind of sense.
[32:24.0] Mike: Mm-hmm. That’s huge.
[32:25.1] Saul: That mag 17 is a large number.
[32:27.2] Mike: And if history is any guide, that’s going to just keep increasing.
[32:31.9] Saul: It’s true. Well, I wanted to say that the website magworld.pw has a lot of these things on there, ha- about mag computer in terms of the levels of compute, the different eras that we’ve gone through, and this little chart of memory. So I would encourage people to check out the website.
[32:51.7] Mike: Mag World.pw.
[32:53.4] Saul: Absolutely. So before we go today, I wanted to talk about, we have mentioned in the past, I don’t think it’s actually made it into an episode yet, but I mentioned making, maybe vibe coding a game, Maggle, on our website. And I’m, I got a lot of things to do, and it’s not my, been my top priority. But-
[33:12.8] Mike: Unlike naming the game
[33:14.0] Saul: … that is the most top priority. I mean, that’s the fun part, right? But I did, a friend of mine sent me a game recently. It’s called Magnitudle. And it is basically, like, what I was going to do in mag, on Mag World for Maggle, with the only exception being that you hit enter things in, in mag notation. But otherwise, it’s, it’s a Fermi estimation game. It’s how many football fields can fit into France or, I don’t know, whatever, things like that.
[33:39.1] Mike: For real?
[33:39.7] Saul: For real. Magnitudle.com. Yeah, you can go play now. So I, I need to reach out to them and, uh, see what they’re up to and see if we can maybe do some trade or something. But yeah, Magnitudle. Anyway, so the other thing I wanted to talk about was I have just been reading a book called Project Hail Mary by Andy Weir. Have you heard of this?
[34:02.1] Mike: Yes. I read that. Uh, my daughter breezed through it because she wanted to finish in time to watch the premiere of the movie.
[34:11.0] Saul: As I’m reading this book, though, there’s some math in the book. I mean, it’s not very complicated math, but it’s still, like, you’re doing some, some physics. It’s some, some hard sci-fi, really.
[34:19.1] Mike: It’s merely rocket science.
[34:20.8] Saul: Right. And but I’m finding that because I’ve been doing this stuff with Mag World, that things are just kind of falling into place. I’m doing some fact-checking in my head as I’m reading the book. Like, he talks about how much energy is in this one gram sample of astrophage, and I do the math, and I’m like, “Yep, your number is correct.” Anyway, I think that we’ll talk about this next time. We’ll do mag energy next time.
[34:43.9] Mike: Great.
[34:45.3] Saul: So-
[34:45.4] Mike: Well, I’m looking forward to doing that math.
[34:48.2] Saul: Yeah. Awesome, Mike. And so, yeah, that’s all we have for our episode today. Um, in closing, I’m going to lead out with some music. Let’s call it music. This is from a Polish orchestra made out of old computer parts, and so this is called the Floppotron. And yeah, this is what it used to sound like, although usually it was only one thing at a time. And thank you to the Recurse Center for sponsoring this episode, and I will say, though, that as much as this thanks is to them, I have an order of magnitude more thanks for introducing me to their wonderful community and ultimately to you, Mike. Thank you for being here.
[35:27.7] Mike: Thank you, Saul. Thank you, RC.
Show Notes
On Mag World:
Mag Computer ↑0–↑6 — the pre-internet levels, from the human (↑0) to the mid-90s PC (↑6)
Mag Computer ↑7–↑12 — the networked levels, from dialup CGI (↑7) up to the supercomputer (↑12+)
A Mag History of RAM (1960–2025) — an interactive chart of memory across Nintendo, PlayStation, iPhone, the Apollo Guidance Computer, and more
A Mag History of RAM (1960–2025)
Mag Latency - Mag numbers for latencies every software engineer should know
Referenced in the episode:
The Recurse Center — this episode’s sponsor; spend 6 or 12 weeks programming at the edge of your abilities with a community of kind, motivated peers, with batches offered in New York City or remotely
Moore’s Law — Gordon Moore’s 1965 observation, revised to a two-year doubling in 1975
Apollo Guidance Computer — ↑3 RAM in hand-wound core-rope memory; it got them to the moon
Altair 8800 (1975) — the ↑2 kit computer that ran the first Microsoft BASIC
Doom (1993) — the ↑6 computer game where 3D rendering started to look realistic
Doom)
Chuck Moore and Forth — the language written to drive telescopes on almost no memory; “I can make the amount of software required to do a particular thing ten times or even a hundred times less, but the general principle eludes me”
ASML and EUV lithography — the only fabs that can etch ↑-9m (3 nm) features, using tin droplets and laser-timed mirrors
Magnitudle — the mag-estimation guessing game mentioned
The Floppotron by Paweł Zadrożniak — the outro “music,” played on an orchestra of old computer parts. Here’s “Tainted Love”: