Before the World Wide Web, savvy computer users were flocking to USENET to participate in discussions on everything from the latest advance in computing to the worst jokes you could imagine. USENET is still around today. So what the heck is it?
Welcome to tech Stuff, a production from iHeartRadio. Hey Thearin, Welcome to tech Stuff. I'm your host, Jonathan Strickland. I'm an executive producer with iHeartRadio. And how the tech are you today? I thought we'd talked about something that's both a throwback and it's still very much current, which is the networked communication system called us net. While I think just about everyone is familiar with certain Internet related services, you know, like email or web browsing, maybe to a lesser extent, things like FTP, that sort of thingt I think depends on the person. Some of you may have only heard or seen references to UNT, some of you may never have heard of it at all, And of course I think some of you are probably very familiar with it. In fact, from what I understand you, SET is actually kind of experiencing some lot of a renaissance among younger users like millennial and gen Z users. I would not know. I'm gen X, so I'm out of the loop everywhere. We're not even included in those articles that are about all the different generations and how they differ from each other. Gen X is always left out of that, so I don't know what's going on with my own generation. But I'm getting off track. So usenet is older than the Web. When Tim berners Lee introduced the technology that would underlie the Worldwide Web in the early nineties, Usenet had already been a thing for more than a decade at that point. So our story begins in North Carolina, primarily at Duke University. You had a couple of graduate students at Duke named Jim Ellis and Tom Truscott, and they were experimenting. They were creating a protocol. And remember, a protocol is a set of rules, a set of instructions that a computer follows, and the protocols define how computers do certain tasks. They were working on one called Unix to Unix Copy Protocol or UUCP. Now, you also had a student at the University of North Carolina Chapel Hill named Stephen Bellevian who would create software that would leverage this protocol and allow people to interface with it. He would become very important in the creation of us neet. In fact, also he ended up writing several articles about the history of us net that were incredibly useful when I was putting this piece together. He wrote them for a circleid dot com, so you can go there and read. I think it's a nine part series where he goes into detail, sometimes great technical detail, which I will not be doing for this episode. I think it goes beyond the scope of what I want to do. Plus I'd be more likely to mess things up than to save them correctly. Anyway, this protocol made it possible for different computers to exchange files and messages if they were networked to one another. And by computers I'm really talking about like mainframes and mini computers at this point, primarily mini computers. Now, a mini computer is not the same thing as a desktop or a laptop. Those would be micro computers. A mini computer was still a big old honkin computer system, like back in the seventies. So a mainframe could potentially take up an entire room or sometimes even an entire floor of a building. That's how big mainframe computers could be. A mini computer might be as small quote unquote as the size of an entire desk or maybe a couple of refrigerators. These machines often had pretty significant computing power for the time, but they were not nearly as powerful as mainframes, or did they have the capacity to process as much information per unit of time as a mainframe could. They were, however, marginally not marginally, they were significantly less expensive than mainframes, so they became a core component of computer science departments of various you know, schools that were specifically centered around technology. One of those mini computers was the PDP eleven from DC and it could run the Unix operating system. So you know, there's lots of different opring systems out there. Unix is one of the big ones, often used in things like like web servers and things of that nature. Unix and Linux, whereas you know, we're mostly familiar with either Mac up Brain system or Windows, depending on the type of machine you use. Anyway, the mini computer PDP eleven was the sort of computer system that graduate students had access to. This this was before many computers would be powerful enough to do anything significant in the computing world. Like they were great for hobbyists. They had lots of useful tasks, but they were extremely limited. They would not be useful in a research capacity for the most part. So the students also had access to dial up modems, which at that time ran at a blistering three hundred bps that stands for bits per second, so the three hundred bits per second is actually not blistering. That was just a joke. Also, at this point, hardware displays were really rare. Like computer minors, you just didn't really have them in a lot of these systems. So for a lot of these mini computers, you would not get your results displayed on a screen. Instead, it would print on a sheet of paper. So you would make some code and then you would get a print out of results and that would be when you'd find out whether or not your code was good. If it wasn't, it had to go back and fix it and then print out again. It was pretty wild stuff back in those days. Well, it gets even better than that. The original use net network relied on acoustic coupler modems. So this is the kind of modem that, as the name implies, uses sound acoustics to make connections between computers. So each computer would have a modem that would have a coupler, and the coupler would hold a handset of a landline telephone, so you would take the telephone out of its cradle. You would put the telephone into the coupler the right way, so that the microphones on one side, the speakers on the other. You would then dial the number of the computer system that you were hoping to connect to, which also would have a handset and a coupler, and then your computer would make noises indicating either a zero or a one, so bits in other words, through the coupler, and these noise would travel over the phone lines, just as voice communication would if two people were just talking on the phone. So the motive on the other end would receive these noises and the coupler would convert them into voltage signals that the computer would then interpret as bits as zeros are wants, and you could have communication between computers this way. Now, the reason why the students went with this specific method wasn't because that's the only technology they had access to. Instead, it was because they were trying to get around some pretty tough regulatory hurdles. Back in those days, AT and T essentially had a monopoly over phone communications within the United States, so this approach helped sidestep some obstacles they would otherwise have faced when trying to connect the systems together. Right because you're doing a voice call rather than a data call, at least as far as the telephone system is concerned. It's just sending signals of sound across You're not using it as a data connection per se. And so it was a way to kind of sidestep the very steep prices you would have to pay in order to have a data connection. So, by the way, AT and T tried desperately to fight that and to make it so that they could still charge enormous amounts of money for those connections, but ultimately they lost that particular court battle. Anyway, Stephen Bellvan wrote software to create a sort of client, kind of like a web browser. It was a client that would serve as the interface for use net and the UUCP protocol, which I know, I just was repetitive. It's like saying ATM machine. And also the computer systems and modems were all doing the heavy lifting behind the scenes and stalling the software on different mini computers made it possible to exchange information and files between these mini computers INI, not A and Y. In fact, there were not very many computers connected to use net at all initially, so they called it use net, which is short for Users Network. The first three computers connected to the network were in Duke University Duke Medical School and the University of North Carolina Chapel Hill. Obviously it would grow from there. But we're gonna have to really sit down and explain what use net is and a bit about how it's organized, to really get an understanding of what makes it special. So if you're at all familiar with Reddit, the description of use net is going to sound a little bit familiar to you on a surface level. So the use net network consists of discussion groups. These are called newsgroups. These would be a lot like subreddits on Reddit. So each newsgroup is dedicated to a specific subject or topic. Users can read and at least in most cases post to newsgroups. There are limitations on that. I'll talk about that in a bit. And you can subscribe to specific newsgroups so that you can keep up with the new stuff that's posted to them. At least you can do that. Now, that's not how it worked initially, because originally the client that Stephen Bellvan programmed was limited to updating the information based on the last time you access the newsgroup according to the computer's logs, and it would give you all the updates, so you couldn't just jump around and grab just the stuff you were interested in in the one newsgroup that you were Let's say that you are just absolutely fascinated with Dudgeons and dragons, and that's the only newsgroup you wanted to be part of, so that was all you were focusing on. Well, with Bellevan's client, you would get everything, not just that newsgroup. The reason for this is that Bellvan felt that usenet wouldn't get that much traffic. He thought maybe it would generate an article or two per day, people would write a piece for use net, or maybe you would get two pieces in a day, and so it'd be a very small update each day. So there's no reason to worry about it because he was you know, at the time, we were talking about three computers, and again there were many computers. They were ones that you had to have access to as a student to be able to even use in the first place, so by by definition, it was limited. Plus this is a you know, a newsgroup you would only have access to if you had booked time with that mini computer, which was a process. Right, you had to access it, and that required you to sign up, and you know, sometimes you had a log book and everything so that you could have access to it so that other students would know that at that time you were accessing the computer system. Anyway, we're gonna take a quick break. When we come back, I'll talk a little bit more about some interesting comparison's contrasts with Reddit. Okay, we're so. Reddit tends to be a place where users share links to interesting stuff to a relevant subreddit reddit, and then conversation will flow from there. So a fairly typical Reddit post might have a headline associated with a news story, and you could click through and that will actually take you to the news story itself, or you can click into the messages and read the conversation around the news item. But I mean, you could just have a Reddit post that just is an original piece, a user generated piece. Maybe it includes a photo or a gift or something like that. But that's typically the way reddits work, especially in the subreds I go to. Those are almost always about sharing articles and links to interesting stuff. Well, us neet was just originally limited to plain old text. There was no Worldwide Web yet, so you couldn't hyperlink to a web page. Web pages didn't exist initially, you couldn't upload a file to use net Folks did see that would be really useful if you could do that, if you could create a way to upload and download files to and from us net, but it would require a bit of work in order to make this possible. As such, students started to work on a way to allow for the uploading and downloading of binary files. So binary refers to basic computer language. You know, it's zeros and ones. Those zeros and ones can represent anything, you know. We're talking everything from software to orchestral symphonies to the dancing baby gif that was all the rage on the internet thanks to Ali mcbeil man. I'm dating myself with this episode. So these smarty pants students came up with a way to encode binary data into text because that's what us net could handle. It could handle text. So you would take your program. You would then have to convert your program into binary code. You would have to convert the binary code into text and then upload that. Someone else could then grab that thread of posts and then use a decoder to decode the text back into binary, and then finally convert the binary into whatever the original file format was. So there are a lot of steps, but it would work. But there was a catch because used that articles have a limit as to how much data they can hold. So typically uploading a file means that the file is going to be split across multiple posts, and by multiple I mean potentially lots of posts, because if you're trying to download something significantly large, like today, if you were trying to download a high resolution video, you could be talking about thousands of posts that make up this video. So part of the issue was that early encoding methods would inflate the file size significantly, by like forty percent, so the file would get even larger because of the encoding method Now, fortunately, folks would come up with alternatives and they would find ways to encode the binary into texts that would reduce that overhead. Also, keep in mind that when us net was a new thing, the folks who were using it were relying on those dial up modems. They had really hefty limitations and how much data they could download per second, so we weren't typically talking about big files at least in the early days of use net. Anyway, the support for binary files meant that us neet would become a way to communicate not just through text, but through files as well. This is as you might imagine that led to some issues with piracy, but well that's a matter for another episode, so we're just going to move on and keep on talking about us net. So a fundamental feature of us net is that it's not a centralized service. This is another way we can contrast it with Reddit. While Reddit has some surface level similarities to us net, beneath the hood things are very different. So Reddit is essentially centralized. It means that the service ultimately runs on servers that Reddit either owns or leases from, like a cloud competing provider such as Amazon Web Services. But the point is the underlying foundation for Reddit is in a centralized group of servers. Centralized in this case doesn't necessarily mean geographically by the way. That's because networking gets all whibbly wobbly, but it does mean that's in this definite, finite number of servers, and that is where Reddit lives. Use net, on the other hand, is decentralized. Anyone could create a use net server. You'd have to download the right software, but you could do it. So the way use networks is more like a peer to peer network. So let's say you create a use net server. You connect your server to the use net network at large, You indicate which news groups in particular you want to prioritize on your server, and your server will regularly ping other servers, the bigger nodes in use Net's network to look for updates to those newsgroups and thus pull in new information whenever there is new information, and then your users on your server can access that. Now, let's say that your users on your server create a newsgroup that gets a lot of popularity, and eventually other servers on the us net network are interested in that newsgroup, so they start pigging your server and they get updates on your user's newsgroup and they serve it to their users. So the whole network is decentralized and to an extent asynchronous. You might access a newsgroup and read an article and some of the posts beneath it, and then you decide you want to write your own response, So you create your own response and you send it to post. Now that's going to go to whatever server you're connected to through usnet, But that doesn't automatically populate across every use net server that carries that newsgroup. That you have to wait for them all to sort of synchronize, which means that they're there's going to be a time where some versions of that news group are going to have your reply, some versions are not, and it means that other people might reply. They might even make the same point you made. If you're unlucky, they might make it better than you did, and then you just seem to be, you know, repeating what someone else said. But when you wrote your post, you had that that reply didn't exist, or at least you hadn't seen it yet because yours had not synchronized with the most recent information. So it gets a little bit like muddy when you're talking about, you know, what's the latest information on us net. Well, it really depends upon which server you're talking about and whether or not that was the point where new material was generated or if in fact it's waiting to be propagated across the entire network. Now. One benefit of this approach is that us net is somewhat shielded against censorship and interference because there's no one to shut down it's distributed right now. That doesn't mean that an authority cannot try, and even in extreme cases, sometimes succeed to take stuff off us net. But it ain't easy. It is a pretty tough thing to do because there's not any one entity you can go to like you can't. You could go to Reddit and demand Reddit remove something, which you may or may not work, depending on the nature of the thing, but you can't really do that with us net because again, it's not one single entity that controls use net. It's it's a hive mind. The structure of newsgroups actually follows hierarchies, and that is like large categorizations. Right today, you could argue the most famous use net hierarchy is the ALT hierarchy, which is funny because the ALT in this case stands for alternative, specifically an alternative to the big seven newsgroup hierarchies that are seen as the more establish but whatever, that's not how everything started. There were no seven hierarchies when use net first started. In fact, in the beginning, there were only three hierarchies of newsgroups, and all that was posted should fall under one of the three, and it was adequate. So the three hierarchies were net, FA, FA, and mod mod, So all newsgroups started with one of those designations. So you might have one that says like net dot discussion that would be a newsgroup. The net hierarchy was pretty much the catch all. It started as a hierarchy for newsgroups that were focusing on conversations about the network itself, but that rapidly expanded to non network topics, so you would get people creating like net dot politics or net dot jokes. The FA hierarchy was an acronym for from arpinnet. It got started after a student at the University of California at Berkeley named Mark Horton joined usenet in nineteen eighty and started to add material from an arpinet mailing list to the network. Access to Arpinet was really limited. It was very difficult to get access, so what Horton was doing was kind of a service, was providing other people the opportunity to read these mailing lists, when otherwise they would not have any access to those mailing lists. So arpinnet was sort of a predecessor to the Internet. It was the network where computer scientists and engineers were developing the rules that would allow for Internet communication. So some people say, like, it's the grandfather of the Internet if you think of it that way. I think that's a little oversimplification, but it kind of gets the idea across. So Horton's contributions would lead to a new hierarchy dedicated to sharing information from Arbennet mailing lists, the FA hierarchy, And originally that was a read only hierarchy, so stuff that was posted to that newsgroup you could read, but you could not contribute to it unless you were one of the authorized entities that could post from mailing lists to that hierarchy. Then, as for the mod hierarchy, that was for newsgroups that needed to be a bit more controlled and buttoned up, because it was pretty clear that as more students started to use Usenet, the tendency to do goofy stuff like contributing off topic posts to a thread or intentionally trying to start flame wars on use net. That's really where that got started. These things would, you know, derail conversations or sometimes completely pull focus from whatever the topic at discussion was supposed to be. So the mod hierarchy was moderated. That's what MOD stood for. It was a moderated hierarchy. Every post under the mod hierarchy would have to be approved by a moderator before it was posted to. A newsgroup is often used for stuff like official news and official announcements, that kind of stuff where you didn't want the Hoi poloi to start mucking everything up and confusing the message. So those were your three You had your NET, your FA, and your MOD and things would carry along that way until we would get to the mid eighties. I'll explain more, but first let's take another quick break to thank our sponsors. So we're up to the mid eighties. So remember us NET kind of got start in seventy nine, really got going in nineteen eighty. We get up to around nineteen eighty six and into nineteen eighty seven, and a few things happened that would shake things up for us NET in a really significant way. For one, graduate students Phil Lapsley of the University of California, Berkeley and Brian Cantor of University of California, San Diego wrote a Request for Comments or RFC in nineteen eighty six in which they described the specification for a new set of rules called the Network News Transfer Protocol or in INTP. SO from nineteen seventy nine to nineteen eighty six, the primary way that you know, people were accessing use net was through these time shared mini computer systems, So you didn't really have access unless you had booked that time with the mini computer, and as I said, that had really limited things. NNTP made it more possible to access us net using a microcomputer a personal computer, because by the mid eighties these were now a thing that were capable enough to be able to do something like that. So you would be able to access read and post to use neet groups by connectting your mini computer to whatever use net server you happened to be able to access. So NNTP is what made that all possible, and that would greatly extend the access for use net, which meant also that volume the posts on us net would increase dramatically as a result. This then led to the Great Renaming, and yes that's what it's referenced as the Great Renaming. It gets a bit dramatic, but as I mentioned earlier, Steve Bellevan, the guy who created the first use net client, imagine that us net would see maybe one to two new articles posted there per day, but us net was generating way more than that by the mid nineteen eighties, and the creation of n NTP would complicate matters because access to us net would grow substantially, and so it was becoming a bit of a mess, right, You just had these three hierarchies that were trying to handle everything, and the really the only differenterentiation between the hierarchies was that the net hierarchy was essentially the one where anyone could post content. The FA or FAW hierarchy was the one that was just for posting content from mailing lists, and the mod was for moderated posts, and that was it. Everything had to fall into those, but it made it really hard to organize and find stuff, like if you were searching for something specific, it was hard to find. Now, the use net network is decentralized, as I mentioned, but Stephen Bellivan explained in those incredible posts that I talked about on circleid dot com. They're well worth checking out. Some of the nodes in this network held more sway than others. They saw more traffic, and so the administrators for those use net servers had a bit more power than someone who's running a server that gets very little traffic at all. And those nodes, along with some of the folks who actually were responsible for building use net, initially they became known as the backbone cabal. This became like a long standing joke in usenet circles about whether or not this cabal existed or didn't. But really, what it just meant was that collectively they had the power to decide what would or would not get wide distribution across all of use Net. So let's say that there's some sort of extraordinary circumstance that comes up and these administrators collectively decide that those newsgroups are bad news, like it's just it's dangerous or harmful or whatever you may think. And it would have to be extraordinary for this to happen, but it could happen. So collectively they decide that they're not going to propagate that newsgroup. Well, that would limit the circulation of that newsgroup. It wouldn't prevent it from existing. It would still exist on whatever home server it was posted to. It just wouldn't propagate to other use Net servers, or at least not the big ones, which means means it would limit the number of people who actually saw and could interact with that post. So effectively, it could be kind of like censoring that newsgroup. But again, this didn't happen that frequently. It's just it was possible because of the way usenet works. So this cabal debated and discussed creating a new group of hierarchies that thet newsgroups could inhabit to make things a little more manageable. This in itself became a very long and detailed debate because there are so many different ways you can classify information. If you don't believe me, ask a librarian, Ask a librarian about things like taxonomies and finding out like how do you determine where does a book belong in? What category should you put any one book? It gets really complicated and subjective, So, for example, how would you classify video games? Should you put it in like recreation or hobbies? Or does it belong in technology or in business or in programming. It kind of depends upon the point of view, So some decisions would appear mostly arbitrary because when you get down to it, they kind of were. So the new big seven hierarchies that were created in this great renaming were COMP for computer news, which is self explanatory, REC for recreation, CIG for science sci, SoC for social sec, TALK for general discussion, and MISK for miscellaneous. These were not the only hierarchies, but these were the big ones, the ones that had the most newsgroups under them. Then you had alt which became the alternative hierarchy for newsgroups that didn't really fit into the other ones, or that were not being accepted into the other hierarchies. At least initially, it was possible for a newsgroup to migrate to a different hierarchy over enough time, but yeah, it was it would make the organization a little wonky. So it again extraordinary circumstances. The Big Seven in the nineties would depending on whom you ask, would become the Big Eight because there was a hierarchy called Humanities that joined the party that some people would say kind of belongs with the Big Seven. But honestly, there are hundreds of hierarchies. Some of them are also pretty big, like biz biz. That's a hierarchy dedicated to business newsgroups. It's a pretty happening newsgroup or hierarchy, i should say. So there are lots of them out there. Some of them just get very specific to things like education, that kind of stuff. And so numerous use net groups formed. Some of them flourished, some of them failed. Some would become the backbone to future web content. IMDb dot Com famously started off as a use neet group before it became the Internet Movie database, and I'm sure no one thought that it was going to grow into something that would later get scooped up by Amazon when it was still part of a use net group. And usnet is still going on strong today. There are news groups that are dead, but that's because they were focusing on topics that are themselves obsolete. Today it would be very strange to find a lively discussion about like five and a quarter inch disk drives. Who the heck is using them? Besides like very niche hobbyists, use net is very different other types of Internet activity as well. Use net does not track user behaviors, So that's one major difference between using us net versus the web. When you're on the web, everyone wants to track you, right, There's so many different ways to track user behaviors and user web history, and that information becomes valuable. That is the currency of the Internet. That's what's being bought and sold by data brokers. That's the stuff that is valuable about you. You don't get to realize that value. You are the product you are being bought and sold. And that's just the way it is when you're using the web. Use net though it doesn't track user behavior, so you can subscribe to whichever use net groups you want. The use net server isn't paying attention to who is doing what, and so there's a lot more privacy in that regard as to you know what you are actually consuming on use net versus on the web. There are lots of use net clients out there. There are a lot of newsreaders out there that you can use. Some of them are built into web browsers. There are also a lot of use net service providers out there, some of which have subscriptions that you have to pay in order to access them, but it means that you can access that particular provider and get advantages that you would over say a smaller provider, like a really big provider will have features that you won't find with smaller providers. One of the things that net servers have is a retention policy. That's how long they will hold a file before deleting it. So you know, if you find one that has a retention policy of a thousand days or more, that's pretty good. I think I saw one that had more than five thousand days retention policy, and that means you'll have access to the most files that you can, because ultimately, as time passes, they will get deleted off use net. So depending on which provider you're using to connect to use net, you will have access to certain user generated content. If you're on a different provider, they might delete files more frequently, you're not going to get access to them. There's also like limitations to things like how much you can pull down per month for some providers. Some of them offer unlimited plans that kind of thing. So there's a lot of variety when it comes to the actual providers out there. But yeah, that's kind of the lowdown on us net. What's interesting to me is that, again I have read things that suggested that millennials and Gen Z users are kind of migrating over to us net more and more, which is news to me. I think that's really interesting, and especially since like I associate use net with an old method of accessing the Internet. And for the record, I had access to use net a little bit in my youth and my college years and everything, but truth be told, I kind of went from BBS's bulletin board systems, which are you know, that's where you're logging into a local computer or a single computer not a network, and you're getting you're posting and retrieving information to that compute, and you're not networked into a larger system. I went from that to telnet, and from tel net I went to the World Wide Web. So I didn't use us net that much when I was in college, which was in the early nineties, so this was mostly me learning about it because I had only had limited access to usnet. I did hear about it quite a bit, like you would always hear about the various alt news groups that were catering to some rather varied interests. I will say, but I never frequented them because I was more too busy tell netting into chat rooms and chatting with people in other colleges just for the heck of it. So that's it. That's our little overview of what us net is. I almost said was, but it really is still a thing. I hope you are all well, and I will talk to you again really soon. Tech Stuff is an iHeartRadio production. For more podcasts from iHeartRadio, visit the iHeartRadio app, Apple Podcasts, or wherever you listen to your favorite shows.