Usenet in the Workplace:
A Proposal to Streamline Business
and Improve Personal Work-Flow

(Part Two of the "Using Usenet" article series.)
by Patrick Salsbury
First draft: 11-09-00
Second draft: 02-09-02
Document URL: http://reality.sculptors.com/~salsbury/Articles/using-USENET2.html

Note: This document is still under development, and will be expanding as time permits.

Synopsis:


Introduction

In the first article of this series, we discussed Usenet from a user's perspective, including an overview of some of the basics, and a brief glimpse into some of its more advanced possibilities. This essay will explore things from a few other perspectives, including that of an advanced user, a system administrator, and a business manager with numerous people working under him or her. The intent is to explore ways in which the technology of Usenet may be employed to overcome certain problems which plague most businesses, but which are so common that most people overlook them. The hope is that we can find ways to utilize these tools to help streamline our day-to-day info-processing, and reduce the overall time we spend going through repetitive info.

The third article in this series (not yet written), is intended to be a technical how-to for site administrators who wish to implement a Usenet-based conferencing system in their environment, and begin to harness some of the benefits laid out in these first two articles.

Why Usenet? I've already got email!

Not very long ago, most people on the planet hadn't heard of email. Despite this fact, a surprising amount of work actually managed to get done. Compare this with today, and you'll see that in a very short time, email has utterly pervaded the workplace, and at the same time, has increased in volume for almost everyone. Whereas 10 years ago, it was uncommon for anyone but computer professionals to get more than a handful of messages, nowadays people will routinely get dozens, or even hundreds of messages per day. Given this fact, it's a wonder we manage to get any work done at all. :-)

This article takes the attitude that "more email" doesn't necessarily equal "more accomplished". Quite often, just the reverse is true. How often have you heard (or said) "I'm really behind in my email, and I haven't seen your note"? How many times have you asked yourself "*WHY* am I getting this message? I don't even know this person!" and then deleting it in frustration because it's wasted your time and kept you from whatever you were supposed to be doing?

Some Numbers

An article about the 2003 E-mail Survey of 1,100 US employers have put some actual (and surprising) numbers behind what we all know already: We're getting buried under the email load, and it's getting worse. Here are some exerpts from their findings:

"The average respondent spends about 107 minutes (1 hour 47 minutes) on e-mail every day...about 25% of the workday. While 24% report spending less than one hour, 31% spend more than two hours and 8% more than four hours.

76% of respondents say that they have lost time in the last year due to e-mail system problems. 35% estimate they lost only half a day, but 24% think they have lost more than two days.

[...]

Eighty-six percent of respondents agree that e-mail has made them more efficient, in spite of the fact that 92% receive spam mail at work. Fully 47% say spam constitutes more than 10% of all their e-mail; 7% report spam represents over 50% of all e-mail received."

It's interesting to note that people estimate only 1/2 to 2 days of lost time in the past year due to "system problems", yet they don't seem to tally up the 25% of the workday that's lost every day by just about every worker. Given that there are roughly 2000 hours in the standard work-year, that means that people spend (on average) roughly 500 hours, or 12.5 weeks per year going through their mailboxes. Some people are spending twice that. It adds up.

If we start with just the 10% spam figure, that's about 2.5% of the work-year...about 50 hours. But at the other end, with those claiming spam is more than 50% of their mail load, you now begin approaching 250-500 hours per year, depending on how much time they spend per day. (Hint: The people who do less than one hour of email per day aren't the ones getting hammered. They haven't been as visible on the net, so they are spared some of the brunt. Those who use mail as their daily tool to communicate with customers and the various online communities necessary for their job, or who receive the mail to addresses posted on the company website are the ones who get hammered.)

Perhaps many people don't consider that 25% of their day "lost time", just as many people don't seem to consider having unsolicited advertisements popping up on their screen to be out-of-the-ordinary. (...After all, it's always been that way on TV, right? Don't we just have to accept those ads on our browsers? And the hours lost each week sifting through and deleting spam? Isn't it simply our lot in life to come in tomorrow...and next week...and next month...just to do it all over again?)

Not necessarily...

Some Background

First off, I'll admit that I'm a chronic email-junkie, and at the same time, I'm chronically overwhelmed by email. Since joining the Internet in 1987, I've found it to be an invaluable tool, and a royal pain. I currently run close to 3 dozen mailing lists, my own domain, and am on several dozen other mailing lists that I don't run. I've been Postmaster for numerous Silicon Valley computer companies, and routinely handle hundreds of emails per day, occasionally spiking to thousands on days when things really go wrong. I am fluent (or at least conversant) in about 13 mail programs that I can think of offhand, and I use probably 6-7 of those on a regular basis, interchangeably.

I love email, and I hate it. It sucks away my time, I constantly feel like I'm drowning in it, (I currently have 2424 unseen messages waiting for me in 21 different folders as of the moment I'm writing this) and my friends and loved-ones all know better than to rely on email as a way to get my attention in a timely manner. So as you can imagine, I'm keen to find ways to improve on this situation, short of running screaming into the woods and never looking at another computer again. :-)

One thing to understand is that, with the speed of modern computers and network connections, the bottleneck in info-processing is NOT the computer, it's us! We are the slow link in the chain. And as such, we need to develop new ways of approaching our info-processing. We need a new architecture and paradigm, a different set of tools and techniques, in order to get a performance boost in our daily routines.

To do this, we're going to have to adopt a slightly different perspective about Usenet for the explorations of this article. Most people view Usenet from an end-user perspective. They see lots of spam, lots of discussion about all manner of topics, lots of unregulated (and sometimes offensive) discussion, etc. In short, they're viewing the content of Usenet, not the underlying architecture. We're going to explore the structure and technical underpinnings of Usenet, and find out how to turn it to our local advantage in the office.

An excellent place to get some background info on what Usenet is (and is not) is the "What is Usenet" article, which is also periodically posted to the news.announce.newusers, news.admin.misc, and news.answers newsgroups. (These links should connect you to the local copies of these groups on your own local news server. If they don't work, you need to configure your browser to correctly access that server. If your ISP isn't providing you with Usenet access, you should be asking yourself why...)

You can also use Google Groups to access Usenet via your web browser, but you won't have access to the rich functionality of a dedicated newsreader program. (Such as customizing your display, advanced threading features, scoring, auto-selection, auto-killing, easily canceling your old posts, etc....) See Article 1 for more details, and the Google Groups Help pages for what they do (and don't) provide. If you're interesting in trying out a dedicated newsreader program for your platform, or finding more Usenet-related resources and info than you can ever possibly read, check out http://www.newsreaders.com/. It's impressive.

Two major forms of email

There are two very broad categories for which email is generally used today: public messages, (which are discussions on mailing lists, large aliases in the office, project work groups, etc.) and private messages, (invites to lunch, confidential discussions, talk with just a couple of friends, etc.) Obviously, there are finer gradations, but you can easily categorize most email into one of these two categories, based solely on the intended recipient audience. (In fact, one of the more common gaffes that everyone seems to make at one time or another is to reply to a public forum with what they intended to be a private email.)

One of the primary arguments of this essay is that those public messages should be posted in a common "public" area of the network, and not be placed in everyone's inbox. Let's use a paper-based example: Many of today's email messages that go to a group say things like "please don't leave your coffee cups in the sink" or "we're going to be cleaning the fridge on Friday afternoon." In a paper-based office, sending a note like this would be similar to typing up a memo, making a bunch of photocopies, and walking them around to every person's desk, dropping a copy into their inbox, rather than just posting one copy on the bulletin board in the break room, or some other public place where people know to look for such announcements. This essay, and the following one, will explore how we can implement that public "bulletin board" in our electronic workspace. It will explore how to implement, and more importantly, why we'd want to do such a thing.

Regional Newsgroups

If you've used Usenet, you already know that there are a "Big Eight" hierarchies of newsgroup categories, namely: comp.*, humanities.*, misc.*, news.*, rec.*, sci.*, soc.*, and talk.*. There is also the alt.* hierarchy, which is quite widespread, but not part of the "official" top hierarchies. There are also countless other distributions, focusing on specific interests or regions, such as biz.*, ba.*, clari.*, princeton.*, sco.*, ucb.*, utexas.*, etc. It is these smaller distributions that we're going to focus on. This regional hierarchy architecture allows us to create a unique and private set of newsgroups for our own specific needs, interests, work-related projects, and team discussions. These are all the "public" messages that currently travel through email, and if we can integrate them into a new architecture, with pre-sorted topics and interest areas, we can begin to realize a performance boost in our own daily info-processing.

Email vs. Usenet - A Philosophical Difference

How many times have you gotten an email in your mailbox that made you mutter to yourself "...And just why am I getting this?!?" as you hit the "Delete" key? Most of us get some of this type of mail. Many of us get some of it every day. It might be a notice sent to everyone about someone you've never met having forgotten their pager or cell phone, and giving you 17 other ways to contact them. It might be an announcement that there are cookies on the desk of someone you've never met in an office 2500 miles away. (Don't laugh. I've gotten these.) It might be the arrival of a new baby in the family of a person you've never met in an office hundreds or thousands of miles away. It could be whatever, but the central point is that it's now in your inbox, and it's demanding your attention.

The main philosophical difference between email and Usenet is that of personal importance of the information to you. Email has an unstated assumption that everything is 100% important to you (otherwise, why would it be in your inbox?), whereas Usenet takes the position that 99.9999% of the information out there is cruft (specifically, definition 4.) Email, like our example of putting a piece of paper into everyone's inbox, demands your attention, but let's face it, not everything that lands in your inbox really concerns you. If we could manage somehow to get all of the stuff that we're not interested in out of our way, how much more time would we have to devote to things that actually do concern us?

This is where Usenet tools come in. Usenet was designed as a global conferencing system, with tens of thousands of categories, and millions of people, all merrily chattering away. At magnitudes like this, you cannot possibly keep up with the data flow. However, due to the architecture of this information, and the tools available to help you select what you want to see (and auto-prune out what you don't want to see), you can actually process this information in a fairly quick and painless fashion, pro-actively weeding out the things that you know don't concern you, and freeing up your future time to deal with things that do. (For more info on this aspect of filtering & selecting, please see Part One of this series.)

This architecture works. At various points over the past 10 years, I've kept current with 50-70 different newsgroups, a full email load, and still managed to hold down a full-time job! :-)

Benefits of a conferencing system vs. email:

Having a public conferencing system such as a local hierarchy of newsgroups vs. a simple flat email system has numerous benefits. First off, much of the public mail has been pre-sorted into categories. (And if something is inappropriately posted to the wrong group, the readers of that group are usually quick to point this out.) This makes it easy to select things that are important to you at the moment, and/or skip over the things that aren't so urgent.

Another benefit is that a conferencing system may be configured to archive all public discussions, and keep them on the server as long as desired (or even permanently), so they may be gone back to as needed. (How many times have you deleted a message by accident, only to need it later? Conversely, and much more likely, how many times have you saved a message, then never looked at it again?) Usenet-style discussions have a concept of "marking something as read", rather than out-and-out deleting. When you read a newsgroup, it shows you things you haven't seen yet. After you've seen them, they vanish, but they aren't really gone. They're all still on the server, and you can access them with a keystroke. This provides good peace-of-mind, as many people cannot easily decide whether or not to delete an email message, and some obsessively keep every single message they ever send or receive. (Even keeping email pages saying things like "Call me now at XXX-YYYY"!)

One exceedingly good example of this was a woman who worked at a former company with me. As her system administrator, I was responsible for backups, and she had close to 500 megabytes of email in her directories. Every note she had ever sent or received, for more than 5 years, all neatly arranged in lots of clearly labeled and nested directories. Every meeting announcement, every lunch invite, every forgotten cell phone & pager, every baby announcement. Obviously, the timeliness of this information was long since past, but there was so much of it, that she couldn't process it anymore to weed out the cruft. And as such, she just kept adding to it. Perhaps she still is to this day, I don't know. This is where "marking as read" would come in handy. She'd never have to worry about keeping copies. They'd just exist on the servers, and if she ever needed to go back to search for something, she could.

Another problem that can be mediated by a conferencing system is that of spam being sent to aliases. My girlfriend works at a large company in Silicon Valley. There are several aliases she's on that have upwards of 15,000 people on them. Invariably, she'll either get notes from someone across the country about a forgotten cell phone, or an actual spam that's been sent to the alias, because the name of that internal alias has leaked out of the company (or simply has a generic enough name such as "everyone@company.com" that it's easy enough for spammers to guess.) Conferencing systems not only provide easier and more flexible tools for filtering at an end-user level, but can be configured on the back-end to allow posting only from certain groups, networks, machines, etc. Email aliases generally accept postings from anyone who knows the address. Mailing list manager software can be configured to accept posts only from members of the list, but this is not always done, and many companies don't run list manager software, they merely create an alias inside their mail system.

Usenet based discussion also has the benefit of threads being auto-generated in the general course of discussion. (Some advanced email programs such as mutt and exmh also contain the ability to thread email, but many common email clients seem to lack this ability. Even ones that provide rudimentary capabilities, such as sorting messages into threads, lack the ability to perform more advanced tasks, such as killing replies to a specific message or a sub-thread, or maintaining a kill file to pro-actively kill future replies to a specific message or sub-thread.) As mentioned earlier, the discussion history is also auto-archived, so if you get pulled into a discussion at some point in the middle (suddenly getting recruited into an ongoing project, for example), you can backtrack through the previous discussions on the server to find out where things stand.

I used to work as a Network Design and Engineering Consultant for a company called Predictive Systems. At Predictive, we had a large and growing talent pool, which we drew upon as needed for different sorts of projects. These projects were of varying duration, with varying numbers of people working on them. Sometimes people would be involved in multiple projects at once. Sometimes these were one or two person jobs, sometimes they had 40-50 people working on them. In this type of environment, having project-related groups on a Usenet-style server makes a lot of sense vs. regular, flat-architecture, everything-in-one-inbox-and-filter-it-yourself email.

For one thing, you can create an extranet space which is used on a per-project basis between your employees, your customers, and any third party vendors or partners you may be working with. I worked on a project in 2000 that had probably 40-50 people all emailing each other. There was one central email alias for this project, and people from three different companies were all mailing to everyone on it. There were many different discussion threads, and certainly some of them didn't apply to the parts of the project I was working on, but I was on the alias, so I would get 20-30 off-topic messages per day, which I had to sift through, evaluate, and delete. The load this imposed and the time it took could certainly have been better spent elsewhere, and just about everyone suffers with this same problem, to a greater or lesser degree. Even when I was transferred to another project, I was kept on the original project alias, and for several months I continued to get mail from people on the old project. In an ideal system, where this was being managed on a newsgroup server, I could have just stopped reading the project newsgroup, and rejoined it later if I went back to work on that project. Sadly, this was not the case, as we were all just using email.

Another problem that kept surfacing during this time was email attachments. I would often get 6 or 7 copies of the same document, as people replied to one another, forgetting to trim the attachment. When these are 2-3Mb files, it can quickly fill up your drive. Not only was I getting multiple copies of the same doc, but people would send updates as attachments, sometimes several times per day, containing only minor edits. This proved to be a logistical nightmare, as we very quickly got swamped with multiple parallel versions of the same document, all with different edits. (This is a problem better addressed with CVS or some other version-control software, but having a central newsgroup server could certainly cut down on the amount of attachments flowing out to dozens or hundreds of individual inboxes.) We could only hope that a Usenet conferencing system would put an end to the "huge attachments going to hundreds of people" problem, but in all honesty, I think we're going to have to face that one for a while, until users learn how to apply the appropriate tool to the task at hand, and perhaps learn the wisdom of posting a URL to their document, rather than attaching the actual document each time they mail.

Further Benefits

An extranet discussion space, aside from providing project-related areas for you and your customers, can also be secured with password protection and encryption to protect your private business data (most email is still sent "in the clear", allowing potential enemies, competitors, or system crackers to snoop on it.) Different regional offices within one company can carry groups of local interest to their users, as well as the standard Company groups. Additionally, if your extranet partners run their own Usenet servers, you can cross-feed each other the specific groups of mutual interest. This will allow project members from different companies to simply subscribe to a new newsgroup on their local servers, rather than having to connect across the net to one centralized server. It removes the need to maintain mail aliases, and reduces the likelihood of forgetting to add a team-member to an email message, or of annoying a person not involved in the specific project. (Having multiple servers also builds in redundancy and makes for a more reliable overall system.) As more companies adopt this NNTP-based architecture, you will be able to build an expanding network of secure, collaborative discussion spaces with your business partners and customers, without swamping the email inboxes of people on those projects. Hopefully, this will lead to project teams that are more focused on their tasks, and more productive overall.

The discussion groups are, by their very nature and design, pre-sorted into specific topics, making them easier to parse when desired, or avoid when they're not your top priority. Usenet tools have a far more advanced set of filter/selection/kill capabilities than normal email tools. (Yes, many mail programs claim to have filtering capabilities, but how many people actually use them? I currently use 78 different procmail rules to filter my email into dozens of pre-sorted mail folders as of this writing. However, most people don't like to program filters. Despite claims of filter functionality, in practice, many mail filters simply take more time to implement that most users wish to invest. And without more advanced capabilities such as filtering on threads, author, size of message, or scoring, many email filter systems are quite labor-intensive to maintain effectively. With Usenet news readers, simple filtering and killing can be 1-2 keystrokes away, with more complex filters not far behind that.

Many existing email programs and browsers are already capable of accessing NNTP (Network News Transfer Protocol) based news servers. Programs such as exmh, Internet Explorer, Lotus Notes, Netscape Navigator, Outlook Express, Opera, and Pine have news functionality built-in. There are also NNTP-specific readers such as Forte Agent, Gnus, nn, rn, slrn, tin, and trn which have advanced feature-sets that take full advantage of news. Some of them simply present a newsgroup as though it were another mail folder, albeit with pre-sorted, topic-specific messages. Many users, when presented with such a setup, might not even know that some of these messages are coming from a different back-end architecture or server. All of the data can be presented in a uniform and familiar way.

Another feature of Usenet newsgroups is the ability to cancel messages. Many times, users of email will fire off a message that they either wish they hadn't sent (often just seconds after hitting the 'send' button), or that they realize is unnecessary upon further reading of a thread. (Thanks to Alan Murdock for raising this point.) I have often trailed behind discussion on mailing lists, and in trying to catch up, I'll be responding to articles as I go, only to find that other people posted the same information a few messages later (and several weeks earlier!) (In fact, I've now made it a habit to try and read through all of the list traffic before replying at all, in case this has very thing has happened. Unfortunately, that makes it even less likely that I'll post anything, since often I have more mail than I can catch up on in one sitting!) In those cases, it'd be nice to just cancel my messages, so they don't clutter everyone's mailboxes, but email isn't designed to do that. However, Usenet is.

It has long been a part of the Usenet design that news servers support 'cancel' messages, allowing a user to retract what they've said. Certainly, it isn't 100% guaranteed, and some machines might be incorrectly configured so as to not honor the cancel messages. Also, there have historically been "cancel message wars" on the newsgroups, where some rogue people go out forging cancel messages and canceling thousands of other people's messages in an effort to censor speech that they personally don't like. But these abuses should not eclipse the very practical uses that the cancel function enables. (Also, keep in mind that user behavior and usage patterns will differ greatly in a corporate environment, on private newsgroups, than they might on the free-wheeling, global and public newsgroups. It seems extremely unlikely that corporate staff members would engage in internal cancel wars.)

For example, one use that would be perfect (and sadly, isn't widely used, since many users don't know they can cancel messages) is in the various *.forsale newsgroups. Someone will post a sequence of messages like this:

43 Poster@Somewhere.org   Free TV! (You pick up)
44 Another@Elsewhere.com  (Different Subject)
45 Poster@Somewhere.org   TV is gone. Please don't call.

Given the short interval between the original post and the "TV is gone" post, we can guess that it didn't last long. However, we might see the above example on our news server up to a week or two later. This is a perfect time for Poster@Somewhere.org to use the cancel function and simply remove message 43, rather than posting a followup telling people not to call. Then it would save everyone time by simply not bothering them with old data. The cancel message would propagate throughout the network of news servers, and key upon the original Message ID header, removing the old message.

At this point, an illustrative example might be useful to understand how a local-office/company-wide private newsgroup hierarchy could be structured, and how it would fit into an existing intranet or extranet.

An example "office.*" hierarchy:

As you can see, the groups are arranged in a series of logical groupings, and can be as generic or as specific as desired. Some of these might not get a whole lot of traffic, and some might be extremely busy. But they follow the philosophy of "a place for everything, and everything in its place." Once you've created a space such as office.jokes, it's suddenly a lot harder for a misguided-but-well-meaning co-worker to justify sending it to office.announce.general. Similarly, by breaking down into regional groups, you simultaneously reduce the email load for everyone, as they may all now subscribe to their own regions, and not have to wade through extraneous postings of interest only to people in other regions.

The companyname.extranet.project.* groups are designed to provide a unique newsgroup name for groups designed to leave your local networks. Thus, even if every company used office.* or local.* for their internal hierarchies, (and properly configured their servers not to send these groups outside the local net), the extranet groups would be obvious to all involved parties, and won't be confused with internal newsgroups at any of the participating sites.

Cross-posting

Postings can also be cross-posted to multiple newsgroups, so if something really should go to several regions, but isn't necessarily as widespread as office.announce.general, then it can be distributed by simply adding the appropriate group names to the Newsgroups: line. News servers will store the multi-group posting in an intelligent way on the server, often using symlinks, so there's really only one copy taking up disk space, with several pointers to it from various groups. (Compare with an email server, which will happily copy the same message into hundreds of mailboxes, eating up untold amounts of disk space for one message.)

For the security-conscious admins out there, know that you can configure your news server (at least, if you're running INN) in such a way that not only won't it pass messages posted to internal groups to the outside world, but you can also configure it so that it won't pass messages that have been cross-posted to internal groups, either! This means that if a user, through malice or ineptitude, posts a message to both office.intranet.project.a and talk.politics (or some other public forum), your server will spot the illegal post and refuse to pass it outside of your domain. The local users might even see it show up in the talk.politics group, but it would never be added to the list of articles to feed elsewhere. (Check the INN newsfeeds(5) man page for details about this function and search for the "@" character.)

Most advanced newsreader software will also handle the cross-post intelligently, marking it as read the first time you see it, so that you aren't bothered with it again upon entering subsequent cross-posted groups. (Alas, the people who post spam on the newsgroups break even this simple guideline of netiquette, using software that creates a new message (and a new Message-ID:) for each copy of a spam article that they post. So if you read 20 groups of the 6000 or so that they hit, you'll likely see 20 copies of their message (unless you're actively tending to your kill file). Had they simply done a cross-post, you would have seen their message once (and only once), and might have actually found it useful, rather than infuriating.

Auto Filtering

And that mention of filtering brings us back to the whole reason we started on this journey: To lighten our daily email load. We're closing in on the goal, now, with a system that's designed to handle large data loads, and help you sift through to find the information that's important to you, personally, while filtering out the things that aren't important to you, personally. (They're important to someone else, or they wouldn't have been posted in the first place. But since they're not important to you, why should your time be sacrificed daily in wading through them and hitting the "delete" key?)

If people actually use this system the way it's intended, many previously unsorted things that all landed in your inbox will start to magically appear in their respective categories, which you can read, skim, or ignore, as you wish. Even if people don't use this system as a news front-end, and continue to send out emails to "Everyone" or "North_America" or "All-Staff" with all of the minutia of their daily work-lives, you can still be free from most of it by using a tool called a gateway (described below) and some well-crafted rules to pre-sort mail messages, sending them to appropriate groups. Then you and the other news-users at your company can get your info via the newsgroups, and remove yourself from the aliases that get filled with random blather.

Many of the most annoying messages are the ones you see over and over. And by using regular expressions to scan those messages as they pass through the gateway from the mail server to the news server, they can be pre-sorted into the appropriate groups. Messages about cleaning the fridge or the sink are pretty easy to spot, as are baby announcements, and the always-forgotten cell phones and pagers. Press Releases have a definite format to them, which can be scanned and routed to the appropriate group, as well as "work at home" or "out sick" messages that get sent to a much wider audience than necessary.

Migration Strategy

Assuming for the moment that you're now sold on the idea of off-loading most of your generic office email to a Usenet server, and getting back the glorious days of only having a few messages to deal with per day, you must be asking yourself: "How do we get from here to there?" The technical implementation details will be tacked in Part Three of this article series (not yet written, alas.) But there are certain considerations which should be brought up early in the planning stages, whether you're a system administrator hoping to implement this system in your network and office environment, or whether you're a manager who wants to roll this out to your staff.

Resistance

Let's be honest. You're going to get resistance on this one from some people, somewhere in the company, no matter who you are. People develop habits, get set in their ways, and many don't want to bother learning something new. I've been in many different work environments and no matter whether a change comes from On High or from the grass roots, there are always some who won't like it, won't use it, and will complain mightily every chance they get. And in this case, that's OK.

One thing to keep in mind is that we're developing an extension to the existing communication system, not a replacement. This is where the Mail-to-News gateways come in, for backwards compatibility with existing email servers. When properly implemented, you can have messages passing seamlessly back and forth between email and news systems automatically, allowing each user to choose to operate where they feel most comfortable, and most effective. (And this may also provide an avenue for managers who have staff that are lagging in productivity. Many people are simply swamped with their info-load, yet don't know any way to effectively deal with it, short of ignoring some messages. This system might provide a retraining/migration path as well as some new tools for staff who are truly floundering and feeling their productivity suffer.)

Also remember that many mail clients, as mentioned earlier, already know how to interface with Usenet servers, and present the messages in a uniform manner that is consistent with the user's email format. Once again, it's not the technology that is a limiting factor, but the users, who simply need to add some new tools to their toolbox.

Gateways

Essentially, a gateway in the Usenet context is a program that massages the headers of a message in order to port it from one system to another. There are two common scripts used on *nix-style systems, which change the headers depending on which direction they're sending the message. These two are called mail2news and news2mail. With these, and some well-crafted filters, you can have both mail and news systems working side-by-side, and not force anyone out of their preferred info-environment.

(From Letter to Mike D'Agosta - 05-12-03

Actually, check out this article I wrote a while back (I need to finish it, actually...) It explains some of my thoughts about email vs. newsgroups, and how we (as admins) could implement the systems to complement each other and return people's email box to them as something relatively personal and easy to manage.

The problem as I see it is that most people nowadays (even sysadmins) don't know what the whole newsgroup framework is for, or what it's good at. It was supplanted by the web, and then people who didn't necessarily understand newsgroups wrote bad web interfaces to and implementations of it, which made it even harder for newbies to use effectively or appreciate. DejaNews sucked. Google's news archive, while impressive, is a pain to use for very long. And have you ever tried to follow a thread in Slashdot? It's incredibly tedious, and most threads die out after just a few posts. No one can realistically follow the hundreds of replies, yet everyone posts some, and there are many short, disjointed threads (seldom more than a dozen replies to any single thread), rather than good discussion on topics that develops over days and weeks.

In fact, Slashdot *prevents* further replies after a few days, quashing any long-term discussion on a topic! Why is that? Just because the interface makes it difficult to find/follow topics more than a few hours/days old? Usenet was designed when reply latency might be up to a week, so people took time to write good posts, and the thread might be followed for days, weeks, or even months.

Instead, we seem to find the Slasdot community eerily mirroring behaviors in the electronic world that Neal Stephenson so aptly described of people in the physical world in "Snow Crash":

"All these beefy Caucasians with guns! Get enough of them together, looking for the America they always believed they'd grow up in, and they glom together like overcooked rice, form integral, starchy little units. With their power tools, portable generators, weapons, four-wheel-drive vehicles, and personal computers, they are like beavers hyped up on crystal meth, manic engineers without a blueprint, chewing through the wilderness, building things and abandoning them, altering the flow of mighty rivers and then moving on because the place ain't what it used to be." --Neal Stephenson, "Snow Crash", p. 274
When you look at Slasdot, it appears to be a very well organized and tightly-knit community. But in reality, none of us know many people on slashdot, and indeed, the default posting identity there is "Anonymous Coward" if you decide not to register with them or log in to your account. Compare this with Usenet newsgroups, where generally are required to post under their own username. Thus, there is some accountability, and consequently, people care about their online reputations (at least, in theory.) As a result of this, people build persistent online communities over the course of years. People joke with each other, have long and passionate debates about politics, music, etc. Sadly, Slashdot seems rather hollow in comparison. There are lots of short little posts (often one-liners shorter than the writer's .sig file) which can be tagged as "funny", "informative", "insightful", "flamebait", etc. There are usually less than 6 replies to any post. Rarely do you see 12-20 replies on any thread, whereas newsgroups routinely have dozens, and sometimes hundreds of replies in one thread.

To be continued...


Bio: Patrick Salsbury lives in a dome in the mountains near Silicon Valley. He has held various "Newsmaster" and "Postmaster" positions at various companies, and is sorely acquainted with the problems of info-overload. His resume can be found here.

Last modified: Saturday, 11-Aug-2007 20:09:11 PDT