Killfiles And You

(or, how to read Usenet in 5 minutes a group)

What's a Killfile?

Long ago there were a few hundred Usenet newsgroups, and if some of them had a dozen new articles a day, it was considered heavy traffic. Newsreader programs would show you, for each group to which you were subscribed, every single article in order. (Or out of order, in case of really primitive ones.)

Nowadays, your site may receive 20,000 newsgroups, many of which get 1,000 articles a day. Because of that volume, reading everything, even in a single group, is no longer possible. Fortunately, modern newsreader programs will show you the list of the 1,000 "Subject:" lines and let you choose which ones to read. Unfortunately, that means you still have to read 1,000 "Subject:" lines. And because 90% of that is crap (Sturgeon's Law), that means you'll have to read a few hundred irrelevant subject lines.

A killfile is a list of what you don't ever want to consider even thinking about reading. A killfile is a bozo filter. It makes invisible any person, place, or thing you don't want to see. In other words, you can tell the newsreader you never want to see the phrase "MAKE BIG MONEY FAST" again, and those articles will never again appear on your screen. If you can think up the right list of words or phrases, you can trim out a lot of the crap.

Not only can you eliminate articles with bad words in their "Subject:" lines, you can block out things from specific people, sites, newsgroups, or concoct even more interesting formulae. (Anything with a certain number of lines; anything crossposted in certain ways; anything which is a followup; etc.) Internally, what happens is the newsreader program marks those articles as "already been read", just like the old stuff it doesn't show you because you saw it yesterday.

The point of this article is to show some of the fun things you can do with killfiles to make Usenet much quicker, and less tedious, to read.

Communist Tactics

There are two basic strategies for filtering any sort of communication (email, Usenet, etc.): the American method and the Russian method. (I didn't make up these terms.) American is "anything goes, except: ..." and Russian is "Nothing is allowed, except: ..."

In an environment where 90% of the stuff is crap, you'll probably want to use the Russian method. Modern newsreaders, which show the list of subject lines so that you can choose which articles to read, make this simple. It will show the list of articles; you'll click on the ones that look appealing; then once you've seen them, you can make all the others go away. ("catchup" or "select all, mark as read") So you're going out of your way to pick the good articles.

Besides "killing" things that are bad, you can also have the newsreader highlight things which are good. Depending on the newsreader, you can label their subject lines in pretty colors and/or move their subject lines to the top of the list. These operations are performed by filters called "selectfiles" or "scorefiles". (Your newsreader may call all these guys "killfiles" or just "filters". The terminology isn't important.) Basically, if you take out the bad stuff, and highlight the good stuff, you can be sure to read all the good stuff, then read as much of the unselected stuff as you feel like (before your lunch hour ends). So you get three levels of priority: good (labelled/selected), unknown (everything that didn't match a killfile or selectfile), and bad (killed). If the newsreader sorts things in that order, you can start reading at the top, and surrender anywhere in the middle, and you'll have seen all the good articles.

A killfile has highlighted some things and moved them to the top, and killed others at the bottom.
So, I'd read the colored part and maybe some of the black if I had time.
(It's set to show the killed articles in gray so I can check on what it's doing.
Picture simulated: I'd never actually kill John F. Winston.)

Free Clue Here

Are you using Netscape Navigator to read Usenet? Stop right now! Get thee to my page of useful links and look for a real Usenet newsreader. Web browsers aren't good newsreaders (and vice versa). As of this writing, Netscape Navigator has no support for killfiles. Trust me on this; switch to something that does. You'll soon revel in the feeling of power you get when you can press a key to make people you don't like disappear forever!

And if you're still using one of those online services that make you use their crummy killfile-less software, consider switching to a regular Internet Service Provider (ISP) to get what's called a PPP or SLIP connection: this will allow you to use your choice of Usenet, mail, and Web tools!

How Killfiles Work

There are many different newsreader programs in use today, and they all differ in how they store their list of what to kill or select. There are two basic kinds of newsreaders, distinguished by their user interface: ones designed to communicate with you in text, and ones with graphical ("click here") user interfaces.

Text-based newsreaders, like the UNIX standbys trn and nn, use text files with lists of commands. (For instance, "/bozo@bozoville.com/f:j" will junk articles from Bozo. By the way, if you're using rn, consider trn or strn; trn is rn plus threading, strn is trn plus scorefiles. They have the same commands as rn, plus new features.) You can put the commands into the killfile by typing them in, and there are usually also some commands you can press in the newsreader program to automatically kill things. (In trn, press shift-K to kill the current subject line in the current group; press control-K to edit the current group's killfile manually.)

Graphical newsreaders, such as Agent, Gravity, YA-NewsWatcher, etc. typically have easier ways of dealing with this: you can build a killfile entry by just filling in some blanks in a statement like "[Kill/Select] all articles whose [Subject/From/etc.] header [contains/doesn't contain] the phrase _________." This is as powerful and flexible as the old-fashioned way, but considerably easier for most people to use. These programs usually have menu or keyboard commands for "Kill this author" and "Kill this subject" that will automatically make the most common killfile entries.

Setting a graphical newsreader to kill golden showers and select articles by John Baez.
This one has some snazzy options, like ranking items numerically and
having these commands evaporate after a while to purge the old ones.

In both cases, you should be able to have the killfile (or selectfile) apply to articles in a specific newsgroup or to all groups (the "global killfile".) Some newsreaders also let you operate on a cluster of groups, so you could have one killfile used for all of "comp.sys.*" instead of trying to make separate identical killfiles for each of those groups. Something bad which shows up in many newsgroups ("MAKE MONEY FAST") can be put into your global killfile to make sure you kill it wherever it pops up, but you should put group-specific things into killfiles which operate only on those groups, because if you were to have all your commands in the global killfile, soon you'd have a long delay before opening each group (while it processes all those commands.)

You will also have options you can set somewhere. They include such things as whether to match capitals/lowercase (so you could kill "MONEY" but not "money") and whether to match a complete word or any part of a word (so you could kill "Mac" but not "Macarena".) You can have it kill/select the entire thread or just one article when it finds an article with a keyword. In a colorful graphical newsreader, you can usually pick colors (labels) for things that match the entries in your killfile/selectfile. You can sometimes even control what happens to killed articles: Do they just disappear forever? Do they go to the bottom of the list? Do they get displayed in faint gray? (I wish you could make them explode.)

Practice Makes Deadly

At this point, you should fire up your newsreader program, find its manual and its FAQ, and practice making some simple killfile entries. Try making an alt.religion.kibology filter that highlights everything from "kibo@world.std.com" and removes everything whose subject contains "longest thread" (as in "Longest Thread Ever"). Once you've got the hang of the basics, it's time to draw up a list of the things you want to kill, and the things you want to select.

A good method of building your list of bad things is to ask yourself, when you see something particularly worthless, "What is the key word or phrase in the subject line? Will searching for it find similar articles and not worthwhile ones?" If so, it's a perfect item to kill. Similarly, if there's some bozo who just won't shut up, kill that author (or select that author if he's an entertaining bozo.)

Be sure to see my page of useful links, which lists some FAQs and other help files.

What to Kill, What to Kill... Me Wanna Kill...

Here's my list of your opinions on bad stuff:

• Kill advertisements for pyramid scams. Take out any subject line with "MONEY" or "CASH" in all caps (ads usually are); zap anything with "Earn" as the first word.

• Ads for phone sex. "900", "976", "SEX" (in caps), etc.

• Kill anything about spam: kill subject lines with the word "spam". In the wake of advertisements, large numbers of spam complaints follow.

• Get rid of flamewars. A start is to kill anything with a "Newsgroups:" header which contains the string "alt.flame" (this will include all alt.flame.xxx-yyy variants.) If your newsreader lets you search the "Xref:" header, use that instead of "Newsgroups:". (Xref is like Newsgroups only with more technical information; many newsreaders download all the Xref lines by default but not Newsgroups, so Xref can be faster to check.) If a persistent flamewar shows up in one of your favorite groups, I bet all the articles in the thread are crossposted somewhere (alt.flame? alt.slack? news.groups? rec.org.mensa?); killing alt.flame crossposts is a start, but check the headers and see if the flamewar's coming from elsewhere.

• While you're killing flamewars, kill any subject lines that contain hot-button words that people argue over. (Unless, of course, you really do want to participate.)

  • Every swear word you can think of.
  • Every racial/ethic slur you're sick of.
  • Any other sort of eternal argument you'd rather not participate in. "Creationism", "abortion", "racis" (works as "racist" or "racism"), "Jesus", "Satan", "Star Trek", "gay", etc.
  • "My computer is better than your computer" wars. These are hard to kill. I suggest zapping the word "Mac" (as a complete word, so as not to kill "MacGregor") if you don't use a Mac, "Win95"/"Windows"/"W95" if you don't use Windows, "DOS", "Linux", and "suck". (Don't specify "suck" as a complete word, to include "sucks".)
  • Names of political candidates and parties, at least those you don't want to read about. During election years, this stuff floods all over.

• To vaporize a particular person you don't like (let's say their email address is bozo@bozoville.com), search the "From:" header for "bozo@bozoville.com". Don't bother searching for the "real name", as that's something which people can change: the email address remains relatively constant.

• If there's a particular site where all the users seem to be pinheads (let's say "luzerland.com") then just search the "From:" header for "@luzerland.com".

• In newsgroups for "binaries" (this means computer programs or dirty pictures), kill anything shorter than 100 lines. That'll get rid of every single "me too", "can you please send me this", or "call my phone sex line" posting in alt.binaries.* and alt.sex.pictures.*.

• In some newsgroups, you may want to see only original postings, not the followup discussions. This isn't always a good idea, but you can remove any subject line starting with "Re:" to do that. (In some newsreaders, you may need to "escape" the colon, since it's often a character with special meaning.)

(The following three ideas talk about "regular expressions", which are a way of specifying patterns of things to search for rather than specific words or phrases. You can use them to search for things like "anything which looks like a phone number", for instance; it's not as fast as searching for a specific phone number, but powerful. Not all newsreaders can use regular expressions, but the good ones can; they also vary a little from program to program. By the way, searching for a "regexp" is called "grepping".)

• Search for phone numbers (American-style) with the regular expression "[0-9][0-9][0-9][-.][0-9][0-9][0-9][0-9]" and URLs with "(http|ftp):". (Do not include the quote marks I've placed around these expressions.)

• Find and stamp out massively crossposted articles. (A crossposted article is one with multiple items on the "Newsgroups:" header, posted once, that shows up in multiple places.) Kill anything with four or more newsgroups on the "Newsgroups:" line. (Sometimes there are good reasons for something to be important and "on-topic" in three newsgroups, but four seems to be stretching it.) This is a tricky operation: I suggest concocting a regular which looks for three commas anywhere on the "Newsgroups:" line, or five colons on the "Xref:" line. ("Newsgroups:" has commas between groups, "Xref:" has a colon before every group plus one extra.) A relatively efficient regular expression describing a string with five colons (take off the quote marks) is: "[^:]+:[^:]+:[^:]+:[^:]+:[^:]+:" In trn, try this killfile entry: "[^:]+:[^:]+:[^:]+:[^:]+:[^:]+:/Hxref:j" (see Matt McIrvin's killfile page for more details.)

• Here's a doozy suggested by Matt as a pattern to kill ads: "^(Re:)? *[^0-9A-Za-z"'(- ]" (Take off the outer quote marks. Yes, it's supposed to have some in the middle.) If you search subject lines for this pattern, it finds all those which do not start with a letter or number. Why? Because ads often start with exclamation points or asterisks or control characters, because many newsreaders sort funny characters to the top of the list.

Finding The Good Stuff

Here's my strategy for selecting interesting articles, in the order they are sorted in my newsreader. I am not suggesting you need to copy my color-coding scheme; you may not even want to colorize your browser. Your mileage may vary. But you probably will want to "promote" interesting-seeming articles if you can.

• Articles by me (green). Why do I select these, if I've already read them while writing them? So I can see if they came out right, archive them, etc.

• Followups to my articles (brown). My newsreader creates these filters automatically, to select anything with the same subject line I posted. This means that if I post "Subject: Roy Scheider is turning brown", I need to create a filter which checks the Subject headers for "Roy Scheider is turning brown", and it won't work if the people responding to me change the Subject header. If you can use regular expressions, a better method, which will find all followups to any of your postings, is to look at one of your own postings and see what your "Message-ID:" header says (it depends on your program); if it says something like "kibo123456789@news.std.com", search for any articles with a "References:" header containing the regular expression "kibo.+@news\.std\.com". (The References header lists the Message-IDs of the articles it is a response to. The ".+" in the expression here means "some stuff", and "\." means "a period.") This not only finds all followups to me, even followups to followups, it can find them even if headers such as "Subject:" or "Newsgroups:" have been altered by the respondent.

• Articles from good people (blue). There are a few people out there who always say something worth reading, intelligent, funny, and/or useful. You may want to search for articles from these people, such as moi, "kibo@world.std.com".

• Articles from entertaining wackos (purple). Okay, you probably won't enjoy seeing Alexander Abian rant about how we must blow the Moon out of orbit. But crazy science is funnier than funny comedy is scientific.

• Hot keywords (red). If I were Joe Piscopo's biggest fan, I might select Subject lines which contained "Piscopo". If I loved cheese, I could select "cheese" and "cheez". (Well, even if I loved cheese, I might kill "cheez".) The red stuff includes stuff I want to be informed about, or things that interest me.

• Warm keywords (orange). These are things which are kind of vague and may or may not be interesting. Mine include "Boston", "story", "index", "FAQ". I don't want to read every article that mentions Boston, so the orange "warm keywords" go below the red "hot keywords".

• And then after that comes the vast unexplored bulk of the miscellaneous stuff (black) which none of my killfile/selectfile entries matched, and finally the killed stuff (gray). Occasionally I go dipping into the black to look for surprises. The gray garbage at the bottom, the killed stuff, is only visible because I want to see what my killfile is doing. You may prefer to actually make the killed articles disappear completely rather than just pushing them down.

Part of my global killfile: these commands apply to all articles in all newsgroups.
I also use additional commands (not shown) for specific groups or sets of groups.
Picture simulated.