Online Community Technologies and Concepts

In a recent email conversation between Louis Rosenfeld and myself, I put down a lot of my thoughts about online communities. I wrote specifically about reputation management, collaborative filtering, mailing list management, content management systems, and distributed categorization and indexing efforts. Believe it or not all of these things are related.

The concept of an online community is very broad. It can be anything from a small close-knit group of people who email each other about a mutual hobby to a mailing list or Web site with thousands and thousands of users. There are thousands of these communities scattered across the Web, covering far too many topics to even attempt to list. Online communities are one of those subjects that encompasses a very wide spectrum of concepts. I'll try to break out a few areas that I have examples for.

Reputation Management: Reputation management is one of the areas that is getting pretty hot in the online community management space. The best example of this is a site called Advogato. It's basically a site where registered members can post articles and essays they've written for discussion. It's similar to Slashdot but without the news bent. The most interesting thing about Advogato is the reputation management aspect called the "trust metric" that lets users certify other users at one of four experience levels. Since this site is a community of mostly open-source software developers, the peer certification is mapped to that aspect of expertise. There's no reason why this trust metric couldn't be mapped differently for a different community.

Here's my profile, which shows that a bunch of people have certified me as an Apprentice and Journeyer. This is dead-on, because it's clear to the people who know me that I am not an experienced software programmer who has earned the right to be certified at a higher level. This system works very well. The more you contribute to the community and the associated software devleopment projects, the higher you are certified.

Note that each profile page also includes a diary/journal feature. Some developers make great use of this and their diaries are widely-read, with the main page of Advogato listing the most recently updated. This is a great aspect of building relationships between members of the Advogato community. It also ties into the important aspect of knowledge sharing within communities.

Collaborative Filtering: This is a dying area ever since NetPerceptions (they pioneered the space) changed their focus a while back, but the concept has been successfully integrated into Amazon's stores with their Recommendations system. They were one of the first companies to buy into NetPerception's idea of collaborative filtering. You'd have to check with Amazon to see how well they define it as a success or not. I shop at Amazon a lot and I don't use the system, but I'm sure that others do. There's no other reason for Amazon to keep it around.

I only include this concept here because a company called Firefly pioneered this space, but was acquired by Microsoft in April 1998. Microsoft abandoned the Firefly technology and used their own implementation of the same idea. Microsoft's greatest value from this acquisition is the concept of the digital "Passport". In fact, Firely even called their technology the Firely Passport. Microsoft has since trademarked the term Passport.Net and is using this terminology for their distributed authentication service. It will be interesting to see what happens in the future should the U.S. government decide to offer some sort of digital identity service that can be used in place of a traditional passport.

Mailing List Management: One of the most interesting concepts I've seen and used successfully is the email-to-newsgroup mailing list gateway script. This is exactly what you think it is. You basically set up an email alias to send all mailing list posts to a newsgroup archive. We very successfully used this while I was at Alphanumerica. But this idea did not migrate well over to CollabNet, after they acquired us. CollabNet, to this day, is still using lots of mailing lists, but no one has made the effort to archive them in a way that allows non-subscribers to read the archives.

Steven Champeon at hesketh, inc. is working on developing a mailing list management system that takes user profiling, permissions, and Web-based archives into consideration. He's building it from scratch to manage his long-running and excellent WebDesign-L mailing list. There is a little bit of discussion about it at Derek Powazek's Design For Community Web site.

Content Management and Document Management: These are really two very different things, but people often clump them together. The only two document management systems worthy of looking at are Xerox's Docushare and IBM's LotusNotes stuff, though both are very proprietary and expensive and offer no community or profiling features. They are often used solely for managing the electronic documents within large corporations.

Content management systems are often much more flexible. There are literally thousands of CMS solutions available. The best-known ones are the high-end expensive proprietary systems like Vignette's Storyserver, Interwoven, and Red Dot Solution's Content Management Server. Although I really like what Zope is doing with their Content Management Framework.

But don't count out NCompassLabs, which was acquired by Microsoft in April 2001 and recently renamed and rolled out as the Microsoft Content Management Server (how very unoriginal). It has a tight integration with the Microsoft Office products, which make it a natural rollout in any Microsoft-centric office.

However, very few of these CMS solutions have community features built into them. You can build a community around existing functionality, but it's always very apparent that it was never designed to handle those features.

CollabNet, my former employer, is developing an interesting solution called SourceCast, which is specifically designed to enable communication between software developers. It has a number of interesting community-centric features. The first being a technology called Eyebrowse, which is a Web-based mailing list management system that uses the same user permissions architecture as the rest of the technologies within the SourceCast suite of applications. This permissions structure was called Helm, but CollabNet recently made the decision to close the technology and fork it into a proprietary application.

One thing that I expect to see happen is for Project Liberty, an open-source competitor to Microsoft's Hailstorm to become one of the foundations for online user profiling and distributed authentication. Project Liberty has a lot of backing, far more than Microsoft has. However, it has an uphill battle against Microsoft's monopolies in the browser and OS markets. This battle will be won by whoever manages to convince the ecommerce companies to use their authentication frameworks. It's likely that big sites like Amazon will use both, creating some kind of middleware or API layer that can accept and process requests from both sets services/technologies. The problem is that Microsoft will say to companies like Amazon, "Sorry, but you have to use Hailstorm/Passport technology on your servers if you want our 60-some million Hotmail (and thus Passport) users to be able to access your site and simultaneously use our authentication services." It's a huge plus that AOL-Time Warner has backed Project Liberty. Also note that Yahoo has already created a similar service called Yahoo Wallet. Note that eBay is part of both projects. Smart.

Categorization and Indexing: You might also take a look at projects like Netscape's Open Directory project, called Dmoz. It uses contributors across the Web to build a Yahoo-like categorical directory, but without the overhead Yahoo has of paying people to manually index and cateorize the Web.

It might also help to look a the history of IMDb and CDdb, both which started out as distributed contributor projects, but were swallowed up by larger companies. Amazon bought IMDb and put too many ads on it, but it's still a good resource (it's the data that counts, right?). CDdb was swallowed up by a company called Gracenote. A lot of people were pissed off that they closed off free access to the CDdb servers, and now you need a license to allow your software to access their servers. Software developers were understandably upset since they could no longer use the CDdb technology for free.

Look at what About.com (formerly Mining Company) is doing. They've built a network of sites and are paying their "community guides" to maintain their individual sections, write articles, and basically be an expert in their field. I'm very surprised they've never decided to add mailing list capabilities to their sites. It's relatively easy to do and would be a great addition to the community services they already provide.

And last, but not least, take a look at what Google is doing with their newsgroup archiving. It's simply phenomenal. I completely expect Google to acquire a company like mail-archive.com or LSoft, and do the same thing for mailing lists. Yahoo has a head-start on them, since they acquired eGroups and built a very nice community and mailing list management service called Yahoo Groups.

The killer app of the Internet is email. That's been repeated by so many people that it's hard to dispute. Now, take a look at all of the above examples and how they contribute to the concept of online community. What is Google missing? What is Yahoo missing? If Google ever gets into the business of archiving mailing lists so they are searchable (especially if they spam-proof them, like Yahoo does), I think the geeks could never be happier. Can you imagine being able to search the Web, all of your mailing lists, an image directory, and newsgroups all from one place? I drool uncontrollably just thinking about it. Google is poised to be the mother of all online communities, by becoming the multi-linked hub between communities.

Anyway, there is no one company or no one solution that is doing everything correctly. Community management is a very difficult thing and absolutely requires a strong leader, list-mom, and watchkeeper who can eject trolls and people who misbehave. Without this person or persons, your community will fail.