Nothing three

Aggregation vs. Collaboration

2010-02-08T11:39:00.009-05:00

There was recently a Nieman Journalism Lab article comparing news coverage between Wikipedia and its smaller sister project Wikinews. It's an interesting topic, and certainly relevant to Wikimedia strategy in the long run: is Wikinews tenable? Would it be better to discontinue it, fold it into Wikipedia, keep it as is, or some other path?

Andrew Lih, who was interviewed for the article, attributed the differerence (and, apparently, a preference for Wikipedia) to a few factors. First, a formulaic structure in Wikipedia articles: an "inverted pyramid" going from general description to finer details. Second, a redundance in Wikinews articles: new events in a series require an entirely new story, with a new narrative and a new summary of contextual information—one that increases the workload and lessens the motivation of a Wikinewsie. Finally, he argues that the wiki process does not lend itself well to narrative processes like Wikinews, giving as an example the (essentially failed) A Million Penguins project by Penguin Publishing.

While Andrew Lih's commentary is good, I think there's an interesting generalization that can be made that I'm not sure is evident in what he says. My generalization is as follows:

Aggregative content production is easier than collaborative content production, but lacks the same quality.

This generalization is intended to highlight one of the biggest failings, in my opinion, of the "Web 2.0" shift: a lack of real collaboration. Take any of a number of Web 2.0 sites, and they can be roughly categorized as either primarily aggregative or primarily collaborative. For example, Flickr and Wikimedia Commons are primarily aggregative: add a decent image and you've improved the collection. There might be some collaborative elements, particularly in managing the metadata on Commons, but the broad thrust of these sites is aggregative. Content could in theory be added automatically.

I feel that most sites will tend towards aggregation over collaboration: aggregation is far simpler and easier. You don't worry about whether your funny cat videos really add to YouTube: you simply trust that enough people will upload enough videos of sufficient quality to keep you amused. People don't go around deleting bad YouTube videos, or solely improving other peoples' work. Aggregation doesn't require high-quality reviews, or any sort of endorsement of content, but instead generally takes more general statistics and perhaps-ignorant numerical approximations that can be made automatically. Search engines like Google are good examples of aggregative content: people create websites on the Internet, then Google aggregates most of those websites and applies an automatic process to rank their relevance to any given keyword.

I think that in the Wikinews vs. Wikipedia debate, we're missing a key component of the difference: which project is more collaborative, and which is more aggregative? I see that even if only one author writes a set of Wikinews articles, dozens of articles may be required to get the same information that will be present in a single paragraph of a decent Wikipedia entry, each written by a person or team. A Wikipedia article can be updated by simply adding a sentence with the update and perhaps a citation confirming it. Wikipedia is, in this sense, more aggregative than Wikinews. As ironic as that may seem (since Wikipedia is generally more collaborative than many "Web 2.0" sites) Wikipedia can more easily compile small contributions that might be worthless on their own into a high-quality aggregate article. Wikipedia is more formulaic, more automatic than Wikinews in some senses, especially given the lack of narrative. (That being said, we should not ignore the fact that Wikipedia's model is, in fact, essentially collaborative, but my argument is that Wikipedia has more of the low-input aggregative mode in this respect than Wikinews does.)

Now, automation and aggregation are not necessarily good or bad things: Google is leader in the search market precisely because it does an aggregative task well. But aggregation has severe limitations. There is a certain lack of human oversight in many aggregative processes, and they can easily be gamed: for example, Google bombing or search engine optimization or even simple sock puppets on sites without close moderation.

Collaboration is highly desirable. Good collaboration produces much better-quality content than aggregation, in general, since there are few forces to cause aggregated content to be improved. Standards can be built; corrections can be made. Google Knol is a good example of this problem. Individual "knols" are essentially controlled by their original author(s) and those designated by them, and there can be any number of knols on the same subject. Edits are often only allowed when manually approved by the original author. It is thus usually much, much easier to create a new, mediocre knol than to attempt to improve on someone else's knol. On average, I would bet on the Wikipedia process more than the Knol process.

The primary problem with collaboration is apathy. It's very, very hard to get good collaboration going, and when it does manage to make those few steps you'll still see only a tiny fraction of users contribute meaningfully to the end result. Apathy is a powerful enemy of collaboration, and without any interest in collaboration, a collaborative project will die a ghost town or be filled with irrelevant material or perhaps simply be taken over as a soapbox for a vocal minority. Aggregation solves the apathy problem by taking a route around it: make the apathy irrelevant by bringing in as much content as possible. Some users will still contribute high-quality material, and if they are numerous enough, the service will ultimately be useful. One does not have to care very much in an aggregative environment, and that helps overcome apathy to collect the end goal: the product; the content.

There can—and probably should—be a balance between collaborative and aggregative processes. Wikipedia, for example, harnesses aggregative forces in small edits and new articles, which fuels a platform for collaborative production. The difference in ease-of-growth between aggregation and collaboration, I think, is best illustrated by Wikipedia's own content statistics. A tiny fraction of the English Wikipedia's articles get significant collaborative influence and see recognition and promotion to grades like "Good" or "Featured", while a sad majority of articles remain short (but still hopefully useful) "stubs" and "Start-class" articles that have not seen significant editing by people other than their creator (or bots, which tend to break statistics looking at human patterns). Now, I think that the balance that ought to be sought is one that continues to accept the powerful aggregative influence, but that greatly promotes collaboration where possible, since collaboration most reliably produces good results.

The long-term goal needs to be to foster collaboration. Whether this will, or should, occur, at the expense of, or fueled by, an aggregative process, remains an interesting question.

Wikipedia, the balancing act

2009-10-10T16:28:00.003-04:00

I've been mostly inactive online lately: tweets here and there, some edits, a few sprees of emptying proposed deletion categories. Amidst all of the lab reports and other frantic work, I've taken little moments to consider a tricky question: Why does Wikipedia work? Why should it work? The oft-quoted "zeroth law of Wikipedia" has long been the following:

Wikipedia only works in practice. In theory, it can never work.

Wikipedia's size and complexity mean that generalizations quickly become over-generalizations, and many of my initial thought experiments had to be discarded. I finally hit on a model for experimentation that works: Wikipedia is a balancing act.

Wikipedia almost always falls between two extremes, at a point where most of the advantages of each can be maximized and the disadvantages minimized. With my chemistry background, I suppose I like to think of it as an equilibrium of sorts: there's a mixture of either "extreme" whose balance can change based on environmental conditions. Heating a chemical solution, for example, can shift a chemical equilibrium to favour an otherwise low-yield product; heating a discussion with a flame war can shift a social equilibrium to favour the otherwise low-use technique of removing members from the community. More seriously, whenever social conditions or attitudes change, any practice dependent on those attitudes will change correspondingly.

Wikipedia welcomes new content, so historically the barriers to creating new content have been low: to this day any registered user can create any page on a whim. Since Wikipedia also wants a certain quality of article, however, those whimsical pages might be deleted mere moments after they are created. This is one of the classic equilibria of Wikipedia, one that's been argued over countless times: inclusionism and deletionism. It's also a good example because it's shifted over time: the early Wikipedia was radically inclusionist, because any content was better than no content. As Wikipedia's content base has grown, so has deletionism: new articles are no longer so highly valued, and quality is increasingly valued, so low-quality articles are more likely to be deleted. Deletionism is entirely a product of a community that does not want to include content below a certain minimum of quality. Wikipedia depends on the balance between inclusionism and deletionism to be maintained: too much deletionism and you lose good content, too much inclusionism and you risk massive amounts of mediocre content.

I've not fully considered what facets of Wikipedia can be considered a balancing act in this regard, but some of the applications of the idea of balance seem obvious:

Governance: anarchy (open wiki editing), bureaucracy (wiki policies, the Arbritration Committee), and other systems find a balance (incidentally, it reminds me somewhat of the idea of "sociocracy" I found recently). In general, the anarchism of open wiki editing prevails as the most open, but when problems come up, bureaucracy and such can be used to make a relatively final decision rather than having continuous edit wars.
Page protection: open editing generally works, but as vandals arrive or edit wars begin, open editing becomes less tenable. High-profile articles like "George W. Bush", "Pie", or "Abortion" thus tend to end up protected. The forthcoming flagged protection feature will make this balance smoother by providing more layers of possible states of protection between the extremes of unprotection, semi-protection, and full protection.

Ultimately, the idea of equilibrium is too simple: it doesn't answer enough questions about how the balance started in the first place, or how stable each one is at the moment. What it can inform, however, is ideas on how to improve Wikipedia (e.g. the current strategic planning). People looking at problems in Wikipedia can target imbalances in the community, the software, or the available resources, and attempt to compensate. Ideas for optimizing Wikipedia for some desirable trait should take into account that Wikipedia depends on running its high-wire act between openness and standards, inclusiveness and quality, anarchism and bureaucracy—falling on either side would have unforeseen consequences.

Flagged Revisions: a confusing development

2009-08-25T14:09:00.012-04:00

In the past day or so, a number of news organizations, beginning with the New York Times, have been publishing stories about how Wikipedia is closing off editing by adding "flagged revisions" software. While there are plans to add an implementation of the "FlaggedRevs" extension to the English Wikipedia, these plans are being critically misinterpreted by the media.

Among the critical errors are assertions that Wikipedia is closing off otherwise open editing with the new software, or confusions of various proposed implementations of the software. It's a cloud of doubt that does not help Wikipedia: there is more than enough fear, uncertainty and doubt around all things "flagged revisions", and that is unhealthy for community discussions on how to run the project. As such, this post will serve to point out key facts about flagged revisions and its history, and outline where the future might be headed.

The FlaggedRevs software

FlaggedRevs is an extension to the MediaWiki software that runs Wikipedia.
FlaggedRevs is highly configurable. Different settings in the software can produce vastly different systems of authentication, some even simultaneous.
As the software, FlaggedRevs should not be confused with any given implementation of the software.

The planned implementation

The planned implementation of FlaggedRevs on the English Wikipedia is a test. The test is expected to run for around two months, after which the community will evaluate its impact and debate longer-term plans.
The planned implementation has two parts, "flagged protection" and "patrolled revisions".
- Flagged protection is a system not unlike Wikipedia's current page protection system. In the current system, pages can be "protected" or "semi-protected" by administrators, with the former restricting editing to administrators and the latter to "autoconfirmed" users (any user with at least ten edits whose account is at least four days old). With flagged protection, all users will be able to edit an article, but only certain users will be able to mark a version of the article as acceptable so that it appears as the main version of the article. Only articles that would otherwise be protected will end up "flag-protected".
- Patrolled revisions is a review feature. While it will apply to all articles, its primary effect will be to mark versions of an article as reviewed. This will help keep out vandalism, since it will be possible to easily check all the changes made since the last patrolled revision. It won't affect things otherwise.
The current test was ratified in a poll held from 17 March 2009 through 1 April 2009; 324 users participated and 259, or approximately 80%, supported the implementation.
The planned implementation should not be confused with the original suggested implementation, which is substantially different.

The original implementation

This implementation of the software was the original design of the software.
This implementation can be called "flagged revisions", but one must be careful not to confuse that phrase with other instances of the FlaggedRevs software.
The German Wikipedia elected to use this implementation of the software, and continues to do so, having started in May 2008. Language versions of Wikipedia are largely independent and can use separate articles, policies, and software extensions.
The original implementation would have effectively applied "flagged protection" to every article on the wiki. All edits would have to be manually reviewed by established editors before they appeared as the main version of the page.
A test of the original implementation was proposed in a poll on the English Wikipedia, but failed to gain consensus, with only 59.6% support from 720 users. On Wikipedia, majority does not suffice; "consensus" for large numbers of users typically entails at least a 75% supermajority, though 80% or greater is preferred.

Availability of FlaggedRevs

The FlaggedRevs software has been available since at least 2007. Its controversial nature in the community has been the main bar to its implementation: the Wikimedia developers typically do not implement such major software changes without broad community support.

Nevertheless, the English Wikipedia presents a unique technical challenge because of its size, and so despite the success of the poll requesting the flagged protection & patrolled revisions implementation, FlaggedRevs is not available on the English Wikipedia yet. The specific details of the implementation also cause trouble, in both that they stretch the original design of the software and that FlaggedRevs is controversial enough that arguments may be had over the minutiae of any implementation.

In any event, a test implementation is up and available at http://flaggedrevs.labs.wikimedia.org/wiki/, and developers have indicated that the software will go live within the next few weeks.

Debating the original proposal

The original implementation of FlaggedRevs, as a measure whose actions are essentially restrictive, is very controversial within Wikipedia and, I get the impression, outside it as well. It has certain evident advantages:

With flagged revisions, effective vandalism is virtually impossible, and in any event lacks all thrill when it will typically not be seen by any but the user that removes it.
By extension, libel and other problems are similarly unlikely when manual review is required for all edits. Wikipedia's real-world responsibilities, e.g. to avoid negatively affecting the lives of those it describes, become more manageable.
Flagged revisions offers unique opportunities for long-term review of changes to Wikipedia articles. While tools for examining the differences (or "diffs") between articles have long been a part of the core MediaWiki software, there are not yet tools for "flagging" revisions as having particular levels of quality.

Flagged revisions has certain potential disadvantages as well:

In large projects, it's hard to tell how much backlog of edits would be incurred, or how that backlog would be distributed. There's a significant concern that the backlog could be days or weeks long, much like the existing backlog for reviewing new pages.
Flagged revisions represents a serious barrier to openness. The Wikipedia community not only prides itself on its openness, but is fueled by it: many new users become long-term members after discovering editing spontanously.
Were disruptive elements to abuse the flagged revisions system by "flagging" inappropriate material, the damage would be greater than to a system agnostic of editorial approval.

These concerns and advantages are each far from trivial, but represent a crossroads: can Wikipedia integrate further restrictions without sacrificing its essential character? I think not. While the community at large would like some implementation of FlaggedRevs, there has not been enough support to justify it in that implementation.

There are a number of various groups on Wikipedia pushing either way, and though I do not wish to draw misleading borders, they tend to generally fall into valuing either prevention of vandalism and libel, or the openness of the site, higher than the other. Some would go as far as to suggest that all contributors be required to provide real-name authentication: others, like myself, worry that Wikipedia would lose its greatest strength were it to become less open.

Openness of the planned system

The planned system, flagged protection & patrolled revisions, represents a serious improvement, in my view, on the original flagged revisions proposal. It removes much of the restrictiveness of that system, at the expense, even, of the potential gains. Most of the restrictions are now only for features where users would already be restricted from editing, or to new features. The focus is on the process of review, rather than the process of editing.

In this planned system, flagged protection would likely help make Wikipedia more open. Where pages are currently protected so that certain groups of users are restricted completely from editing, it would surely be more open for them to be able to edit but require review of those edits. There exists a system of requesting edits on Wikipedia already: I imagine that this system could be superseded by a simpler method of requesting confirmation of edits to those pages. Flagged protection can replace older, cruder methods of protecting pages, and thus be more inclusive.

Patrolled revisions is a feature that will be restricted to established users, but that considered, it's important to note that it does not remove any permission from any user. It is merely a sensitive permission that, to prevent abuse, won't be given to all users. In the long term, it is my hope that patrolled revisions can serve as a sort of insurance for Wikipedia's reliability: people worried about viewing vandalism can view a reviewed version of an article to be sure to avoid it, while the current revision of the article remains completely open for editing.

The future of Wikipedia

FlaggedRevs as a software highlights much of Wikipedia's character: there is a constant balancing act going on between editorial oversight and openness, and it's always tempting to see what would happen were it pushed one way or another. On the one hand lies potential stagnancy through lack of contributions, and on the other lies stagnancy through a glut of mediocre content. Developing Wikipedia serves as a challenge that is also an experiment: no group of people has before managed such an ambitious project so openly, let alone had the success that Wikipedia has enjoyed. I, along with many others, worry that whatever is implemented will shape public opinion of Wikipedia, influence the size and shape of its contributor base, or be a platform for increasing (or decreasing) restrictions on Wikipedia's content. We can only hope that the future Wikipedia will continue to improve and succeed.

Scientology "banned" from Wikipedia

2009-05-31T15:53:00.013-04:00

In an effort to close some long-standing conflicts on Scientology-related topics, the Wikipedia Arbitration Committee (ArbCom, for short) has used some interesting measures in an attempt to settle the problem more thoroughly.

Most prominently, ArbCom has called for a blanket ban on editing from Scientology-associated IP addresses. Specifically, the sanction is the following:

2) All IP addresses owned or operated by the Church of Scientology and its associates, broadly interpreted, are to be blocked as if they were open proxies. Individual editors may request IP block exemption if they wish to contribute from the blocked IP addresses.
Passed 10 to 1 at 13:31, 28 May 2009 (UTC)

This is an interesting development, but I think it's being misinterpreted to some degree in the media. Given Scientology's reputation for attempting to influence media in their favour, it's entirely understandable that a group in Wikipedia's position would want to bar them from contributing, so I don't blame journalists and the public for misunderstanding the IP ban for an organizational ban—but they can't be excused for missing the fact that a number of anti-Scientology activists were topic-banned (disallowed from editing Scientology-related articles, on penalty of blocking) as part of the decision.

To a certain degree, it is the case: Wikipedia doesn't need the kind of relentless view-pushing that Scientologists present. Whether they're "fighting religious discrimination" or "suppressing the truth", their drive to stamp out criticism of the movement is undeniable, and on Wikipedia, that's unacceptable.

It's evidently not the case that Wikipedia is outright banning the organization. The above sanction only blocks the IP addresses rather than bans the organization. If you read the principles defined for the definition, this becomes more evident. Specifically, the following principle is interesting:

11) It is rarely possible to determine with complete certainty whether several editors from the same IP or corporate server are sockpuppets, meat puppets, or acquaintances who happen to edit Wikipedia. In such cases, remedies may be fashioned which are based on the behavior of the user rather than their identity. The Arbitration Committee may determine that editors who edit with the same agenda and make the same types of edits be treated as a single editor.
Passed 7 to 4 (with 1 abstention) at 13:31, 28 May 2009 (UTC)

It's not only the principle that justifies the block, in my opinion—since multiple uniformly pro-Scientology editors using Scientology IP addresses can't be distinguished from a single one abusing multiple accounts—but one of the more controversial principles in the case.

I find this curious, and so looked at the votes placed on the principle. Risker opposed the principle with the concern that it could be used badly, since users might be attacked using this principle based on the point of view that their edits support.*

*Having edits which support a particular point of view is, in my opinion, not inherently in violation of Wikipedia's policy of neutrality; even if someone's edits uniformly present a particular point of view, correcting other imbalances, or presenting material favourable to one view neutrally, is not necessarily a problem.

As I was writing this post, I stumbled upon a Huffington Post article which deals with this kind of concern. It's interesting in part because of the points that it misses, but in part because of the relevance that it has to the subtler implications of the ArbCom decision, and suggests that it may be a dangerous precedent. In "Wikipedia Removes Semi-Protection from Civil Liberties", Leah Anthony Libresco argues that Wikipedia's decision to ban Scientology is misguided. Libresco argues that setting the precedent of banning an organization like Scientology, a whole class of people, is akin to taking away civil liberties. There is some confusion: Libresco says, for example, that WikiScanner can "identify the sources of anonymous edits made on Wikipedia by analyzing the IP addresses of the perpetrators", when this isn't really the case (WikiScanner correlates edits made without a user account with a database of known IPs of organizations, but can't equal tools like CheckUser which can investigate registered users). I think that Libresco misunderstands the depth of the problem (that Wikipedia's methods for investigating abuse are insufficient for the Scientology IP addresses) and the seriousness of the remedy, while falling into the usual assumption of an organizational ban rather than a technical block. Libresco does, however, make some cogent points about the need for open discussion—if speech is suppressed, neutrality becomes more difficult to create and, worse, to expect.

This kind of sentiment was echoed in another article which criticized the decision. In "Why Wikipedia was wrong to ban Scientology", Evgeny Morozov attacks the ArbCom decision as one which suppresses the group from joining in the debate about itself. Two sections of his article summarize the article well for me:

I am no fan of Scientology, but I think that banning them from Wikipedia is going to be counterproductive. Unfortunately, it presents the Wikipedia admins/editors as a non-neutral group that opposes a particular set of ideas. In an ideal world, I don't think that the Wikipedia editors should be making any value judgements on whether a particular idea is good or bad, for it undermines the trust that users place in an open encyclopedia, no matter how innovative it is.

[…]

However, bowing down to Scientology-bashers is almost guaranteed to trigger similar requests from people who hate satanism, fascism, or even pokemons. [sic] Granted it's harder to identify and ban the more decentralized community of, say, satanists than that of scientologists [sic] (who have registered physical addresses), but I am sure that very soon somebody will request that another group is excluded from online deliberations over what kind of materials to publish about it. In a way, Wikipedia's decision opens Pandora's box : why allow Christians to edit articles on Christianity, for example?

It misses many of the obvious considerations, and has been thoroughly criticized so far in the comments, but it does raise a good point in the larger scheme of things: is a decision to block particular sets of IP addresses on these grounds tenable? It's certainly possible that it may—as the first quoted section, and some other articles have suggested—have negative public image effects for Wikipedia. It's certainly a reminder that, especially in administrative actions, even the appearance of impropriety in an action can be damaging without the need for any true abuse.

On another issue, the criticism raises questions. Libresco's article in particular makes the parallel of civil liberties in suggesting that the suppression of any particular group is troubling. While I think it's fallacious to make a direct comparison of Wikipedia to many existing political and economic systems, there are parallels that should not be ignored. I plan to outline some interesting parallels in a future article.

Is the block dangerous, justified, or merely ugly and unfortunate? I'd like to hear your opinion.

Hello, world!

2009-05-24T15:01:00.006-04:00

I've thought a while about various Wikipedia-related issues, and I follow Wikipedia-related news, and I've decided that I should blog about them. Here are some basic ground rules:

The blog will follow NPOV, where NPOV is "Nihiltres' point of view". :) While it would be nice to follow a Wikipedia-like neutral point of view, I don't feel like working endlessly to appease critics. I generally think Wikipedia is a good thing, but not perfect—criticism will try to be constructive when applicable.
I plan to allow and encourage comments, but I reserve the right to delete posts for any reason. Specifically, comments which are unhelpful or derogatory will be targeted.
You can ask me to write about something! Contact me on my Wikipedia user talk page, by email (wiki dot nihiltres at gmail dot com), through Twitter (@wiki_nihiltres) through IRC ("Nihiltres" on freenode), or even "wikinihiltres" on AIM, and I'll consider writing about whatever topic you suggest.
I plan to try to make this blog accessible to non-Wikipedians, though it'll inevitably describe things that only Wikipedians are really interested in. Keep your WP:WTF (acronyms) out of comments, please!

I'm currently in the process of writing a first ("real") article, so watch this space.