CSCW Session – Wikipedia and Coordination

10Nov08

I’m taking notes as the sessions go…

Mopping up: Modeling Wikipedia promotion decisions

Moira Burke and Robert Kraut – CMU (Bob is a failry big figure in CSCW and CHI)

How are promotion decisions made?

  • Large groups of strangers colaborate to choose caretakers known as administrators
  • We model successful candidates based on simple metrics that can be computed quickly in real time
  • How does the community user evidence to build consensus, and are there opportunities for tools to support decision-making?

Policy capture theory:

  • compare organizations stated criteria for making decisions with actual behaviors
  • typically: a disconnect (now, imagine that)
  • beause of:
  • difficulty finding information
  • cognitive overload
  • weighting simple things too heavily
  • process-blocking or bandwagon effects in collective process.

Method: Looked at all Admin-approvals.

  • Categorized previous contributions based on RFA guide
  • Modeled promotion success based on contribution history
  • Several criteria listed from RFA

Hard to measure criteria:

  • trustworthiness
  • quality of edits

How to use: apply their model of admin-effectiveness to:

  • voter dashboard or self-evaluation tool
  • Admin finder bot
  • Similar model for decisionmaking in other online environments like WOW

Harnessing the wisdom of crowds in Wikipedia: Quality through coordination

Aniket Kittur and Bob Kraut, CMU

Online collective intelligence:

  • Predicitng
  • Filtering
  • Organizaing
  • Recommending (netflix)

Assumptions:

  • people are making independent judgements
  • and you can automatically aggregate these assumptions

But that doesn’t really work for complex information processing.

Need to coordinate, collaborate. So: HOw do we harness the wisdom of crowds for complex things.  Just throwing people together won’t work. “Adding manpower to a late software project makes it: later.”

Previous research indicates that more work / more people on wikipedia leads to better articles (“Feature articles”).

Interested in coordination among authors editors:

  • Explicit coordination (direct communication between authors / editors)
  • Implicit coordination (structuring wrork so it is concentrated in core group; leadership role in setting scope and direction)

How to measure quality of artciles? Why, with the Wikipedia 1.0 Quality Assessment Scale!

Findings:

  • Incrasing # of editors had NO INCREASE in quality
  • Increasing coordination in communication and concentration resulted in higher quality.
  • Communication does not scale to the crowd. High communication with few editors leads to qualtity. But scale up the editors, quality goes down.

Interesting stuff.

Articulations of WikiWork. (Good paper!)

Mass collaporations, a la wiki, are going to become more important to society.

We want to know how this work is sustained, so we studied Barnstars on wikipedia. To figure out how work is valued on wikipedia.

They review the breakdown of barnstars on wikipedia:

  • Editing work: 27%
  • Social and Community support actions: 25%
  • Border Patrol: 11%
  • Administrative Actions”: 9%
  • Collaborative ations and dispositions (collaboration on pages like mediating conflicts) 8%
  • Meta cntent work, the cration of tools, templates, etc. 5%
  • And, undifferentiated: 14%

So note that editing only accounts for 27%. Reputation systems must award more than simple production, right?

Need reputation frameworks for complex cooperative work. Very interesting.

Advertisements


No Responses Yet to “CSCW Session – Wikipedia and Coordination”

  1. Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: