Crowdsourcing Links and Question of the Day

• Here's a new startup that sounds interesting. Intelligence-Tools

"The idea here is similar to Mechanical Turk, but with a twist. We place all prices on the system in a market similar to the NYSE. This allows for instantaneous completion (or at least contracting/assignment) of tasks."

Example: Check out their document editor. You paste your text in a box and get a quote for editing. That quote is only good for 15-minutes, so think fast.. Currently the system doesn't have editors and I imagine right now Intelligence-Tools has the same problem most crowdsourcing startups do. How to get critical mass.

• Checking in on Spot Us

We have news. Our first pitch has been successfully crowdfunded. That only took 10 days! Perhaps there is something to "community funded reporting." But as I note in the Idea Lab blog, it's too early to start waving hands and claiming success. Still, I believe a w00t is an order.

 

• via Scripting Enabled

"A few weeks ago a lot of Tweets, emails and instant messages started chatting about a new system that would allow screen reader users to fix issues with web sites in a crowdsourcing manner: WebVisum"
Read the interview with Marc Dohnal, the initiator of WebVisum.

Note: IBM has their own crowdsourced project to tackle the accessibility of the web. Social Accessibility Project.

• I like this post on strategies for harnessing group potential.

"The first is what I’d call the “million monkeys” strategy.....More exciting is truly collaboratively crowdsourcing"

• This brings me to the question of the day!!!

What are the criteria through which crowdsourcing does and doesn't work.

Often people ask: "is this a good thing to throw crowdsourcing at." As if crowdsourcing were Dungeons and Dragons and "crowdsourcing" was a spell you can caste. No, crowdsourcing isn't magic and you can't just sprinkle it on websites and make everything viral.

Example - Crowdsourcing is great if you want to write an encyclopedia entry about a specific automobile maker. Proof.
But there are certain things I would never trust the crowd to do.
Examples: Brain surgery (especially my brain), fly an airplane (especially my flight), raise a child (especially my child and yes, that is despite the 'it takes a village' saying). I think you get the idea.

Crowdsourcing can be very powerful but it can also be destructive. I'm willing to bet that even Jeff Howe would agree - there are times when you DON'T want to employ crowdsourcing. So my question again is "what are the criteria through which you can determine if crowdsourcing should/should not be applied."

Comments

[Mike Krieger]

Hi David,

Great job writing these guest posts, and great question you've posed today. This is a major part of what I'm looking at in my Master's thesis, and I think the following might work as a way of thinking when/where crowdsourcing is appropriate:

-- Is the *diversity* of the crowd going to benefit this task/project substantially? (ie: in InnoCentive's case, surprising solutions from other domains come from the diversity of participants)
-- Are there *small chunks & delegate-able actions* that would benefit from parallel crowd contributions? (Poring over satellite data, collaborative filtering)
-- Can the crowd's contributions be evaluated by the crowd itself/other verification mechanism to make sure good data is coming in? (Luis von Ahn's games all have some verifying element built in; Dolores Labs talked recently about the thresholding approach they use to make sure they're getting good data)
-- Is the crowd contributing something they enjoy doing, or wish they had more chances to do? (Designing/voting on tshirts; looking at galaxy maps, etc)
-- Will participants in the crowd be better than computers at performing a task? (tagging, 'serious games', etc)
-- Is there still a central vision for the project that guides and moderates the crowd's input and direction? (this is why you wouldn't want a crowd to design a whole product, or fly a plane -- there still should be an architect/core team, a la Fred Brooks)

And perhaps, most importantly:
-- Is the crowd being brought in because of its strengths -- diversity, rapid generation of knowledge/feedback, tacit knowledge (as humans, as users of a product, as members of a community...) -- or simply to reduce costs?

What do you think of these guidelines/questions?

[alan booker]

Ditto David, fantastic postings!

“What are the criteria through which crowdsourcing does and doesn't work.”

I wonder if the question could be, can one apply CS, as a concept, to any initiative or should it be rather be that as an initiative develops, is the CS term relevant or not. That puts emphasis on the process rather than the concept!

As much as the term CS is still in its infancy there can be no doubt that as a process, even if unnamed, it has enjoyed its place in many instances throughout the march of time.

The recent Obama campaign certainly demonstrated the power of the people. CS, when it comes to small individual donations collectively blowing away traditional models of fund raising!

At the moment the focus appears to be upon initiatives that harness the power of crowds. I am sure that as one experiences a decline in living standards, political dissatisfaction and other dramatic changes in the social/cultural sphere, collective input/the will for change is going to demonstrate that CS might in fact be the term that adequately describes the participatory process that supports or forces change of existing models!

Collective input might initially be focused upon a very different set of goals than an end outcome such as model change, but that could well be the end result of collective participation. To put it differently, the criteria might be completely unknown until a new model is created that allows for collective participation.

I think confusion arises when the words such as job, employ, employee or agent are used.

Using the principle of open source as an example tends to turn the mind to the question of free as apposed to paid participation which then degenerates into questions of being used, or not, rather than the higher original principle of participatory inclusion.

The difference between free or “free as in freedom” might better give a clue! Those criteria could warrant some pretty intense conversation.

In following the argument regarding “spec work,” http://andrewhyde.net/spec-work-is-evil-why-i-hate-crowdspring/
it appears that regardless of the objective facts mentioned in the many postings, never the twain shall meet and therein is wedged the challenge.

As new technologies are introduced disruption of already existing industries, institutions or businesses are unavoidable.

Change is constant and most of the arguments against it are sour grapes unless unethical practices are used!

Regards, Alan