Return of the Turk...

A old idea made new.

In 1770, the inventor Wolfgang von Kempelen unveiled to the court of Habsburg Empress, Maria Theresa, a remarkable device; an automaton that mimicked the appearance of an Ottoman Sultan seated at a chess table. This machine, the “Mechanical Turk” was able to play chess well enough to challenge experienced players, and continued to amaze audiences for 50 years before it was revealed that the machine’s ability to play chess was facilitated by a cleverly concealed human operator.

Now there’s a new “Mechanical Turk”, which works in much the same fashion, using human operators to perform tasks that, although seemingly as simple as moving a chess piece, are currently beyond the ability of computer programs to reliably perform.

What sort of tasks? Well, determining from a sentence whether the general sentiment of the text is positive, negative, mixed or neutral, or determining whether a particular image is relevant to text of a search query that revealed said picture in a website, and the significance of the same image in terms of size, placement, and quality on the same page.

Today’s Turk is a little-known application hosted by Amazon that allows businesses to post HITS (Human Intelligence Tasks) that an army of human workers (known as “Turkers”) can work remotely from their computers for compensation ranging from a few cents to several dollars per task.

On the surface, it’s an interesting idea – there is work to be found here – and with about 100,000 HITS available to be worked on a daily basis at an average value of perhaps 50 cents, there’s also money to be made, which has drawn an estimated 100,000 Turkers in 100+ countries (although many are based out of the US, and demographics suggest they are generally lower-income individuals.)

Because of the low pay and repetitive nature of much of the work, the Turk has been criticized as a sort of modern sweatshop, and certainly it’s not the most interesting sort of remote work that can be found on the internet, which includes everything from farming gold in World of Warcraft to selling designer clothes and shoes in Second Life.

On the other hand, anyone can get started Turking with just basic computer skills, a few easily accessible web tools - I used Google Translate to help with that task of determining the sentiment in sentences (which were written in German) – and some time on their hands.

That’s what I thought the strength of this approach to crowd-sourcing might be; a student in a boring class or a worker in an office job that requires them to be present but not necessarily busy might find the Turk to be both a welcome distraction and a means of extracting a little more monetary value from time spent glued to a seat by the demands of an antiquated approach to work or education.

To check it out, I set up an account (easy to do if you already shop with Amazon) and jumped in looking for HITS while sitting in a class of the aforementioned Neolithic style, then continued the exercise while stuck in a taxi in for a couple of hours in city traffic.

My experience was that tasks that pay well (for example, the German sentence sentiment exercise) actually involved a fairly significant degree of both work (translating, rating, etc.) and concentration, which made difficult to multi-task effectively between class participation and HIT completion (HITs have a specific timeframe, ranging from a few minutes to several hours for completion.) Tasks that pay lesser amounts, such as the “quality/relevance of pictures in webpages” HIT are quicker to complete, but still require some effort. Some people may find these tasks interesting, despite their repetitive nature – but I doubt anyone could happily make a living by Turking.

I also found the interface a bit clumsy, and many tasks required “qualification” tests which, while useful to the business in finding qualified Turkers, was annoying to the person who wanted to get right into doing some electronic drudge work. There was also a good number of tasks that appeared to be scammish in nature (requiring a website sign-up to complete the task) and HIT-spam, where a single job site posted 60 or 80 very similar HITS, which, due to the nature of the sort/filter tools available, forced the Turker to flip through several pages worth of undesirable HITS to find a more desirable source of employment.

These are strange complaints for a site associated with Amazon, but it turns out the association is very loose indeed – the Turk was originally designed by an Amazon employee to help the company find and eliminate duplicates in its own product description page, but expanded to become a crowd-sourcing tool which has been used for such tasks as finding missing persons, transcribing podcasts, generating blog content, and driving traffic to websites. Now, although Amazon hosts the site and takes a 10% commission off each job, it doesn’t actively monitor the service and refers people who complain to the company or individual who posted the offending HIT.

The bottom line? The Turk represents an interesting first effort at creating something that will be increasingly common in the future workplace; a market where content developers and “human intelligence” workers can find employment, on their own time and terms, capable of working remotely from anywhere on the planet, and potentially reducing pollution and congestion by allowing digital workers to earn their income from the comfort of their own homes. However, the current market lacks protections for the workers (in the form of stipulated minimum compensation) or protections for society, since many of the jobs offered are designed to skew perspectives of search results, etc. Is the website that pops to the top of your search queue really popular (and thus, presumably, a valuable source of your desired information) or has it attained that rank due to blind clicks by a mercenary army of Turkers?

The race should be on to develop a common global online market, where anyone from retail giants such as Amazon and Walmart to individuals sitting in their underwear in front of a laptop can offer their goods and services, accept compensation, and schedule delivery to any other citizen in the world; secure from spammers and scammers, with fair-trade wages and prices paid, and a feedback system in place to reward good performance and reveal poor performance. The pieces of such a system already exist – we have price comparison sites, online retailers, auction sites, and rating sites. But to tie them together in a simple, easy-to-use interface – that would be a parlor trick worthy of Wolfgang von Kempelen.