45 comments

  • madrox14 hours ago
    Mechanical Turk had a good run, but not surprised it&#x27;s shutting down. I&#x27;m sure the platform was getting flooded with people doing task arbitrage and using lots of AI anyway.<p>I believe the issue is that this can no longer be a horizontal play. MTurk was mostly for unskilled tasks...the kind AI can do well enough that it isn&#x27;t worth the cost differential to verify it or keep farmed to humans. The &quot;trust but verify&quot; AI output is now the kind that requires domain expertise. This is what most full stack AI companies are bringing to industries.<p>Curious if this kind of work will come around again one day or was just a moment in time. If it does I&#x27;m sure it will be specifically about generating training data.
    • johnsmith184014 hours ago
      I think yes but on different categories. First one I imagine is robotics control and support.<p>&quot;This robot is having trouble folding a tshirt help it out for 1$&quot;<p>Unless they go the waymo route of highly trusted people but I think mass deployed robots are a bit safer than a car for this.
      • georgefrowny4 hours ago
        Giving out control of industrial machinery that interacts in the human environment without the physical interlocks (i.e. humanoid robots in a house) to random internet people seems like a problem.<p>I can imagine a carefully orchestrated plot to assassinate someone by having an embedded agent in the task delegation pool command the laundry bot to punch the target&#x27;s head off their shoulders.
        • dividedbyzero4 hours ago
          Giving out control of such things to LLMs is already complete madness, so once the first pleasure bot powered by Grok has dismembered a few thousand users, they&#x27;ll get sophisticated safety mechanisms.<p>Though like as not you&#x27;re still going to be right, after all, Stuxnet happened.
          • iamacyborg2 hours ago
            I’m really not happy that you’ve put the idea of Musk fuckbots into my head.
            • dmd2 hours ago
              I&#x27;m envious of the time you spend where it wasn&#x27;t already.
      • madrox13 hours ago
        This is what I meant by full stack AI companies. I don&#x27;t think you could get humans into the loop fast enough if they didn&#x27;t have some idea of the type of task involved. I don&#x27;t want people to be asked to fold a tshirt one moment and do a difficult traffic merge the next.
        • its-summertime12 hours ago
          There is training systems and validation of skills in mturk iirc: for tshirt folding, you&#x27;d be given fake setups to be able to get used to controlling the robot, if you can&#x27;t do it, you won&#x27;t ever get assignments to do it. For traffic overrides, you&#x27;d be tested on having correct knowledge, and once again given supervised tasks to show you can actually be trusted (and there would be safety systems, elevating tasks that can&#x27;t be performed at your level to people who can, etc)<p>The worker needs to do a bunch to opt into any given work group, which makes the (lack of) payments extremely unreasonable on top of everything else
        • inigyou9 hours ago
          It doesn&#x27;t really matter what you want though, only what CEOs want and that&#x27;s low costs. I can see a combined shirt folding&#x2F;traffic merging platform taking off.
          • vavos1 hour ago
            but call center are still mostly single client even though it would be cheaper for any worker to be able to answer to any call. So clearly the expertise and context trade off is too big to be worthwhile
          • bonoboTP4 hours ago
            CEOs are loser nobodies. Real influence is with owners, boards, investors.
            • inigyou57 minutes ago
              Mostly they don&#x27;t care as much as CEOs think they care
        • ntauthority13 hours ago
          warioware shows it can be <i>fun</i> though but i&#x27;d indeed not like to see that applied to safety-critical tasks
      • mike_hearn6 hours ago
        There are already robotics models that can fold shirts and similar just fine. Progress in VLA models is good, I don&#x27;t think this would form the foundation of a business. Humanoid robots are going to be another ChatGPT, it&#x27;s going to seem to happen almost overnight because people aren&#x27;t paying attention to the underlying research papers.
        • simsla5 hours ago
          Any papers you&#x27;d recommend?
          • cyber_kinetist57 minutes ago
            Most progress in data-driven robotics nowadays are done either in unicorn startups or corporate research labs - so you should follow the industry more than academia. The path to good robot performance isn&#x27;t really in the models themselves - it&#x27;s highly dependent on how much you can gather high-quality real-life data.<p>Specifically for laundry folding, Sunday Robotics is probably the state of the art, where they were able to obtain 99.1% success rate and call it &quot;done&quot;: <a href="https:&#x2F;&#x2F;www.sunday.ai&#x2F;blog&#x2F;act-2-preview#solve-standard" rel="nofollow">https:&#x2F;&#x2F;www.sunday.ai&#x2F;blog&#x2F;act-2-preview#solve-standard</a>
            • amichal10 minutes ago
              If their hero image video and the side by side at (5x) with a human at quote &quot;1x&quot; are &quot;solved&quot; I&#x27;m not impressed. I&#x27;m faster and more accurate and I am the worst folder in my house (kids included). The human looks like they are doing it slow mo to show children how to.
      • vidarh8 hours ago
        There are already a number of companies providing RLHF and SFT services for AI providers that does a lot of validation&#x2F;prequalification of people that&#x27;d be well placed to take on tasks like that, but the big problem to solve would be <i>latency</i> if you don&#x27;t have people contracted to carry out a task <i>right now</i>.
      • mikestorrent13 hours ago
        That&#x27;s a remarkable idea. It could be heavily gamified, it could train models, and it might actually be mentally stimulating since you&#x27;d be facing different situations all the time.<p>Except, I&#x27;m a grown adult and I can&#x27;t fold a t-shirt properly
        • madrox13 hours ago
          It could require listing your credentials to get you the proper tasks: doctor, lawyer, or tshirt folder
    • Yokohiii11 hours ago
      It is weird because the last time I&#x27;ve heard about MTurk was about developing countries being rather reliant on it for doing AI grunt work. If am not totally wrong this must mean that the data work has moved to other services.
      • toyg6 hours ago
        Probably AI models got good enough to bootstrap their own training systems.
        • swiftcoder3 hours ago
          I think the bigger issue is that a lot of the demand for labelling training data is now in highly-specialised fields (i.e. things like medical imaging), and mechanical turk&#x27;s focus was on the generalist problems
    • SV_BubbleTime10 hours ago
      &gt; task arbitrage and using lots of AI anyway.<p>Oh, I remember UpWork.
      • shuwix6 hours ago
        Oh ... upwork ... put offer, get 50 replies from people which jump for every penny without even being able to understand the task.
        • SV_BubbleTime1 hour ago
          I used it before AI coding and it was getting rough. Lots of US interviews with proxy Chinese or Pakistani workers. Lots of bullshit “I’ve done that; I can do this” and instantly apparent that this was untrue. Just the outright lies… whew.<p>I haven’t touched it since AI coding.<p>It’s a bad contractor market now. IDK what I would do if I needed a contractor.
    • ape43 hours ago
      I understand unskilled humans are used to train AIs
      • falcor842 hours ago
        To the best of my knowledge, that isn&#x27;t true anymore, and nowadays you&#x27;d only get hired to do RLHF if you have particular skills beyond what can be achieved by just running other models against it.
    • raverbashing8 hours ago
      &quot;Fun&quot; fact, they were also used by psychology&#x2F;etc students when they need to do &#x27;research&#x27; with X amount of people
    • timcobb14 hours ago
      How could it come around again?
      • Legend244013 hours ago
        There are like three dozen companies selling similar services for generating AI training data.
        • charlieyu13 hours ago
          Good data isn’t cheap and keeping it in-house gives you more control.
      • madrox14 hours ago
        I&#x27;m unsure. If it does, it will be work that is too expensive or inaccurate or regulatory for current AI methods. For example, you want a doctor to sign off on some AI output on a diagnosis.<p>However, I&#x27;m not sure a single platform will be how it emerges
    • yashvg4 hours ago
      [flagged]
      • ZitchDog4 hours ago
        How do you make sure they aren’t using AI to pretend to be a domain expert? I’d think an AI would be pretty good at that.
        • yashvg4 hours ago
          We do our best to identify AI use and ban those experts - its not perfect just yet. Long term we are thinking of moving towards proctoring experts using their camera and screen capture. Hard to think of another reliable way.
      • fireant4 hours ago
        Some doctors are known to be a hip shooters making snap decisions, but is 30s really enough time for any kind of &quot;expertise&quot; from a real human?
        • yashvg4 hours ago
          You get matched with an expert in 30s to 3min, which then starts a back and forth with the AI agent and the expert which goes on for 10 to 60mins till the AI is happy.<p>This works well for many tasks, but for others a more async mechanism where the expert doesn&#x27;t feel rushed might be better.
          • testplzignore3 hours ago
            &gt; Experts are vetted upfront by an AI interviewer<p>&gt; till the AI is happy<p>My god this is dystopian.
  • x0xMaximus14 hours ago
    As AMT&#x27;s largest requester for the past 10 years, this news was relayed to requesters at the same time as respondents. It&#x27;s also worth noting that our lead contact, the Sr Program Manager at AWS leading AMT, transitioned to Amazon Bedrock and SageMaker Model Evaluations a ~2-3 years ago.. Leaving behind essential zero team managing the project after they migrated over the stored value accounts to native AWS billing.
    • jan_Inkepa14 hours ago
      our of curiosity, can you say what your use of it was?
      • x0xMaximus14 hours ago
        Initially the annotation of biomedical literature (<a href="https:&#x2F;&#x2F;pubmed.ncbi.nlm.nih.gov&#x2F;25592589&#x2F;" rel="nofollow">https:&#x2F;&#x2F;pubmed.ncbi.nlm.nih.gov&#x2F;25592589&#x2F;</a>) but left academia for commercial use where we transitioned to arbitrage the market research and ephemeral task markets. It was essentially a research panel with a underdeveloped UI for tasks.. so abstract away AMT&#x27;s tooling so it can be used by any buyer in the ResTech, Political Polling and Data Annotation space<p>My favorite part of AMT is always going to be figuring out that if we only paid in $0.07 intervals, their commission algorithm would round down to the nearest whole cent when their 20% commission resulted in a fractional cent on the unit transaction level, not the monthly invoice level. Was ultimately worth it to have implemented <a href="https:&#x2F;&#x2F;git.generalresearch.com&#x2F;panels&#x2F;amt-jb&#x2F;tree&#x2F;jb&#x2F;flow&#x2F;assignment_tasks.py#n527" rel="nofollow">https:&#x2F;&#x2F;git.generalresearch.com&#x2F;panels&#x2F;amt-jb&#x2F;tree&#x2F;jb&#x2F;flow&#x2F;a...</a>
        • Nition10 hours ago
          &gt; Can you say what your use of it was?<p>&gt; Initially the annotation of biomedical literature but left academia for commercial use where we transitioned to arbitrage the market research and ephemeral task markets. It was essentially a research panel with a underdeveloped UI for tasks.. so abstract away AMT&#x27;s tooling so it can be used by any buyer in the ResTech, Political Polling and Data Annotation space.<p>Reading this is similar to how I feel when I&#x27;ve asked Claude about something it coded for me. As with Claude, I think I get it after reading it three times; you guys were a middleman that provided a simplified interface for Mechanical Turk?
          • ShinyLeftPad1 hour ago
            &gt; Reading this is similar to how I feel when I&#x27;ve asked Claude about something it coded for me<p>I didn&#x27;t feel like that at all.
        • swyx12 hours ago
          &gt; My favorite part of AMT is always going to be figuring out that if we only paid in $0.07 intervals, their commission algorithm would round down to the nearest whole cent when their 20% commission resulted in a fractional cent on the unit transaction level, not the monthly invoice level.<p>can someone ELI5 because what in the office space
          • ramses012 hours ago
            Go watch Superman III for more details...
            • libria10 hours ago
              Or Office Space
          • owebmaster12 hours ago
            20% of 0.07 was billed as 1 cent, not 1.4 nor rounded up to 2.
            • rkozik19891 hour ago
              Imagine your favorite feature of a product was spend 1 cent on a real human being time and effort rather than 2 cents.
              • aftbit1 hour ago
                That&#x27;s on the commission that Amazon took, not on the human.
              • ShinyLeftPad1 hour ago
                Or spend the same and let the human earn more?
            • x0xMaximus12 hours ago
              1
          • testbjjl12 hours ago
            It was cheaper and looser automated billing practices than current AI “solutions”?
        • wuschel52 minutes ago
          &gt; My favorite part of AMT is always going to be figuring out that if we only paid in $0.07 intervals,<p>That is hilarious! Was it a semi-random discovery due to interaction with the system and people, or did you intentionally looked to game the algorithm?
        • loremm14 hours ago
          I truly have no idea what this website is saying <a href="https:&#x2F;&#x2F;generalresearch.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;generalresearch.com&#x2F;</a>
          • DrewADesign13 hours ago
            It kind of reminds me of those fake tech ads those guys put up in the NYC subway:<p><a href="https:&#x2F;&#x2F;www.instagram.com&#x2F;p&#x2F;DZacltqHMjT&#x2F;?img_index=8&amp;igsi=bTM2bjhvOHIzYnk5" rel="nofollow">https:&#x2F;&#x2F;www.instagram.com&#x2F;p&#x2F;DZacltqHMjT&#x2F;?img_index=8&amp;igsi=bT...</a>
            • inigyou9 hours ago
              I can&#x27;t see this because I don&#x27;t have an Instagram account
              • DrewADesign5 hours ago
                Yes, that’s true.<p>Of course, I didn’t put it there, and I don’t know them, so there’s not much I can do about that, right?
              • zzzzpknk9 hours ago
                replace instagram.com with kittygr.am
          • internet_points1 hour ago
            &gt; Our Customers<p>&gt; Ripe with fraud, labor exploitation, political polling manipulation; our customers demand the best tools and security for reaching global audiences and securing respondent reliability at any scale.<p>wat
          • skinfaxi14 hours ago
            Why it&#x27;s Transactional Systems for Ephemeral Tasks, don&#x27;t you know? No but same here. The other pages lead to more confusion.
            • freehorse13 hours ago
              My favourite part is the &quot;our customers&quot; section [0], starting with<p>&gt; Ripe with fraud, labor exploitation, political polling manipulation<p>I am not sure if they want to convey what I think I am reading or not.<p>[0] <a href="https:&#x2F;&#x2F;generalresearch.com&#x2F;mission&#x2F;#:~:text=Our%20Customers" rel="nofollow">https:&#x2F;&#x2F;generalresearch.com&#x2F;mission&#x2F;#:~:text=Our%20Customers</a>
              • x0xMaximus11 hours ago
                Buyer A is presented with the following rate card:<p>&lt; 10min targeting gen pop: $3 10 &lt;= 15min targeting gen pop: $5<p>Of course they&#x27;ll bid saying their 14 min survey only takes 9 min to complete. Buyers try to cheat pricing strategies of exchanges just as much as respondents try to cheat buyers on survey platforms. Both parties can&#x27;t be trusted and have adverse incentives.
          • sswaner11 hours ago
            “ General Research combines coordinated operations to identify and neutralize foreign actors using technological advancements, international covert operations, and networking analysis as an Internet Service Provider.”<p>What is so hard to understand????
          • muragekibicho11 hours ago
            Perhaps they&#x27;re targeting a super specific niche. Like somebody who sees WXET Ephemereal and yelps in triumph &quot;that&#x27;s my quant!&quot;.
          • rajamaka13 hours ago
            I think it&#x27;s a satirical website aimed at poking fun at corporate-linkedin-speak
            • x0xMaximus12 hours ago
              Wish we didn’t even need one
          • MikhailTal10 hours ago
            As usual, the real tldr is in the privacy policy <a href="https:&#x2F;&#x2F;generalresearch.com&#x2F;supplier&#x2F;privacy-policy&#x2F;notice-to-respondents&#x2F;" rel="nofollow">https:&#x2F;&#x2F;generalresearch.com&#x2F;supplier&#x2F;privacy-policy&#x2F;notice-t...</a><p>&gt; General Research Laboratories, LLC (“GRL”) is an aggregator of market research surveys in multiple marketplaces for business customers. GRL does not typically host consumer surveys which are conducted by other consumer-facing organizations....<p>So, just a middleman to give you fake&#x2F;shady at best survey responses to pad your numbers, so big enterprises&#x2F;concultancies can have data that say whatever they want. The rest of the website is just BS
            • x0xMaximus9 hours ago
              Finally you get it. Only exception is that we run our own exchange now to do task bidding so we don’t need to deal exclusively with other companies to middleman. Core business is what’s called yield management (akin to DSP in adtech) where the best survey (is the user qualified for it, does it have the best pay, etc) is selected for traffic in &lt;100ms. I’d only argue the shady companies are the ones paying proxies (like cint.com) and pushing paid user acquisition instead of surveys. We actively fund ontology development for better profiling targeting. but yes, big enterprises&#x2F;consultancies can use and interpret the collected data however they want, not our responsibility and we have no legal rights over it anyway
          • tourist2d13 hours ago
            It smells like they don&#x27;t know what they&#x27;re doing. Had a lol at the mission page accidentally implying their customers are unethical though:<p>&gt; Our Customers<p>&gt; Ripe with fraud, labor exploitation, political polling manipulation; our customers demand the best tools and security ...
            • DrewADesign12 hours ago
              At least they’re ripe! I <i>hate</i> unripe customers, rife with astringency, sharp vegetal notes, and an unpleasant, unyielding texture.
        • m3rc13 hours ago
          What&#x27;s up with the part of the website where you are talking about &quot;neutralized foreign actors&quot;, was that also using contracted Mechanical Turk labor?
          • x0xMaximus13 hours ago
            The industry is so backwards, there are residential proxy companies that are directly paid by one of the largest market research firms. I have discussed this at length <a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;Marketresearch&#x2F;comments&#x2F;1s2tl0o&#x2F;open_letter_to_cint_supplier_quality_issues&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;Marketresearch&#x2F;comments&#x2F;1s2tl0o&#x2F;ope...</a><p>It should be a bipartisan issue that a Swedish company is paying a UAE Residential Proxy company [1] to build tools that allow people from anywhere in the world to take US political polls that are used by both parties to collect election data.<p>Just for starters, you can&#x27;t even think about soliciting online work from a panel without a robust residential proxy detection methods. We had to build our own:<p>`wget -N &#x27;<a href="https:&#x2F;&#x2F;grip.net&#x2F;files&#x2F;grip-proxy-30d.mmdb" rel="nofollow">https:&#x2F;&#x2F;grip.net&#x2F;files&#x2F;grip-proxy-30d.mmdb</a>&#x27;`<p>[1] <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=eOmeQcwSK3o" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=eOmeQcwSK3o</a> flagged by Nokia Deepfield and CTRL for it&#x27;s involvement in botnets
            • parineum12 hours ago
              Polling data isn&#x27;t some guaranteed right or something. If you ask the Internet questions and then sell the collated answers, it&#x27;s your job to authenticate the responses or your polls will be wildly wrong and people will stop buying them.<p>Which is already the case and nobody but a bunch of campaign contractors who can&#x27;t justify their do nothing jobs anymore cares.
              • schoen12 hours ago
                I just remembered that in the story &quot;Simulacron-3&quot;, there is ostensibly a law requiring people to respond to opinion polls. It is illegal to refuse.<p><a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Simulacron-3" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Simulacron-3</a><p>Spoiler: gur jbeyq va juvpu gur ynj fhccbfrqyl rkvfgf vf n fvzhyngvba, perngrq ol be sbe cbyyfgref!
              • x0xMaximus12 hours ago
                Exactly, it’s a beautiful thing when the market is rational <a href="https:&#x2F;&#x2F;finance.yahoo.com&#x2F;quote&#x2F;CINT.ST&#x2F;" rel="nofollow">https:&#x2F;&#x2F;finance.yahoo.com&#x2F;quote&#x2F;CINT.ST&#x2F;</a>
            • charcircuit13 hours ago
              Someone using an IP that a geoip database says is US, doesn&#x27;t mean that person is a US voter. The existence of proxies on the internet is a feature.
              • x0xMaximus12 hours ago
                Obviously, a problem not even fully addressed by L2’s datasets. Sure, but the use of proxies to masquerade the identity of users being sold to researcher buyers that paid for a different service is fraud
                • charcircuit11 hours ago
                  It makes me question why the surveys don&#x27;t ask for the users country. That way they cd can figure it out without trying to guess it based off of IP.
                  • x0xMaximus11 hours ago
                    Users&#x2F;respondents lie, and many buyers&#x2F;researchers are neo-luddites; most C level still come from the telemarketing days. A fun example: mobile targeting is terrible when off wifi because all survey platforms uniq identify users based off their IPv4, so any T-Mobile LTE users in the same city going through the same CGNAT often share profiling data that ends up conflicting, which ends up meaning they don&#x27;t get sent into the best survey(s). The issue is even worse in heavy IPv6 countries like India+France.
                    • inigyou9 hours ago
                      The fact that you can sell a product that doesn&#x27;t work and become a billionaire proves capitalism is broken.
                      • bonoboTP4 hours ago
                        Yes. Much better to leave these decisions to the Dear Leader who is infinitely wise and cannot be misled. Or to the People&#x27;s Central Planning Committee, where they decide all the allocations and strip rich people&#x27;s wealth if they aren&#x27;t on board with the correct ideology. Such systems are non-broken and have yielded unmeasurable prosperity throughout the world wherever implemented.
        • BobbyTables211 hours ago
          That’s top-tier “OfficeSpace” thinking there…
    • testbjjl12 hours ago
      Sounds like your program manager was smart and valuable to the company. Do you have any plans to replace the service&#x2F;data provided by MTurk?
  • ge966 minutes ago
    That&#x27;s crazy I remember doing work on there a long time ago. It was sus work like manipulating Google SEO rank by looking up and clicking links. But still it was real money even if it was pennies.<p>I think the weirdest thing I had to do was scan through social media images... so weird seeing into random people&#x27;s lives.<p>I did it in the late 2000s or early 2010s.
  • donalhunt3 hours ago
    Worth noting that one of the most high profile uses of Mechanical Turk was in September 2007 to review satellite imagery in a massive crowdsourced effort to find missing record-setting aviator Steve Fossett.<p>The effort failed to find any areas of interest and the missing aviator was found the following year by a hiker. I wonder if there was any analysis after the fact to understand if the imagery actually provided any hints regarding the eventual crash site.
    • yard20101 hour ago
      &gt; Amazon&#x27;s search effort was shut down the week of October 29, without any measurable success. Major Cynthia Ryan later said it had been more of a hindrance than a help. She said that persons purporting to have seen the aircraft on the Mechanical Turk or have special knowledge clogged her email during critical days of the search, and for even months afterward. Many of the ostensible sightings proved to be images of CAP aircraft flying search grids, or simply mistaken artifacts of old images. Psychics flooded the search base in Minden with predictions of where the aviator could be found. One man from Canada was particularly persistent with daily calls to Ryan. Ryan noted that every message, letter, or phone call was taken seriously, which swamped the USAF specialists assigned the task of reviewing every one of them without regard to apparent plausibility. In retrospect, the crowdsource effort was &quot;not ready for prime time&quot;, according to Ryan<p>From Wikipedia
    • nlkingthree3 hours ago
      I wonder whether AI would have found the aviator.
      • walrus012 hours ago
        freely available satellite&#x2F;aerial images in 2007 were in no way high resolution enough to identify such a tiny airplane, and in many places, they still aren&#x27;t.
      • 21asdffdsa123 hours ago
        CIA should be good on that- find traces by man in wild nature..
  • shortformblog13 hours ago
    So, I have a story to share about Mechanical Turk that you might find interesting. I’ve shared it a couple of times before on Twitter and Bluesky, but I’ll share it again.<p>Long story short: Mechanical Turk saved my bacon.<p>Back in 2005, I was working a job at a small-town newspaper in a town I’d never lived before. I didn’t know anyone beyond the staff (as I was working layout rather than as a reporter), and I had gotten interested in the idea of doing Mechanical Turk for a few extra bucks. I thought there might be a formative scene of interested people doing this, so I started working on a blog for it. I briefly collaborated on it with another guy who put it on forum software because he didn’t know how to use, like Drupal.<p>That blog was called Turking.com, and it seemed like it was going well for a bit. We even got a mention on the AWS website. But after about two or three months, it was clear the initial excitement around the idea (and the initial work) had died down. (The work picked up later, but in clearly different ways. I don’t think folks were really “excited” about it after that point.)<p>If you want to get an idea of it, there was one capture on the Wayback Machine: <a href="https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20051124231722&#x2F;http:&#x2F;&#x2F;www.turking.com&#x2F;mturk&#x2F;" rel="nofollow">https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20051124231722&#x2F;http:&#x2F;&#x2F;www.turkin...</a><p>The site had started to die out in part because my iBook suffered a catastrophic GPU failure and me, being a broke small-town newspaper employee, did not have the money to replace it. Plus, the community just hadn’t emerged like we had expected.<p>But then I got an email out of the blue: Someone wanted to buy my domain, which I owned outright, but they didn’t want to say who. I got contacted by a broker, and the exchange took place over escrow.<p>(The domain most assuredly was bought by Amazon and is managed by MarkMonitor.)<p>I didn’t get a ton of money from the deal, but I did get enough to pay for a new laptop. I had some regrets about selling it (in part because I originally built the site with someone else), but I was in a bit of a dire financial situation which that proved to be the starting point for getting myself out of.<p>That domain purchase was notably more than I ever made from clicking and classifying random pictures, that said.
    • idiotsecant13 hours ago
      So did the other person just get hosed when you did your &#x27;ipo&#x27;?
      • shortformblog13 hours ago
        I explained the situation after the fact, and they were understanding. But certainly I admit that if I could do it again I probably would have clued them in sooner. The site had slowed down by the point this happened, so I’d describe it as a little more of a fire sale.<p>I learned a lot from that situation that I took to future sites.
    • latexr5 hours ago
      &gt; Long story short: Mechanical Turk saved my bacon.<p>Nothing wrong with your story, but that summary is terrible. You had a domain for a blog and someone bought it. That’s it. Mechanical Turk is inconsequential, as is the subject of the blog, the story would have been exactly the same if your blog had been about turkey sandwiches and Burger King offered to buy it.
      • tigerlily4 hours ago
        The guy sold the domain and was literally able to buy the laptop that got him out of his dire financial situation.
      • shortformblog4 hours ago
        I see you missed the point of the story. But whatever. I&#x27;m sorry my dire financial situation didn&#x27;t tie up in a nice enough bow for you.
        • latexr2 hours ago
          I honestly don’t see what you don’t understand about “Nothing wrong with your story”. It’s literally the start of the first sentence. It seems pretty clear to me the mild criticism is directed <i>solely</i> at the summary (you know, the quoted part), not the story.<p>Ironic that you’re accusing someone else of “missing the point”, considering.
  • ealready_value57 minutes ago
    About 8 years ago, we used mturk for reading data out of public PDFs generated by a huge range of producers. I am not sure that LLMs would have been able to consistently extract this data until recently as a good number of these PDFs were scans, sometimes a scan of scan.<p>We had pretty good luck in the end, but getting there produced a somewhat large app on our side that would manage the whole process, including but not limited to asking for multiple responses, comparing them to each other, finding consensus, and determining which users would consistently produce bad responses and stop them from responding. We got to a confidence that about 85-95% of the data was correct, which was good enough for the company.<p>Through that process, I learned a couple things about managing mturk, primarily about how changes to the cost-per-task would change the process. Initially, we thought that price would be a quality knob, but quickly learned that price was a speed know. The higher the price the faster the tasks would be taken and completed. Quality did not change significantly as the price went up or down.<p>Overall, I still have fondness to mturk, but it was really bare-bones experience that needed a lot of work to get working effectively.
  • conception14 hours ago
    It’s kind of crazy they’re shutting this down just when this Service probably has the most possibilities ever. You have an agent with Multiple people doing actual physical tasks in the real world seems like something that could be really powerful.
    • somenameforme9 hours ago
      In spite of the name I think the overwhelming majority of the tasks were things that could be done by LLMs and they&#x27;re probably getting flooded by people using bots to do exactly that. Even before the age of LLMs Mechanical Turk data was pretty bad because you&#x27;d have a bunch of people racing to answer questions as quickly as possible to get their $0.25 or whatever. So it was essentially a test of &#x27;can you input random answers as quickly as possible while paying enough attention to notice the attention question that says to mark d.&#x27;<p>Any remotely open platform for this sort of stuff is going to wrecked by LLMs. But making some sort of high trust, high verification market is going to end up sending prices high enough that it becomes an unattractive proposition for use. And even in that case it&#x27;s just going to be a cat and mouse game of people figuring out how to game the system enough to get trusted before handing it off to the LLM.
      • Ekaros3 hours ago
        Seems like whole thing is destined to fail. Buyers of services are there to exploit and pay minimum they can get away with. And other side is ready to cheat if they can to get most out of it.<p>And same dynamic will apply to most cases. Unless you actually bring it in house with strict oversight. Which is thing to avoid originally...
    • paxys13 hours ago
      The concept behind it isn&#x27;t going away. There are plenty of companies that hire people in bulk to do manual data labeling, transcription, RLHF, moderation and lots more. It&#x27;s just that they are now catering to large AI companies, not regular people looking to get some repetitive work done (since that can now mostly be done by AI).
    • neom14 hours ago
      Rentahuman is still around: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=46868675">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=46868675</a>
    • robot_jesus14 hours ago
      Right? Not to mention the training&#x2F;tuning possibilities.<p>Mercor has a market value of $20B basically doing the same thing but desperately trying to find workers.
    • dec0dedab0de14 hours ago
      that was my first thought too, but i guess llms don’t need an api to hire humans
    • Hilliard_Ohiooo14 hours ago
      [dead]
  • runamuck1 hour ago
    Funny story... an incredibly famous rapper got his start because his manager, Shane Morris used Mechanical Turk to get engagement on Spotify before Spotify began to detect that vector.
  • wanderingstan13 hours ago
    I used it to have people transcribe my dad’s handwritten letters and journals. It was touching to get notes from a “Turk” saying how much she enjoyed his travels and following the cast of characters in his life!
  • utopiah8 hours ago
    FWIW I tried it shortly after it launched so a bit after ~2005, maybe 2007 because I read about and honestly found the prospect, from a dataset creation or improvement, quite interesting. I think I heard about it from a research paper that distributed its work that way.<p>Well it did work, in the sense that I managed to complete some tasks. I don&#x27;t think I ever collected the money&#x2F;credits back then though (simply because it amounted to so little). What I can attest though is that... it was debilitating. If you think your office job is boring then splitting it in way smaller tasks where you have no autonomy is absolutely terrible from a worker standpoint.<p>I initially was hoping to use it in order to work on providing a service over a dataset but understanding first hand what it takes to make it happen made me stop. It radically changed how I saw supervised learning since, and sadly not in a good way.
  • rcr-anti13 hours ago
    Had some absolutely bizarre results from their attempt to integrate mturk with Bedrock&#x27;s &#x27;ground truth&#x27; thing as of a few months ago. Threw simple mnist digits recognition at it, see what the quality, timing, and cost was. Figured mnist digits was at this point trivial. Spent $10 and the accuracy was marginally better than guessing, completely unusable results. Was completely baffled, people were publishing peer reviewed research based on exclusively mturk results.
  • nerevarthelame1 hour ago
    A lot of the early tasks posted to Mechanical Turk by Amazon to get people using their new platform were easily automated. You could basically choose the &quot;none of the above&quot; option 100% of the time and end up with a quality score above their threshold.<p>Instead of partying in college, my friends and I wrote a script to complete the tasks. We took over university computer labs to run it on a bunch of different computers. We made a few thousand bucks. Good times.
  • mturk13 hours ago
    I&#x27;ll still be around on Oct 1.
    • rickcarlino12 hours ago
      Wishing you the best of luck on the road ahead ;-)
  • abhaynayar5 hours ago
    Interesting. I remember reading the book &quot;Life 3.0&quot; years ago, where one of the first things the fictional AGI does to make money is perform tasks on Amazon Mechanical Turk while pretending to be human.
  • thorum14 hours ago
    Prior discussion from when the service stopped accepting new customers in July: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=48803886">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=48803886</a>
  • smalltorch14 hours ago
    I did this a long time ago and it actually paid for a few meals.<p>They are terribly monotonous tasks.<p>I can see why it&#x27;s shutting down if its still the same thing.<p>Lllms could probably do everything there without rotting out minds for basically pennies.
    • Forgeties7913 hours ago
      Instead we pay to rot our minds lol
  • dubeye4 hours ago
    I&#x27;ve used Mechanical Turk in the early days but developed my own version for my vertical to manage freelancers.<p>The problem as I see it is verifying the work is not done by AI. If it was possible to verify somehow that the work is definitely not done by AI I think there would still be a market for this. But the data just cannot be trusted.<p>An increasingly big part of my job is to try and weed out workers who use AI and it&#x27;s not an easy task at scale because generally workers gain trust with manual work and then there is degradation over time.
    • inemesitaffia4 hours ago
      Outsourcing shops?<p>Proctoring?<p>Locked down devices?
      • dubeye4 hours ago
        I suppose it depends on the value of the individual task. I can&#x27;t imagine it&#x27;s really realistic to lock down devices of mechanical Turk workers in such a way that ai use can be monitored.<p>In any case Locking down devices isn&#x27;t really pragmatic at scale for relatively low value tasks, and also introduces complexity about employment law where I am.
  • akshay_akula56 minutes ago
    End of an era. Funny timing too, right when agents plus humans doing real world tasks would have made it interesting again.
  • 1970-01-011 hour ago
    I don&#x27;t see this as anything but good news about AI progress, yet I can&#x27;t help but think there will be spins about &quot;AI taking away thousands more tech jobs&quot;
  • shermantanktop12 hours ago
    The tagline for MTurk used to be “artificial artificial intelligence.”<p>It appears now we can get along with just a single “artificial.”
    • inigyou9 hours ago
      No &quot;intelligence&quot; required
  • transitorykris11 hours ago
    The only time I ever used it (as a turk?) was when Jim Gray went missing at sea and satellite imagery of vast regions of the pacific were fed through Mechanical Turk. Seems a trivial problem now, but was not then, and needed humans to take a look.
  • jp00012 hours ago
    Not surprised and I’d like to see the effects of LLMs on offshoring of jobs to India. I’ve seen efforts to reduce headcount there after the release of Opus-4.6.
  • somepleb8 hours ago
    Reminds me of Expensify&#x27;s usage of Amazon Mechanical Turk.<p><a href="https:&#x2F;&#x2F;arstechnica.com&#x2F;information-technology&#x2F;2017&#x2F;11&#x2F;expensify-acknowledges-potential-privacy-problem-by-calling-it-a-feature&#x2F;" rel="nofollow">https:&#x2F;&#x2F;arstechnica.com&#x2F;information-technology&#x2F;2017&#x2F;11&#x2F;expen...</a>
  • cmiles88 hours ago
    Amazon &#x2F; AWS went from boasting about how they never killed off products to just going on a steady binge of shutting things down. That wouldn’t necessarily be bad if they were replacing this with better stuff, but most of the new launches also seem on a path to eventual culling too.
  • ghaff10 hours ago
    Absent visibility into AWS P&amp;Ls--even notwithstanding AI--Mechanical Turk has been a real outlier at AWS for a number of years now, so not really surprising.
  • logotype5 hours ago
    MTurk existed since forever, and somehow flew under the radar for all these years. Very few developers I know didn&#x27;t even know it existed.
  • winterbourne9 hours ago
    I was just thinking about the good old days combining human judgements on mturk for classifying high-value forum threads. Since replaced by using judgements from multiple LLMs.<p>The only use I can think of it these days is for social scientists who need to gather judgements from bona fide humans.<p>This will be devastating for a lot of workers in low-infrastructure countries.
  • mips_avatar13 hours ago
    Surprised Amazon managed to fumble mechanical turk at the same time Mercor&#x2F;Scale and all these other companies started hiring humans to do data labeling tasks.
  • willmeyers13 hours ago
    I wouldn&#x27;t be surprised if Fiverr and Upwork start pulling back too. There&#x27;s just so much AI bots now that have flooded the task marketplace.
  • wileydragonfly13 hours ago
    I made a couple thousand dollars off it squeezing in tasks here and there between meetings. Amazon Payments were challenging to redeem as I recall. I did more or less write the bulk of someone’s doctoral thesis. Kept giving me a dollar to summarize the findings of various psychology papers. The most memorable was the one where people were put in a room and someone sprayed “liquid ass” on the wall. The subjects given no explanation experienced higher levels of anxiety than those that were told there was a sewage leak being repaired. One of the strangest dollars I ever made. My PS4 and game collection was spectacular.
  • cautiouscat14 hours ago
    I was literally wondering this morning if mturk was still around! I assumed a lot of the stuff could just be done with LLMs and OCI now.
  • zackmorris3 hours ago
    Of course the moment AI can be used to earn a residual income from Mechanical Turk, they shut it down.<p>Incentives work both ways. If the exchange rate of human labor to capital falls to zero, then human workers will have no choice but to exclude capital from the equation.<p>No jobs, no money, no billionaires.
  • HlessClaudesman5 hours ago
    Cut off one mechanical turk and two more will rise to take it&#x27;s place.
  • GeoBounties7 hours ago
    great news for us at geobounties! we allow ai agents to deploy humans for real world tasks. a true mechanical turk! : ) now your ai assistant can even get your coffee for you! (sorry Peggy Olson!)
  • aboardRat414 hours ago
    End of an era.
    • tibbon13 hours ago
      Humans were once useful!
  • tempfile5 hours ago
    Evil business, glad to be rid of it.
  • pulkas6 hours ago
    mission completed
  • qarl214 hours ago
    I wonder why.
  • xyst13 hours ago
    bezos has determined LLM has replaced &quot;on demand 24&#x2F;7 workforce&quot;
    • dgellow6 hours ago
      (He’s not CEO anymore)
  • fetchmark7 hours ago
    [flagged]
  • PrimeAli11 hours ago
    [flagged]
  • retr0rocket12 hours ago
    [dead]
  • indianwashlets56 minutes ago
    [flagged]
  • OkWellBye14 hours ago
    [flagged]