More than 140 Kenya Facebook moderators sue after diagnoses of PTSD

Even if 1% of images are disturbing that’s multiple per hour, let anyone across months.

US workman’s comp covers PTSD acquired on the job, and these kinds of jobs are rife with it.

> They are there more as a political org so that facebook can say: "We employ 140 moderators in Kenya alone!" while they do indifferent work that facebook already can filter out.

Why assume they're just token diversity hires who don't do useful work..?

Have you ever built an automated content moderation system before? Let me tell you something about them if not: no matter how good your automated moderation tool, it is pretty much always trivial for someone with familiarity with its inputs and outputs to come up with an input it mis-predicts embarrassingly badly. And you know what makes the biggest difference.. is humans specifying the labels.

Content moderation also involves reading text, so you’d imagine that there’s a benefit to having people who can label data and provide ground truth in any language you’re moderating.

Even with images, you can have different policies in different places or the cultural context can be relevant somehow (eg. some country makes you ban blasphemy).

Also, I have heard of outsourcing to Kenya just to save cost. Living is cheaper there so you can hire a desk worker for less. Don’t know where the insistence you’d only hire Kenyans for political reasons comes from.

>Also a doctor is paid $$$$$

>Content moderator is a min wage job

So it's purely a monetary dispute?

>bad working hours, no psychological support, and you spend your day looking at rape, child porn, torture and executions.

Many other jobs have the same issues, though admittedly with less frequency, but where do you draw the line?

I assumed that, at the margin, Meta would prioritise reducing false negatives. In other words, they would prefer that as many legitimate posts are published as possible.

So the things that are flagged for human review would be on the boundary, but trend more towards disturbing than legitimate, on the grounds that the human in the loop is there to try and publish as many posts as possible, which means sifting through a lot of disturbing stuff that the AI is not sure about.

There’s also the question of training the models - the classifiers may need labelled disturbing data. But possibly not these days.

However, yes, I expect the absolute most disturbing shit to never be seen by a human.

—

Again, literally no experience, just a guy on the internet pressing buttons on a keyboard.

I watch surgery videos sometimes, out of fascination. It's not gore to me - sure it's flesh and blood but there is a person whose life is going to be probably significantly better afterwards. They are also not in pain.

I exposed myself to actual gore vids in the aughts and teens... That stuff still sticks with me in a bad way.

Context matters a lot.

I agree. But that might be comorbid with PTSD. It’s probably not good for you to be _that_ desensitised to this sort of thing.

I also feel like there’s something intangible regarding intent that makes moderation different from being a doctor. It’s hard for me to put into words, but doctors see gore because they can hopefully do something to help the individual involved. Moderators see gore but are powerless to help the individual, they can only prevent others from seeing the gore.

Desensitization is only one stage of it. It's not permanent & requires dissociation from reality/humanity on some level. But that stuff is likely to come back and haunt one in some way. If not, it's likely a symptom of something deeper going on.

My guess is that's why it's after bulldozing hundreds of Palestinians, instead of 1 or 10s of them, that Israeli soldiers report PTSD.

If you haven't watched enough videos of the ongoing genocides in the world to realize this, it'll be a challenge to have a realistic take on this article.

I might have misremembered that, but remember hearing the story. Maybe it was only after unsuccessful CPR attempts or something like that.

Maybe it's different in the US where ambulances cost money, but here in Germany the typical paramedic will see a wide variety of cases, with the vast majority of patients surviving the encounter. Giving your paramedic a day off after witnessing a death wouldn't break the bank. In the ICU or emergency room it would be a different story.

Maybe a different country than yours ?

Yes, they will be much more able to deal with the consequences of that trauma than someone who gets a pittance to do the same thing. A low-wage peon won't even be able to afford therapy if they need it.

At least they can pay for therapy and afford to stop working or find another job

I've lived places where the cops make $100k+. It all depends on location.

Frankly

there’s a difference between a one or two or even ten off exposure to the brutality of life, where various people in your life will support you and help you acclimate to it

Versus straight up mainlining it for 8 hours a day

hey kid, hope you're having a good life. I'll look at the screen full of the worst the internet and humanity has produced on the internet for eight hours.

I get your idea but in the context of this topic I think you're overreaching

Actually reckoning with this stuff leads people into believing in anti-natalism, negative utilitarinism, Scopenhaur/Philipp Mainlander (Mainlander btw was not just pro-suicide, he actually killed himself!), and the voluntary extinction movement. This terrified other philosophers like Nietzsche, who spends most of his work defending reality even if it's absolute shit. "Amor Fati", "Infinite Regress/Eternal Recurrence", "Übermensch" vs the literal "Last Man". "Wall-E" of all films was the modern quintessential nietzschian fable, with maybe "Children of Men" being the previous good one before that.

You're literally not allowed to acknowledge that this stuff is bad and adopt one of the religions that see this and try to remove suffering - i.e. Jainism, because at least historically doing so meant you couldn't use violence in any circumstances, which also meant that your neighbor would murder you. There's a reason that Jain's population are in the low millions

Reality is actually bad, and it should be far more intuitive to folks. The fact that positive experience is felt "quickly" and negative experience is felt "slowly" was all the evidence I needed that I wouldn't just press the "instantly and painlessly and without warning destroy reality" (benevolent world-exploder) button, I'd smash it!

... okay.

Emergency personnel might need to braze themselves for car accidents every day. That Kenyans need to be traumatized by Internet Content in order to make a living is just silly and unnecessary.

I spent my civil service as a paramedic assistent at the countryside, close to a mountainroad that was very popular with bikers. I was never interested in motorbikes in the first place, but the gruesome accidents I've witnessed turned me off for good.

You’re only about 20x as likely to die on a motorcycle as in a car.

What can I say? People like to live dangerously.

Watch how a group of wild dogs kill their prey, then realise that for milenia human like apes were part of their diet. Even the modern battlefield is more humane than the African savannah.

>Crossbows can increase this to ~400 J.

Funny you mention crossbows; the Church at one point in time tried to ban them because they democratized violence to a truly trivial degree. They were the nuclear bombs and assault rifles of medieval times.

Also, I will take this moment to also mention that the "problem" with weapons always seem to be how quickly they can kill rather than the killing itself. Kind of takes away from the discussion once that is realized.

Humans can render other humans unrecognizable with a rock.

Brutal murder is low tech.

That’s an excellent way to put it, resonates with my (non medical) experience. It’s the emotional stuff that will try to follow me around and be intrusive.

I won’t watch most movies or TV because they are just some sort of tragedy porn.

I'd be asking "how bad is the fentanyl situation in your are?"

It's also super easy to come up with better questions: "What's the funniest call you've ever been on?" "What call do you feel like you made the biggest difference?" "What's the best story you have?"

I get more upset watching people lightly smack and yell at each other on public freakout than I do watching people die. It's not that I don't care about the dead either, I watched wpd and similar sites for years. I didn't enjoy watching it, but I liked knowing the reality of what was going on in the world, and how each one of us has the capacity to commit these atrocities. I'm still doing a lousy job at describing why I like to watch it. But I do.

That's the tragic consequence of being human. Either the people you care about leave first or you do, but in the end, everyone goes. We are blessed and cursed with the knowledge to understand this. We should try to maximize the time we spend with those that are important to us.

Well, i think it goes to a point. I'd imagine there's some goldilocks zone of time spent with the animal, care experienced from the animal, dependence on the animal, and manner/speed of death/ time spent watching the thing die.

I say animal to explicitly include humans. Finding my hamster dead in fifth grade did change me. But watching my mother slowly die a horrible, haunting death didn't make me a better person. I'm just saying that there's a spectrum that goes something like: easy to forget about, I'm able to not worry, sometimes i think about it when i dont want, often i think about it, often it bothers me, and do on. You can probably imagine the cycle of obsession and stress.

This really goes for all traumatic experiences. There's a point where they can make you a better person, but there's a cliff after which you have no guarantees that it won't just start obliterating you and your life. It's still a kind of perspective. But can you have too much perspective? Lots of times i feel like i do

My brother, an Eastern-European part-time farmer and full-time lorry driver, just texted me a couple of hours ago (I had told him I would call him in the next hour) that he might be with his hands full of meat by that time as “we’ve just butchered our pig Ghitza” (those sausages and piftii aren’t going to get made by themselves).

Now, ask a laptop worker to butcher an animal whom used to have a name and to literally turn its meat into sausages and see what said worker’s reaction would be.

What is it about partaking in or witnessing the killing of animals or humans that makes one more connected to reality?

> Footage of people being shot is actually allowed on many platforms.

It's also part of almost every American cop and military show and movie. Of course it's not real but it looks the same.

Many (all?) Japanese schools don't have janitors. Instead students clean on rotation. Never been much into Japanese stuff but I absolutely admire this about their culture, and imagine it's part of the reason that Japan is such a clean and at least superficially respectful society.

Living in other Asian nations where there are often defacto invisible caste systems can be nauseating at times - you have parents that won't allow their children to participate in clean up efforts because their child is 'above handling trash.' That's gonna be one well adjusted adult...

You can thank IRL privacy activists for the lack of cameras in every room in each house; Just imagine how much faster domestic disputes could be resolved!

Unsurprising lack of response to this statement. It's 100% true, and any cost-benefit calculation of privacy should account for it.

Rubbish. The reason Facebook doesn't want to demand ID for most users is that it adds friction to using their product, which means fewer users and less profit.

Introducing a free unlimited hosting service where you could only upload pictures, text or video. There’s a public page to see that content among adds and links to you friends free hosting service pages. TOS is a give-give: you give them the right to extract all the aggregated stat they want and display the adds, they give you the service for free so you own you content (and are legally responsible of it)

Isn't that forcing? Who decides how much censorship people can voluntarily opt into?

If given control, I think many/most people would opt into a significant amount of censorship.

No amount? So you wouldn't accept a job to moderate Facebook for a million dollars a day? If you would, then surely you would also do it for a lower number. There is an equilibrium point.

The same is true for poverty and the poor that will work for any amount, the cheap labor the rich needs to make riches.

Of course. Not all at once, but gradually over time like the EU has begun to do. If capital and goods are free to move, then so must labor be. The labor market is very far from free if you think about it.

vegans die of malnutrition.

There's no ethical consumption under capitalism.

Artificially creating local market rates by trapping people is.

That's only exploitation if you combine it with fact of the enclosure of the commons and that all land and productive equipment on Earth is private or state property and that it's virtually impossible to just go farm or hunt for yourself without being fucked with anymore, let alone do anything more advanced without being shut down violently.

Sure, but who did the proactive flagging back then? Probably moderators. Seems like a shitty job nonetheless

As far as I can tell, Facebook is still terrible.

I have a throwaway Facebook account. In the absence of any other information as to my interests, Facebook thinks I want to see flat earth conspiracy theories and CSAM.

When I report the CSAM, I usually get a response that says "we've taken a look and found that this content doesn't go against our Community Standards."

Then what is freedom of speech if every plattform deletes your content? Does it even exist? Facebook and co. are so ubiquitous, we shouldn't just apply normal laws to them. They are bigger than governments.

The solution to most social media problems in general is:

`select * from posts where author_id in @follow_ids order by date desc`

At least 90% of the ills of social media are caused by using algorithms to prioritize content and determine what you're shown. Before these were introduced, you just wouldn't see these types of things unless you chose to follow someone who chose to post it, and you didn't have people deliberately creating so much garbage trying to game "engagement".

Because edge cases exist, and it isn't worth it for a company to hire enough staff to deal with them when one user with a problem, even if that problem is highly impactful to their life, just doesn't matter when the user is effectively the product and not the customer. Once the AI works well enough, the staff is gone and the cases where someone's business or reputation gets destroyed because there are no ways to appeal a wrong decision by a machine get ignored. And of course 'the computer won't let me' or 'I didn't make that decision' is a great way for no one to ever have to feel responsible for any harms caused by such a system.

Yes. As opposed to running it on their servers like they do now.

And it was only for iCloud synced photos.

My understanding was the FP risk. Everything was on device. People designed images that were FPs of real images.

> Also, with the advent of GenAI, CSAM has been redefined to include generated imagery that does not contain any of {children, sex, abuse}

It hasn’t been redefined. The legal definition of it in the UK, Canada, Australia, New Zealand has included computer generated imagery since at least the 1990s. The US Congress did the same thing in 1996, but the US Supreme Court ruled in the 2002 case of Ashcroft v Free Speech Coalition that it violated the First Amendment. [0] This predates GenAI because even in the 1990s people saw where CGI was going and could foresee this kind of thing would one day be possible.

Added to that: a lot of people misunderstand what that 2002 case held. SCOTUS case law establishes two distinct exceptions to the First Amendment – child pornography and obscenity. The first is easier to prosecute and more commonly prosecuted; the 2002 case held that "virtual child pornography" (made without the use of any actual children) does not fall into the scope of the child pornography exception – but it still falls into the scope of the obscenity exception. There is in fact a distinct federal crime for obscenity involving children as opposed to adults, 18 USC 1466A ("Obscene visual representations of the sexual abuse of children") [1] enacted in 2003 in response to this decision. Child obscenity is less commonly prosecuted, but in 2021 a Texas man was sentenced to 40 years in prison over it [2] – that wasn't for GenAI, that was for drawings and text, but if drawings fall into the legal category, obviously GenAI images will too. So actually it turns out that even in the US, GenAI materials can legally count as CSAM, if we define CSAM to include both child pornography and child obscenity – and this has been true since at least 2003, long before the GenAI era.

[0] https://en.wikipedia.org/wiki/Ashcroft_v._Free_Speech_Coalit...

[1] https://www.law.cornell.edu/uscode/text/18/1466A

[2] https://www.justice.gov/opa/pr/texas-man-sentenced-40-years-...

My understanding was the FP risk. The hashes were computed on device, but the device would self-report to LEO if it detects a match.

People designed images that were FPs of real images. So apps like WhatsApp that auto-save images to photo albums could cause people a big headache if a contact shared a legal FP image.

Weird take. The point of on-device scanning is to enable E2EE while still mitigating CSAM.

> they don’t own to report said devices to police for having illegal files on them

They do this today. https://www.apple.com/child-safety/pdf/Expanded_Protections_...

Every photo provider is required to report CSAM violations.

Already happened/happening. I have an ex-coworker that left my current employer for my state's version of the FBI. Long story short, the government has a massive database to crosscheck against. Often times, the would use automated processes to filter through suspicious data they would collect during arrests.

If the automated process flags something as a potential hit, then they, the humans, would then review those images to verify. Every image/video that is discovered to be a hit is also inserted into a larger dataset as well. I can't remember if the Feds have their own DB (why wouldn't they?), but the National Center for Missing and Exploited Children run a database that I believe government agencies use too. Not to mention, companies like Dropbox, Google, etc.. all has against the database(s) as well.

Not what I was saying. I'm questioning the need for the thing entirely.

Uhh no I'm not giving up my privacy because a few people want to misbehave. Screw that. My friends know who I am but the social media companies shouldn't have to.

Also, it'll make social media even more fake than it already is. Everyone trying to be as fake as possible. Just like LinkedIn is now. It's sickening, all these people toting the company line. Even though they do nothing but complain when you speak to them in person.

And I don't think it'll actually solve the problem. People find ways to get through the validation with fake IDs.

These accounts like yours and this report of PTSD don't line up. Both of them are credible. What's driving them crazy but not Old Internet vets?

Could it be:

  - the fact that moderators are hired and paid  
  - that kids are young and a lot more tolerant  
  - that moderators aren't intended audiences  
  - backgrounds, sensitivity in media at all  
  - the amount, of disturbing images  
  - the amount, in total, not just bad ones  
  - anything else?

Doesn't it sound a bit too shallow and biased to determine it was specifically CSAM or whatever specific type of data that did it?

Did your parents know what you were seeing? Advice to others to not have kids see this kind of stuff, let alone get desensitized to it?

What drew you to 4chan?

The point is that you don't know which one will stick. Even people who are desensitized will remember certain things, a person's facial expression or a certain sound or something like that, and you can't predict which one will stick with you.

That fucking guy torturing monkeys :(

I’m not sure what is behind your assumption, if it’s the Autistic people does not have empathy myth, please read up on the topic.

I'd wager they'd still have PTSD, but wouldn't be able to communicate it as well as a normal person.

What you really want is AI doing this job. Or psychopaths/unempathetic people if that's not an option.

I never said Facebook is the responsible party. I’m saying these workers deserve our sympathy and I’m saying it’s not a case of people who had a simple choice but willingly chose a job that caused them PTSD.

I don’t think Facebook is blameless though. They practically brag about their $40B of AI spend per year and absolutely brag about how advanced their AI is. You can’t focus some of your R&D to flag content that’s instantly recognizable as disgusting content, like pedophilia, necrophilia, and beastiality? There’s already a ton of pre-labeled data they can use from all these workers. No, they don’t get a pass on that. I think it’s shameful they focus all their AI compute and engineering on improving targeted ads and not put a major focus on solving this specific problem that’s directly hurting so many people.

Maybe the solution is that Facebook shouldn't exist. It solves both the problem of distribution and the problem of moderation.

Exploitation nearly always involves paying. Plenty of people caught up in sex trafficking still get paid, they just don't have a viable way out. Plenty of people working in sweat shops still get paid, but again not enough with enough viable alternatives to get out.

[dead]

> Perhaps if looking at pictures of disturbing things on the internet gives you PTSD than this isn’t the kind of job for you?

Perhaps these are jobs people are forced to do because the price of labour isn't as rich as other countries, trafficked and the likes.

> I know lots of people who can and do look at horrible pictures on the internet and have been doing so for 20+ years with no ill effects.

Looking at is different to moderating. I've sen my fair share of snuff from the first iraqi having their head cut off in 2005 all the way down to: ogrish/liveleak, goatse, tubgirl, 2girls1cup shock sites.

But when you are faced with imagery of gruesome material day-in day-out on 12-hour shifts if not longer non-stop, being paid very little, it would take a toll on anyone.

I've done it, lone-wolf sysop for a adult dating website for two years and the stuff I saw was moderate but still made me feel mentally disturbed. The normality wears off very quickly.

Could you work a five days week looking at extreme obscenity imagery for $2 an hour?

> Switch to a federated social systems like Mastodon, with only a few thousand or ten thousand users per instance, and perpetrators will never be able to grow too large.

The #2 and #3 most popular Mastodon instances allow CSAM.

Tricky. It also gives perpetrators a lot more places to hide. I think the jury is out on whether a few centralized networks or a fediverse makes it harder for attackers to reach potential targets (or customers).

Tech billionaires usually advocate for some kind of return to the gilded age, with minimal workers rights and corporate tax. Markets were freer back then, how did that work out for the average man? Markets alone don't do anything for the average quality of life.

Not defending this person in particular, but you should take a look at how anti-LGBT, including at the government level, most countries are in Africa. Maybe a decent number of them do regard seeing homosexuality as PTSD inducing.

There are several places where they legally can and will kill you for homosexuality.

[flagged]

[flagged]

PTSD is what happens when you see someone standing next to you reduced to a chunky red salsa in a split second.

The idea that seeing images of that can match the real thing can only be said by people who haven't smelled the results.

It is exactly what I mean.

If you don't nurture that outrage every day than you'd be rather surprised what can happen to a culture in a single generation.