DISARMframeworks/generated_pages/techniques/T0097.208.md

9.0 KiB
Raw Blame History

Technique T0097.208: Social Cause Persona

  • Summary: Online accounts which present themselves as focusing on a social cause are presenting the Social Cause Persona. Examples include accounts which post about current affairs, such as discrimination faced by minorities.

    While presenting as an account invested in a social cause is not an indication of inauthentic behaviour, such personas have been used by threat actors to exploit peoples legitimate emotional investment regarding social causes that matter to them (T0143.002: Fabricated Persona, T0097.208: Social Cause Persona).

    Legitimate accounts focused on a social cause could use their persona for malicious purposes, or be exploited by threat actors (T0143.001: Authentic Persona, T0097.208: Social Cause Persona). For example, the account holders could take money for using their position to provide legitimacy to a false narrative, or be tricked into doing so without their knowledge.

    Associated Techniques and Sub-techniques:
    T0097.103: Activist Persona: Analysts should use this sub-technique to catalogue cases where an individual is presenting themselves as an activist related to a social cause. Accounts with social cause personas do not present themselves as individuals, but may have activists controlling the accounts.

  • Belongs to tactic stage: TA16

Incident Descriptions given for this incident
I00074 The Tactics & Tropes of the Internet Research Agency “The Black Matters Facebook Page [operated by Russias Internet Research Agency] explored several visual brand identities, moving from a plain logo to a gothic typeface on Jan 19th, 2016. On February 4th, 2016, the person who ran the Facebook Page announced the launch of the website, blackmattersus[.]com, emphasizing media distrust and a desire to build Black independent media; [“I DIDNT BELIEVE THE MEDIA / SO I BECAME ONE”]”

In this example an asset controlled by Russias Internet Research Agency began to present itself as a source of “Black independent media”, claiming that the media could not be trusted (T0097.208: Social Cause Persona, T0097.202: News Outlet Persona, T0143.002: Fabricated Persona).
I00114 Carols Journey: What Facebook knew about how it radicalized users This report examines internal Facebook communications which reveal employees concerns about how the platforms algorithm was recommending users join extremist conspiracy groups.

In summer 2019, a new Facebook user named Carol Smith signed up for the platform, describing herself as a politically conservative mother from Wilmington, North Carolina. Smiths account indicated an interest in politics, parenting and Christianity and followed a few of her favorite brands, including Fox News and then-President Donald Trump.

Though Smith had never expressed interest in conspiracy theories, in just two days Facebook was recommending she join groups dedicated to QAnon, a sprawling and baseless conspiracy theory and movement that claimed Trump was secretly saving the world from a cabal of pedophiles and Satanists.

Smith didnt follow the recommended QAnon groups, but whatever algorithm Facebook was using to determine how she should engage with the platform pushed ahead just the same. Within one week, Smiths feed was full of groups and pages that had violated Facebooks own rules, including those against hate speech and disinformation.

Smith wasnt a real person. A researcher employed by Facebook invented the account, along with those of other fictitious “test users” in 2019 and 2020, as part of an experiment in studying the platforms role in misinforming and polarizing users through its recommendations systems.

That researcher said Smiths Facebook experience was “a barrage of extreme, conspiratorial, and graphic content.”


Facebooks Algorithm suggested users join groups which supported the QAnon movement (T0151.001: Social Media Platform, T0151.002: Online Community Group, T0153.006: Content Recommendation Algorithm, T0097.208: Social Cause Persona).

Further investigation by Facebook uncovered that its advertising platform had been used to promote QAnon narratives (T0146: Account Asset, T0114: Deliver Ads, T0153.005: Online Advertising Platform):

For years, company researchers had been running experiments like Carol Smiths to gauge the platforms hand in radicalizing users, according to the documents seen by NBC News.

This internal work repeatedly found that recommendation tools pushed users into extremist groups, findings that helped inform policy changes and tweaks to recommendations and news feed rankings. Those rankings are a tentacled, ever-evolving system widely known as “the algorithm” that pushes content to users. But the research at that time stopped well short of inspiring any movement to change the groups and pages themselves.

That reluctance was indicative of “a pattern at Facebook,” Haugen told reporters this month. “They want the shortest path between their current policies and any action.”

[...]

By summer 2020, Facebook was hosting thousands of private QAnon groups and pages, with millions of members and followers, according to an unreleased internal investigation.

A year after the FBI designated QAnon as a potential domestic terrorist threat in the wake of standoffs, alleged planned kidnappings, harassment campaigns and shootings, Facebook labeled QAnon a “Violence Inciting Conspiracy Network” and banned it from the platform, along with militias and other violent social movements. A small team working across several of Facebooks departments found its platforms had hosted hundreds of ads on Facebook and Instagram worth thousands of dollars and millions of views, “praising, supporting, or representing” the conspiracy theory.

[...]

For many employees inside Facebook, the enforcement came too late, according to posts left on Workplace, the companys internal message board.

“Weve known for over a year now that our recommendation systems can very quickly lead users down the path to conspiracy theories and groups,” one integrity researcher, whose name had been redacted, wrote in a post announcing she was leaving the company. “This fringe group has grown to national prominence, with QAnon congressional candidates and QAnon hashtags and groups trending in the mainstream. We were willing to act only * after * things had spiraled into a dire state.”

While Facebooks ban initially appeared effective, a problem remained: The removal of groups and pages didnt wipe out QAnons most extreme followers, who continued to organize on the platform.

“There was enough evidence to raise red flags in the expert community that Facebook and other platforms failed to address QAnons violent extremist dimension,” said Marc-André Argentino, a research fellow at Kings College Londons International Centre for the Study of Radicalisation, who has extensively studied QAnon.

Believers simply rebranded as anti-child-trafficking groups or migrated to other communities, including those around the anti-vaccine movement.

[...]

These conspiracy groups had become the fastest-growing groups on Facebook, according to the report, but Facebook wasnt able to control their “meteoric growth,” the researchers wrote, “because we were looking at each entity individually, rather than as a cohesive movement.” A Facebook spokesperson told BuzzFeed News it took many steps to limit election misinformation but that it was unable to catch everything.
I00128 #TrollTracker: Outward Influence Operation From Iran [Meta removed a network of assets for coordinated inauthentic behaviour. One page] in the network, @StopMEK, was promoting views against the Peoples Mujahedin of Iran (MEK), the largest and most active political opposition group against the Islamic Republic of Iran Leadership.

The content on the page drew narratives showing parallels between the Islamic State of Iraq and Syria (ISIS) and the MEK.

Apart from images and memes, the @StopMEK page shared a link to an archived report on how the United States was monitoring the MEKs movement in Iran in the mid-1990s. The file was embedded as a QR code on one of the pages images.


In this example a Facebook page presented itself as focusing on a political cause (T0097.208: Social Cause Persona, T0151.001: Social Media Platform, T0151.002: Online Community Group). Within the page it embedded a QR code (T0122: Direct Users to Alternative Platforms, T0153.004: QR Code Asset), which took users to a document hosted on another website (T0152.004: Website Asset).
Counters Response types

DO NOT EDIT ABOVE THIS LINE - PLEASE ADD NOTES BELOW