Emerging From the Shadows: Shadow Banning and Misinformation
12-minute read
Recent state and federal laws have been proposed to target misinformation and filter bubbles on social media platforms. But another technique that some suggest could combat these problems is referred to as “shadow banning”.
What is shadow banning exactly? That’s the thing: it remains incredibly mysterious, as the practice is hidden from the public by social media tech giants.
Shadow banning might actually work to combat misinformation and other harmful online content, but more transparency is needed to determine its practical and ethical implications.
Amid the black leather and stained wood of the Dirksen Senate Office Building in Washington, D.C., Republican Senator Ted Cruz directed a salvo of questions at Twitter over the controversial practice that he referred to as “shadow banning.” Undoubtedly sweating under his suit and tie, Twitter’s Director of Public Policy and Philanthropy, Carlos Monje Jr., met the question with a succinct response, “No, sir, we do not shadow ban users.”
This 2018 hearing was in response to growing concerns from conservatives over social media platforms censoring Republican users’ content without notifying them. Later that year, Twitter CEO Jack Dorsey would testify to Congress reiterating, “…we do not shadow ban anyone based on political ideology. In fact, from a simple business perspective and to serve the public conversation, Twitter is incentivized to keep all voices on the platform.” These moments took the practice of shadow banning from relative online obscurity and thrust it into public political discussion. Recently, the way social media companies treat content posted on their platforms has been at the center of both state and federal legislation. Shadow banning is one practice that’s thrown around, either as an unfair restriction on free speech or a noble tool to combat misinformation, (depending on who you ask). But it's not only the practice’s ethics that have been called into question. Many have cast doubt over the very existence of shadow banning.
Shadow banning is the practice of blocking (or partially blocking) a user from a social media site without their knowledge so that it’s not readily apparent to the user that they have been banned. Its usage is controversial for two main reasons. Firstly, it’s viewed as an unethical method of restricting user behavior because it is done surreptitiously, which cedes more power to platforms to make unilateral choices on content restrictions. At the heart of this concern is the idea that social media platforms function as public forums and are key to human communication. If social media platforms function as “public squares,” we come to expect ideological neutrality from them. So, when shadow banning is used on certain ideological groups and not others, it elicits fears of censorship and marginalization. Controversy also stems from a more fundamental question: is shadow banning implemented on any platforms in the first place? While Twitter has denied using shadow banning as a moderation technique, most other social media platforms remain similarly silent on the topic.
Even defining shadow banning becomes increasingly difficult once platforms weigh in on the methods they use to display content. Claims of shadow banning have been recast as definitional misunderstandings. This was the case when Twitter was accused of shadow banning conservative accounts by removing the auto-population feature when people searched for them (yet not removing the accounts or their content at all). Twitter denied that this was shadow banning and explained it as a kind of content filtering.
Shadow banning is also often conflated with “deboosting,” a practice whereby platforms de-prioritize user content in different — and often subtle — ways. An example of deboosting was alleged by individuals who claimed that Facebook deprioritized certain livestreams on its platform by not providing users with notifications about or access to the streams. These individuals suggested Facebook limited videos’ visibility and discriminated on political grounds. Facebook denies that they implement deboosting. But allegations such as these nonetheless compound the definitional confusion surrounding shadow banning. Does neglecting to notify users that someone is beginning a livestream mean the streamer has been shadow banned? What about when accounts are technically prevented from auto-populating in the search bar?
The term shadow banning suffers from ambiguity, but seems primarily to be used when platform limitations are applied in ways that are discriminatory, (meaning that they are enforced against certain users but not others). Social media platforms are likely to market any “deprioritizing” of content for certain users as an innocuous design decision, rather than calling it shadow banning. Nonetheless, social media practices that restrict the visibility of users’ accounts or intentionally decrease exposure to a users’ content can be considered shadow banning. The more obvious cases are those in which specific user accounts have their content rendered inaccessible to audiences that the platform would normally allow them to reach.
Dorsey’s testimony did not concede that Twitter participated in “shadow banning” but he did admit to their algorithm unfairly filtering around 600,000 user accounts from auto-populating in the search bar. This way of decreasing account visibility was driven by algorithmic decisions that were designed to filter out spam. Automatic practices like this drive most of the content moderation for online spam, and companies often claim that alleged shadow bans are accidental overreach of algorithmic content filtering. The curated and filtered nature of platforms gives them a defensible rationale for why they might unfairly shadow ban user content: “It was the algorithm’s fault”. Blame aside, it’s important to note that the logic of shadow banning allows social media platforms to sidestep confrontation with users. This lack of confrontation only incentivizes platforms to use shadow banning as a surreptitious tool.
Shadow banning, when it has been admitted to, is justified as a way of curbing online misinformation by fighting spam accounts, bots, and other types of posting that are considered platform misuse. Bots and spam accounts traffick in a variety of misinformation, from political propaganda to conspiracies. There are two overarching theories behind why shadow banning might be deployed. One is to “quarantine” a user to a smaller set of social nodes. When Reddit banned an anti-vaccine and anti-government subreddit called “r/NoNewNormal”, they quarantined the posts on the subreddit from being viewable on the front page where it could be seen by users not already affiliated with the community. This theoretically isolated the content posted in the community and prevented it from gaining additional traction, while still allowing the “r/NoNewNormal” community to interact within itself. Shadow bans also try to prevent cycles of platform misuse where banned users end up immediately creating a new account to circumnavigate the ban. By not informing users that their account has been banned or quarantined, the account might continue to spew misinformation – false or misleading content – into a void that doesn’t interfere with users. For bots and spam accounts that are vectors of misinformation, shadow bans would be fairly effective. Bot accounts vie for influence through quantity and reach of posts. Cordoning off bot and spam posts without outright banning the accounts helps to ensure that bot creators will not simply create a new username for the bot to be associated to. Lastly, shadow banning became a prominent strategy on sites like Twitter in response to foreign influence operations such as Russian spam accounts. Shadow banning foreign accounts that seek to infiltrate the US’s information ecosystem allows Twitter to restrict bad actors from influencing audiences.
However, shadow banning is an intervention that fails to address many other forms of misinformation. Shadow bans disincentivize banned users from posting content by cutting them off from the “currency” of social media, be it likes, comments, or shares. If users’ posts receive no interaction, they are implicit signaled that their content is not being received or shared positively, which may curb their use of the platform. Whether this would push users to post more extreme content to attract greater attention remains to be seen. But this sort of potential backfire effect could amplify the effects of misinformation in a disconcerting way. Beyond targeting bots and spam accounts, shadow banning is unable to address the very real problem of legitimate accounts that tout views that could radicalize people but are not considered spam. This includes posts that legitimize extremists or conspiracy theories while not themselves being conspiratorial or extremist.
Examining specific cases of shadow banning and measuring their efficacy is challenging given the practice’s lack of acknowledgement by online platforms. Allegations of shadow banning are almost always anecdotal and evidenced through end-user observations as they interact with the platform. Interestingly, one study assessed the plausibility that shadow banning could be implemented on Twitter (a practice which Twitter denies in the first place, as we know). This study adopted a statistical approach rather than addressing the anecdotes that pervade this research topic. Twitter has claimed that any instances of alleged shadow banning amount to “bugs” in their algorithms that filter users and content. Yet this team of scholars found that “…bans appear as a local event, impacting specific users and their close interaction partners, rather than resembling a (uniform) random event such as a bug.” In other words, the way in which certain user characteristics were overwhelmingly prevalent among shadow banned users indicates that the practice is unlikely to randomly target accounts. If, instead, shadow banning can be proven to be an intentional targeting tactic, shadow banning cannot be explained away as a software anomaly. Online moderation scholar Dr. Carolina Are analyzed the implementation of shadow bans on Instagram as a tool for restricting sexually risqué content such as nudity and pole dancing. Though not an effort to quell misinformation specifically, the analysis exposes the likely existence and impact of shadow bans on Instagram. Dr. Are’s narrative of managing a popular pole dancing account catalogs the way her increase in followers curiously led to less follower engagement with her content. She notes that a common shadow ban approach that Instagram admitted to is the censoring of specific hashtags, which runs counter to the supposed community-driven function of hashtags that is commonly understood. Dr. Are acknowledges that her speculation about shadow banning is driven by the opaque policies of social media platforms like Instagram on their hand in content visibility. As Dr. Are released what felt like “content posted into a void,” it seemed that the practice of shadow banning to deliver its purpose of restricting the spread of content with a troubling efficiency.
More recently, the social media platform TikTok has come under fire for similar and alleged shadow banning practices that restrict hashtags and video visibility, and that even remove whole videos without notifying creators. Black content creators have claimed that TikTok has shadow banned certain users and their videos that were associated to the #BlackLivesMatter movement. The platform admitted to what they termed a “glitch” that affected the view count displays of videos with the hashtags #BlackLivesMatter and #GeorgeFloyd. The effect was noticed amidst the skyrocketing attention received by these hashtags during the summer of 2020. Outrage at this supposed “glitch” pointed out that curbing this sort of content ran directly counter to statements the company made on their support of Black content creators. Black content creators also claim that they had videos entirely removed by TikTok that contained #BlackLivesMatter and #GeorgeFloyd content, without permission or notification. TikTok has also conceded that it shadow banned videos of disabled, queer, and plus-size content creators in a bizarre attempt to implement so-called “anti-bullying practices.” It is notable that this shadow ban was done manually, through a system of moderators flagging specific user accounts. Regarding automated moderation on the platform, TikTok concedes that their algorithms create a risk of “…presenting an increasingly homogenous stream of videos…” but leaves the extent of their filtering practices at that. The platform’s track record for dealing with content is a mélange of both automated and manual shadow banning practices, neither of which have resulted in public insight into how the platform filters or bans content.
Ultimately, shadow bans are an extreme form of restricting content from being seen by others. However, it remains unclear how well shadow bans work to combat online misinformation. They are “effective” insofar as shadow bans target the right accounts i.e., accounts that purvey misinformation. There remains scant concrete evidence of the perspective of platforms on shadow banning, but the social media giant Reddit has provided some context. In 2015, Reddit phased out its shadow ban policy, replacing it with the more traditional form of moderation: account suspensions. What’s crucial in this change of strategy is that Reddit explicitly acknowledged that it had implemented shadow banning to quickly hide spam on the platform. Rather than denying its existence, Reddit alluded to shadow banning as an official moderation strategy. The practice appeared to function on a manual basis, meaning Reddit employees, (called admins), were responsible for implementing shadow bans on user accounts. After moderating in this fashion for an unknown amount of time, Reddit officially announced that shadow bans had “outgrown their usefulness” and that they had realized shadow bans were “…great for dealing with bots/spam rings, but woefully inadequate for real human beings.” This conclusive evidence of the practice of shadow banning seems to lend authority to unconfirmed or denied shadow banning allegations by other platforms. It highlights that platforms like Reddit evaluated the effectiveness of shadow banning based on the kinds of users it targeted, rather than by the content it removed.
Overwhelmingly, Reddit’s conclusion suggests the practice did not seem to serve the purpose for which it was intended. Instead, Reddit noted that its shadow bans created a toilsome system of ambiguous bans and user appeals to moderators. The benefit of deploying shadow bans over account suspensions was that spammers were unable to circumvent their suspended account. But it was decided that the juice was not worth the squeeze, and Reddit ultimately began operating under a system of formalized account suspensions. Reddit’s very public change in moderation tactics acknowledges the need for more transparency around shadow banning practices by social media platforms. Platforms have little incentive to make these policies explicit, and some would argue they are actually disincentivized to do so. Reddit co-founder Steve Huffman noted that their company’s content policies function best when they are “specifically vague” because it prevents bad actors from exploiting loopholes in their policy.
The future of understanding shadow banning relies on further and widespread transparency on the part of social media platforms. Shadow banning’s effectiveness to combat harmful and disturbing content online is challenging to measure because we lack widespread recognition that this practice is even used to moderate in the first place. There are massive gaps in empirical data on shadow bans. Before social media platforms are transparent about the practice, we cannot know the extent to which it tackles the problems of misinformation, how widespread its usage is, and whether it is used equitably. Despite the merit of user-centric analyses like Dr. Are’s, we need more backend insight into how shadow ban decisions are made by algorithms or human moderators within the architecture of platforms. that we ought to employ despite some ethical criticism. But in order to ever justify or refute the practice, academic studies will need concrete data on the context and rationales of user bans from platforms. This is especially the case amid growing legislation on how content is managed on social media platforms. Further studying shadow banning a tall order for a social media landscape that has many reasons to shirk transparency. But we can only hope that continued calls for platform accountability will force companies to pull the practice of shadow banning into the spotlight.