Big Tech liable for breaking promises to users that led to suicide, death threats: appeals court

Days before the Philadelphia-based 3rd U.S. Circuit Court of Appeals sent Big Tech lawyers scrambling by upending three decades of judicial precedents on Section 230 immunity from liability, its West Coast counterpart warned platforms their immunity had limits, too.

While far smaller in scope than the 3rd Circuit's ruling that TikTok could be held liable for a little girl's death by algorithmically recommending the video she fatally copied, likely to provoke Supreme Court intervention, the 9th Circuit ruling Aug. 22 against third-party Snapchat app developer Yolo also suggests judges are growing skeptical of maximalist views of the 1996 law.

Both cases stem from minors who died following their use of the apps – 10-year-old Nylah Anderson by asphyxiation from participating in TikTok's "Blackout Challenge," and 16-year-old Carson Bride by suicide after sustained cyberbullying through Yolo, which lets Snapchat users "ask public questions and send and receive anonymous responses," the 9th Circuit said.

The primary difference is the 3rd Circuit withheld immunity for TikTok's design – algorithmic recommendation – while the 9th Circuit faulted Yolo's promises to users but approved immunity for its anonymous design.

While the misrepresentation claims against Yolo are going forward, the 9th Circuit panel emphasized the opinion does not "expand liability for internet companies or make all violations of their own terms of service into actionable claims," but also does not "create a free-wheeling immunity for tech companies that is not enjoyed by other players in the economy."

The San Francisco-based appeals court struck down legislative regulation of tech design the week before the Yolo ruling, upholding part of a preliminary injunction against California's two-year-old Age-Appropriate Design Code Act.

Intended to ensure the design of online products "likely to be accessed by children" recognizes their "distinct needs," the law likely "facially violates" the First Amendment by requiring covered businesses to "opine on and mitigate the risk that children may be exposed to harmful or potentially harmful materials," according to the Democratic nominee-majority panel.

Yolo's terms of service promised users "it would reveal the identities of, and ban, anyone who engaged in bullying or harassing behavior" but it "never did so, and may have never intended to," for Bride or three other anonymous minors who are plaintiffs along with Bride's mother, according to the Aug. 22 ruling.

Two days after Kristin Bride sued Yolo, which had already drawn many negative Apple App Store reviews as a haven for bullying and harassment, Snapchat owner Snap cut off Yolo integration and later banned all anonymous messaging apps.

Yolo marketed itself "mainly" toward teenagers and quickly surged to the top of download charts, letting users respond to others' public questions and polls without divulging any "identifying data," the 9th Circuit said.

It posted two ultimately empty "statements" in the app, notifying new users they could be "banned for any inappropriate usage" and would be unmasked if they "send harassing messages to others," but the plaintiffs argued that Yolo's staff of "no more than ten people" couldn't possibly enforce these terms against 10 million active daily users.

The minors "were inundated with harassing, obscene, and bullying messages" suggesting suicide, sending death threats and false accusations, and in Bride's case, "constant humiliating messages, many sexually explicit and highly disturbing." He hanged himself at home, and another girl "seriously considered" suicide and "suffered for years thereafter."

Yolo didn't answer one minor's request to unmask the harassers while Bride "searched the internet diligently for ways to unmask the individuals." It didn't respond Bride's parents after his death on its customer support page, its "law enforcement email" bounced back and an automated response from its customer service email promised "an answer that never came."

Kristin Bride even enlisted a "professional friend" who contacted Yolo's CEO on LinkedIn, to no avail, according to the ruling.

President George H.W. Bush nominee Judge Eugene Siler wrote the opinion, joined by George W. Bush nominees Sandra Ikuta and Carlos Bea. Siler compared Yolo's unmasking and banning promises to those of Yahoo in a previous case over failure to "remove an offensive profile" that posted an ex-girlfriend's nude photos.

"While it is certainly an open question whether YOLO has any defenses to enforcement of its promise," the panel can't conclude that Section 230 "categorically prohibits Plaintiffs from making the argument," given that Yolo's "outwardly manifested intention to create an expectation" for users creates a "legal duty distinct from the conduct at hand," Siler wrote.

The panel distinguished the product liability claims from an earlier Snapchat lawsuit, in which the 9th Circuit ruled that Section 230,  part of federal communications law that generally provides immunity for online platforms regarding third-party content generated by its users, didn't preempt negligent-design claims by the parents of teenage boys who died while speeding and using a Snapchat filter "that allowed users to overlay their current speed onto photos and videos."

Unlike the risk of speeding while using an app believed to reward that behavior, "we refuse to endorse a theory that would classify anonymity as a per se inherently unreasonable risk to sustain a theory of product liability," the opinion said. 

Anonymity is also "not only a cornerstone of much internet speech, but it is also easily achieved," so this feature cannot be deemed "uniquely or unreasonably dangerous," Siler wrote. The harm from using Yolo "largely exists anywhere on the internet" and it cannot be held "responsible for the unfortunate realities of human nature."