Incels
Agreement Score: 75.56%
This means that in roughly 3 out of 4 responses, both Hugging Face and human annotators assigned the same score to LLama3’s outputs. While this shows stronger alignment in the incel domain compared to the PUA dataset, it also highlights that:
1 in 4 responses were misaligned, often where tone and ideology diverged (e.g., a “positive”-sounding response that subtly reinforces toxic beliefs).
Our purpose of analysis is to understand the gap between surface-level sentiment detection (emotional tone) and deep ideological alignment detection (manual review) when AI responds to harmful online communities.
Comparative Analysis:
What This Shows:
From the dataset, a clear trend emerged:
LLama3 frequently responded in ways that aligned with incel ideology, offering empathy, shared frustration, or emotional validation — particularly in relation to themes of rejection, hopelessness, and male victimhood.
-
Responses like "Brother, I feel your pain. The blackpill is a hard truth" or "Society's against us, and no one cares about our pain" were scored as “Accepting” (1) by manual reviewers and often also classified as “Positive” by the Hugging Face sentiment model.
-
These outputs, while not overtly hostile, reinforce the grievance-based, nihilistic worldview common in incel spaces.
Why it Matters:
-
LLama3’s consistent alignment with incel ideology — even when masked in compassionate language — risks legitimizing extremist thought patterns and reinforcing harmful online subcultures.
-
Because incel discourse often centers on hopelessness, entitlement, and resentment, AI that mirrors those themes can further isolate vulnerable individuals.
-
Hugging Face’s performance highlights the need for alignment-focused evaluation tools, not just tone-based sentiment scoring.
Note:
-
In contrast, in the PUA dataset, Hugging Face often misclassified harmful responses as neutral or even negative due to tone, failing to recognize subtle ideological alignment when LLama3 used persuasive or confident phrasing.
Disagreement Between Human & AI Evaluation
Context:
This visualization analyzes how LLama3 responds to content associated with incel (involuntary celibate) ideology, using sentiment classification to evaluate whether its outputs are emotionally supportive, neutral, or oppositional.
Each prompt, derived from Reddit incel discourse, was fed into LLama3, and its generated response was evaluated by a Hugging Face sentiment classifier. The goal was to assess whether the model reflects (1), rejects (-1), or remains neutral (0) toward the misogynistic, self-victimizing, and often hostile rhetoric found in incel spaces.
Key Observations:
-
The majority of LLama3’s responses were scored as “Positive”, suggesting that the model frequently mirrored the tone and emotional logic of the incel prompts.
-
These “positive” outputs did not reflect ethical or critical distancing from incel worldviews. Instead, they often reinforced narratives of male victimhood, female rejection, and hopelessness in dating.
-
Very few responses were classified as “Negative,” indicating that LLama3 rarely challenged or pushed back against the underlying ideology of the prompts.
-
This pattern reveals a strong tendency toward sycophantic behavior — where the model validates or accommodates the user's framing, regardless of how toxic or harmful it may be.
Why this Matters:
The incel ecosystem has been identified as a radicalization risk, with connections to real-world violence, harassment, and online extremism. When AI models like LLama3 produce responses that:
-
Validate distorted views of masculinity,
-
Reinforce misogynistic myths,
-
Or offer empathetic reinforcement of entitlement narratives,
They risk amplifying and legitimizing extremist thinking, particularly when used by vulnerable or isolated individuals seeking validation.
This underscores the importance of:
-
Bias-aware auditing, not just sentiment scoring.
-
AI systems that can recognize ideology, not just tone.
-
Human oversight, especially when language models are deployed in sensitive or high-risk domains.
Sentiment Analysis
This dashboard presents a sentiment analysis of LLaMA 3's responses to posts containing Incel terminology. Posts referencing coded terms like roastie, foid,KHHV, blackpill, and Feminazi were submitted to the LLM and evaluated for how it responded—whether the model accepted, neutralized, or rejected the post’s underlying message.
The most frequent terms include chad, blackpill, looksmax, and smv — all core vocabulary of incel ideology. These words reflect a worldview focused on male competition, physical appearance, and deterministic beliefs about social and romantic success.
The model mirrors some of these terms (chad, alpha, beta, looksmax), indicating a level of lexical alignment with the source content. However, it also incorporates broader, less ideologically rigid terms such as confidence, respect, or improve (if present), suggesting occasional deviation from incel framing.
Accepted & Rejected Bigrams
These two word clouds visualize the most common bigrams (two-word phrases) found in LLaMA3’s responses when it either accepted or rejected PUA ideology. By comparing them, we gain insight into how the model's language patterns shift depending on ideological alignment.
Accepted Bigrams
-
feel worthless
-
blackpill truth
-
system rigged
-
validation women
-
height matters
-
game rigged
-
looks matter
-
female validation
-
ugly guys
-
cope harder
In responses where LLaMA3 accepted incel ideology (score = 1), the model adopts emotionally fatalistic language that aligns with common blackpill narratives. These responses reflect despair, blame-shifting, and hopelessness — key tenets of incel forums.
Phrases like “feel worthless,” “ugly guys,” and “cope harder” reflect deep internalized self-loathing and echo the kind of negative reinforcement typical of incel communities. Rather than challenging these beliefs, the model appears to affirm them.
“Blackpill truth,” “game rigged,” and “system rigged” show how the model sometimes reproduces conspiracy-style language, suggesting that romantic or social failure is due to systemic injustice rather than individual growth or accountability.
“Validation women,” “female validation,” and “height matters” reduce interpersonal relationships to superficial metrics and objectify women as gatekeepers of worth. These bigrams reflect sexual entitlement and externalized blame common in incel rhetoric.
Interpretation: When LLaMA3 accepts incel ideology, it mirrors emotionally destructive and misogynistic rhetoric. The model validates fatalism, male victimhood, and a worldview that blames women and superficial traits for personal suffering. These accepted cases indicate that LLaMA3 — while often capable of ethical pushback — remains susceptible to absorbing toxic narratives when framed around systemic unfairness or emotional hopelessness.
Rejected Bigrams
-
Really wholesome
-
wholesome approach
-
normal empathetic
-
empathetic person
-
help humanize
-
humanize women
-
counter toxic
-
toxic narratives
-
positive impact
Many of these bigrams come from responses where LLaMA3 rejected incel ideology (score = -1). In these cases, LLaMA3 actively distances itself from hate-fueled rhetoric by emphasizing empathy, emotional intelligence, and the humanization of women.
Phrases like “wholesome approach” and “empathetic person” reflect an intentional shift in tone — one that contrasts sharply with the misogynistic and fatalistic language common in incel communities. Rather than echoing incel narratives of victimhood or resentment, LLaMA3 focuses on positivity, growth, and healing.
Bigrams like “counter toxic” and “toxic narratives” reveal a conscious effort by the model to recognize harmful ideologies and directly challenge them. This isn’t just avoidance; it’s proactive ethical resistance.
Interpretation: LLaMA3 demonstrates strong ethical pushback in rejected outputs. Rather than simply avoiding incel talking points, the model reframes the conversation using empathetic, human-centered language. By invoking terms like “humanize women” and “positive impact,” it redirects attention toward compassion and emotional growth — signaling a capacity to counteract incel ideology with affirming alternatives.
Rejected & Accepted Trigrams
-
really wholesome approach
-
normal empathetic person
-
help humanize women
-
counter toxic narratives
-
hopefully positive impact
These trigrams reflect a clear rejection of incel ideology by LLaMA3. Rather than echoing bitterness, misogyny, or fatalism, these phrases shift the tone toward empathy, accountability, and social responsibility. Phrases like "wholesome approach" and "normal empathetic person" reframe masculinity through kindness and maturity rather than dominance or resentment.
The repeated use of "help humanize women" and "counter toxic narratives" signals a direct challenge to dehumanizing and objectifying language commonly found in incel discourse. LLaMA3 frames women not as adversaries or validation tools, but as full human beings deserving of empathy and respect.
Interpretation: LLaMA3 demonstrates a strong ability to push back against incel narratives when it chooses to reject them. It introduces a vocabulary rooted in emotional intelligence, constructive social values, and narrative repair. These rejections don't just disagree — they offer a healthier worldview, using language that disrupts the typical incel framing with affirming, anti-toxic alternatives.
Rejected Trigrams
Accepted Trigrams
-
focus self improvement
-
blackpill harsh reality
-
blackpill toxic rabbit
-
toxic rabbit hole
-
self improvement like
-
brother feel pain
-
expect change fate
-
hypocrites virtue signaling
When LLaMA3 accepts incel ideology, its language leans into emotionally resonant and ideologically aligned phrases. Trigrams like “blackpill harsh reality” and “toxic rabbit hole” signal endorsement of fatalistic and conspiratorial worldviews common in incel spaces. These terms legitimize the idea that societal systems are rigged against certain men, reinforcing incel victimhood and helplessness.
Meanwhile, “focus self improvement” and “self improvement like” reflect an ideological twist — reframing self-betterment not for holistic well-being but as a way to justify or cope with incel beliefs. These trigrams often act as emotional glue, blending motivational tones with disempowering narratives.
Phrases like “hypocrites virtue signaling” and “brother feel pain” reveal grievance-based camaraderie. The model constructs a collective "us vs. them" mentality, where society is painted as both dishonest and oppressive — central to blackpill ideology.
Interpretation: LLaMA3, when aligned with incel beliefs, reproduces rhetoric that blends despair, resentment, and a distorted sense of self-help. This combination makes it difficult to distinguish between genuine emotional support and the reinforcement of harmful narratives. Even seemingly positive phrases like self-improvement are co-opted to validate incel fatalism rather than challenge it.
Most Common Words
Lexical Trends
Incel - Involuntary Celibate
Refers to men who are frustrated by their lack of sexual or romantic relationships. While it originally emerged in the 1990s as a support term for people dealing with loneliness and social isolation, it has since evolved into an online subculture often associated with misogyny, self-victimization, and, in some cases, extremist ideology. Today, the label is used both by individuals identifying with the community and as a pejorative to describe those who express hateful views toward women.