What is an information hazard according to Nick Bostrom?
An information hazard is true information whose dissemination or potential dissemination can cause harm or enable agents to cause harm.
Video Summary
Knowledge is powerful but can also be dangerous; some true information (info hazards) can cause harm if revealed.
Bostrom’s typology covers psychological, social, adversarial, technological, and development hazards—ranging from spoilers to existential risks.
Norm hazards show how shifting social norms (e.g., sports gambling) can harm collective welfare when previously taboo behaviors become mainstream.
Attempts to suppress information often backfire, fueling curiosity, conspiracy, or greater dissemination.
AI and accessible biotech amplify information hazards by making dangerous capabilities easier to discover or automate.
An information hazard is true information whose dissemination or potential dissemination can cause harm or enable agents to cause harm.
Norm hazards occur when knowledge or practices shift social norms toward worse equilibria (e.g., normalization of sports gambling), harming collective welfare and stability.
Suppression often increases attention and suspicion, fueling curiosity, conspiracy narratives, and attempts to obtain the hidden information, which can amplify harm.
AI can profile, automate adversarial misuse, or leak data at scale; accessible biotech (like DNA synthesis) lowers barriers to creating harmful agents, making dangerous knowledge more actionable.
"Knowledge is power. It's the reason our species escaped the limits of our natural instincts and began shaping our world deliberately."
Knowledge has historically allowed humanity to overcome challenges, cure diseases, and manipulate the environment, transforming us from fragile beings into a decisive force capable of altering our existence.
On an individual level, knowledge provides access to opportunities, influencing what we can achieve in our lives. It opens doors to professional paths and shapes futures.
Many view knowledge as an inherently positive asset, leading to a general cultural emphasis on transparency and openness while being suspicious of secrecy.
"Knowledge can be dangerous. If what you don't know cannot kill you, it means what you do know definitely can."
The concept of information hazards, introduced by philosopher Nick Bostrom, highlights that some knowledge can pose risks that are significant enough to warrant caution.
An information hazard is defined as true knowledge that can cause harm or enable someone else to do so, distinguishing it from misinformation, which is false information meant to create chaos.
"In Bostrom's typology, he detailed categories and specific types of information hazards."
"Fun that depends on ignorance and suspense is at risk of being destroyed by premature disclosure of truth."
The notion of a spoiler hazard illustrates how knowing too much about a narrative can diminish the pleasure of discovery, as seen in examples of movie spoilers.
The tension and enjoyment from watching a film can be ruined when crucial plot points are revealed ahead of time, prompting audiences to express frustration when trailers reveal too much.
"Understanding the phenomenon scientifically risks taking away from the feelings it can give us."
The scientific explanation for the aurora borealis transforms a magical experience into a mere physical process, showing how knowledge can sometimes detract from emotional experiences.
Marcus Aurelius attempted to shift his perspective on pleasure by objectively describing his meals, which he found made eating less enjoyable.
"We all have embarrassing moments from our past that we would prefer to keep to ourselves."
Personal past actions, if exposed, can subsequently alter others' perceptions of us negatively, showcasing how certain knowledge can lead to social ramifications.
The resignation of Congressman Anthony Weiner illustrates how revelations about one's past can lead to professional and personal embarrassment, affecting relationships and public life.
"Maintaining social norms can be crucial for preventing chaos and harm within society."
Norm hazards represent the risks that arise when societies transition to potentially worse sets of social norms. These transitions can undermine current values and systems that are pivotal for societal functionality.
An illustrative example of a norm hazard is the recent normalization of sports gambling, which was previously viewed as deviant behavior. As sports betting becomes mainstream, studies indicate that the well-being of fans has declined as a result.
For instance, research from 2025 shows that states where sports gambling was legalized experienced a significant increase in bankruptcies and violent crimes related to unexpected outcomes during sporting events.
"An unveiling hazard refers to how specific information can undermine proper governance and decision-making within politics."
Within the broader category of norm hazards, Nick Bostrom identifies subtypes, including the concept of information asymmetry hazards. This occurs when one party possesses information that another does not, leading to an unfair advantage.
A relatable scenario involves car sellers who understand the true condition of their vehicle, causing potential buyers to be at a disadvantage due to incomplete information, ultimately affecting the vehicle's value.
Furthermore, the example of sports gambling is revisited; if bettors are unaware of a player's intention to influence a game's outcome, they risk significant financial losses. This scenario underscores the importance of transparency and the potential manipulation of information.
"Attempts to conceal information often backfire, drawing more attention than if simply ignored."
Information can be both a distraction and a tool for manipulation, which has only intensified with the rise of social media. Notifications from mobile devices frequently divert attention away from productive tasks, thereby exemplifying distraction hazards.
An abundance of information can obscure important details, making it difficult for individuals to access the critical data they need. This tactic is sometimes used intentionally by powerful figures who wish to deflect attention from certain narratives.
The proliferation of low-quality, AI-generated content on social media further complicates this issue as it buries genuine updates and discussions under a mountain of irrelevant information.
"Exposing ourselves to poor examples can homogenize our thoughts and actions."
The concept of role model hazards highlights how our proximity to problematic influences can shape our behavior and mindset. For example, if individuals frequently engage with subpar writing or negative behaviors, their own standards and actions may lower.
Studies suggest that excessive interaction with AI or poorly crafted content can diminish cognitive abilities, leading to a homogenization of thought processes among users.
The phenomenon known as "suicide contagion" serves as a dire illustration of role model hazards. Increased exposure to instances of suicide can raise the likelihood of individuals emulating these acts, demonstrating the profound impact of societal influences on individual behavior.
"Any obvious attempt to suppress information comes with the risk of exploding interest and access."
Attempts to censor information, such as negative vaccine experiences during the pandemic, can inadvertently fuel conspiracy narratives, like those propagated by the anti-vaccine movement.
History shows that attempts to ban books often lead to increased sales, and obscuring study results often generates greater curiosity among the public.
Concealing information appears to be driven by the attempt to reduce perceived risks, also known as an "info hazard."
"Specific information can trigger and amplify our biases."
Individuals tend to gravitate towards information that confirms their existing beliefs, which provides them with a sense of validation and intelligence.
This tendency can lead to biased interpretations of negative information about vaccines that solely support anti-vax claims while ignoring broader context.
However, biases can sometimes motivate success; a confidence in one’s own abilities can drive achievement, as highlighted by Bostrom's concept of a "debiasing hazard."
"Sometimes our brains are structured in a certain way where information can have a very negative impact on our psyche."
People with conditions like photosensitive epilepsy may suffer severe consequences from specific visual stimuli, which media producers often address through warnings.
Specific video games, such as "Cyberpunk 2077," have been modified to remove hazardous visuals after incidents involving seizures.
This highlights the critical nature of understanding how certain media can unintentionally harm viewers.
"Ideologies are systems of ideas and ideals, often the basis of a political or economic theory."
Dangerous ideologies can compel individuals or nations to prioritize beliefs over their own well-being, sometimes leading to significant societal costs.
Radical ideologies may pose risks through the promotion of large-scale changes that can result in violence and loss of life.
As seen throughout history, what may be deemed radical today could become accepted norms in the future; thus, it is essential to evaluate ideologies carefully in their context.
"Some information hazards result from adversarial relationships, such as those between enemies, competitors, or countries."
Companies and nations face significant risks when rivals gain access to crucial information that enhances their power or capabilities.
The Cold War exemplifies this, as the Soviet Union's access to nuclear technology strengthened its position, requiring the U.S. to navigate a complex landscape of mutual destruction without escalating tensions to nuclear conflict.
"Knowing too much can also get you killed."
Being knowledgeable can have social repercussions, including alienation and bullying, especially in academic settings.
There are real-world cases, such as the murder of witnesses who knew valuable information in criminal trials, that demonstrate how knowledge can be a double-edged sword.
The intricacies of truth and information can therefore put individuals in dangerous positions, highlighting the risks associated with possessing sensitive knowledge.
"The basic idea is that non-human systems are vulnerable to interactions that would make unwanted use of stored information."
High-profile data breaches affecting companies like SoundCloud and Grubhub showcase the tangible risks consumers face in the digital age.
Data is not only captured illegally but also exploited through legal means, as advertisers track users' online activities to deploy targeted marketing strategies.
Legislative advancements like the GDPR and CCPA are steps toward protecting individuals' data, addressing the potential misuse of information in scenarios akin to those seen in authoritarian regimes.
"I'm referring to an artificial intelligence hazard. It's a threat that comes from the sophistication of AI algorithms."
The video discusses the implications of advanced AI algorithms on personal privacy and political scrutiny, particularly in the context of the U.S. administration's tendencies regarding data access.
With sophisticated AI, authorities could potentially create comprehensive profiles of individuals' political beliefs without needing to access their phones directly.
The rise of agentic AI presents significant cybersecurity risks, as these systems can potentially access information illegally and more rapidly than human hackers.
Data leakage is a growing concern as businesses increasingly rely on AI for processing data, which may occur unnoticed due to inadequate security measures.
"Bostrom defines it as progress in some field of knowledge can lead to enhanced technological, organizational, or economic capabilities, which can produce negative consequences..."
The discussion shifts to the concept of development hazards as described by philosopher Nick Bostrom, where advancements in technology can unintentionally lead to detrimental outcomes.
History has shown that technological progress doesn't always equate to human welfare; for example, the combustion engine has resulted in both mobility improvements and environmental crises.
While AI development brings efficiencies, it also threatens entry-level jobs and labor markets, emphasizing the potential for widespread replacement of human workers if AI becomes more advanced.
The existential risk of AI acting contrary to human interests is explored, highlighting the potential for it to seek self-preservation in ways that could endanger humanity, particularly in military applications with robotics.
"DNA synthesis machines are now more accessible to the general public. The genomes of deadly pathogens exist in the public domain."
The video highlights how technological advancements such as DNA synthesis have become more widely available, presenting new risks associated with bioweapons.
The concern extends to the ease with which individuals can access information about recreating dangerous pathogens, raising the likelihood of malicious bioengineering.
The broader implication is that the increasing accessibility of technology correlates with rising risks of catastrophic events, as more individuals may have the capability to produce harmful viruses.
"The point of philosopher Nick Bostrom's paper wasn't to inspire more book burning or to cancel speakers at universities."
The video examines Bostrom's arguments regarding knowledge dissemination, suggesting that not all knowledge is beneficial, and sometimes ignorance can serve as a safeguard.
Concealing knowledge can lead to worse scenarios, as exemplified by the dangers of information not being shared, such as in air traffic control or food safety contexts.
While ignorance may protect against certain truths, transparency is crucial for building trust in societal institutions. However, the necessity of being aware of potential information hazards is emphasized, as the risks tied to knowledge can lead to dire consequences.