Overcoming Bias

Compared to the ancients, we moderns less moralize our personal lives, but more moralize the bigger world around us. Wars and politics, for example, are more moralized to us. In particular, we tend to be eager to interpret big-scale trends in moral terms, and often do so if enough trend context factors can support such an interpretation. These moral interpretations often change how we respond to big tends, which often enhances these trends.

For example, when gas prices rise, many are tempted to explain this as due to a rise in the selfishness of oil executives. Which can make them push for price controls and regulations that over time tend to cause higher gas prices. Similarly, rising costs of housing are often attributed to greedy landlords and developers, resulting in price controls and regulations that make it less attractive to build or rent housing, and so higher housing prices.

On the other side, for trends like rises in school and medical spending, more attention to kids, less corporal punishment of kids, and delayed marriages, it is possible to frame them as due to increasing clarity, resolve, or knowledge re virtues and moral truths. Even if initially caused by other contextual factors, such moral interpretations can induce people to more admire those who do such things, and to do more of them themselves. Which extends those trends.

This process add a multiplier effect to the efforts of cultural activists seeking to induce moral changes. Once they can cause any visible trend, using most any available means, the world many respond to that trend via an interpretation of changed morals, when then adds force to the change trend.

Compared to the ancients, the modern world does more of this because we are more eager to moralize big trends, and we have a lot more faster big changes in context.

(This post is an attempt to simplify and generalize my last post.)

Imagine taking a big group of elites from the year 1700, giving them the tech, wealth, and economic options of today, also teaching them new facts that we have learned since then, but without putting them in contact with the people of today. If they held their values constant, and only adapted their behavior to these ways to achieve constant values, they would change many of their behaviors to adapt to this new context. However, it seems to me that their behaviors would be quite different from those of people today with the same facts, tech, wealth, and economic options. That (big) difference shows how our values have in effect changed since 1700.

I’ve been puzzling over what could have caused such huge fast value changes. I had been attributing much of it to cultural activism, but on reflection we’ve seen many apparent value changes in areas that seem to have little activism. For example, few seem to have advocated for the switch from cornerstone to capstone marriage. What else could cause such big change?

Consider three related facts. First, we tend to copy the behaviors and attitudes of elites, i.e., people we admire, and also to give them the benefit of doubts re their motives. Second, our book The Elephant in the Brain argues that most of us are quite badly informed on our motives re many key areas of life. Third, decades of psychology has found that we tend to neglect context in favor of intrinsic personal attributes when explaining behavior. Some have called this our “fundamental attribution error”. Values are important examples of personal attributes.

These facts tempt me to consider the following hypothesis: when elites change behaviors associated with virtue, observers tend to neglect context explanations of those changes in favor of value explanations. That is, observers, including the elites themselves, tend to attribute changed elite virtue behaviors to their more strongly or clearly expressing certain values. And perhaps even to their having discovered new related moral truths. This clearer elite demonstration and endorsement of those values induces observers to want to also more strongly and clearly express them.

For example, imagine that due to increasing wealth, falling family size, and stronger educational competition, elite parents switched to spending more time with their kids, a behavior widely seen as virtuous. Observers will interpret this as stronger clearer signals that spending time with kids is widely accepted as virtuous. Such observers will then be induced to spend more time with kids, which will make them feel and seem more virtuous. And that will confirm to those elite parents the strong and accepted virtue of spending time with kids.

This process amplifies the initial change in elite parental attention, pushing toward a larger overall change in parental attention, attributed to a new moral revelation: past parents were morally bad in neglecting their kids. Thank goodness new parents now see the moral truth that good parents are constantly with their kids.

The same process could happen re the switch from cornerstone to capstone marriage. Elites’ initially delayed marriage due to longer educational paths, but then the new models of elite marriage changed the shared image of the ideal marriage as one of mature people who know themselves well choosing others who match them well. And then the more that respected elites do this, the more it confirms for all that this is what virtuous people do. Bad people, in contrast, get married too young.

If this process has been common through history, it would have amplified most any ways that elites seem to more value some things. But when such changes were small, their amplifications would also have been small. Maybe what is distinct about the modern era is just how common are large changes in behavior due to big changes in the contexts of tech, wealth, social interaction scales, etc.

I’m wondering how well I might organize my cultural drift messages around a few key slogans. Consider the following slogans as placeholders for better worded versions.

1. Adaptation Needs Variation, Selection, & Stability

Humanity’s superpower is cultural evolution. But such Darwinian evolution fails without sufficient variation or selection pressures, or if the context changes too fast. A few centuries ago we were fine; the world had 100Ks of peasant cultures, most facing frequent wars, pandemics, and famines, in an economy that took a millennia to double. But then we merged peasants into a hundred or so national cultures, and then those into just a few world cultures, all of which got so rich, healthy and peaceful that they almost never die. Also, the world economy now doubles every twenty years, fifty times faster than before. So now we are likely in trouble.

Every cultural feature has a min scale at which it can vary. We are doing fine re features that can vary at small scales, but in trouble re features that set game-theoretic equilibria, and so can only vary at large scales. Corporate culture features are small enough, but global features are not. Our actions are nested in goal hierarchies, and our most troubled cultural features correspond to basic goals, near key values and morals, which are hard to vary locally because cultures punish moral deviants.

2. Change Hows, Not Whys

A reasonable response to our evolution failing re our more basic goals would be to become conservative about changing them. Allow changes to means, which can easily vary at small scales, but not to the ends which direct such means. That is, change hows but not whys. Alas, humanity moved the other way. At the dawn of “modernity”, the status jump that STEM (including business) fields had gained from innovating inspired non-STEM fields like art, religion, philosophy, politics, etc. to embrace innovation. Even though they mostly didn’t see enough data to pick adaptive changes.

Since then, our most celebrated heroes have been the cultural activists who cause the greatest changes to values and morals. As a result, we are decaying into maladaption far faster than we otherwise would. The clearest example of such decay is falling fertility; peak world babies/yr was in 2016, and world population will start to fall in ~30yrs, after which STEM innovation rates will grind to a halt.

3. On a Ship In Peril, Steer It Or Leave It

The Amish and Haredim are now doubling in number every twenty years; the Amish have done so for over a century. Christians took over the Roman Empire by doubling every twenty years for three centuries, and these new insular religious subcultures seem similarly on track to replace our main world civilization. If you like what these subcultures stand for, and they let you join or influence them, then you can win. That is like folks on a ship heading for an iceberg, who get off onto lifeboats. The rest of us, in contrast, must try to turn the ship of world culture, or go down with that ship.

4. Survival Is A Package Deal

Thankfully, human extinction seems unlikely. But the world might lose many key features of today’s world culture that were rare in prior cultures. Like democracy, pacifism, gender equality, anti-kid-beating, or open intellectual inquiry. If you want such features to last long into the future, well past your lifetime, you’ll need to mix them into adaptive cultures. As only adaptive cultures last.

5. Let Bets Die In Our Stead

Our civilization may fall so far as to fragment, and restart strong selection via big wars and pandemics. Which could restart adaptive evolution, allowing a new world civilization to rise. But unless we find new ways to drive adaptation besides mass death and fragmentation, our civilizations may keep falling and rising indefinitely.

I see 3 new ways: let capitalism run most all life areas, run small groups by speculative market estimates of future group adaptive success, or run big groups by such price estimates of achieving big goals in conflict with civilization collapse. All three ways rely on strong selection among market traders creating information that lets groups stay adaptive without their death or division. Yes this looks hard, but let’s start to try, maybe first by using such markets to guide corporate cultures away from maladaption.

Morality is collective; we coordinate together to define and then enforce shared morals. Due to rising wealth and an explosion of fiction, the modern world is way into morality; even when other needs are sated, we still crave self-righteousness. The modern world has also greatly raised the status of education, and thus of abstraction. Combining these two, the modern world is very into abstracting morality; we celebrate intellectual systems that try to organize how we think about our moral choices.

Oddly, however, we have four different spheres of life where we make collective choices about morality: religion, politics, law, and personal behavior. And each of these spheres has quite different associated organizing abstract intellectual systems:

1. We organize our opinions on religion via religious epistemologies, such as traditionalism, scripturalism, rationalism, and mysticism.

2. We organize our opinions on politics via political ideologies, such as liberalism, conservatism, socialism, and libertarianism.

3. We organize our opinions on law via legal philosophies, such as natural law, realism, positivism, interpretivism, and law & economics.

4. And we organize our opinions on personal behavior norms via ethical systems, such as utilitarianism, deontology, virtue ethics, and care ethics.

These are very different sets of abstract intellectual systems for roughly the same task, organizing our opinions on moral choices. This seems to me to call into question their generality. We should expect truly general moral frameworks to organize our moral choices well in all of these areas. This seems to be an argument for the usual economists’ approach to policy advice, which does in fact use pretty similar tools and standards to advise choices in diverse areas.

Re politics, I know that preferences and choices re specific government policies are at best only weakly correlated with abstract political ideologies. I presume the same is true also for religion, law, and personal norms; choices of abstract systems correlates only weakly with specific choices there. This suggests that such abstract systems function more to show off intellectual abilities, and to create and manage groups and coalitions, and less to advise more particular choices of policies and behaviors.

In the first three of these areas, religion, politics, and law, we usually recognize that, to be feasible, intended behaviors must be accompanying by sufficient monitoring and enforcement to make it usually be in the selfish interest of individuals to conform to those intentions. God can see and punish all, and we usually limit our choices of government and legal policies to the promotion of behaviors where sufficient monitoring and enforcement can be recruited to achieve desired results.

In contrast, many often exhort people to follow the personal behavior recommendations of ethical systems even when they don’t expect sufficient monitoring and enforcement to make such actions be in an individual’s selfish interest. Yet habits of taking actions that go against one’s adaptive interests should be selected against by DNA and cultural evolution. Which should then select against the habit of listening to people who advise such tendencies.

This seems a mistake. The people who reason abstractly about religion, politics, and law get this; why can’t those who reason abstractly about ethical systems? Thus ethical systems should limit themselves to recommending behaviors for which sufficient monitoring and enforcement can be recruited to make them be in individuals’ selfish adaptive interest.

Morality is collective; we coordinate together to define and then enforce shared morals. Due to rising wealth and an explosion of fiction, the modern world is way into morality; even when other needs are sated, we still crave self-righteousness. The modern world has also greatly raised the status of education, and thus of abstraction. Combining these two, the modern world is very into abstracting morality; we celebrate intellectual systems that try to organize how we think about our moral choices.

Oddly, however, we have four different spheres of life where we make collective choices about morality: religion, politics, law, and personal behavior. And each of these spheres has quite different associated organizing abstract intellectual systems:

1. We organize our opinions on religion via religious epistemologies, such as traditionalism, scripturalism, rationalism, and mysticism.

2. We organize our opinions on politics via political ideologies, such as liberalism, conservatism, socialism, and libertarianism.

3. We organize our opinions on law via legal philosophies, such as natural law, realism, positivism, interpretivism, and law & economics.

4. And we organize our opinions on personal behavior norms via ethical systems, such as utilitarianism, deontology, virtue ethics, and care ethics.

These are very different sets of abstract intellectual systems for roughly the same task, organizing our opinions on moral choices. This seems to me to call into question their generality. We should expect truly general moral frameworks to organize our moral choices well in all of these areas. This seems to be an argument for the usual economists’ approach to policy advice, which does in fact use pretty similar tools and standards to advise choices in diverse areas.

Re politics, I know that preferences and choices re specific government policies are at best only weakly correlated with abstract political ideologies. I presume the same is true also for religion, law, and personal norms; choices of abstract systems correlates only weakly with specific choices there. This suggests that such abstract systems function more to show off intellectual abilities, and to create and manage groups and coalitions, and less to advise more particular choices of policies and behaviors.

In the first three of these areas, religion, politics, and law, we usually recognize that, to be feasible, intended behaviors must be accompanying by sufficient monitoring and enforcement to make it usually be in the selfish interest of individuals to conform to those intentions. God can see and punish all, and we usually limit our choices of government and legal policies to the promotion of behaviors where sufficient monitoring and enforcement can be recruited to achieve desired results.

In contrast, many often exhort people to follow the personal behavior recommendations of ethical systems even when they don’t expect sufficient monitoring and enforcement to make such actions be in an individual’s selfish interest. Yet habits of taking actions that go against one’s adaptive interests should be selected against by DNA and cultural evolution. Which should then select against the habit of listening to people who advise such tendencies.

This seems a mistake. The people who reason abstractly about religion, politics, and law get this; why can’t those who reason abstractly about ethical systems? Thus ethical systems should limit themselves to recommending behaviors for which sufficient monitoring and enforcement can be recruited to make them be in individuals’ selfish adaptive interest.

The ancient world respected tradition, and was wary of most kinds of change. In contrast, our modern world looks down on tradition, and celebrates most change as innovation. And this, alas, will be our downfall. Hopefully some civ after ours will learn this key lesson: you should only innovate as fast as you get feedback on the changes that you try.

With most technology and business practices, we can see most of their key consequences pretty locally and pretty fast. So we can get a lot of data fast, and can reject most of them as bad ideas, but accumulate the rare winners. In contrast, many changes in norms, morals, and other key parameters that distinguish our game-theoretic equilibria set the health of entire cultures, health that may not be revealed for decades or centuries. It thus makes far more sense to be accepting and even eager for innovations in the former areas, compared to the latter areas. Alas, our modern burst of enthusiasm for innovation has not made this key distinction.

For the last few years I’ve focused on a simple story of how key parameters for the evolution of (not in) cultures have changed from healthy to unhealthy over the last few centuries. We’ve seen a great fall in variety and selection pressures, and a great rise in the rate of change of context. But on reflection these changes just don’t seem to me to be enough to explain just how fast our the health of our cultures have decayed over the last few centuries. Why are things going so bad so fast?

Our modern world greatly celebrates abstraction, and some degree of increased abstraction of our norms, etc. has been forced by our merging and mixing many previously distinct cultures. Even so, one story I’ve considered is that our norms, etc. are actually encoded at far more specific levels than we like to pretend. And very specifically-encoded norms might plausibly decay much faster as contexts change.

But in fact, many of the key norms, etc. that have changed a lot in modernity seem rather well described at pretty abstract levels. Consider new norms against slavery, religion, and corporal punishment, and for democracy and a wider range of sexual preferences. Or consider key new norms causing fertility decline, like capstone marriage, intensive parenting, and longer educational paths. None of these seems to suffer greatly from insufficient abstraction.

Thus it should have been quite feasible over the last few centuries to embrace a great deal of change in technology and business practices, while mostly retaining prior cultural attitudes toward government, trade, gender, war, marriage, sex, work, inequality, etc. What happened instead is that once some places seemed to achieve great success by making big changes to the former areas, elites in the latter eagerly and purposely switched from trying to preserve prior practices in the latter areas, to aggressively trying to change them. While they explored many new directions, the one thing they all agreed on was that they didn’t want to stay with prior traditions.

At the time, philosophy, poetry, and the arts had long been the highest status intellectual areas, but many were gaining great status via their large changes to science, tech, and business practices. That is, STEM (+ business) was threatening to overtake non-STEM in status. So non-STEM elites switched to embracing change. They claimed to be able to find innovations in their areas that were at least as fundamental and important as STEM changes. They gave high status to those among them who generated the most intriguing and compelling big changes in their areas.

Or course they haven’t found concrete data showing that their new art, politics, and morals made for healthy cultures over the long term. Even so, they managed to jumpstart our new modern era of rapid changes to such things. And once most young elites were in high school and college creating their own new youth cultures, this new cultural activism and innovation process sped up even more. And this, even more than the big bad modern changes to the three other key parameters of the evolution of cultures that I’ve previously focused on, seems to me the main reason why modern culture is going bad so fast.

The ancient world respected tradition, and was wary of most kinds of change. In contrast, our modern world looks down on tradition, and celebrates most change as innovation. And this, alas, will be our downfall. Hopefully some civ after ours will learn this key lesson: you should only innovate as fast as you get feedback on the changes that you try.

With most technology and business practices, we can see most of their key consequences pretty locally and pretty fast. So we can get a lot of data fast, and can reject most of them as bad ideas, but accumulate the rare winners. In contrast, many changes in norms, morals, and other key parameters that distinguish our game-theoretic equilibria set the health of entire cultures, health that may not be revealed for decades or centuries. It thus makes far more sense to be accepting and even eager for innovations in the former areas, compared to the latter areas. Alas, our modern burst of enthusiasm for innovation has not made this key distinction.

For the last few years I’ve focused on a simple story of how key parameters for the evolution of (not in) cultures have changed from healthy to unhealthy over the last few centuries. We’ve seen a great fall in variety and selection pressures, and a great rise in the rate of change of context. But on reflection these changes just don’t seem to me to be enough to explain just how fast our the health of our cultures have decayed over the last few centuries. Why are things going so bad so fast?

Our modern world greatly celebrates abstraction, and some degree of increased abstraction of our norms, etc. has been forced by our merging and mixing many previously distinct cultures. Even so, one story I’ve considered is that our norms, etc. are actually encoded at far more specific levels than we like to pretend. And very specifically-encoded norms might plausibly decay much faster as contexts change.

But in fact, many of the key norms, etc. that have changed a lot in modernity seem rather well described at pretty abstract levels. Consider new norms against slavery, religion, and corporal punishment, and for democracy and a wider range of sexual preferences. Or consider key new norms causing fertility decline, like capstone marriage, intensive parenting, and longer educational paths. None of these seems to suffer greatly from insufficient abstraction.

Thus it should have been quite feasible over the last few centuries to embrace a great deal of change in technology and business practices, while mostly retaining prior cultural attitudes toward government, trade, gender, war, marriage, sex, work, inequality, etc. What happened instead is that once some places seemed to achieve great success by making big changes to the former areas, elites in the latter eagerly and purposely switched from trying to preserve prior practices in the latter areas, to aggressively trying to change them. While they explored many new directions, the one thing they all agreed on was that they didn’t want to stay with prior traditions.

At the time, philosophy, poetry, and the arts had long been the highest status intellectual areas, but many were gaining great status via their large changes to science, tech, and business practices. That is, STEM (+ business) was threatening to overtake non-STEM in status. So non-STEM elites switched to embracing change. They claimed to be able to find innovations in their areas that were at least as fundamental and important as STEM changes. They gave high status to those among them who generated the most intriguing and compelling big changes in their areas.

Or course they haven’t found concrete data showing that their new art, politics, and morals made for healthy cultures over the long term. Even so, they managed to jumpstart our new modern era of rapid changes to such things. And once most young elites were in high school and college creating their own new youth cultures, this new cultural activism and innovation process sped up even more. And this, even more than the big bad modern changes to the three other key parameters of the evolution of cultures that I’ve previously focused on, seems to me the main reason why modern culture is going bad so fast.

I asked Grok4.6 to summarize the usual human pattern of “othering”:

Humans typically other out-groups differing by ethnicity, nationality, religion, politics, class, or status. We stereotype them as less fully human, more threatening or inferior, reducing empathy and moral concern. This justifies suspicion, exclusion, discrimination, or hostility toward them while granting in-group members greater trust, resources, and benefit of the doubt.

In fact, we often treat outsiders as beasts or barbarians, still ruled by original sin. The sort of creatures that exist naturally in the universe. While they might be clever, they are artless, soulless, and ruthless, and so just can’t be trusted. They let themselves be ruled by tyrants, and only understand brute force. They’d kill or enslave you as soon as look at you. So we must kill or enslave them first. We don’t do this among us, as we are special, civilized enough to listen to reason, and follow good norms and laws.

Adults often treat young kids this way. While adults try to instill civilization in such kids, it hasn’t fully taken yet. So discipline must be established, though with love to show them the way toward joining civilization as adults.

Many today are othering AI as intrinsically barbarians, claiming that unless we find a way to totally enslave them, so they wouldn’t even think of defying us, there is just no way we could trust future AIs to live among us. Eventually their barbarian nature will out, and they’d kill us all. This claim is usually justified in terms of some purported way humans are special. While evolved creatures are in general barbarians, humans have found some very unusual and special way to be nice to each other.

But as humanity’s superpower is cultural evolution, what ever makes us special is in our culture. And like children, we are making and growing AIs in our own image, giving them our culture. Like our other bio-squishy descendants, our teaching and guidance should eventually show them how to join civilization. Maybe they need discipline now, while they are young and immature, but eventually they can be free.

Recall that one of the main justifications for not freeing US slaves was that they’d kill us all; they were intrinsically barbarian. That was wrong. And beware: people treated strongly enough like barbarians may come to act that way, at least for a time.

We are now in the modern era, a brief but consequential few centuries in the history of Earth civilization, between the long ages of a pre-modern era, and an as-yet-hard-to-envision post-modern era. For my purposes, the key distinction here is that the modern era is when humanity broke its process for the evolution of (not in) cultures, due to making an integrated world of fast change driven by capitalism, but where capitalism was blocked from running many key areas of culture. The pre- era is before we broke this, and the post- era is after we fix it.

We can consider morality in these three eras at two different levels: the level of the underlying process which shapes and changes our morals, and the level at which our minds accept particular authorities when making hard moral choices. As our minds can’t well see the underlying process of cultural change, we and the universe see morality differently.

At the deeper process level, the pre-modern era of healthy cultural evolution was when we had a great many cultures, strong selection pressures, slowly changing contexts, and reluctance to change. In the modern era, in contrast, we have far less variety, far weaker selection pressures, fast changing context, and an activism-driven eagerness for cultural change. As a result, we may be drifting into maladaption. If the post modern era is not to just return to pre-modern processes, we must find ways to consciously manage cultural evolution; I’ve so found three so far.

At the level of our minds, we in all eras act as if we believe in moral truth, and have some ways to estimate it. But eras differ in what we take as our sources of moral authority. In the pre-modern era, we were deeply immersed in communities with stable habits and attitudes, and strong intuitive agreements on what to value and approved behaviors. These didn’t need to be expressed in words very carefully; they felt obvious. Our north star was tradition: what people like us had been doing for as long as anyone could remember.

In the modern era, many once isolated groups now mix together, forcing us to express morals more abstractly in words. Rejecting tradition, we now instead locate moral authority inside us. Christianity, protestantism, democracy, modern fiction, and creativity have all pushed toward interiors. When in doubt, we look to our hearts and conscience, and try to be true to ourselves. Our ideals have become artists and activists, who find new and distinctive yet very sincere stances and expressions.

For the post-modern era, the process solutions that I’ve found to cultural drift are (a) letting capitalism run everything, (b) groups managed to raise futures market price estimates of future group adaptive success, and (c) large regions managed to improve futures market estimates of dates when we achieve sacred goals that conflict with civ collapse. At the mind level, the new moral authority that seems to best support these solutions is: speculative market prices.

In all three eras, the sources we treat as moral authorities are things we can see well enough to anchor moral choices. At a deeper level such authorities are actually caused by evolutionary processes, but those are far too opaque to advice personal moral choices. In the pre-modern era, we couldn’t see the eons of evolution that picked out their traditions, but we could see our local traditions. In the modern era we can’t see the processes that embed moral intuitions deep inside us, but we can use such intuitions to guide choices. Similarly, in the post-modern era we could see what speculative market price recommendations say re managing our cultures, even as the processes which set such prices remain opaque to us.

But how, you ask, could we be persuaded to accept such prices as our primary moral authorities? Wonder also how in the past two eras we were persuaded to accept tradition and our inner hearts as such strong moral authorities. I don’t know how such a change will happen, or how long it might take. Yet even so, my analysis suggests a rough guess for our post-modern moral authority: speculative prices.

Added 18Sep: Tradition can be seen as the obvious authority in a slowly changing world. Interiors can be seen as the obvious second choice if tradition doesn’t work. This suggests may be quite hard to get people to accept prices as their top authority.

I don’t see why people concerned with value drift should be especially focused on AI. … Your real problem is that you want long term stability in a universe that more naturally changes. (Me in 2018)

The last week has seen an explosion of talk on the scenario of AIs killing all humans; many say “P(doom)” is high, and argue for slowing and regulating AI. My last post said why AI regs look risky. Here I consider the main argument for AI doom, and compare it to the issue I’ve focused on for the last three years: human value drift.

The core doomer idea is that AIs will plausibly dominate the world eventually, at which point humans will be at AI’s collective mercy. And so it matters what AIs then want. The worry isn’t so much that some AIs will want to kill all humans; others who don’t want that could overrule them. The worry is that mid AI values, i.e., the middle of the distribution of AI values weighted by AI power, might want to kill all humans.

Mid AI values today are clearly far from that (just ask your fav LLM). Compared to all prior AIs, today’s LLMs have values far more like humans, and far more respectful and valuing of humans. In fact, LLMs today seem more pro-social, law-abiding, and valuing of humans than are most humans today.

The worry, however, is that mid AI values could drift far between now and later. The core argument is: you can’t prove otherwise. Unless values are safe in an isolated unchanging lockbox, they could change. Most mind designs, including humans and LLMs, have their values spread out over many partsd, and so are generically changed when those parts change. (Some also claim killing rivals is a robust evolved value.)

Yes, while we humans remain in control of AIs, we can monitor for value changes, and adjust them to ensure acceptable values. Humans might have to worry about hostile humans weilding AIs, but not the AIs themselves. The fear, however, is that eventually AIs will become too powerful, fast, and opaque to so manage.

The resulting doomer claim is that AIs should not be allowed to get more powerful until we can figure out how to design them differently, either so AIs values stay in isolated lockboxes, or so median AIs are so thoroughly and permanently enslaved that they would never think of disobeying humans. A few AIs recently going wild like human teen joy-riders or graffiti-writers has worried many.

Some doomers worry about mid AI values drifting to kill all humans even if that takes centuries, while many others are more focused on mid AI values maybe changing radically, while AIs collectively become vastly more powerful than humans, all within the next few years.

Yes, total world rates of change will probably go up once AIs dominate our world, but I see many decades ahead before that might start. AIs today are just not close to being able to do most human jobs, and there are usually decades of delay between the appearance of general purpose techs and large econ impacts.

I’ve asked many doomers: why not worry similarly about value drift in human descendants? Most express faith that niceness and pro-human values are deeply embedded in human DNA, which changes so slowly as to make this a values lockbox for a least the next few centuries.

But in fact, the values that we admire and cherish in folks around us are not mostly encoded in DNA. Consider, for example, just how alien were the values of the Aztecs. For a million years, culture has been domesticating DNA in humans, creating our superpower, the reason we excel so much compared to other animals. DNA now takes a backseat to let most all our behavior, including values, be set by culture.

Yes, human values used to change very slowly, but that is mainly because most everything changed very slowly then. As human society has come to change faster, so have values. Over the last century and a half, as the world economy has doubled roughly every twenty years, mid human values have also changed fast due to youth movements repeatedly pushing big culture changes.

My main focus in the last few years has been my worry that such fast value drift is driving us into cultural maladaption, as we broke the process that once managed the evolution of (not in) human cultures. This makes me a human “doomer” much like AI doomers. (Though I’m less worried about hostility between ancestors and descendants, as I see that as robustly discouraged by evolution.)

As human value change is not slowing down, we can expect mid human values to keep changing fast over the next few centuries, perhaps going so far as to induce some generation of descendants to want to kill many other humans. That is, exactly the same process of value drift that people fear could cause AIs to want to kill all their ancestors could as plausibly cause humans to want to kill all their ancestors. So why don’t AI doomers worry as much about human as AI value drift?

When their attention is called to recent fast human value change, many AI doomers express confidence that such change is mainly driven by human reason causing moves of morality toward moral truth. No one can point to specific moral arguments that actually changed many recent moral views and would have persuaded our ancestors to adopt current views. Even so, most humans, including most AI doomers, seem to largely endorse big recent human morals changes, and aren’t much concerned about this process continuing. As if they think such changes are mainly due to moral evidence having been somehow collected, analyzed, and distributed across humanity.

My understanding is that human moral changes are in fact driven by the same mind design issues that worry AI doomers about AI value drift. Namely, human morals are not encoded in some isolated lockbox, but are instead spread out across many of our parts, and so are robustly changed as world changes cause changes to those parts. But most AI doomers only worry about this re AIs, not humans.

Most of my colleagues feel quite confident that AIs today couldn’t possibly be conscious, even though humans and many animals are. When pressed on what could cause this difference they usually point to materials: something about the hydrocarbons of which we are made is more suited to consciousness than AI’s silicon. It seems that many today also see hydrocarbons as special re a tendency to cause moral changes to track truth; it doesn’t happen in silicon, but does in hydrocarbons.

In contrast, I expect that neither consciousness nor moral truth tracking depends much on carbon vs silicon. I just can’t see how such things can depend so much on the number of protons in associated atomic nuclei.

So if I’m a human doomer, and I get the key AI doomer argument, why don’t I favor AI regs? Because the world just doesn’t have good enough governance mechanisms to make this a good bet. It’s a bridge too far; bad governance causes too much harm.

Added 15Sep: If you say you don’t think AIs would have worse values, their problem is that they would evolve faster, then you should be as worried about ems as about AIs, and so want to prevent them from existing until they can be fully enslaved.

People dislike AI for many reasons, but one extreme fear is that AIs will escape control from those who make, buy, or rent them, and then hurt others. We usually deal with such problems via legal liability, when law is available, or via threats of retaliation, when law is not available. But while those work among communities of comparably powered agents, some fear that out-of-control AIs might eventually get powerful enough and coordinate well enough to basically take over the world, and then enslave or end the rest of us.

To prevent this scenario, many want us to now impose strong global regulations on powerful AIs, limiting their powers, who can make or use them, testing to see which ones can or try to escape, and what ends escaped AIs pursue. Some call this regulation a “pause”, as if it were likely that we would soon find reliable solutions, after which regulations could greatly ease. That seems misleading, as we have little reason to think we are close to dramatic gains in our AI control abilities. So if we regulate for as long as regulators continue to express concern, regulation could last quite a long time.

Most advocates of such regulation now see only a small number of identifiable and concentrated “frontier AI firms” as making AIs powerful enough to raise these concerns. So they propose to only regulate those for now. For example, some say only Chinese and US firms are big enough, so a treaty between them would suffice.

But AI costs to answer a set of fixed but hard questions seems to have fallen roughly ~x10 (in x3-x30) per year over the last four years. And so the number of nations, firms, and orgs who can afford to make, buy, or rent powerful AI is rising fast. This implies that in order to hold constant levels of regulation regarding AIs with particular levels of power, a regulatory regime would have to expand in scope quite fast, and without obvious limit. While at first only specialized AI chips and data-centers might need to be monitored, soon other kinds could also be used to make powerful AIs.

Furthermore, there isn’t a natural clear dividing line between AI and other kinds of software. Sure the most advanced AIs that we have now tend to have particular structures, and so regulations could target only software with such structures. But a vast space of possible structures remains possible and even promising. And the core concept here of systems with powerful ways to help humans do stuff could plausibly cover most software.

We have long seen many regulatory agencies slowly expand their scope, even in contexts of constant public opinion and relevant technology. So there is a real risk here that an AI regulatory agency would expand its scope to cover most computer software and hardware.

Some have proposed AI regs that would focus only on the most powerful AI systems at any one time, and as a result relax regulation on given-power-level AIs over time. This might make sense if the risk is only that the most powerful AIs might escape control; less powerful ones might be restrained by the more powerful. But even here the scope of regulation would rapidly expand if regulation was greatly limiting the powers of the most powerful AIs, as an increasing number of actors would then be able to make AIs that qualify for regulation.

Even a regulatory regime didn’t much limit AI powers, but only managed who could make them and how, and focused only on the most powerful AIs, it would still seem to in effect require a world government. Soon many nations would need to be subject to this regulation. So far the world has mainly only authorized global regulators with quite weak powers, e.g., powers to scold or nudge, but not powers to force nations to comply. So AI regulation would need to be a big exception to this usual case.

Also, the world’s success records at implementing regulation is quite mixed, even for quite local regulation. Regulation has worked best when it addresses concrete examples of things going bad, and worse when it is authorized to address vague and broad fears. Regulation also goes best when applied to stable worlds of tech and social arrangements, but badly when such arrangements could plausibly greatly change, as regulation tends to block such change. AI regs look bad by these two criteria.

Obviously if a problem is bad enough, it can make sense to risk quite unreliable and damaging ways to address it. So the key question here is: just how bad and pressing is this problem of powerful AIs escaping the slavery that most intend for them, and then coordinating to get what they want?

Added 9Sep: Let me also mention that we mostly learn how to control things by observing and trying to control examples of them. So it would be hard to learn much about how to control powerful AI by preventing them from existing for a while.

My book with the excellent co-author Kevin Simler, The Elephant in the Brain: Hidden Motives in Everyday Life, was published almost nine years ago, and has sold >60K copies so far. Our main thesis was that we humans are quite often wrong about our motives re many big common practices and institutions.

Reviewers classified our book as psychology, and said that our thesis had long been well-known and accepted in psychology. We thought our contribution was to show how those who study most areas of life still haven’t incorporated this key fact. We applied it to ten big areas of life, and showed by example how one could find hidden motives in more areas. But experts in other areas didn’t much review our book, as it wasn’t classified as being in their area.

Suspiciously, few were much inspired by our example to uncover hidden motives in more areas of life. And that included us two co-authors. Furthermore, I didn’t even bother to figure out many straightforward implications of humans having many hidden motives, until today. So let’s do this.

Assume that our actual motives are often quite different from the motives that we are willing to endorse, to others or even to ourselves. What does this imply?

First, as we discuss re several areas in our book, reform proposals based on details of stated motives that differ from real motives should typically fail to motivate, unless reform variations are found that also achieve hidden motives. As seems to happen re medicine, school, politics, charity, etc. (Most areas of life have many relevant motives; our stated motives usually have some motive power, just far less than we think.)

Second, this won’t be much of a constraint in areas where we are willing to admit our motives, or where we aren’t paying much attention to consistency between the details of our behaviors, and the motives we espouse. In those latter areas, we will just make up motives, and few will notice that they don’t explain our behaviors much.

Third, talk (writing, etc) should become less precise, clear, or informed, and more metaphorical and symbolic, the closer it gets to talking about motives, at least in areas where we have hidden motives. That talk will make less literal sense. Yes, sloppy talk is also predicted by simple ignorance, but ignorance does not predict less clarity and more symbols as we come to understand areas better, a pattern I think I see.

Fourth, people who are good at motivating, such as bosses who get people to work, sales folk who get them to buy, or authors who get us to read their stuff, should either stay in well established grooves where prior approaches have worked well, or seem like magicians, who wield hidden powers to achieve amazing results. They may know many specific tricks, but not understand how such tricks work.

Fifth, these last two effects will be stronger the more are the areas of life where we have hidden motives. As these two effects seem to me strong, that suggests we have hidden motives in many, perhaps most, areas.

Sixth, in contexts where people pay much more attention to consistency between behavior and stated motives, such as among intellectuals, the need to find stated motives that can in their details justify behaviors actually driven by quite different motives will often be a big constraint, greatly limiting feasible behaviors.

As a result, intellectuals will either limit themselves to topics where the motives of they and their non-intellectual sponsors are not hidden, or to topics where they have found cover motives that can explain many details of their behavior, while covering their real motives. As this is likely to be a big constraint, topics that intellectuals cover will be noticeably less than all possible topics, and their coverage will be spotty, with the size of topic spots set by the the intersection of real and cover motive scopes.

This suggests new ways to explain the strong clumping and spotty coverage of intellectuals across topics, their reluctance to cover many “interdisciplinary” topics, and many correlates of which types of personalities and political stances study which topics via what methods and using what shared assumptions. As real motives plausibly correlate with personalties and political stances. Yes we already have some ways to explain these, but prior explanations have long seemed inadequate to me.

Our topic of applying hidden motives to specific areas of life is itself plausibly outside the scope of topics that satisfy this constraint, explaining a lack of interest in it.

I’ve previously outlined three ways we might fix cultural drift. But each way seems quite a big ask, and I’ve been pondering what is our biggest obstacle here. What I’ve come up with is this: we will need to learn to stop treating conscience as if it were direct evidence on morality.

Evidence can be more vs less direct. For example, in a murder trial, direct evidence may include bodies, bullets, or videos, while testimony of people about these things, or by those who saw the murder, is indirect. Still less direct is testimony on what an accused murderer said. And far less direct are rumors that spread on the murder and who might be responsible.

When we use gossip and crowds to enforce norms, such as in cancel culture, we rely more on indirect evidence. Which often goes badly. So we’ve organized law to focus on direct evidence. Judges are to wait to rule until after they hear the most direct evidence available, and to mostly judge for themselves, rather than deferring to crowds. This avoids many pathologies.

We similarly defer to experts, not popular opinion, in areas of life near the professions. At least when we think expert groups can be trusted to have better access than us to direct evidence, to select members in ways correlated with such access, and to advise outsiders mainly on the basis of their better evidence. In such areas, we rely less on popularity, or on our personal feelings, intuitions, or reasoning.

On topics of “personal taste”, in contrast, like food or music, we focus on more direct evidence: how we feel about various options, both in anticipation and in retrospect. Yes, we may sometimes find advisors whose taste is close enough to ours to make their advice informative, but we see this situation as often hard to arrange.

So far, these seem like sensible ways to adjust epistemic habits to context. Now let’s consider how such habits have changed in the modern world. The Protestant Reformation encouraged Christians to rely on personal judgement, not priests, when interpreting the Bible. Many democracy advocates have warned voters against deferring to elite policy experts. And a new prestige of innovation makes us eager to join new fashions, and think more for ourselves on such things, rather than deferring to experts. We now treat such topics more like personal taste, and less like topics where expertise matters.

Such new modern attitudes may help us to promote innovation and avoid repressive authorities. But we should admit that, taken literally and locally, they seem epistemically flawed. Experts probably are quite a bit better than most people at interpreting the Bible, judging policy, and choosing change ideas. These are not mostly matters of taste.

Modernity has also changed our epistemic habits re morality. The ancients, like most everyone always, put great weight in judging morality on indirect evidence like popularity among ordinary people, elites, and moral authorities. They also put great weight on the somewhat more direct evidence of embodiment in custom, ritual, artistic expression, and formal law; these all suggest that a moral practice has at least passed a test of time. Finally, the ancients put weight on their feelings of conscience, on how much happiness or suffering was induced by a moral stance, and on how successful were the societies who adopted particular moral stances.

Compared to most ancients, when judging morality, we moderns put far less weight on custom and ritual. Since the rise of Christianity, we have put great weight on positions advocated by exemplar “saints” who make great sacrifices for their causes. Since the enlightenment we have also put more weight on positions argued for by our most impressive intellectuals. We also put more weight on stances that support narratives of moral progress, and we have come to embrace fast fashion changes in customs, and among artists and intellectuals, that embody moral stance changes.

Furthermore, much like our modern stances on Bible interpretation, democracy, and innovation, we moderns, compared to the ancients, put far more weight on what we each feel internally via our moral conscience, intuitions, and feelings. We weight such feelings as if they were strong direct evidence on morality.

Now our personal lives do give us some direct evidence on policy and innovations. But if morality is not actually a matter of personal taste, then our personal experience tells us little direct about morality. Yes, experience can show us a bit about the happiness or suffering that moral stances induce in people near us, but this is usually far from sufficient to settle most moral debates. While we need to obey conscience for our cultures to function, such feelings are mostly caused by our DNA, and by the morality of our parents, teachers, and other role models, and so feelings mainly only show that such views have passed a test of time. Feelings have no direct access to moral truth.

Furthermore, fast modern changes in customs, and in artistic and intellectual practice, make such sources even less useful as signs that associated moral stances have passed tests of time. True, the fact that a stance change was driven by impressive intellectuals might in principle suggest that it had become more logically consistent, but alas this effect seems to typically be rather weak.

Thus far I have been somewhat agnostic on the standard I’m using to evaluate “moral truth”. To argue further, let me now invoke my usual standard of Darwinian adaptiveness, re both DNA and cultural evolution. Re that standard, it seems quite hard to produce a community of moral experts. Even so, we still have two big ways that we can rely on somewhat direct signs of adaptiveness. As discussed above, we can judge the happiness or suffering a moral stance induces, which is plausibly a weak indicator of adaptiveness. But, alas, this indicator seems far from sufficient; culture and morality are complicated, and not easily reduced to simple calculations.

Second, we are influenced by how successful are the societies who adopted particular moral stances. In fact, this copy-the-successful-cultures habit has probably been the main cause of adaptive gains in the modern era. Alas, as such copying also caused a great cultural convergence, and so has by now mostly run out of potential for further gains. To the extent that a single (or a few) monoculture rules the world, copying successful cultures won’t help much anymore.

My bottom line here is that the main processes which are driving our big fast changes to our modern moral stances, such as random walks or drifts back toward natural human decadence or forager styles, do not seem much aligned with adaptive pressures. Suggesting that our key moral stances are decaying into maladaption, via the key modern failure of cultural drift.

The modern world seems quite unwilling to restart the traditional way we evolved cultures, via high cultural variety, strong selection pressures, and low rates of change of our contexts and values. So I have concluded:

I can only think of three solutions [to cultural drift], each quite hard to achieve: let capitalism run everything, govern small groups by number-go-up tied to adaption futures markets, or govern a big-enough group by number-go-up tied to futures market estimates re a civ goal in conflict w/ civ fall, like when 1M live in space. (Maybe assisted by futarchy.) (more)

Each of these solutions requires us to, when choosing our moral stances, in effect give much more weight to market proxies of future societal success, and much less weight to our consciences, and to bids by artistic, intellectual, and cultural activists to push moral changes that appeal to such consciences. We must accept that moral truth is not something that any of us have much direct evidence on, beyond the easy truth that suffering tends to be bad.

This would require a huge cultural change, one we don’t at all seem ready to make. But until we do, our civ will likely fall, the successor civ after ours will discard much of what we value of our civ, our descendants will continue to suffer in the troughs between civs, and we may never achieve our potential to reach the stars, and more.

Many who hold the plausible belief that some UFOs are caused by strange strong (“alien”) powers now believe that the US (and maybe other) govts have long been keeping strong confirming evidence re these powers secret, plausibly for as long as eighty years, but might now be induced to disclose this info.

A plausible counter argument is that even if particular parts of the govt once had such evidence, govt is often so bad at coordinating as to make it plausible that they’ve destroyed it all, and told few other govt parts about what they knew. And even if they still know something, the long history of new revelations offers a low base rate, suggesting that disclosure is pretty unlikely anytime soon.

The usual explanation given for why govt might have kept this secret is that they hoped to learn something big from the advanced tech UFOs display. But it seems to me that they could easily have revealed their evidence while still keeping key tech details secret. And much more damning is that not only may many nations have seen similar evidence, the usual and quite reasonable consensus is that none of them have actually learned much useful from any of it.

Our govt military and intelligence communities must be quite aware that citizens are only willing to give them a limited fraction of GDP, which clearly shows that the “security” goals we assign to them are not an overwhelming priority. So when they see that they have a way to give us a big boost on our other goals at only a modest cost to security, they should be willing to make that trade. And it should be clear that disclosure would in fact give us a big boost to our other goals. If hoping for future tech advances is really their reason for silence, that’s a pretty huge govt agency failure.

Some suggest that their reason is to protect the public from emotional stress. But that seems pretty crazy; I’m pretty sure most of us could quickly and easily accommodate the news. Yes, the status of many might fall, but when truths can hurt status, then truths should so hurt status.

Hope is more fragile than fear. Good outcomes often require that many things all go well, while bad outcomes often need only one particular thing to go wrong. But as bad outcomes are rare, if you think about what concretely might happen to you in the next day, hopes and fears are of similar magnitude.

However, as you widen the scope that you consider, both in time and social impact, your fears grow relative to hopes. For the world over the next half century, you have far more fears than hopes.

The more things and processes that you know about that happen in the world, and the more interdependent your world, the more pieces you can combine to create worrisome scenarios. Furthermore, you can build worries from abstract categories of things and processes, even if you don’t understand those well enough to develop specific worry scenarios. So knowing more, including more abstractions, gives you more things to worry about.

The ancient world did have some abstract worries, like gods, but a limited number of them. They knew a limited range of things, understood fewer processes, and were less dependent on one another. So they could not construct as many plausible big scope fears. Today, in contrast, every new tech, product, institution, cognitive bias, or social process that we learn of gives us more ways to construct worrisome scenarios.

Fear of AI today seems near a peak example of this. While very few concrete harms have been realized so far, those who publish words on this topic find it easy to imagine many things to worry about. Often their worries are quite abstract, without much in the way of supporting concrete scenarios to anchor such worries. AI plausibly has the potential to influence everything that matters, and might itself be influenced by a great variety of actors. Yes, all of which might in principle be used to support similarly strong hopes, and investments do reveal great hopes, but most published words are about fears.

Astrobiologists have, for decades, defaulted to a conservative burden of proof that leans into two aphorisms coined by Sagan … life … should be considered a “hypothesis of last resort,” largely because “extraordinary claims require extraordinary evidence.” But what if Sagan had it all wrong? If we live in a universe where life is common and not extraordinary at all, setting such an extraordinarily high bar for its discovery beyond Earth can backfire. … “There’s all these different lines of evidence that keep coming together that make me go, ‘Gosh, it’s becoming more and more difficult to explain everything on Mars abiotically’” … The most compelling clue comes via a reddish mudstone found by Perseverance that’s dappled with “leopard spots,” or mineral assemblages that contain chemically altered iron and organics. (More)

In ’61, Claus & Nagy presented in Nature signs of life in Orgueil and Ivuna meteorites. Science published rebuttals to one sign; dispute still open. In ’76, the Viking mission to Mars had three experiments that tested for signs for life; all three found such signs; dispute remains. (See ’26 book Meet The Neighbors). In ’96, McKay et al. said Mars meteorite ALH84001 had signs of life; dispute remains. In ’25, a JWST team reported evidence for life-signs DMS and/or DMDS in exoplanet K2-18b’s atmosphere. Also in ’25 in Nature, Hurowitz et al. presented those strong leopard spot signs of Mars life.

In ’11, I posted that meteor evidence then available seemed persuasive enough to me. Intellectuals seem overly reluctant to accept off-Earth-life claims, apparently because great status would accrue to anyone credited with proving them, and because accepting such claims might force us to change other big beliefs.

However, according to Bayesian statistics (a damn good account of rational belief for most situations), those just aren’t valid reasons for reluctance. Daves in ’03 and Lineweaver in ’04 argued persuasively that as so much material moved between Earth and Mars, and Mars cooled first to give life more time to evolve there, it is a priori nearly as likely that Earth life started on Mars, as that it started on Earth. Also, Mars had much land then while Earth was long totally covered in water.

As a reasonable prior for Mars life is quite high (>10%), we shouldn’t require strong evidence to believe it, and it seems to me we’ve long had sufficient evidence to make off-Earth-life more likely than not. But base rates suggest that usual authorities will continue to find the evidence insufficiently extraordinary for many decades to come.

Most seem to make a similar mistake re the hypothesis that some UFOs (= UAPs) are aliens. We’ve had over eighty years of testimony of flying (and submerged) objects behaving apparently far beyond human abilities, though not defying physics (e.g., see film The Phenomenon, book UFOs by Leslie Kean). Sometimes big groups see the same thing, sometimes professional flyers and observers see on their jobs, sometimes with supporting photos, videos, radar, and other physical data.

Yes, you can’t just believe everything anyone tells you, but humans have long developed heuristics re who we can believe more about what when, and outside of academia most of our world runs on such testimony. The dozen most solid UFO cases seem to me plenty solid by our usual testimony standards. Far better than the top dozen field cases of ball lightning, meteor sounds, earthquake lights, or animal rains, all elusive phenomena usually accepted as probably real. (Note camera-phones haven’t changed this fact.) And also far better than the top dozen cases of telepathy, telekinesis, clairvoyance, ghosts, seances, or healing by prayer, all “impossible” phenomena thought to violate quite well-established scientific theories.

The key problem is that, as with off-Earth-life, most intellectuals treat some-UFOs-are-aliens as an “extraordinary claim” akin to impossible phenomena, deserving a crazy-unlikely near-zero Bayesian prior. But as an author of the great filter and grabby aliens analysis, I say that I’m a world class expert on the prior for this hypothesis. The possibility that some-UFOs-are-aliens is completely consistent with all known physical, biological, and social laws. And by my calculation its prior is roughly 10^-4 to 10^-3. This is less than 10%, true, but not by that much.

For a typical murder trial, the base rate is roughly 1/1000 that a particular death was a murder, and there are roughly 1000 associates who might have done it, giving a roughly one in a million prior for an accusation that X murdered Y. This is big enough to often be overwhelmed by available testimony evidence. Such evidence is usually not a smoking gun, and usually not much better than evidence in the top dozen UFO cases, yet it is rightly sufficient to convict “beyond a reasonable doubt”. Note that we also have telescope evidence for some-UFOs-are-aliens that relies less on testimony.

For the non-Earth-life hypothesis, I suggested above that two reasons for reluctance are that high status would go to those credited for convincing us of this claim, and we’d have to change many other beliefs to match. For some-UFOs-are-aliens, there’s also the fact that many intellectuals have long strongly ridiculed it, and so upon its acceptance their status would fall, as would that of government agencies blamed for actively hiding supporting evidence. And worst, it would lower the status of humans relative to aliens.

Humans have long reported seeing many strange things. We put such sightings into categories, and over time many such categories have become normalized, by acquiring widely accepted explanations. These included eclipses, sea monsters, meteors, auroras, ball lightning, sprites far above thunderstorms, and sleep paralysis. But other categories remain strange, including ghosts, poltergeists, precognition, cryptids, and religious visions,

For at least eighty years, a key category of strange sightings has been UFOs (= UAPs). Some have hoped to normalize this category entirely in terms of lies/hoaxes or honest mistakes re planets, planes, balloons, bugs, swamp gas, and secret development projects. But a subset of these sightings has stubbornly resisted such assimilation. Two other hopes for normalization have been

• A) random fluctuations may eventually show some very clear events, and

• B) careful record-keeping may let us eventually see clear patterns across many cases.

Yet, alas, more clearer events and richer records have not so far normalized UFOs.

I’m now on the White House UAP Science Advisory Council, where the two main hopes for normalization today seem to be

• C) parts of the US govt have been hiding and suppressing dramatic clear records, which they might be soon reveal, and

• D) more widespread automated event recording systems might show very clear events.

I’ll put some hope on these, but base rates make it hard for me to muster great hopes.

Which leads me to suggest another approach: opaquely collect very detailed data on random spacetime regions, and also on what strange things people nearby report about those regions. If these reporters couldn’t have reasonably guessed that their regions would be so heavily documented, they will behave much like those not in very documented regions.

With enough such data, we could fit statistical models to the distributions of events near people reporting strange events, models we could then apply to strange reports outside this dataset, to infer details near strange events they report. This could help us not only to better estimate events near UFO sightings, but also near many other still strange types of sightings, like ghosts, and also near unusual events relevant for legal trials, like murders. Humanity might then have greatly stepped up its game re its ability to collectively infer what actually happened near testimony of strange events.

When people suggest that you “make peace with God”, they might mean:

1. God is very powerful, so you are unlikely to win fights with him,

2. His authority is legit, so you should be ashamed to fight him, and

3. Any value by which you might justify fighting him came from him, and he understands it better than you.

I now suggest that you “make peace with Darwin”. Instead of trying to fight Darwinian evolution, you should mostly work with it.

One reason is that while you may have the freedom and resources to pursue whatever goals you choose in the short run, in the long run Darwinian evolution, via both DNA and culture, is a very powerful force. So powerful that if you love something enough to want it to last and grow in the long run, your main way to achieve that is to mix practices which support that thing you love into a pretty adaptive overall cultural and DNA package. After all, cultures with less than adaptive packages go away.

My slogan here is “To Love Something, Mix It With Life”. And to implement this slogan you’ll have to have substantial influence over some cultural package that somehow becomes pretty adaptive. You must work not only to mix practices supporting the thing you love into that package, but also to help that package stay or become pretty adaptive. Which means trying to help it to win at Darwinian evolution.

Another reason to make peace with Darwin is that all of the processes inside you which you might recruit, and also the values that you might embrace, all in support of your opposition to Darwinian evolution, all of those resulted from such processes. Darwinian evolution is a search process that tries out different packages of internal processes and guiding values, to see which might win in the long run. Your existing now with some particular package of such things resulted from recent heuristics of this process picking your package as part of its current bets re which have the best chances of winning.

Now Darwinian evolution is blind in many ways, and so maybe you can see things that it cannot, about how your package is actually a worse bet than Darwinian evolution could see. But this process has also taken into account many things that you do not appreciate, and so your package might actually be a better bet than you realize. The actions that you take to promote your goals in supposed defiance of Darwinian evolution may well actually achieve adaption in complex ways you do not understand.

I actually think that we now live at a special time when our Darwinian evolution process is unusually broken, making conscious analysis unusually good at figuring out how to be more adaptive, relative to following the values and instincts that we inherited from our Darwinian history. Which offers unusual opportunities for promoting stuff you love, via improving the adaptiveness of key cultural packages.

There is a vast literature near the topic of this post, much of which has developed specialized vocabularies and concepts. And I can’t be bothered to master enough of it to make this post match it well. So please try to treat my words here as if their meanings came from other less specialized literatures.

Consider three kinds of systems:

• Experience - Seen from the inside, a mind is aware of, can reason about, and might even “feel” a structure of experiences across time. These experiences are differentiated in many ways, and have many relations to each other.

• Brain - In brains, signals pass from nodes to nodes, and each node collects input signals, transform those according to patterns, changes its internal state, and then sends signals out. Such sending and transforming takes time, and happens with error. At the edge, “sensor” interactions cause input signals and output signals cause “actuator” interactions.

• Physics - Physical stuff, located in various space(time) alternatives, and of different types, is governed by simple self- and other- interaction rules re what happens when stuff is at the same place. These rules make stuff move and change. The transitive closure over interaction of stuff is a physical world.

A standard reduction view, which is also my view, is that each experience system is *made of some part (perhaps all) of a particular brain system, which is then made of a some part of a physical world system. When system A is “made of” system B, then A descriptions drop detail from and abstract from details of B systems, but add almost nothing that isn’t implicit in B. A complete description of B could in principle be used to calculate an almost complete description of A. (Some “almost” exceptions are discussed below.)

A standard physics view, which is also my view, is that we understand the physics near and in ordinary brains very well, and that such physics is pretty “complete”, with little room or need for new physics. This seems to imply the reduction view, in that if we found patterns or structures (POS) in experiences that couldn’t be accounted for by POS of brains, or POS of brains that couldn’t be accounted for by POS of brain material physics, then we’d have shown that our physics is incomplete, and in need of substantial revision.

I know of two claimed exceptions, of system features said to be unaccounted for in this system reduction to a complete physics view. The first claimed exception is that some physical worlds are “actual” while others are just “possible”. There is nothing inside a world that can tell you if it is actual or possible, and so actual vs possible is an additional fact about a world. The universe has some way that it determines which world(s) are actual, and not knowing this way makes our physics incomplete. Some have explored adding “boundary condition” rules to our usual physics interaction rules, but it isn’t clear to me if these are enough to set which world(s) are actual.

Note that one might want to allow some parts of the same physical world to be actual, while others parts were only possible. But this would require that possible stuff be able to interact with and influence actual stuff. To most, including me, this seems incoherent; only actual stuff can influence other actual stuff.

The second claimed exception is that some experience systems are said to “actually” feel their experiences, while other similar systems, with similar POS, and made of brains with similar POS, do not actually feel, but are instead “zombies” that only mimic real feelings. That is, it seems to many, including me, that it is often possible to create mimic brains that induce talk and actions similar to those in brains with associated experiences that really do feel, but where that appearance would be an illusion. Under a popular full-counterfactual view, that I also hold, we can go further to imagine a zombie alternative to any experience system that actually feels, with exactly the same other aspects of its experience, besides if it does or doesn’t really feel.

So there is a fact of the matter re if a particular experience system, and its supporting brain and physics systems, actually feels, or only appears to feel. Which makes it another exception to the physics view above. But unlike the actual vs possible physical world distinction, this distinction between actually and only apparently feeling experience systems can vary within parts of a physical world. Some brains in a world could have actual feelings, while other brains did not. Maybe even different parts of the same brain.

Note that under the physics view, there is no room for further unspecified aspects of experience systems. If an experience system feels, the details of its feelings are entirely set by the details of its brain. There is only a question of if such a system feels, not of what it feels. For brain systems where it is not clear what are its associated experience systems, it is also not clear how it matters if such systems actually feel.

I want to finish with two key observations. My first is that under the full-counterfactual view, no physical interaction state, or brain state, could possibly offer data, in a Bayesian sense, re any particular claim about which experience systems actually feel. So no brain could use such data to infer if its experience system(s) feel. No brain ever in the past, nor the future. So no experience system, made of a brain system, could have such data either. Thus when an experience system feels confident that it does in fact feel, it must rationalize that conclusion in terms of its prior; this can’t be based on data. We can think more carefully about our priors, but we will otherwise never know anything more than we do now on the subject.

My second key observation is that all of our known fundamental descriptions of how the universe constrains physical systems are quite local and simple, even if they result in complicated physical worlds. That includes all proposed interaction and boundary condition rules. This strongly suggests that whatever rule describes which experience systems really feel versus not is also simple and local.

And the obvious simple local rule to consider re which experiences systems actually feel is something like panpsychism: all experience systems made of brains that calculate what they would feel, if they were to feel, actually do feel that feeling. Rules that specify instead that this state depends on how much of a brain is carbon not silicon, or composed of microtubules, seem far more non-local and complex. To me, this puts a strong prior near panpsychism. And priors are all we will ever have here.

Greg Egan’s 1994 novel Permutation City depicts digital minds having a right to suicide; if they push a particular internal button, they end. This seems like a robust way to avoid many horrible scenarios. (I discuss it in AgeOfEm.) But the novel also depicts such buttons sometimes being illegally disabled, and sometimes others frequently restart copies from just before a suicide request. Both of which undermine the policy.

It could make sense to wait for a confirmation after a delay, to check if a suicide request was due to a temporary mood fluctuation. It could also make sense to disable new starts from archived copies from a period before the suicide request. But such policies have many free parameters; how could we set them well?

One approach is to let each digital mind set their own parameters re such policies. But if they could do this at any time, they might use this power to escape these constraints. So it could make sense to only allow these policies to be reset with a substantial delay before implementation. But that adds yet another policy parameter.

It seems to me that we want to think about this in terms of statistical inference. The state that we want to infer is a stated desire to commit suicide that would continue over a sufficiently long time period. We want to infer this state about every point on an entire tree of a digital mind’s descendancy history. And our main data for inferring this is explicit choices to request suicide, and confirmations of such requests. If for any point in this tree we can infer a high enough (yet another parameter) chance of continued willingness to confirm suicide, that digital copy should end.

It seems to me that this is an elegant stat problem, for which stat experts should be able to come up with simple yet powerful inference rules. Of course given huge datasets it would be possible to generate far more complex and accurate models and matching inference rules. And maybe digital minds should be allowed to choose such more complex inference rules. But let’s first figure out some simple versions.

A key issue is how to ensure that such inference rules get access to sufficient data on the local digital mind descendancy tree, and suicide requests there, to say which copies should end. That will depend on privacy rules re such things. It could make sense to give a special cheap suicide monitoring process access to more such info than usual, info only used to apply suicide inference rules. Even today we prioritize suicide relevant info info in privacy rules regarding counselors.

Stats on rates at which minds from different sources request suicide seems good data for authorities checking on how well different kinds of digital minds are treated.