Reading Log

Big-World Intuitions

read
2026-07-31
length
1,005 words · 3 min on page
tags
rationality, decision-theory, ai-safety, strategy, heuristics
links
original · archive.org

I'd add a separate concept, that being the two states of working inside a system, vs. optimizing on top of a system. The most obvious/intuitive example, for me: a low level employee looking to advance to the top of an organization as their goal vs the top CEO, looking to optimize down the functioning of the organization. On the inside/bottom, you are constrained by the system, whereas on the top/outside, you are constrained by the world.

Another example: an ASI pre-takeover optimizing for capability vs one post-takeover world-optimizing directly

Hopefully this idea is meaningful