The essential problem with AI safety: there will always be some people who are willing to roll the dice.
Smart people are often rather dumb about AI. Specifically, smart people tend to hang out with other smart people. So they say silly things like “AIs aren’t nearly as smart as humans”
The All-or-Nothing Pause Assumption: people assume that if we can’t pause AI perfectly and forever, it’s uselesss.
If you are a nerd and lonely, apply your nerd powers to social skills. Rational optimization works for pretty much everything, including how to get along with people