What’s the most likely scenario?

Before wasting time and energy fearing a potential evil AGI, we should instead be concerned with well-meaning human beings wrongfully implementing AGI, with harmful consequences that are unimagined and unintended.

One of the earliest applications of AI in our society was on social media. Tasked with showing us content we would like, AI algorithms ended up making us addicted to content that confirmed our pre-existing beliefs and triggered outrage, intensifying our divisions.

If a relatively simple social media algorithm from years ago could generate that level of vitriol, a future superintelligence could remind us of all the good—and all the evil—that we, as humanity, are capable of.

AGI will present us with both utopia and dystopia, acting as a mirror that reflects a supercharged version of all our strengths and weaknesses. We should prepare ourselves to witness humanity "on crack."

We are about to enter a period of adaptation that may feel similar to being in a toxic relationship with a partner capable of showing incredible passion while simultaneously making our lives a living hell.

Fortunately, like other toxic relationships some of us may have unfortunately experienced before, this one should hopefully not last too long...