But… Why don’t we align AGI with human values?

It is currently not possible to guarantee a future AGI will behave in ways that align with our own human objectives.

A first step would theoretically be to develop AI systems that are transparent and explainable, so their decision-making processes can be understood and evaluated by humans. We could then attempt to align its algorithms with human values... The problem is, WHOSE values?

In an era in which society seems entrenched in opposite ends of a culture war, how can we even agree on what values to program?

Will AI be subject to a singular moral code or a multiplicity of cultural perspectives? Who should define AI’s moral values? Corporations, developers, regulatory bodies, or society as a whole?

Whoever can solve the AI alignment problem would probably be the first triple Nobel Prize winner for the Peace, Economics, and Science prizes.