I wanted to quote a couple things from this OpenAI blog post on alignment by Jakub Pachocki:
The fundamental challenge of AI alignment is generalization. As machines become smarter, they find themselves working on higher-level concepts, and placed in environments increasingly different from those they encountered in training. They can fail at generalizing from the values taught and reinforced in their training process to those new situations; and it can be hard for us to be sure how they will act.
And:
We are facing a transition to a world with incredibly intelligent machines, and we need to ensure that transition works out well for humanity. We need to find ways to preserve human agency and enshrine an intrinsic value to being human, in a world where most tasks could be performed by AI.
When I read this sort of thing, it strengthens my belief that the big AI labs are fundamentally different from the ad-based big tech companies in Silicon Valley like Meta. In terms of public backlash, they are all lumped together. It’s that foundation of distrust in big tech that the more significant frustration with AI is built on.
But the technology and vision are different. It’s hard to imagine Meta being as thoughtful as Jakub is in the essay linked above. Mark Zuckerberg does have his op-eds, but they usually ring hollow.
