Will AI Enslave Humanity?
A friend recently asked me:
“Do you think AI will one day enslave humanity?”
I think there are two ways to look at this question.
1. AI reward is not human desire.
Humans are driven partly by biological mechanisms such as dopamine. These endogenous drives can create desire and impulses—even causing us to act against the rules and long-term goals we consciously set for ourselves.
AI is different.
AI may have reward mechanisms and objectives, but these are fundamentally different from biological desire:
AI: externally specified objective → reward → optimization → behavior
Human: endogenous biological drive → desire/impulse → behavior
So:
Reward is not desire.
Optimization is not desire.
Goal-directed behavior does not necessarily imply subjective desire.
We currently have no evidence that AI experiences desire in the human sense.
But this does not mean AI cannot produce dangerous outcomes.
An AI does not need to want to enslave humanity. If the objectives, rewards, or constraints we give it are poorly designed, optimization itself could produce outcomes we never intended.
AI may not escape the objective, but it can escape the designer’s intent.
2. Perhaps technology is already “enslaving” us.
Kevin Kelly, in What Technology Wants, describes technology as an evolving system—the technium.
Humans create technology, but technology also reshapes humans.
Smartphones changed our attention.Social media changed our relationships.The Internet changed how we acquire knowledge.AI is now changing how we work, learn, create, and think.
Today, companies, engineers, investors, and nations are racing to develop increasingly powerful AI.
Why can’t we simply stop?
Because if one company stops, another continues. If one country stops, another continues.
Technology begins to look like a self-propelling evolutionary system—not because it has consciousness or desire, but because human competition keeps driving it forward.
So perhaps the real question is not:
“Will AI want to enslave humanity?”
But:
“Are we still using technology—or is technology beginning to use us as well?”
AI 會奴役人類嗎？
最近，一位朋友問我：
「你覺得 AI 有一天會奴役人類嗎？」
我認為，可以從兩個角度來看這個問題。
1. AI 的獎勵機制，不等於人類的慾望
人類的行為部分受到多巴胺等生物機制驅動。這些內生的驅動會形成慾望與衝動，甚至讓我們違反自己理性上設定的規則與長期目標。
AI 不一樣。
AI 雖然也有 reward mechanism 和 objective，但它們與人類的生物性慾望有本質上的不同：
AI：外部指定的目標 → 獎勵 → 最佳化 → 行為
人類：內生的生物驅動 → 慾望／衝動 → 行為
所以：
Reward 不等於 desire。Optimization 不等於 desire。Goal-directed behavior 也不代表一定存在主觀慾望。
至少目前，我們沒有證據顯示 AI 具有像人類一樣的主觀慾望。
但這不代表 AI 不可能造成危險的結果。
AI 根本不需要「想」奴役人類。如果我們給它的目標、獎勵或約束條件設計不當，單純的最佳化過程，就可能產生我們從未預期的結果。
AI 不一定會跳脫 objective，卻可能跳脫 designer’s intent。
2. 也許科技早就在「奴役」我們
Kevin Kelly 在 What Technology Wants 中，把科技描述成一個持續演化的系統——technium。
人類創造科技，但科技也會反過來塑造人類。
智慧型手機改變了我們的注意力。社群媒體改變了我們的人際關係。Internet 改變了我們獲取知識的方式。AI 正在改變我們工作、學習、創作，甚至思考的方式。
今天，企業、工程師、投資者與國家都在競相發展更強大的 AI。
為什麼我們不能停下來？
因為一家公司停下來，另一家公司會繼續；一個國家停下來，另一個國家會繼續。
科技於是開始呈現出一種自我推進的演化系統——不是因為科技具有意識或慾望，而是因為人類自身的競爭不斷推動它前進。
所以，也許真正的問題不是：
「AI 會不會想奴役人類？」
而是：
「究竟是我們還在使用科技，還是科技也已經開始反過來使用我們？」
