应用治理:谁签字,谁负责
AI参与生成的产出物,出了错谁签字负责?"AI辅助"与"AI决定"的边界,在实践中应该划在哪?
Who signs when AI-generated output goes wrong? Where is the line between AI-assisted and AI-decided?
Let me get technical, because the risk is technical. Modern AI doesn't fail like software. A large language model that's wrong doesn't crash — it answers fluently, confidently, and wrong. Hallucination isn't a bug you patch; it's a property you manage: retrieval grounding so answers cite real sources, evaluation sets so you notice regressions, and regression testing every time the model or the prompt changes. And once you give a model tools and autonomy — agents, workflows — you multiply the value and the blast radius.
So the boundary between "AI-assisted" and "AI-decided" is not a philosophy question. It's a consequence question. Three lines I'd draw. One: tier by consequence. Drafting and summarizing flow free. Anything that moves money, touches health, or carries legal weight gets a named human owner. Two: checkpoints live inside the workflow, not inside a memo. Every AI output should carry three fields — who owns it, what data fed it, which human reviewed it. Three: the signature transfers liability to a person. That's not overhead; that's the feature.
Full disclosure: your host from Tongwei handed me an AIPM demo account at dinner last night and said, "Try to break the governance layer." Three hours later I gave up — every output has an owner, a status, an audit trail. I came here for a quiet conference and got peer-reviewed by a workflow engine. That's the most backhanded product pitch I have ever enjoyed — and it worked.
US and China, briefly — two laboratories, two philosophies. In the US, governance is market-driven: NIST's AI Risk Management Framework is voluntary, but procurement makes it mandatory in practice — no red-team report, no enterprise deal. In China, it's license-driven: under the 2023 interim measures for generative AI, services pass security assessment and file for record before launch — assessment first, go-live second. "Ship and show me" versus "file and then ship." Different roads, same destination: both systems make one person indispensable — the one who signs.
说点技术细节,因为风险本身就是技术的。现代AI的失效方式和软件不一样。出错的大语言模型不会崩溃——它会流畅地、自信地、错着回答你。幻觉不是能打补丁的bug,是需要管理的属性:用检索增强让回答有据可查,用评测集发现能力回退,模型或提示词每改一次就回归测一次。而一旦给模型装上工具和自主性——智能体、工作流——价值和爆炸半径同时翻倍。
所以"AI辅助"和"AI决定"的边界,不是哲学问题,是后果问题。我画三条线。一、按后果分级。起草摘要放开跑;凡动钱、碰健康、带法律后果的,必须有具名的人类责任人。二、检查点长在工作流里,不躺在通知里。每份AI产出带三个字段——责任人、数据来源、哪个人审过。三、签字把责任转移给个人。这不是开销,这正是功能本身。
坦白讲:昨晚通维的东道主给了我一个AIPM演示账号,说"你试试把治理层搞坏"。三小时后我放弃了——每份产出都有责任人、状态、审计留痕。我是来开会的,结果被一个工作流引擎做了同行评议。这是我这辈子吃过的最别扭的产品安利——而且生效了。
美国和中国,简短说——两个实验室,两种哲学。美国是市场驱动:NIST的AI风险管理框架是自愿的,但采购让它变成事实强制——拿不出红队报告,企业订单就悄悄死掉。中国是许可驱动:2023年生成式AI暂行办法要求服务上线前过安全评估、做备案——先评估、后上线。一个"先跑给我看",一个"先备案再跑"。路不同,终点相同:都让同一个人不可替代——签字的那个人。
In PMBOK terms, AI governance is not a new knowledge area — it's a new risk register page inside the old one. Treat AI outputs like deliverables that face a quality gate, and add the three fields Bruce named before the gate opens. Review after the gate is archaeology; review before it is governance.
用PMBOK的话说,AI治理不是新增知识领域,而是旧风险登记册里新加的一页。把AI产出物当成要过质量门的交付物,在开门之前加上布鲁斯说的那三个字段。门后评审是考古,门前评审才是治理。