Written with assistance from ChatGPT.

I’ve been thinking about the difference between our current version of AI, artificial general intelligence (AGI), and what I’d call human-determined AI (HDAI).

Current AI can already reason across many subjects, write code, analyze information, and use tools, but its goals, permissions, and boundaries are still heavily determined by humans. AGI is usually defined by general intellectual capability, but general intelligence doesn’t necessarily imply independence from human direction.

This also seems important for AI safety. Perhaps the goal shouldn’t simply be making AI more intelligent, but making increasingly capable AI remain reliably aligned with human intentions, constrained where necessary, transparent about uncertainty, and subject to meaningful human oversight.

Could we eventually have something that qualifies as AGI while remaining highly human-determined? Would that be safer than increasingly autonomous AI, or could dependence on human direction introduce its own risks—especially when humans disagree about what AI should be allowed to do?

Is highly autonomous AI / AGI important? Why / why not?