Comments on: Why an AI-based singularity? http://ferrouswheel.me/2009/08/why-an-ai-based-singularity/ watching the world turn. Sat, 10 Dec 2011 02:46:55 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Joel http://ferrouswheel.me/2009/08/why-an-ai-based-singularity/comment-page-1/#comment-985 Joel Sun, 02 Aug 2009 22:29:52 +0000 http://ferrouswheel.me/?p=324#comment-985 There are some subtle issues with an AI's desire being to "make humans happy". The first part of http://singinst.org/upload/CEV.html gives some idea of this. But friendliness is definitely a concern. OpenCog is a long way off requiring active friendliness engineering efforts. There are some subtle issues with an AI’s desire being to “make humans happy”.

The first part of http://singinst.org/upload/CEV.html gives some idea of this.

But friendliness is definitely a concern. OpenCog is a long way off requiring active friendliness engineering efforts.

]]> By: James MM http://ferrouswheel.me/2009/08/why-an-ai-based-singularity/comment-page-1/#comment-982 James MM Sun, 02 Aug 2009 06:18:59 +0000 http://ferrouswheel.me/?p=324#comment-982 "If we gave an AI an extremely strong desire/goal to make humans happy, and the AI design had some kind of reward based system (such that the AI was trying to maximise these) and achieving these rewards caused other effects in the AI’s mind, such a propensity to use positive phrases to describe the world, then is there any reason to believe that the AI isn’t happy itself?" I find this design-plan suggestion very attractive! Promise me you'll work on this project ;-) “If we gave an AI an extremely strong desire/goal to make humans happy, and the AI design had some kind of reward based system (such that the AI was trying to maximise these) and achieving these rewards caused other effects in the AI’s mind, such a propensity to use positive phrases to describe the world, then is there any reason to believe that the AI isn’t happy itself?”

I find this design-plan suggestion very attractive! Promise me you’ll work on this project ;-)

]]>