Ben Goertzel recently asked several people for comment about open-source AGI development for a couple of pieces of writing he’s working on. I thought I’d share my own responses, and I’ll update the post later with Ben’s finished product that will have responses from others too.
What are the benefits you see of the open-source methodology for an AGI project, in terms of effectively achieving the goal of AGI at the human level and ultimately beyond? How would you compare it to a traditional closed-source commercial methodology; or to a typical university research project in which software code isn’t cleaned up and architected in a manner conducive to collaborative development by a broad group of people.
I believe open source software is beneficial for AGI development for a number of reasons.
Making an AGI project OSS gives the effort persistence and allows some coherence in an otherwise fragmented research community.
Everyone has there own pet theory of AGI, and providing a shared platform with which to test these theories I think invites collaboration. Even if the architecture of a project doesn’t fit a particular theory, learning that fact is something that is valuable to know along with where the approaches diverge.
More than one commercial projects with AGI-like goals have run into funding problems. If the company then dissolves there will often be restrictions on how the code can be used or it may even be shut-away in a vault and never be seen again. Making a project OSS means that funding may come and go, but the project will continue to make incremental progress.
OSS also prompts researchers to apply effective software engineering practices. Code developed for research often can end up a mess due to being worked on by a single developer without peer review. I was guilty of this in the past, but working and collaborating with a team means I have to comment my code and make it understandable to others. Because my efforts are visible to the rest of the world there is more incentive to design and test properly instead of just doing enough to get results and publish a paper.
How would you say OpenCog has benefitted specifically from its status
as an OSS project so far?
I think OpenCog has benefited in all the ways I’ve described above.
We’re fortunate to also have had Google sponsor our project for the Summer of Code in 2008 and 2009. This initiative brought in new contributors as well as helped us improve documentation and guides for
making OpenCog more approachable to newcomers. As one might imagine, there is a steep learning curve to learning the ins and outs to a AGI framework!
In what ways would you say an AGI project differs from a typical OSS project? Does this make operating OpenCog significantly different from operating the average OSS project?
One of the most challenging things of building an OSS AGI project compared to any other is that most OSS projects have an end use. A music player plays music, a web server serves web pages, and a statistical library provides implementations of statistical functions.
An AGI on the other hand doesn’t really reach it’s end use until it’s complete. Thus creating packaged releases and the traditional development cycle is not as well defined. We are working to improve this with projects that are applying OpenCog to game characters and other domains, but the core framework is still a mystery to most people. It takes a certain level of investment before you can see how might apply the server and other aspects of OpenCog in your applications.
However, a number of projects associated with OpenCog have made packaged releases. RelEx, the NLP relationship extractor, and MOSES, a probabilistic genetic programming system, are both standalone tools.
Some people have expressed worries about the implications of OSS
development for AGI ethics in the long term. After all, if the code
for the AGI is out there, then it’s out there for everyone, including
bad guys. On the other hand, in an OSS project there are also
generally going to be a lot more people paying attention to the code
to spot problems. How do you view the OSS approach to AGI on balance
— safer or less safe than the alternatives, and why? And how
confident are you of your views on this?
I believe that the concerns of OSS development of AGI are exaggerated. We are still in the infancy of AGI development and scare-mongering by saying that any such efforts shouldn’t happen won’t solve anything. Much like prohibition, making something illegal or refusing to do it will just leave it to more unscrupulous types.
I’m also completely against the idea of a group of elites developing AGI behind closed doors. Why should I trust self-appointed guardians of humanity? This technique is often used by the less pleasant rulers of modern-day societies: “Trust us – everything will be okay! Your fate is in our hands. We know better.”
The open-source development process allows developers to catch the coding mistakes of one another. When a project reaches fruition, they typically have many contributors and many eyes on the code will catch what smaller teams may not. However, it also allows other Friendly AI theorists to inspect the mechanism behind an AGI system and make specific comments about the ways in which Unfriendliness could occur. When everyone’s AGI system is created behind closed doors, these specific comments can not be made, or proven to be correct.
Further, a lot behind the trajectory of an AGI system will be dependent on the initial conditions. Indeed, even the apparent intelligence of the system may be influenced by whether it has the right environment and whether it’s bootstrapped with knowledge about the world. Just like having an ultra intelligent brain sitting in a jar with no external stimulus will be next to useless, so will a seed AI that doesn’t have a meaningful connection to the world… (despite potential claims otherwise I can’t see seed AI developing in a ungrounded null-space).
I’m not 100% confident of this, but I’m a rational optimist. Much like I’m a fan of open governance, I feel the fate of our future should also be open.
Are there any other relevant questions you think I should have asked?
If so feel free to pose and answer them for me 😉 …
When will the singularity occur? … would be the typical question the
press would ask so that they can make bold claims about the future
But my answer to that is NaN. 😉