AGI影响专家和友好的AI专家

||分析

Miri的使命是“确保创造聪明的智慧具有积极的影响。”实现这项任务的核心策略是找到和培训一个人可能会打电话“agi.影响专家“和”Friendly AI专家。“

agi.impact experts发展与预测技术发展有关的技能(例如,建筑计算模型of AI development or reasoning aboutintelligence explosion microeconomics),预测AGI可能对社会的影响,并确定哪些干预措施最有可能增加人类安全地驾驭AGI创造的机会。有关概述,请参阅Bostrom&Yudkowsky(2013年);Muehlhauser&Salamon(2013)

友好的AI专家develop skills useful for the development of mathematical architectures that can enable AGIs to betrustworthy(或“人类友好”)。这项工作是在Miri研金宝博娱乐究研讨会在各种出版物中,例如Christiano et al. (2013);Hibbard (2013)。请注意,选择“友好的AI”术语(部分)以避免我们理解这一主题的建议非常好 - 像“道德AI”一样的短语可能听起来像是通过寻找它的那种东西在百科全书中,我们对此值得信赖的AI的理解是贫穷的。

Now, what do we mean by “expert”?

Read more »

“Intelligence Explosion Microeconomics” Released

||Papers

米里’s new, 93-page technical report by Eliezer Yudkowsky, “智力爆炸微观经济学“现在已经发布了。该报告解释了我们研究计划的一个开放问题之一。金宝博娱乐这是抽象:

I. J. J.良好的“情报爆炸”论文是一个充分先进的机器智能可以建立一个更聪明的版本本身,这可能反过来建立一个甚至更聪明的版本,并且这个过程可以继续超过人类智慧。As Sandberg (2010) correctly notes, there are several attempts to lay down return-on-investment formulas intended to represent sharp speedups in economic or technological growth, but very little attempt has been made to deal formally with I. J. Good’s intelligence explosion thesis as such.

我将关键问题确定为对认知再投资的回报 - 投资更多计算能力,更快的计算机或改进的认知算法的能力,以产生产生更大的大脑的认知劳动,更快的大脑或更好的思维设计。世界上有许多现象,这是与本问题的证据表明相关的,从观察到的同性恋演变,摩尔定律,摩尔法的竞争力,更多。金宝博官方我对这种辩论进行了一些深度,然后出现了如何解释这种证据。我建议在智力爆炸上分析职位的下一步前进将是正式化投资曲线,使得每个立场可以正式说,他们认为它们被已经制作的历史观察伪造的可能微吻合。更一般地说,我对“认知再投资”或“智力爆炸微观经济学”的回报造成了多个开放问题。虽然此类问题迄今为止收到了很少的关注,但它们看起来与影响地球始于智能生活的结果的政策选择非常相关。

公众讨论这项研究的首选地方是金宝博娱乐这里。还有一个用于技术讨论者的私人邮件列表,您可以申请加入这里

“奇点假设”发表

||Papers

奇点假设Singularity Hypotheses: A Scientific and Philosophical Assessment现在已由Springer发布,在HardCover和电子书表格中。

The book contains 20 chapters about the prospect of machine superintelligence, including 4 chapters by MIRI researchers and research associates.

“Intelligence Explosion: Evidence and Import”PDF.)由卢克·迈翰万家和(以前的Miri研究员)Anna Salamon评论金宝博娱乐

关于三个索赔的证据:(1)有一个大量的机会,我们将在2100之前创造人级AI,即(2)如果创造人级AI,则会有一个很好的机会,超人艾美岛将遵循一个“情报爆炸”,(3)一个不受控制的情报爆炸可能会摧毁我们的价值的一切,但如果我们能够实现它,受控情报爆炸会使人类受益。我们结束了建议,以增加相对于不受控制的智力爆炸的受控情报爆炸的几率。

“智力爆炸和机器伦理”PDF.) by Luke Muehlhauser and Louie Helm discusses the challenges of formal value systems for use in AI:

许多研究人金宝博娱乐员认为,一个自我完善的artificial intelligence (AI) could become so vastly more powerful than humans that we would not be able to stop it from achieving its goals. If so, and if the AI’s goals differ from ours, then this could be disastrous for humans. One proposed solution is to program the AI’s goal system to want what we want before the AI self-improves beyond our capacity to control it. Unfortunately, it is difficult to specify what we want. After clarifying what we mean by “intelligence,” we offer a series of “intuition pumps” from the field of moral philosophy for our conclusion that human values are complex and difficult to specify. We then survey the evidence from the psychology of motivation, moral psychology, and neuroeconomics that supports our position. We conclude by recommending ideal preference theories of value as a promising approach for developing a machine ethics suitable for navigating an intelligence explosion or “technological singularity.”

“友好的人工智能”by Eliezer Yudkowsky is a shortened version ofYudkowsky (2008)

最后,“人工综合情报和人体心理模型”PDF.)由罗马yampolskiy和(Miri Research As金宝博娱乐sociate)约书亚福克斯评论了人为机器智能的危险:

当建立第一次人工综合智能时,它们可能会改善到遥远的人类水平。关于这种未来实体的猜测已经是受拟人偏见的FF,这导致了与人类思想的错误类比。在本章中,我们应用了一个面向目标的对智力的理解,以表明人类只占据可能的思想的设计空间的一小部分。这个空间远远大于我们熟悉人类的实例;并且未来的高度精神的心理架构和目标无需具有人类思想的大部分性质。一种新的认知科学和哲学的方法,不需要在人类的例子中,需要帮助我们了解我们在大于美国的力量时所面临的挑战。

这本书还包括对大多数章节的简短,关键响应,包括Eliezer Yudkowsky和(以前的Miri Staffer)Michael Anissimov所写的答复。

Altair’s Timeless Decision Theory Paper Published

||Papers

Altair纸前面在他作为Miri研究员的时间,Alex A金宝博娱乐ltair写了一篇论文永恒的决策理论(TDT) that has now been published: “关于Newcomblike问题的决策算法比较。”

Altair的纸张既不同简洁,也更加精确于其TDT的制定比Yudkowsky的早期纸张“永恒的决策理论。” Thus, Altair’s paper should serve as a handy introduction to TDT for philosophers, computer scientists, and mathematicians, while Yudkowsky’s paper remains required reading for anyone interested to develop TDT further, for it covers more ground than Altair’s paper.

Altair的抽象读:

When formulated using Bayesian networks, two standard decision algorithms (Evidential Decision Theory and Causal Decision Theory) can be shown to fail systematically when faced with aspects of the prisoner’s dilemma and so-called “Newcomblike” problems. We describe a new form of decision algorithm, called Timeless Decision Theory, which consistently wins on these problems.

我们可能会在稍后提交日志,但我们已经将当前版本发布到我们的网站,以便读者不需要等待两年(从提交到接受出版物)来阅读它。

为了温和地介绍整个规范决策理论(包括TDT),参见Muehlhauser和Williamson的Decision Theory FAQ

米里’s April newsletter: Relaunch Celebration and a New Math Result

||新闻稿


来自执行董事的问候

亲爱的朋友们,

这些是Miri的令人兴奋的时期。

经过多年的提高认识和能力建设,我们终于将自己转化为一个专注于制作建立值得信赖的数学研究(或“人类友好”)机器智能所需的数学研究。金宝博娱乐随着我们最忠诚的支持者知道,这一直是我们大约十年的目标,这是一个令人生气的过渡。

与哲学研究和技术预测研究相比,看看可以获得数学牵引力的学术牵引力越来越快,也很令人兴奋。金宝博娱乐之内小时of publishing a draft ofour first math result那Field Medalist Timothy Gowers had seen the draft and commented on it (这里),以及其他几个职业数学家。

我们celebrated our “relaunch” at an April 11th party in San Francisco. It was a joy to see old friends and make some new ones. You can see photos and read some details below.

有关我们新的战略优先事项的更多详细信息,请参阅我们的博客文章:Miri 2013年的战略

Cheers,

卢克·穆罕沃斯
Executive Director

Read more »

Miri 2013年的战略

||米里Strategy

这篇文章不是一个详细的战略计划。目前,我只想提供更新what MIRI is doing in 2013 and why

我们的使命保持不变。创造智慧比人类的智慧可能是人类历史上最重要的事件和Miri存在有助于确保此事件具有积极影响。

Still, much has changed in the past year:

  • 我们的短期目标August 2011 strategic planwere largelyaccomplished
  • 我们改变了我们的名字from “The Singularity Institute” to “The Machine Intelligence Research Institute” (MIRI).
  • 我们were once doing three things — research, rationality training, and the Singularity Summit. Now we’re doingonething: research. Rationality training was spun out to a separate organization,CFAR.那and the Summit wasacquired由奇点大学。我们仍然共同生产了奇点大学的奇点峰会,但这需要我们的努力。
  • 2013年1月至3月的数十小时战略规划之后,并输入了20多个外部顾问的投入,we’ve decided to (1) put less effort into public outreach, and to (2) shift our research priorities to Friendly AI math research

这是最后一对更改,我希望在下面更详细地解释。

Read more »

面对智力爆炸ebook

||消息

面对智力爆炸is nowavailable as an ebook

You can get it这里。It is available as a “pay-what-you-want” package that includes the ebook in three formats: MOBI, EPUB, and PDF.

It is also available on Amazon Kindle (US加拿大UK,以及大多数人)和Apple IBookstore(US加拿大UK和大多数人)。

All sources are DRM-free. Grab a copy, share it with your friends, and review it on Amazon or the iBookstore.

All proceeds go directly to funding the technical and strategic金宝博娱乐 机器智能研究所金宝博娱乐

精益的非营利组织

||米里Strategy

Can精益启动methods work for nonprofits?

The Lean Startup作者埃里克里似乎这思想如此:

A startup is a human institution designed to create a new product or service under conditions of extreme uncertainty… Anyone who is creating a new product or business under conditions of extreme uncertainty is an entrepreneur whether he or she knows it or not, and whether working in a government agency, a venture-backed company, a nonprofit, or a decidedly for-profit company with financial investors.

In the past year, I helped launch one new nonprofit (应用理性中心), I massively overhauled one older nonprofit (米里),我与许多非营利组织和董事一起咨询。现在我想分享一些关于“精益非营利组织”的想法的初步思考。

Read more »