New Paper: “Predicting AGI: What can we say when we know so little?”

||Papers

Predicting AGIMIRI research associate Benja Fallenstein and UC Berkeley student Alex Mennen have released a new working paper titled “预测AGI：当我们知道这么少时，我们能说什么？”

From the introduction:

This analysis does not attempt to predict when AGI will actually be achieved, but instead, to predict when this epistemic state with respect to AGI will change, such that we will have a clear idea of how much further progress is needed before we reach AGI. Metaphorically speaking, instead of predicting when AI takes off, we predict when it will start taxiing to the runway.

The paper argues for a Pareto distribution for “time to taxi,” and concludes:

in general, a Pareto distribution suggests that we should put a much greater emphasis on short-term strategies than a less skewed distribution (e.g. a normal distribution) with the same median would.

New Paper: “Racing to the Precipice”

||Papers

Racing to the edge卡尔·舒尔曼（Carl Shulman）担任美里研究金宝博娱乐员（Miri Research Chemel）期间，他为现在作为FHI技术报告提供的论文做出了贡献：Racing to the Precipice: a Model of Artificial Intelligence Development.

Abstract:

本文介绍了AI军备竞赛的简单模型，其中几个开发团队竞争建立第一个AI。假设第一个AI将非常有力和变革性，每个团队都会激励首先完成 - 如果需要，请浏览安全预防措施。本文介绍了此过程的NASH均衡，每个团队都采取了正确的安全预防措施。在团队之间拥有额外的开发团队和额外的敌意可以增加AI-Disaster的危险，尤其是在冒险面前比开发AI的技能更重要的情况下。令人惊讶的是，信息也增加了风险：对彼此的能力（以及他们自己的）的了解越多，危险就越多。

Update:截至2015年7月，本文已发表在《杂志》上AI & Society.

MIRI’s November 2013 Newsletter

||Newsletters

Machine Intelligence Research Institute

亲爱的朋友们，
We’re experimenting with a new, ultra-brief newsletter style. To let us know what you think of it, simply reply to this email. Thanks!

News Updates

  • You can nowsupport MIRI for freeby shopping at smile.amazon.com instead of amazon.com. Update your bookmarks!
  • Louie Helm will be onsite for the Nov. 24th Marin county screening of Doug Wolens’ new documentaryThe Singularity. Details on this and other screenings arehere.
Research Updates
Other Updates
  • Many of our friends have said Louie Helm’sRockstar Researchis among their favorite new sources of news; check it out!
  • Know an exceptionally bright, ambitious person younger than 20? Tell them toapply for The Thiel Fellowship! $100,000 to skip college and develop one’s skills and ideas, with an incredible network of mentors in the Bay Area.
  • CFAR hasupcoming rationality workshopsin February (Melbourne), March (Bay Area), and April (NYC). Tell your friends!
Cheers,
Luke Muehlhauser
Executive Director


Support MIRI by Shopping at AmazonSmile

||News

通过Amazonsmile支持Miri。If you shop at the newAmazonSmile, Amazon donates 0.5% of the price of your eligible purchases to a charitable organization of your choosing.

MIRI is an eligible charitable organization, so the next time you consider purchasing something through Amazon, support MIRI by shopping atAmazonSmile!

If you get to Amazon.com via an affiliate link, remember to change “amazon.com” to “smile.amazon.com” in the address bar before making your purchase. Or, even easier, use theSmileAlwaysChrome extension.

格雷格·莫里塞特（Greg Morrisett）on Secure and Reliable Systems

||Conversations

格雷格·莫里塞特（Greg Morrisett）肖像格雷格·莫里塞特（Greg Morrisett）is the Allen B. Cutting Professor of Computer Science at Harvard University. He received his B.S. in Mathematics and Computer Science from the University of Richmond in 1989, and his Ph.D. from Carnegie Mellon in 1995. In 1996, he took a position at Cornell University, and in the 2003-04 academic year, he took a sabbatical and visited the Microsoft European Research Laboratory. In 2004, he moved to Harvard, where he has served as Associate Dean for Computer Science and Engineering, and where he currently heads the Harvard Center for Research on Computation and Society.

Morrisett has received a number of awards for his research on programming languages, type systems, and software security, including a Presidential Early Career Award for Scientists and Engineers, an IBM Faculty Fellowship, an NSF Career Award, and an Alfred P. Sloan Fellowship.

他曾担任《功能编程杂志》的首席编辑，并担任编程语言和系统和信息处理信件的ACM交易的副编辑。金宝博官方他目前担任《 ACM杂志》的编辑委员会，并担任ACM通信的研究重点列的总合编辑。金宝博娱乐此外，莫里塞特（Morrisett）曾在DARPA信息科学技术研究（ISAT）小组，NSF计算机和信息科学与工程（CISE）咨询委员会，Microsoft Research的技术咨询委员会以及Microsoft的Trusthworthy Computing Accuperication咨询委员会中任职。金宝博娱乐

Luke Muehlhauser： 中的一个interesting projects您参与的是SAFE, a DARPA-funded project “focused on a clean slate design for resilient and secure systems.” What is the motivation for this project, and in particular for its “clean slate” approach?

Read more »

From Philosophy to Math to Engineering

||Analysis

For centuries, philosophers wondered how we could learn what causes what. Some argued it was impossible, or possible only via experiment. Others kepthacking awayat the problem,clarifying ideaslikecounterfactualandprobabilityandcorrelationby making them more precise and coherent.

Then, in the 1990s, a breakthrough: Judea Pearl and othersshowedthat, in principle, we can sometimes infer causal relations from data even without experiment, via the mathematical machinery of probabilistic graphical models.

Next, engineers used this mathematical insight to writesoftwarethat can, in seconds, infer causal relations from a data set of observations.

Across the centuries, researchers had toiled away, pushing our understanding of causality from philosophy to math to engineering.

From Philosophy to Math to Engineering (small)

And so it is withFriendly AIresearch. Current progress on each sub-problem of Friendly AI lies somewhere on a spectrum from philosophy to math to engineering.

Read more »

Robin Hanson on Serious Futurism

||Conversations

Robin Hansonis an associate professor of economics at George Mason University and a research associate at the Future of Humanity Institute of Oxford University. He is known as a founder of the field of prediction markets, and was a chief architect of the Foresight Exchange, DARPA’s FutureMAP, IARPA’s DAGGRE, SciCast, and is chief scientist at Consensus Point. He started the first internal corporate prediction market at Xanadu in 1990, and invented the widely used market scoring rules. He also studies signaling and information aggregation, and is writing a book on the social implications of brain emulations. He blogs atOvercoming Bias.

汉森获得了学士学位1981年加州大学尔湾分校的物理学博士学位1984年芝加哥大学的物理学和硕士学位硕士学位和博士学位。在1997年获得加州理工学院的社会科学。在获得博士学位之前，他研究了洛克希德（Lockheed），美国宇航局（NASA）和其他地方的人金宝博娱乐工智能，贝叶斯统计和超文本出版。

Luke Muehlhauser: In an earlier blog post, Iwroteabout the need for what I called AGI impact experts who “develop skills related to predicting technological development, predicting AGI’s likely impact on society, and identifying which interventions are most likely to increase humanity’s chances of safely navigating the creation of AGI.”

In 2009, you gave a talk called, “How does society identify experts and when does it work?“考虑到你所做的研究和expertise, what do you think of humanity’s prospects for developing these AGI impact experts? If they are developed, do you think society will be able to recognize who is an expert and who is not?

Read more »

New Paper: “Embryo Selection for Cognitive Enhancement”

||Papers

IES first pageDuring his time as a MIRI research fellow, Carl Shulman co-authored (withNick Bostrom) a paper that is now available as a preprint, titled “Embryo Selection for Cognitive Enhancement: Curiosity or Game-changer?”

Abstract:

Human capital is a key determinant of personal and national outcomes, and a major input to scientific progress. It has been suggested that advances in genomics will make it possible to enhance human intellectual abilities. One way to do this would be via embryo selection in the context ofin vitrofertilization (IVF). In this article, we analyze the feasibility, timescale, and possible societal impacts of embryo selection for cognitive enhancement. We find that embryo selection, on its own, may have significant impacts, but likely not drastic ones, over the next 50 years, though large effects could accumulate over multiple generations. However, there is a complementary technology, stem cell-derived gametes, which has been making rapid progress and which could amplify the impact of embryo selection, enabling very large changes if successfully applied to humans.

The last sentence refers to “iterated embryo selection” (IES), a future technology first described by MIRIin 2009. This technology has significant strategic relevance for Friendly AI (FAI) development because it might be the only intelligence amplification (IA) technology (besidesWBE人类聪明)来产生足够大的影响ce to substantially shift our odds of getting FAI before arbitrary AGI, if AGI isdeveloped sometime this century.

Unfortunately, it remains unclear whether the arrival of IES would shift our FAI chances positively or negatively. On the one hand, a substantially smarter humanity may be wiser, and more likely to get FAI right. On the other hand, IES might accelerate AGI relative to FAI, since AGI is more parallelizable than FAI. (For more detail, and more arguments pointing in both directions, seeIntelligence Amplification and Friendly AI.)