Ending collaborations with FHI and Sculpting Evolution

BERI has ended our formal collaborations with Oxford’s Future of Humanity Institute (effective May 5) and MIT’s Sculpting Evolution group (effective July 10).

The Future of Humanity Institute (FHI) was shut down by the University of Oxford on April 16. I’d known that this was a possibility since early 2023, but it wasn’t until March of this year that I heard the official decision. At that point we started to wind down all of our existing commitments to FHI researchers, the last of which ended on May 5. For FHI’s perspective on their shut down, see their memorial page.

FHI had a long history before BERI even started, and they were one of our first three university collaborators. While the last couple of years have been difficult at times, I’m proud of what we were able to accomplish with them, including supporting early work form Robert Trager on AI governance and Owain Evans on Truthful AI, and rapidly spinning up forecasting and evaluation of nonpharmaceutical interventions during the early days of the COVID-19 pandemic.

The Sculpting Evolution group is not shutting down—they just don’t need BERI anymore. Within the first year of our collaboration it was clear that their plans would ultimately lead to the creation of an independent 501(c)(3) (now called SecureBio) which would replace and expand on the type of support BERI specialized in providing. Now that SecureBio has become a mature organization with a full complement of staff, it was the right time to end our formal collaboration and grant them the remaining budget of approximately $25,000.

I’ve been continually impressed and encouraged by SecureBio’s important work, and I’m proud that BERI was able to accelerate the early stages of that work. While this grant ends BERI’s formal relationship with SecureBio and Sculpting Evolution, I personally remain a member of SecureBio’s board, and I look forward to working with them more in the future.

BERI 2023 Annual Report

We’ve released our 2023 annual report here.

This document includes both high-level and program-specific summaries of our activities in 2023 and plans for each program in 2024.

Previous annual reports have included a section containing predictions for the coming year. In order to publish our predictions in a timely manner, this year we opted to publish our predictions and goals for 2024 in a blog post, viewable here.

2023 was a time of change for BERI. We grew from one to two full-time employees, brought on a record 14 new trial collaborations, and doubled our direct spending on our collaborations program. We’re excited to continue this work in 2024 while also finding additional ways to support and impact the existential risk community.

We are extremely grateful to all of our donors for continuing to make our work possible. Thank you!

2024 Collaborator Survey Results

In January 2024 we sent a survey to 70 of our collaborators, attempting to measure their opinions of BERI. The survey was based off the previous collaborator survey sent in 2022. No questions were omitted from this year’s verision, but several new questions were added.

Who responded to the survey?

We sent the survey to 70 individuals, including at least one person from each of our main university collaborations and key individuals at more established trial collaborations. We biased our selection of recipients towards those we’ve helped recently (in 2023), but also included some individuals who had particularly involved relationships with BERI prior to 2023.

Like our last survey in 2022, this survey was anonymous. There were 30 responses, all of which were submitted in January and February of 2024.

For more information on BERI, please refer to our website or email contact@existence.org.

Quantitative Feedback

Additionally, 2024 respondents report that:1

• 44% of projects which received BERI’s help would not have happened otherwise.

  • This is higher than the 35% reported in 2022

• 60% of projects which received BERI’s help would still have happened otherwise, but more slowly or in a way that would have taken more of their time or effort.

  • This is slightly higher than the 54% reported in 2022

• 1% of projects which received BERI’s help were made slower or more difficult by BERI’s involvement.2

  • This is the same as was reported in 2022

Qualitative Feedback

In order to keep comments anonymous, we’ve removed sentences and censored phrases that included identifying information. We’ve also made minor grammatical corrections.

What do you like about BERI?

• BERI is easy to work with and has provided very useful support in a variety of ways.

• Quick and clear answers, open communication. I could trust BERI to do the right things.

• In general, BERI’s quick resourcing amplifies and speeds our impact. Among qualities, my favorite is that BERI staff speak plainly and act on their words. Among resources, my favorite is resourcing minimal procedural delay - enabling in days (or hours) what would cost weeks or months to shepherd through University processes.

• Makes it very easy to buy things compared to [university] purchasing

• Fast and flexible way to do things university bureaucracy is bad at

• Sawyer is friendly, pragmatic, communicates clearly, just overall great.

• So responsive, genuine problem solvers, feels like they genuinely are invested in our group and want us to succeed

• Things are easy.

• They focus on asking what we need in order to carry out our research. Too many times the academic grant process and the way universities focus on research income encourages researchers to ask for as much funding as they think they can get regardless of what is needed, or alternatively we have to dramatically change the direction of our research to fit the focus of the funding scheme, which almost always results in weak research outcomes.

• Very responsive. Very transparent wrt decision-making. Very positive/upbeat.

• Super fast turnaround times. No bureaucratic headache.

• Its mission

• BERI staff extremely approachable and constructive

• Really on top of everything, never have to worry about a dropped thread

What do you dislike about BERI?

• I worry we build [our group] dependent on / accustomed to BERI resource support…which, should BERI lose funding, would disappear. A solution to that might be to designate as “endowment” a portion of BERI’s funding for evergreen or long-wind-down staff.

• Nothing really, as a BERI-supported researcher it’s all upside

• I’m not quite sure what their scope or funding situation is, or what kinds of work they want to help with.

• Wish they had partner offices abroad to further expand global impact (this is probably a relatively unique issue for our group tho)

• Nothing

• Nothing so far.

• It should be in the UK

• Too small - could be bigger!

What is one thing that BERI doesn’t currently do, but that you wish we did?

• I did not know of the existence of BERI until a colleague pointed me to it. Might be useful for you to also advertise your existence a bit more, so people know what options are out there (or I might have just been in a bubble)

• Wild idea: a central “comms” / PR resource that could fractionally help BERI-supported orgs with communications. This would seem to make sense as we get a river of press inquiries, and the positioning of [our lab] as well as other BERI-supported orgs could be somehow clarified and coordinated by a single comms resource. When (e.g.) the BBC wants a view on LLM impacts on LAWS, comms could (1) assure consistent messaging, and (2) suggest other BERI-supported orgs’ possible interviewees.

• One thought that came to mind (didn’t have it before this question) is maybe doing some training on best practices for organization management? Many of us are starting orgs for the first time and are learning on the fly. If there are best practices we can learn to scale up quickly in this area it might make all of our orgs more efficient!

• Am planning a May global meeting on AI safety, perhaps could act as admin hub for that

• I don’t know exactly all the things BERI does. In my case, it’s quite flexible.

• Perhaps keep/share high-level documentation on different collaborations. As someone who was hired into an existing collaboration, it would have been useful to have a doc called something like ‘BERI + ‘my org name’: overview’. This might have included info on what ways BERI had supported us in the past, how much funding BERI had earmarked for the org, and what constraints there are on the use of those funds.

• Fund PhDs

Appendix

Thinking about projects of yours that have received BERI help, what percentage of those projects would probably **not** have happened without BERI's involvement?

Continuing to think about projects of yours that have received BERI help, what fraction of those projects **would** still have happened without BERI's involvement, but more slowly or in a way that would have taken more of your time or effort?

Thinking about projects of yours that involved BERI in some way, what percentage of those projects were made slower or more difficult by BERI's involvement?

Make it stand out

Whatever it is, the way you tell your story online can make all the difference.

Notes

1. See the appendix section for details on these numbers. These simplified results implicitly assume that BERI works on the same number of projects with each of our collaborators, which is not the case. 

2. Specifically, all respondents said 0% except for one respondent, who said 5%. 

BERI Returns All FTX Funds

We are happy to announce that as of January 31, 2024 BERI has returned funds received from the FTX Foundation (also referred to as the FTX Future Fund) in the amount of $389,745.

BERI received the following grants from the FTX Foundation, each for a specific project:

• $100,000 on April 7, 2022,

• $55,000 on May 18, 2022,

• $130,000 on September 22, 2022, and

• $98,000 on October 18, 2022.

The returned fees equaled the $393,000 less legal fees incurred in preparing the settlement.

In discussion with the lawyers involved, FTX was open to a settlement for significantly less than the full amount donated. However, we chose to return the full amount granted to ensure that BERI does not benefit from theft or fraud in any way.

We’re excited to put this situation firmly behind us and to continue working towards our 2024 goal of further diversifying our funding base.

BERI’s Goals and Predictions for 2024

Previous BERI annual reports have concluded with a list of predictions (2021, 2020) about BERI’s activities for the following year. Annual predictions are best made early in the year. In contrast, our 2022 annual report wasn’t released until July of 2023, the year we would have been predicting. To avoid that issue this year, we’ve opted to publish our 2024 predictions as a blog post instead.

We’ve also decided to publish a list of goals for the year. In lieu of publishing an annual strategic plan or theory of change, we are choosing to publish these specific goals to support our commitment to transparency and accountability.

Goals for 2024

We categorized outcomes as goals (vs. predictions) if:

1. Their achievement will likely require an intentional, focused effort;

2. Success or failure will primarily depend on our competence and resources as opposed to any external factors; and

3. Pursuit of the goal is unlikely to have negative side effects for any of BERI’s current collaborations.

Here are our six headline goals for BERI in 2024:

• Launch a new program, expanding our work to activities beyond the currently-dominant Collaborations Program.

  • This would include successfully raising funds with an explicit plan for this new program, but it would also include committing to a significant reallocation of our existing resources, both in terms of staff time as well as financial budget (while still respecting all existing commitments to funders).

• Exercise BERI’s potential “convening power” to increase or broaden our impact.

  • This could include a one-off conference or meeting, or establishing a durable community made up of BERI collaborators, with the goal of connecting people concerned with existential risk and human flourishing.

  • This is an admittedly vague goal, and it may be unclear at the end of 2024 whether or not we’ve accomplished it. We’re just not sure how much convening it makes sense for BERI to do, and so we don’t want to commit to some action that ends up seeming counterproductive when we’re actually doing it. On the other hand, this idea has been such a frequent topic of internal discussion at BERI that it felt wrong to leave it off our list of goals entirely.

• Establish a standardized package of default services that BERI offers to all collaborators as part of onboarding.

  • Example services could include:

    • Uber, but with stated defaults around who gets access, Eats vs Rides, what default restrictions are placed, etc.

    • Amazon, but with stated defaults around who gets access, what can be purchased, default budgets, etc.

  • BERI already offers both of these services (and many others) to many of our collaborations. However, no services are consistently offered to even half of our main collaborations, there’s no cross-collaboration instructions or documentation, and policies and procedures (when they exist) vary between groups.

  • Moving from an entirely ad hoc service model towards a standardized foundation of services allows us to broaden BERI’s impact without overwhelming our small team. Many of our current processes are simply not scalable and will need to be significantly redesigned. With 37 collaborations and likely more to come, it’s critical that we fix this now.

• Establish metrics and targets for each collaboration which we use to measure our performance. While there will always be special exceptions, there should be some set of metrics and goals that we assume to be the default for any given collaboration.

  • As we’ve extended BERI support to more and more academic research groups, it’s become increasingly difficult to balance competing needs and limited staff time between them. Often the result is that groups with bandwidth to reach out more get more help, and groups that are drowning in unnecessary administrative tasks are ignored. These standardized metrics and goals will make it easier for us to manage the expanding BERI community in a way that supports our mission and is fair to each of our collaborators.

  • Some examples of possible metrics:

    • Time between launching a trial collaboration a making our first expense

    • Frequency of check-in meetings with collaboration liaisons.

    • Some distribution of total expenses across all collaborations, e.g. BERI’s spending on our most active program should be no more than five times that of our median program.

• Refresh our website with new branding and a clearer focus on what we want to communicate.

  • BERI’s website has remained essentially unchanged since 2018, and it shows. While we historically haven’t been a particularly public-facing organization, we do want our online presence to clearly communicate what BERI is and what we do. A relatively small amount of effort could likely make significant gains in this area.

• Raise >$100k from a new donor, i.e. one from whom we’ve never received a grant or donation.

  • We are extremely thankful to have strong funding partners in Open Philanthropy and Jaan Tallinn. Including all of their various philanthropic partners (e.g. SFF), these two donors have made up 84% of BERI’s total income since our founding in 2017, a share that has stayed relatively constant over the years.

  • This dependable support has allowed BERI and our collaborators to accomplish far more than we would have been able to if we’d had to rely on expensive and time-consuming public fundraisers. However, it also puts BERI in a precarious position. If either of these two funders were to change their priorities in such a way that funding BERI no longer made sense, our operations would be severely impacted.

  • To ensure that BERI can survive in a changing funding environment, we need to proactively broaden our funding base. Our initial approach will be to find a new foundation looking to fund work on existential risk reduction.

Predictions for 2024

The rest of this post makes predictions about BERI’s 2024, as opposed to the goals laid out above. BERI’s focus is on furthering our mission and supporting our collaborators—if this leads some of the predictions below to be judged as inaccurate, that’s a failure of forecasting and not a failure of our mission.

The purpose of these predictions is to concretely express what we expect to happen for BERI in 2024. While goal-setting is a standard practice which needs no further justification, publishing predictions as an organization is less common. We believe that making public predictions is good practice for BERI because:

• By thinking quantitatively about the coming year, we clarify our beliefs, both in our own minds and in our communication with colleagues.

• By establishing an organizational habit around quantifying our beliefs, we improve our overall communication and decision making capabilities.

• Compared to only making these predictions internally, publishing predictions makes us take the exercise more seriously. Also, if a public prediction proves inaccurate, we’ll feel more obligated to provide an explanation than we would be otherwise.

With all that in mind, here are thirteen predictions for what BERI will be like in 2024:

• [40%] We open applications for new trial collaborators and run a major round of publicity and advertising, at least on the scale of our 2022 effort.

  • Example of what would count: emailing >150 people individually and posting on at least one public forum.

  • Example of what would not count: taking on three new collaborators, each of whom proactively reached out to us.

• [85%] Conditional on BERI deciding to run another round of trial collaborations, BERI will accept 4-8 new trial collaborations.

• For our collaborations program, our total spend on direct support2 will be between:

  • [60%] $3M and $3.7M3

  • [80%] $2.6M and $4.2M

• For our collaborations program, our summed √$ of direct support will be between:

  • [60%] 5,900√$ and 6,300√$

  • [80%] 5,500√$ and 6,700√$

• Our all-in overhead rate4 for all programs will be between:

  • [60%] 10% and 13%

  • [80%] 8% and 15%

• [80%] Four collaborations will have ended (not including MATS).

• [60%] As of Dec 31 we will have at least 2.25 FTE of core employees.

• [30%] As of Dec 31 we will have at least 3 FTE of core employees.

• [75%] BERI will sign on to one or more public statements calling for some form of regulation of AI.

• [80%] On our 2024 collaborator survey (2022 results here), the average overall satisfaction with BERI will be at least 4.5 out of 5.

Notes

1. You can see previous non-collaboration programs that BERI has run on this page. 

2. Direct support does not include operations expenses like core staff time. 

3. These expense numbers, as well as numbers in the other financial predictions, exclude potential grants to MATS Research, Inc. The MATS program is currently a fiscally-sponsored project of BERI’s. MATS is currently in the process of incorporating as an independent not-for-profit. Once this is completed, BERI will transfer the balance of MATS funds to the new entity, MATS Research, Inc. While this transfer will technically be a grant, including it here would only muddle our predictions and artificially deflate our overhead rate. 

4. All-in overhead rate = [Total expenses - direct expenses] / [Total expenses] 

OCPL and SRL Converted to Main Collaborations

BERI has received grants in support of two of our trial collaborations:

1. A $252,000 grant from the Future of Life Institute through the Survival and Flourishing Fund to support our collaboration with the Oxford China Policy Lab (OCPL) at the University of Oxford. We originally launched a trial collaboration with OCPL in 2022.

2. A $52,000 grant from Jaan Tallinn through the Survival and Flourishing Fund in support of our collaboration with the Safe Robotics Laboratory (SRL) at the University of Princeton. We originally launched a trial collaboration with SRL in 2021.

With these donations we’re converting these trial collaborations into main collaborations.

We are truly thankful to have donors like these to support our work and mission. We’re also thankful for groups like OCPL and SRL, with the vision and ambition to work towards a long future of survival and flourishing for human civilization. BERI is proud to support their work.

These are the second and third instances of BERI converting trial collaborations into main collaborations in 2023; we believe these donations provide evidence for the strength of the trial collaborations approach. For more information on trial collaborations, see this post.

New University Collaborators

This year marked the fourth year that BERI has publicly solicited applications for new university collaborations. We had an unexpected doubling in the number of applications we received compared to any previous year, accompanied by an increase in the overall quality of those applications. As usual, applications came from a wide variety of academic disciplines and geographical areas, with the majority being outside of the US.

We ultimately identified 14 collaborations that we think would benefit from BERI’s support and services and who align with our mission. For comparison, we have only accepted an average of 6 new collaborations for the previous years. A generous donation from Jaan Tallinn via the Survival and Flourishing Fund has allowed us to bring on all 14 of the groups we identified as being a good fit for new collaborations.

After careful consideration, we’ve decided to start collaborations with the following groups:

• ARAAC - The Australian Responsible Autonomous Agents Collective at Federation University Australia

• Ben Levinstein’s group at the University of Illinois

• CAML Lab - The Cambridge Applied Machine Learning Lab at the University of Cambridge

• DMIP - The Data Mining, Machine Intelligence and Inductive Programming Group (DMIP) at the Universitat Politècnica de València

• Duke Center on Risk at Duke University

• Lira Lab at the University of Southern California

• Oliver Crook, Research Fellow at the University of Oxford

• Oxford Control and Verification Group at the University of Oxford

• Oxford Martin AI Governance Initiative at the University of Oxford

• R. Daniel Bressler, PhD candidate at Columbia University

• Roger Grosse’s Lab at the University of Toronto

• Samwald Research Group at the University of Vienna

• SpyLab - Secure and Private AI Lab at ETH Zurich

• Torr Vision Group at the University of Oxford

These groups join 23 others as active BERI collaborators. This is the largest new cohort of BERI collaborators yet (by a large margin), and we’re excited to get to work!

We consider these 14 new groups to be “trial” collaborations. They will initially be supported with BERI’s general funds (including the SFF grant mentioned above), as opposed to the collaboration-specific grants that support our main collaborations. If a trial collaboration is successful (i.e. they find it useful and we find it cost-effective), we will likely attempt to raise additional funds in support of specific collaborations. For more information about BERI’s trial collaborations, see this blog post.

We greatly appreciate the time each applicant took to apply. It’s inspiring to see so many people passionate about the long-term survival and flourishing of humankind!

Applications open for new university collaborators

The Berkeley Existential Risk Initiative (BERI) is once again accepting applications from university-affiliated research groups and individual academics interested in receiving our support. Applications are due August 31st.

Selected applicants will be eligible for free support and services from BERI that would be difficult or impossible to obtain through normal university channels. Common examples include purchasing equipment, software, and food, accessing an Uber account for easy travel, and hiring experts for research support; see How does BERI help our collaborators? for more information. If you’re an academic doing research that aligns with BERI’s mission, we encourage you to apply. If you know someone who might be interested, please share this with them!

BERI is a public charity whose mission is to improve human civilization’s long-term prospects for survival and flourishing. We’ve been working with university research groups since 2017, and have provided over $6 million of administrative and operational support on a variety of projects at UC Berkeley, Oxford, Stanford, MIT, and more.

• To read what past BERI collaborators have said about us, please see our recent collaborator survey results.

• For more information on what we do for our collaborators, see this one-pager or our FAQ.

• For more information about what BERI’s been up to, check out our website or our recently-released annual report on 2022.

If you have any questions, please comment below or send an email to contact@existence.org.

CLTC converted to main collaboration

BERI was recently recommended a $522,000 grant via the Survival and Flourishing Fund to support our collaboration with the Center for Long-Term Cybersecurity (CLTC) at UC Berkeley. This grant will support the ongoing work by Dr. Anthony Barrett and the team at CLTC on developing AI standards, in particular creating an AI risk management standards “profile” with supplemental guidance for developers of increasingly general-purpose AI.

We deeply appreciate the support of grantmakers such as the Survival and Flourishing fund for their ongoing support of BERI’s work and mission. We would also like to acknowledge the great work being done by the CLTC team in bringing together academia, industry, and the federal government to collaborate on this profile.

This is the sixth instance of BERI converting a trial collaboration into a main collaboration. Grants such as this from SFF allow BERI to validate our trial collaborations approach. For more information on trial collaborations, see this post.

New BERI Deputy Director Elizabeth Cooper

I'm excited to welcome Elizabeth Cooper as BERI's new Deputy Director!

Elizabeth joined BERI on February 24 to support operations and grow new collaborations. In her previous roles, she worked as a Chief of Staff at various tech start-ups, including Corvid Technologies. She holds a B.A. in Economics and Peace, War, and Defense from the University of North Carolina. In her free time, she is an avid hiker, scuba diver, and home chef.

Elizabeth starts at ~20 hours per week, but we're planning to increase that to full-time later this year.

After three years of running BERI by myself, I started to feel that further expansion would have jeopardized the success of our existing programs. Meanwhile, risks to human civilization from emerging technologies are more salient than ever, and the need for more research on these issues is only increasing. Bringing Elizabeth on board will ensure that BERI is able to continue supporting the expanding field of x-risk research.

In addition to allowing for a greater quantity of university collaborations, Elizabeth will also enable improvements in the quality of BERI's services. This means a more structured and proactive approach, and a smoother and more comprehensive experience for the academic researchers we support.

Finally, I'm looking forward to Elizabeth enabling and taking the lead on entirely new BERI programs, leveraging existing BERI networks and expertise to support the field of x-risk research in whatever ways seem most promising.

New University Collaborators

We’ve completed our evaluation of the applications for new collaborators. We received a total of 19 applications from groups and individuals, across a wide variety of academic disciplines, geographical areas, and project types. After careful consideration, we’ve decided to start collaborations with the following groups:

• Charles Whittaker's group at Imperial College London

• The Intelligent and Interactive Autonomous Systems Lab (ILIAD)

• Joshua Lewis's group at New York University

• Karina Vold's lab at the University of Toronto

• Lionel Levine's lab at Cornell University

• The Oxford Applied and Theoretical Machine Learning Group (OATML)

• The Oxford China Policy Lab (OCPL)

• Sam Bowman's lab at New York University

These groups join 14 others as active BERI collaborators. This is the largest new cohort of BERI collaborators yet, and we're excited to get to work!

We consider these 8 new groups to be “trial” collaborations. They will initially be supported with BERI's general funds (raised primarily through individual donations), as opposed to collaboration-specific grants like with our main collaborations. If a trial collaboration is successful (i.e. they find it useful and we find it cost-effective), we will likely attempt to raise additional funds in support of specific collaborations. For more information about BERI's trial collaborations, see this blog post.

We greatly appreciate the time each applicant took to apply. It's inspiring to see so many people passionate about the long-term survival and flourishing of humankind!

BERI 2022 Annual Report

We’ve released our 2022 annual report here.

This document includes both high-level and program-specific summaries of our activities in 2022 and plans for each program in 2023. Due to how late in the year we began writing this report, we have not included quantitative predictions for 2023. We do, however, look back at our predictions for 2022 and qualitatively discuss potential changes for BERI in 2023 and beyond.

I devoted extra attention to this edition’s “Letter from the Executive Director.” 2022 was a roller coaster of a year, and I wanted to reflect on both the challenges and opportunities that faced BERI and the x-risk community as a whole.

We are extremely grateful to all of our donors for making this work possible. Thank you!

Applications open for new university collaborators

BERI is once again accepting applications from university-affiliated groups and individuals interested in receiving our support.

Winning applicants would be eligible for free services from BERI, like purchasing equipment, food, and software, maintaining an Uber account for easy travel, and hiring experts for research support; see How does BERI help our collaborators? for more info on that. If you’re a member of a research group, or an individual researcher, working on long-termist projects, I encourage you to apply. If you know anyone who might be interested, please share this with them!

BERI is a public charity whose mission is to improve human civilization’s long-term prospects for survival and flourishing. We’ve been working with university groups since 2017, and have provided over $4 million of administrative and operational support on long-termist projects with groups at UC Berkeley, Stanford, Oxford, Cambridge, and elsewhere.

Applications are due June 26th. For more information on what we do for our collaborators, see our FAQ.

If you have any questions, email contact@existence.org.

Funding for our collaboration with David Krueger's lab

BERI has received a $40,000 grant from Open Philanthropy to support our collaboration with Professor David Krueger's lab at the University of Cambridge. We originally launched a trial collaboration with David in July 2021, and with this donation we’re converting our trial collaboration with him into one of our main collaborations.

We are truly thankful to have donors like Open Philanthropy to support our work and mission. We’re also thankful for researchers like David, with the vision and ambition to work towards a long future of survival and flourishing for human civilization. We're proud to support his work and the work of his students—we're especially excited that several of David's PhD students are alumni of CHAI, another BERI collaborator!

These is the fifth instance of BERI converting a trial collaboration into a main collaboration; we believe these donations provide ongoing evidence for the strength of the trial collaborations approach. For more information on trial collaborations, see this post.

BERI 2021 Annual Report

We've released our 2021 annual report here.

This document includes both high-level and program-specific summaries of our activities in 2021, plans for each program in 2022, and some specific, quantitative predictions for the rest of 2022.

New this year, the report also includes a slightly more personal "Letter from the Executive Director" (me), and a two-page analysis of BERI's impact so far.

We are extremely grateful to all of our donors for making this work possible. Thank you!

BERI 2022 collaborator survey results

A summary of the results of our first formal collaborator survey is now available here. One of BERI's main goals is to support university x-risk researchers in whatever way they need, so it's useful for us to check whether those researchers actually think BERI is helping them. Surveys like this helps BERI to monitor and improve our impact.

Our intention moving forward is to carry out and publish a survey similar to this one annually.

Thank you so much to everyone who responded to the survey!

What does BERI do and why is it important?

I recently gave a presentation about BERI to the Global Priorities Institute at Oxford University. My preparations for the talk helped crystallize a few thoughts about BERI that I’ve had in rough form for a while. So I thought I’d take the more novel parts of the presentation and turn them into a blog post.

This post will discuss (1) what BERI does in general, and (2) why I think it’s important.

What does BERI do?

Like it says on our home page, BERI collaborates with university research groups working to reduce existential risk by providing them with free services and support. Sometimes a university’s existing administrative structures are unable to carry out a particular request, for some idiosyncratic or unanticipated reason. In these situations BERI can step in to fill this gap.

In principle, we can do anything that a charity or university can do. In practice, most of our activities can be categorized as either hiring or purchasing.

Hiring can be broken down further into short-term hiring and long-term hiring. Short-term hiring often means hiring research assistants for specific, bounded projects, but can also span a wide variety of skill-sets, including copy editors, web designers, translators, and productivity coaches. This is useful because university hiring processes are often slow and cumbersome, optimized more for hiring tenured professors than for 10-hour freelancers. Even paying a service provider as a vendor (instead of as an employee) presents challenges to an organization as large and complex as a university. For many excellent reasons, built up over decades of experience, many universities have elaborate approval processes which simply aren’t worth navigating for a short project. BERI avoids these complexities by being much smaller and focused on a very narrow body of work—existential risk reduction.

BERI also does long-term hiring. For example, we recently hired a machine learning engineer to support Professor Phil Thomas of the Autonomous Learning Lab at UMass Amherst. Many universities have trouble hiring skilled software engineers, because typical industry salaries are much higher than universities expect to pay to someone in a non-tenure track position. BERI does not have this restriction, which allows us to pay significantly higher salaries to non-academics than most universities are able to pay.

Purchasing is more varied, and I don’t have a one-size-fits-all explanation of why purchasing is a useful category of BERI services. Perhaps your department will only supply laptops from a preset list, but your work would benefit from a different laptop. Or maybe your research group would benefit from some piece of software which requires a monthly subscription (e.g. Slack or Asana), which would require monthly reimbursements for small amounts of money through a burdensome submission system. BERI can easily purchase the laptop, or apply our credit card to monthly software payments.

Another way to understand BERI is to imagine a spectrum of administrative tasks. On one end are the things your university or department is already able to do for you, without too much trouble. On the other end are things your university or department doesn’t want you to do, at all. BERI support exists somewhere in the middle of this spectrum. We try to help with things that are permitted and encouraged by your university and department, but which are either difficult or impossible to do within existing university structures.

Why do I think BERI’s work is important?

The “best of both worlds” argument

I think that universities can be good homes for high-impact work. Perhaps there’s something about the university setting that enables high-impact work. After all, universities have lots of resources—financial, administrative, and reputational—which can be directed towards important altruistic endeavors like reducing x-risk. But even if you’re skeptical of that, there are still going to be people who have found themselves in academia for reasons independent of impact, and are only now trying to optimize their careers to improve human civilization’s long-term prospects for survival and flourishing. Either way, I’m confident that there are university researchers doing important work on existential risk.

I also think universities present obstacles to impact. The causes for this are as varied as the institutions themselves, but obstacles do exist. If researchers trying to leverage university resources to reduce x-risk are being frustrated by the obstacles they find, I don’t want them to leave academia for industry just to get better operations support for their work. Instead, I want BERI to give them the best of both worlds, by clearing obstacles so that the benefits of universities can be accessed with less difficulty. In this way, BERI can have an amplifying effect on important work reducing x-risk.

The “comparative advantage” argument

Researchers often spend time on operational and administrative tasks which (1) they don’t want to do, and (2) aren’t their specialty. This slows progress on research. Meanwhile, there are ops people (like those at BERI) who (1) enjoy operational and administrative tasks, and (2) don’t have a background in research. Each of these groups should spend more time on what they’re good at and want to be doing, and less time on what they’re not good at and don’t want to be doing. By shifting work time towards each contributor’s comparative advantage, BERI opens up more researcher time, and speeds up research agendas that try to reduce x-risk.

Summary

BERI collaborates with university research groups working to reduce existential risk by providing them with free services and support. In principle, we can do anything that a charity or university can do. In practice, most of our activities can be categorized as either hiring or purchasing. There are multiple ways to think about BERI’s importance and impact. I highlighted two of them in this post, which I called the “best of both worlds” argument and the “comparative advantage” argument.

InterACT converted to main collaboration

BERI has received a $135,000 grant from the Long-Term Future Fund to support our collaboration with the Interactive Autonomy and Collaborative Technologies Laboratory (InterACT) at UC Berkeley. We originally launched a trial collaboration with InterACT in August 2020, and with this donation we’re converting our trial collaboration with InterACT into one of our main collaborators.

We are truly thankful to have donors like the Long-Term Future Fund to support our work and mission. We’re also thankful for groups like InterACT, with the vision and ambition to work towards a long future of survival and flourishing for human civilization. We're proud to support their work.

These is the fourth instance of BERI converting a trial collaboration into a main collaboration; we believe these donations provide ongoing evidence for the strength of the trial collaborations approach. For more information on trial collaborations, see this post.

Listing Our Donors Publicly

Starting today, most of BERI's donors will be listed on our transparency page.

By default, BERI does not publish donor information. Each individual on the list has given their explicit permission for BERI to publish this information.

As a public charity, we feel an obligation to be as transparent as possible. The publication and ongoing maintenance of this list is an important step towards the fullfilment of this obligation. In addition, we're proud to have the support of so many donors — showing our appreciation in this way is the least we can do.

As of writing, ~93% (by dollar) of our donors are named on this list. For donors who have not given permission, reasons vary: Many were not contacted or did not respond. Some donors simply prefer privacy. BERI respects the right of each of our donors to make their own decisions in this regard.

Sculpting Evolution and ALL Converted to Main Collaborations

BERI has received grants in support of two of our trial collaborations:

1. A $250,000 grant from the Long-Term Future Fund to support our collaboration with the Autonomous Learning Laboratory (ALL) at the University of Massachusetts Amherst. We originally launched a trial collaboration with ALL in August 2020.

2. A $30,000 grant from Open Philanthropy (via a recommendation to their DAF at SVCF) to support our collaboration with the Sculpting Evolution group at MIT. We originally launched a trial collaboration with Sculpting Evolution in July 2021.

With these donations we’re converting these collaborations trials into main collaborators.

We are truly thankful to have donors like these to support our work and mission. We’re also thankful for groups like ALL and Sculpting Evolution, with the vision and ambition to work towards a long future of survival and flourishing for human civilization. BERI is proud to support their work.

These is the second and third instances of BERI converting trial collaborations into main collaborations; we believe these donations provide evidence for the strength of the trial collaborations approach. For more information on trial collaborations, see this post.