As advancing AI capabilities gained widespread attention in late 2022 and 2023, interest in governing and regulating these systems has grown. Discussion of the potential catastrophic risks of misaligned or uncontrollable AI has become more prominent, potentially opening up opportunities for policy that could mitigate the threats.
There’s still a lot of uncertainty about which AI governance strategies would be best. Many have proposed policies and strategies aimed at reducing the largest risks, which we discuss below.
But there’s no roadmap here. There’s plenty of room for debate about what’s needed, and we may not have found the best ideas yet in this space. In any case, there’s still a lot of work to figure out how promising policies and strategies would work in practice. We hope to see more people enter this field to develop expertise and skills that will contribute to risk-reducing AI governance and policy.
In a nutshell: Advanced AI systems could have massive impacts on humanity and potentially pose global catastrophic risks. There are opportunities in the broad field of AI governance to positively shape how society responds to and prepares for the challenges posed by the technology.
Given the high stakes, pursuing this career path could be many people’s highest-impact option. But they should be very careful not to accidentally exacerbate the threats rather than mitigate them.
If you are well suited to this career, it may be the best way for you to have a social impact.
Based on an in-depth investigation
Table of Contents
“What you’re doing has enormous potential and enormous danger.” — US President Joe Biden, to the leaders of the top AI companies
Artificial intelligence has advanced rapidly. In 2022 and 2023, new language and image generation models gained widespread attention for their abilities, blowing past previous benchmarks.
And the applications of these models are still new; with more tweaking and integration into society, the existing AI systems may become easier to use and more ubiquitous.
We don’t know where all these developments will lead us. There’s reason to be optimistic that AI will eventually help us solve many of the world’s problems, raising living standards and helping us build a more flourishing society.
But there are also substantial risks. Advanced AI could be used to do a lot of harm. And we worry it could accidentally lead to a major catastrophe — and perhaps even cause human disempowerment or extinction. We discuss the arguments that these risks exist in our in-depth problem profile.
But we need a range of strategies for risk reduction. Public policy and corporate governance in particular may be necessary to ensure that advanced AI is broadly beneficial and low risk.
Governance generally refers to the processes, structures, and systems that carry out decision making for organisations and societies at a high level. In the case of AI, we expect the governance structures that matter most to be national governments and organisations developing AI — as well as some international organisations and perhaps subnational governments.
Some aims of AI governance work could include:
We need a community of experts who understand modern AI systems and policy, as well as the severe threats and potential solutions. This field is still young, and many of the paths within it aren’t clear and are not sure to pan out. But there are relevant professional paths that will provide you valuable career capital for a variety of positions and types of roles.
The rest of this article explains what work in this area might involve, how you can develop career capital and test your fit, and some promising places to work.
There are a variety of ways to pursue AI governance strategies, and as the field becomes more mature, the paths are likely to become clearer and more established.
We generally don’t think people early in their careers should aim for a specific high-impact job. They should instead aim to develop skills, experience, knowledge, judgement, networks, and credentials — what we call career capital — that they can use later to have an impact.
This may involve following a standard career trajectory or moving around in different kinds of roles. Sometimes, you just have to apply to many different roles and test your fit for various types of work before you know what you’ll be good at. Most importantly, you should try to get excellent at something for which you have strong personal fit and that will let you contribute to solving pressing problems.
In the AI governance space, we see at least six broad categories of work that we think are important:
Thinking about the different kinds of career capital that are useful for the categories of work that appeal to you may suggest some next steps in your path. (We discuss how to assess your fit and enter this field below.)
You may want to move between these different categories of work at different points in your career. You can also test out your fit for various roles by taking internships, fellowships, entry-level jobs, temporary placements, or even doing independent research, all of which can serve as career capital for a range of paths.
We have also reviewed career paths in AI technical safety research and engineering, information security, and AI hardware expertise, which may be crucial to reducing risks from AI. These fields may also play a significant role in an effective governance agenda. People serious about pursuing a career in AI governance should familiarise themselves with these subjects as well.
Taking a role within an influential government could help you play an important role in the development, enactment, and enforcement of AI policy.
We generally expect that the US federal government will be the most significant player in AI governance for the foreseeable future. This is because of its global influence and its jurisdiction over much of the AI industry, including the most prominent AI companies such as Anthropic, OpenAI, and Google DeepMind. It also has jurisdiction over key parts of the AI chip supply chain. Much of this article focuses on US policy and government. 1
But other governments and international institutions matter too. For example, the UK government, the European Union, China, and others may present opportunities for impactful AI governance work. Some US state-level governments, such as California, may have opportunities for impact and gaining career capital.
What would this work involve? Sections below discuss how to enter US policy work and which areas of the government that you might aim for.
In 2023, the US and UK governments both announced new institutes for AI safety — both of which should provide valuable opportunities for career capital and potential impact.
But at the broadest level, people interested in positively shaping AI policy should gain skills and experience to work in areas of government with some connection to AI or emerging technology policy.
This can include roles in: legislative branches, domestic regulation, national security, diplomacy, appropriations and budgeting, and other policy areas.
If you can get a role already working directly on this issue, such as in one of the AI safety institutes or working for a lawmaker focused on AI, that could be a great opportunity.
Otherwise, you should seek to learn as much as you can about how policy works and which government roles might allow you to have the most impact. Try to establish yourself as someone who’s knowledgeable about the AI policy landscape. Having almost any significant government role that touches on some aspect of AI, or having some impressive AI-related credential, may be enough to go quite far.
One way to advance your career in government on a specific topic is what some call “getting visibility.” This involves using your position to learn about the landscape and connect with the actors and institutions in the policy area. You’ll want to engage socially with others in the policy field, get invited to meetings with other officials and agencies, and be asked for input on decisions. If you can establish yourself as a well-regarded expert on an important but neglected aspect of the issue, you’ll have a better shot at being included in key discussions and events.
Career trajectories within government can be broken down roughly as follows:
Read more about how to evaluate your fit and get started building relevant career capital in our article on policy and political skills.
There’s still a lot of research to be done on AI governance strategy and implementation. The world needs more concrete policies that would really start to tackle the biggest threats; developing such policies and deepening our understanding of the strategic needs of the AI governance space are high priorities.
Other relevant research could involve surveys of public and expert opinion, legal research about the feasibility of proposed policies, technical research on issues like compute governance, and even higher-level theoretical research into questions about the societal implications of advanced AI.
Some research, such as that done by Epoch AI, focuses on forecasting the future course of AI developments, which can influence AI governance decisions.
However, several experts we’ve talked to warn that a lot of research on AI governance may prove to be useless. So it’s important to be reflective and seek input from others in the field about what kind of contribution you can make. We list several research organisations below that we think pursue promising research on this topic and could provide useful mentorship.
One approach for testing your fit for this work — especially when starting out — is to write up analyses and responses to existing work on AI policy or investigate some questions in this area that haven’t received much attention. You can then share your work widely, send it out for feedback from people in the field, and evaluate how you enjoy the work and how you might contribute to this field.
But don’t spend too long testing your fit without making much progress, and note that some are best able to contribute when they’re working on a team. So don’t over-invest in independent work, especially if there are few signs it’s working out especially well for you. This kind of project can make sense for maybe a month or a bit longer — but it’s unlikely to be a good idea to spend much more than that without funding or some really encouraging feedback from people working in the field.
If you have the experience to be hired as a researcher, work on AI governance can be done in academia, nonprofit organisations, and think tanks. Some government agencies and committees, too, perform valuable research.
Note that universities and academia have their own priorities and incentives that often aren’t aligned with producing the most impactful work. If you’re already an established researcher with tenure, it may be highly valuable to pivot into work on AI governance — your position may even give you a credible platform from which to advocate for important ideas.
But if you’re just starting out a research career and want to focus on this issue, you should carefully consider whether your work will be best supported inside academia. For example, if you know of a specific programme with particular mentors who will help you pursue answers to critical questions in this field, it might be worth doing. We’re less inclined to encourage people on this path to pursue generic academic-track roles without a clear idea of how they can do important research on AI governance.
Advanced degrees in policy or relevant technical fields may well be valuable, though — see more discussion of this in the section on how to assess your fit and get started.
You can also learn more in our article about how to become a researcher.
Internal policy and corporate governance at the largest AI companies themselves is also important for reducing risks from AI.
At the highest level, deciding who sits on corporate boards, what kind of influence those boards have, and the incentives the organisation faces can have a major impact on a company’s choices. Many of these roles are filled by people with extensive management and organisational leadership experience, such as founding and running companies.
If you’re able to join a policy team at a major company, you can model threats and help develop, implement, and evaluate proposals to reduce risks. And you can build consensus around best practices, such as strong information security, using outside evaluators to find vulnerabilities and dangerous behaviours in AI systems (red teaming), and testing out the latest techniques from the field of AI safety.
And if, as we expect, AI companies face increasing government oversight, ensuring compliance with relevant laws and regulations will be a high priority. Communicating with government actors and facilitating coordination from inside the companies could be impactful work.
In general, it seems better for AI companies to be highly cooperative with each other 2 and with outside groups seeking to minimise risks. And this doesn’t seem to be an outlandish hope — many industry leaders have expressed concern about catastrophic risks and have even called for regulation of the frontier technology they’re creating.
That said, cooperation will likely take a lot of effort. Companies creating powerful AI systems may resist some risk-reducing policies, because they’ll have strong incentives to commercialise their products. So getting buy-in from the key players, increasing trust and information-sharing, and building a consensus around high-level safety strategies will be valuable.
People outside of government or AI companies can influence the shape of public policy and corporate governance with advocacy and lobbying.
Advocacy is the general term for efforts to promote certain ideas and shape the public discourse, often around policy-related topics. Lobbying is a more targeted effort aimed at influencing legislation and policy, often by engaging with lawmakers and other officials.
If you believe AI companies may be disposed to advocate for generally beneficial regulation, you might work with them to push the government to adopt specific policies. It’s plausible that AI companies have the best understanding of the technology, as well as the risks, failure modes, and safest paths — and so are best positioned to inform policymakers.
On the other hand, AI companies might have too much of a vested interest in the shape of regulations to reliably advocate for broadly beneficial policies. If that’s right, it may be better to join or create advocacy organisations unconnected from the industry — perhaps supported by donations — that can take stances opposed to commercial interests.
For example, some believe it might be best to deliberately slow down or halt the development of increasingly powerful AI models. Advocates could make this demand of the companies themselves or of the government. But pushing for this step may be difficult for those involved with the companies creating advanced AI systems.
It’s also possible that the best outcomes will result from a balance of perspectives from inside and outside industry.
Advocacy can also:
However, note that advocacy can sometimes backfire because predicting how information will be received isn’t straightforward. Be aware that:
It’s important to keep these risks in mind and consult with others (particularly those who you respect but might disagree with tactically). And you should educate yourself deeply about the topic before explaining it to the public.
You can read more in the section about doing harm below. We also recommend reading our article on ways people trying to do good accidentally make things worse and how to avoid them. And you may find it useful to read our article on the skills needed for communicating important ideas.
Case study: the Center for AI Safety statement
In May 2023, the Center for AI Safety released a single-sentence statement saying: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
Most notably, the statement was supported by more than 100 signatories, including leaders of major AI companies, including OpenAI, Google Deepmind, and Anthropic, as well as top researchers in the field, Geoffrey Hinton and Yoshua Bengio. It also includes a member of the US Congress, other public officials, economists, philosophers, business leaders, and more.
This statement drew media attention at the time, and UK Prime Minister Rishi Sunak and White House press secretary both reacted to the statement with expressions of concern. Both the UK government and the US government have subsequently moved forward with efforts to start to address these risks.
The statement has also helped clarify and inform the discourse about AI risk, as evidence that being concerned about catastrophes on the scale of human extinction is not a fringe position.
If regulatory measures are put in place to reduce the risks of advanced AI, some agencies and outside organisations will need to audit companies and systems to make sure that regulations are being followed.
Governments often rely on third-party auditors when regulating because the government lacks much of the expertise that the private sector has. There aren’t many such opportunities available in this type of role for AI-related auditing that we know of, but such roles play a critical part of an effective AI governance framework.
AI companies and the AI systems they create may be subject to audits and evaluations out of safety concerns.
One nonprofit, Model Evaluation and Threat Research (METR, formally known as ARC Evals), has been at the forefront of work to evaluate the capabilities of advanced AI models. 3 In early 2023, the organisation partnered with two leading AI companies, OpenAI and Anthropic, to evaluate the capabilities of the latest versions of their chatbot models prior to their release. They sought to determine if the models had any potentially dangerous capabilities in a controlled environment.
The companies voluntarily cooperated with METR for this project, but at some point in the future, these evaluations may be legally required.
Other types of auditing and evaluation may be required as well. METR has said it intends to develop methods to determine which models are appropriately aligned — that is, that they will behave as their users intend them to behave — prior to release.
Governments may also want to employ auditors to evaluate the amount of compute that AI developers have access to, their information security practices, the uses of models, the data used to train models, and more.
Acquiring the technical skills and knowledge to perform these types of evaluations, and joining organisations that will be tasked to perform them, could be the foundation of a highly impactful career. This kind of work will also likely have to be facilitated by people who can manage complex relationships across industry and government. Someone with experience in both sectors could have a lot to contribute.
Some of these types of roles may have some overlap with work in AI technical safety research.
For someone with the right fit, working to improve coordination with China on the safe development of AI could be a particularly impactful career path.
The Chinese government has been a major funder in the field of AI, and the country has giant tech companies that could potentially drive forward advances.
Given tensions between the US and China, and the risks posed by advanced AI, there’s a lot to be gained from increasing trust, understanding, and coordination between the two countries. The world will likely be much better off if we can avoid a major conflict between great powers and if the most significant players in emerging technology can avoid exacerbating any global risks.
We have a separate career review that goes into more depth on China-related AI safety and governance paths.
As we’ve said, we focus most on US policy and government roles. This is largely because we anticipate that the US is now and will likely continue to be the most pivotal actor when it comes to regulating AI, with a major caveat being China, as discussed in the previous section.
But many people interested in working on this issue can’t or don’t want to work in US policy — perhaps because they live in another country and don’t intend on moving.
Much of the advice above still applies to these people, because roles in AI governance research and advocacy can be done outside of the US. 4 And while we don’t think it’s generally as impactful in expectation as US government work, opportunities in other governments and international organisations can be complementary to the work to be done in the US.
The United Kingdom, for instance, may present another strong opportunity for AI policy work that would complement US work. Top UK officials have expressed interest in developing policy around AI, a new international agency, and reducing extreme risks. And the UK government announced the creation of a new AI Foundation Model Taskforce in 2023 to drive forward safety research.
The European Union has shown that its data protection standards — the General Data Protection Regulation (GDPR) — affect corporate behaviour well beyond its geographical boundaries. EU officials have also pushed forward on regulating AI, and some research has explored the hypothesis that the impact of the EU’s AI regulations will extend far beyond the continent — the so-called “Brussels effect.”
And any relatively wealthy country could fund some AI safety research, though much of it requires access to top talent and state of the art tech. Any significant advances in AI safety research could inform researchers working on the most powerful models.
Other countries might also develop liability standards for the creators of AI systems that could incentivise corporations to proceed cautiously before releasing models.
And at some point, there may be AI treaties and international regulations, just as the international community has created the International Atomic Energy Agency, the Biological Weapons Convention, and Intergovernmental Panel on Climate Change to coordinate around and mitigate other global catastrophic threats.
Efforts to coordinate governments around the world to understand and share information about threats posed by AI may end up being extremely important in some future scenarios. The Organisation for Economic Cooperation and Development, for instance, has already created the AI Policy Observatory.
Third-party countries may also be able to facilitate cooperation and reduce tensions betweens the United States and China, whether around AI or other potential flashpoints.
People working in AI policy have proposed a range of approaches to reducing risk as AI systems get more powerful.
We don’t necessarily endorse all the ideas below, but what follows is a list of some prominent policy approaches that could be aimed at reducing the largest dangers from AI: 5
The details, benefits, and downsides of many of these ideas have yet to be fully worked out, so it’s crucial that we do more research and get more input from informed stakeholders. And this list isn’t comprehensive — there are likely other important policy interventions and governance strategies worth pursuing.
You can also check out a list of potential policy ideas from Luke Muehlhauser of Open Philanthropy, 8 an article about AI policy proposals from Vox‘s Dylan Matthews, and a survey of expert opinion on best practices in AI safety and governance.
Helen studied engineering before getting a research job at GiveWell and then Open Philanthropy. From there, she explored a couple of different cause areas, and eventually moved to Beijing to learn about the intersection of China and AI. When the Center for Security and Emerging Technology (CSET) was founded, she was recruited to help build the organisation. CSET has since become a leading think tank in Washington, DC on the intersection of emerging technology and national security.
Learn more
Ben graduated from Yale in 2016, where he majored in physics, math, and philosophy. After graduating, Ben became a researcher at the Centre for Effective Altruism and then moved to the Centre for the Governance of AI, where he became the director You can check here to see if GovAI is hiring.
Learn more
If you’re early on in your career, you should focus first on getting skills and other career capital to successfully contribute to the beneficial governance and regulation of AI.
You can gain career capital for roles in many ways. Broadly speaking, working in or studying fields such as politics, law, international relations, communications, and economics can all be beneficial for going into policy work.
And expertise in AI itself, gained by studying and working in machine learning and technical AI safety, or potentially related fields such as computer hardware and information security, should also give you a big advantage.
Try to find relatively “cheap” tests to assess your fit for different paths. This could mean, for example, taking a policy internship, applying for a fellowship, doing a short bout of independent research, or taking classes or courses on technical machine learning or computer engineering.
It can also involve talking to people doing a job and finding out what the day-to-day experience of the work is and what skills are needed.
All of these factors can be difficult to predict in advance. While we grouped “government work” into a single category above, that label covers a wide range of roles. Finding the right fit can take years, and it can depend on factors out of your control, such as the colleagues you work closely with. That’s one reason it’s useful to build broadly valuable career capital that gives you more options.
Don’t underestimate the value of applying to many relevant openings in the field and sector you’re aiming for to see what happens. You’ll likely face a lot of rejection with this strategy, but you’ll be able to better assess your fit for roles after you see how far you get in the process. This can give you more information than guessing about whether you have the right experience.
Try to rule out certain types of work if you gather evidence that you’re not a strong fit. For example, if you invest a lot of effort trying to get into reputable universities or nonprofit institutions to do AI governance research, but you get no promising offers and receive little encouragement, this might be a significant signal that you’re unlikely to thrive in that path.
That wouldn’t mean you have nothing to contribute, but your comparative advantage may lie elsewhere.
A mix of people with technical and policy expertise — and some people with both — is needed in AI governance.
While anyone involved in this field should work to maintain an understanding of both the technical and policy details, you’ll probably start out focusing on either policy or technical skills to gain career capital.
This section covers:
Much of this advice is geared toward roles in the US, though it may be relevant in other contexts.
The chapter of the 80,000 Hours career guide on career capital lists five key components that will be useful in any path: skills and knowledge, connections, credentials, character, and runway.
For most jobs touching on policy, social skills, networking, and — for lack of a better word — political skills will be a huge asset. This can probably be learned to some extent, but some people may find they don’t have these kinds of skills and can’t or don’t want to acquire them.
That’s OK — there are many other routes to having a fulfilling and impactful career, and there may be some roles within this path that demand these skills to a much lesser extent. That’s why testing your fit is important.
To gain skills in policy, you can pursue education in many relevant fields, such as political science, economics, and law.
Many master’s programmes offer specific coursework on public policy, science and society, security studies, international relations, and other topics; having a graduate degree or law degree will give you a leg up for many positions.
In the US, a master’s, a law degree, or a PhD is particularly useful if you want to climb the federal bureaucracy. Our article on US policy master’s degrees provides detailed information about how to assess the many options.
Internships in DC are a promising route to test your fit for policy and get career capital. Many academic institutions now offer a strategic “Semester in DC” programme, which can let you explore placements in Congress, federal agencies, or think tanks.
The Virtual Student Federal Service (VSFS) also offers part-time, remote government internships. Students in this program work alongside their studies.
Once you have a suitable background, you can take entry-level positions within parts of the government and build a professional network while developing key skills. In the US, you can become a congressional staffer, or take a position at a relevant federal department, such as the Department of Commerce, Department of Energy, or the Department of State. Alternatively, you can gain experience in think tanks (a particularly promising option if you have an aptitude for research). Some government contractors can also be a strong option.
Many people say Washington, D.C. has a unique culture, particularly for those working in and around the federal government. There’s a big focus on networking, bureaucratic politics, status-seeking, and influence-peddling. We’ve also been told that while merit matters to a degree in US government work, it is not the primary determinant of who is most successful. People who think they wouldn’t feel able or comfortable to be in this kind of environment for the long term should consider whether other paths would be best.
If you find you can enjoy government and political work, impress your colleagues, and advance in your career, though, you may be a good fit. Just being able to thrive in government work can be a valuable comparative advantage.
US citizenship
Your citizenship may affect which opportunities are available to you. Many of the most important AI governance roles within the US — particularly in the executive branch and Congress — are only open to, or will at least heavily favour, American citizens. All key national security roles that might be especially important will be restricted to those with US citizenship, which is required to obtain a security clearance.
This may mean that those who lack US citizenship will want to consider not pursuing roles that require it. Alternatively, they could plan to move to the US and pursue the long process of becoming a citizen. For more details on immigration pathways and types of policy work available to non-citizens, see this post on working in US policy as a foreign national. Consider also participating in the annual diversity visa lottery if you’re from an eligible country, as this is low effort and allows you to win a US green card if you’re lucky.
Technical experience in machine learning, AI hardware, and related fields can be a valuable asset for an AI governance career. So it will be very helpful if you’ve studied a relevant subject area for an undergraduate or graduate degree, or did a particularly productive course of independent study.
We have a guide to technical AI safety careers, which explains how to learn the basics of machine learning.
Working at an AI company or lab in technical roles, or other companies that use advanced AI systems and hardware, may also provide significant career capital in AI policy paths. Read our career review discussing the pros and cons of working at a top AI company.
We also have a separate career review on how becoming an expert in AI hardware could be very valuable in governance work.
Many politicians and policymakers are generalists, as their roles require them to work in many different subject areas and on different types of problems. This means they’ll need to rely on expert knowledge when crafting and implementing policy on AI technology that they don’t fully understand. So if you can provide them this information, especially if you’re skilled at communicating it clearly, you can potentially fill influential roles.
Some people who may have initially been interested in pursuing a technical AI safety career, but who have found that they either are no longer interested in that path or find more promising policy opportunities, might also decide that they can effectively pivot into a policy-oriented career.
It is common for people with STEM backgrounds to enter and succeed in US policy careers. People with technical credentials that they may regard as fairly modest — such as a computer science bachelor’s degree or a master’s in machine learning — often find their knowledge is highly valued in Washington, DC.
Most DC jobs don’t have specific degree requirements, so you don’t need to have a policy degree to work in DC. Roles specifically addressing science and technology policy are particularly well-suited for people with technical backgrounds, and people hiring for these roles will value higher credentials like a master’s or, better even, a terminal degree like a PhD or MD.
There are many fellowship programmes specifically aiming to support people with STEM backgrounds to enter policy careers; some are listed below.
Policy work won’t be right for everybody — many technical experts may not have the right disposition or skills. People in policy paths often benefit from strong writing and social skills, as well as being comfortable navigating bureaucracies and working with people holding very different motivations and worldviews.
Ernest Moniz started his career as a physicist, becoming a professor at MIT in the 1970s. He gained management experience as a department head and leader of the MIT research council, and in the 1990s, he became the associate director for science in the Office of Science and Technology Policy in the White House.
He then became under secretary in the Department of Energy, which set him up to eventually become the Secretary of Energy under President Barack Obama — a highly influential role where he used his technical expertise in international negotiations over nuclear weapons.
Secretary Moniz shows how far someone with a technical background can get in terms of influencing policy — but of course you don’t have to be an MIT professor or lead a federal agency to have an impact.
There are other ways to gain useful career capital that could be applied in this career path.
Because this is one of our priority paths, if you think this path might be a great option for you, we’d be especially excited to advise you on next steps, one-on-one. We can help you consider your options, make connections with others working in the same field, and possibly even help you find jobs or funding opportunities.
Since successful AI governance will require work from governments, industry, and other parties, there will be many potential jobs and places to work for people in this path. The landscape will likely shift over time, so if you’re just starting out on this path, the places that seem most important might be different by the time you’re pivoting to using your career capital to make progress on the issue.
Within the US government, for instance, it’s not clear which bodies will be most impactful when it comes to AI policy in five years. It will likely depend on choices that are made in the meantime.
That said, it seems useful to give our understanding of which parts of the government are generally influential in technology governance and most involved right now to help you orient. Gaining AI-related experience in government right now should still serve you well if you end up wanting to move into a more impactful AI-related role down the line when the highest-impact areas to work in are clearer.
We’ll also give our current sense of important actors outside government where you might be able to build career capital and potentially have a big impact.
Note that this list has by far the most detail about places to work within the US government. We would like to expand it to include more options over time. (Note: the fact that an option isn’t on this list shouldn’t be taken to mean we recommend against it or even that it would necessarily be less impactful than the places listed.)
We have more detail on other options in separate (and older) career reviews, including the following:
Here are some of the places where someone could do promising work or gain valuable career capital:
In Congress, you can either work directly for lawmakers themselves or as staff on legislative committees. Staff roles on the committees are generally more influential on legislation and more prestigious, but for that reason, they’re more competitive. If you don’t have that much experience, you could start out in an entry-level job staffing a lawmaker and then later try to transition to staffing a committee.
Some people we’ve spoken to expect the following committees — and some of their subcommittees — in the House and Senate to be most impactful in the field of AI. You might aim to work on these committees or for lawmakers who have significant influence on these committees.
House of Representatives
Senate
The Congressional Research Service, a nonpartisan legislative agency, also offers opportunities to conduct research that can impact policy design across all subjects.
In general, we don’t recommend taking low-ranking jobs within the executive branch for this path because it’s very difficult to progress your career through the bureaucracy at this level. It’s better to get a law degree or a relevant graduate degree, which can give you the opportunity to start with more seniority.
The influence of different agencies over AI regulation may shift over time. For example, in late 2023, the federal government announced the creation of the US Artificial Intelligence Safety Institute, which may be a particularly promising place to work.
Whichever agency may be most influential in the future, it will be useful to accrue career capital working effectively in government, creating a professional network, learning about day-to-day policy work, and deepening your knowledge of all things AI.
We have a lot of uncertainty about this topic, but here are some of the agencies that may have significant influence on at least one key dimension of AI policy as of this writing:
Readers can find listings for roles in these departments and agencies at the federal government’s job board, USAJOBS; a more curated list of openings for potentially high impact roles and career capital is on the 80,000 Hours job board.
We do not currently recommend attempting to join the US government via the military if you are aiming for a career in AI policy. There are many levels of seniority to rise through, many people competing for places, and initially you have to spend all of your time doing work unrelated to AI.
However, having military experience already can be valuable career capital for other important roles in government, particularly national security positions. We would consider this route more competitive for military personnel who have been to an elite military academy, such as West Point, or for commissioned officers at rank O-3 or above.
Policy fellowships are among the best entryways into policy work. They offer many benefits like first-hand policy experience, funding, training, mentoring, and networking. While many require an advanced degree, some are open to college graduates.
(Read our career review discussing the pros and cons of working at a top AI company.)
Our job board features opportunities in AI safety and policy:
As we discuss in an article on accidental harm, there are many ways to set back a new field that you’re working in when you’re trying to do good, and this could mean your impact is negative rather than positive. (You may also want to read our article on harmful careers.)
There’s a lot of potential to inadvertently cause harm in the emerging field of AI governance. We discussed some possibilities in the section on advocacy and lobbying. Some other possibilities include:
We have to act with incomplete information, so it may never be very clear when or if people in AI governance are falling into these traps. Being aware that they are potential ways of causing harm will help you keep alert for these possibilities, though, and you should remain open to changing course if you find evidence that your actions may be damaging.
And we recommend keeping in mind the following pieces of general guidance from our article on accidental harm:
We think this work is exceptionally pressing and valuable, so we encourage our readers who might be interested to test their fit for governance work. But going into government, in particular, can be difficult. Some people we’ve advised have gone into policy roles with the hope of having an impact, only to burn out and move on.
At the same time, many policy practitioners find their work very meaningful, interesting, and varied.
Some roles in government may be especially challenging for the following reasons:
So we recommend speaking to people in the kinds of positions you might aim to have in order to get a sense of whether the career path would be right for you. And if you do choose to pursue it, look out for signs that the work may be having a negative effect on you and seek support from people who understand what you care about.
If you end up wanting or needing to leave and transition into a new path, that’s not necessarily a loss or a reason for regret. You will likely make important connections and learn a lot of useful information and skills. This career capital can be useful as you transition into another role, perhaps pursuing a complementary approach to AI governance.
We’ve been concerned about risks posed by AI for years. Based on the arguments that this technology could potentially cause a global catastrophe, and otherwise have a dramatic impact on future generations, we’ve advised many people to work to mitigate the risks.
The arguments for the risk aren’t completely conclusive, in our view. But the arguments are worth taking seriously, and given the fact that few others in the world seemed to be devoting much time to even figuring out how big the threat was or how to mitigate it (while at the same time progress in making AI systems more powerful was accelerating) we concluded it was worth ranking among our top priorities.
Now that there’s increased attention on AI, some might conclude that it’s less neglected and thus less pressing to work on. However, the increased attention on AI also makes many interventions potentially more tractable than they had been previously, as policymakers and others are more open to the idea of crafting AI regulations.
And while more attention is now being paid to AI, it’s not clear it will be focused on the most important risks. So there’s likely still a lot of room for important and pressing work positively shaping the development of AI policy.
If you’re interested in this career path, we recommend checking out some of the following articles next.
These degrees are highly valuable for those hoping to take on important roles in the US federal government.
The US government is likely to be a key actor in how advanced AI is developed and used in society, whether directly or indirectly.
Working at a leading AI company is an important career option to consider, but the impact of any given role is complex to assess.
