Guangyu Xu, a student just finishing his Ph.D. at the Centre for Particle Physics at Durham University, recently sent me a public letter he wrote, explaining the story of his job search, in hopes that it might be useful to others in a similar situation. As he acknowledges, his research record has been rather weak, so not surprising that his postdoc applications were not successful.
Back when I was writing Not Even Wrong, I did some detailed research into whatever information I could find about the HEP-TH job market, but I haven’t tried to do this more recently. Erich Poppitz did some analysis of data from the Theoretical Particle Physics Jobs Rumor Mill (available here), but only up to 2017. Given the large investment of various government agencies in the support of Ph.D. students, I would think that there would be data on career outcomes gathered by such agencies, but haven’t tried to look. Any pointers to this kind of data from anyone who has been looking into it would be appreciated.
Also of interest would be any up-to-date job search advice from those like Guangyu Xu who have been going through this recently.
Bibliometrics allow to infer most hires in hep worldwide. Fig. 6b of https://direct.mit.edu/qss/article/2/1/225/99129/Gender-issues-in-fundamental-physics-A shows the fraction of InSpire authors hired in hep up to ≈now as function of the date of their first paper. There are systematic issues, one needs to avoid misinterpreting, but the fraction is clearly declining.
I joined the tech industry after completing a physics PhD. My job search and later career experience differ quite a bit from Guangyu’s.
1. It is very easy for a hep-th PhD to become a strong software engineer or data scientist with a few months’ preparation. These subjects are much easier than hep-th, so with some practice hep-th PhDs should be very attractive candidates. Even at high-paying FANG companies many engineers and machine learning researchers have physics PhDs.
2. Directly applying to the companies leads to nowhere. In this industry one gets hired through recruiters or referrals by current employees. As a new grad one should try to get referrals from one’s network. Since many physics PhDs already work at these companies, it should not be too hard.
3. I find it unfortunate that Guangyu thought head hunters / recruiters useless. They come in different qualities and good recruiters can be very useful in navigating the application process.
4. Most of these jobs are located in the US so American candidates will probably have an easier time. It’s possible that being in the UK made Guangyu’s job search difficult.
I think getting hired in the tech industry as a physics PhD has become harder compared to a few years ago and especially at this moment with all the layoffs happening that route is difficult.
I don’t quite agree that software engineering is significantly easier in itself; with the larger number of jobs available (in normal times) one doesn’t need to be all that great in it to get hired, but being a top software engineer is very hard and those people are very rare.
The competition for professor level positions in research universities in technical areas is roughly as difficult as the competition to play professionally in first division in the Premier League or in the NBA (this could be off by an order of magnitude – in either direction – but I’d be surprised if it were more off than that). The main difference is that once obtained a professorship lasts for decades while a professional sports career lasts less than a decade (this makes the comparison difficult in certain ways), but the total net economic rewards are probably comparable (ignoring extreme cases).
In healthy development of potential professional athletes there is a serious effort to prepare them for alternative careers that is lacking in most development of potential future professors.
Those who make it suffer from various biases of perception. One is attributing success to hard work and innate talent and not to luck. Another is underestimating how much innate talent is needed to make it, and how many others simply lack that talent.
The notion that any high energy physics PhD can just succeed in programming or finance seems to me a fallacy of a similar sort. I’m among those who made it as a professor (something I attribute to stubborness), and despite some trying I’ve never gotten good at programming nor managed to understand finance. I suspect that most of those for whom these alternatives seem easy are simply better suited for those alternatives rather than that these alternatives are inherently “easier” (this reeks of arrogance).
The systemic failure is the lack of feedback until the end of the PhD program. Too few students are told that although they might finish the PhD program their prospects to continue are not good (one must be careful – I know several persons who *were* told their prospects were dim who persisted and had successful careers – but perhaps this apparently irrational persistence is precisely one of the qualities necessary to succeed as a professor).
I don’t think any of this is particular to high energy physics. There are always temporal variations in the employability of different fields, for diverse reasons, but the perspectives are difficult in any technical area because the competition is serious because the rewards are high.
I doubt that the prestige of the doctoral institution is as important as the letter write believes. In the US this probably matters more than in Europe, but mostly what is observed is akin to observing that Real Madrid’s youth development places more players in European leagues than does the Rayo Vallecano’s – one signs all the future Messis at age 12, the other signs from among the rest. More bluntly – one expects more future professors to come out of Imperial than out of Durham because of who entered these institutions.
I left physics almost 22 years ago for a quant role. Today I am responsible for a team of 95 people, about 1/3 being PhD level quant, and another 1/3 being MSc level quant. We deploy trading strategies, maintain massively parallel code bases used for extensive simulation, have risk managers, and a decent amount of reporting and analytics to support the business side. In retrospect I am glad I left physics, but I did turn one set of challenges in for another.
1. There’s more to the crossover opportunity than simply going and finding “alphas” at a fund. Educate on the space of opportunities.
2. Educate oneself on business in general. Often junior quants are hammers looking for nails. Further, they don’t always progress much beyond quant factotums. This is because everything gets rendered as a quant problem as opposed to a business problem (that may have a quant solution). Understanding the business is the key to mobility. Note that when I started out I basically build pricing engines – writing down equations, solving them, and coding them. These were state secrets at the time. Eventually they became shared software, then commodified for sale, then free, and essentially useless today; so the ability to adapt becomes crucial.
3. Starting out try to find ways to contextualize ones’ research so that it can be differentiated. Example: many CV today indicate “[…] using machine learning”. When asked why the answer is usually along the lines of “If the grant proposal doesn’t involve some type of ….” Being able to show some conviction and use cases can help in a tight job market or simply just getting the job one wants.
4. Try to find more than one reason (money) to want to become a quant. Many junior quants wash out for one reason or another after a few years (how many CVs have “1 year 3 months” at investment firm) while others seem to simply not progress beyond a certain point. Make sure the grass is really greener; or, as I tell people, that you’re pulled at least as much as you feel pushed into the profession.
5. Understand that it is an evolving space. There aren’t really any textbooks or guides. There’s dated materials out there and there’s a bias for people that write to either not really know or not have spent enough time to really be in a position to tell people. It’s amazing how many guides and books exist written by people with little or no actual experience. Some of the better physicists I know moved away from HFT when the first books were being published and into crypto before there were any books on crypto. Make diverse friends and be curious about trends to be able to see around corners.
6. Use all resources. Headhunters included. Show some grit in looking. About 10 or so years ago I started getting cold-called (eg on bloomberg) from job seekers writing a note, attaching a white-paper, and a link to some code, indicating that they worked on some problem they thought would helpful. When I started there were no email addresses (shops indicated they didn’t solicit, rather they recruited; iow, they had direct contact with certain select universities and didn’t need other avenues.) Things have changed and potential candidates have evolved. Don’t be closed to any type of advocacy.
Best of luck to all.
mimi:
I think you underestimate how hard learning to program is for some people. If you’re a good programmer (which I expect the majority of physics graduate students are) finding a job in machine learning, finance, or similar areas shouldn’t be that hard.
However, in my experience, people’s programming abilities vary widely, and if you are on the lower end of the scale, a few month’s preparation will not get you very far. I have met at least one top-notch mathematician who absolutely could not think algorithmically, and thus would have made an utterly hopeless programmer. These people will have a much harder time finding a job.
Luckily for Guangyu Xu, he was a competent programmer, although at the start of his job search he didn’t have any evidence for this. Eventually, he was able to amass enough evidence to convince at least one potential employer of this, and he was able to find a job.
BB,
I don’t want to turn this into a discussion of the problems with getting permanent positions at research universities. That competition via postdocs/tenure-track jobs/tenure is pretty well understood by all involved. It’s much less obvious what advice to give to the huge majority of people entering the field who won’t succeed at this competition.
A couple comments though about the specific situation of hep-th:
1. The other field I have a lot of first-hand information about is pure mathematics. The job situation there is nowhere near as bad as hep-th.
2. The comparison to successful professional athletes is flawed in one crucial respect: they win games and provide joy to spectators. For several decades now basically all of those successful in the hep-th competition have failed miserably at the game of understanding nature, depressing everyone who watches closely.
I read Guangyu Xu’s open letter. The first time I read it, it struck a chord with me and reminded me of my own job-hunting struggles after I got my Ph.D. I felt a kinship to him. But after reading his letter a few more times, I find it curious that he does not even mention his native China and its need for well-trained physicists like him.
I bailed out of doing high energy theory postdoc ~ 15 years ago because I had stopped believing the field was going anywhere and I didn’t think I was willing to do what it takes to become a professor. I found the transition to being a hedge fund quant relatively easy, but I was a good candidate because I cared about and decently understood programming, happened to know a reasonable amount of probability theory, and at heart was someone who was good at and just wanted to be able to spend time solving problems in applied math. With a bit of preparation I was able to brush up on options math and passably fake knowing a few facts about finance, statistics and data analysis. Some observations:
1. as a physicist who could program, I could go to a website, download a useful library, and write some files of code that would compile and do some calculation for me that I could use in a paper. This skillset is far from knowing basic software engineering, where you write code that interacts with hundreds of thousands of lines of code that may change and any time and that was written by hundreds of different people, use source control tools, unit testing, have a build and release process, etc. This was a shock.
2. Even fifteen years ago, the time when physicists could just show up and get a job was over. Banks were starting to hire from quant finance masters programs (incidentally, I suspect one of the main functions of these programs is not so much to teach anything especially useful to employers, but to produce OPT visas for bright students from abroad). The big funds never focused exclusively on physicists, but have always hired people from CS, math, EE, stats, operations research etc. You still sometimes meet candidates who think because they have a string theory PhD from a fancy school they will have the red carpet rolled out for them. Not the case.
3. Another ex-physicist told me before I left academia that the hardest math I would ever need again was to take the moving average of some numbers. That has turned out to be far from the truth. On the other hand, I’m pretty sure Rentec isn’t so astoundingly successful because they have the deepest bench of algebraic geometers.
4. By far the biggest change over the past fifteen years has been the rise of “data science” and machine learning. A lot of nonsensical marketing fluff goes along with this, but obviously there are real skills there. By and large physicists are cut off from this, at least at the cutting edge, though you do sometimes meet people who have used machine learning in a legitimate way for a real project. This seems to be true for theorists as well as people doing, say, high energy experiment or working on big data projects in astronomy or whatever.
5. No doubt being from a well known, fancy school helps you get an interview.
Having made the transition from academia to industry myself several years ago, I’d say the most useful skill was knowing how to learn. In interviews I’d tell them that I didn’t know much about computer networks or satellites, but I was willing and able to learn. That ended up getting me into a pretty good career.
Now that I sit on the other side of the table, I find that recruiting people is often less about the specifics of what someone knows, but more about the attitude they bring to the team. I’d much rather hire someone smart who is willing to stick with tough problems than take someone who looks good on paper but won’t actually deliver all that much.
Ben:
I would be very reluctant to infer anything from the lack of references to China. There are many, many reasons why someone may not consider returning to their original country a desirable option.
I sympathize with grad students facing dim prospects for making a career out of their passion, but join the very, very large club. Most people don’t get to pursue their passion as their career. If you got in a PhD in your passion, you had years more passion than most of us, and you are not haunted by the question of “what if I had…”
There are plenty of options besides software or finance for PhD physicists who aren’t tenure track. Among them:
1. TEACHING high school, community college, adjunct professor, teaching at smaller liberal arts colleges. At a high school like Andover, you’d get to mentor some of the top students in the country.
2.LAW SCHOOL (in the US), potentially going into IP law. (Please first spend time with attorneys and see if you can imagine spending your days doing law. Average career satisfaction is low, but some are happy with their choice. I know a former Physics PhD making millions as a partner at a big firm, but the stars really aligned for him).
3. MEDICINE (in the US). You may need 1-2 years to take the bio and chem prerequisites (this can be done at an “extension school”). Then 4 years of med school, then 3 to 7 years of residency (where you’re getting paid $50K – $70K per year). Then income ranging from $200-600K depending on the specialty. But make sure you like medicine first.
4. CONSULTING. If you have good people skills, you might be able to land a position at consulting firms like McKinsey or BCG, which can serve as a platform for other business jobs.
5. If you have trouble breaking into finance/software, get a master’s in Statistics, Data Science, Financial Engineering, CS, etc. (or possibly do a post-doc in these subjects). For software, there are “coding camps” that take less than 1 year, and that have good placement rates. The intellectual content of these programs should be less demanding than a physics PhD, and the MS degree (or post-doc, or coding camp) should help get you into the field.
Peter,
Sorry for the off-topic comment, but you, and your readers, would surely be interested to hear this.
I’m at CERN—I’d like to remain anonymous. CMS+Atlas picked up a 1350 GeV four-lepton excess, currently at three sigma. That’s the rumor. There will likely be a press release this coming week.
Various commenters have said that software engineering is not necessarily easy to learn. My point was not that it was easy on an absolute scale but only relatively for hep-th PhD students. I’ve worked at some top tech companies and have met quite a few other physics/math PhDs there. All of them agree that software engineering / data science are much easier than what they had to study in grad school. I’m sure there are some physicists/mathematicians for whom programming is hard (as Peter Shor pointed out) but most can become quite good with a few months of intensive work.
CPT,
Hadn’t heard anything like that, and that rumor sounds implausible (a joint CMS-ATLAS press release????). Informed sources encouraged to confirm or deny….
I’ll make one last comment in response to mimi and Peter S re programming that may be useful to physics/math people thinking about a transition:
It’s been my experience that there’s roughly two types of programmer that do what I do – the scrappy programmer and the programmer’s programmer. The former tends to be a quant with good problem solving skills that can accurately generate prototype models that solve a problem. c++, cuda, python, matlab, apl applied to pdes, optimizatin, simulation, valuation, etc. What they tend not to be good with is how the latter excels. Coding standards, documentation, scalability, extensibility, compute costs. I’ve run into real problems early on in my career mediating this interaction. It’s not uncommon to inherit tens, if not hundreds of thousands of lines of “spaghetti code”, perhaps calling other code across a zoo of languages. Long ago I figured out how to solve this. I have a dedicated tech team that has front, middle, and back end programmers; and they own the code base. The back end are real low-level programers with PhD comp sci, hard physics, backgrounds. The front tends to be app dev. The middleware folks deal with the transmission of data. The pure quant teams prototype while the tech teams productionalize according to standards we’ve developed, inclusive of testing and documentation.
So one bit of advice that I have is an incoming quant that is a scrappy programmer, who I assume is good at problem solving in their IDE/language of choice, is to be amenable to learning about the broader ecosystems to allow for smoother handoff and assimilation of prototypes. I’ve seen places literally shut down or taken over because people ignored design, architecture, costs, and standards as growth overpowered considerations around continuity and sustainability. Some math/physics types can’t be bothered with this stuff, or think it’s beneath them, but the ones that do think about it tend to do better and certainly have an easier time having their ideas accepted amongst a team and, most gratifyingly, their solutions put into production.
I recently retired from leading an organization with thousands of engineers, mathematicians, and scientists. I’d hired a lot of them. Over 35 years I learned to look to hire physicists (and mathematicians) for engineering roles. When we had to invent a solution to a novel problem, the physicists always started from first principles. To make a long story short, they helped the team find solutions our competitors never dreamed of. If you are a leader looking to amplify innovation in your team, bring some physicists in.
@CERN: by four-lepton excess, do you mean the possible discovery of a new lepton, in addition to the electron, tau and muon ?
@martibal Presumably the poster meant collision events with four leptons coming out and a reconstructed mass of 1350 GeV. Four leptons were one of the main signatures looked at in the original announcement of the Higgs boson discovery
All,
At least one of the comments here about a supposed new LHC result was an impersonation by a troll. Most likely the whole thing is trolling. So, deleting any more comments about this that come in (unless you send me the draft of the paper, with the plots…).
FWIW, my experience (heading a global financial modeling group) agrees with @cgh in every particular. The different approaches to programming is a point that cannot get too much emphasis.