Published on

thoughts on iteration times & the research university system

Biotech has two big problems today: (a) iteration cycles in biotech are fundamentally too slow, which we believe has fundamentally technological roots and (b) the world is currently relying on universities for our research, which are failing at their dual mandate of innovating and training. These mandates are often in fundamental conflict, and by trying to do both, the university can do neither well. Universities systematically push the best scientists either into industry or into middle management roles as professors the minute that their PhD training is finished, preventing the formation of well-gelled teams of experienced individual contributors.

To make meaningful progress in bioengineering, a uniformly-excellent team of scientists working on important problems is table-stakes. But academia as built today makes those teams only coincidentally. Academia has no system for removing underperforming team members without incurring substantial reputational and professional costs to everyone involved - instead, underperformers linger in labs for years, when both sides would be better off with a different fit. Academia's cultural norms around publications and grants force a "one-size-fits-all" strategy to team construction, in which one senior mentor (last author) advises one junior mentee (first author), and a longer list of significantly less valued supporting roles (all the middle authors). This disparity in recognition between the first author and middle authors introduces an uncomfortable social dynamic between would-be co-authors which discourages collaboration. Moreover, in today's academic culture, wasting 3 years producing an output of questionable usefulness is seen as superior to moving on and publicly declaring failure once it becomes clear that the project will not produce anything of substantial value. In short, academic culture systematically undermines it's potential to form great teams - those that do form are transient, fragile, and unsupported by the institution at large.

This old system is breaking down. As of this writing, the University of California is seeing ~50,000 of these trainees and researchers on strike for higher pay, which is the largest higher education labor strike in history. More researchers than ever before are fleeing academia to take positions in industry, lured by the meme of higher pay for reduced freedom to conduct high-impact catalytic research. Many postdoctoral positions are going unfilled. Researchers are increasingly dissatisfied with the bureaucracy of filling out grants, squabbling over publication credits, and navigating increasingly political and petty university administrations. Universities are every day becoming less attractive places to do cutting edge research, and for many, universities have passed their peak.

Returning to the problem of slow iteration cycles, let me explain why that is a problem. It can easily take 10 years between when you first design a therapeutic, and when you know that the therapeutic is safe and effective. Add in the fact that most therapeutics fail at some point in those 10 years, and if you consider the average duration of a career in biotech, it is not uncommon that even talented researchers will never see a drug reach a patient. That's insane! Success and learning is a compounding, iterative process, but most scientists developing drugs barely get to crack a double digit number of projects before they retire. It's nearly impossible to get good at something if you (a) only get feedback once every several years,  (b) that feedback is vague, noisy, and can be impacted by external factors beyond your control or understanding and (c) the feedback that your peers receive is kept secret, because they want to prevent their rivals from gaining insight that they just paid millions of dollars and years of their life to learn.  Finally this feedback cadence also really, really boring for most ambitious people - ambitious people like to be in areas where they are constantly growing and learning, succeeding and failing.

And so, the multi-trillion dollar question: why does it take 10 years to iterate once in biotechnology? In a word, regulations. Manufacturing is slow because GMP procedures are stringent; testing is slow because clinical trials must satisfy a long list of requirements. Why are there so many regulations? Those regulations are there, of course, because many prospective therapeutics will in fact regularly maim or kill the patient far worse than the disease they hope to cure. Why do therapeutics maim and kill patients? Fundamentally, because we do not understand how to predictably design safe & effective drugs. Why don't we know how to predictably design new drugs? Because our tools to predict the behavior of the human body are insufficient for the task. If we had the tools to design safe and effective drugs, the regulators would eventually adjust. The FDA, contrary to some critics, does has a demonstrated track record of eventually reducing regulatory burden once it is demonstrably the case that a given intervention is reliably safe (see the recent history of hearing aids). Therefore, the root of the iteration cycle problem is a technological one - develop a process for designing predictably safe and effective drugs, prove that your process works, and this curse of slow iteration speeds will be lifted.

This magic box I am describing, which accepts as inputs an uncured disease and some money and outputs a cure within a week - what does it look like? War-weary biopharma veterans will insist that this box does not exist, and will never exist - we are doomed to forever evaluate drugs like a bookie tending to his sportsbook - discounting the odds, papering over real-world variances with the application of probabilistic portfolio management. But, if you wanted to try to prove them wrong, though, here is a playbook:

  1. Hire the best scientists you can find, the ones who are getting squeezed away from the bench in academia. Help them stay at the bench where they have a demonstrated track record of success.
  2. Set them to work building new catalytic technologies, that improve our ability to predictably intervene in humans.
  3. In the process, reset the culture of this new team to focus above all on deploying these technologies in the real world.

Postscripts:

  1. Previous models (e.g. Bell Labs) focused on innovation first, and training happened as a matter of course. This model was famously effective at generating innovative results for the for-profit monopolies that supported them, but is today generally considered to be less profitable than simply licensing the outputs of research universities. These outputs are now increasingly subsidized through a mix of government support and below-market wages paid to "trainees" undercutting the economics of an in-house research lab. Cynically, one could say that the only remaining justification for substantial in-house R&D is that in-house R&D can justify the continued existence of the monopoly that supports it, much as Bell Labs justified the existence of a much more profitable telephone company. Outside of PR and recruiting needs, it's hard to explain the existence of a legit protein engineering research program at Salesforce, whose primary product line is a database to track sales leads.
  2. In computer science, having worked on 10 projects, of which 5 actually worked, would mark you as a mediocre student coming out of your first spring semester introduction to computer science course. In biotechnology, those same numbers would mark you as a seasoned industry veteran with a track record of uncommon success.
  3. To carry the computer science comparison one step further, dealing with the human body and genome is like dealing with a 3 billion year old continuously updated legacy codebase. Most of the biggest revolutions in biotechnology have simply come from developing better "debuggers" such as sequencing. Most people who have coded can relate to this: if you don't know the language very well, and the codebase is completely undocumented and massively complex, sometimes your best bet is to just setup a sandbox environment and repeatedly run and re-run your script until it seems to be doing what you want. For biology, though, we don't have very accurate sandboxes for the human body, and every time you press command-B you spend 10 years and $X00M waiting for the command to run, like it's the EC3 typo from hell.
  4. To go one level further, why aren't our tools good enough? A few reasons: (a) biology doesn't follow many straightforward and abstractable rules, (b) slow iteration cycles introduce a recursive "chicken and egg" problem, since it's hard to get better quickly if your experiments are so slow, and (c) we've underinvested in robust tool development.
  5. One parallel (but much less critical) reason why iteration cycles are long is that because success is uncertain, finance will sometimes insist on a waterfall approach: only pay for the next step once you've started the previous step. And so, steps that could be pursued in parallel are instead pursued in sequence, only once the cheaper derisking step has been completed.