Research

Abstract: New AI tools have the potential to change the way workers perform and learn, but little is known about their impacts on the job. In this paper, we study the staggered introduction of a generative AI-based conversational assistant using data from 5,179 customer support agents. Access to the tool increases productivity, as measured by issues resolved per hour, by 14% on average, including a 35% improvement for novice and low-skilled workers but with minimal impact on experienced and highly skilled workers. We provide suggestive evidence that the AI model disseminates the best practices of more able workers and helps newer workers move down the experience curve. In addition, we find that AI assistance improves customer sentiment, increases employee retention, and may lead to worker learning. Our results suggest that access to generative AI can increase productivity, with large heterogeneity in effects across workers.

Abstract: This paper examines the role of spillover learning in shaping the value of exploratory versus incremental R&D. Using data from the pharmaceutical industry, we show that novel drug candidates generate more dynamic spillovers than incremental ones. That is, despite being more likely to fail in the development process, novel drugs are more likely to inspire the development of subsequent successful drugs. Motivated by this fact, we develop a model where firms are better able to evaluate the viability of incremental drugs, but where investing in novel drugs helps firms learn about future related projects. Our model provides an empirical diagnostic for assessing the relative value of evaluation versus learning, namely that if firms place greater value on learning, then they should set a lower revenue threshold for investing in novel relative to incremental drugs. We in fact find that firms place less value on learning: they are less likely to invest in novel drugs and in turn novel drugs have higher revenues on approval. Finally, we provide suggestive evidence that some of these patterns are driven by concerns about the appropriability of spillover knowledge.

Abstract: We show that widely-used subjective assessments of employee "potential" contribute to gender gaps in promotion and pay. Using data on 30,000 management-track employees from a large retail chain, we find that women receive substantially lower potential ratings despite receiving higher job performance ratings. Differences in potential ratings account for 30-50% of the gender promotion gap. Women's lower potential ratings do not appear to be based on accurate forecasts of future performance: women outperform male colleagues with the same potential ratings, both on average and on the margin of promotion. Yet, even when women outperform their previously forecasted potential, their subsequent potential ratings remain low, suggesting that firms persistently underestimate the potential of their female employees.

Abstract: This paper views hiring as a contextual bandit problem: to find the best workers over time, firms must balance "exploitation" (selecting from groups with proven track records) with "exploration" (selecting from under-represented groups to learn about quality). Yet modern hiring algorithms, based on "supervised learning" approaches, are designed solely for exploitation. Instead, we build a resume screening algorithm that values exploration by evaluating candidates according to their statistical upside potential. Using data from professional services recruiting within a Fortune 500 firm, we show that this approach improves the quality (as measured by eventual hiring rates) of candidates selected for an interview, while also increasing demographic diversity, relative to the firm's existing practices. The same is not true for traditional supervised learning based algorithms, which improve hiring rates but select far fewer Black and Hispanic applicants. In an extension, we show that exploration-based algorithms are also able to learn more effectively about simulated changes in applicant hiring potential over time. Together, our results highlight the importance of incorporating exploration in developing decision-making algorithms that are potentially both more efficient and equitable.

Abstract: This paper studies how insurance coverage policies affect incentives for pharmaceutical innovation. In the United States, the majority of drugs are sold to Pharmacy Benefit Managers (PBMs), which administer prescription drug plans on behalf of insurers. Beginning in 2012, PBMs began adopting "closed formularies", excluding coverage for certain drugs, including many newly approved drugs, when adequate substitutes were available. We show that this policy reshaped upstream R&D activity and led pharmaceutical firms to shift investment away from therapeutic classes at greater risk of facing coverage exclusions. This move translated into a relative decline in the number of drug candidates that appear more incremental in their therapeutic contribution: that is, those in drug classes with more pre-existing therapies and less scientifically novel research.

Abstract: We analyze firms' decisions to invest in incremental and radical innovation, focusing specifically on pharmaceutical research. We first develop a new measure of drug novelty that is based on the chemical similarity of new drug candidates to existing drugs. We show that drug candidates that we identify as ex-ante novel are riskier investments, in the sense that they are subsequently less likely to be approved by the FDA. However, conditional on approval, novel candidates are more likely to be both socially as well as privately valuable. Second, we shed light on the role of financial constraints in firms' decisions to invest in novel drug compounds. We use variation in the expansion of Medicare prescription drug coverage in the United States, which differentially benefited firms based on their drug portfolio, to isolate exogenous variation in firm cash flows. We find that firms that benefit more from the expansion of drug coverage develop more new drug candidates as a result. This increase is primarily driven by the development of more molecularly novel drug compounds.

Abstract: Absent explicit quotas, incentives, reporting, or fiscal year-end motives, drug approvals around the world surge in December and at the end of each month. This pattern is found in a large, global data set consisting of drug approvals from the United States, the European Union, Japan, China, and South Korea, suggesting that this pattern reflects an empirical regularity common across cultures and regulatory regimes. In the United States, the number of December drug approvals is roughly 80% larger than in any other month. Similar approval spikes occur at the end of each calendar month. Additionally, approvals spike before holidays, such as before Thanksgiving in the United States and the Chinese New Year in China (but not vice versa). Drugs approved in December and at month-ends are associated with significantly more adverse effects, including more hospitalizations, life-threatening incidents, and deaths. This pattern is consistent with a model in which regulators rush to meet internal production benchmarks associated with salient calendar periods: this type of "desk-clearing" behavior results in more lax review, which leads both to increased output and increased safety issues.

Abstract: We propose a macroeconomic model in which variation in the level of trust leads to higher innovation, investment, and productivity growth. The key feature in the model is a hold-up friction in the creation of new capital. Innovators generate ideas but are inefficient at implementing them into productive capital on their own. Firms can help innovators implement their ideas efficiently, but cannot ex ante commit to compensating them appropriately.

Abstract: The best worker isn’t always the best candidate for manager. In these cases, do firms promote the best potential manager or the best worker in their current job? Using data on the performance of sales workers from 131 firms, we find evidence consistent with the Peter Principle: firms prioritize current job performance when making promotion decisions, at the expense of other observable characteristics that better predict managerial quality. We estimate that the costs of managerial mismatch are substantial, suggesting that firms are either making inefficient promotion decisions or that the incentive benefits of emphasizing current performance must also be high.

Abstract: We quantify the impact of scientific grant funding at the National Institutes of Health (NIH) on patenting by pharmaceutical and biotechnology firms. Our paper makes two contributions. First, we use newly constructed bibliometric data to develop a method for flexibly linking specific grant expenditures to private-sector innovations. Second, we take advantage of idiosyncratic rigidities in the rules governing NIH peer review to generate exogenous variation in funding across research areas. Our results show that NIH funding spurs the development of private-sector patents: a $10 million boost in NIH funding leads to a net increase of 3.26 patents. Though valuing patents is difficult, we report a range of estimates for the private value of these patents using different approaches.

Abstract: Job testing technologies enable firms to rely less on human judgement when making hiring decisions. Placing more weight on test scores may improve hiring decisions by reducing the influence of human bias or mistakes but may also lead firms to forgo the potentially valuable private information of their managers. We study the introduction of job testing across 15 firms employing low-skilled service sector workers. When faced with similar applicant pools, we find that managers who appear to hire against test recommendations end up with worse average hires. This suggests that managers often overrule test recommendations because they are biased or mistaken, not only because they have superior private information.

Abstract: Evaluators with expertise in a particular field may have an informational advantage in separating good projects from bad. At the same time, they may also have personal preferences that impact their objectivity. This paper develops a framework for separately identifying the effects of expertise and bias on decision making and applies it in the context of peer review at the US National Institutes of Health (NIH). I find evidence that evaluators are biased in favor of projects in their own area, but that they also have better information about the quality of those projects. On net, the benefits of expertise tend to dominate the costs of bias. As a result, limiting the influence of personal preferences may also reduce the quality of funding decisions.

Abstract: Scientists and policy-makers have long argued that public investments in science have practical applications. Using data on patents linked to U.S. National Institutes of Health (NIH) grants over a 27-year period, we provide a large-scale accounting of linkages between public research investments and subsequent patenting. We find that about 10% of NIH grants generate a patent directly but 30% generate articles that are subsequently cited by patents. Although policy-makers often focus on direct patenting by academic scientists, the bulk of the effect of NIH research on patenting appears to be indirect. We also find no systematic relationship between the “basic” versus “applied” research focus of a grant and its propensity to be cited by a patent.

Abstract: This paper examines the success of peer-review panels in predicting the future quality of proposed research. We construct new data to track publication, citation, and patenting outcomes associated with more than 130,000 research project (R01) grants funded by the U.S. National Institutes of Health from 1980 to 2008. We find that better review scores are consistently associated with better research outcomes and that this relationship persists even when we include detailed controls for an investigator's publication history, grant history, institutional affiliations, career stage, and degree type. A one-standard deviation worse peer-review score among awarded grants is associated with 15% fewer citations, 7% fewer publications, 19% fewer high-impact publications, and 14% fewer follow-on patents.

Abstract: This chapter provides an overview of grant funding as an innovation policy tool aimed at both practitioners and science policy scholars. We first discuss how grants relate to other contractual mechanisms such as patents, prizes, or procurement contracts, and argue that, among these, grants are likely to be the most effective way of supporting early stage, exploratory science. Next, we provide a brief history of the modern scientific grant and discuss the current state of knowledge regarding several key elements of the design of grant programs: the choice of program scope, the design of peer review, as well as approaches for creating incentives for risk-taking and translation for grant recipients. We argue that, in making these choices, policy-makers should adopt a portfolio-based mindset that seeks a diversity of approaches, while accepting that high failure rates for individual projects is in fact part of an effective grant-making program. Finally, we close with a call for increased rigor in the evaluation of grant programs. By adopting randomized controlled trials and other quasi-experimental techniques, policy makers can both communicate and improve the impact that grant programs have on discovery and innovation, thereby creating a stronger justification for their expansion or continued existence.

Abstract: The move toward increased school accountability may substantially affect the career risks that school leaders face without providing commensurate changes in pay. Since effective school leaders likely have significant scope in choosing where to work, these uncompensated risks may undermine the efficacy of accountability reforms by limiting the ability of low-performing schools to attract and retain effective leaders. This paper empirically evaluates the economic importance of principal mobility in response to accountability by analyzing how the implementation of No Child Left Behind (NCLB) in North Carolina affected principal mobility across North Carolina schools and how it reshaped the distribution of high-performing principals across low- and high-performing schools. Using value-added measures of principal performance and variation in pre-period student demographics to identify schools that are likely to miss performance targets, I show that NCLB decreases average principal quality at schools serving disadvantaged students by inducing more able principals to move to schools less likely to face NCLB sanctions. These results are consistent with a model of principal-school matching in which school districts are unable to compensate principals for the increased likelihood of sanctions at schools with historically low-performing students.

Abstract: The effect of vouchers on sorting between private and public schools depends upon the price elasticity of demand for private schooling. Estimating this elasticity is empirically challenging because prices and quantities are jointly determined in the market for private schooling. We exploit a unique and previously undocumented source of variation in private school tuition to estimate this key parameter. A majority of Catholic elementary schools offer discounts to families that enroll more than one child in the school in a given year. Catholic school tuition costs therefore depend upon the interaction of the number and spacing of a family’s children with the pricing policies of the local school. This within-neighborhood variation in tuition prices allows us to control for unobserved determinants of demand with a set fine geographic group fixed effects while still identifying the price parameter. We analyze this variation by using data on over 3,700 school tuition schedules collected from Catholic schools around the nation, matched to restricted Census data that identifies precise location that can be matched to the nearest Catholic school. We find that a standard deviation decrease in tuition prices increases the probability that a family will send its children to private school by one half percentage point, which translates into an elasticity of Catholic school attendance with respect to tuition costs of -0.19. Our subgroup results suggest that a voucher program would disproportionately induce into private schools those who, along observable dimensions, are unlike those who currently attend private school.