Performance Measures For The Socio-Economic Impact

Jointly published by Akadmiai Kiad, Budapest and Springer, Dordrecht
Scientometrics, Vol. 68, No. 1 (2006) 4171
Performance measures for the socio-economic impact of government spending on R&D

BRIAN P. COZZARIN
Department of Management Sciences, University of Waterloo, Waterloo, Ontario (Canada)
The aims of this paper are to summarize Canadian government programs pertaining to research and development (R&D) and R&D support programs, and to propose a method for analyzing their socio-economic impact. The programs under investigation include: Canada Research Chairs Canada Millennium Scholarship Foundation Canada Foundation for Innovation Technology Partnerships Canada (TPC) Industrial Research Assistance Program (IRAP) Natural Sciences and Engineering Research Council (NSERC) Social Sciences and Humanities Research Council (SSHRC) Canada Institutes of Health Research (CIHR) Canadian Institute of Advanced Research (CIAR) Pre-Competitive Advanced Research Networks (PRECARN) Networks of Centres of Excellence
Introduction The aims of this paper are to summarize Canadian government programs pertaining to research and development (R&D) and R&D support programs, and to propose a method for analyzing the socio-economic impact of these programs. A relevant question
Received November 7, 2005 Address for correspondence: BRIAN P. COZZARIN Department of Management Sciences, University of Waterloo 200 University Avenue West, Waterloo, Ontario, Canada, N2L 3G1 E-mail: bpcozzar@engmail.uwaterloo.ca 01389130/US $ 20.00 Copyright 2006 Akadmiai Kiad, Budapest All rights reserved
B. P. COZZARIN: Government spending on R&D
to ask is why do governments fund R&D? As the survey of 15 national innovation systems by NELSON (1993) found, the similarities in technology policy across the 15 market based economies was striking. The relatively small differences in technology policy were attributed to political priorities and national economic conditions (i.e country size and overall affluence). Thus the funding of R&D, innovation and diffusion is apparently of national importance. The economic argument for the existence of such funding stemmed initially from SCHMOOKLER (1959) from a social standpoint private firms engage in too little R&D. Schmookler asserted that the government should finance R&D because the costs are too great for private firms and that the prospective returns are too uncertain, too distant, and appropriability is weak. ARROW (1962) also demonstrated that firms will under-invest in research in a competitive economy because they cannot fully appropriate the returns to their investment (due to spillovers). Thus spillovers distort firms investment behaviour and the resulting R&D will be suboptimal (DIXIT & STIGLITZ, 1977; HALL, 1986; STONEMAN, 1987). PAVITT & WALKER (1976) listed a number of alternative reasons for government intervention in the R&D market: weak management ability lowers the chances of successful innovation, information asymmetry in downstream product markets may prevent users/consumers from buying innovative products, private R&D will not occur when innovations have predominantly external benefits in areas such as safety, environment or job satisfaction, the structure of the industry or industrial climate may inhibit innovation, and finally, short-term time horizons prohibit long-term investments in R&D. What is innovation? The Oslo (OECD, 1997) manual defines an innovation as the commercial introduction of a new or significantly improved product or process to the market. Historically innovation has been viewed as a linear process [research invention development innovation diffusion] (UTTERBACK, 1974; UTTERBACK & ABERNATHY, 1975; PAVITT & WALKER, 1976). The way the model depicts the process is as follows: if more innovations are desirable then all that has to be done is fund more basic research. More recently, the linear view of technological change has been superseded by a non-linear conceptual model that features feedback loops emanating from each stage.* Treating diffusion separately from development and innovation has been criticized (FREEMAN, 1994; METCALFE, 1994). Empirical work has shown that as products and processes diffuse they change considerably (METCALFE, 1981, 1988; MANSFIELD et al., 1977; MANSFIELD, 1989). TEECE (1996) associates the following
One needs only to conduct a cited reference search using Web of Science (THOMSON SCIENTIFIC, 2005) to see that a vast literature on the innovation process has developed from UTTERBACK (1974) with 151 cited references and UTTERBACK & ABERNATHY (1975) with 264.
*
42
Scientometrics 68 (2006)
characteristic with invention and innovation: uncertainty, path dependency, cumulativeness, irreversibilities, technological inter-relatedness, tacitness and nonappropriability. Invention and innovation are uncertain, since they involve the generation of new ideas, and the development of new markets. The three types of uncertainty related to innovation are primary (due to randomness), secondary (due to communication), and behavioural (due to opportunism). Secondary uncertainty can be controlled through organizational form. Choosing one path may block or delay access to alternate paths. The cumulative nature of innovation is tied to the cumulative property of learning in general. Spillovers can occur within firms, between firms in the same industry, between firms in different industries, and between countries. In general, intra-industry spillovers are more common than inter-industry spillovers (DE BONDT, 1996). One well understood reason for government intervention in the provision of R&D is spillovers. Table 1 lists some recent empirical evidence concerning spillovers. In general, spillovers are good for the economy as a whole but bad for innovators themselves. PAVITT (1984) recognized two central characteristics of innovation: that most knowledge applied by firms is not general purpose, nor is it easily transmitted nor easily reproduced, their search for innovation is constrained since firms cannot evaluate all innovation possibilities thus technological change is localized, cumulative and firmspecific; secondly, innovation exhibits much variety in terms of products and processes, by sector, by the size/pattern of technological diversification of innovating firms. For instance, VON HIPPEL (1995) found that users of scientific instruments (gas chromatograph, nuclear magnetic resonance spectrometer, ultraviolet absorption spectro photometer, transmission electron microscope) developed 77 percent of innovations which were then incorporated by manufacturers. In a well-known paper, COHEN & LEVINTHAL (1990) assert that at the level of the individual prior knowledge permits the assimilation and exploitation of new knowledge. In human beings, prior knowledge assists in learning, and the storage of knowledge is enhanced by associative learning wherein linkages are established. Thus, learning is cumulative and learning performance is greatest when that which is being learned is related to prior knowledge. Their argument for firm-level learning hinges on this simple model of the human learning process. They call the knowledge base of a firm its absorptive capacity. In order for a firm to learn and therefore innovate it must possess some baseline absorptive capacity. And, in order for a firm to learn it must be able to absorb new information from the outside environment i.e. from universities, competitors, technical institutes, government R&D labs, conferences, personal contact etc. Cohen and Levinthal cite literature showing that firms who conduct their own R&D are better at assimilating and understanding external knowledge. Thus their main argument is that firms should conduct R&D in order to become better innovators especially in an environment where the ease of learning is difficult.
43
Table 1. Empirical evidence related to spillovers Results of MOHNEN (1992), BERNSTEIN (1997), MCFETRIDGE (1995), JAFFE (1989) Private rates of return to R&D investments exceed those of other investments Social rates of return to R&D investment are up to five times greater than private rates of return; the size of spillover benefits varies Social rates of return on basic R&D are higher than applied R&D Public R&D yields lower rates of return than private R&D, but higher rates of return than public infrastructure capital R&D spillovers reduce variable costs and enhance productivity (varies by firm or industry level), similar qualitative results were found with corporate R&D projects (Henderson and Cockburn, 1993). R&D spillovers increase output and help reduce prices R&D spillovers are generally partial substitutes for labour and materials, but complements to capital (other than R&D capital), spillovers reduce labour and material demand, but increase demand for capital, a major component of R&D capital is skilled labour R&D spillovers cause an increase in R&D capital investment in R&D performing firms, in low R&D intensive firms spillovers act as a substitute for R&D capital investment, at an industry-level spillovers reduce total industry R&D capital investment of the receiving industry R&D spillovers lead to productivity gains in other countries, international spillovers depend on trade and other relations between countries, large R&D-intensive economies subsidize small economies with low R&D intensity through spillovers Corporate patent activity responds positively from knowledge spillovers emanating from university research
Technology policy Introduction Technology policy has come to the forefront because Western governments believe that technological change is the real engine of economic growth. The old Keynesian macroeconomic levers that governments used to rely on (i.e. trying to increase employment by increasing aggregate demand by increasing the money supply for example; of special interest was increasing the demand for products with a high elasticity of employment, KEYNES, 1936, ch. 20) have been discarded in favour of stable monetary policy. What is left for governments to do? They have decided that they can try and address the market failure of too little R&D in a competitive economy. If governments believe in an evolutionary framework,* then they believe that purely deductive logic cannot yield the optimal sorts of policy that should be pursued. Instead, decision processes are inductive, involving localized search (ARTHUR, 1994). Thus, policy makers have to engage in inductive experiments to determine which policies work. Not only that, but learning and hence innovation has three faces that influence policy-making: it is a joint product between other processes that use and produce technology, it can result from collaboration or interaction with other institutions, and it can result from a process internal to the firm i.e. an R&D initiative (MALERBA, 1992).
*
I really have no idea whether this is the case. Yet, the R&D policy initiatives of the Government of Canada lead me to believe that they do.
44
METCALFE (1994) asks what are the central issues in technology policy from an evolutionary perspective? He lists two central issues. The first, are policies that take innovation possibilities of firms as given and hence are aimed at reducing R&D costs or increasing returns to R&D. For example, the cost of R&D can be reduced through subsidies (STONEMAN, 1991) or through tax incentives. Policies that can increase the payoff of R&D include government procurement policies or an increased length and/or scope of patent protection (NORDHAUS, 1969). The second, are policies which try to enhance the innovation possibilities of firms. For example, they can enhance innovation possibilities through R&D collaboration (PECK, 1986; BAUMOL, 1992; GEROSKI, 1992), or through the establishment of linkages (informal or formal) to public R&D institutions (universities, government labs). These linkages will facilitate spillovers from public R&D to private firms. There is empirical evidence that such spillovers are important for innovation possibilities (MANSFIELD, 1980; NELSON, 1981; COHEN & LEVINTHAL, 1989; JAFFE, 1989). NELSON (1995) asked three questions related to national technology policy: what makes an innovative system effective, how do we infer causality, and do high-tech industries benefit more than other industries from government intervention? He divided countries into three groups large, high-income; small, high-income; and low-income. In some high-income countries (France, UK and U.S.) defence R&D accounts for the major differences in government funding of R&D. Furthermore, the institutional characteristics of countries seem everlasting since many 19th century institutions still exist today. Technology policy should address two main features of innovation: the variety of innovations introduced into an economy, and the selection process that changes the relative economic importance of innovations (METCALFE, 1994). As an example of the selection process, think of Liquid Paper that was invented and marketed in 1956 by Bette Nesmith Graham. The dawn of the personal computer and word processing spelled the end of the electric typewriter and Liquid Papers indispensability. Collaboration Firm collaboration can overcome the free rider problem and the problems associated with first-mover advantage, by jointly pooling resources. The free rider problem is basically one of moral hazard, wherein some incumbents will let other firms perform R&D and will wait for the ensuing spillovers to occur. The problem with seeking a first mover advantage occurs when incumbents independently pursue being first to market with a new innovation, however, in so doing they duplicate each others
45
R&D efforts. Since R&D represents a sunk cost, duplication is essentially a dead weight loss to the economy.* Antitrust policy has recently welcomed collaboration (U.S. National Cooperative Research Act, 1984) instead of confusing it with the formation of a cartel. Fiscal and investment policy Fiscal and investment policy can artificially increase the returns on R&D investments for private firms, hence increasing the incentive for innovation. According to MCFETRIDGE (1995), industry-wide direct subsidies have a high rate of return, while firm specific direct subsidies have a low rate of return. His conclusion is that tax incentives are generally more effective than direct subsidies, although he admits that only a limited amount of empirical evidence is available (MCFETRIDGE, 1995). Government procurement is a type of public investment that may encourage R&D. Governments attempt to change demand conditions by acting as a buyer (i.e. the U.S. defense industry). Like direct subsidies, government procurement tends to have a negative impact on competitive advantage and the innovation rate, but it can be beneficial in some cases (PORTER, 1990). Government procurement is harmful if it creates a guaranteed market, but is beneficial if it stimulates early demand, creates demanding and sophisticated buyers, reflects international needs, or creates processes that facilitate innovation (PORTER, 1990). According to MCFETRIDGE (1995) tax incentives are preferable to both direct subsidies and government procurement in achieving economic growth. He also suggests that current tax incentives have a positive effect, but that additional benefits would not necessarily arise from making them more generous. Table 2 summarizes the effects of tax incentives and direct subsidies on innovation. An additional factor in choosing between tax incentives and direct subsidy is that of moral hazard, or opportunism. Tax incentives have a higher chance of provoking dishonesty since the fund allocation mechanism depends on a human decision, which can be subject to influence and bias.
* I am talking here about R&D projects that are exact or nearly exact replicas of one another. NELSON (1961) showed that parallel (but somewhat different) R&D efforts (he used the example of developing a new fighter aircraft for the USAF) are beneficial. One of the main reasons for parallel efforts pertains to sub-optimal lock-in effects (of the type that Arthur discusses) because early on a particular technology looks promising and all others are discarded. However, as time passes (and as history has shown) other technologies may supersede the early winner in terms of cost and/or performance. Nelson also showed that with some fairly straightforward analysis, the optimal number of parallel efforts can be found.
46
Table 2. Comparison of tax incentives and direct subsidies Tax incentives Markets determine which investments will be undertaken; decision-making remains with investors Direct subsidies Direct subsidies involve discretionary government control over decision-making; funds are selectively channeled to sectors, or to firms, or to investments identified as having the greatest potential for growth or the most need for assistance Direct subsidies targeted to relatively few sectors, firms or investments Direct subsidies can be accessed by both taxpaying and non-taxpaying firms Revenue cost for direct subsidies is capped by the granting authority at a specific level Funding levels for direct subsidies are often established on an annual basis and may vary, sometimes significantly, from year to year Direct subsidies can be more costly in terms of administration and compliance; harder to access, less timely, less certain and more burdensome than tax incentives
Tax incentives typically structured to deliver assistance to a broad range of sectors, firms or investments Tax incentives accessed by both taxpaying and nontaxpaying firms only through the use of refunds or loss-transfer provisions Revenue cost of tax incentives depends on marketdetermined levels of investment Tax system can be more effective in encouraging longer-term investments; firms can reasonably expect to receive ongoing benefits when multi-year projects are undertaken Existing tax administration structure is used, thus making tax incentives less costly in terms of administration and compliance; easier to access, more timely, more certain and less burdensome than direct subsidies Source: DEPARTMENT OF FINANCE (1997)
Technical standards Government can encourage technological innovation by creating technical standards in areas where technological change is inter-dependent and poorly coordinated (LINK & KAPUR, 1994). By imposing adequate technical standards, government can remove unnecessary uncertainty and allow firms to move ahead. According to PORTER (1990, p.653), government policy encourages the upgrading of competitive advantage if it supports early adoption of technical standards that embody a high overall level of technology. However, such standards should be imposed with caution since the adoption of standards that do not endure can be very harmful and impede on innovation. Human capital We know that learning in general enables more learning (COHEN & LEVINTHAL, 1990). Training differs in its specificity and thus who is willing to pay for it. For example, training can either be funded by the agent themselves, a trade association, or by the government. Highly specific training is difficult to transfer from one firm to
47
another or even one department to another (i.e. training for a company-specific application). General training, on the other hand, is subject to spillovers (since it is a desirable screening device in the labour market), so policy is necessary to ensure that it is available at the socially optimal level. Thus, government support of general training is prudent (GUNDERSON, 1974). Adequate training of employees is also crucial to the successful implementation of most innovations, since technology generally causes an increase in required skill levels. Closing note A closing note for this section is relevant from PAVITT & WALKER (1976). They stated many years ago, that the objectives, level of financial assistance, and type of government programs for R&D are entirely dependent on political preferences not on economic theory. Canadian Government Programs* This section briefly describes the 11 programs under investigation in this study. The intention is not to be exhaustive, but to highlight the budget, key goals of each program, the expected outputs and the overall expected outcome for Canadian society. Canada Research Chairs Established in 2000 by the Government of Canada, the Canada Research Chairs main objective is to achieve and maintain the highest quality research possible, attract and keep researchers, improve the training of employees, improve universities capability for producing and applying new knowledge, and taking full advantage of available research resources. The Government of Canada allocated $900 million to set up 2,000 research chairs across Canada by 2005. Targeting quality research, new researchers, and institutes to carry out research, the program is a strong supporter of the national strategy to establish Canada as one of the worlds top five countries for research and development. To help achieve this goal, the Chairs Program helps universities and their affiliated research institutes and hospitals become highly respected centres of research and training. This contributes to higher levels of competitiveness in the global economy, improved health of Canadians and enriched social and cultural life.
Most of the information in this section was either found directly in INDUSTRY CANADA (2005) or from links contained therein to other Government of Canada websites.
48
Millennium Scholarships Established in 1998, with an endowment of $2.5 billion from the Government of Canada, the Canada Millennium Scholarship Foundation is a private, independent, nonprofit organization. The Millennium Scholarship Foundations mandate is: to focus on assisting Canadian students who demonstrate financial need and academic merit; to avoid duplication of existing financial aid programs and costly repetition of administrative procedures; to ensure fairness and equity in the delivery of its programs and resources.
The Foundation targets students who are motivated, but lack the financial resources to pursue higher education. The program strives to provide better access to postsecondary education to these students. The Foundation provides students an opportunity to learn, but also challenges them to make a difference in their schools and communities and develop long-term goals. Canada Foundation for Innovation An independent corporation, the Canadian Foundation for Innovation (CFI) was established by the Government of Canada in 1997 to fund research infrastructure. The CFIs main goal is to strengthen Canadian universities, colleges, research hospitals, and other non-profit institutions in the quality of research and technology development. The CFI has a budget of $3.65 billion and funds up to 40% of a projects infrastructure costs. These funds are invested in partnership with eligible institutions and their funding partners who provide the remaining 60% of a projects cost. Based on this, the total capital investment by the CFI, the research institutions and their partners, will be over $10 billion by 2010. Technology Partnerships Canada (TPC) Since 1996, Technology Partnerships Canada has been geared towards helping Canadian companies execute R&D that brings new technology closer to the marketplace. As a technology investment fund, TPC hopes to increase economic growth,create more jobs, and establish sustainable development. TPC invests in quality research and targets industries such as small and medium-sized businesses that are involved in many different types of technology. As of December 31, 2004, TPC has funded 673 projects which represent a total investment of over $2.7 billion, $1.97 billion of which had been disbursed. Out of these projects, 89% target small to mediumsized companies across Canada.
49
Industrial Research Assistance Program (IRAP) The National Research Councils Industrial Research Assistance Program (NRCIRAP) is a well known Canadian innovation assistance program available for small and medium-sized Canadian enterprises (SMEs). IRAP provides Canadian SMEs with valuable technological and business advice, financial assistance, and many more types of innovation assistance. Through technological innovation, NRC-IRAPs mandate is to generate wealth for Canada. This is accomplished by their assistance to SMEs through technology and innovation. Strategic objectives include the hope to increase the innovative capacity of Canadian SMEs and become the national enabler of technological innovation for Canadian SMEs. Six programs implement their mandate: IRAP-H (contributions to firms employing undergraduates), IRAP-L (contributions to laboratory investigations), IRAP-M (contributions to small projects), IRAP-P (contributions to large projects), IRAP-R (contributions to major projects involving technology transfer), IRAP-S (international technology services), (COOPER, 2003). NRC-IRAPs annual budget is approximately $150 million, 43% of which is for Nonrepayable contributions for R&D Activities. Natural Sciences and Engineering Research Council (NSERC) The Natural Sciences and Engineering Research Council of Canada invests in people, discovery, and innovation through grants, scholarships and awards. NSERC invests in more then 20,000 university students across Canada. The budget for university-based research and training in natural sciences and engineering in 20042005 was $850 million. NSERC hopes to promote Canadas discovery and innovation, and will do this by targeting investment in research and people. Social Sciences and Humanities Research Council (SSHRC) Created by an act of Parliament in 1977, the Social Sciences and Humanities Research Council of Canada (SSHRC) is a federal agency that develops and funds university-based research and training in social sciences and humanities. SSHRCs goal is to fund research that will stimulate innovative thinking about issues that affect Canadians in their everyday lives. SSHRCs budget for 20042005 is $230 million.
50
Canada Institutes of Health Research (CIHR) Canada Institutes of Health Research is Canadas leading health research funding agency, funding over 8,500 researchers across Canada. CIHR invests in researchers, since and training. CIHRs mandate is to excel internationally in scientific excellence, finding new knowledge to translate into improved health for Canadians. CIHR hopes to strengthen Canadas health research community as well as address health problems, focus on specific health issues such as disease prevention and cure, improve the health status of susceptible populations, and support health innovations. CIHRs budget is $662 million for 20042005, 70% of which is funding for research that is investigatordriven, and 30% of which is reserved for strategic ideas and programs. Canadian Institute of Advanced Research (CIAR) The Canadian Institute for Advanced Research (CIAR) was founded in 1982 to support basic research. The institutes mandate is to select leading researchers from all over the world to help solve complex problems in the sciences and social sciences. CIAR accomplishes this mandate by providing funds to enable collaboration between researchers through meetings and interaction. Pre-Competitive Advanced Research Networks (PRECARN) PRECARN Incorporated is a member-owned not-for-profit consortium funding research and collaboration between firms, universities, and government researchers. The main goal is to support development of intelligent systems technologies. Intelligent systems mimic human abilities in reasoning, decision-making, and adapting to change depending on their environment. These technologies include: sensors, robots, knowledge software and human-machine interfaces. With the application of these technologies it is expected that firms (especially SMEs) will increase their productivity and competitiveness. Networks of Centres of Excellence The Canadian Institutes of Health Research (CIHR), Natural Sciences and Engineering Research Council of Canada (NSERC), and Social Sciences and Humanities Research Council of Canada (SSHRC) in concert with Industry Canada manage the Networks of Centres of Excellence program. The Networks of Centres of Excellence program is a partnership between government and non-government
51
organizations, universities and industry. The goal is to encourage multidisciplinary research and facilitate the real-world application of academic research. The expected outcome of the program is to improve the lives of Canadians.
Table 3. Government program summary Institutional affiliation (Governance mechanism) Government of Canada Total budget ($mil.) Annual budget ($mil.)
Program Canada Research Chairs
Inception date
End date
Mandate & goals Strengthen research excellence
2000
2005
900
180 Improve training Improve universities ability to generate and apply new knowledge
Canada Millennium Scholarship Foundation
Private and autonomous 1998 Governed by a Board of Directors (15 members) none 2,500
Provide access to secondary education to economically disadvantaged students through financial aid
Canada Foundation for Innovation
Independent corporation 1997 none 3,650
Invest in infrastructure in universities, colleges, teaching hospitals, not-for-profit institutions Strengthen Canada's capacity for innovation Attract and retain researchers Training of young Canadians Promote networks, collaboration and multidisciplinary research Creates necessary conditions for sustainable long-term economic growth Commercialization of innovations, creation of spin-off ventures
Technology Partnerships Canada
Special operating agency of Industry Canada
TPC is a technology investment fund consisting of 673 projects $1.97 billion has been disbursed 597 (89%) projects target SMEs 1996 none 2,700
52
Table 3. (continued) Institutional affiliation (Governance mechanism) National Research Council Total budget ($mil.) Annual budget ($mil.)
Program Industrial Research Assistance Program
Inception date
End date
Mandate & goals Stimulate wealth creation in Canada through innovation in SMEs Increase innovation capacity of SMEs Focus on commercializing publicly funded R&D (collaboration between SMEs and research organizations)
1961
none
150
Natural Sciences and Engineering Research Council Social Sciences and Humanities Research Council
Government of Canada
1978 Government of Canada through a 22 member council Government of Canada
none
850
Funds basic and applied research, funding for 10,000 professors, 20,000 university students, encourage 500 firms to invest in university research Funds university research and training in the social sciences and humanities
1977
none
230 Funds 8,500 researchers in university, teaching hospitals, research institutes
Canada Institutes of Health Research (formerly Medical Research Council)
2000
none
662 Creation of new knowledge for more effective health services, products and improved health for Canadians Funds four main areas: biomedical, clinical, health systems and services, population and public health
Canadian Institute of Advanced Research
unknown
1982
none
25
Supports basic research, researchers selected by the agency to solve "complex problems in the sciences and social sciences" Funds only people, not infrastructure
53
Table 3. (continued) Institutional affiliation (Governance mechanism) Memberowned notfor-profit industrial consortium corporations, research institutes, government Total budget ($mil.) Annual budget ($mil.)
Program PreCompetitive Advanced Research Networks
Inception date
End date
1987 Networks of Centres of Excellence Industry Canada with assistance from NSERC, SSHRC, CIHR 1990
2010
20*
Mandate & goals To make Canadian firms more competitive through development and use of intelligent systems technologies especially for SMEs; to help firms improve productivity and competitiveness; funding supports collaboration between industry, universities, and government researchers to increase commercialization efforts and diffusion of new innovations A partnership between universities, industry, government and nongovernment organizations encourage multidisciplinary research, facilitate real-world application of academic research Ultimate goal is to improve the lives of Canadians
none
77
Note: c = continuously funded, = not applicable * The 2000 Federal budget gave $20 million for the Phase III program for the period 2000 to 2005. The 2005 budget gave another $20 million over five years (COOPER, 2005).
Table 3 summarizes the 11 programs discussed above. For the five programs which have a limited duration (i.e. Canada Research Chairs, Canada Millennium Scholarship Foundation, Canada Foundation for Innovation, Technology Partnerships Canada and Pre-Competitive Advanced Research Networks) the total funding base is $9.785 billion. For the six remaining programs with an ongoing, indefinite mandate (Industrial Research Assistance Program, NSERC, SSHRC, CIHR, Canadian Institute of Advanced Research and Networks of Centres of Excellence) the annual expenditures total $1.819 billion. The present worth of the annual expenditures for the ongoing programs, in contrast to the five limited duration programs, is approximately $38.295 billion.*
I am assuming here a real discount rate of 4.75%, and that the investment of $1.819 billion (in 2005 dollars) will occur indefinitely into the future. The relevant formula for the present worth under these assumptions is equal to the annual payout (A) divided by the discount rate (i) or A/i.
54
Literature on social and economic analysis of government programs In 1996, the Science and Technology Redesign Project (now a full-fledged division within Statistics Canada) began its mandate of developing better science and technology measures especially measures related to outputs from the R&D process (STATISTICS CANADA, 1998). Much of the groundwork for science and technology measurement emanated from the OECD. In particular, GODIN (1996) mentions five key OECD documents: the Frascatti Manual (1994) first published in 1963, Supplement on R&D in the Higher Education Sector (1989), Proposed Standard Practice for the Collection and Interpretation of Data on the Technological Balance of Payments (1990), Oslo Manual (1997), and the Technology-Economy-Productivity (TEP) Program (1992). At the time of writing MOWERY (1995) lamented that there was little in the way of literature on technology policy and that the theoretical literature on this topic is thin, however, an empirical literature based on the theory is almost non-existentrigorous empirical evaluations of technology policies are scarce. There are different levels of analysis: program level effects, university and/or firm-level effects and economy-wide effects (due to spillovers). PAVITT & WALKER (1976) point to the need for government funded basic research, since most industrial R&D activities focus on short-term, lowrisk innovations. They blame the use of conventional investment methods on this shortsighted behaviour. Thus they intone that governments in particular should fund longterm, risky R&D investment; they caution the use of conventional investment methods to analyze this sort of government program. Techniques for evaluation of R&D Policies A number of papers have recently appeared concerning science and technology indicators (GODIN, 1996), overall methods of evaluating publicly funded R&D (ARNOLD & BALAZS, 1998; KOSTOFF, 1996; GEISLER, 1994), economic methods concerned with evaluating public R&D (LINK, 1996; TASSEY, 1999, 2003; MCFETRIDGE, 1995; LIPSEY & CARLAW, 1998; SALTER & MARTIN, 2001), and bibliometric methods (NARIN & HAMILTON, 1996; GAUTHIER, 1998; VAN LEEUWEN et al., 2001). Only a few papers have appeared that specifically analyze Canadian policies towards R&D; those that have will be summarized below. (i) Net present value Net present value has its roots in corporate finance literature and in engineering economics (BREALEY & MYERS, 2002; PARK, 2001; FRASER et al., 2000).
55
The net present value of an investment is determined in the following manner:
NPV = C0 +
Bt
t
t =1 (1 + i)
(1)
Where C0 is the initial cost of the investment, Bt is the annual return from the investment (in this case we assume that B is a constant, but it can increase or decrease as well), t is time in years, and i is the relevant Minimum Acceptable Rate of Return (MARR). Usually the MARR is computed as the weighted average cost of capital for the investing firm or individual. Since the benefits are divided by the MARR for each period up to and including n, they are discounted. If the NPV of the investment is greater than zero then the firm or individual should make the investment. If it is less than zero then the investment should be rejected. If it equals zero then the investor is indifferent between investing and not investing since the return on the investment just equals the MARR. RANK & WILLIAMS (1999) used what they call a partial net present value approach to evaluate Canadian Networks of Centres of Excellence (Table 4). The approach was deemed partial, because the authors chose to concentrate on nine big winner projects as identified by interviews and surveys of program participants from a total of 41 projects and eight networks. Their results for the net present value of the partial program from 19891998 was $33.6 million (after deducting all relevant costs, and using a MARR of 8%).
(ii) Internal rate of return
The internal rate of return uses the same equation as (1), however, NPV is set to zero, and the equation is solved for the MARR as shown in equation (2). The decision rule for the investor depends on whether the internal rate of return exceeds the MARR, then the project should proceed, if it is less than the MARR then the project should be rejected: 0 = C0 +
n
Bt
t
t =1 (1 + i )
( solve for i )
(2)
In some respects the internal rate of return is a more useful evaluation technique than NPV because decision makers are better able to judge the rate of return with market rates of return such as Treasury Bill Yields, the prime lending rate, yields on corporate bonds of similar risk, and yields on stock market portfolios. Thus, it is easier to compare the internal rate of return amongst projects rather than NPV, as it is difficult to compare a portfolio of projects (with different initial costs, some may have distant benefits, others may have smaller benefits but in the immediate future) when the NPVs are all positive but significantly different from one another.
56
The major problem with the internal rate of return method occurs when complex cash flows occur which could cause the future net cash-flow stream to have multiple crossover points on the x-axis (or whichever axis is chosen for the interest rate). In such instances, the internal rate of return method will give multiple solutions and another more complicated approach (called the external rate of return) must be utilized. (iii) Benefit-cost analysis Benefit-cost analysis computes a ratio of discounted benefits to discounted costs. If the benefit cost ratio is greater than one then the project should proceed; if it is less than one then the project should be discarded. The relevant equation is below:
BCA =
t =1 (1 + i ) n Ct t =1 (1 + i )
Bt
t
(3)
t
The BCA equation above is actually just a rearrangement of the NPV equation shown in (1).* Thus the benefit-cost ratio is much like the NPV method in that it does not yield an easily comparable measure across alternative projects such as the internal rate of return. According to TASSEY (2003) it is inferior to the NPV and internal rate of return methods because a slight error or outlier in the benefits or costs can cause wide variations in the ratio. Another potential problem results because benefits and/or costs can be treated arbitrarily. Tassey cites two instances where a project fee was either treated as a cost (added to the denominator) or treated as a negative benefit (subtracted from the numerator) within his institution (National Institute of Standards and Technology). The two differing approaches yielded very different benefit-cost ratios for the two projects. (iv) Microeconomic analysis The method for analyzing the economic effects of R&D in microeconomics uses standard neoclassical welfare techniques. A supply and demand curve are estimated and a new supply curve results after the effects of the R&D policy have had an effect on
* Let N be the net present value, let C be costs and B be the benefits. Then we have N = C+B, dividing by C and re-arranging we have N/C1 = B/C. We call N/C1 the benefit-cost ratio, denoted as BCA.
57
product or process innovation.* The difference between the old and new producers surplus plus the difference between the old and new consumers surplus is the social value of the new product or process. MANSFIELD et al. (1977) used this method and found the consumer and producers surplus for 17 industrial innovations. They then solved for the internal rate of return the private rate of return for the innovating firm alone, and the social rate of return for all firms plus the innovating firm that utilized the new innovation in either their production process(es) or product(s). While on the surface this techniques seems straightforward, it is not. Determining the value of the innovation to other firms is quite difficult, not to mention time consuming. Mansfield et al. visited 17 firms and conducted in-depth interviews with numerous key employees; utilized as much accounting information as was feasible and/or authorized (for each year marketing, production, R&D, plant and equipment costs, and start-up costs for the innovation had to be calculated); and calculated rough estimates of the elasticity of demand through interviews. (v) Bibliometrics Scientometrics is the field of inquiry devoted to the measurement of scientific research. For instance, Canada had a 4% share of world scientific publications from 19891993 (BRAUN et al., 1995) and the United States is the worlds largest gatekeeper of scientific journal publications (BRAUN & DISPATONYI, 2005). Bibliometrics quantifies scientific output so that it is amenable to statistical analysis (GAUTHIER, 1998). Bibliometrics is a method for analyzing the importance of research publications by an author, or by many authors within a field. The method typically uses counts of publications or of citations (called literature bibliometrics) to measure the impact of an author or program on a particular field. However, two other categories are important: patent bibliometrics and linkage bibliometrics (NARIN & HAMILTON, 1996). Citations of papers and of patents and the linkage between patents and journal papers serve as measures of the quality of the research, but also highlights linkages between basic and applied research or between different fields or between technology in general and science (NARIN & HAMILTON, 1996). International co-authorships are an important measure of quality. Canadas 6% share of all internationally co-authored publications ranks fifth in the world behind Sweden, Italy, France and Germany (GAUTHIER, 1998).**
This procedure works for product or innovations that are first for the firm or nation i.e. for innovations of low to moderate importance. For product innovations of high importance i.e. a first in the world, the demand function does not initially exist. The sum of the producers and consumers surplus (once enough data has been amassed to calculate a demand elasticity) in this case will yield the total social welfare of the new product. ** Bibliometrics has its own set of unique problems; see GAUTHIER (1998) and MACROBERTS & MACROBERTS (1996).
*
58
The Science Citation Index from the Institute for Scientific Information provides cited references to 3,700 science and technical journals, if the computerized version Web of Science is used the coverage increases to 5,800 journals (ISI, 2005). The search engine is quite powerful in that literature can be tracked both forward and backward from any cited article. The Current Impact Index is used to analyze the citation patterns in the current year of an organizations previous patents (five years previous, moving wall) (NARIN & HAMILTON, 1996). (vi) Econometric studies Governments have been funding R&D for a long time. For example, LINK (1996) traced government funding back to the U.S. Navys research in 1789, and the Department of Agricultures establishment of land grant universities stemming from the Morrill Act of 1862. Link summarized the major features of economic studies that evaluate government R&D funding. In particular, econometric studies typically follow the same approach: a multi-input, multi-output production function is first hypothesized. If the function is homothetic and weakly separable in inputs, the separable function is additive, and the output vector is replaced by a composite good index, then the equation becomes:
q = A(t ) f ( x)
(4)
Where q is the composite output index, A(t) is current technology (it is neutral and disembodied), x is a vector of inputs into the production process. Total factor productivity (TFP) can be derived from (4) as the ratio of output to all inputs: TFP = A(t ) = q f ( x) (5)
Technological change is defined as the percentage change in TFP over time. The dTFP change in TFP over time is TFP' = so therefore, the percentage change in TFP is: dt
TFP' A(t )' A' = = TFP A(t ) A (6)
If we assume that the production function is Cobb-Douglas (a standard assumption), and we take the natural logarithm and differentiate with respect to time we obtain an equation of the following form:
TFP' OT ' GT ' = + + + TFP OT GT
(7)
Where is disembodied technology, OT is the firms own technology from internal R&D, GT is government technology which comes from accumulated government-
59
subsidized research, and is the error term. Finally, Link points out that can be combined with OT/OT, and can be combined with GT/GT to yield an estimating equation:
SFR GSR TFP' = 0 + 1 q + 2 q + TFP
(8)
Where SFR is self-financed research and GSR is government-sponsored research. The intercept term 0 is disembodied growth, and 2 is really the coefficient of interest. LINKs (1981) own research using a collection of 51 manufacturing firms, shows that the coefficient on basic government research is highly significant but its magnitude is about half of the coefficient on self-financed basic research.* In a review paper by DAVID et al. (2000) the authors report that there are at least four different types of econometric analyses used to assess the effects of government policy on private R&D: cross-section studies of firms with different levels of government R&D support, panel studies at the firm-level wherein unobserved heterogeneity between firms can be controlled for (the data also reveal the firms performance over time with respect to its level of government support), macroeconomic studies that measure changes in the economys performance relative to government funding of R&D, and studies that use instrumental variables that attempt to control for simultaneity between private and public R&D funds. Their specific objective in their review of the literature was to determine whether government funding of R&D substitutes or complements private R&D funding. Their findings were mixed, but one result was clear at higher levels of aggregation (from the firm to the industry to the macroeconomy) more tests revealed complementarity between the two forms of R&D funding. There are issues with micro-level studies that examine the effects of government R&D funding because of selection bias. For example, LICHTENBERG (1984) said: federal contracts do not descend upon firms like manna from heaven (p. 74). Indeed, firms are selected for federal funding because of their existing commitment to R&D and innovation. Macroeconomic studies on the other hand, can capture the indirect effects of government R&D policies. Perhaps this is one reason for the differences in empirical results found in David et al.s literature review. GRILICHES (1992) completed a thorough literature review of studies attempting to measure social returns to public research. His results show that on average the social returns to public R&D are 41% (for the agriculture industry).Detailed results from his survey are included in Table 4.
A related productivity measurement can be found with Data Envelope Analysis wherein the best performers exhibit minimum costs while achieving highest performance (i.e. minimal R&D expenditure producing high productivity such as patents, market share or profitability). See GRUPP (1993) for an example.
60
Table 4. Empirical results of program evaluation Author(s) MANSFIELD et al. (1977) Program under investigation 17 industrial innovations across 17 industries Method Social Rate of Return, Private Rate of Return Results SRR(median) = 56%, PRR(median) = 25%
MANSFIELD (1965)
TASSEY (1999)
effects of R&D expenditure for 10 manufacturing industries effect of NIST measurement and standards lab projects
Private Rate of Return
PRR(min) = 3%, PRR(max) = 99%
BCA, Social Rate of Return
DAVID et al. (2000)
whether government support for R&D substitutes or complements private R&D support
literature survey of econometric studies
BERNSTEIN (1997)
measurement of spillovers from R&D for the electrical and electronic products industries in Canada
GRILICHES (1992)
measurement of returns to government R&D for agriculture
econometric estimation of a cost function Private Rate of Return Social Rate of Return Spillover Return econometric analysis to calculate Social Rates of Return
Photonics SRR=145%, BCA=13:1 Machine tool software SRR=99%, BCA=118:1 Thermocouples SRR=32%, BCA=3:1 Radio pharmaceuticals SRR=138%, BCA=97:1 Alternative refrigerants SRR=433%, BCA=4:1 Advanced ceramics SRR=33%, BCA=10:1 Cross-section studies 14 of which 7 found complementarities Panel studies 4 of which 1 found complementarities Industry-level studies 5 of which 4 found complementarities Aggregate studies 7 of which 6 found complementarities Electrical products PRR=17.2%, SRR=110.02%, Spillover Rate of Return=92.82%
Griliches (1958) hybrid corn SRR=3540% hybrid sorghum SRR=20% Peterson (1967) poultry SRR=21-25% Schmitz-Seckler (1970) tomato harvester SRR=37-46% Griliches (1964) aggregate SRR=3540% Evenson (1968) aggregate SRR=4150% Knutson-Tweeten (1979) aggregate SRR=28-47% Huffman-Evenson (1991) crops SRR=45-62% livestock SRR=11-83% aggregate SRR=43-67%
61
Table 4. (continued) Author(s) LINK (1981) Program under investigation impact of government financed basic research, government financed applied research Method econometric estimation of a production function he dependent variable was average annual rate of change in total factor productivity from 19731978 data was for 51 major manufacturing firms Data Envelope Analysis, patents, publications, interviews Results coefficients on self-financed applied research and government funded applied research were not significant government funded basic research was significant the coefficient on self-financed basic research was significant and about twice as large as the coefficient on government funded basic research
GRUPP (1993)
impact of government intervention in telecommunicatio ns R&D in 10 countries
Sweden (followed by the Netherlands, Germany, Japan Italy, France, UK and USA) is most efficient in terms of converting R&D expenditures into export market shares countries with highly deregulated telecom industries (i.e. USA and UK) appear to perform the worst in terms of export shares IRAP met 16 out of the 19 success criteria
LIPSEY & CARLAW (1998)
IRAP
RANK & WILLIAMS (1999)
Canadian Networks of Centres of Excellence
tested to see the number out of 19 criteria of success that were followed partial net present value analysis from 1989 to 1998
case study benefits of 9 big winners from the NCE program net of implementation and production costs yielded a net present value of $611.9 million (constant 1996 dollars) total NCE program and partner research costs of $578.3 million (constant 1996 dollars) lower bound estimate of the net present value for the whole NCE program = 611.9 578.3 = $33.6 million minimum acceptable rate of return used in the analysis=8%
MANSFIELD (1991) evaluated the social returns for university research for the preceding 15 years. He found that 11 percent of new products and 9 percent of new processes could not have been developed without substantial delay in the absence of academic
62
research. His calculated social rate of return for university research was 28 percent. The overall finding of the research was that academic R&D was widely diffused, and that it was difficult to measure the links between academic research and industrial innovation. Nonetheless, the results were convincing that the contribution of academic research in pharmaceuticals, instruments and information technology has been considerable. BERNSTEIN (1997) produced estimates of the private rate of return, social rate of return and spillover rate of return for 10 Canadian manufacturing industries. Estimates of the social rate of return range from a low of 85.8% in the food and beverages industry, to a high of 191.27% in the petroleum products industry. Spillover rates of return range from a low of 68.6% in the food and beverages industry to a high of 174.07% in the petroleum products industry. It was found that spillovers from the electrical products industry affected eight out of the nine remaining industries (food and beverages, fabricated metals, non-electrical machinery, non-metallic minerals, paper and allied products, petroleum products, primary metals and transportation equipment) by reducing their production costs. The second effect of spillovers from the electrical products industry was capital enhancing (increasing physical capital intensity) and/or increasing R&D intensity. Proposed methods of analysis for Canadian programs The mandate and goals of programs may or may not lead to easily measured outcomes. Here the outcome is meant to be the ultimate effect of the program on society or the economy in general. It should be relatively clear to the reader by now, that measurement of any outcomes related to government sponsored research, development, commercialization, infrastructure etc. should be multidimensional; thus the process will be time-consuming, expensive (in terms of data collection) and fraught with methodological pitfalls. This is borne out by the testimonials of RANK & WILLIAMS (1999) who analyzed Canadian Networks of Centres of Excellence and by LIPSEY & CARLAW (1998) in their analysis of the Defence Industry Productivity Program, four programs designed to enhance technological change Industrial Research and Development Incentives Act (enacted 1966), Program for the Advancement of Industrial Technology (enacted 1965), Enterprise Development Program (enacted 1972), Industrial and Regional Development Program (enacted in 1983), and NRCs Industrial Research Assistance Program (enacted 1961). In Table 5 I have grouped the 11 programs into four main categories under the heading Type. The following list tries to give the reader an idea of which procedures might be beneficial for each category: Infrastructure or capital acquisition programs benefits derived from new infrastructure are typically assumed to occur because of embodied technical change meaning that new capital equipment contains new technology thus making it more
63
productive than earlier generations of similar equipment. Infrastructure and capital acquisition lend themselves to NPV, benefit-cost analysis and IRR analysis. However, it would be difficult to estimate societal effects using corporate finance measures. To arrive at social benefits, the most logical method to use is TFP analysis. Either an index number or econometric approach would work depending on the data constraints. Capital stocks would be subject to depreciation (in use, as well as over time) and an annual rental value for the capital stock would have to be computed (using some imaginative techniques) before the index construction or econometric estimation. Academic programs relevant indicators are the number of graduate students who are trained, other tangibles such as the number of publications derived from the grant, the number and quality of citations (bibliometrics), and the number of patents (patents would normally only be relevant for the natural and medical sciences, not the social sciences and humanities). Spin-off companies from university research are another important measure of success. Intangibles include the following enhancing the reputation of the researcher, the prestige bestowed upon the university by housing such grants and producing quality graduates. Scholarships bright students who would otherwise not be able to obtain an education are granted access through a scholarship. One of the benefits of an educated populace is an expanded tax-base due to education i.e. the recipients obtain better jobs as an immediate consequence of their education. But, that is probably not a sufficient indicator of program success, although the number of students who become gainfully employed after graduation is. Or the number of students who successfully enroll in higher education is important. Furthermore, completion rates and the fields of study may be useful metrics of program performance. Other characteristics of an educated populace relate to many intangibles (well-being for instance) or even technological change could increase because of increased numbers of highly educated workers however, these effects are in the distant future and are again very hard to quantify. Commercialization and/or Development programs the difference in sales and/or profit before and after the grant/loan. IRR of the new projects; their NPV; econometric estimation of firm-level or industry-level effects of the program. Data: A way forward In order to perform an econometric analysis of the firm-level effects of R&D programs, data on R&D spending both private and public are required at the firm level. Some measure of growth in sales or profit or labour productivity is also required to test the effects of the program on firm performance. If Statistics Canadas R&D surveys were linked together this sort of analysis could be accomplished. I do not know of a survey at Statistics Canada that contains information explicitly on government program type and level of funding at the firm level. Of course, depending on the needs of the
64
evaluating agency (which is of paramount importance in any evaluation exercise!), oneoff surveys would need to be administered for many of the programs listed in Table 5.
Table 5. Proposed methods of analysis Program Canada Research Chairs Type Academic Relevant method of analysis Bibliometric analysis (number of papers, paper citations, international co-authorships) Patent citations Spin-offs Number of students educated Increase in tax base Intangibles non-measurable Economic analysis IRR, NPV or econometric study Economic analysis IRR, NPV or econometric study Spin-offs Spillovers Economic analysis IRR, NPV or econometric study Spin-offs Bibliometric analysis Patent citations Spin-offs Bibliometric analysis Patent citations Spin-offs Bibliometric analysis Patent citations Spin-offs Bibliometric analysis Patent citations Economic analysis IRR, NPV or econometric study Spin-offs Spillovers Economic analysis IRR, NPV or econometric study Spin-offs Spillovers
Canada Millennium Scholarship Foundation Canada Foundation for Innovation Technology Partnerships Canada
Scholarship
Infrastructure DevelopmentCommercialization
Industrial Research Assistance Program NSERC
Infrastructure, DevelopmentCommercialization Academic
SSHRC
Academic
CIHR
Academic
Canadian Institute of Advanced Research Pre-Competitive Advanced Research Networks
Academic DevelopmentCommercialization
Networks of Centres of Excellence
DevelopmentCommercialization
Below are my brief thoughts on what could possibly be done with some existing surveys currently residing within Statistics Canadas walls.
65
R&D survey STATISTICS CANADAs (1997) Research and Development in Canadian Industry is a multi-year ongoing survey of Canadas R&D performing firms. Below I have listed the relevant variables for economic analysis. Canadian Federal Government R&D grants, Industry Canada Defence Industry Productivity Program, Industry Canada Technology Partnerships Program, National Research Council Industrial Research Assistance Program, other grant programs Economic variables in the R&D survey: revenue, percent of revenue generated by new or substantially improved products/services by the firm in the previous three-year period, number and education of scientists and engineers (Bachelors, Masters, Doctoral), expenditures for R&D broken down by: wages and salaries, other current costs, capital expenditures including land, buildings, equipment and other.
Survey of Electronic Commerce Up until now I have not discussed the importance of spin-off companies that are created with the help of government R&D or commercialization funding. The Survey of Electronic Commerce and Technology (STATISTICS CANADA, 1999, 2000, 2003) asks three questions pertaining to technology transfer for all types of technologies and products. The first question asks: Over the past 3 years, has the firm licensed technologies from any of the following? The binary (yes or no) responses include a Canadian university, a Canadian hospital, and/or a federal government of Canada laboratory. The second question asks: Did the acquisition of technology from any of the following sources play a major role in the firms inception or growth over the past 3 years? The responses are the same as for the first question. The third question asks whether the firm is a spin-off from a Canadian university. BORDT & EARL (2004) found that technology transfer was important for 4,400 firms out of the 21,000 firms sampled for the 2003 Survey of Electronic Commerce and Technology. They also discovered that 1,350 firms generated spin-offs. If the SECT were linked to the R&D survey we could undertake econometric studies of the effect of the Technology Partnerships Program and IRAP (possibly others depending on how the Other grants programs boxes are filled in).
66
Biotechnology survey The Biotechnology Use and Development Survey (STATISTICS CANADA, 1999, 2001, 2003) asks whether the firm (or more properly the employee answering the survey) is a spin-off from a university/hospital, another company, or a government agency/lab (question 13). Question 22 of the 2001 Biotechnology Use and Development Survey asks the same question except the response categories are a spin-off from: a university/hospital, another biotech company, a non-biotech firm, or a corporate agency or lab. In addition to a question about spin-offs, the biotechnology surveys (mentioned above) ask questions about external sources of information (spillovers) that include federal research organizations, federal information programs and research consortia. If the biotechnology surveys could be linked to the R&D survey we could undertake econometric studies of the effect of the Technology Partnerships Program and IRAP (possibly others depending on how the Other grants programs boxes are filled in). Intellectual property commercialization survey The Survey of Intellectual Property Commercialization in the Higher Education Sector (STATISTICS CANADA, 1998, 2000, 2001) also asks a question about the number and type of spin-off firms from university related research. Since SIPCHES targets institutions of higher education it would be relatively easy to collect data on the funding level of NSERC, SSHRC and CIHR for each university and college in Canada. The survey asks whether the institution has federal government research contracts as well. The levels of tri-council funding should theoretically influence the number of spin-offs, patent applications, licences granted and royalties received for each educational institution. Innovation survey Using the Oslo definition of innovation the Survey of Innovation (Statistics Canada, 1999, 2003) asks questions on whether the firm had product and or process innovations that were world-first or Canada-first. The 2003 survey asks how many new or significantly improved products and/or processes the firm introduced in the three years previous to the survey. Thus, if the innovation survey could be linked to the R&D survey it would be possible to correlate innovation outputs to R&D inputs including IRAP and Technology Partnerships in an econometric analysis.
67
Conclusion This paper has briefly summarized technology policy, Canadian programs directed at R&D, commercialization, academic research, education, and the analysis of such programs. In the previous section I proposed some preliminary methods for the evaluation of eleven of Canadas technology programs. Evaluation of even one of these programs will be an arduous task. I suspect the government knows this, since NRC has conducted only two evaluations in recent memory (HOLBROOK, 2000); LIPSEY & CARLAW (1998) evaluated six programs, one of which was IRAP; finally, RANK & WILLIAMS (1999) did a partial net present value analysis of Networks of Centres of Excellence. Despite the difficulties, it is surprising given the commitment of the government in terms of funding, that so few evaluations of Canadas programs have been performed. I caution the reader of this paper on a number of fronts: first of all, the methods that I suggest for evaluation of the eleven programs are tentative. My initial thoughts on the subject are contained herein, and they will undoubtedly be revised (perhaps after a comment to the journal has been published!). Second, much has been written on the topic of technology policy and its evaluation. However, there is no academic consensus as to how benefits either social or economic should be measured. Therefore, before any substantive empirical work could begin, consultations with government, industry and academia must take place. I would suggest a roundtable discussion with key government individuals who could discuss and agree upon appropriate measures. Afterwards, consultations with industry and universities could proceed, wherein their comments on the proposed methods and measures would be taken into consideration. *
I am grateful for the financial support provided by Marketplace Innovation Division, Industry Canada who also commissioned the report on which this paper is based. Id like to express my gratitude to Denys Cooper, IRAP, National Research Council for providing essential information on IRAP and other government programs. The Editor, Tibor Braun and the anonymous reviewers gave constructive comments that improved the paper. Finally, thanks to Paisley Cozzarin for her invaluable research assistance.
References
ARROW, K. (1962), Economic welfare and the allocation of resources for invention, In: The Rate and Direction of Inventive Activity: Economic and Social Factors, Princeton University Press, pp. 609625. ARTHUR, W. B. (1989), Competing technologies, increasing returns, and lock-in by historical events, Economic Journal, 99 (394) : 116131. ARTHUR, W. B. (1994), Inductive reasoning and bounded rationality, American Economic Review, 84 (2) : 406411. BAUMOL, W. J. (1992), Horizontal collusion and innovation, Economic Journal, 102 (410) : 129137.
68
BERNSTEIN, J. (1997), Interindustry R&D spillovers for electrical and electronic products: The Canadian case, Economic Systems Research, 9 (1) : 111126. BORDT, M., L. EARL (2004), Public sector technology transfer in Canada, 2003, Statistics Canada, Science, Innovation and Electronic Information Division Working Papers, Catalogue no. 88F0006XIE No. 018. BRAUN, T., W. GLNZEL, H. Grupp (1995), The scientometric weight of 50 nations in 27 science areas, 19891993. Part I. All fields combined, mathematics, engineering, chemistry and physics, Scientometrics, 33 (3) : 263293. BRAUN, T., I. DISPATONYI (2005), The journal gatekeepers of major publishing houses of core science journals, Scientometrics, 64 (2) : 113120. BREALEY, R. A., S. C. MYERS (2002), Brealey & Myers on Corporate Finance: Capital Investment and Valuation, McGraw-Hill. COHEN, W., R. LEVINTHAL (1989), Innovation and learning: The two faces of R&D, Economic Journal, 99 : 569596. COHEN, W., R. LEVINTHAL (1990), Absorptive capacity: A new perspective on learning and innovation, Administrative Science Quarterly, 35 (1) : 128152. COOPER, D. (2003 and 2005), Industrial Research Assistance Program, National Research Council Canada, 1200 Montreal Road, Ottawa, Personal Communication. DAVID, P. A., B. H. HALL, A. A. TOOLE (2000), Is public R&D a complement or substitute for private R&D? A review of the econometeric evidence, Research Policy, 29 : 497529. DEBONDT, R. (1996), Spillovers and innovative activities, International Journal of Industrial Organization, 15 (1) : 128. DEPARTMENT OF FINANCE (1997), Why and How Governments Support Research and Development,
http://www.fin.gc.ca/resdev/why1_e.html
DIXIT, A. K., J. E. STIGLITZ (1977), Monopolistic competition and optimum product diversity, American Economic Review, 67 : 297308. FRASER, N., I. BERNHARDT, E. JEWKES, M. TAJIMA (2000), Engineering Economics in Canada, 2nd ed., Prentice-Hall Canada. FREEMAN, C. (1994), The economics of technical change, Cambridge Journal of Economics, 18 : 463514. FREEMAN, C., L. SOETE (1997), Aspects of public policy for science, technology and innovation, In: Economics of Industrial Innovation, 3rd ed., MIT Press, pp. 373395. GAUTHIER, E. (1998), Bibliometric Analysis of Scientific and Technological Research: A Users Guide to the Methodology, Science and Technology Redesign Project, Statistics Canada, September, 88F0006XPB No. 8, ST-98-08. GEISLER, E. (1994), Key output indicators in performance evaluation of research and development organizations, Technological Forecasting and Social Change, 47 : 189203. GEROSKI, P. A. (1992), Vertical relations between firms and industrial policy, Economic Journal, 102 (410) : 138147. GODIN, B. (1996), The State of Science and Technology Indicators in the OECD Countries, Statistics Canada, Science and Technology Redesign Project, Research Paper. GOVERNMENT OF CANADA (2001), Achieving Excellence: Investing in People, Knowledge and Opportunity. Communications and Marketing Branch, Industry Canada.
http://innovation.gc.ca/gol/innovation/site.nsf/vDownload/Page_PDF/$file/achieving.pdf
GRILICHES, Z. (1992), The search for R&D spillovers, Scandinavian Journal of Economics, 94 (Supplement) : 2947. GRUPP, H. (1993), Efficiency of government intervention in technical change in telecommunications: Ten national economies compared, Technovation, 13 (4) : 187220. GUNDERSON, M. (1974), The case of government supported training program, Industrial Relations Industrielles, 29 : 709723. HALL, P. (Ed.) (1986), Technology, Innovation and Public Policy. P. Allen, Oxford. HENDERSON, R., I. COCKBURN (1993), Scale, Scope and Spillovers: The Determinants of Research Productivity in the Pharmaceutical Industry, Working Paper No. 4466, National Bureau of Economic Research, September.
69
HOLBROOK, A. (2000), Evaluation of research sponsored by federal granting councils in Canada: The social contract, Research Evaluation, 8 (1) : 4756. INDUSTRY CANADA (2005), Innovation, Science and Technology, and Research.
www.ic.gc.ca/cmb/welcomeic.nsf/ICPages/SubjectInnovation
ISI (2005), Science Citation Index, http://www.isinet.com/products/citation/sci/ JAFFE, A. B. (1989), Real effects of academic research, American Economic Review, 79 (5) : 957970. KEYNES, J. M. (1936), The General Theory of Employment, Interest and Money, Harcourt, Brace and Company. KOSTOFF, R. N. (1996), Performance measures for government-sponsored research: Overview and background, Scientometrics, 36 (3) : 281292. LICHTENBERG, F. R. (1984), The relationship between federal contract R&D and company R&D, American Economic Review, 74 (2) : 7378. LINK, A. N. (1981), Basic research and productivity increase in manufacturing: Additional evidence, American Economic Review, 71 (5) : 11111112. LINK, A. N. (1996), Economic performance measures for evaluating government-sponsored research, Scientometrics, 36 (3) : 325342. LINK, A. N., P. KAPUR (1994), A note on the diffusion of flexible manufacturing systems technology, Economics Letters, 46 : 369374. LIPSEY, R. G., K. CARLAW (1998), A Structuralist Assessment of Technology Policies Taking Schumpeter Seriously on Policy, Working Paper Number 25, October 1998, Industry Canada Research Publications Program. MACROBERTS, M. H., B. R. MACROBERTS (1996), Problems of citation analysis, Scientometrics, 36 (3) : 435444. MALERBA, F. (1992), Learning by firms and incremental change, Economic Journal, 102 : 845859. MANSFIELD, E. (1980), Basic research and productivity increase in manufacturing, American Economic Review, 70 (5) : 863873. MANSFIELD, E. (1989), The diffusion of industrial robots in Japan and the United States, Research Policy, 18 : 183192. MANSFIELD, E. (1991), Social returns from R&D: Findings, methods and limitations, Research Technology Management, 34 (6) : 2427. MANSFIELD, E., J. RAPPOPORT, A. ROMEO, S. WAGNER, G. BEARDSLEY (1977), The Production and Application of New Industrial Technology, New York, Norton. METCALFE, J. S. (1981), Impulse and diffusion in the study of technical change, Futures, 13 (5) : 345359. METCALFE, J. S. (1988), The Diffusion of Innovations: An Interpretive Survey, In: DOSI, G. et al. (Eds). METCALFE, J. S. (1994), Evolutionary economics and technology policy, Economic Journal, 104 : 931944. MOHNEN, P. (1992), The Relationship between R&D and Productivity Growth in Canada and Other Major Industrialized Countries, Economic Council of Canada, Ottawa: Canada Communications Group. MOWERY, D. (1995), The practice of technology policy, In: P. STONEMAN (Ed.), Handbook of the Economics of Technological Change, Blackwell Publishers, pp. 513544. NARIN, F., K. S. HAMILTON (1996), Bibliometric performance measures, Scientometrics, 36 (3) : 293310. NELSON, R. R. (1961), Uncertainty, learning and the economics of parallel research and development efforts, Review of Economics and Statistics, 43 (4) : 351364 NELSON, R. R. (1981), Research on productivity growth and productivity differences: Dead ends and new departures, Journal of Economic Literature, 19 (3) : 10291064. NELSON, R. R. (1993), A retrospective, In: NELSON (Ed.), National Innovation Systems: A Comparative Analysis, Oxford University Press, pp. 503523. NELSON, R. R. (1995), Why should managers be thinking about technology policy? Strategic Management Journal, 16 : 581588. NORDHAUS, W. D. (1969), Economic theory of technological change, American Economic Review, 59 (2) : 1828. OECD (1989), R&D Statistics and Output Measurement in the Higher Education Sector. Paris, France : Organisation for Economic Co-operation and Development.
70
OECD (1990), Main Science and Technology Indicators. Paris, France: Organisation for Economic Cooperation and Development. OECD (1992), Technology-Economy-Productivity Programme (TEP). Paris, France: Organisation for Economic Co-operation and Development. OECD (1994), Proposed Standard Practice for Surveys of Research and Experimental Development (Frascatti Manual). Paris, France: Organisation for Economic Co-operation and Development. OECD (1997), Proposed Guidelines for Collecting and Interpreting Technological Innovation Data (Oslo Manual). Paris, France: Organisation for Economic Co-operation and Development. PARK, C. S. (2001), Contemporary Engineering Economics, 3/E, Prentice Hall PAVITT, K. (1984), Sectoral patterns of technical change Towards a taxonomy and a theory, Research Policy, 13 (6) : 343373. PAVITT, K., W. WALKER (1976), Government policies towards industrial innovation: A review, Research Policy, 5 : 1197. PECK, M. J. (1986), Joint research and development: The case of microelectronics and computer technology corporation, Research Policy, 15 (5) : 219231. PORTER, M. (1990), The Competitive Advantage of Nations, New York: The Free Press. RANK, D., D. WILLIAMS (1999), Partial benefit/cost in the evaluation of the Canadian networks of centres of excellence, Evaluation and Program Planning, 22 : 121129. SALTER, A. J., B. R. MARTIN (2001), The economic benefits of publicly funded basic research: A critical review, Research Policy, 30 : 509532. SCHMOOKLER, J. (1959), Bigness, fewness and research, Journal of Political Economy, 67 (6) : 628632. STATISTICS CANADA (1997), Research and Development in Canadian Industry, Science and Technology Redesign Project. http://www.statcan.ca/english/concepts/science.htm STATISTICS CANADA (1998, 2000, 2001), Survey of Intellectual Property Commercialization in the Higher Education Sector, Science, Innovation and Electronic Information Division.
http://www.statcan.ca/english/concepts/science.htm
STATISTICS CANADA (1999, 2000, 2003), Electronic Commerce and Technology Survey, Science, Innovation and Electronic Information Division.
STATISTICS CANADA (1999, 2003), Survey of Innovation, Innovation and Electronic Information Division.
STATISTICS CANADA (1999, 2001, 2003), Biotechnology Use and Development Survey, Science, Innovation and Electronic Information Division. http://www.statcan.ca/english/concepts/science.htm STONEMAN, P. (1987), The Economic Analysis of Technology Policy. Oxford University Press. STONEMAN, P. (1991), The use of a Levy Grant system as an alternative to tax based incentives to research and development, Research Policy, 20 (3) : 195201. TASSEY, G. (1999), Lessons learned about the methodology of economic impact studies: The NIST experience, Evaluation and Program Planning, 22 : 113119. TASSEY, G. (2003), Methods for Assessing the Economic Impacts of Government R&D, U.S. Department of Commerce, National Institute of Standards and Technology, Program Office, Strategic Planning and Economic Analysis Group, September, Planning Report 03-1.
http://www.nist.gov/director/prog-ofc/report03-1.pdf
TEECE, D. J. (1996), Firm organization, industrial structure, and technological innovation, Journal of Economic Behavior & Organization, 31 : 193224. THOMSON SCIENTIFIC (2005), Web of Science, http://scientific.thomson.com/products/wos/ U.S. National Cooperative Research Act (1984), United States Code, Title 15, Chapter 69 Cooperative Research. http://www.law.cornell.edu/uscode/html/uscode15/usc_sup_01_15_10_69.html UTTERBACK, J. M. (1974), Innovation in industry and the diffusion of technology, Science, 183 : 620626. UTTERBACK, J. M., W. J. ABERNATHY (1975), A dynamic model of process and product innovation, OmegaInternational Journal of Management Science, 3 (6) : 639656. VAN LEEUWEN, T. N., L. J. VAN DER WURFF, A. F. J. VAN RAAN (2001), The use of combined bibliometric methods in research funding policy, Research Evaluation, 10 (3) : 195201. VON HIPPEL, E. (1995), The Sources of Innovation, New York, NY: Oxford University Press.
71

Performance Measures For The Socio-Economic Impact

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Performance Measures For The Socio-Economic Impact

Uploaded by

Copyright:

Available Formats

Jointly published by Akadmiai Kiad, Budapest and Springer, Dordrecht

Scientometrics, Vol. 68, No. 1 (2006) 4171

Performance measures for the socio-economic impact of government spending on R&D

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

Program Canada Research Chairs

Mandate & goals Strengthen research excellence

Canada Millennium Scholarship Foundation

Canada Foundation for Innovation

Independent corporation 1997 none 3,650

Technology Partnerships Canada

Special operating agency of Industry Canada

B. P. COZZARIN: Government spending on R&D

Program Industrial Research Assistance Program

1978 Government of Canada through a 22 member council Government of Canada

230 Funds 8,500 researchers in university, teaching hospitals, research institutes

Canada Institutes of Health Research (formerly Medical Research Council)

Canadian Institute of Advanced Research

B. P. COZZARIN: Government spending on R&D

Program PreCompetitive Advanced Research Networks

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

The net present value of an investment is determined in the following manner:

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

TFP' OT ' GT ' = + + + TFP OT GT

B. P. COZZARIN: Government spending on R&D

SFR GSR TFP' = 0 + 1 q + 2 q + TFP

B. P. COZZARIN: Government spending on R&D

Private Rate of Return

PRR(min) = 3%, PRR(max) = 99%

BCA, Social Rate of Return

DAVID et al. (2000)

literature survey of econometric studies

measurement of returns to government R&D for agriculture

B. P. COZZARIN: Government spending on R&D

impact of government intervention in telecommunicatio ns R&D in 10 countries

LIPSEY & CARLAW (1998)

RANK & WILLIAMS (1999)

Canadian Networks of Centres of Excellence

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

Industrial Research Assistance Program NSERC

Infrastructure, DevelopmentCommercialization Academic

Canadian Institute of Advanced Research Pre-Competitive Advanced Research Networks

Networks of Centres of Excellence

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D

B. P. COZZARIN: Government spending on R&D