New Indicators of the Technological Impact of Scientific Production

doi:10.2478/jdis-2021-0028

Abstract

Abstract:

Purpose: Building upon pioneering work by Francis Narin and others, a new methodological approach to assessing the technological impact of scientific research is presented.
Design/methodology/approach: It is based on the analysis of citations made in patent families included in the PATSTAT database that is to scientific papers indexed in Scopus.
Findings: An advanced citation matching procedure is applied to the data in order to construct two indicators of technological impact: on the citing (patent) side, the country/region in which protection is sought and a patent family’s propensity to cite scientific papers are taken into account, and on the cited (paper) side, a relative citation rate is defined for patent citations to papers that is similar to the scientific paper-to-paper citation rate in classical bibliometrics.
Research limitations: The results are limited by the available data, in our case Scopus and PATSTAT, and especially by the lack of standardization of references in patents. This required a matching procedure that is neither trivial nor exact.
Practical implications: Results at the country/region, document type, and publication age levels are presented. The country/region-level results in particular reveal features that have remained hidden in analyses of straight counts. Especially notable is that the rankings of some Asian countries/regions move upwards when the proposed normalized indicator of technological impact is applied as against the case with straight counts of patent citations to those countries/regions’ published papers.
Originality/value: In our opinion, the level of sophistication of the indicators proposed in the current paper is unparalleled in the scientific literature, and provides a solid basis for the assessment of the technological impact of scientific research in countries/regions and institutions.

Key words: Technological Impact, Patents, Scientometric, PATSTAT, Scopus

P. Guerrero-Bote, Vicente, Henk F. Moed, and Félix Moya-Anegón. "New Indicators of the Technological Impact of Scientific Production." Journal of Data and Information Science, vol.6, no.4, 2021, pp.36-61. DOI: 10.2478/jdis-2021-0028

Figures/Tables 13

Table 1

Patent citation based indicators by designated country/region.

Designated country/region	# Patent Families	Families with cites to papers (%)	# Pat Fam cites to papers	Avg cites to papers per Pat Fam	Weighted avg GDP share	Avg cites to papers per Pat Fam with ≥1 cites	Pat Fam with filing year ≥2003 (%)	Pat Fam cites to papers (%)	Weighted cites to papers (%)
United States	5400857	11.65	3533076	0.65	0.234	33.29	22.47	81.15	45.91
China	8571905	4.24	1463274	0.17	0.108	33.12	35.66	33.61	12.50
Germany	2408775	15.79	2158516	0.90	0.052	35.02	10.02	49.58	6.18
Japan	5096385	4.65	1410294	0.28	0.079	38.61	21.20	32.39	5.88
United Kingdom	2037468	18.02	2136675	1.05	0.041	35.47	8.48	49.08	4.72
France	1887911	19.15	2105069	1.12	0.040	35.66	7.85	48.35	4.47
Italy	1832796	19.28	2085714	1.14	0.031	35.94	7.62	47.91	3.49
Spain	1762192	20.08	2086719	1.18	0.021	35.92	7.33	47.93	2.28
Netherlands	1753318	20.02	2078575	1.19	0.013	36.03	7.29	47.74	1.37
Turkey	1742792	20.05	2074734	1.19	0.011	36.08	7.25	47.66	1.22
Canada	513166	25.89	1111076	2.17	0.024	47.31	2.13	25.52	0.96
Switzerland	1745305	20.05	2076016	1.19	0.009	36.06	7.26	47.69	0.95
Sweden	1748215	20.01	2075724	1.19	0.008	36.07	7.27	47.68	0.82
Belgium	1732148	20.18	2075083	1.20	0.007	36.08	7.21	47.66	0.78
South Korea	2348083	5.96	815412	0.35	0.018	41.86	9.77	18.73	0.78
Poland	1707430	20.20	2059367	1.21	0.007	36.21	7.10	47.30	0.74
Austria	1742992	20.07	2075891	1.19	0.006	36.07	7.25	47.68	0.65
Norway	1264904	22.10	1772202	1.40	0.006	38.36	5.26	40.71	0.55
Denmark	1740152	20.09	2076218	1.19	0.005	36.08	7.24	47.69	0.52
Brazil	296790	20.85	478260	1.61	0.029	47.00	1.23	10.99	0.52
Australia	446278	21.88	857725	1.92	0.018	49.69	1.86	19.70	0.52
Greece	1731130	20.19	2074796	1.20	0.004	36.08	7.20	47.66	0.45
Finland	1741247	20.10	2075883	1.19	0.004	36.07	7.24	47.68	0.41
Ireland	1730187	20.20	2074865	1.20	0.004	36.08	7.20	47.66	0.41
Portugal	1728495	20.23	2076124	1.20	0.003	36.08	7.19	47.69	0.37
Czech Republic	1734030	20.18	2075923	1.20	0.003	36.06	7.21	47.68	0.32
Russian Federation	534368	7.92	275369	0.52	0.025	43.57	2.22	6.33	0.31
Romania	1733753	20.15	2074321	1.20	0.003	36.09	7.21	47.65	0.27

Table 1

Table 2

Table 3

Table 4

Table 5

Figure 1.

Table 6

Table 7

Figure 2.

Figure 3.

Table 8

Figure 4.

Figure 5.

References 39

[1]	Ahmadpoor, M., & Jones, B.F. (2017). The dual frontier: Patented inventions and prior scientific advance. Science, 357(6351), 583-587. doi: 10.1126/science.aam9527 pmid: 28798128
[2]	Archambault, E., Campbell, D., Gingras, Y., & Larivière, V. (2009). “Comparing bibliometric statistics obtained from the Web of Science and Scopus”. Journal of the American Society for Information Science and Technology, 60 (7), 1320-1326. https://doi.org/10.1002/asi.21062 doi: 10.1002/asi.v60:7
[3]	Bryan, K.A., Ozcan, Y., & Sampat, B. (2020). In-text patent citations: A user’s guide. Research Policy, 49(4), 103946.
[4]	Callaert, J., Pellens, M., & Van Looy, B. (2014). Sources of inspiration? Making sense of scientific references in patents. Scientometrics, 98(3), 1617-1629. doi: 10.1007/s11192-013-1073-x
[5]	Coffano, M., & Tarasconi, G. (2014). Crios—Patstat database: Sources, contents and access rules. Center for Research on Innovation, Organization and Strategy, Crios Working Paper n. 1. https://ssrn.com/abstract=2404344 (accessed: 01/06/2020) . https://doi.org/10.2139/ssrn.2404344
[6]	Derwent. (2007) World Patents Index—Derwent patentee codes, Revised edition 8. Thomson Corporation. Leuven Manual. ISBN: 0 901157 38 4. http://ips.clarivate.com/m/pdfs/mgr/patenteecodes.pdf
[7]	European Patent, Office. (2019). Data catalog Patstat global. Versión 5.13. EPO Patstat customers. https://www.epo.org
[8]	Gorraiz, J., Gumpenberger, C., & Wieland, M. (2011). “Galton 2011 revisited: A bibliometric journey in the footprints of a universal genius”. Scientometrics, 88(2), 627-652. https://doi.org/10.1007/s11192-011-0393-y doi: 10.1007/s11192-011-0393-y
[9]	Guerrero-Bote, V.P., & Moya-Anegón, F. (2015). “Analysis of scientific production in food science from 2003 to 2013”. Journal of food science, 88(12), R2619-R2626. https://doi.org/10.1111/1750-3841.13108
[10]	Guerrero-Bote, V.P., Sánchez-Jiménez, R., & Moya-Anegón, F. (2019). “The citation from patents to scientific output revisited: A new approach to Patstat / Scopus matching”. El profesional de la información, 28(4), e280401. https://doi.org/10.3145/epi.2019.jul.01
[11]	Hane, P.J. (2004). “Elsevier announces Scopus service”. Information today. http://newsbreaks.infotoday.com/nbreader.asp?ArticleID=16494 (accessed: 01/06/2020).
[12]	Hinze, S., & Schmoch, U. (2004). Opening the black box. In: Moed, H.F., Glänzel, W., and Schmoch, U. (2004) (eds.). Handbook of quantitative science and technology research. The use of publication and patent statistics in studies of S&T systems. Dordrecht (the Netherlands): Kluwer Academic Publisher, 215-235.
[13]	Jacsó, P. (2011). “The h-index, h-core citation rate and the bibliometric profile of the Scopus database”. Online information review, 35(3), 492-501. https://doi.org/10.1108/14684521111151487 doi: 10.1108/14684521111151487
[14]	Jefferson, O.A., Jaffe, A., Ashton, D., Warren, B., Koellhofer, D., Dulleck, U., Bilder, G., Ballagh, A., Moe, J., DiCuccio, M., Ward, K., Bilder, G., Dolby, K., & Jefferson, R.A. (2018). “Mapping the global influence of published research on industry and innovation”. Nature biotechnology, 36(1), 31-39. https://doi.org/10.1038/nbt0818-772a doi: 10.1038/nbt.4049 pmid: 29319684
[15]	Kang, B., & Tarasconi, G. (2016). “Patstat revisited: Suggestions for better usage”. World patent information, 46, 56-63. https://doi.org/10.1016/j.wpi.2016.06.001 doi: 10.1016/j.wpi.2016.06.001
[16]	Leydesdorff, L., Moya Anegón, F., & Guerrero-Bote, V.P. (2010). “Journal maps on the basis of Scopus data: A comparison with the Journal Citation Reports of the ISI”. Journal of the American Society for Information Science and Technology, 61(2), 352-369. https://doi.org/10.1002/asi.21250
[17]	Lissoni, F. (2012). “Academic patenting in Europe: An overview of recent research and new perspectives”. World patent information, 34(3), 197-205. https://doi.org/10.1016/j.wpi.2012.03.002 doi: 10.1016/j.wpi.2012.03.002
[18]	Lotti, F., & Marin, G. (2013). “Matching of Patstat applications to AIDA firms: Discussion of the methodology and results”. Bank of Italy occasional paper, n. 166. https://ssrn.com/abstract=2283111 (accessed: 01/06/2020) . https://doi.org/10.2139/ssrn.2283111
[19]	Lundberg, J. (2007). “Lifting the crown—citation z-score”. Journal of Informetrics, 1(2), 145-154. https://doi.org/10.1016/j.joi.2006.09.007 doi: 10.1016/j.joi.2006.09.007
[20]	Magerman, T., Van-Looy, B., & Song, X.Y. (2006). Data production methods for harmonized patent statistics: Patentee name standardization. Technical report, K.U. Leuven. https://ec.europa.eu/eurostat/documents/3888793/5836029/KS-AV-06-002-EN.PDF (accessed: 01/06/2020).
[21]	Maraut, S., & Martínez, C. (2014). “Identifying author-inventors from Spain: Methods and a first insight into results”. Scientometrics, 101(1), 445-476. https://doi.org/10.1007/s11192-014-1409-1 doi: 10.1007/s11192-014-1409-1
[22]	Martínez, C. (2011). “Patent families: When do different definitions really matter?”, Scientometrics, v. 86, pp. 39-63. https://doi.org/10.1007/s11192-010-0251-3
[23]	Moya-Anegón, F., Chinchilla-Rodríguez, Z., Vargas-Quesada, B., Corera-Álvarez, E., Muñoz-Fernández, F.J., González-Molina, A., & Herrero-Solana, V. (2007). “Coverage analysis of Scopus: A journal metric approach”. Scientometrics, 73(1), 53-78. https://doi.org/10.1007/s11192-007-1681-4 doi: 10.1007/s11192-007-1681-4
[24]	Moya-Anegón, F., Guerrero-Bote, V.P., López-Illescas, C., & Moed, H.F. (2018). “Statistical relationships between corresponding authorship, international co-authorship and citation impact of national research systems”. Journal of informetrics, 12(4), 1251-1262. https://doi.org/10.1016/j.joi.2018.10.004 doi: 10.1016/j.joi.2018.10.004
[25]	Moya-Anegón, F., López-Illescas, C., Guerrero-Bote, V.P., & Moed, H.F. (2020). The citation impact of social sciences and humanities upon patentable technology. Scientometrics, to be published.
[26]	Narin, F., & Olivastro, D. (1992). Status report: Linkage between technology and science. Research Policy, 21, 237-249. doi: 10.1016/0048-7333(92)90018-Y
[27]	Narin, F., Hamilton, K.S., & Olivastro, D. (1997). The increasing linkage between US technology and public science. Research Policy, 26, 317-330. doi: 10.1016/S0048-7333(97)00013-9
[28]	Organisation for Economic Co-operation and Development. (2009). OECD Patent Statistics Manual, OECD, Paris. https://www.oecd.org/sti/inno/oecdpatentstatisticsmanual.htm
[29]	Pickering, B. (2004). “Elsevier prepares Scopus to rival ISI Web of science”. Information world review, n. 8.
[30]	Poege, F., Harhoff, D., Gaessler, F., & Baruffaldi, S. (2019). Science quality and the value of inventions. Science advances, 5(12), easy7323.
[31]	Price, D.J.D (1965). Networks of scientific papers. Science, 149, 510-515. doi: 10.1126/science.149.3683.510
[32]	Raffo, J.D., & Lhuillery, S. (2009). “How to play the ‘Names game’: Patent retrieval comparing different heuristics”. Research policy, 38(10), 1617-1627. https://doi.org/10.2139/ssrn.1441172 doi: 10.1016/j.respol.2009.08.001
[33]	Schmoch, U., & Khan, M. (2019). Methodological challenges for creating patent indicators. In: Glanzel, W., Moed, H.F., Schmoch, U., & Thelwall, M. (eds.), Springer Handbook of Science and Technology Indicators. Cham (Switzerland): Springer Natur, 907-927.
[34]	Schoen, A., Heinisch, D., & Buenstorf, G. (2014). “Playing the ‘Name game’ to identify academic patents in Germany”. Scientometrics, 101(1), 527-545. https://doi.org/10.1007/s11192-014-1400-x doi: 10.1007/s11192-014-1400-x
[35]	SCImago Research, Group. (2010). SIR Methodology: General Considerations. https://www.scimagoir.com/methodology.php (accessed: 01/06/2020).
[36]	Thoma, G., & Torrisi, S. (2007). Creating powerful indicators for innovation studies with approximate matching algorithms. A test based on Patstat and Amadeus databases (No. 211). KITeS, Centre for Knowledge, Internationalization and Technology Studies, Università Bocconi, Milano, Italy. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.573.8107&rep=rep1&type=pdf (accessed: 01/06/2020).
[37]	Van Raan, A.F.J. (2017). Patent Citations Analysis and Its Value in Research Evaluation: A Review and a New Approach to Map Technology-relevant Research. Journal of Data and Information Science, 2, 13-50. doi: 10.1515/jdis-2017-0002
[38]	Veugelers, R., & Wang, J. (2019). Scientific novelty and technological impact. Research Policy, 48(6), 1362-1372. doi: 10.1016/j.respol.2019.01.019
[39]	Waltman, L., van Eck, N.J., van Leeuwen, T.N., Visser, M.S., & van Raan, A.F. (2011). “Towards a new crown indicator: Some theoretical considerations”. Journal of informetrics, 5(1), 37-47. https://doi.org/10.1016/j.joi.2010.08.001 doi: 10.1016/j.joi.2010.08.001

Document type	# Patent families	Avg cites to papers	Avg refs in patents	Avg cites to papers (GDP-based weighting)	Avg cites to papers (fractional GDP-based weighting)
Article	670831	0.008	38.9	0.020	0.0023
Conference Paper	340577	0.018	17.4	0.025	0.0048
Review	177172	0.012	47.1	0.018	0.0019
Short Survey	24580	0.020	48.1	0.018	0.0013

Year	# Patent families	Avg cites to papers	Avg refs in patents	Avg cites to papers (GDP-based weighting)	Avg cites to papers (fractional GDP-based weighting)
2003	230308	0.019	31.49	0.0218	0.00400
2004	232905	0.019	33.77	0.0212	0.00350
2005	229644	0.019	35.01	0.0208	0.00318
2006	214822	0.019	35.43	0.0203	0.00292
2007	205747	0.019	37.53	0.0202	0.00278
2008	192682	0.019	39.03	0.0201	0.00261
2009	177198	0.019	39.74	0.0195	0.00255
2010	160483	0.019	40.16	0.0193	0.00249
2011	140461	0.019	41.07	0.0191	0.00244
2012	116753	0.018	42.90	0.0189	0.00233
2013	87978	0.016	46.07	0.0188	0.00207
2014	61586	0.014	42.73	0.0186	0.00178
2015	38225	0.011	39.68	0.0185	0.00134
2016	18082	0.006	42.64	0.0189	0.00064
2017	6046	0.002	42.45	0.0202	0.00022
2018	806	0.000	38.90	0.0191	0.00003

Scientific Area	# Patent families	Avg cites to papers	Avg refs in patents	Avg cites to papers (GDP-based weighting)	Avg cites to papers (fractional GDP-based weighting)
Computer Science	251649	0.019	16.10	0.025	0.00525
Engineering	394749	0.018	21.55	0.024	0.00466
Materials Science	228670	0.018	23.88	0.024	0.00428
Physics and Astronomy	204802	0.017	24.03	0.025	0.00401
Energy	56034	0.020	15.61	0.022	0.00395
Mathematics	111652	0.021	18.10	0.024	0.00386
Chemistry	241128	0.015	27.28	0.020	0.00350
Chemical Engineering	154459	0.017	33.27	0.020	0.00309
Decision Sciences	11859	0.021	13.79	0.028	0.00280
Health Professions	21004	0.017	29.71	0.021	0.00247
Environmental Science	51047	0.015	24.88	0.020	0.00230
Pharmacology, Toxicology and Pharmaceutics	120068	0.011	40.29	0.018	0.00227
Agricultural and Biological Sciences	78973	0.012	34.36	0.018	0.00193
Dentistry	4080	0.011	28.90	0.019	0.00192
Biochemistry, Genetics and Molecular Biology	271492	0.007	45.62	0.018	0.00187
Earth and Planetary Sciences	19357	0.009	15.67	0.022	0.00172
Medicine	270703	0.006	46.07	0.018	0.00164
Multidisciplinary	67184	0.017	55.33	0.019	0.00161
Immunology and Microbiology	94414	0.009	46.12	0.018	0.00160
Business, Management and Accounting	10229	0.010	11.89	0.030	0.00154
Veterinary	10619	0.012	32.96	0.018	0.00153
Neuroscience	43846	0.011	55.51	0.019	0.00141
Nursing	13688	0.011	54.48	0.018	0.00132
Arts and Humanities	37896	0.015	50.43	0.021	0.00128
Social Sciences	22800	0.007	19.65	0.025	0.00100
Economics, Econometrics and Finance	2044	0.004	11.76	0.034	0.00062
Psychology	6026	0.006	63.15	0.021	0.00059

Country/Region	%Families	%Output	%BF	%Exc10	%Exc1	%Fam.Cit.	%TFO
United States	22.47	24.25	36.37	37.76	46.45	43.74	36.88
China	35.66	14.05	11.48	13.09	13.58	6.68	14.32
Germany	10.02	6.13	8.32	8.71	10.05	7.93	8.31
Japan	21.20	5.21	4.80	4.59	4.10	7.64	7.83
United Kingdom	8.48	7.05	10.85	11.23	14.03	7.19	6.50
France	7.85	4.30	5.57	5.86	6.56	4.59	4.74
South Korea	9.77	2.50	2.53	2.71	2.65	3.51	4.56
Canada	2.13	3.66	5.50	5.76	7.16	4.25	4.40
Italy	7.62	3.65	5.01	5.26	5.82	3.27	3.29
India	0.23	3.80	3.06	2.89	2.52	1.53	2.74
Netherlands	7.29	2.02	3.55	3.90	5.22	2.64	2.56
Spain	7.33	2.97	3.71	3.98	4.41	2.19	2.49
Switzerland	7.26	1.50	2.70	2.91	4.11	2.48	2.48
Australia	1.86	2.97	4.52	4.83	6.05	2.22	2.36
Chinese Taiwan	2.74	1.44	1.43	1.58	1.34	1.59	2.19
Sweden	7.27	1.33	2.16	2.31	2.93	1.70	1.65
Belgium	7.21	1.12	1.81	1.94	2.58	1.47	1.48
Singapore	0.31	0.66	1.11	1.25	1.79	0.97	1.33
Denmark	7.24	0.83	1.49	1.59	2.18	1.03	1.12
Austria	7.25	0.82	1.23	1.27	1.63	1.05	1.10
Israel	0.32	0.75	1.10	1.15	1.41	1.18	1.03
Finland	7.24	0.69	1.06	1.10	1.33	0.83	1.03
Chinese Hong Kong	0.30	0.69	1.13	1.31	1.71	0.79	1.00
Brazil	1.23	2.14	1.84	1.66	1.55	0.75	0.92
Russian Federation	2.22	2.12	1.36	1.06	1.04	0.60	0.78
Poland	7.10	1.41	1.23	1.07	1.15	0.59	0.76
Norway	5.26	0.67	1.08	1.08	1.37	0.51	0.72
Iran	0.00	1.28	1.16	1.28	1.17	0.33	0.65

	%Families	%Output	%BF	%Exc10	%Exc1	%Fam.Cit.	%TFO
%Families	1	0.72	0.60	0.61	0.58	0.55	0.68
%Output	0.72	1	0.97	0.97	0.95	0.92	0.987
%BF	0.60	0.97	1	0.99	0.99	0.98	0.98
%Exc10	0.61	0.97	0.99	1	0.99	0.97	0.98
%Exc1	0.58	0.95	0.99	0.99	1	0.97	0.97
%Fam.Cit.	0.54	0.92	0.98	0.97	0.97	1	0.97
%TFO	0.68	0.98	0.98	0.98	0.97	0.97	1