Can Crossref Citations Replace Web of Science for Research Evaluation? The Share of Open Citations

doi:10.2478/jdis-2020-0037

Abstract

Abstract:

Purpose: We study the proportion of Web of Science (WoS) citation links that are represented in the Crossref Open Citation Index (COCI), with the possible aim of using COCI in research evaluation instead of the WoS, if the level of coverage was sufficient.

Design/methodology/approach: We calculate the proportion on citation links where both publications have a WoS accession number and a DOI simultaneously, and where the cited publications have had at least one author from our institution, the Czech Technical University in Prague. We attempt to look up each such citation link in COCI.

Findings: We find that 53.7% of WoS citation links are present in the COCI. The proportion varies largely by discipline. The total figures differ significantly from 40% in the large-scale study by Van Eck, Waltman, Larivière, and Sugimoto (blog 2018, https://www.cwts.nl/blog?article=n-r2s234).

Research limitations: The sample does not cover all science areas uniformly; it is heavily focused on Engineering and Technology, and only some disciplines of Natural Sciences are present. However, this reflects the real scientific orientation and publication profile of our institution.

Practical implications: The current level of coverage is not sufficient for the WoS to be replaced by COCI for research evaluation.

Originality/value: The present study illustrates a COCI vs WoS comparison on the scale of a larger technical university in Central Europe.

Key words: Open citations, Crossref Open Citation Index, Web of Science, Current Research Information System

Chudlarský, Tomáš, and Jan Dvořák. "Can Crossref Citations Replace Web of Science for Research Evaluation? The Share of Open Citations." Journal of Data and Information Science, vol.5, no.4, 2020, pp.35-42. DOI: 10.2478/jdis-2020-0037

Figures/Tables 5

Figure 1.

Table 1

Figure 2.

Table 2

Table 3

References 10

[1]	La Bruzzo, S., Manghi, P., & Mannocci, A., , (2019). OpenAIRE’s DOIBoost—Boosting Crossref for Research. In: Manghi, P., Candela, L., Silvello, G. (eds) Digital Libraries: Supporting Open Science. IRCDL 2019. Communications in Computer and Information Science, vol 988. Springer, Cham, DOI 10.1007/978-3-030-11226-4_11
[2]	Chudlarský, T., & Dvořák, J. (2019). Can Crossref Citations Replace Web of Science for Research Evaluation? The Share of Open Citations. [poster abstract] In Proceedings of the 17th Conference of the International Society for Scientometrics and Informetrics.(pp. 2551-2552). Rome, Italy: Edizioni Efesto. ISBN 978-88-3381-118-5.
[3]	Crossref. Retrieved from https://www.crossref.org
[4]	Dvořák, J., Chudlarský, T., & Špaček, J. (2019). Practical CRIS Interoperability. In 14th International Conference on Current Research Information Systems: Practical CRIS Interoperability. Amsterdam: Elsevier B.V., pp. 256-264. Procedia Computer Science. vol. 146. ISSN 1877-0509. DOI 10.1016/j.procs.2019.01.077
[5]	van Eck, N.J., Waltman, L., Larivière, V., & Sugimoto, C. (2018). Crossref as a new source of citation data: A comparison with Web of Science and Scopus. Retrieved from https://www.cwts.nl/blog?article=n-r2s234
[6]	Fraumann, G., & Waltman, L. (2019). The 2019 Workshop on Open Scientometric Data Infrastructures at Leiden University. Retrieved from https://www.cwts.nl/blog?article=n-r2x274&title=the-2019-workshop-on-open-scientometric-data-infrastructures-at-leiden-university
[7]	ISSI (2017). Open citations: A letter from the scientometric community to scholarly publishers. Retrieved from http://issi-society.org/open-citations-letter/
[8]	OpenCitations. (2018). Open Citation Indexes: COCI, the OpenCitations Index of Crossref open DOI-to-DOI references. Dump, the “Citation data (CSV)” file. DOI 10.6084/m9.figshare. 6741422.v3
[9]	Shotton, D. (2017). Milestone for I4OC—open references at Crossref exceed 50%. Retrieved from https://opencitations.wordpress.com/2017/11/24/milestone-for-i4oc-open-references-at-crossref-exceed-50/
[10]	The DOI Handbook. (2012). The DOI Foundation, 2012. DOI 10.1000/182

Faculty or University Institute	WoS publications	WoS citations	Of which in COCI	Coverage
Institute of Experimental and Applied Physics	1,122	24,348	15,225	62.5%
Faculty of Nuclear Sciences and Physical Engineering	4,225	54,470	32,398	59.5%
Faculty of Transportation Sciences	567	15,830	9,329	58.9%
Faculty of Mechanical Engineering	1,778	26,114	14,999	57.4%
Czech Technical University (whole)	12,796	90,675	48,707	53.7%
Faculty of Electrical Engineering	3,959	16,726	7,768	46.4%
Faculty of Biomedical Engineering	478	2,050	950	46.3%
Czech Institute of Informatics, Robotics and Cybernetics	219	459	191	41.6%
Faculty of Civil Engineering	1,727	7,131	2,539	35.6%
University Centre of Energy Efficient Buildings	114	232	72	31.0%
Klokner Institute	126	255	78	30.6%
Faculty of Information Technology	347	654	185	28.3%

Field ( / Subfield)	WoS publications	WoS citations	Of which in COCI	Coverage
- Physical sciences / Astronomy (including astrophysics, space science)	117	1,028	803	78.1%
- Physical sciences / Fluids and plasma physics (including surface physics)	521	2,444	1,552	63.5%
- Physical sciences / Particles and field physics	1,426	35,838	22,320	62.3%
Physical sciences (whole)	4,307	57,877	35,152	60.7%
- Physical sciences / Nuclear physics	868	12,604	7,585	60.2%
- Physical sciences / Other	788	3,810	2,187	57.4%
Biological sciences	114	991	545	55.0%
Czech Technical University (whole)	12,796	90,675	48,707	53.7%
Clinical medicine	131	652	316	48.5%
Chemical sciences	200	1,083	524	48.4%
Earth and related environmental sciences	252	1,468	711	48.4%
Electrical engineering, Electronic engineering, Information engineering	2,834	10,523	4,951	47.0%
Mathematics	820	2,303	942	40.9%
- Physical sciences / Optics (including laser optics and quantum optics)	590	2,253	789	35.0%
Computer and information sciences	1,000	3,097	1,071	34.6%
Materials engineering	745	4,184	1,404	33.6%
Mechanical engineering	542	1,562	516	33.0%
Environmental engineering	223	617	200	32.4%
Civil engineering	942	2,555	740	29.0%
Medical engineering	103	177	38	21.5%

Discipline	WoS publications	WoS citations	Of which in COCI	Coverage
Astronomy, Celestial Mechanics, Astrophysics	114	1,025	803	78.3%
Plasma and Gas Discharge Physics	376	1,986	1,389	69.9%
Theoretical Physics	375	1,957	1,353	69.1%
Elementary Particles and High Energy Physics	1,398	35,792	22,308	62.3%
Nuclear, Atomic and Molecular Physics, Colliders	934	12,720	7,635	60.0%
Czech Technical University (whole)	12,796	90,675	48,707	53.7%
Nuclear & Quantum Chemistry	101	463	241	52.1%
Sensors, Measurement, Regulation	377	1,139	572	50.2%
Computer Applications, Robotics	530	3,807	1,868	49.1%
Solid Matter Physics & Magnetism	294	1,466	670	45.7%
Electronics & Optoelectronics, Electrical Engineering	1,385	3,149	1,416	45.0%
Computer Hardware & Software	636	2,611	1,168	44.7%
General Mathematics	730	1,993	802	40.2%
Other Materials	152	977	355	36.3%
Fluid Dynamics	161	489	172	35.2%
Optics, Masers, Lasers	584	2,247	787	35.0%
Control Systems Theory	324	1,528	528	34.6%
Informatics, Computer Science	577	1,362	462	33.9%
Non-nuclear Energetics, Energy Consumption & Use	212	535	175	32.7%
Composite Materials	281	2,000	641	32.0%
Civil Engineering	633	1,761	521	29.6%
Building Engineering	256	682	195	28.6%
Metallurgy	166	658	188	28.6%
Nuclear Energetics	119	247	62	25.1%