Exploring the Potentialities of Automatic Extraction of University Webometric Information

doi:10.2478/jdis-2020-0040

Abstract

Abstract:

Purpose: The main objective of this work is to show the potentialities of recently developed approaches for automatic knowledge extraction directly from the universities’ websites. The information automatically extracted can be potentially updated with a frequency higher than once per year, and be safe from manipulations or misinterpretations. Moreover, this approach allows us flexibility in collecting indicators about the efficiency of universities’ websites and their effectiveness in disseminating key contents. These new indicators can complement traditional indicators of scientific research (e.g. number of articles and number of citations) and teaching (e.g. number of students and graduates) by introducing further dimensions to allow new insights for “profiling” the analyzed universities.

Design/methodology/approach: Webometrics relies on web mining methods and techniques to perform quantitative analyses of the web. This study implements an advanced application of the webometric approach, exploiting all the three categories of web mining: web content mining; web structure mining; web usage mining. The information to compute our indicators has been extracted from the universities’ websites by using web scraping and text mining techniques. The scraped information has been stored in a NoSQL DB according to a semi-structured form to allow for retrieving information efficiently by text mining techniques. This provides increased flexibility in the design of new indicators, opening the door to new types of analyses. Some data have also been collected by means of batch interrogations of search engines (Bing, www.bing.com) or from a leading provider of Web analytics (SimilarWeb, http://www.similarweb.com). The information extracted from the Web has been combined with the University structural information taken from the European Tertiary Education Register (https://eter.joanneum.at/#/home), a database collecting information on Higher Education Institutions (HEIs) at European level. All the above was used to perform a clusterization of 79 Italian universities based on structural and digital indicators.

Findings: The main findings of this study concern the evaluation of the potential in digitalization of universities, in particular by presenting techniques for the automatic extraction of information from the web to build indicators of quality and impact of universities’ websites. These indicators can complement traditional indicators and can be used to identify groups of universities with common features using clustering techniques working with the above indicators.

Research limitations: The results reported in this study refers to Italian universities only, but the approach could be extended to other university systems abroad.

Practical implications: The approach proposed in this study and its illustration on Italian universities show the usefulness of recently introduced automatic data extraction and web scraping approaches and its practical relevance for characterizing and profiling the activities of universities on the basis of their websites. The approach could be applied to other university systems.

Originality/value: This work applies for the first time to university websites some recently introduced techniques for automatic knowledge extraction based on web scraping, optical character recognition and nontrivial text mining operations (Bruni & Bianchi, 2020).

Key words: Development of data and information services, Webometrics indicators, Higher education institutions, Automatic extraction, Machine learning, Optimization

Bianchi, Gianpiero, Renato Bruni, Cinzia Daraio, Antonio Laureti Palma, Giulio Perani, and Francesco Scalfati. "Exploring the Potentialities of Automatic Extraction of University Webometric Information." Journal of Data and Information Science, vol.5, no.4, 2020, pp.43-55. DOI: 10.2478/jdis-2020-0040

Figures/Tables 4

Table 1

Selected indicators to be used for profiling Italian universities’ websites.

No.	Indicator name	Source	Description	Rationale
1.	Access to research	Scraping and mining	Number of research articles / log(number of professor)	Measures the ability to provide access to publications produced by the university, slightly normalized by the size of the institution
2.	Access to content	Scraping and mining	Number of pdf, ppt, doc, rtf, ps / log(number of professor)	Measures the ability to provide consents, slightly normalized by the size of the institution
3.	Orientation to external collaborations	Scraping and mining	Number of research institutions (IT+EU) and research-oriented companies mentioned in the website/ log(number of professor)	Measures the ability of the website to provide a comprehensive description of the extent of on-going research (or Third-mission) collaborations, slightly normalized by the size of the institution
4.	Access to information on teaching	Scraping and mining	Percentage of teachers providing their emails	Measures the possibility to easily get in touch with professors
5.	Visibility	Search engine	Number of HTML pages pointing to the university website from external domains / log(number of professor)	Measures how the university website is visible from outside, slightly normalized by the size of the institution
6.	Usability	Analytics	Percentage of contacts from mobile devices	Level of use by the mobile-oriented audience (largely including students).
7.	Relevance	Analytics	(1/national ranking by visitors) / log(number of students)	Websites’ popularity at the national level, slightly normalized by the size of the institution
8.	Intensity of use	Analytics	average time spent on the website / number of pages visited	A key indicator of website effectiveness: the more time is spent on the website, the more relevant will be the available contents
9.	International orientation	Analytics	Percentage of foreign contacts	Popularity abroad as a condition to attract customers (incl. students) and partners
10.	Direct access	Analytics	Percentage of direct accesses	Percentage of non-casual visitors as an indicator of popularity and ability to connect to a population of regular users

Table 1

Table 2

Figure 1.

Figure 2.

References 26

[1]	Aggarwal, C.C. (2018). Machine learning for text. Springer.
[2]	Aguillo, I.F., Ortega, J.L., & Fernández, M. (2008). Webometric ranking of world universities: Introduction, methodology, and future developments. Higher education in Europe, 33(2-3), 233-244. doi: 10.1080/03797720802254031
[3]	Aguillo, I.F., & Orduna-Malea, E. (2013) The Ranking Web and the “World-Class” Universities: New Webometric Indicators Based on G-Factor, Interlinking, and Web 2.0 Tools. In book: Building World-Class Universities pp. 197-217. doi: 10.1007/978-94-6209-034-7_13
[4]	Bar-Ilan, J. (2019). Data Collection from the Web for Informetric Purposes. In Springer Handbook of Science and Technology Indicators (pp. 781-800). Springer, Cham.
[5]	Bianchi, G., R., Bruni, & F., Scalfati, (2018). Identifying e-Commerce in Enterprises by means of Text Mining and Classification Algorithms. Mathematical Problems in Engineering, 7231920.1-7231920.8. doi: 10.1155/2017/2696108 pmid: 29578548
[6]	Bianchi, G., Bruni, R., Laureti Palma, A., Perani, G., & Scalfati, F. (2019). The corporate identity of Italian Universities on the Web: a webometrics approach. In the Proceedings of the 2019 ISSI Conference ISSI (pp. 2273-2278).
[7]	Björneborn, L., & Ingwersen, P. (2004). Toward a basic framework for webometrics. Journal of the American Society for Information Science and Technology, 55(14), 1216-1227. doi: 10.1002/(ISSN)1532-2890
[8]	Bruni, R., & Bianchi, G. (2015). Effective classification using a small training set based on discretization and statistical analysis. IEEE Transactions on Knowledge and Data Engineering, 27(9), 2349-2361. doi: 10.1109/TKDE.2015.2416727
[9]	Bruni, R., Bianchi, G., Dolente, C., & Leporelli, C. (2019). Logical Analysis of Data as a Tool for the Analysis of Probabilistic Discrete Choice Behavior. Computers & Operations Research, 106, 191-201.
[10]	Bruni, R., & Bianchi, G. (2020). Website categorization: A formal approach and robustness analysis in the case of e-commerce detection. Expert Systems with Applications, 142, 113001. doi: 10.1016/j.eswa.2019.113001
[11]	Bychkova1, M.N., & Okushova, G.A. (2017). Methods of analysis of a modern university’s presence in the Internet communicative space. AI & Society, 32, 89-100.
[12]	Daraio, C., & Bonaccorsi, A. (2017). Beyond university rankings? Generating new indicators on universities by linking data in open platforms. Journal of the Association for Information Science and Technology, 68(2), 508-529. doi: 10.1002/asi.2017.68.issue-2
[13]	Daraio, C., & Glänzel, W. (2016). Grand challenges in data integration—State of the art and future perspectives: An introduction. Scientometrics, 108(1), 391-400. doi: 10.1007/s11192-016-1914-5
[14]	Daraio, C., Bruni, R., Catalano, G., Daraio, A., Matteucci, G., .. , Scannapieco, M & Lepori, B. (2020). A tailor-made data quality approach for higher educational data. Journal of Data and Information Science, 5(3), 129-160. doi: 10.2478/jdis-2020-0029
[15]	Elgohary, A.E. (2008). Arab universities on the web: A webometric study. Electronic Library, 26(3), 374-386. doi: 10.1108/02640470810879518
[16]	Glänzel, W., Moed, H.F., Schmoch, U., & Thelwall, M. (2019). Springer Handbook of Science and Technology Indicators. Springer Nature. doi: 10.1038/nature01416 pmid: 12594512
[17]	Göransson, B., & Brundenius, C. (2010). Universities in transition: The changing role and challenges for academic institutions. Springer Science & Business Media.
[18]	Islam, M.A., & Alam, M.S. (2011). Webometric study of private universities in Bangladesh. Malaysian Journal of Library and Information Science, 16(2), 115-126.
[19]	McCoy, C.G., Nelson, M.L., & Weigle, M.C. (2018) Mining the Web to approximate university rankings. Information Discovery and Delivery, 46(3), 173-183. doi: 10.1108/IDD-05-2018-0014
[20]	Pal, A., Sarkar, A., & Bhattacharya, U. (2019). Webometric analysis of open universities in India. Library Philosophy and Practice. 3038.
[21]	Seeber, M., Lepori, B., Lomi, A., Aguillo, I., & Barberio, V. (2012). Factors affecting web links between European higher education institutions. Journal of informetrics, 3, 435-447. doi: 10.1016/j.joi.2012.03.001
[22]	Thelwall, M. (2019). Online Indicators for Non-Standard Academic Outputs. In Springer Handbook of Science and Technology Indicators (pp. 835-856). Springer, Cham.
[23]	Thelwall, M. (2009). Introduction to webometrics: Quantitative web research for the social sciences. Synthesis lectures on information concepts, retrieval, and services 1. 1 p. 1-116.
[24]	Thelwall, M., Vaughan, L., & Björneborn, L. (2005). “Webometrics. ” Annual Review of Information Science and Technology 39.1 p. 81-135.
[25]	Vaughan, Liwen., & R., Yang. (2013). Web traffic and organization performance measures: Relationships and data sources examined. Journal of Informetrics 7.3 p. 699-711.
[26]	Wouters, P., Zahedi, Z., & Costas, R. (2019). Social media metrics for new research evaluation. In Springer handbook of science and technology indicators (pp. 687-713). Springer, Cham.

Cluster1 (blue)	unito,unimi, polimi, unipd, unibo, unifi,unipi, uniroma1, unina
Cluster2 (red)	unisg, hunimed, iusspavia, unibz, sissa, imtlucca, sssup
Cluster3 (green)	univda,liuc,uninsubria,iulm,unisr,uniurb,unicam,unitus,lumsa,uniroma4,unicampus,unint.eu,unieurm,unilink,unicas,unisannio,uniparth,unisob.na,univaq,unite,unich,unimol,unifg,lum,unisale,unibas,unicz,unirc,unime,unikore,uniss
Cluster4 (yellow)	polito,unige,unibocconi,unicatt,unimib,unibg,unibs,unipv,unitn,univr,unive,iuav,uniud,units,unipr,unimore,unife,univpm,unimc,sns,unisi,unipg,uniroma2,luiss,uniroma3,unisa,poliba,uniba,unical,unipa,unict,unica