Journal of Data and Information Science ›› 2022, Vol. 7 ›› Issue (2): 84-106.doi: 10.2478/jdis-2022-0009
• Research Papers • Previous Articles
Fei Pu†(), Zhongwei Zhang, Yan Feng, Bailin Yang
Received:
2021-11-03
Revised:
2022-01-13
Accepted:
2022-03-10
Online:
2022-05-20
Published:
2022-04-19
Contact:
Fei Pu
E-mail:puf2008@gmail.com
Add to citation manager EndNote|Ris|BibTeX
URL: http://manu47.magtech.com.cn/Jwk3_jdis/EN/10.2478/jdis-2022-0009
http://manu47.magtech.com.cn/Jwk3_jdis/EN/Y2022/V7/I2/84
This work is licensed under the Creative Commons Attribution 4.0 International License.
Table 1.
Parameters and scoring functions in SOTA baselines and in ContE model.
Model | Scoring function ψ(e1,r,e2) | Parameters |
---|---|---|
TransE | | |
ComplEx | | |
SimplE | | |
ConvE | | |
ConvKB | | |
Rotate | | |
HAKE | | |
| | |
ContE | | |
Table 2.
Relation pattern modeling and inference abilities of baseline models.
Model | Symmetry | Antisymmetry | Inversion | Composition |
---|---|---|---|---|
TransE (Antoine et al., | × | √ | √ | √ |
DistMult (Yang et al., | √ | × | × | × |
ComplEx (Trouillon et al., | √ | √ | √ | × |
SimplE (Seyed & David, | √ | √ | √ | × |
ConvE (Dettmers et al., | - | - | - | - |
ConvKB (Nguyen et al., | - | - | - | - |
RotatE (Sun et al., | √ | √ | √ | √ |
HAKE (Zhang et al., | √ | √ | √ | √ |
KGCR (Pu et al., | √ | √ | √ | × |
LineaRE (Peng & Zhang, | √ | √ | √ | √ |
ContE | √ | √ | √ | √ |
Table 3.
Comparison of SOTA baselines and ContE model in terms of time complexity and number of parameters.
Models | #Parameters | Time Complexity |
---|---|---|
TransE | O(ned + nrd) | O(d) |
NTN | O(ned + nrd2k) | O(d3) |
ComplEx | O(ned + nrd) | O(d) |
TransR | O(ned + nrdk) | O(dk) |
SimplE | O(ned + nrd) | O(d) |
ContE | O(ned + 2nrd) | O(d) |
Table 5.
Experimental results for UMLS.
UMLS | ||||
---|---|---|---|---|
MRR | Hits@N | |||
1 | 3 | 10 | ||
TransE (Antoine et al., | 0.7966 | 0.6452 | 0.9418 | 0.9841 |
DistMult (Yang et al., | 0.868 | 0.821 | - | 0.967 |
ComplEx (Trouillon et al., | 0.8753 | 0.7942 | 0.9531 | 0.9713 |
ConvE (Dettmers et al., | 0.957 | 0.932 | - | 0.994 |
NeuralLP (Yang, Zhang, & Cohen, | 0.778 | 0.643 | - | 0.962 |
NTP-λ (Rocktaschel et al., | 0.912 | 0.843 | - | 1.0 |
MINERVA (Das et al., | 0.825 | 0.728 | - | 0.968 |
KGRRS+ComplEx (Lin et al., | 0.929 | 0.887 | - | 0.985 |
KGRRS+ConvE (Lin et al., | 0.940 | 0.902 | - | 0.992 |
Rotate (Sun et al., | 0.9274 | 0.8744 | 0.9788 | 0.9947 |
HAKE (Zhang et al., | 0.8928 | 0.8366 | 0.9387 | 0.9849 |
LineaRE (Peng & Zhang, | 0.9508 | 0.9145 | 0.9856 | 0.9992 |
ContE | 0.9677 | 0.9501 | 0.9811 | 1.0 |
Table 6.
Experimental results for Nations.
Nations | ||||
---|---|---|---|---|
MRR | Hits@N | |||
1 | 3 | 10 | ||
TransE (Antoine et al., | 0.4813 | 0.2189 | 0.6667 | 0.9801 |
DistMult (Yang et al., | 0.7131 | 0.5970 | 0.7761 | 0.9776 |
ComplEx (Trouillon et al., | 0.6677 | 0.5274 | 0.7413 | 0.9776 |
ConvE (Dettmers et al., | 0.5616 | 0.3470 | 0.7155 | 0.9946 |
Rotate (Sun et al., | 0.7155 | 0.5796 | 0.7985 | 1.0 |
HAKE (Zhang et al., | 0.7157 | 0.5945 | 0.7786 | 0.9851 |
LineaRE (Peng & Zhang, | 0.8146 | 0.7114 | 0.8881 | 0.9975 |
ContE | 0.8412 | 0.7587 | 0.9179 | 1.0 |
Table 7.
Experimental results for FB15K-237.
FB15K-237 | ||||
---|---|---|---|---|
MRR | Hits@N | |||
1 | 3 | 10 | ||
TransE (Antoine et al., | 0.279 | 0.198 | 0.376 | 0.441 |
DistMult (Yang et al., | 0.281 | 0.199 | 0.301 | 0.446 |
ComplEx (Trouillon et al., | 0.278 | 0.194 | 0.297 | 0.45 |
ConvE (Dettmers et al., | 0.312 | 0.225 | 0.341 | 0.497 |
ConvKB (Nguyen et al., | 0.289 | 0.198 | 0.324 | 0.471 |
R-GCN (Schlichtkrull et al., | 0.164 | 0.10 | 0.181 | 0.30 |
SimplE (Seyed & David, | 0.169 | 0.095 | 0.179 | 0.327 |
CapsE (Nguyen et al., | 0.150 | - | - | 0.356 |
Rotate (Sun et al., | 0.338 | 0.241 | 0.375 | 0.533 |
ContE | 0.3445 | 0.2454 | 0.3823 | 0.5383 |
Table 8.
Experimental results for Countries_S1.
Countries_S1 | ||||
---|---|---|---|---|
MRR | Hits@N | |||
1 | 3 | 10 | ||
TransE (Antoine et al., | 0.8785 | 0.7708 | 1.0 | 1.0 |
DistMult (Yang et al., | 0.9028 | 0.8125 | 1.0 | 1.0 |
ComplEx (Trouillon et al., | 0.9792 | 0.9583 | 1.0 | 1.0 |
Rotate (Sun et al., | 0.8750 | 0.7708 | 1.0 | 1.0 |
HAKE (Zhang et al., | 0.9045 | 0.8333 | 0.9792 | 1.0 |
LineaRE (Peng & Zhang, | 1.0 | 1.0 | 1.0 | 1.0 |
ContE | 1.0 | 1.0 | 1.0 | 1.0 |
Table 9.
Experimental results for Countries_S2 and Countries_S3.
Countries_S2 | Countries_S3 | |||||||
---|---|---|---|---|---|---|---|---|
MRR | Hits@N | MRR | Hits@N | |||||
1 | 3 | 10 | 1 | 3 | 10 | |||
TransE | 0.6997 | 0.50 | 0.9375 | 1.0 | 0.1206 | 0.00 | 0.0833 | 0.3542 |
DistMult | 0.7813 | 0.5833 | 1.0 | 1.0 | 0.2496 | 0.0625 | 0.333 | 0.6250 |
ComplEx | 0.7934 | 0.6042 | 0.9792 | 1.0 | 0.2731 | 0.0833 | 0.3958 | 0.6667 |
Rotate | 0.6979 | 0.4792 | 0.9583 | 1.0 | 0.1299 | 0.00 | 0.0833 | 0.4792 |
HAKE | 0.6667 | 0.4583 | 0.8333 | 0.9583 | 0.2472 | 0.0625 | 0.3333 | 0.5417 |
LineaRE | 0.7873 | 0.6458 | 0.9583 | 0.9792 | 0.2393 | 0.0625 | 0.3542 | 0.5208 |
ContE | 0.8370 | 0.7292 | 0.9583 | 0.9792 | 0.4695 | 0.3542 | 0.5 | 0.625 |
[1] | Ali M., Berrendorf M., Hoyt C.T., Vermue L., Sharifzadeh S., Tresp V., & Lehmann J. (2021). PyKEEN 1.0- A python library for training and evaluating knowledge graph embeddings. Journal of Machine Learning Research, 22, 1-6. |
[2] | Antoine B., Nicolas U., Alberto G.D.,Weston, J & Yakhnenko, O. (2013). Translating embeddings for modeling multirelational data. Proceedings of Advances in Neural Information Processing Systems (NIPS), 2787-2795. |
[3] | Chang K.W., Yih W.T., Yang B.S., & Meek C. (2014). Typed tensor decomposition of knowledge bases for relation extraction. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 1568-1579. |
[4] | Das R, Dhuliawala S., Zaheer M., Vilnis L., Durugkar I., Krishnamurthy A., Smola A., & McCallum A. (2018). Go for a walk and arrive at the answer:Reasoning over paths in knowledge bases using reinforcement learning. Proceedings of 6th International Conference on Learning Representations (ICLR). |
[5] | Dettmers T., Minervini P., Stenetorp P., & Riedel S. (2018). Convolutional 2D knowledge graph embeddings. Proceedings of 32nd AAAI Conference on Artificial Intelligence, 1811-1818. |
[6] | Devlin J., Chang M.W., Lee K., & Toutanova K. (2019). Bert:Pre-training of deep bidirectional transformers for language understanding. Proceedings of 57th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 4171-4186. |
[7] | Ding B.Y., Wang Q., Wang B., & Guo L. (2018). Improving knowledge graph embedding using simple constraints, Proceedings of 56th Annual Meeting of the Association for Computational Linguistics (ACL), 110-121. |
[8] | Dong X., Gabrilovich E., Geremy H., Horn W., Lao N., Murphy K., Strohmann T., Sun S.H., & Zhang W. (2014). Knowledge vault: A web-scale approach to probabilistic knowledge fusion. Proceedings of 20th ACM SIGKDD conference on Knowledge Discovery and Data Mining (KDD), 601-610. |
[9] |
Galarraga L.A., Teflioudi C., Hose K., & Suchanek F.M. (2015). Fast rule mining in ontological knowledge bases with AMIE+. VLDB Journal, 24(6), 707-730.
doi: 10.1007/s00778-015-0394-1 |
[10] | Gjergji K., Fabian M.S., Georgiana I., Maya R. & Gerhard W. (2018). Naga: Searching and ranking knowledge. Proceedings of 24th IEEE International Conference on Data Engineering (ICDE), 953-962. |
[11] | Guillaume B., Sameer S., & Theo T. (2015). On approximate reasoning capabilities of low-rank vector spaces. Proceedings of AAAI Spring Syposium on Knowledge Representation and Reasoning (KRR): Integrating Symbolic and Neural Approaches, 6-9. |
[12] | Guo S., Wang Q., Wang L.H., Wang B., & Guo L. (2018). Knowledge graph embedding with iterative guidance from soft rules. Proceedings of 32th AAAI Conference on Artificial Intelligence, 4816-4823. |
[13] | Guo L.B., Sun Z.Q., & Hu W. (2019). Learning to exploit long-term relational dependencies in knowledge graphs. Proceedings of 36th International Conference on Machine Learning (ICML), 2505-2514. |
[14] | Hao Y.C., Zhang Y.Z., Liu K., He S.Z, Liu Z.Y., Wu H., & Zhao J. (2017). An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge. Proceedings of 55th Annual Meeting of the Association for Computational Linguistics (ACL), 221-231. |
[15] | Ji G.L., He S.Z., & Xu L. (2015). Knowledge graph embedding via dynamic mapping matrix, Proceedings of 53rd Annual Meeting of the Association for Computational Linguistics and 7th International Joint Conference on Natural Language Processing, 687-696. |
[16] | Ji G.L., Liu K., He S.Z., & Zhao J. (2017). Distant supervision for relation extraction with sentence-level attention and entity descriptions. Proceedings of 31st AAAI Conference on Artificial Intelligence, 3060-3066. |
[17] | Ji G.L., Liu K., He S.Z., & Zhao J. (2016). Knowledge graph completion with adaptive sparse transfer matrix. Proceedings of 30th Conference on Artificial Intelligence, 985-991. |
[18] | Kok S., & Domingos P. (2007). Statistical predicate invention. Proceedings of the 24th International Conference on Machine Learning (ICML), 433-440. |
[19] |
Li H., Y. Liu, Mamoulis N., & Rosenblum D.S. (2019). Translation-based sequential recommendation for complex users on sparse data. IEEE Transactions on Knowledge and Data Engineering, 32(8), 1639-1651.
doi: 10.1109/TKDE.2019.2906180 |
[20] | Lin X.V., Socher R., & Xiong C.M. (2018). Multi-hop knowledge graph reasoning with reward shaping, Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), 3243-3253. |
[21] | Lin Y.K., Liu Z.Y., Luan H.B., Sun M.S., Rao S.W., & Liu S. (2015). Modeling relation paths for representation learning of knowledge bases. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 705-714. |
[22] | Liu H.X., Wu Y.X., & Yang Y.M. (2017). Analogical inference for multi-relational embeddings. Proceedings of 34th International Conference on Machine Learning (ICML), 2168-2178. |
[23] | Liu W.J., Zhou P., Zhao Z., Wang Z.R, Ju Q., Deng H.T., & Wang P. (2020). K-BERT:Enabling language representation with knowledge graph, Proceedings of 34th AAAI Conference on Artificial Intelligence, 1112-1119. |
[24] | Liu Y., Li H., & Alberto G.D. (2019). MMKG: Multi-modal knowledge graphs, European Semantic Web Conference, LNCS, 11503, 459-474. |
[25] | Mikolov T., Sutskever I., Chen K., Corrado G.S., & Dean J. (2013). Distributed representations of words and phrases and their compositionality. Proceedings of Advances in Neural Information Processing Systems (NIPS), 3111-3119. |
[26] | Minervini P., Costabello L., Munoz E., Novacek V., & Vandenbussche P.Y. (2017). Regularizing knowledge graph embeddings via equivalence and inversion axioms. Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML-KDD), 668-683. |
[27] | Nathani D., Chauhan J., Sharma C., & Kaul M. (2019). Learning attention-based embeddings for relation prediction in knowledge graphs. Proceedings of 57th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 4710-4723. |
[28] | Nguyen D.Q., Nguyen T.D., Nguyen D.Q., & Phung D. (2018). A novel embedding model for knowledge base completion based on convolutional neural network. Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT), 327-333. |
[29] | Nguyen D.Q., Vu T., Nguyen T.D., Nguyen D.Q., & Phung D. (2019). A capsule network-based embedding model for knowledge graph completion and search personalization. Proceedings of 57th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2180-2189. |
[30] | Peng Y.H., & Zhang J. (2020). LineaRE: Simple but Powerful Knowledge Graph Embedding for Link Prediction. Proceedings of 20th IEEE International Conference on Data Mining (ICDM), 422-431. |
[31] | Pu F., Yang B.L., Ying J.C, You L.Z., & Xu C.O. (2020). A Contextualized Entity Representation for Knowledge Graph Completion. Proceedings of 13th International Conference on Knowledge Science, Engineering and Management (KSEM), 77-85. |
[32] | Rocktaschel T., & Riedel S. (2017). End-to-end differentiable proving. Proceedings of Advances in Neural Information Processing Systems (NIPS), 3791-3803. |
[33] | Schlichtkrull M.S., Thomas N.K., & Bloem P. (2018). Modeling relational data with graph convolutional networks. Gangemi A. et al. (eds.) ESWC 2018, LNCS, vol. 10843, 593-607, Springer, Heidelberg. |
[34] | Seyed M.K., & David P. (2018). Simple embedding for link prediction in knowledge graphs. Proceedings of 32nd Conference on Advances in Neural Information Processing Systems (NIPS), 4289-4300. |
[35] | Socher R., Chen D.Q., Manning C.D., & Andrew Ng. (2013). Reasoning with neural tensor networks for knowledge base completion. Proceedings of Advances in Neural Information Processing Systems, 926-934. |
[36] | Sun Y., Wang S., Li Y., Feng S.K., Chen X.Y., Zhang H., Tian X, Zhu D.X., Tian H., & Wu H., (2019). ERNIE: Enhanced representation through knowledge integration. arXiv preprint arXiv:1904.09223. |
[37] | Sun Z.Q., Deng Z.H., Nie J.Y., & Tang J. (2019). Rotate:knowledge graph embedding by relational rotation in complex space. Proceedings of 7th International Conference on Learning Representations (ICLR). |
[38] | Sun Z.Q., Vashishth S., Sanyal S., Talukdar P., & Yang Y.M. (2020). A re-evaluation of knowledge graph completion methods. Proceedings of 58th Annual Meeting of the Association for Computational Linguistics (ACL), 5516-5522. |
[39] | Trouillon T., Welbl J., Riedel S., Gaussier E., & Bouchard G. (2016). Complex embeddings for simple link prediction. Proceedings of International Conference on Machine Learning (ICML), 2071-2080. |
[40] | Vaswani A., Shazeer N., Parmar N, Uszkoreit J, Jones Llion J., Gomez A.N., Kaiser L., & Polosukhin I. (2017). Attention is all you need. Proceedings of Advances in Neural Information Processing Systems (NIPS), 5998-6008. |
[41] | Wang Z., Zhang J.W., Feng J.L., & Chen Z. (2014). Knowledge graph embedding by translating on hyperplanes. Proceedings of 28th AAAI Conference on Artificial Intelligence, 1112-1119. |
[42] | Xiong C.Y., Russell P., & Jamie C. (2017). Explicit semantic ranking for academic search via knowledge graph embedding. Proceedings of 26th International Conference on World Wide Web (WWW), 1271-1279. |
[43] | Yang B.S., & Tom M. (2017). Leveraging knowledge bases in LSTMs for improving machine reading. Proceedings of 55th Annual Meeting of the Association for Computational Linguistics (ACL), 1436-1446. |
[44] | Yang B.S., Yih W.T., He X.D., Gao J.F., & Deng L. (2015). Embedding entities and relations for learning and inference in knowledge bases. Proceedings of the International Conference on Learning Representations (ICLR), 1-12. |
[45] | Yang F, Yang Z.L., & Cohen W.W. (2017). Differentiable learning of logical rules for knowledge base reasoning. Proceedings of Advances in Neural Information Processing Systems (NIPS), 2319-2328. |
[46] | Zhang N.Y., Deng S.M., Sun Z.L., Wang G.Y., Chen X., Zhang W., & Chen H.J. (2019). Long-tail relation extraction via knowledge graph embeddings and graph convolution networks, Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (ACL), 3016-3025. |
[47] | Zhang Z.Q., Cai J.Y., Zhang Y.D., & Wang J. (2020). Learning hierarchy-aware knowledge graph embeddings for link prediction. Proceedings of 34th AAAI Conference on Artificial Intelligence, 3065-3072. |
[48] | Zhong H.P., Zhang J.W., Wang Z., Wan H., & Chen Z.(2015). Aligning knowledge and text embeddings by entity descriptions. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 267-272. |
[1] | Raf Guns. Predictive Characteristics of Co-authorship Networks: Comparing the Unweighted, Weighted, and Bipartite Cases [J]. Journal of Data and Information Science, 2016, 1(3): 59-78. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||