ABSTRACT
We develop a categorical compositional distributional semantics for Lambek Calculus with a Relevant Modality $\mathbf{!L^*}$, a modality that allows for the use of limited editions of contraction and permutation in the logic. Lambek Calculus has been introduced to analyse syntax of natural language and the linguistic motivation behind this modality is to extend the domain of the applicability of the calculus to fragments which witness the discontinuity phenomena. The categorical part of the semantics is a monoidal biclosed category with a $!$-functor, very similar to the structure of a Differential Category. We instantiate this category to finite dimensional vector spaces and linear maps via "quantisation" functors and work with three concrete interpretations of the $!$-functor. We apply the model to construct categorical and concrete semantic interpretations for the motivating example of $\mathbf{!L^*}$: the derivation of a phrase with a parasitic gap. The efficacy of the concrete interpretations are evaluated via a disambiguation task, on an extension of a sentence disambiguation dataset to parasitic gap phrase one, using BERT, Word2Vec, and FastText vectors and relational tensors.
► BibTeX data
► References
[1] BERT-Base pretrained model. https://storage.googleapis.com/bert_models/2018_10_18/uncased_L-12_H-768_A-12.zip.
https://storage.googleapis.com/bert_models/2018_10_18/uncased_L-12_H-768_A-12.zip
[2] BERT-as-service. https://bert-as-service.readthedocs.io/en/latest/.
https://bert-as-service.readthedocs.io/en/latest/
[3] NoSketch engine. https://www.clarin.si/noske/wacs.cgi/first_form.
https://www.clarin.si/noske/wacs.cgi/first_form
[4] Python notebooks. https://drive.google.com/drive/folders/19agebprIhTEPknZt1YzqAaMyKLlZmTrX.
https://drive.google.com/drive/folders/19agebprIhTEPknZt1YzqAaMyKLlZmTrX
[5] NumPy. https://numpy.org/.
https://numpy.org/
[6] BERT WordPiece tokenization algorithm. https://huggingface.co/course/chapter6/6?fw=pt.
https://huggingface.co/course/chapter6/6?fw=pt
[7] BERT transformers. https://huggingface.co/docs/transformers/model_doc/bert.
https://huggingface.co/docs/transformers/model_doc/bert
[8] ukWaC corpus. https://wacky.sslmit.unibo.it/doku.php?id=corpora.
https://wacky.sslmit.unibo.it/doku.php?id=corpora
[9] John Baez and Mike Stay. Physics, topology, logic, and computation: A Rosetta Stone. In B. Coecke, editor, New Structures in Physics, volume 813 of Lecture Notes in Physics. Springer. Springer, 2011. 10.1007/978-3-642-12821-9_2.
https://doi.org/10.1007/978-3-642-12821-9_2
[10] Guy Barry, Mark Hepple, Neil Leslie, and Glyn Morrill. Proof figures and structural operators for categorial grammar. In Conference of the European Chapter of the Association for Computational Linguistics, 1991. 10.3115/977180.977215.
https://doi.org/10.3115/977180.977215
[11] Richard Blute, Prakash Panangaden, and Robert Seely. Fock space: A model of linear exponential types. Manuscript, revised version of the MFPS paper Holomorphic models of exponential types in linear logic, pages 474–512, 1994. https://ncatlab.org/nlab/files/BPSLinear.pdf.
https://ncatlab.org/nlab/files/BPSLinear.pdf
[12] Rrichard Blute, Robin Cockett, and Robert Seely. Differential categories. Mathematical Structures in Computer Science, 16 (6): 1049–1083, 2006. 10.1017/S0960129506005676.
https://doi.org/10.1017/S0960129506005676
[13] Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 2017. 10.1162/tacl_a_00051.
https://doi.org/10.1162/tacl_a_00051
[14] Alonzo Church. The weak theory of implication. Journal of Symbolic Logic, 18 (2): 177–178, 1953. 10.2307/2268954.
https://doi.org/10.2307/2268954
[15] Bob Coecke, Mehrnoosh Sadrzadeh, and Stephen Clark. Mathematical foundations for a compositional distributional model of meaning. Lambek Festschrift. Linguistic Analysis, 36: 345–384, 2010. 10.48550/arXiv.1003.4394.
https://doi.org/10.48550/arXiv.1003.4394
[16] Bob Coecke, Edward Grefenstette, and Mehrnoosh Sadrzadeh. Lambek vs. Lambek: Functorial vector space semantics and string diagrams for Lambek calculus. Annals of Pure and Applied Logic, 164: 1079 – 1100, 2013. 10.1016/j.apal.2013.05.009.
https://doi.org/10.1016/j.apal.2013.05.009
[17] James Richard Curran. From Distributional to Semantic Similarity. PhD thesis, University of Edinburgh, 2003. https://era.ed.ac.uk/handle/1842/563.
https://era.ed.ac.uk/handle/1842/563
[18] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. 10.18653/v1/N19-1423. https://aclanthology.org/N19-1423.
https://doi.org/10.18653/v1/N19-1423
https://aclanthology.org/N19-1423
[19] Elisabet Engdahl. Parasitic gaps. Linguistics and Philosophy, 6: 5–34, 1983. 10.1007/BF00868088.
https://doi.org/10.1007/BF00868088
[20] John R Firth. A synopsis of linguistic theory, 1930–1955. In Studies in Linguistic Analysis, pages 1–32. Blackwell, Oxford, 1957.
[21] Jean Yves Girard, Andre Scedrov, and Philip J. Scott. Bounded linear logic: A modular approach to polynomial-time computability. Theoretical Computer Science, 1992. 10.1016/0304-3975(92)90386-T.
https://doi.org/10.1016/0304-3975(92)90386-T
[22] Edward Grefenstette and Mehrnoosh Sadrzadeh. Experimental support for a categorical compositional distributional model of meaning. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1394–1404, Edinburgh, Scotland, UK., July 2011. Association for Computational Linguistics. https://www.aclweb.org/anthology/D11-1129.
https://www.aclweb.org/anthology/D11-1129
[23] Edward Grefenstette and Mehrnoosh Sadrzadeh. Concrete models and empirical evaluations for the categorical compositional distributional model of meaning. Computational Linguistics, 2015. 10.1162/COLI_a_00209.
https://doi.org/10.1162/COLI_a_00209
[24] Gregory Grefenstette. Explorations in Automatic Thesaurus Discovery. Springer New York, NY, 1994. 10.1007/978-1-4615-2710-7.
https://doi.org/10.1007/978-1-4615-2710-7
[25] Zellig S. Harris. Distributional structure. Word, 1954. 10.1080/00437956.1954.11659520.
https://doi.org/10.1080/00437956.1954.11659520
[26] James Edward Humphreys. Introduction to Lie Algebras and Representation Theory. Springer-Verlag, 1972. 10.1007/978-1-4612-6398-2.
https://doi.org/10.1007/978-1-4612-6398-2
[27] Martin Hyland and Andrea Schalk. Glueing and orthogonality for models of linear logic. Theoretical Computer Science, 294: 183–231, 02 2003. 10.1016/S0304-3975(01)00241-9.
https://doi.org/10.1016/S0304-3975(01)00241-9
[28] Peter Johnstone. Sketches of an Elephant: A Topos Theory Compendium. Oxford Logic Guides. Oxford University Press, New York, NY, 2002. 10.1017/S1079898600003462.
https://doi.org/10.1017/S1079898600003462
[29] Max Kanovich, Stepan Kuznetsov, and Andre Scedrov. Undecidability of the Lambek calculus with a relevant modality. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9804 LNCS: 240–256, 2016. 10.1007/978-3-662-53042-9_14.
https://doi.org/10.1007/978-3-662-53042-9_14
[30] Dimitri Kartsaklis and Mehrnoosh Sadrzadeh. Prior disambiguation of word tensors for constructing sentence vectors. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1590–1601, Seattle, Washington, USA, oct 2013. Association for Computational Linguistics. https://www.aclweb.org/anthology/D13-1166.
https://www.aclweb.org/anthology/D13-1166
[31] Dimitri Kartsaklis, Mehrnoosh Sadrzadeh, and Stephen Pulman. Separating disambiguation from composition in distributional semantics. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pages 114–123, Sofia, Bulgaria, August 2013. Association for Computational Linguistics. https://www.aclweb.org/anthology/W13-3513.
https://www.aclweb.org/anthology/W13-3513
[32] Yves Lafont. Soft linear logic and polynomial time. Theoretical Computer Science, 2004. 10.1016/j.tcs.2003.10.018.
https://doi.org/10.1016/j.tcs.2003.10.018
[33] Thomas K. Landauer and Susan T. Dumais. A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 1997. 10.1037/0033-295X.104.2.211.
https://doi.org/10.1037/0033-295X.104.2.211
[34] Jean-Simon Pacaud Lemay. Why FHilb is not an interesting (co)differential category. Electronic Proceedings in Theoretical Computer Science, 318: 13–26, May 2020. 10.4204/eptcs.318.2.
https://doi.org/10.4204/eptcs.318.2
[35] Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. Introduction to Information Retrieval. Cambridge University Press, Cambridge, UK, 2008. 10.1017/CBO9780511809071.
https://doi.org/10.1017/CBO9780511809071
[36] Lachlan McPheat, Mehrnoosh Sadrzadeh, Hadi Wazni, and Gijs Wijnholds. Categorical vector space semantics for Lambek Calculus with a relevant modality (extended abstract). EPTCS Proceedings of the third Annual International Applied Category Theory Conference 2020 (ACT 2020), 328, 2020. 10.4204/EPTCS.333.12. 6-10 July 2020.
https://doi.org/10.4204/EPTCS.333.12
[37] Lachlan McPheat, Gijs Wijnholds, Mehrnoosh Sadrzadeh, Adriana Correia, and Alexis Toumi. Anaphora and ellipsis in Lambek Calculus with a relevant modality: Syntax and semantics. Journal of Cognitive Science, 22 (2): 1–34, 2021. 10.17791/jcs.2021.22.2.1.
https://doi.org/10.17791/jcs.2021.22.2.1
[38] Paul-André Melliès. Categorical semantics of linear logic. In Pierre-Louis Curien, Hugo Herbelin, Jean-Louis Krivine, and Paul-André Melliès, editors, Interactive Models of Computation and Program Behaviour, volume 27 of Panoramas et Synthèses, pages 1–196. Société Mathématique de France, 2009.
[39] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems, page 3111–3119, 2013. 10.5555/2999792.2999959.
https://doi.org/10.5555/2999792.2999959
[40] Dmitrijs Milajevs, Dimitri Kartsaklis, Mehrnoosh Sadrzadeh, and Matthew Purver. Evaluating neural word representations in tensor-based compositional settings. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 708–719, Doha, Qatar, oct 2014. Association for Computational Linguistics. 10.3115/v1/D14-1079.
https://doi.org/10.3115/v1/D14-1079
[41] Michael Moortgat. Multimodal Linguistic Inference. Logic Journal of IGPL, 1995. 10.1093/jigpal/3.2-3.371.
https://doi.org/10.1093/jigpal/3.2-3.371
[42] Michael Moortgat and Gijs Wijnholds. Lexical and derivational meaning in vector-based models of relativisation. In Proceedings of the 21st Amsterdam Colloquium, 2017. https://doi.org/10.48550/arXiv.1711.11513.
https://doi.org/10.48550/arXiv.1711.11513
[43] Glyn Morrill. Grammar logicised: Relativisation. Linguistics and Philosophy, 40: 119–163, 2017. 10.1007/s10988-016-9197-0.
https://doi.org/10.1007/s10988-016-9197-0
[44] Glyn Morrill. A note on movement in logical grammar. Journal of Language Modelling, pages 353–363, 2018. 10.15398/jlm.v6i2.233.
https://doi.org/10.15398/jlm.v6i2.233
[45] Glyn Morrill and Oriol Valentín. Computational coverage of TLG: Nonlinearity. In Proceedings of NLCS'15. Third Workshop on Natural Language and Computer Science, volume 32, pages 51–63. EasyChair Publications, 2015. 10.48550/arXiv.1706.03033.
https://doi.org/10.48550/arXiv.1706.03033
[46] Glyn Morrill and Oriol Valentín. On the logic of expansion in natural language. In Maxime Amblard, Philippe de Groote, Sylvain Pogodalla, and Christian Retoré, editors, Logical Aspects of Computational Linguistics. Celebrating 20 Years of LACL (1996–2016), pages 228–246, Berlin, Heidelberg, 2016. Springer Berlin Heidelberg. 10.1007/978-3-662-53826-5_14.
https://doi.org/10.1007/978-3-662-53826-5_14
[47] Glyn Morrill, Neil Leslie, Mark Hepple, and Guy Barry. Categorial deductions and structural operations. In Studies in Categorial Grammar, volume 5 of Edinburgh Working Papers in Cognitive Science, pages 1–21. Centre for Cognitive Science, 1990.
[48] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. GloVe: Global vectors for word representation. In Empirical Methods in Natural Language Processing, 2014. 10.3115/v1/d14-1162.
https://doi.org/10.3115/v1/d14-1162
[49] Martin Pickering and Steven Frisson. Processing ambiguous verbs: Evidence from eye movements. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27: 556–73, 03 2001. 10.1037/0278-7393.27.2.556.
https://doi.org/10.1037/0278-7393.27.2.556
[50] Anne Preller. Natural language semantics in biproduct dagger categories. Journal of Applied Logic, 12 (1): 88–108, 2014. 10.1016/j.jal.2013.08.001. Logic Categories Semantics.
https://doi.org/10.1016/j.jal.2013.08.001
[51] Anne Preller and Mehrnoosh Sadrzadeh. Semantic vector models and functional models for pregroup grammars. Journal of Logic, Language and Information, 20 (4): 419–443, 2011. 10.1007/s10849-011-9132-2.
https://doi.org/10.1007/s10849-011-9132-2
[52] Herbert Rubenstein and John B. Goodenough. Contextual correlates of synonymy. Communications of the ACM, 8 (10): 627–633, October 1965. 10.1145/365628.365657.
https://doi.org/10.1145/365628.365657
[53] Mehrnoosh Sadrzadeh. Datasets. https://msadrzadeh.com/datasets/.
https://msadrzadeh.com/datasets/
[54] Mehrnoosh Sadrzadeh, Stephen Clark, and Bob Coecke. The Frobenius anatomy of word meanings I: Subject and object relative pronouns. Journal of Logic and Computation, 2013. 10.1093/logcom/ext044.
https://doi.org/10.1093/logcom/ext044
[55] Mehrnoosh Sadrzadeh, Stephen Clark, and Bob Coecke. The Frobenius anatomy of word meanings II: Possessive relative pronouns. Journal of Logic and Computation, 26 (2): 785–815, 2016. 10.1093/logcom/exu027.
https://doi.org/10.1093/logcom/exu027
[56] Mehrnoosh Sadrzadeh, Dimitri Kartsaklis, and Esma Balkır. Sentence entailment in compositional distributional semantics. Annals of Mathematics and Artificial Intelligence, 2018. 10.1007/s10472-017-9570-x.
https://doi.org/10.1007/s10472-017-9570-x
[57] Mehrnoosh Sadrzadeh, Michael Moortgat, and Gijs Wijnholds. A Frobenius algebraic analysis for parasitic gaps. Journal of Applied Logics, 7: 823–852, 2020. 10.48550/arXiv.2005.05639.
https://doi.org/10.48550/arXiv.2005.05639
[58] Gerard Salton. A document retrieval system for man-machine interaction. In 19th ACM National Conference, New York, New York, USA, 1964. ACM Press. 10.1145/800257.808923.
https://doi.org/10.1145/800257.808923
[59] Mark Steedman. Combinatory grammars and parasitic gaps. Natural Language and Linguistic Theory, 5: 403–439, 1987. 10.1007/BF00134555.
https://doi.org/10.1007/BF00134555
[60] Knut Taraldsen. The theoretical interpretation of a class of marked extractions. In Adriana Belletti, Luciana Brandi, and Luigi Rizzi, editors, Theory of Markedness in Generative Grammar: Proceedings of the 1979 GLOW Conference. Scuole Normale Superiore di Pisa, 1979. 10.1007/978-1-4757-5718-7_13.
https://doi.org/10.1007/978-1-4757-5718-7_13
[61] Gijs Wijnholds. compdisteval-ellipsis. https://github.com/gijswijnholds/compdisteval-ellipsis.
https://github.com/gijswijnholds/compdisteval-ellipsis
[62] Gijs Wijnholds and Mehrnoosh Sadrzadeh. Classical copying versus quantum entanglement in natural language: The case of vp-ellipsis. EPTCS Proceedings of the second workshop on Compositional Approaches for Physics, NLP, and Social Sciences (CAPNS), 2018. 10.4204/EPTCS.283.8.
https://doi.org/10.4204/EPTCS.283.8
[63] Gijs Wijnholds and Mehrnoosh Sadrzadeh. A type-driven vector semantics for ellipsis with anaphora using Lambek calculus with limited contraction. Journal of Logic, Language and Information, 28: 331–358, 2019a. 10.1007/s10849-019-09293-4.
https://doi.org/10.1007/s10849-019-09293-4
[64] Gijs Wijnholds and Mehrnoosh Sadrzadeh. Evaluating composition models for verb phrase elliptical sentence embeddings. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 261–271, Minneapolis, Minnesota, June 2019b. Association for Computational Linguistics. 10.18653/v1/N19-1023.
https://doi.org/10.18653/v1/N19-1023
[65] Gijs Jasper Wijnholds. Coherent diagrammatic reasoning in compositional distributional semantics. In International Workshop on Logic, Language, Information, and Computation, pages 371–386. Springer, 2017. 10.1007/978-3-662-55386-2_27.
https://doi.org/10.1007/978-3-662-55386-2_27
Cited by
Could not fetch Crossref cited-by data during last attempt 2024-11-21 09:37:25: Could not fetch cited-by data for 10.32408/compositionality-5-2 from Crossref. This is normal if the DOI was registered recently. On SAO/NASA ADS no data on citing works was found (last attempt 2024-11-21 09:37:25).
This Paper is published in Compositionality under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. Copyright remains with the original copyright holders such as the authors or their institutions.