Szanowni Państwo,
Serdecznie zapraszamy na Seminarium naukowe Instytutu Informatyki, które odbędzie się 11 kwietnia 2024 r. o godzinie 10:15 w sali 110INF (ul. Oleska 48, 45-052 Opole).Referat pt.: „Fragments of Language and Transformer-based Language Models” wygłosi prof. dr Ian Pratt-Hartmann.
Abstract:
The impressive performance of transformer-based language models (TLMs) in natural language inference tasks has stimulated the development of a number of linguistic and deductive reasoning benchmarks to evaluate these models. The central question these benchmarks aim to resolve is whether TLMs are really learning the logical principles governing the language in question, or simply mimicking human annotators’ judgements guided by superficial cues. Perhaps the sternest test for language models in this regard concerns their ability to recognize valid entailments understood in a strictly logical sense, where the inference does not depend on background knowldge. Such an ability, if demonstrated, would indicate a genuine grasp of the logical semantics of natural language.
In this study, we investigate how the ability of TLMs to learn to solve the satisfiability problem in natural language is impacted by the array of available grammatical constructs featured in the problem instances to which they have been exposed. To carry out this evaluation faithfully, we additionally conduct an empirical study of how the probability of satisfiability in the fragments of natural language we consider depends on various salient parameters of the problem instance.