Dictionary example sentences play an important role in illustrating word definitions and usage, but manually creating quality sentences is challenging. Prior works have demonstrated that language models can be trained to generate example sentences. However, they relied on costly customized models and word sense datasets for generation and evaluation of their work. Rapid advancements in foundational models present the opportunity to create low-cost, zero-shot methods for the generation and evaluation of dictionary example sentences. We introduce a new automatic evaluation metric called OxfordEval that measures the win-rate of generated sentences against existing Oxford Dictionary sentences. OxfordEval shows high alignment with human judgments, enabling large-scale automated quality evaluation. We experiment with various LLMs and configurations to generate dictionary sentences across word classes. We complement this with a novel approach of using masked language models to identify and select sentences that best exemplify word meaning. The eventual model, FM-MLM, achieves over 85.1% win rate against Oxford baseline sentences according to OxfordEval, compared to 39.8% win rate for prio
The purpose of the present paper is to analyze several variants of Solovay's theorem on the existence of doubly partially conservative sentences. First, we investigate $Θ$ sentences that are doubly $(Γ, Λ)$-conservative over $T$ for several triples $(Θ, Γ, Λ)$. Among other things, we prove that the existence of a $Δ_{n+1}(\mathsf{PA})$ sentence that is doubly $(Σ_n, Σ_n)$-conservative over $T$ is equivalent to the $Σ_{n+1}$-inconsistency of $T$ over $\mathsf{PA}$. Secondly, we study $Θ$ sentences that are hereditarily doubly $(Γ, Λ)$-conservative over $T$ for several triples $(Θ, Γ, Λ)$.
This study explores how sentence types affect the Lombard effect and intelligibility enhancement, focusing on comparisons between natural and grid sentences. Using the Lombard Chinese-TIMIT (LCT) corpus and the Enhanced MAndarin Lombard Grid (EMALG) corpus, we analyze changes in phonetic and acoustic features across different noise levels. Our results show that grid sentences produce more pronounced Lombard effects than natural sentences. Then, we develop and test a normal-to-Lombard conversion model, trained separately on LCT and EMALG corpora. Through subjective and objective evaluations, natural sentences are superior in maintaining speech quality in intelligibility enhancement. In contrast, grid sentences could provide superior intelligibility due to the more pronounced Lombard effect. This study provides a valuable perspective on enhancing speech communication in noisy environments.
This article presents a neural language architecture of sentence structure in the brain, in which sentences are temporal connection paths that interconnect neural structures underlying their words. Words remain 'in-situ', hence they are always content-addressable. Arbitrary and novel sentences (with novel words) can be created with 'neural blackboards' for words and sentences. Hence, the unlimited productivity of natural language can be achieved with a 'fixed' small world like network structure. The article focuses on the neural blackboard for sentences. The architecture uses only one 'connection matrix' for binding all structural relations between words in sentences. Its ability to represent arbitrary (English) sentences is discussed in detail, based on a comprehensive analysis of them. The architecture simulates intra-cranial brain activity observed during sentence processing and fMRI observations related to sentence complexity and ambiguity. The simulations indicate that the observed effects relate to global control over the architecture, not to the sentence structures involved, which predicts higher activity differences related to complexity and ambiguity with higher comprehens
In the domain of Aspect-Based Sentiment Analysis (ABSA), generative methods have shown promising results and achieved substantial advancements. However, despite these advancements, the tasks of extracting sentiment quadruplets, which capture the nuanced sentiment expressions within a sentence, remain significant challenges. In particular, compound sentences can potentially contain multiple quadruplets, making the extraction task increasingly difficult as sentence complexity grows. To address this issue, we are focusing on simplifying sentence structures to facilitate the easier recognition of these elements and crafting a model that integrates seamlessly with various ABSA tasks. In this paper, we propose Aspect Term Oriented Sentence Splitter (ATOSS), which simplifies compound sentence into simpler and clearer forms, thereby clarifying their structure and intent. As a plug-and-play module, this approach retains the parameters of the ABSA model while making it easier to identify essential intent within input sentences. Extensive experimental results show that utilizing ATOSS outperforms existing methods in both ASQP and ACOS tasks, which are the primary tasks for extracting sentimen
Given a countable mathematical structure, its Scott sentence is a sentence of the infinitary logic $\mathcal{L}_{ω_1 ω}$ that characterizes it among all countable structures. We can measure the complexity of a structure by the least complexity of a Scott sentence for that structure. It is known that there can be a difference between the least complexity of a Scott sentence and the least complexity of a computable Scott sentence; for example, Alvir, Knight, and McCoy showed that there is a computable structure with a $Π_2$ Scott sentence but no computable $Π_2$ Scott sentence. It is well known that a structure with a $Π_2$ Scott sentence must have a computable $Π_4$ Scott sentence. We show that this is best possible: there is a computable structure with a $Π_2$ Scott sentence but no computable $Σ_4$ Scott sentence. We also show that there is no reasonable characterization of the computable structures with a computable $Π_n$ Scott sentence by showing that the index set of such structures is $Π^1_1$-$m$-complete.
Random groups of density d<\frac{1}{2} are infinite hyperbolic, and of density d>\frac{1}{2} are finite. We prove that for any given system of equations Σ, all the solutions of Σover a random group of density d<\frac{1}{2} are projected from solutions of Σover the free group F_{k}, with overwhelming probability, where k is the rank of the group. We conclude that any given sentence in the Boolean algebra of universal sentences, is a truth sentence over F_{k} if and only if it is a truth sentence over random groups of density d<\frac{1}{2}, with overwhelming probability.
There is a longstanding debate in the logico-philosophical community as to why the Gödelian sentences of a consistent and sufficiently strong theory are true. The prevalent argument seems to be something like this: since every one of the Gödelian sentences of such a theory is equivalent to the theory's consistency statement, even provably so inside the theory, the truth of those sentences follows from the consistency of the theory in question. So, Gödelian sentences of consistent theories should be true. In this paper, we show that Gödelian sentences of only sound theories are true; and there is a long road from consistency to soundness, indeed a hierarchy of conditions which are satisfied by some theories and falsified by others. We also study the truth of Rosserian sentences and provide necessary and sufficient conditions for the truth of Rosserian (and also Gödelian) sentences of theories.
Random groups of density d<\frac{1}{2} are infinite hyperbolic, and of density d>\frac{1}{2} are finite. We prove the existence of a uniform quantifier elimination procedure for formulas of minimal rank (probably the superstable part of the theory). Namely, given a minimal rank formula V(p), we prove the existence of a formula \varphi(p) that belongs to the Boolean algebra of two quantifiers, so that the two formulas V(p) and \varphi(p) define the same set over the free group F_{k} and over a random group of density d<\frac{1}{2}. We conclude that any given sentence of minimal rank is a truth sentence over the free group F_{k} if and only if it is a truth sentence over random groups of density d<\frac{1}{2}.
The noisy labeling problem has been one of the major obstacles for distant supervised relation extraction. Existing approaches usually consider that the noisy sentences are useless and will harm the model's performance. Therefore, they mainly alleviate this problem by reducing the influence of noisy sentences, such as applying bag-level selective attention or removing noisy sentences from sentence-bags. However, the underlying cause of the noisy labeling problem is not the lack of useful information, but the missing relation labels. Intuitively, if we can allocate credible labels for noisy sentences, they will be transformed into useful training data and benefit the model's performance. Thus, in this paper, we propose a novel method for distant supervised relation extraction, which employs unsupervised deep clustering to generate reliable labels for noisy sentences. Specifically, our model contains three modules: a sentence encoder, a noise detector and a label generator. The sentence encoder is used to obtain feature representations. The noise detector detects noisy sentences from sentence-bags, and the label generator produces high-confidence relation labels for noisy sentences.
We demonstrate how to parse Geach's Donkey sentences in a compositional distributional model of meaning. We build on previous work on the DisCoCat (Distributional Compositional Categorical) framework, including extensions that model discourse, determiners, and relative pronouns. We present a type-logical syntax for parsing donkey sentences, for which we define both relational and vector space semantics.
The standard recurrent neural network language model (RNNLM) generates sentences one word at a time and does not work from an explicit global sentence representation. In this work, we introduce and study an RNN-based variational autoencoder generative model that incorporates distributed latent representations of entire sentences. This factorization allows it to explicitly model holistic properties of sentences such as style, topic, and high-level syntactic features. Samples from the prior over these sentence representations remarkably produce diverse and well-formed sentences through simple deterministic decoding. By examining paths through this latent space, we are able to generate coherent novel sentences that interpolate between known sentences. We present techniques for solving the difficult learning problem presented by this model, demonstrate its effectiveness in imputing missing words, explore many interesting properties of the model's latent sentence space, and present negative results on the use of the model in language modeling.
Disinformation is often presented in long textual articles, especially when it relates to domains such as health, often seen in relation to COVID-19. These articles are typically observed to have a number of trustworthy sentences among which core disinformation sentences are scattered. In this paper, we propose a novel unsupervised task of identifying sentences containing key disinformation within a document that is known to be untrustworthy. We design a three-phase statistical NLP solution for the task which starts with embedding sentences within a bespoke feature space designed for the task. Sentences represented using those features are then clustered, following which the key sentences are identified through proximity scoring. We also curate a new dataset with sentence level disinformation scorings to aid evaluation for this task; the dataset is being made publicly available to facilitate further research. Based on a comprehensive empirical evaluation against techniques from related tasks such as claim detection and summarization, as well as against simplified variants of our proposed approach, we illustrate that our method is able to identify core disinformation effectively.
The ability to fuse sentences is highly attractive for summarization systems because it is an essential step to produce succinct abstracts. However, to date, summarizers can fail on fusing sentences. They tend to produce few summary sentences by fusion or generate incorrect fusions that lead the summary to fail to retain the original meaning. In this paper, we explore the ability of Transformers to fuse sentences and propose novel algorithms to enhance their ability to perform sentence fusion by leveraging the knowledge of points of correspondence between sentences. Through extensive experiments, we investigate the effects of different design choices on Transformer's performance. Our findings highlight the importance of modeling points of correspondence between sentences for effective sentence fusion.
Atomic clauses are fundamental text units for understanding complex sentences. Identifying the atomic sentences within complex sentences is important for applications such as summarization, argument mining, discourse analysis, discourse parsing, and question answering. Previous work mainly relies on rule-based methods dependent on parsing. We propose a new task to decompose each complex sentence into simple sentences derived from the tensed clauses in the source, and a novel problem formulation as a graph edit task. Our neural model learns to Accept, Break, Copy or Drop elements of a graph that combines word adjacency and grammatical dependencies. The full processing pipeline includes modules for graph construction, graph editing, and sentence generation from the output graph. We introduce DeSSE, a new dataset designed to train and evaluate complex sentence decomposition, and MinWiki, a subset of MinWikiSplit. ABCD achieves comparable performance as two parsing baselines on MinWiki. On DeSSE, which has a more even balance of complex sentence types, our model achieves higher accuracy on the number of atomic sentences than an encoder-decoder baseline. Results include a detailed error
Local (first order) sentences, introduced by Ressayre, enjoy very nice decidability properties, following from some stretching theorems stating some remarkable links between the finite and the infinite model theory of these sentences. We prove here several additional results on local sentences. The first one is a new decidability result in the case of local sentences whose function symbols are at most unary: one can decide, for every regular cardinal k whether a local sentence phi has a model of order type k. Secondly we show that this result can not be extended to the general case. Assuming the consistency of an inaccessible cardinal we prove that the set of local sentences having a model of order type omega_2 is not determined by the axiomatic system ZFC + GCH, where GCH is the generalized continuum hypothesis
This study is to review the approaches used for measuring sentences similarity. Measuring similarity between natural language sentences is a crucial task for many Natural Language Processing applications such as text classification, information retrieval, question answering, and plagiarism detection. This survey classifies approaches of calculating sentences similarity based on the adopted methodology into three categories. Word-to-word based, structure based, and vector-based are the most widely used approaches to find sentences similarity. Each approach measures relatedness between short texts based on a specific perspective. In addition, datasets that are mostly used as benchmarks for evaluating techniques in this field are introduced to provide a complete view on this issue. The approaches that combine more than one perspective give better results. Moreover, structure based similarity that measures similarity between sentences structures needs more investigation.
Sentence scoring and sentence selection are two main steps in extractive document summarization systems. However, previous works treat them as two separated subtasks. In this paper, we present a novel end-to-end neural network framework for extractive document summarization by jointly learning to score and select sentences. It first reads the document sentences with a hierarchical encoder to obtain the representation of sentences. Then it builds the output summary by extracting sentences one by one. Different from previous methods, our approach integrates the selection strategy into the scoring model, which directly predicts the relative importance given previously selected sentences. Experiments on the CNN/Daily Mail dataset show that the proposed framework significantly outperforms the state-of-the-art extractive summarization models.
Large language models (LLMs) are increasingly used for topic modeling, outperforming classical topic models such as LDA. Commonly, pre-trained LLM encoders such as BERT are used out-of-the-box despite the fact that fine-tuning is known to improve LLMs considerably. The challenge lies in obtaining a suitable labeled dataset for fine-tuning. In this paper, we build on the recent idea of using bags of sentences as the elementary unit for computing topics. Based on this idea, we derive an approach called FT-Topic to perform unsupervised fine-tuning, relying primarily on two steps for constructing a training dataset in an automatic fashion. First, a heuristic method identifies pairs of sentence groups that are assumed to belong either to the same topic or to different topics. Second, we remove sentence pairs that are likely labeled incorrectly. The resulting dataset is then used to fine-tune an encoder LLM, which can be leveraged by any topic modeling approach that uses embeddings. In this work, we demonstrate its effectiveness by deriving a novel state-of-the-art topic modeling method called SenClu. The method achieves fast inference through an expectation-maximization algorithm and ha
Several backbone models pre-trained on general domain datasets can encode a sentence into a widely useful embedding. Such sentence embeddings can be further enhanced by domain adaptation that adapts a backbone model to a specific domain. However, domain adaptation for low-resource languages like Japanese is often difficult due to the scarcity of large-scale labeled datasets. To overcome this, this paper introduces SDJC (Self-supervised Domain adaptation for Japanese sentence embeddings with Contrastive learning) that utilizes a data generator to generate sentences, which have the same syntactic structure to a sentence in an unlabeled specific domain corpus but convey different semantic meanings. Generated sentences are then used to boost contrastive learning that adapts a backbone model to accurately discriminate sentences in the specific domain. In addition, the components of SDJC like a backbone model and a method to adapt it need to be carefully selected, but no benchmark dataset is available for Japanese. Thus, a comprehensive Japanese STS (Semantic Textual Similarity) benchmark dataset is constructed by combining datasets machine-translated from English with existing datasets.