We introduce two Bayesian models for unsupervised semantic role labeling (SRL) task. The models treat SRL as clustering of syntactic signatures of arguments with clusters corresponding to semantic roles. The ﬁrst model induces these clusterings independently for each predicate, exploiting the Chinese Restaurant Process (CRP) as a prior. In a more reﬁned hierarchical model, we inject the intuition that the clusterings are similar across different predicates, even though they are not necessarily identical.
Unsupervised learning of linguistic structure is a difﬁcult problem. A common approach is to deﬁne a generative model and maximize the probability of the hidden structure given the observed data. Typically, this is done using maximum-likelihood estimation (MLE) of the model parameters. We show using part-of-speech tagging that a fully Bayesian approach can greatly improve performance. Rather than estimating a single set of parameters, the Bayesian approach integrates over all possible parameter values. ...
Stochastic Optimality Theory (Boersma, 1997) is a widely-used model in linguistics that did not have a theoretically sound learning method previously. In this paper, a Markov chain Monte-Carlo method is proposed for learning Stochastic OT Grammars. Following a Bayesian framework, the goal is ﬁnding the posterior distribution of the grammar given the relative frequencies of input-output pairs. The Data Augmentation algorithm allows one to simulate a joint posterior distribution by iterating two conditional sampling steps. ...
Chance events are commonplace in our daily lives. Every day we face situations where the result is uncertain, and, perhaps without realizing it, we guess about the likelihood of one outcome or another. Fortunately, mastering the concepts of probability can cast new light on situations where randomness and chance appear to rule. In this fully revised second edition of Understanding Probability, the reader can learn about the world of probability in an appealing way.
We introduce a novel Bayesian approach for deciphering complex substitution ciphers. Our method uses a decipherment model which combines information from letter n-gram language models as well as word dictionaries. Bayesian inference is performed on our model using an efﬁcient sampling technique. We evaluate the quality of the Bayesian decipherment output on simple and homophonic letter substitution ciphers and show that unlike a previous approach, our method consistently produces almost 100% accurate decipherments. ...
Tuyển tập các báo cáo nghiên cứu khoa học quốc tế về bệnh thú y đề tài:
Estimation of Paratuberculosis Prevalence in Dairy Cattle in a Province of Korea using an Enzyme-linked Immunosorbent Assay: Application of Bayesian Approach
Tuyển tập các báo cáo nghiên cứu về y học được đăng trên tạp chí y học quốc tế cung cấp cho các bạn kiến thức về ngành y đề tài: Risk assessment of pre-hospital trauma airway management by anaesthesiologists using the predictive Bayesian approach
We investigate the problem of acoustic modeling in which prior language-speciﬁc knowledge and transcribed data are unavailable. We present an unsupervised model that simultaneously segments the speech, discovers a proper set of sub-word units (e.g., phones) and learns a Hidden Markov Model (HMM) for each induced acoustic unit.
We present an unsupervised, nonparametric Bayesian approach to coreference resolution which models both global entity identity across a corpus as well as the sequential anaphoric structure within each document. While most existing coreference work is driven by pairwise decisions, our model is fully generative, producing each mention from a combination of global entity properties and local attentional state. Despite being unsupervised, our system achieves a 70.3 MUC F1 measure on the MUC-6 test set, broadly in the range of some recent supervised results. ...
Machine transliteration is deﬁned as automatic phonetic translation of names across languages. In this paper, we propose synchronous adaptor grammar, a novel nonparametric Bayesian learning approach, for machine transliteration. This model provides a general framework without heuristic or restriction to automatically learn syllable equivalents between languages.
In this work I address the challenge of augmenting n-gram language models according to prior linguistic intuitions. I argue that the family of hierarchical Pitman-Yor language models is an attractive vehicle through which to address the problem, and demonstrate the approach by proposing a model for German compounds. In an empirical evaluation, the model outperforms the Kneser-Ney model in terms of perplexity, and achieves preliminary improvements in English-German translation.
Learning a tree substitution grammar is very challenging due to derivational ambiguity. Our recent approach used a Bayesian non-parametric model to induce good derivations from treebanked input (Cohn et al., 2009), biasing towards small grammars composed of small generalisable productions. In this paper we present a novel training method for the model using a blocked Metropolis-Hastings sampler in place of the previous method’s local Gibbs sampler.
Speech style accommodation refers to shifts in style that are used to achieve strategic goals within interactions. Models of stylistic shift that focus on speciﬁc features are limited in terms of the contexts to which they can be applied if the goal of the analysis is to model socially motivated speech style accommodation. In this paper, we present an unsupervised Dynamic Bayesian Model that allows us to model stylistic style accommodation in a way that is agnostic to which speciﬁc speech style features will shift in a way that resembles socially motivated stylistic variation.
Many semantic parsing models use tree transformations to map between natural language and meaning representation. However, while tree transformations are central to several state-of-the-art approaches, little use has been made of the rich literature on tree automata. This paper makes the connection concrete with a tree transducer based semantic parsing model and suggests that other models can be interpreted in a similar framework, increasing the generality of their contributions.
Tree substitution grammars (TSGs) offer many advantages over context-free grammars (CFGs), but are hard to learn. Past approaches have resorted to heuristics. In this paper, we learn a TSG using Gibbs sampling with a nonparametric prior to control subtree size. The learned grammars perform signiﬁcantly better than heuristically extracted ones on parsing accuracy.
We describe a statistical approach for modeling agreements and disagreements in conversational interaction. Our approach ﬁrst identiﬁes adjacency pairs using maximum entropy ranking based on a set of lexical, durational, and structural features that look both forward and backward in the discourse. We then classify utterances as agreement or disagreement using these adjacency pairs and features that represent various pragmatic inﬂuences of previous agreement or disagreement on the current utterance. ...
At the other extreme, each disk controller now has tens of mega-
bytes of storage and a very capable processor. It is quite feasible
to have intelligent disks that offer either database access (SQL or
some other non-procedural language) and even web service ac-
cess. Moving from a block-oriented disk interface to a file inter-
face, and then to a set or service interface has been the goal of
database machine advocates...
This chapter deals with independent component analysis (ICA) for nonlinear mixing models. A fundamental difﬁculty in the nonlinear ICA problem is that it is highly nonunique without some extra constraints, which are often realized by using a suitable regularization. We also address the nonlinear blind source separation (BSS) problem. Contrary to the linear case, we consider it different from the respective nonlinear ICA problem. After considering these matters, some methods introduced for solving the nonlinear ICA or BSS problems are discussed in more detail.
This study, given its Bayesian approach, is related to the recent article by Baks, Metrick,
and Wachter (2001), who estimate funds' alphas using informative prior beliefs about alpha.
They investigate the degree to which informative priors can preclude an investor from infer-
ring that at least one actively managed fund has a positive alpha. This inference relates to
an investment problem of a mutual fund investor who can also earn the hypothetical costless
returns on the benchmark indexes.