Mô hình nlp

Bài viết giới thiệu một phương pháp sử dụng trọng lượng từ BM25 kết hợp xử lý ngôn ngữ tự nhiên (BM25-NLP). Hiệu quả của phương pháp này được minh chứng thông qua việc thực nghiệm với ba phần mềm mã nguồn mở SVN, Argo UML, và Apache.

7p vijihyo2711 25-09-2021 26 1   Download

Luận văn nghiên cứu tổng quan về Bộ thủ tục hành chính trên Cổng thông tin điện tử thành phố Bà Rịa. Nghiên cứu các thành phần cấu tạo Chatbot. Tìm hiểu các kỹ thuật xử lý ngôn ngữ trong Natural Language Understanding (NLU), Natural Language Processing (NLP) như biểu diễn ngôn ngữ, phân loại ý định (Intent Classification hay Intent Detection), trích xuất thông tin (Information Extraction). Nghiên cứu mô hình cây quyết định (Decision Trees) để xây dựng hệ thống Chatbot.

103p interstellar 20-09-2021 37 3   Download

Nội dung luận văn được chia ra làm 3 phần như sau: Chương 1/ Giới thiệu tổng quan về hệ thống trợ lý ảo, cấu trúc hệ thống trợ lý ảo, trình bày về xử lý ngôn ngữ tự nhiên NLP và ứng dụng NLP trong chatbot. Chương 2/ Nghiên cứu một số kĩ thuật được sử dụng trong chatbot, tìm hiểu quản lý hội thoại, mô hình sinh hội thoại. Chương 3/ Trình bày về quá trình thực nghiệm và đánh giá, các kết quả được thực nghiệm và xây dựng chatbot.

52p hanh_tv26 04-04-2019 111 12   Download

Most natural language processing tasks require lexical semantic information. Automated acquisition of this information would thus increase the robustness and portability of NLP systems. This paper describes an acquisition method which makes use of fixed correspondences between derivational affixes and lexical semantic information. One advantage of this method, and of other methods that rely only on surface characteristics of language, is that the necessary input is currently available.

7p bunmoc_1 20-04-2013 33 2   Download

Corpus-based sense disambiguation methods, like most other statistical NLP approaches, suffer from the problem of data sparseness. In this paper, we describe an approach which overcomes this problem using dictionary definitions. Using the definitionbased conceptual co-occurrence data collected from the relatively small Brown corpus, our sense disambiguation system achieves an average accuracy comparable to human performance given the same contextual information.

8p bunmoc_1 20-04-2013 40 2   Download

In m a n y applications of natural language processing it is necessary to determine the likelihood of a given word combination. For example, a speech recognizer m a y need to determine which of the two word combinations "eat a peach" and "eat a beach" is more likely. Statistical NLP methods determine the likelihood of a word combination according to its frequency in a training corpus. However, the nature of language is such that m a n y word combinations are infrequent and do not occur in a given corpus. ...

7p bunmoc_1 20-04-2013 33 1   Download

Chinese sentences are written with no special delimiters such as space to indicate word boundaries. Existing Chinese NLP systems therefore employ preprocessors to segment sentences into words. Contrary to the conventional wisdom of separating this issue from the task of sentence understanding, we propose an integrated model that performs word boundary identification in lockstep with sentence understanding. In this approach, there is no distinction between rules for word boundary identification and rules for sentence understanding. These two functions are combined. ...

3p bunmoc_1 20-04-2013 45 1   Download

This paper describes a new discourse module within our multilingual NLP system. Because of its unique data-driven architecture, the discourse module is language-independent. Moreover, the use of hierarchically organized multiple knowledge sources makes the module robust and trainable using discourse-tagged corpora. Separating discourse phenomena from knowledge sources makes the discourse module easily extensible to additional phenomena.

8p bunmoc_1 20-04-2013 31 2   Download

To resolve or not to resolve, that is the structural ambiguity dilemma. The traditional wisdom is to disambiguate only when it matters in terms of the meaning of the utterance, and to do so using the computationally least costly information. NLP work on PP-attachment has followed this wisdom, and much effort has been focused on formulating structural and lexical strategies for resolving noun-phrase and verb-phrase (NP-PP vs. VP-PP) attachment ambiguity (e.g. [8, 11]).

2p bunmoc_1 20-04-2013 44 4   Download

Although most NLP researchers agree that a level of "logical form" is a necessary step toward the goal of representing the meaning of a sentence, few people agree on the content and form of this level of representation. An even smaller number of people have considered the complex action sentences that are often expressed in taskoriented dialogues. Most existing logical form representations have been developed for single-clause sentences that express assertions about properties or actual actions and in which time is not a main concern.

2p bunmoc_1 20-04-2013 39 1   Download

We present an architecture for the integration of shallow and deep NLP components which is aimed at flexible combination of different language technologies for a range of practical current and future applications. In particular, we describe the integration of a high-level HPSG parsing system with different high-performance shallow components, ranging from named entity recognition to chunk parsing and shallow clause recognition.

8p bunmoc_1 20-04-2013 35 2   Download

Context is used in many NLP systems as an indicator of a term’s syntactic and semantic function. The accuracy of the system is dependent on the quality and quantity of contextual information available to describe each term. However, the quantity variable is no longer fixed by limited corpus resources. Given fixed training time and computational resources, it makes sense for systems to invest time in extracting high quality contextual information from a fixed corpus.

8p bunmoc_1 20-04-2013 33 2   Download

We describe a distributed, modular architecture for platform independent natural language systems. It features automatic interface generation and self-organization. Adaptive (and nonadaptive) voting mechanisms are used for integrating discrete modules. The architecture is suitable for rapid prototyping and product delivery.

4p bunrieu_1 18-04-2013 57 2   Download

Most documents are about more than one subject, but many NLP and IR techniques implicitly assume documents have just one topic. We describe new clues that mark shifts to new topics, novel algorithms for identifying topic boundaries and the uses of such boundaries once identified. We report topic segmentation performance on several corpora as well as improvement on an IR task that benefits from good segmentation. Introduction Dividing documents into topically-coherent sections has many uses, but the primary motivation for this work comes from information retrieval (IR). ...

8p bunrieu_1 18-04-2013 34 3   Download

Non-compositional expressions present a special challenge to NLP applications. We present a method for automatic identification of non-compositional expressions using their statistical properties in a text corpus. Our method is based on the hypothesis that when a phrase is non-composition, its mutual information differs significantly from the mutual informations of phrases obtained by substituting one of the word in the phrase with a similar word.

8p bunrieu_1 18-04-2013 40 2   Download

One area in which artificial neural networks (ANNs) may strengthen NLP systems is in the identification of words under noisy conditions. In order to achieve this benefit when spelling errors or spelling variants are present, variable-length strings of symbols must be converted to ANN input/output form--fixed-length arrays of numbers. A common view in the neural network community has been that different forms of input/output representations have negligible effect on ANN performance.

3p bunrieu_1 18-04-2013 47 2   Download

Much effort has been put into computational lexicons over the years, and most systems give much room to (lexical) semantic data. However, in these systems, the effort put on the study and representation of lexical items to express the underlying continuum existing in 1) language vagueness and polysemy, and 2) language gaps and mismatches, has remained embryonic.

7p bunrieu_1 18-04-2013 37 3   Download

Chinese word segmentation is the first step in any Chinese NLP system. This paper presents a new algorithm for segmenting Chinese texts without making use of any lexicon and hand-crafted linguistic resource. The statistical data required by the algorithm, that is, mutual information and the difference of t-score between characters, is derived automatically from raw Chinese corpora. The preliminary experiment shows that the segmentation accuracy of our algorithm is acceptable.

7p bunrieu_1 18-04-2013 48 2   Download

In this paper we examine how the differences in modelling between different data driven systems performing the same NLP task can be exploited to yield a higher accuracy than the best individual system. We do this by means of an experiment involving the task of morpho-syntactic wordclass tagging. Four well-known tagger generators (Hidden Markov Model, Memory-Based, Transformation Rules and Maximum Entropy) are trained on the same corpus data. After comparison, their outputs are combined using several voting strategies and second stage classifiers. ...

7p bunrieu_1 18-04-2013 48 5   Download

Separable verbs are verbs with prefixes which, depending on the syntactic context, can occur as one word written together or discontinuously. They occur in languages such as German and Dutch and constitute a problem for NLP because they are lexemes whose forms cannot always be recognized by dictionary lookup on the basis of a text word. Conventional solutions take a mixed lexical and syntactic approach. In this paper, we propose the solution offered by Word Manager, consisting of string-based recognition by means of rules of types also required for periphrastic inflection and clitics. ...

5p bunrieu_1 18-04-2013 43 2   Download