Evaluation methodology

Xem 1-20 trên 81 kết quả Evaluation methodology
  • Despite the rising interest in developing grammatical error detection systems for non-native speakers of English, progress in the field has been hampered by a lack of informative metrics and an inability to directly compare the performance of systems developed by different researchers. In this paper we address these problems by presenting two evaluation methodologies, both based on a novel use of crowdsourcing.

    pdf6p hongdo_1 12-04-2013 17 2   Download

  • In this paper, I discuss issues pertinent to the design of a task-based evaluation methodology for a spoken machine translation (MT) system processing human to human communication rather than human to machine communication. I claim that system mediated human to human communication requires new evaluation criteria and metrics based on goal complexity and the speaker's prioritization of goals. ystem

    pdf4p bunrieu_1 18-04-2013 18 2   Download

  • The GIVE Challenge is a recent shared task in which NLG systems are evaluated over the Internet. In this paper, we validate this novel NLG evaluation methodology by comparing the Internet-based results with results we collected in a lab experiment. We find that the results delivered by both methods are consistent, but the Internetbased approach offers the statistical power necessary for more fine-grained evaluations and is cheaper to carry out.

    pdf4p hongphan_1 15-04-2013 31 1   Download

  • Obtaining large volumes of inference knowledge, such as entailment rules, has become a major factor in achieving robust semantic processing. While there has been substantial research on learning algorithms for such knowledge, their evaluation methodology has been problematic, hindering further research. We propose a novel evaluation methodology for entailment rules which explicitly addresses their semantic properties and yields satisfactory human agreement levels. The methodology is used to compare two state of the art learning algorithms, exposing critical issues for future progress. ...

    pdf8p hongvang_1 16-04-2013 13 1   Download

  • many groups and individuals have been motivated to consider the potential for producing ethanol. Across the country, farmer cooperatives, rural development coalitions, bio-energy advocates and others have gathered to explore the process and prospects for developing ethanol production facilities. In many cases these efforts have resulted in the successful development of ethanol plants.

    pdf52p lulanphuong 22-03-2012 38 7   Download

  • Partial evaluation technology continues to grow and mature. ACM SIGPLANsponsored conferences and workshops have provided a forum for researchers to share current results and directions of work. Partial evaluation techniques are being used in commercially available compilers (for example the Chez Scheme system). They are also being used in industrial scheduling systems (see Augustsson's article in this volume), they have been incorporated into popular commercial products (see Singh's article in this volume), and they are the basis of methodologies for implementing domain-specific languages....

    pdf445p hotmoingay3 09-01-2013 25 3   Download

  • In Vietnam, project cycle management (PCM) of road investment projects consists of investment preparation, implementation, construction, and operation processes. The postevaluation of projects during operation has not yet been considered through PCM in a systematic and effective manner. This paper discussed project management issues of road infrastructure projects in Vietnam. Then the paper introduced the post-evaluation process for integrating into Vietnam’s PCM using the PCM methodology developed by Foundation for Advanced Studies on International Development (FASID).

    pdf16p uocvong07 27-10-2015 13 3   Download

  • How might one evaluate the In order to take steps towards establishing a methodology for evaluating Natural Language systems, we relative contributions of each of these factors or comconducted a case study. We attempt to evaluate two pare two approaches to the same problem? different approaches to anaphoric processing in disIn order to take steps towards establishing a course by comparing the accuracy and coverage of methodology for doing this type of comparison, we two published algorithms for finding the co-specifiers conducted a case study. ...

    pdf11p bungio_1 03-05-2013 16 2   Download

  • It is not always clear how the differences in intrinsic evaluation metrics for a parser or classifier will affect the performance of the system that uses it. We investigate the relationship between the intrinsic evaluation scores of an interpretation component in a tutorial dialogue system and the learning outcomes in an experiment with human users. Following the PARADISE methodology, we use multiple linear regression to build predictive models of learning gain, an important objective outcome metric in tutorial dialogue.

    pdf11p bunthai_1 06-05-2013 17 2   Download

  • In recent years there has been a growing interest in crowdsourcing methodologies to be used in experimental research for NLP tasks. In particular, evaluation of systems and theories about persuasion is difficult to accommodate within existing frameworks.

    pdf9p nghetay_1 07-04-2013 11 1   Download

  • The idea of “nugget pyramids” has recently been introduced as a refinement to the nugget-based methodology used to evaluate answers to complex questions in the TREC QA tracks. This paper examines data from the 2006 evaluation, the first large-scale deployment of the nugget pyramids scheme. We show that this method of combining judgments of nugget importance from multiple assessors increases the stability and discriminative power of the evaluation while introducing only a small additional burden in terms of manual assessment. ...

    pdf8p hongvang_1 16-04-2013 21 1   Download

  • In this paper we present a methodology for extracting subcategorisation frames based on an automatic LFG f-structure annotation algorithm for the Penn-II Treebank. We extract abstract syntactic function-based subcategorisation frames (LFG semantic forms), traditional CFG categorybased subcategorisation frames as well as mixed function/category-based frames, with or without preposition information for obliques and particle information for particle verbs.

    pdf8p bunbo_1 17-04-2013 18 1   Download

  • Lexical-semantic resources are used extensively for applied semantic inference, yet a clear quantitative picture of their current utility and limitations is largely missing. We propose system- and application-independent evaluation and analysis methodologies for resources’ performance, and systematically apply them to seven prominent resources. Our findings identify the currently limited recall of available resources, and indicate the potential to improve performance by examining non-standard relation types and by distilling the output of distributional methods. ...

    pdf9p bunthai_1 06-05-2013 21 1   Download

  • Interpreting, like playing chess, is a game of problem solving, evaluation, critical thinking, intuition and forecasting. Every game is different and each game is a challenge, which requires interpreters, to unceasingly develop knowledge and experience. It is disciplined study and repeated practice of many techniques and skills that bring victory to the interpreter. Apart from basic requirements of language mastery and culture sensitivity, there are quite a few skills that need to be acquired for successful interpreting. One of them is note-taking skill....

    pdf61p hattieubenho2009 13-11-2012 145 66   Download

  • Designing for the disabled is about making buildings accessible to and usable by people with disabilities. Universal design is about making buildings safe and convenient for all their users, including people with disabilities. A theme of this book is the similarities and differences of the two, between their correspondences and affinities on the one hand, and their discordancies and diverse methodologies on the other.

    pdf129p badao259 12-12-2009 166 60   Download

  • This book provides an authoritative outline of current mass appraisal techniques being used internationally and in-depth research into state-ofthe- art developments that are likely to permeate the industry over the next decade.

    pdf346p toanktxd 18-05-2011 202 45   Download

  • Forensic Engineering Investigation is a compendium of the investigative methodologies used by engineers and scientific investigators to evaluate some of the more common types of failures and catastrophic events. In essence, the book provides analyses and methods for determining how an entity was damaged and when that damage may have legal consequences. The material covers 21 common types of failures, catastrophic events, and losses that forensic engineers routinely assess.

    pdf50p ken333 28-06-2012 60 20   Download

  • Many experts from the field of IT Service Management have assisted in putting together this first edition of The Guide to IT Service Management Volume I. Without these authors, who have done a lot of work to formulate their knowledge and insights and put them down on paper, a book like this would not be possible. I owe these authors my gratitude. The names of the authors are mentioned in their respective contributions, but you can be assured that many more were involved in the writing and evaluating of the final texts....

    pdf849p phoebe75 01-02-2013 29 7   Download

  • Advances in patient management have often been closely linked to the development of critical quantitative analysis methods. Flow cytometry is such an important methodology. It can be applied to individual cells or organelles allowing investigators interested in obtaining information about the functional properties of cells to assess the differences among cells in a heterogeneous cell preparation or between cells from separate samples.

    pdf214p wqwqwqwqwq 09-07-2012 23 6   Download

  • The project was characterised by high uncertainty, since neither cost nor time could be accurately estimated. Times of completion were based on probabilities: optimistic, pessimistic, and most likely. This led to what has come to be known as the programme evaluation review technique (PERT). Later a new methodology known as project planning and scheduling (PPS) was introduced in the private sector. PPS required realistic estimates of cost and time, and was considered more definitive than PERT.

    pdf0p tainhacmienphi 25-02-2013 22 6   Download

CHỦ ĐỀ BẠN MUỐN TÌM

Đồng bộ tài khoản