Building Web Reputation Systems- P1

Chia sẻ: Cong Thanh | Ngày: | Loại File: PDF | Số trang:15

0
54
lượt xem
4
download

Building Web Reputation Systems- P1

Mô tả tài liệu
  Download Vui lòng tải xuống để xem tài liệu đầy đủ

Building Web Reputation Systems- P1:Today’s Web is the product of over a billion hands and minds. Around the clock and around the globe, people are pumping out contributions small and large: full-length features on Vimeo, video shorts on YouTube, comments on Blogger, discussions on Yahoo! Groups, and tagged-and-titled Del.icio.us bookmarks. User-generated content and robust crowd participation have become the hallmarks of Web 2.0.

Chủ đề:
Lưu

Nội dung Text: Building Web Reputation Systems- P1

  1. Building Web Reputation Systems F. Randall Farmer and Bryce Glass Beijing • Cambridge • Farnham • Köln • Sebastopol • Taipei • Tokyo
  2. Building Web Reputation Systems by F. Randall Farmer and Bryce Glass Copyright © 2010 F. Randall Farmer and Bryce Glass. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://my.safaribooksonline.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Editor: Mary E. Treseler Indexer: Ellen Troutman Zaig Production Editor: Loranah Dimant Cover Designer: Karen Montgomery Copyeditor: Genevieve d’Entremont Interior Designer: David Futato Proofreader: Loranah Dimant Illustrator: Robert Romano Printing History: March 2010: First Edition. Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc. Building Web Reputation Systems, the image of a Pionus parrot, and related trade dress are trademarks of O’Reilly Media, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a trademark claim, the designations have been printed in caps or initial caps. While every precaution has been taken in the preparation of this book, the publisher and authors assume no responsibility for errors or omissions, or for damages resulting from the use of the information con- tained herein. TM This book uses RepKover™, a durable and flexible lay-flat binding. ISBN: 978-0-596-15979-5 [M] 1267644619
  3. Table of Contents Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Part I. Reputation Defined and Illustrated 1. Reputation Systems Are Everywhere . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 An Opinionated Conversation 3 People Have Reputations, but So Do Things 4 Reputation Takes Place Within a Context 4 We Use Reputation to Make Better Decisions 5 The Reputation Statement 6 Explicit: Talk the Talk 6 Implicit: Walk the Walk 6 The Minimum Reputation Statement 6 Reputation Systems Bring Structure to Chaos 7 Reputation Systems Deeply Affect Our Lives 8 Local Reputation: It Takes a Village 8 Global Reputation: Collective Intelligence 9 FICO: A Study in Global Reputation and Its Challenges 10 Web FICO? 12 Reputation on the Web 12 Attention Doesn’t Scale 13 There’s a Whole Lotta Crap Out There 13 People Are Good. Basically. 15 The Reputation Virtuous Circle 17 Who’s Using Reputation Systems? 18 Challenges in Building Reputation Systems 19 Related Subjects 20 Conceptualizing Reputation Systems 20 2. A (Graphical) Grammar for Reputation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 The Reputation Statement and Its Components 22 iii
  4. Reputation Sources: Who or What Is Making a Claim? 23 Reputation Claims: What Is the Target’s Value to the Source? On What Scale? 24 Reputation Targets: What (or Who) Is the Focus of a Claim? 25 Molecules: Constructing Reputation Models Using Messages and Processes 26 Messages and Processes 26 Reputation Model Explained: Vote to Promote 28 Building on the Simplest Model 29 Complex Behavior: Containers and Reputation Statements As Targets 30 Solutions: Mixing Models to Make Systems 33 From Reputation Grammar to… 35 Part II. Extended Elements and Applied Examples 3. Building Blocks and Reputation Tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Extending the Grammar: Building Blocks 39 The Data: Claim Types 39 Processes: Computing Reputation 46 Routers: Messages, Decisions, and Termination 54 Practitioner’s Tips: Reputation Is Tricky 57 The Power and Costs of Normalization 57 Liquidity: You Won’t Get Enough Input 58 Bias, Freshness, and Decay 60 Implementer’s Notes 64 Making Buildings from Blocks 65 4. Common Reputation Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Simple Models 67 Favorites and Flags 68 This-or-That Voting 69 Ratings 70 Reviews 70 Points 71 Karma 72 Combining the Simple Models 74 User Reviews with Karma 75 eBay Seller Feedback Karma 78 Flickr Interestingness Scores for Content Quality 82 When and Why Simple Models Fail 89 Party Crashers 89 Keep Your Barn Door Closed (but Expect Peeking) 91 Reputation from Theory to Practice 94 iv | Table of Contents
  5. Part III. Building Web Reputation Systems 5. Planning Your System’s Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 Asking the Right Questions 97 What Are Your Goals? 98 Content Control Patterns 102 Incentives for User Participation, Quality, and Moderation 111 Consider Your Community 121 Better Questions 123 6. Objects, Inputs, Scope, and Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 The Objects in Your System 125 Architect, Understand Thyself 126 What Makes for a Good Reputable Entity? 129 Determining Inputs 131 User Actions Make Good Inputs 131 But Other Types of Inputs Are Important, Too 134 Good Inputs 135 Common Explicit Inputs 136 Common Implicit Inputs 143 Constraining Scope 146 Context Is King 146 Limit Scope: The Rule of Email 148 Applying Scope to Yahoo! EuroSport Message Board Reputation 149 Generating Reputation: Selecting the Right Mechanisms 150 The Heart of the Machine: Reputation Does Not Stand Alone 151 Common Reputation Generation Mechanisms and Patterns 151 Practitioner’s Tips: Negative Public Karma 161 Draw Your Diagram 163 7. Displaying Reputation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 How to Use a Reputation: Three Questions 165 Who Will See a Reputation? 166 To Show or Not to Show? 166 Personal Reputations: For the Owner’s Eyes Only 169 Personal and Public Reputations Combined 171 Public Reputations: Widely Visible 171 Corporate Reputations Are Internal Use Only: Keep Them Hush-hush 172 How Will You Use Reputation to Modify Your Site’s Output? 172 Reputation Filtering 173 Reputation Ranking and Sorting 173 Reputation Decisions 174 Table of Contents | v
  6. Content Reputation Is Very Different from Karma 175 Content Reputation 176 Karma 176 Reputation Display Formats 178 Reputation Display Patterns 180 Normalized Score to Percentage 180 Points and Accumulators 182 Statistical Evidence 183 Levels 185 Ranked Lists 189 Practitioner’s Tips 194 Leaderboards Considered Harmful 194 Going Beyond Displaying Reputation 196 8. Using Reputation: The Good, The Bad, and the Ugly . . . . . . . . . . . . . . . . . . . . . . . . 197 Up with the Good 198 Rank-Order Items in Lists and Search Results 199 Content Showcases 200 Down with the Bad 204 Configurable Quality Thresholds 205 Expressing Dissatisfaction 206 Out with the Ugly 207 Reporting Abuse 207 Teach Your Users How to Fish 209 Inferred Reputation for Content Submissions 210 A Private Conversation 212 Course-Correcting Feedback 213 Reputation Is Identity 214 On the User Profile 216 At the Point of Attribution 219 To Differentiate Within Listings 220 Putting It All Together 221 9. Application Integration, Testing, and Tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 Integrating with Your Application 223 Implementing Your Reputation Model 223 Rigging Inputs 225 Applied Outputs 225 Beware Feedback Loops! 226 Plan for Change 226 Testing Your System 227 Bench Testing Reputation Models 228 Environmental (Alpha) Testing Reputation Models 229 vi | Table of Contents
  7. Predeployment (Beta) Testing Reputation Models 230 Tuning Your System 232 Tuning for ROI: Metrics 232 Tuning for Behavior 236 Tuning for the Future 241 Learning by Example 241 10. Case Study: Yahoo! Answers Community Content Moderation . . . . . . . . . . . . . . . . 243 What Is Yahoo! Answers? 243 A Marketplace for Questions and Yahoo! Answers 244 Attack of the Trolls 245 Built with Reputation 247 Avengers Assemble! 248 Initial Project Planning 249 Setting Goals 249 Who Controls the Content? 250 Incentives 250 The High-Level Project Model 251 Objects, Inputs, Scope, and Mechanism 252 The Objects 252 Limiting Scope 254 An Evolving Model 255 Displaying Reputation 268 Who Will See the Reputation? 268 How Will the Reputation Be Used to Modify Your Site’s Output? 268 Is This Reputation for a Content Item or a Person? 268 Using Reputation: The…Ugly 269 Application Integration, Testing, and Tuning 270 Application Integration 270 Testing Is Harder Than You Think 270 Lessons in Tuning: Users Protecting Their Power 271 Deployment and Results 273 Operational and Community Adjustments 274 Adieu 276 A. The Reputation Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 B. Related Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 Table of Contents | vii
  8. Preface What Is This Book About? Today’s Web is the product of over a billion hands and minds. Around the clock and around the globe, people are pumping out contributions small and large: full-length features on Vimeo, video shorts on YouTube, comments on Blogger, discussions on Yahoo! Groups, and tagged-and-titled Del.icio.us bookmarks. User-generated content and robust crowd participation have become the hallmarks of Web 2.0. But the booming popularity of social media has brought with it a whole new set of challenges for those who create websites and online communities, as well as for the users who consume those sites: • Problems of scale (how to manage—and present—an overwhelming inflow of user contributions) • Problems of quality (how to tell the good stuff from the bad) • Problems of engagement (how to reward contributors in a way that keeps them coming back) • Problems of moderation (how to “stamp out” the worst stuff quickly and efficiently) Reputation systems can provide an effective solution to all of these problems. Reputation and Karma: Two Simple Definitions Reputation Information used to make a value judgment about an object or person Karma The reputation(s) for a user ix
  9. What is reputation in an online community? In its broadest sense, reputation is infor- mation used to make a value judgment about an object or a person. There are potentially many components to reputation. For example, karma is a reputation score for a user in a community, and it may be an aggregation of answers to the following: • How long has this person been a member of the community? • What types of activities has she engaged in? • How well has she performed at them? • What do other people think about this person? Having access to a person’s reputation might help you make better-informed judg- ments. Judgments like: • Can I trust this person? • Should I transact with him? • Is it worth my time to listen to him? The same principles of reputation can also be applied to that person’s individual contributions—the video, photo, or textual content that he’s authored. Content rep- utation can answer questions such as: • What’s the best video in this category? • What are the most interesting photos submitted today? • Should I bother reading this blog entry? • Which comments in a thread are truly worth my time? For the site operator, reputation is at its most powerful when karma (people reputation) is used in concert with content reputation to increase your understanding of your online community and the relative value of its participants and contents. For the community itself, reputation is relevance. To the site, reputation represents value and return on investment. So what does a reputation system do? It powers the whole process: it monitors the community, makes note of the actions that community members take, assesses the community’s response to those actions, and keeps a running tally of the history of it all. (Optionally, the system may then show the results of these calculations back to users, enabling them to make judgments about the people and things they encounter.) Reputation systems are the underlying mechanisms behind some of the best-known consumer websites, including Amazon, eBay, Digg, Slashdot, Facebook, and especially media and gaming sites like iTunes and Xbox Live. But, of course, developing a successful reputation system is not easy. Our experience is that reputation is one part each of engineering, social science, psychology, and eco- nomics. Social media architects and designers must pay careful attention to multiple factors: the context in which the community is situated, the incentives they’d like to x | Preface
  10. provide contributors (which often demand a nuanced understanding of the actions they want to encourage and discourage), and the effects that decisions made early in the design process may have on the community spirit downstream. Building Web Reputation Systems provides a complete, soup-to-nuts process for de- signing and developing your own community’s reputation program. In a guided fash- ion, you will start with the hard questions: • How can reputation enhance my business? The community? • What are the right types of behaviors to encourage? • What are the right objects (people, things?) to accrue reputation? • What types of actions should count toward a reputation? and then you’ll learn how to build reputation models to accurately reflect all of these inputs. To do so, you’ll need a reputation grammar, which we’ve developed and applied to many successful reputation systems used across the Web. We also provide well-diagrammed reputation patterns for common web needs (“Digg- style” voting systems, ratings and reviews, simple karma) and some extended reputa- tion system reviews and a case study based on actual reputation deployments at industry-leading social sites, including Yahoo!, Flickr, and eBay. Why Write a Book About Reputation? We wrote this book because we saw how critical reputation has become to the survival and growth of the Web. Though there are many academic research papers on specific generational algorithms and social effects of reputation systems, we couldn’t find a single book that put it all in context—describing it as a separate domain of knowledge, complete with a grammar, emerging best-use patterns, and recurring antipatterns. Until now. Our extensive experience creating and deploying several reputation systems across a wide range of social media at Yahoo! provided much of our base of knowledge for this book. We add a comprehensive review of other existing applications to venture forth into unexplained territory and offer a working definition of web reputation systems: a survivor’s guide for those who would follow. We hope this book will improve the quality of conversation about reputation systems, and help improve the quality of products that are using them. Preface | xi
  11. Who Should Read This Book This book and the supporting blog and wiki (http://buildingreputation.com) are targeted at several audiences: Primary audience Anyone who is building, operating, or participating in a website or online applica- tion that leverages user-generated content: social networking, ratings, votes, re- views, comments, posts, uploads, games, etc. You’ll need to understand reputation systems in order to maximize your user experience and search engine performance, and streamline your operational expenses. A must-have reference for System architects, product managers, community support staff, or user interface designers tasked with designing a reputation system for an online community. (Or any variant thereof: a social collaboration tool, voting mechanism, or a ratings- and-reviews experience.) Also useful to Game designers who deal with reward and incentive systems for social games or any technologist in a role that requires some familiarity with reputation systems who’d like to become conversant in their design and operations. Successful social media design is hard. Many companies and organizations understand that. Rewarding points for user participation may generate activity and build audience, but site owners should be wary of possible negative impact on content quality. They need this book to help them choose the right reputation model for their site and help them evaluate the health of their deployed social collaboration system. This book assumes that you have a rudimentary understanding of application logic, can read basic flowcharts, create content for the Web, and have direct experience in- teracting with users on socially active websites. No programming or web engineering experience is required, though you’ll be aided through some of the more technical aspects of this book by some passing familiarity with programmatic thinking. Organization of This Book This book is broken into three main parts: Reputation Defined and Illustrated, Exten- ded Elements and Applied Examples, and Building Web Reputation Systems. It was written to be consumed sequentially, but if you are already experienced in developing websites heavy with user-generated content, you may be able to move more quickly through Part I. xii | Preface
  12. Part I: Reputation Defined and Illustrated Chapter 1 briefly introduces basic concepts and terminology of reputation and sets reputation systems in their proper historical context. With this context in mind, Chap- ter 2 goes on to define a complete grammar for thinking about reputation and provides a graphical language to clearly describe reputation systems for analysis and development. Part II: Extended Elements and Applied Examples In Chapter 3, the primitive reputation grammar elements described in Part I are combined into higher-level building blocks that can be used as-is for simple reputation systems or combined to create more complex ones. In Chapter 4, we show how the grammar and blocks we’ve described can be used to elucidate many of the reputation systems currently deployed across the Web, including Digg, eBay, and Flickr. In this part, practitioners’ tips become a regular feature of most chapters; the lessons we’ve learned are not to be missed—even by the casual reader. Part III: Building Web Reputation Systems The last section, comprising more than half the book, goes beyond cookie-cutter rep- utation needs to offer detailed advice on designing, building, deploying, and operating a custom reputation system. The project begins not by drawing a model or screen mock- ups, but by answering the three big questions posed in Chapter 5 that define and limit your reputation choices. With those answers, Chapter 6 helps to identify your appli- cation’s objects, methods, and inputs that power your reputation. With all of this in mind, you can start drawing the reputation model. Chapter 7 helps you decide how to display reputation, and Chapter 8 describes other common uses for reputation, such as providing search relevance. At this point, the reputation model is designed, and the screen mocks are ready. Next up is implemen- tation, testing, and deployment, for which strategies and tips are covered in Chap- ter 9. We wrap it all up with a case study in Chapter 10. This chapter ties the entire book together, revisiting each of the steps outlined in detail with a single concrete example. It is a successful reputation model that provides inspiration, many insights, and several notes of caution. Appendix A is intended for technical readers, such as system architects and platform engineers. It describes the reputation framework—an execution environment for mul- tiple reputation models. The Yahoo! Reputation Platform is described in detail as an example of a framework that can scale to tens of thousands of transactions per second but had to make some functional trade-offs that are quite illustrative. Appendix B contains references for related materials for further reading, most of which are available on the Web. Preface | xiii
Đồng bộ tài khoản