The value of innovation : knowing, proving, and showing the value of innovation and creativity : a step by step guide to impact and ROI measurement / Jack J. Phillips and Patricia Pulliam Phillips.

By: Phillips, Jack J [author]
Contributor(s): Pulliam Phillips, Patricia [author]
Language: English Publisher: Hoboken, NJ : Wiley-Scrivener, 2018Description: 1 online resource (359 pages)Content type: text Media type: computer Carrier type: online resourceISBN: 9781119242413Subject(s): InnovationsmanagementGenre/Form: Electronic books.DDC classification: 658.4063 Online resources: Full text available at Wiley Online Library Click here to view
Contents:
TABLE OF CONTENTS Preface xix Acknowledgements xxiii About the Authors xxv 1 The Importance and Challenges of Innovation 1 Innovation Hype 2 Articles 2 Books 2 Jobs 3 Speeches 3 Experience 3 The Realities of Innovation 4 Innovation is Not New 4 Innovation is Necessary for Survival 5 Innovation is Equated with Success 5 Innovation is Truly Global 6 Consumers and Investors Expect Innovation 6 Innovation is Often Disruptive 6 Innovation is Not a Single Event 7 Little Ideas Often Make a Big Difference 7 Innovation Comes in Many Types and Forms 8 Innovation Spans Many Different Horizons 8 Trouble in Paradise: The Misconceptions 9 Misconception 1: Small Companies are More Innovative 10 Misconception 2: Uncontested Markets are Good for Innovation 10 Misconception 3: Spending More on R&D Increases Innovation 10 Misconception 4: Companies Need More Radical Innovation 10 Misconception 5: Open Innovation Turbocharges R&D 11 Misconception 6: R&D Needs to be More Relevant 11 Misconception 7: Wall Street Rewards Innovation 11 Innovation Challenges 12 Innovation is Expensive 12 Managing Innovation is Difficult 13 An Innovation Culture is Necessary for Success 13 Innovation Requires Many Personas 14 Innovation Success Rates Need to Improve 16 The Value of Innovation is Unclear 16 Final Thoughts 17 2 Status and Concerns about Innovation Measurement 19 Innovation: Definition, Models, and Measures 20 Sources of Innovation 21 Measurement Shifts 23 Measurement Shifts are Common 23 Value Perception. 24 The Search for Money 24 Hoping, Knowing, Proving, and Showing Value 25 Innovation is Systematic 25 Macro View of Measurement 27 Industry Level Measures 29 Company Level 30 Concerns about Company Level Measures 30 Micro View of Measurement 32 Final Thoughts 34 3 The Case for a New System 35 Innovation: A Cost or an Investment? 36 The Value of Innovation: A Summary 38 Intangibles and the Fear of not Investing 38 Relationship Between Variables 39 ROI Studies 41 Types of Data 41 Inputs 41 Reaction and Planned Action 42 Learning 43 Application and Implementation 43 Impact 44 Return on Investment 44 How Does Your Current System Stack Up? 45 Focus of Use 45 Standards 47 Contents vii Types of Data 47 Dynamic Adjustments 47 Connectivity 48 Approach 48 Conservative Nature 48 Simplicity 48 Theoretical Foundation 49 Acceptance 49 Using Design Thinking to Deliver and Measure Results 49 Start with Why: Aligning Projects with the Business 50 Make it Feasible: Selecting the Right Solution 51 Expect Success: Designing for Results 52 Make it Matter: Designing for Input, Reaction, and Learning 52 Make it Stick: Designing for Application and Impact 53 Make it Credible: Measuring Results and Calculating ROI 53 Tell the Story. Communicating Results to Key Stakeholders 54 Optimize the Results: Using Black Box Thinking to Increase Funding 54 Requirements for the Value of Innovation: A Measurement Process 55 ROI Measurement Methodology 56 Terminology: Projects, Solutions, Participants . . . 57 Final Thoughts 57 4 Introducing the ROI Methodology 59 The ROI Methodology 60 Types of Data 60 The Initial Analysis 63 The ROI Process Model 65 Planning the Evaluation 66 Evaluation Purpose 66 Feasibility 67 Data Collection Plan 68 ROI Analysis Plan 68 Project Plan 71 Collecting Data 71 Isolating the Effects of the Project 72 Converting Data to Monetary Values 72 Identifying Intangible Benefits 73 Tabulating Project Costs 74 Calculating the Return on Investment 74 Reporting 75 Operating Standards and Philosophy 75 Implementing and Sustaining the Process 76 Benefits of This Approach 76 Aligning with Business 77 Validating the Value Proposition 77 Improving Processes 77 Enhancing the Image and Building Respect 78 Improving Support 78 Justifying or Enhancing Budgets 78 Building a Partnership with Key Executives 79 Earning a Seat at the Table 79 Final Thoughts 79 5 Aligning Innovation Projects to the Organization 81 Creating Business Alignment 83 The Purpose of Alignment 83 Disciplined Analysis 84 Determining the Potential Payoff 86 Obvious Versus not-so-obvious Payoff 87 The Cost of a Problem 89 The Value of an Opportunity 90 To Forecast or not to Forecast? 90 Determining Business Needs 90 The Opportunity 91 Hard Data Measures 91 Soft Data Measures 92 Tangible versus Intangible Benefits: A Better Approach 93 Impact Data Sources 94 Determining Performance Needs 95 Analysis Techniques 95 A Sensible Approach 95 Determining Learning Needs 96 Determining Preference Needs 97 Case Study: Southeast Corridor Bank 98 Payoff Needs 98 Business Needs 99 Performance Needs 99 The Solution 100 Learning Needs 101 Preference Needs 102 Developing Objectives for Innovation Projects 102 Reaction Objectives 102 Learning Objectives 103 Application and Implementation Objectives 103 Impact Objectives 104 ROI Objectives 105 Final Thoughts 106 6 Collecting Data Along Chain of Impact with a Toolbox of Methods 107 Questionnaires and Surveys 109 Types of Questions and Statements 109 Design Issues 110 A Detailed Example 111 Improving the Response Rate for Questionnaires and Surveys 113 Using Interviews 120 Types of Interviews 121 Interview Guidelines 121 Using Focus Groups 122 Applications for Evaluation 122 Guidelines 123 Measuring with Tests 124 Measuring with Simulation 124 Task Simulation 124 Role-Playing/Skill Practice 125 Using Observation 125 Guidelines for Effective Observation 125 Observation Methods 127 Using Action Plans 128 Using Action Plans Successfully 129 Advantages/Disadvantages of Action Plans 132 Using Performance Contracts 133 Monitoring Business Performance Data 134 Existing Measures 134 Developing New Measures 135 Selecting the Appropriate Method for Each Level 135 Type of Data 135 Participants’ Time for Data Input 136 Manager Time for Data Input 136 Cost of Method 137 Disruption of Normal Work Activities 137 Accuracy of Method 137 Utility of an Additional Method 137 Cultural Bias for Data Collection Method 138 Final Thoughts 138 7 Measuring Reaction and Perceived Value 139 Why Measure Reaction and Perceived Value? 140 Customer Satisfaction 141 Immediate Adjustments 141 Predictive Capability 141 Important but not Exclusive 142 Sources of Data 143 Participants 143 Participant Managers 143 Other Team Members 143 Internal or External Customers 144 Project Leaders and Team Members 144 Sponsors and Senior Managers 144 Records and Previous Studies 144 Areas of Feedback 145 Data Collection Timing 146 Data Collection Methods 146 Questionnaires and Surveys 146 Interviews 147 Focus Groups 147 Using Reaction Data 147 Final Thoughts 148 8 Measuring Learning 149 Why Measure Learning and Confidence? 150 The Importance of Intellectual Capital 151 The Learning Organization 152 The Compliance Issue 152 The Use and Development of Competencies 152 The Role of Learning in Innovation Projects 153 The Challenges and Benefits of Measuring Learning 153 Challenges 154 The Benefits of Measuring Learning 154 Measurement Issues 155 Project Objectives 155 Typical Measures 155 Timing 156 Data Collection Methods 157 Questionnaires and Surveys 157 Performance Tests 157 Technology and Task Simulations 158 Case Studies 159 Role-Playing and Skill Practice 159 Informal Assessments 159 Administrative Issues 160 Reliability and Validity 160 Consistency 161 Pilot Testing 161 Scoring and Reporting 161 Using Learning Data 162 Final Thoughts 162 9 Measuring Application and Implementation 163 Why Measure Application and Implementation? 165 Information Value 165 Project Focus 166 Problems and Opportunities 166 Reward Effectiveness 167 Challenges 167 Linking with Learning 168 Building Data Collection into the Project 168 Ensuring a Sufficient Amount of Data 168 Addressing Application Needs at the Outset 169 Measurement Issues 169 Methods 169 Objectives 170 Areas of Coverage 170 Data Sources 170 Timing 170 Responsibilities 171 Data Collection Methods 171 Using Questionnaires to Measure Application and Implementation 172 Using Interviews, Focus Groups, and Observation 172 Using Action Plans 172 Barriers to Application 174 Application Data Use 174 Final Thoughts 175 10 Measuring Impact 177 Why Measure Business Impact? 178 Higher-Level Data 178 A Business Driver for Projects 179 “The Money” for Sponsors 179 Easy to Measure 180 Collecting Effective Impact Measures 180 Data Categories 180 Metric Fundamentals 181 Identifying Specific Measures Linked to Projects 182 Business Performance Data Monitoring 183 Identify Appropriate Measures 184 Convert Current Measures to Usable Ones 184 Develop New Measures 184 Data Collection Methods 185 Using Action Plans to Develop Business Impact Data 185 Using Performance Contracts to Measure Business Impact 187 Using Questionnaires to Collect Business Impact Measures 189 Measuring the Hard to Measure 190 Everything Can Be Measured 190 Perceptions are Important 191 Every Measure Can Be Converted to Money, but not Every Measure Should Be 191 Special Emphasis on Intangibles 192 Final Thoughts 192 11 Isolating the Effects of Innovation 193 Why the Concern over this Issue? 196 Reality 196 Myths 196 Preliminary Issues 198 Chain of Impact 198 Identify other Factors: A First Step 199 Isolation Methods 200 Control Groups 200 Trend Line Analysis 203 Mathematical Modeling 205 Estimates 206 Participants’ Estimate of Impact 206 Manager’s Estimate of Impact 209 Customer Estimates of Project Impact 209 Internal or External Expert Estimates 210 Estimate Credibility: The Wisdom of Crowds 210 Calculate the Impact of other Factors 212 Select the Technique 213 Final Thoughts 214 12 Converting Data to Money 215 Why Convert Data to Monetary Values? 217 Value Equals Money 217 Impact is More Understandable 217 Converting to Monetary Values is Similar to Budgeting 218 Monetary Value is Vital to Organizational Operations 218 Monetary Values are Necessary to Understand Problems and Cost Data 219 Key Steps in Converting Data to Money 219 Standard Monetary Values 222 Converting output Data to Money 222 Calculating the Cost of Quality 223 Converting Employee Time Using Compensation 227 Finding Standard Values 228 When Standard Values are not Available 229 Using Historical Costs from Records 229 Time 229 Availability 230 Access 230 Accuracy 230 Using Input from Experts 230 Using Values from External Databases 231 Linking with other Measures 232 Using Estimates from Participants 233 Using Estimates from the Management Team 233 Using Project Staff Estimates 234 Technique Selection and Finalizing Value 234 Choose a Technique Appropriate for the Type of Data 235 Move from Most Accurate to Least Accurate 235 Consider Source Availability 235 Use the Source with the Broadest Perspective on the Issue 236 Use Multiple Techniques When Feasible 236 Apply the Credibility Test 236 Consider the Possibility of Management Adjustment 238 Consider the Short-Term/Long-Term Issue 238 Consider an Adjustment for the Time Value of Money 239 Final Thoughts 239 13 Addressing Intangibles 241 Why Intangibles are Important 244 Intangibles are the Invisible Advantage 244 We are Entering the Intangible Economy 245 More Intangibles are Converted to Tangibles 245 Intangibles Drive Innovation Projects 246 The Magnitude of the Investment 246 Measuring and Analyzing Intangibles 246 Measuring the Intangibles 247 Converting to Money 249 Identifying and Collecting Intangibles 251 Analyzing Intangibles 252 Final Thoughts 253 14 Measuring ROI 255 Why Monitor Costs and Measure ROI? 258 Fundamental Cost Issues 259 Fully Loaded Costs 259 Costs Reported without Benefits 260 Develop and Use Cost Guidelines 261 Sources of Costs 262 Prorated versus Direct Costs 262 Employee Benefits Factor 263 Specific Costs to Include 263 Initial Analysis and Assessment 264 Development of Project Solutions 264 Acquisition Costs 264 Implementation Costs 264 Maintenance and Monitoring 265 Support and Overhead 265 Evaluation and Reporting 265 The ROI Calculation 265 Benefits/Costs Ratio 266 ROI Formula 267 ROI Objective 269 Other ROI Measures 270 Payback Period (Breakeven Analysis) 270 Discounted Cash Flow 271 Internal Rate of Return 271 Final Thoughts 272 15 Forecasting Value, Including ROI 273 Why Forecast ROI? 278 Expensive Projects 279 High Risks and Uncertainty 279 Postproject Comparison 279 Compliance 280 The Trade-offs of Forecasting 280 Preproject ROI Forecasting 282 Basic Model 282 Basic Steps to Forecast ROI 283 Sources of Expert Input 287 Securing Input 287 Conversion to Money 288 Estimate Project Costs 288 Case Study 289 Forecasting with a Pilot Program 293 Forecasting with Reaction Data 293 Case Study: Forecasting ROI from Reaction Data 294 Use of the Data 295 Forecasting Guidelines 296 Final Thoughts 299 16 Reporting Results 301 The Importance of Communicating Results? 303 Communication is Necessary to Make Improvements 303 Communication is Necessary to Explain the Contribution 303 Communication is a Politically Sensitive Issue 304 Different Audiences Need Different Information 304 Principles of Communicating Results 304 Communication Must Be Timely 305 Communication Should Be Targeted to Specific Audiences 305 Media Should Be Carefully Selected 305 Communication Should Be Unbiased and Modest in Tone 305 Communication Must Be Consistent 306 Make the Message Clear 306 Testimonials Must Come from Respected Individuals 306 The Audience’s Bias of the Project Will Influence the Communication Strategy 306 Storytelling is Essential 307 The Process for Communicating Results 307 The Need for Communication 308 The Communication Plan 309 The Audience for Communications 309 Basis for Selecting the Audience 311 Information Development: The Impact Study 312 Media Selection 312 Meetings 312 Interim and Progress Reports 314 Routine Communication Tools 315 E-mail and Electronic Media 316 Project Brochures and Pamphlets 316 Case Studies 316 Delivering the Message 316 Routine Feedback on Project Progress 317 Storytelling 319 Presentation of Results to Senior Management 320 Reactions to Communication 322 Final Thoughts 322 17 Implementing and Sustaining ROI 323 Why is this Important? 324 Resistance is Always Present 326 Implementation is the Key to Success 326 Consistency is Needed 326 Efficiency 326 Value is Maximized 326 Implementing the Process: Overcoming Resistance 327 Review Current Results 328 Developing Roles and Responsibilities 328 Identifying a Champion 329 Developing the ROI Leader 329 Establishing a Task Force 329 Assigning Responsibilities 330 Establishing Goals and Plans 331 Setting Evaluation Targets 331 Developing a Plan for Implementation 332 Revising or Developing Policies and Guidelines 332 Preparing the Project Team 334 Involving the Project Team 334 Using ROI as a Learning and Project Improvement Tool 334 Teaching the Team 334 Initiating ROI Studies 335 Selecting the Initial Project 335 Developing the Planning Documents 335 Reporting Progress 336 Establishing Discussion Groups 336 Preparing the Sponsors and Management Team 336 Removing Obstacles 337 Dispelling Myths 337 Delivering Bad News 338 Using the Data 338 Monitoring Progress 339 Final Thoughts 340 References 343 Index 351
Summary: Innovation is the life blood of practically every organization. Innovation drives growth, development, and prosperity for many organizations and geographical areas. Sometimes, innovation thrives within a certain geographical location or in certain organizations that are known for their innovative approaches. This outstanding new volume will demonstrate how to measure the success of innovation in all types of organizations. In the last decade, there have been tremendous investments in creativity and innovations sponsored by companies, cities, states, countries, universities, NGO’s, and even non-profits. With the magnitude of emphasis on creativity and innovation, the sponsors and key stakeholders will demand to know the value of these programs. The Value of Innovation: Measuring the Impact and ROI in Creativity and Innovation Programs will show step-by-step how to measure the impact and the ROI of innovation and creativity programs. The process collects six types of data: reaction, learning, application, impact, ROI, and intangibles. Data are collected analyzed and reported using a systematic, logic model. Conservative standards create results that are both CEO and CFO friendly. This proven process has been used now in 5000 organizations and this new book adapts the method directly to this critical area of innovation, showing examples and case studies.
Tags from this library: No tags from this library for this title. Log in to add tags.
    Average rating: 0.0 (0 votes)

ABOUT THE AUTHOR
Jack J. Phillips, PhD is Chairman of ROI Institute and a world-renowned expert on measurement and evaluation. Phillips provides consulting services for Fortune 500 companies and workshops for major conference providers worldwide. Phillips is also the author or editor of more than a hundred books and hundreds of articles. His work has been featured in the Wall Street Journal, Bloomberg Businessweek, Fortune, and on CNN. Patricia Pulliam Phillips, PhD is an internationally recognized author, consultant, and President and CEO of ROI Institute, Inc. Phillips provides consulting services to organizations worldwide. She helps organizations build capacity in the ROI Methodology by facilitating the ROI certification process and teaching the ROI Methodology through workshops and graduate level courses. She has been an author or editor of more than a hundred books and numerous journals.
PERMISSIONS
Request permission to reuse

Includes bibliographical references and index.

TABLE OF CONTENTS
Preface xix

Acknowledgements xxiii

About the Authors xxv

1 The Importance and Challenges of Innovation 1

Innovation Hype 2

Articles 2

Books 2

Jobs 3

Speeches 3

Experience 3

The Realities of Innovation 4

Innovation is Not New 4

Innovation is Necessary for Survival 5

Innovation is Equated with Success 5

Innovation is Truly Global 6

Consumers and Investors Expect Innovation 6

Innovation is Often Disruptive 6

Innovation is Not a Single Event 7

Little Ideas Often Make a Big Difference 7

Innovation Comes in Many Types and Forms 8

Innovation Spans Many Different Horizons 8

Trouble in Paradise: The Misconceptions 9

Misconception 1: Small Companies are More Innovative 10

Misconception 2: Uncontested Markets are Good for Innovation 10

Misconception 3: Spending More on R&D Increases Innovation 10

Misconception 4: Companies Need More Radical Innovation 10

Misconception 5: Open Innovation Turbocharges R&D 11

Misconception 6: R&D Needs to be More Relevant 11

Misconception 7: Wall Street Rewards Innovation 11

Innovation Challenges 12

Innovation is Expensive 12

Managing Innovation is Difficult 13

An Innovation Culture is Necessary for Success 13

Innovation Requires Many Personas 14

Innovation Success Rates Need to Improve 16

The Value of Innovation is Unclear 16

Final Thoughts 17

2 Status and Concerns about Innovation Measurement 19

Innovation: Definition, Models, and Measures 20

Sources of Innovation 21

Measurement Shifts 23

Measurement Shifts are Common 23

Value Perception. 24

The Search for Money 24

Hoping, Knowing, Proving, and Showing Value 25

Innovation is Systematic 25

Macro View of Measurement 27

Industry Level Measures 29

Company Level 30

Concerns about Company Level Measures 30

Micro View of Measurement 32

Final Thoughts 34

3 The Case for a New System 35

Innovation: A Cost or an Investment? 36

The Value of Innovation: A Summary 38

Intangibles and the Fear of not Investing 38

Relationship Between Variables 39

ROI Studies 41

Types of Data 41

Inputs 41

Reaction and Planned Action 42

Learning 43

Application and Implementation 43

Impact 44

Return on Investment 44

How Does Your Current System Stack Up? 45

Focus of Use 45

Standards 47

Contents vii

Types of Data 47

Dynamic Adjustments 47

Connectivity 48

Approach 48

Conservative Nature 48

Simplicity 48

Theoretical Foundation 49

Acceptance 49

Using Design Thinking to Deliver and Measure Results 49

Start with Why: Aligning Projects with the Business 50

Make it Feasible: Selecting the Right Solution 51

Expect Success: Designing for Results 52

Make it Matter: Designing for Input, Reaction, and Learning 52

Make it Stick: Designing for Application and Impact 53

Make it Credible: Measuring Results and Calculating ROI 53

Tell the Story. Communicating Results to Key Stakeholders 54

Optimize the Results: Using Black Box Thinking to Increase Funding 54

Requirements for the Value of Innovation: A Measurement Process 55

ROI Measurement Methodology 56

Terminology: Projects, Solutions, Participants . . . 57

Final Thoughts 57

4 Introducing the ROI Methodology 59

The ROI Methodology 60

Types of Data 60

The Initial Analysis 63

The ROI Process Model 65

Planning the Evaluation 66

Evaluation Purpose 66

Feasibility 67

Data Collection Plan 68

ROI Analysis Plan 68

Project Plan 71

Collecting Data 71

Isolating the Effects of the Project 72

Converting Data to Monetary Values 72

Identifying Intangible Benefits 73

Tabulating Project Costs 74

Calculating the Return on Investment 74

Reporting 75

Operating Standards and Philosophy 75

Implementing and Sustaining the Process 76

Benefits of This Approach 76

Aligning with Business 77

Validating the Value Proposition 77

Improving Processes 77

Enhancing the Image and Building Respect 78

Improving Support 78

Justifying or Enhancing Budgets 78

Building a Partnership with Key Executives 79

Earning a Seat at the Table 79

Final Thoughts 79

5 Aligning Innovation Projects to the Organization 81

Creating Business Alignment 83

The Purpose of Alignment 83

Disciplined Analysis 84

Determining the Potential Payoff 86

Obvious Versus not-so-obvious Payoff 87

The Cost of a Problem 89

The Value of an Opportunity 90

To Forecast or not to Forecast? 90

Determining Business Needs 90

The Opportunity 91

Hard Data Measures 91

Soft Data Measures 92

Tangible versus Intangible Benefits: A Better Approach 93

Impact Data Sources 94

Determining Performance Needs 95

Analysis Techniques 95

A Sensible Approach 95

Determining Learning Needs 96

Determining Preference Needs 97

Case Study: Southeast Corridor Bank 98

Payoff Needs 98

Business Needs 99

Performance Needs 99

The Solution 100

Learning Needs 101

Preference Needs 102

Developing Objectives for Innovation Projects 102

Reaction Objectives 102

Learning Objectives 103

Application and Implementation Objectives 103

Impact Objectives 104

ROI Objectives 105

Final Thoughts 106

6 Collecting Data Along Chain of Impact with a Toolbox of Methods 107

Questionnaires and Surveys 109

Types of Questions and Statements 109

Design Issues 110

A Detailed Example 111

Improving the Response Rate for Questionnaires and Surveys 113

Using Interviews 120

Types of Interviews 121

Interview Guidelines 121

Using Focus Groups 122

Applications for Evaluation 122

Guidelines 123

Measuring with Tests 124

Measuring with Simulation 124

Task Simulation 124

Role-Playing/Skill Practice 125

Using Observation 125

Guidelines for Effective Observation 125

Observation Methods 127

Using Action Plans 128

Using Action Plans Successfully 129

Advantages/Disadvantages of Action Plans 132

Using Performance Contracts 133

Monitoring Business Performance Data 134

Existing Measures 134

Developing New Measures 135

Selecting the Appropriate Method for Each Level 135

Type of Data 135

Participants’ Time for Data Input 136

Manager Time for Data Input 136

Cost of Method 137

Disruption of Normal Work Activities 137

Accuracy of Method 137

Utility of an Additional Method 137

Cultural Bias for Data Collection Method 138

Final Thoughts 138

7 Measuring Reaction and Perceived Value 139

Why Measure Reaction and Perceived Value? 140

Customer Satisfaction 141

Immediate Adjustments 141

Predictive Capability 141

Important but not Exclusive 142

Sources of Data 143

Participants 143

Participant Managers 143

Other Team Members 143

Internal or External Customers 144

Project Leaders and Team Members 144

Sponsors and Senior Managers 144

Records and Previous Studies 144

Areas of Feedback 145

Data Collection Timing 146

Data Collection Methods 146

Questionnaires and Surveys 146

Interviews 147

Focus Groups 147

Using Reaction Data 147

Final Thoughts 148

8 Measuring Learning 149

Why Measure Learning and Confidence? 150

The Importance of Intellectual Capital 151

The Learning Organization 152

The Compliance Issue 152

The Use and Development of Competencies 152

The Role of Learning in Innovation Projects 153

The Challenges and Benefits of Measuring Learning 153

Challenges 154

The Benefits of Measuring Learning 154

Measurement Issues 155

Project Objectives 155

Typical Measures 155

Timing 156

Data Collection Methods 157

Questionnaires and Surveys 157

Performance Tests 157

Technology and Task Simulations 158

Case Studies 159

Role-Playing and Skill Practice 159

Informal Assessments 159

Administrative Issues 160

Reliability and Validity 160

Consistency 161

Pilot Testing 161

Scoring and Reporting 161

Using Learning Data 162

Final Thoughts 162

9 Measuring Application and Implementation 163

Why Measure Application and Implementation? 165

Information Value 165

Project Focus 166

Problems and Opportunities 166

Reward Effectiveness 167

Challenges 167

Linking with Learning 168

Building Data Collection into the Project 168

Ensuring a Sufficient Amount of Data 168

Addressing Application Needs at the Outset 169

Measurement Issues 169

Methods 169

Objectives 170

Areas of Coverage 170

Data Sources 170

Timing 170

Responsibilities 171

Data Collection Methods 171

Using Questionnaires to Measure Application and Implementation 172

Using Interviews, Focus Groups, and Observation 172

Using Action Plans 172

Barriers to Application 174

Application Data Use 174

Final Thoughts 175

10 Measuring Impact 177

Why Measure Business Impact? 178

Higher-Level Data 178

A Business Driver for Projects 179

“The Money” for Sponsors 179

Easy to Measure 180

Collecting Effective Impact Measures 180

Data Categories 180

Metric Fundamentals 181

Identifying Specific Measures Linked to Projects 182

Business Performance Data Monitoring 183

Identify Appropriate Measures 184

Convert Current Measures to Usable Ones 184

Develop New Measures 184

Data Collection Methods 185

Using Action Plans to Develop Business Impact Data 185

Using Performance Contracts to Measure Business Impact 187

Using Questionnaires to Collect Business Impact Measures 189

Measuring the Hard to Measure 190

Everything Can Be Measured 190

Perceptions are Important 191

Every Measure Can Be Converted to Money, but not Every Measure Should Be 191

Special Emphasis on Intangibles 192

Final Thoughts 192

11 Isolating the Effects of Innovation 193

Why the Concern over this Issue? 196

Reality 196

Myths 196

Preliminary Issues 198

Chain of Impact 198

Identify other Factors: A First Step 199

Isolation Methods 200

Control Groups 200

Trend Line Analysis 203

Mathematical Modeling 205

Estimates 206

Participants’ Estimate of Impact 206

Manager’s Estimate of Impact 209

Customer Estimates of Project Impact 209

Internal or External Expert Estimates 210

Estimate Credibility: The Wisdom of Crowds 210

Calculate the Impact of other Factors 212

Select the Technique 213

Final Thoughts 214

12 Converting Data to Money 215

Why Convert Data to Monetary Values? 217

Value Equals Money 217

Impact is More Understandable 217

Converting to Monetary Values is Similar to Budgeting 218

Monetary Value is Vital to Organizational Operations 218

Monetary Values are Necessary to Understand

Problems and Cost Data 219

Key Steps in Converting Data to Money 219

Standard Monetary Values 222

Converting output Data to Money 222

Calculating the Cost of Quality 223

Converting Employee Time Using Compensation 227

Finding Standard Values 228

When Standard Values are not Available 229

Using Historical Costs from Records 229

Time 229

Availability 230

Access 230

Accuracy 230

Using Input from Experts 230

Using Values from External Databases 231

Linking with other Measures 232

Using Estimates from Participants 233

Using Estimates from the Management Team 233

Using Project Staff Estimates 234

Technique Selection and Finalizing Value 234

Choose a Technique Appropriate for the Type of Data 235

Move from Most Accurate to Least Accurate 235

Consider Source Availability 235

Use the Source with the Broadest Perspective on the Issue 236

Use Multiple Techniques When Feasible 236

Apply the Credibility Test 236

Consider the Possibility of Management Adjustment 238

Consider the Short-Term/Long-Term Issue 238

Consider an Adjustment for the Time Value of Money 239

Final Thoughts 239

13 Addressing Intangibles 241

Why Intangibles are Important 244

Intangibles are the Invisible Advantage 244

We are Entering the Intangible Economy 245

More Intangibles are Converted to Tangibles 245

Intangibles Drive Innovation Projects 246

The Magnitude of the Investment 246

Measuring and Analyzing Intangibles 246

Measuring the Intangibles 247

Converting to Money 249

Identifying and Collecting Intangibles 251

Analyzing Intangibles 252

Final Thoughts 253

14 Measuring ROI 255

Why Monitor Costs and Measure ROI? 258

Fundamental Cost Issues 259

Fully Loaded Costs 259

Costs Reported without Benefits 260

Develop and Use Cost Guidelines 261

Sources of Costs 262

Prorated versus Direct Costs 262

Employee Benefits Factor 263

Specific Costs to Include 263

Initial Analysis and Assessment 264

Development of Project Solutions 264

Acquisition Costs 264

Implementation Costs 264

Maintenance and Monitoring 265

Support and Overhead 265

Evaluation and Reporting 265

The ROI Calculation 265

Benefits/Costs Ratio 266

ROI Formula 267

ROI Objective 269

Other ROI Measures 270

Payback Period (Breakeven Analysis) 270

Discounted Cash Flow 271

Internal Rate of Return 271

Final Thoughts 272

15 Forecasting Value, Including ROI 273

Why Forecast ROI? 278

Expensive Projects 279

High Risks and Uncertainty 279

Postproject Comparison 279

Compliance 280

The Trade-offs of Forecasting 280

Preproject ROI Forecasting 282

Basic Model 282

Basic Steps to Forecast ROI 283

Sources of Expert Input 287

Securing Input 287

Conversion to Money 288

Estimate Project Costs 288

Case Study 289

Forecasting with a Pilot Program 293

Forecasting with Reaction Data 293

Case Study: Forecasting ROI from Reaction Data 294

Use of the Data 295

Forecasting Guidelines 296

Final Thoughts 299

16 Reporting Results 301

The Importance of Communicating Results? 303

Communication is Necessary to Make Improvements 303

Communication is Necessary to Explain the Contribution 303

Communication is a Politically Sensitive Issue 304

Different Audiences Need Different Information 304

Principles of Communicating Results 304

Communication Must Be Timely 305

Communication Should Be Targeted to Specific Audiences 305

Media Should Be Carefully Selected 305

Communication Should Be Unbiased and Modest in Tone 305

Communication Must Be Consistent 306

Make the Message Clear 306

Testimonials Must Come from Respected Individuals 306

The Audience’s Bias of the Project Will Influence the Communication Strategy 306

Storytelling is Essential 307

The Process for Communicating Results 307

The Need for Communication 308

The Communication Plan 309

The Audience for Communications 309

Basis for Selecting the Audience 311

Information Development: The Impact Study 312

Media Selection 312

Meetings 312

Interim and Progress Reports 314

Routine Communication Tools 315

E-mail and Electronic Media 316

Project Brochures and Pamphlets 316

Case Studies 316

Delivering the Message 316

Routine Feedback on Project Progress 317

Storytelling 319

Presentation of Results to Senior Management 320

Reactions to Communication 322

Final Thoughts 322

17 Implementing and Sustaining ROI 323

Why is this Important? 324

Resistance is Always Present 326

Implementation is the Key to Success 326

Consistency is Needed 326

Efficiency 326

Value is Maximized 326

Implementing the Process: Overcoming Resistance 327

Review Current Results 328

Developing Roles and Responsibilities 328

Identifying a Champion 329

Developing the ROI Leader 329

Establishing a Task Force 329

Assigning Responsibilities 330

Establishing Goals and Plans 331

Setting Evaluation Targets 331

Developing a Plan for Implementation 332

Revising or Developing Policies and Guidelines 332

Preparing the Project Team 334

Involving the Project Team 334

Using ROI as a Learning and Project Improvement Tool 334

Teaching the Team 334

Initiating ROI Studies 335

Selecting the Initial Project 335

Developing the Planning Documents 335

Reporting Progress 336

Establishing Discussion Groups 336

Preparing the Sponsors and Management Team 336

Removing Obstacles 337

Dispelling Myths 337

Delivering Bad News 338

Using the Data 338

Monitoring Progress 339

Final Thoughts 340

References 343

Index 351

Innovation is the life blood of practically every organization. Innovation drives growth, development, and prosperity for many organizations and geographical areas. Sometimes, innovation thrives within a certain geographical location or in certain organizations that are known for their innovative approaches. This outstanding new volume will demonstrate how to measure the success of innovation in all types of organizations.

In the last decade, there have been tremendous investments in creativity and innovations sponsored by companies, cities, states, countries, universities, NGO’s, and even non-profits. With the magnitude of emphasis on creativity and innovation, the sponsors and key stakeholders will demand to know the value of these programs. The Value of Innovation: Measuring the Impact and ROI in Creativity and Innovation Programs will show step-by-step how to measure the impact and the ROI of innovation and creativity programs. The process collects six types of data: reaction, learning, application, impact, ROI, and intangibles. Data are collected analyzed and reported using a systematic, logic model. Conservative standards create results that are both CEO and CFO friendly. This proven process has been used now in 5000 organizations and this new book adapts the method directly to this critical area of innovation, showing examples and case studies.

There are no comments for this item.

to post a comment.