Don’t Fool Yourself …, Or Why Evaluation Is Important

7 Apr

In 2006, when I was attending Stanford University School of Education for my master’s degree, we were working on our master’s project. One of our professors, who is an expert on evaluation, came to our class to discuss evaluation of our projects. He started by saying:

“Don’t fool yourself into believing that you are developing a great product.”

It was a shocker for us as we all completely believed that we were indeed on our way to developing great products. My project was intended to make it easy for school children to view 3D animations on 3D monitors on concepts that were difficult to visualize in 2D, for example, the rotations and revolutions of planets and the atomic structure of molecules. It was the greatest idea of all time. Why did I need to conduct an evaluation for such a simple, yet powerful concept?

We learned a lot that day, especially not overestimating the usefulness and effectiveness of our ideas. Unfortunately, we didn’t have much time to evaluate our projects so we completed our program and set out, starry-eyed. to change the world.

However, people in real world who develop products and services don’t always care about the users of those products and services. They come up with innovative ideas (or so they think), and start working on the ideas with the premise that they are developing great products. Case in point: Windows 8, iTunes Ping, Google Plus. They ask the users what they think about those ideas, and, thanks to confirmation bias, come back with user feedback that confirm the greatness of their ideas.

They fool themselves into believing that they are developing a great product. They don’t want to look at the analytics because that would disprove their hypothesis. And I have been as guilty of this crime as anyone else.

The right way: Startup companies

Startup companies too start with the premise that they are working on the world’s greatest idea. And they are right. If they don’t think that way, they wouldn’t risk everything to work on that idea. They spend years developing, launching, evaluating, and iterating on their idea. Their best friend? Usage analytics, which gives them insights into how many people are using their products, where, and when. How many people are visiting what parts of the product more? What macro-level trends are visible? Does making a minor change in a color shade increase or decrease the usage of a page or a field?

More than 90% of startup companies die within two years of launching because of several reasons. But they all know how useful and effective their products are (or not) thanks to analytics data. Analytics do not always save an idea but they help fail early, fail fast, fail often, and help people learn and move on. They can fool themselves but not for long. And that’s why evaluation is critical to developing great products.

There’s Something About Print Books

7 Mar

In the middle of the fifteen century, the fist major book – Gutenberg Bible – was printed. With the advent of the printing presses, an advancement in printing technologies and chemicals, and the dawn of the industrial revolution, the print books proliferated. Later, some predicted that the Internet was going to kill books, but the number of books published is increasing every year. However, the sale of print books is in decline, mainly because more and more people are reading books on their electronic devices, mostly tablets, e-readers, and smartphones.


The current generation
The current generation, the so called Digital Natives, are considered to be extremely tech-savvy. They are supposed to own all kinds of electronic devices and live their life in a digital world. The printed paper should be anathema to them. We don’t expect them to use printed books.

So what’s the reality? It’s totally different!

I’ve spoken with more than 5o undergraduate students taking business courses in the past few months. The first thing they do when classes start is buy a print book, either a new book, a used book, or a rented one. Many of them are aware of the existence of ebooks, but they all prefer print books. Cost is a factor but not for everyone. I wonder what’s going on?

I asked some students why they prefer print books and got some vague answers. I’ve even asked myself why I prefer print books. I like the touch of the book, I like to display them on my bookshelf, it’s romantic, it’s nostalgic, but I haven’t been able to come up with a better answer. Do we prefer print books because we have used a print book most of our lives? Is it just a habit that’s difficult to break? People break old habits if something compelling enters their lives. Does it mean the current ebooks are not compelling enough?

Some theories
Humans have evolved to be responsive to visual and tactile signals as these traits helped them survive in the wild for thousands of years. Books are tangible things, ebooks are not. Flipping the pages in a physical book is easy and visual/spatial. Research shows that it’s easy to retain information when using a print book versus an ebook. Reading a print book is faster than reading on a screen. Taking notes and highlighting is easier with a print book. There is no battery to charge, no worries of damaging it. The ebooks are mostly a replica of print books.

Some people say that the next generation who are growing up using digital devices since childhood will be more inclined to use ebooks. However, the schools still use mostly paper books and homework though the tests are computer-based and kids use some ebooks and digital learning tools. It might take another 15-20 years before college students use only ebooks.

There must be something about print books.

Related readings
Why Printed Books Will Never Die
E-Reading Rises as Device Ownership Jumps

When The Buck Doesn’t Stop

7 Feb

This week, Satya Nadella was picked to be the next CEO of Microsoft to succeed Steve Ballmer. Being a fellow Indian, I’m happy for him. However, an MBA replacing another MBA is not an appealing idea to me.

Steve Ballmer has been criticized for missing the search (Google) and mobile (Apple) revolutions. Microsoft’s stock price has been stagnant for a decade. The tech media has been pronouncing MS dead for years now, while MS keeps posting bigger and bigger revenues and profits, quarter after quarter.

Was Steve Ballmer a bad CEO? I don’t think so. He was brought in by Bill Gates to increase the revenues and profits of a public company, which he did quite spectacularly. But he is not a product or design guy like Steve Jobs. He didn’t have vision, but he had strategy. He was not innovative, but he knew how to make money.

When nobody questions why the design of a product sucks, when there are too many cooks in the kitchen, you get Windows 8. Surprisingly, MS got it incredibly right with Windows phone, but Windows 8 has an equally confusing design.

Will Satya Nadella be a numbers and strategy guy or a design and innovation guy? We’ll see soon. When there is nobody at the top with design sensibilities who can question the design of a product, the product usually sucks. In every industry, category, and market. Because the buck doesn’t stop with anyone.

Related Readings
Why Designers Leave

The Revolution That Wasn’t: Part 2

28 Jan

I recently read some comments about MOOCs.

Doubts About MOOCs Continue to Rise, Survey Finds: Babson Survey Research Group, Pearson and the Sloan Consortium

The findings, released in a report on Wednesday, reveal a growing skepticism among academic leaders about the promise of MOOCs. The report also suggests that conventional, tuition-based online education is still growing, although not as swiftly as in past years.

The article – Top Issues Facing Higher Education In 2014 on, ends with:

You may observe a notable omission from this list: MOOCs. Increasing awareness of their limitations for certain audiences combined with a feeling of “enough already” will make these yesterday’s news in 2014.

The pioneer of MOOCs, Stanford Professor and founder of Udacity, predicted in 2012:

In 50 years, there will be only 10 institutions in the world delivering higher education and Udacity has a shot at being one of them.

Recently, however, he changed his opinion of MOOCs:

“I’d aspired to give people a profound education–to teach them something substantial. But the data was at odds with this idea.”
“We were on the front pages of newspapers and magazines, and at the same time, I was realizing, we don’t educate people as others wished, or as I wished. We have a lousy product. (emphasis mine)”

I have taken a few MOOC courses on Udacity, Coursera, Stanford Venture Labs, and NovoEd. I’m enrolled in one or two courses all the time, which I complete at my own pace. I believe it’s a great but overhyped idea, and MOOCs are not a replacement for traditional students and universities.

Meanwhile, I stand by my take on MOOCs two years ago – Are MOOCs A Disruptive Innovation?

Suggested Readings:

Design Of Everyday Things: Faucets

16 Jan

Bad design
A few years ago, at Cincinnati/Norther Kentucky airport restroom, I saw a sign above the wash basins that read something like this - “The faucet may not operate if your clothes are dark colored”. This was a classic example of a designer hellbent on incorporating the sensor technology into a product without thinking about the end user. If you wore dark clothes and wanted to wash your hands, you were out of luck! Fortunately, those faucets are gone now.

Another bad design
In my office, the faucets in the restroom dispense water if you place your hands below them, which is fine if you want the water to flow for about 10 seconds after which the faucet shuts off. It intelligently assumes that nobody needs water flowing for more than 10 seconds. You have to wave your hands again to get the water flowing again. Or, as someone told me, you can tap the top of the faucets to get water. There is absolutely no way for anyone to figure out that tapping the faucet can get you water! An absence of signifiers.
faucetThe faucets look elegant; however, there is no way to control the temperature of water. If you want cold water or hot water, you are out of luck! The same faucets on the first floor restroom dispense almost hot water. And there is no way to reduce the flow of hot water or control the temperature of water.

Good design
I see a great design of faucets in my gym. The faucets look beautiful with the sensor clearly visible at the front so I know where I should place my hands. There is small lever to control the flow of hot water. The water keeps flowing as long as you need it. Simple, obvious, and elegant. Presence of signifiers.

The Meaningful Life Is A Road Worth Traveling

8 Jan

A Stanford research project explored the key differences between lives of happiness and meaningfulness. While the two are similar, dramatic differences exist – and one should not underestimate the power of meaningfulness. “The quest for meaning is a key part of what makes us human,” the researchers concluded.

Read the article on Stanford News

Reading List – 2013

30 Dec

Here is a list of books I’ve read in 2013.

  1. A New Culture of Learning: Cultivating the Imagination for a World of Constant Change by Douglas Thomas and John Seely Brown
  2. To Sell Is Human: The Surprising Truth About Moving Others by Daniel H. Pink
  3. The Signal and the Noise: Why So Many Predictions Fail — But Some Don’t by Nate Silver
  4. You’re Doing It Wrong!: How to Improve Your Life by Fixing Everyday Tasks You (and Everyone Else) Are Totally Screwing Up by Lee Thornton
  5. The New Rules of Lifting for Life by Lou Schuler and Alwyn Cosgrove
  6. The Start-up of You: Adapt to the Future, Invest in Yourself, and Transform Your Career by Reid Hoffman and Ben Casnocha
  7. Who Moved My Cheese by Spencer Johnson
  8. Top Dog: The Science of Winning and Losing by Po Bronson and Ashley Merryman
  9. Nurture Shock: New Thinking About Children by Po Bronson and Ashley Merryman
  10. The Happiness Project: Or, Why I Spent a Year Trying to Sing in the Morning, Clean My Closets, Fight Right, Read Aristotle, and Generally Have More Fun by Gretchen Rubin
  11. The Innovator’s DNA: Mastering the Five Skills of Disruptive Innovators by Jeff Dyer, Hal Gregersen, and Clayton M. Christensen
  12. Earn the Right to Win: How Success in Any Field Starts with Superior Preparation by Tom Coughlin and David Fisher
  13. Moneyball by Michael Lewis
  14. Another Day In Cubicle Paradise: A Dilbert Book by Scott Adams
  15. The Secrets of Happy Families: Improve Your Mornings, Rethink Family Dinner, Fight Smarter, Go Out and Play, and Much More
    by Bruce Feiler
  16. Alone Together: Why We Expect More from Technology and Less from Each Other by Sherry Turkle
  17. The Art of Doing: How Superachievers Do What They Do and How They Do It So Well by Camille Sweeney and Josh Gosfield
  18. The Best Advice I Ever Got: Lessons from Extraordinary Lives by Katie Couric
  19. For the Love of Physics: From the End of the Rainbow to the Edge of Time – A Journey Through the Wonders of Physics by Walter Lewin
  20. Space Chronicles: Facing the Ultimate Frontier by Neil DeGrasse Tyson
  21. What’s Math Got to Do with It?: How Parents and Teachers Can Help Children Learn to Love Their Least Favorite Subject by Jo Boaler
  22. The Inmates Are Running the Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity by Alan Cooper
  23. Rework by Jason Fried
  24. Total Recall: How the E-Memory Revolution Will Change Everything by Gordon Bell and Jim Gemmell
  25. The Kite Runner by Khalid Hosseini
  26. A Thousand Splendid Suns by Khalid Hosseini
  27. And The Mountains Echoed by Khalid Hosseini
  28. Inferno by Dan Brown
  29. Still Foolin’ ‘Em: Where I’ve Been, Where I’m Going, and Where the Hell Are My Keys? by Billy Crystal
  30. The Way We’re Working Isn’t Working: The Four Forgotten Needs That Energize Great Performance by Tony Schwartz, Jean Gomes, and Catherine McCarthy
  31. David and Goliath: Underdogs, Misfits, and the Art of Battling Giants by Malcolm Gladwell
  32. How to Fail at Almost Everything and Still Win Big: Kind of the Story of My Life by by Scott Adams
  33. Creative Confidence: Unleashing the Creative Potential Within Us All by Tom Kelley and David Kelley
  34. Brilliant Blunders: From Darwin to Einstein – Colossal Mistakes by Great Scientists That Changed Our Understanding of Life and the Universe by Mario Livio
  35. Diary of a Wimpy Kid by Jeff Kinney
  36. Mission to Mars: My Vision for Space Exploration by Buzz Aldrin and Leonard David
  37. Eat Move Sleep: How Small Choices Lead to Big Changes by Tom Rath
  38. Tell Me the Truth, Doctor: Easy-to-Understand Answers to Your Most Confusing and Critical Health Questions by Richard Besser

Reading List – 2012


Get every new post delivered to your Inbox.