Don't Be Afraid to Tell Us Our Baby is Ugly

I had coffee with a friend recently to discuss social media marketing and as I was updating her on what we were up to at Trada, she asked, “How do you collect feedback from users in Trada?”

For product managers like me, user feedback is the most fundamental aspect of our jobs. Unfortunately for many of us, it too often ends up being the most ignored.  It’s never purposely ignored – it’s something that tends to happen slowly, over months and years. We get busy with other projects and feel the pressure to get new features to market quickly. Before you know it, you’ve lost touch with your users and as a result they tend to lose touch with your product.

At Trada, as an organization we work hard to not let this happen. Even before Trada was “officially” founded, we started interviewing paid search experts and advertisers (many of whom were one and the same) to find out what they found difficult about paid search advertising and determine if Trada’s fundamental model of a paid search marketplace made sense. Niel and I had both run paid search campaigns in the past and we had our opinions about what advertisers struggled with, but it was important to hear directly from them to help us fully understand the problem we were trying to address.

Shortly after we released our first prototype of the Trada marketplace, we held a series of meetings (we call them Contradas) with paid search experts (we call them optimizers) to learn what they liked and what was lacking in the marketplace. Contradas typically included 8 – 10 optimizers, lasted for 6 – 8 weeks, with a 2 – 3 hour meeting once a week in the evening. Every Trada employee attended. We’d get together with our optimizers and eat pizza and drink beer with them while they told us what they liked, and what was lacking in the marketplace. This allowed us to iterate the UI and functionality incredibly fast. Since then, we’ve also held Contradas with advertisers to gather their feedback as well. Over the course of the last 14 months or so, we’ve held 4 Contradas – about once a quarter. It’s about time for another one.

If it sounds amazing that we could get 8 – 10 highly sophisticated and successful paid search experts to spend a significant amount of time each week for up to two months to review a (then) prototype application without being paid a dime, I have to admit that I was pretty surprised myself. The key is the structure of Contradas and the type of people we recruit to participate.

FREE PIZZA + NEW FEATURES = HAPPY CONTRADA!

To start, people who are great at what they do also tend to be intellectually interested in new technologies in their industry. If you do your part, they’ll give you an amazing amount of their time and energy. But they won’t do it forever – nor should they be expected to. For that reason, our Contradas were specifically time bound and our agendas were very clear and efficient, so members knew exactly what they were committing to, and for how long.

Also, people get excited when they see their ideas become a reality in a short period of time. Notice that I didn’t say, “when users validated or rejected our ideas”. While we definitely came to our users with prototypes to react to, we were careful to ask them open ended questions that often revealed completely new ideas that we’d never thought of before. If we’d simply said, “Here’s a new feature, do you like it?” we wouldn’t have uncovered those pearls that came out as a result of open ended questions and conversation. We also encouraged criticism. We opened every meeting by telling everyone, “Don’t be afraid to tell us our baby is ugly.” Rather than getting defensive of criticism, we embraced it as an opportunity to make the Trada marketplace better. We also offered free counseling to Trada employees to help them deal with the rejection.

When we received feedback from optimizers, we built it into the product very quickly – usually within a week, or two at the most. We could pull this off because, as I mentioned earlier, everyone working at Trada attended Contrada meetings. Engineers heard directly from users, which allowed them to understand the use case quickly and start development without delay. The big caveat to doing this is that you have to have a rock-star engineering team who can produce high-quality work at breakneck speed. We’re fortunate to have that dev team at Trada.

Understand that not everything went perfectly. One of the things we learned quickly is that it’s very difficult to have Contrada members who are remote and dial/web conference into the meetings. There was simply too much conversation and white boarding going on in the room for them to be able to keep up with what was being said and provide the value they wanted to.

Also, I found that when the entire staff is involved in the meetings and they’re as passionate as Contrada members about certain topics, it’s easy to get overly involved in the conversation and start talking/debating employee-to-employee. This wastes valuable time with users and you don’t learn anything talking amongst yourselves. When we realized this was starting to happen, we made a conscious effort to recognize it and put a stop to it. You have all week to fight, er, debate with your teammates – when end users are present stay focused on them!

Finally, remember that a little “thank you” goes a long way. One of our Contradas took place near Thanksgiving, so we took the opportunity to buy everyone pies from a great little bakery across the street from our office. Another took place near Halloween, so we stuffed jack-o-lantern buckets with goodies and a book. Each only cost a few dollars, but went a long ways toward showing Contrada members how much we appreciated their time.

Now that our advertisers and optimizers are literally spread around the globe, it’s a bit more challenging to gather feedback in the same way we do with Contradas. Just today we had a discussion about which tools to implement into our application to gather user feedback and different ways we can include optimizers and advertisers in our product development process. We hope our optimizers and advertisers continue to give us the outstanding feedback they have in the past – and we’ll continue to work hard to make user feedback a core part of our development process.