Benefits and Drawbacks of Questions & Answers

We often receive questions about the benefits and drawbacks of implementing user generated content (UGC) features to websites. Our response is always; it depends. It depends on the type of UGC feature and a few other factors.

Let’s take the example of a question and answer feature like Yahoo’s Answers or Answers.com. This type of feature can generate lots of good and valuable UGC. The UGC can then generate new organic and referred traffic but also returning visitors.

In our example of UGC, questions and answers, it depends on 3 main factors:

1. Your goals with this type of feature.
2. The stage your site is at
3. Your market

Our tests:

Before looking at these 3 factors let’s look at a little research we’ve done twice over the past 6 months for questions and answers. We submitted 20 related topical questions to Yahoo’s Answers and Answers.com and tracked the SERPs of 80 different queries based on:

Example of question: “Where should I go to receive the best low interest car loan in California?”

Example of searches:

1. Exact same search keywords as the questions submitted. Ex: “Where should I go to get the best low interest car loan in California?”.
2. Search with 3 to 4 same keywords of the questions plus a W word (what, when, why, who and how). Ex: “Where is the best interest car loan in Los Angeles?”
3. Some of the search keywords of the questions, except we didn’t use the related W words . Ex: “Best low interest car loan in California”.
4. Search using 3-4 same keywords . Ex: “low interest loan California”.

Our goal was to see the impact of questions and answers on SERPs and related traffic in a very competitive vertical over a few weeks.

Most people usually consider questions and answers as a good tool to get long tail organic traffic. It’s probably true for huge general sites offering questions and answers but in competitive markets the reality can be quite different.

Let see the average SERPs results we got. Interestingly the averages are quite similar in both time period and among the search results.

1- Exact same search (and keywords) as the posted questions

Question example: “Where should I go to receive the best low interest car loan in California?”

Search: “Where should I go to receive the best low interest car loan in California?”

The above results show the posted questions doing very well with high SERP positions during the 1st week and relatively well on week 2 and 3 on Yahoo and disappearing from the SERP on week 6. On Google the questions gets good SERP all the way to week 7 before disappearing from the SERP.

Observation: When the search keywords are so well aligned to the question and the page title we can normally expect these great results. But with only 1-2% of searches matching the exact question you can’t put all your hopes on exact searches.

2- Search with 3-4 keywords of the posted questions (with W word)

Question example: “Where should I go to get the best low interest car loan in California?”

Search example: “Where is the best low interest loan in Southern California?”

The results demonstrate a good position for the 1st and 2nd week on both Yahoo and Google (and also 3rd week for Google). The questions then drop off the SERP pretty fast after week 3 and disappear after week 6.

Observation: Conducting a search with some of the same keywords AND a W word performs well for a shorter amount of time compared to the exact same search (ex. case 1). In general, with about only 5% of searches done with W related keywords betting your organic traffic on questions can be dangerous, unless naturally you’re only targeting this traffic segment.

3- Some of the search keywords of the posted questions (without the W word)

Question example: “Where should I go to get the best low interest car loan in California?”

Search example: “Best low interest car loan in California”

Results show a fairly good position for the 1st  week (SERP of 7 to 16th) but then drop dramatically right before disappearing on week 5 on Yahoo and week 8 on Google.

Observation: Not using the W word in the search has the biggest impact on the SERP even when you have a good amount of keywords. The chance you will rank well for searches not done as a W question is very low.

4- Only 3-4 keywords of the posted questions as a search

Question example: “Where should I go to get the best low interest car loan in California?”

Search example: “low interest loan California”

Our results show a fairly good SERP on Yahoo for week 1 and 2 while they’re not good on Google even on week 1 with a SERP of 40+.

Observation: Doing well on the SERP with only keywords (no W word) is almost impossible in a very competitive market.

Tests’ conclusions:

Using questions and answers in order to acquire organic traffic is quite hard in very competitive markets.

There are a few reasons for this:

Some of your competitors have high page rank and they dominate the SERPs; for exact searches and even when the search is broader and involves a W word and a couple of keywords.

There are so many pages in the SERP for each potential search (sometimes over 1Million) that ranking well is pretty hard even for what could be considered low-hanging fruits. Example: Search query of a couple of keywords found in the question.

Our tests were conducted on sites that have very good page ranks (8). If your site has a lower page rank you can expect even worst results.

Overall, questions and answers can be a good traffic generator feature when you are in a low/medium competitive market or when you have a high page rank and/or a huge volume of questions posted. In the case where you only have a huge volume of questions it will translate into the large majority of your questions not generating any traffic.

Now let’s go back to the initial 3 main factors required to perform well with Questions & Answers.

1. Your goals with this type of feature:

As we’ve seen above if your goal is to generate traffic and you’re in a competitive market, have a low to medium page rank and low volume of questions, the results will be less than minimal. If you’re in a not so competitive market with pretty good page rank and lots of traffic you should do well.

Sites also often launch this type of feature to have returning visitors. As we can see with Quantcast traffic frequency for Yahoo’s Answers‘ and Answers.com‘s , the regulars and addicts are respectively 65% and 49% of their visitors. This means the majority of these sites’ visitors return because they were there before (they are returning visitors), most of them are not new ones coming from the SERPs.

That’s explained by the simple fact they built their user base a long time ago (before the feature was launched in Yahoo’s case and simply a long time ago in Answers.com’s case as they were among the first few who launched Q&A) and most of their traffic is coming from these repeated users. The opposite happened to Google’s Answer; they had the power to dominate the SERPs for any question and build organic traffic and returning users but they simply didn’t have the users base to build upon (more precisely they were not able to tap into their users base to make it work. Crazy uh!).

The traffic balance comes from users who find their site after clicking on their links in the SERP because these sites get good page rank, a high number of questions and visitors.

2. The stage your site is at:

As we started to see in the previous point, if you’re a well-established site with a good ranking and a high volume of visitors, launching a Question & Answer feature is quite easy and brings benefits right away. You can then use your existing high volume traffic to generate lots of questions and content that will then get you more traffic. It becomes a virtuous circle.

Going the other way around by launching the feature and then trying to generate traffic is the long and hard road to take. That will get you into the vicious circle of trying to generate questions from very little traffic. You may even become tempted to try to find “creative” ways to get people to post questions, from rewarding them to maybe even tricking them. You don’t want your site look like an empty mall on a Monday morning. That’s a losing battle; it’s bad for moral and it simply doens’t look right to your visitors. Don’t put your carriage before your horse.

3. Your market:

Your market affects your questions and answers feature planning in 2 broad ways.

Competitive or not: If you’re in a very competitive market, you’ve seen it’s quite hard to succeed to acquire traffic if you’re not well-established, have a high page rank and a huge amount of posted questions.

Narrow vs broad: The second way your market influences your questions & answers is how narrow your market is and how often your visitors will come ask questions. If you have  a site targeting a broad array of topics from the beginning your visitors will have the possibility to come back and ask questions regularly. If your site targets a topic that’s limited in time and scope your visitors won’t be coming back to ask questions often.

For example, if your site is about a car purchase visitors won’t be coming back to your site to ask questions every week or month; this mean you won’t have any addicts or regular visitors for this feature. You will only be able to rely on visitors coming from the SERPs or once in a great while. Again, in a narrow market, launching a question & answers feature before traffic has been built will make you fall into a vicious circle.

Last note

So there you are: you may say, “but isn’t it a chicken and egg thing?” Not really. It’s not about chicken or eggs. It’s about traffic first and foremost.

If you’re a competitive market and unless you already have a large user base, grow your traffic before launching a question and answer feature.

It’s easy to figure out how much traffic you need to acquire before launching it. Simply figure out how many questions you need to get the traffic you want to receive from that feature (lets be modest with 100). Then extrapolate it when you know that about 8%* of your visitors will post questions. There you are, you need 1250 visitors. You now have a pretty good idea about the amount of traffic you need to get your 100 visitors. Naturally that’s if you’re not in a competitive market and have a good page rank. If you are in a competitive market you can easily multiply that by 10 (12,500 questions to get 100 organic visitors) if you don’t have a great page rank you can multiply that by 20 (that’s 25,000 questions to get 100 organic visitors).  Anything below that point will get you into the vicious circle and anything above it into a virtuous circle.

Test it before

Doing a small test with some questions related to your market will give you a good idea of the difficulty of being successful with Q&A in your market. Doing it doesn’t take too long; just a couple of hours to post your questions, then track relevant search results over a few weeks. That’s a pretty cheap way to figure if you’re going to spend thousands of dollars before it’s worth it.

Don’t get us wrong. Questions & Answers are foremost great tools to engage and get your user base to return.

It’s your choice, pick your circle; vicious or virtuous?

* A great example of Q&A well executed is Trulia’s Voice. With about 26M visitors in 2008 their community posted a total of 420,000 questions and answers. That’s only 1.6% of visitors posting a question or an answer.

If you would like to share your thoughts or experiences with these types of features in a competitive market or not, we’d like to get your feedback.