I see a rising trend for computer generated heatmaps of websites, used for landing page optimization and conversion rate optimization. I am updating an older post regarding such tools.
If you’re using services like feng-gui or attention wizard you should be aware of the following:
- computers are machines and even if they can detect skin, faces, smiles or eyes within pictures, they can’t create heatmaps for emotions associated with the image. Be honest, are you looking at the picture below exactly like computers do?! (scroll to see how a computer will look at Pamela Anderson).
Pamela Anderson
Computer Generated Heatmap on Pamela Anderson
And here is how real people will look at the same image: (images and video courtesy of Peter Hartzbech from imotionsglobal.com):
Real Human Heatmap Gaze
Real Human Spotlight Fixation
Real Human Spotlight Gaze
Here’s the video:
- human eyes (the direction they point at) and smiles (large, white teeth smiles) also play an important role on where your visitors will look at. An image depicting model’s eyes looking directly at the user will attract his eyes on the model’s eyes, while an image with the eyes of the “hero” pointing to a product (looking to the left or right) will “heat up” the product image. From GrokDotCom, here’s a real (made on humans) eye tracking heatmap:
Model looks at you (left) and at the product (right)
- a website seen on a 1024×600 resolution will have a different heatmap compared with the same website seen on 1920 x 1200. What’s above the fold still matters, not matter what anyone will say. You need to analyze and optimize the website for most used resolution of your audience. Below are screenshots for the same website taken at different resolutions:
The heatmap at 1024x768 res
Same website heatmap taken at 1920x1200.See the differences? Not sure if it's even technically correct
- foveal and peripheral vision can also affects the eye movement and generate heat maps. The way you place elements on a page (headlines closers to an attractive image) will play a difference and I doubt current computer algorithms are taking this into consideration (via entrepreneur.com)
The hotspot seen by human eyes as per foveal view
- banner blindness also has some influence (how would a computer know that an image is actually a banner and take banner (via useit.com)
Banner Blindness Eye Tracking Study
- animation will change the hot spots
Rate this post!
loading...
No related posts.
Great post!
It is true that computer simulations of visual attention like out AttentionWizard.com service do not exactly predict where real humans would look. But to be fair, they are designed to look at a whole landing page and not just photographs of Pamela.
However, real eye tracking tests cost thousands of dollars. Predicted attention heatmaps are instant and have about a 75% correlation with eye tracking (good enough for government work… :-)
More over, you can use them on mock-up designs and not even live landing pages. The Lite version of our product is free for daily heatmaps.
loading...
Tim, thanks for reading my article and for the comment.
I agree with you, real eye tracking is expensive, that’s why I use tools like yours. They have value, without any doubt, however I am always aware of the 75% correlation. To great greater insights I am trying to do a combo analysis of computer generated heatmap and mouse tracking (i.e. Crazy Egg).
loading...
I think mouse tracking is very helpful, but it only works on LIVE pages. We are big fans of ClickTale.
We see many of our http://AttentionWizard.com users refining their design BEFORE even going live. Why would you launch with a landing page that you know has significant attention leaks (not related to the desired conversion action)?
All of these tools have their place.
loading...
Right. When I work on live projects, ClickTale can provide nice data to analyze. When the page is not live, I still have to use computer generated heatmaps. I agree that they are much better than guessing.
Nice job with attention wizard tool. Does it use the feng-gui.com’s algorithm or is it proprietary?
loading...
I agree, clicktail helps me know what is happening on my site and its heatmpas and videos show me i can increase my conversions.
loading...
No arguments with your main point, but I find automated heatmaps to be a useful object lesson in showing clients just how complex visual attention can be (and simulating the added complexity of adding elements). If someone wants to add a bunch of photos or giant banner ad, a quick algo-generated heatmap is a nice way to show them the consequences.
I would also hypothesize that people attend to Pamela’s dress straps out of concern that they may not have enough structural integrity to hold out ;)
loading...
Pete, there’s no doubt, automated heatmaps are useful. I just wanted to make people aware of some particularities, especially for the layouts including the hero face.
loading...
My view is that you should use the tools that are available and test.
If you can get a 75% correlation from computer generated maps then well and good.
If, as claimed by ClickTale, their new mouse move hover heatmaps (in beta) offer 88/89% correlation with eye-tracking then even better.
For high traffic pages an almost 90% correlation approximates pretty close to full eye-tracking and enables a level of testing not before available to the smaller site owner.
loading...
Having done eyetracking research since 1995, invented eyetracking heatmapping as a visualization technique in 2001, and having a specialty eyetracking company that since 2000 has tested thousands of web pages, I can tell you that there is NOT a 88-90% correlation between mouse movements and eyetracking; that I think that automated heatmaps are cool but that they (like click-based heatmaps) miss the critical details that can save or mislead you; and that eyetracking collection AND ANALYSIS does cost more (minimum of $3500 to test a page at http://eyetools.com ) but that you get expert guidance based on analysts having seen analysis on 1000′s of web pages with increases in conversion ranging from 10% – 1,000%. You can definitely say I’m biased, but I’ve also seen a lot more data than most anyone else :-)
loading...
@rob: I am not sure of the 90% correlation as Greg mentioned
loading...
Great post – you make some really valid points here. I love Attentionwizard.com heatmaps. It allowed me to look at landing pages without having to ask the client’s UX people.
Based on my experience, I recommend using computer-generates heatmaps BEFORE launch. After launch, use real people and live data to confirm behavior.
For example, I was looking at a landing page that had a lot of “heat” on the main call to action. That’s great, but the quantitative data didn’t match (i.e. no one was clicking on the link). We can see from the computer-generated heatmap that people should be looking at the call to action.
Testing the landing page and exploring the content instead of just the design really helped figure out that the call to action content was weak, not the design or layout.
So if you find your coworkers are hung up on why the heatmap didn’t work, tell them the issue may not just be the heatmap, but the content on the page. Thanks again for the great post – good things to think about. :)
loading...
Great post, I’m glad someone has pointed this out.
I think there are several problems with tools like Attention Wizard:
- Do you really want to use a tool where a quarter of the time it will be wrong? And if we don’t know when that will be, why would you use it at all? (If CRO has taught us anything, it’s that we can’t rely on our intuition to ‘know’ when what we see is right or wrong!)
- What’s the 75% figure based on, anyway? A best guess or an actual study? How many eye tracking studies were compared? Has this study been published? What was deemed an accurate comparison vs an inaccurate comparison?
- How correlated is eye tracking attention with conversion rate, anyway? Sure, there might be a strong correlation between complex v simple, but I’m interested in which simple design converts best. If Attention Wizard (or eye tracking) et al really worked, we wouldn’t need to test because it will have made the case already!
As far as I can see, essentially we’re left trusting a vendor on a shaky 75% figure, which means *at best* the tool still misleads us a quarter of the time, and it’s still unknown whether the other times actually do correlate with conversion rate.
I’m not sure I’d want to use that!
Too many leaps of faith between computer generated heat map > eye tracking correlation > conversion rate improvement correlation for me.
Tim?
loading...
Joy – thanks for the thoughtful post – couldn’t have said it better myself.
Attention prediction software like http://AttentionWizard.com is very useful pre-launch of your page or serves as a quick sanity check on live pages.
Greg – of course eye tracking will remain the gold standard in terms of quality, but is still out of the reach of many people and companies except for the high-stakes projects.
I plan to cover AttentionWizard and ClickTale during my Conversion Ninja Toolbox session at http://ConversionConference.com in San Jose this May.
loading...
Tim, looks like our comments were approved at the same time. Could you elaborate a little on the 75% figure – how you arrived at it, what it means etc? Thanks!
loading...
Luke, 75% correlation with real eye-tracking does not mean that the predicted heatmap is “wrong” 25% of the time. You are implying a binary kind of decision, when visual scene analysis is a very complicated matter.
The 75% correlation number comes from comparisons against real eye tracking results.
However, all of this kind of misses the point in my opinion.
What matters is that http://AttentionWizard.com provides a way to get rid of gross visual problems on the landing page. Believe me, most pages do not have subtle issues – they are often riddled with gross graphical distractions. So this is only a first-cut approach to diagnosing conversion issues.
Also, we need to remember that we are comparing apples to bowling balls. The predicted heatmaps are essentially a quick “scene analysis” that is automatically carried out by our brains to locate objects of interest in the first few seconds of viewing a landing page.
They do not simulate a “considered response” over an extended period of time that involves any kind of rational or deliberate thinking. In other words, they will not tell you if your color choices are cheesy, or your headline is stupid, or your brand strength is non-existent, or your value proposition is weak, or that your price is too high.
Obviously all of those things massively impact conversion. However, unless you get the crude visual focus and organization of the page fixed first, it will be a lot harder for a visitor to even consider some of these other factors.
I would invite you to read the following Credo Mobile case study on our blog and see how this diagnostic tool was used in conjunction with a lot of good thinking and a visual redesign to improve conversion by 84% http://sitetuners.wordpress.com/2009/06/03/credo-mobile-case-study-84-conversion-rate-increase-through-consulting/
The bottom line is that thousands of people find our tool very helpful in designing and refining landing pages. It is available in a free version and provides instant feedback that is impossible to get with other approaches.
Is it perfect?
No.
Is it useful to you?
I invite you to try it on some real landing pages (instead of Pamela’s silicone mountains) and judge for yourself.
loading...
Cool, thanks for the response Tim!
loading...
It will be useful is someone who works in the eyetracking business could post (for comparison reasons) samples heatmaps of the same page done with eye tracking, clicktale, crazy egg and attention wizard.
loading...
Animations get covered up w/post-it notes. If the text is really interesting, I will display it using a tool that gets rid of everything except the text.
I hate reading online. And, no, I don’t see anything beyond the text I’m trying to read.
It’s not even important that an ad be seen. You can’t measure the impact of that ad, so what is the point. Online ads are no better than offline ads. They should be, but the technology to make them better does not exist. Ad distribution technology doesn’t improve the situation.
loading...
Great post!
Tim is also right – that this is not to mimic what humans do and to get those real studies can cost thousands of dollars.
But I also think you have to realize it can cost a hell of a lot more than thousands of dollars if people make decisions on imperfect data, assuming that a predictive tool is…well…a predictive tool! :)
Tim, you’ve created an incredible piece of technology, but I don’t think we’re in a stage of analytics maturity where people will be smart with the data. It needs to be able to evaluate when CTAs are below the fold. It needs to be able to evaluate based on shapes and not just colors, etc. It should try to recognize advertising, eyes (the first things people look at). Am I talking about a tool that will take a gazillion dollars to build? Maybe. But that’s what it’s going to take to get data as good as the real thing.
Tim is a super-smart guy who will get this figured out.
And it wouldn’t hurt if you built a boob-recognition engine, too.
Here are a few demos I’ve done to add to what’s here:
http://www.atlantaanalytics.com/web-analytics-tools/sitetuners-attentionwizard-opportunities/
loading...
As someone who is just starting to ramp up some LP testing plans, this article was very helpful. I found out about attentionwizard.com through a webinar this week that Tim gave. Found the results interesting, but definitely questioned the data. This, and the comment thread, really shed some light on things. Thanks.
For what it’s worth from a “newbie”, even before this article, I assumed AttentionWizard was really an “early” testing tool to create a base understanding. Definitely not the “end-all” for heatmap data.
loading...