Web 2.0 might be the latest fad but the identity management part is certainly intriguing. The key idea is that users have one unique online identity instead of dozens of site-specific profiles. Not a new concept (Microsoft Passport, etc) but this time the concept seems to get a lot more traction.
In this article, the author toys with the idea of “Identity management” as a service:
One of the things that’s becoming evident as organizations deploy service-oriented architectures is that identity management (access control, user authorizations) has to be implemented as a service. Anything else rapidly becomes too unwieldy to maintain and manage as the number of discrete application services increases.
Earlier today I was myself playing with a similar idea, with a different twist: instead of having companies outsource identity management to a trusted 3rd party, individual users (people) let a trusted 3rd party manage their online identity and garantee privacy protection. Clearly, this isn’t fully baked but in my scenario the 3rd party works for the consumer, and is likely to do a better job than any 3rd party which works for a business. Businesses always want to retain as much information as possible.
Circling back to analytics, I do see a trend - users wanting to regain control of their identity and wanting to stay anonymous. If the trend continues in the next few years the value of the so-called “lifetime” analytics reports will continue to erode and inevitably become irrelevant.
Interesting for those of you selling software, interesting for those of you buying software - great article on pricing: Camels and Rubber Duckies ( from the “Joel on Software” blog).
A|B tests have a margin of error, which you can roughly estimate by keeping a small control group (A|B|A’, 50/40/10%, A’ being the control group with same experience as the A group). This isn’t statistically the right way to do it but it does give you some indication.
Now if your test ways B is winning the test (let’s say +4% revenue increase), with a margin of error of +/-5%, it means that it’s possible that after going live on B your revenues will decrease. Think of A|B tests as election polls:
- When a presidential poll says 52% Bush and 48% Kerry with +/-5% margin of error, it means that if the millions were asked to vote, Bush would likely win but it’s possible that Kerry could win with 53% of votes.
- Likewise, when an A|B test says B increases revenue by 4% with a +/-5% margin of error, it means that if millions of users visit your site with B, you would likely get a 4% increase in revenue but it’s possible you could lose 1%.
For an A|B test to be conclusive you need a big gap between challenger and champion, or enough traffic to bring down your margin of error. By the way, there is no test based on sampling which can give you a 0% margin of error. Uncertainty is a natural thing.
We have in many ways come full circle on Path Analysis. The top paths reports were the cool feature in the late 90’s, since they provided some insight on what users “were doing on the site”. But they quickly became the least used of all reports, because they were difficult to analyze and customers were asking for more detail. In 2001, a handful of vendors rolled out a new generation of Path Analysis tools – more detail, better UIs, more flexibility – and along with that came some new interest for Path analysis and browser overlays.
While analysts now have the tools to analyze and visualize user browsing behavior they still spend 95%+ of their time on other reports, such as marketing campaigns or segmentation analysis:
“It’s fascinating data, but always leads to more questions, generally more complex.”
“We want to know where to start, what to look for, what to do”.
Like this earlier post was showing (Path analysis quiz), there isn’t one answer.
Here is a screenshot from Fireclick’s Site Explorer (browser overlay) today.
Percentage labels indicate click-throughs on the link, orange bars the number of orders for people who clicked on the link, green bars revenue. A and B point to the same page. C points to a page where people can find used copies of the Six Feet Under DVD.
Question 1 - The data indicates the following:
A (Image link) - click-through rate: 0.7%
B (Text link) - click-through rate: 1.2%
Similar conversion rate for A and B, similar revenue.
What is the most appropriate action:
i - Eliminate the image link
ii - Reduce the size of the image link
iii - Reduce the font size for the text link
iv - Do nothing
Question 2 - “C” (”Buy used”) has a 4.1% click-through rate and twice the conversion rate of A or B. The “Buy new” link is getting less than 0.1% of the clicks on this page.
What is the most appropriate action:
i - Eliminate link C
ii - Increase the size of link C
iii - Swap the location of “Buy new” and “Buy used”
iv - Remove some of the clutter around “Buy new”
Driving through an unfamiliar foreign country is fun. Sicily is beautiful, people are nice, roads are safe – but there are few, very few signs to help you find your destination. I find it a lot easier to find my way here in California – even easier in France where I grew up, because I know the roads better and because the signs are more intuitive to me.
I think online browsing is somewhat similar. I have a handful of Web sites I know well and visit often. New and unfamiliar sites however require me to think about where to click to find what I’m looking for. Just like driving around home or driving around Sicily for me.
I mention this because I have been thinking lately a lot about the path analysis set of reports – looking for a new “framework” for interpreting the data. In fact, while I think there are many great tools that build great-looking path analysis reports, these only scratch the surface of what can be done. And path analysis data is often misinterpreted.
That’s why I’m talking about driving and browsing. It’s not the ultimate metaphor, but I understand driving a lot better than browsing. For example, to tell that I’m confused driving in Sicily, what would you be looking at first? The path I took - or my driving speed as I’m approaching intersections? Likewise, to tell someone is confused on a Web site, wouldn’t you be looking at the read time per page rather?
For the next few weeks, I’ll be posting on the topic. If you have any ideas, drop a line.
