Long Tail Statistics (Kilkki's Formulation Hides Latent Demand)

The expression “The Long Tail” first started to buzz with the WIRED article of that name by Chris Anderson a couple of years ago.  Later, he expanded the article into a book explaining the ideas more fully and giving examples from book sales, music and movies.

The idea is explained in many places.  Start with Wikipedia.  I won’t try to reproduce a basic explanation.

However, recently, I read a paper by Kalevi Kilkki titled "A practical model for analyzing long tails". In it, Kilkki explains an alternate Long Tail distribution to the traditional Power Law distribtution.

I don’t see Kilkki’s motivation–or rather, I don’t trust it. He seems to be blindly "refining" the distribution to get more precision.  I don’t trust this for two reasons (1) precision isn’t going to help prediction or design (without understanding underlying processes, at the very least) and (2) I think his quest to line up the dots with the lines hides important effects. We will see. Finally, I think Kilkki over emphasizes the differences and benefits of his model. To try out my main critique, I built a (rough) Mathematica notebook to look at Kilkki’s model and the traditional power law.

As usual, I generated more questions than answers, but, unless I really misunderstood Kilkki, I don’t see the benefits he is pointing to.  I hope you enjoy a quick look at The Long Tail and send comments if you have any thoughts.