The Ascent of Ranking Algorithms

Algorithmic ranking is on the rise. Everywhere I turn, something or the other is being ranked analytically.

Ranking web pages based on relevance, pioneered by Google’s PageRank, maybe the best-known example of algorithmic ranking.

Also ubiquitous are ranking algorithms inside recommender systems. Given an individual’s behavior (browsing history, rating history, purchase history and so on), the idea is to rank the huge universe of things (e.g., books, movies, music) out there based on likely appeal to the individual and show the top-rankers. If you are an Amazon or Netflix customer, you have doubtless been at the receiving end of these ranked recommendations for books and movies that you may find of interest.  Plenty of complex and occasionally elegant math goes into quantifying and predicting “likely appeal” (Netflix Prize winning approach).

Despite its age, recommendation ranking is far from mature and different flavors of recommender systems are popping up every day. Just last week, BusinesWeek had a story on The Filter, a new recommendation ranking system that is allegedly leaving the other approaches in the dust (aside: One of the founders of The Filter is Peter Gabriel, legendary musician and member of Genesis, one of my favorite rock bands).

So far, I have listed “old” examples of ranking: web pages, books, movies, and music.  But  recently, I came across something new: SpotRank.

Skyhook Wireless, the company that provides location information to Apple devices (when you fire up Google Maps on your iPhone, your exact location is pinpointed using a combination of GPS information and Skyhook’s wifi database – details) announced SpotRank a few months ago.

By tracking the number of “location hits” their servers receive from Apple devices, Skyhook can determine which spots are popular and when they are popular. They capture this in the form of a popularity score and, as the name suggests, SpotRank ranks locations by their popularity score.

Next time you are in a strange part of town, have time to kill and are looking for popular spots, maybe SpotRank can help you (at least if you like hanging out with Apple fans).

Now that places are being ranked, what’s next? Ranking people?

It is already being done. Heard of UserRank?

UserRank was created by Nextjump, a NYC-based company that runs employee discount and reward programs for 90,000 corporations, organizations and affinity groups. Next Jump connects 28,000 retailers and manufacturers to the over 100 million consumers who work in the companies in its network, typically getting the merchants to offer deep discounts.

NextJump calculates a UserRank for every one of the 100m consumers in its database.

The more a user shops on our network, the higher their UserRank™ will be. Users with high UserRank™ are more likely to spend and are typically your best customers.

NextJump creates value by allowing retailers/merchants to use UserRank in offer targeting. For instance, an offer can be targeted only to consumers with a minimum UserRank.

I wonder what my UserRank is?

My final example is from the field of drug discovery. In a recent article, MIT News describes fascinating work done by researchers at MIT and Harvard on applying ranking algorithms to this area.

The drug development process typically starts with identifying a molecule that’s associated with a disease. Depending on the disease, this “target” molecule either needs to be suppressed or promoted. A drug that’s successful in treating the disease is a chemical (which, of course, is just another molecule) that suppresses or promotes the target molecule without causing bad side-effects.

How is such a drug found? Over the years, researchers have amassed a large catalog of chemicals that can help suppress or promote target molecules. From this library, drug developers find the most promising ones to use as drug candidates for further testing and clinical trials. Unfortunately,

majority of drug candidates fail — they prove to be either toxic or ineffective — in clinical trials, sometimes after hundreds of millions of dollars have been spent on them. (For every new drug that gets approved by the U.S. Food and Drug Administration, pharmaceutical companies have spent about $1 billion on research and development.) So selecting a good group of candidates at the outset is critical.

This sounds like a ranking problem: given a target molecule, rank  the chemicals in the database according to their likely effectiveness in being a viable drug for the chosen target.

The drug companies weren’t slow to recognize this, of course. They have been using machine-learning algorithms since the 90s with some success. However, the MIT-Harvard researchers showed that a

rudimentary ranking algorithm can predict drugs’ success more reliably than the algorithms currently in use.

What was the key idea?

At a general level, the new algorithm and its predecessors work in the same way. First, they’re fed data about successful and unsuccessful drug candidates. Then they try out a large variety of mathematical functions, each of which produces a numerical score for each drug candidate. Finally, they select the function whose scores most accurately predict the candidates’ actual success and failure.

The difference lies in how the algorithms measure accuracy of prediction. When older algorithms evaluate functions, they look at each score separately and ask whether it reflects the drug candidate’s success or failure. The MIT researchers’ algorithm, however, looks at scores in pairs, and asks whether the function got their order right.

(italics mine)

Rather than scoring each drug candidate in isolation and then ranking them all, the key idea was to build pairwise ranking into the construction of the matching algorithm itself.

As the data deluge gets larger and larger, finding information most relevant to one’s needs (be they mundane needs like in shopping or profound needs like in drug discovery) gets harder and harder. Perhaps this is why we are seeing ranking algorithms everywhere.

Have you seen any interesting examples of algorithmic ranking at work? Please share in the comments.

(HT to Karan Singh and Florent De Gantes for making me aware of the MIT News article and NextJump, respectively)

Share/Bookmark

67 thoughts on “The Ascent of Ranking Algorithms”

  1. We pick-up and disappear to san francisco bay area Global (SFO) Airport, San Jose International Airport (SJC),
    Oakland International Airport (OAK).

  3. This made him change his mind about not turning
    state witness against Enron’s CEO Kenneth Lay. Now, Samsung’s
    first 16-inch notebook R610’s debut, full HD will be on the laptop all the dreams
    into reality photos. Apparently, it was the job of the Federal
    Reserve to ruin the once strong America and weaken it’s
    currency under the smokescreen of economic crisis, wars and now climate change mania.

  4. An intriguing discussion is definitely worth comment.
    I do think that you need to publish more about this subject matter, it might not be a
    taboo matter but generally people don’t talk about
    such topics. To the next! Many thanks!!

  5. Fun and youth full in many songs, no one can believe that Reba Mc – Entire is now over 50.

    This is very common now in the post ‘dot-com’ era, but remember folks there is a reason it’s the “post” dot-com era and not the dot-com boom.

    CBR’s Brett White shared his thoughts about the sexist shirts on Robot 6.

  6. Aw, this was an exceptionally nice post. Spending some time and actual effort to produce a good article… but what
    can I say… I hesitate a lot and don’t manage to get anything done.

  7. Americans have never given much credence to world opinion but they have been surprised of late
    by world behavior as in, actions speak louder than words. Parents
    are able to intercept the emails and choose to approve or disapprove the
    content of such while teaching their children how to protect themselves from the internet attacks for their range.
    These are just a few ideas for foot tattoo designs for women.

  10. Thank you on your excellent posting! I truly really enjoyed browsing your post, you could be a excellent copy writer.

    I might ensure that I mention your blog site
    and may come back later in life. I want to induce you to definitely maintain your perfect posts,
    have a nice holiday weekend!

  11. Undeniably imagine that which you stated. Your favourite justification seemed to be at the
    web the simplest factor to have in mind of. I say to
    you, I certainly get irked at the same time as other folks consider issues that they just
    do not realize about. You controlled to hit the nail upon the highest and also defined out the whole thing with no need side-effects ,
    other folks can take a signal. Will probably be again to get more.

    Thank you

  12. If you happen to get mired down in a specific place while playing, you can lose the
    whole enthusiasm for the game. It is about to be released in the mid of 2014 for the Play – Station 3 and Windows based devices along
    with the other devices like Xbox One consoles and Xbox 360.

    Each and every time a bid is placed the product costs improves one penny (hence the name penny auctions).

  13. I th?nk wh?t you typed made a bunc? of sense. However,
    what about this? what iff yo? added a little content?
    I ain’t saying your content is not solid, butt what if
    you added a title that grabbed people’s attention?
    I mean The Analytic Age ? Blo? Archive ?
    The Ascent of Ranking Algorithms is kind? plain. ?ouu could peek at Yahoo’s front
    page and see hhow they create post titles to grab v?ewers tto click.
    You migh add a related video or a ?elated picture or two too grab readers ?nterested
    ?bout what you’ve got to sa?. ?ust myy opinion, it
    might bring your pots a little bbit ore interesting.

  14. Additional to that we also rate Exposed Skin Care as the best for all skin forms (apart from
    those with sensitive skin).

    My site best acne treatment products walmart – Joie,

  15. The next time I read a blog, Hopefully it does not fail me
    just as much as this one. After all, I know it was
    my choice to read, however I genuinely thought you would probably have something
    useful to talk about. All I hear is a bunch of moaning about
    something that you could fix if you were not
    too busy seeking attention.

  16. It’s distributed by VCI Entertainment and Capitol Christian Distribution. He vowed to his father
    that he will surely make Wushu great. So I have written an article that will help you to learn all the ways in which you can watch videos on your brand new
    iphone.

  18. Android SDK contains essential tools, technologies and resources that help developers to create dynamic applications.

    One of them was hardware incompatibility of Android devices which in some
    cases can lead to severe fragmentation. It is a complete journey from the initial
    idea to create a business and lead to success.

  19. “The Complete Grape Growers Guide” will cover ALL of
    these mistakes in detail. He was a skilled professional and the best soldier who could be relied
    on to run a Legion on a campaign and in battle. One important thing about wine grapes is that they
    do not affect Botrytis cinerea.

  20. Hi, I think your blog might be having browser compatibility issues.
    When I look at your blog site in Opera, it looks fine but when opening
    in Internet Explorer, it has some overlapping.
    I just wanted to give you a quick heads up! Other then that, awesome blog!

  21. Bricking device: contextlinks2### our device bricks the device Sometimes
    when we ### it. You can receive purposes and your paperwork do the job effectively and more arranged.
    Moreover you can add ringtones for your calls and messages.

  23. s Haiti Relief Bounty Tournament will match all bounty money won and donate the total relief efforts to Haiti up to $50,000 and
    also contribute all tournament fees collected. In the
    long run sports like soccer, cricket, volleyball and other exclusive sports events World Cup and Olympics are more enclosed because of their
    ideal viewership in the world which furthermore captured a large number of the
    dignified viewers. A team with 2 losses might have lost both games right before the upcoming game, and be on a losing streak.

  24. It mostly comes down to personal preference, the type of photographer you are and your level of experience.
    Several digital camera models even enable you to take brief videos.
    Yet, a d – SLR nikon dslr camera
    uses light sensor chips and digital memory cards instead of films.

  25. (A broadband connection will deliver superior
    audio and video quality). And this law of game codified
    in England by the football association in 1863. After you have master
    it then move on to another one.

  28. Epoxy pipeline lining in West Palm Beach is utilized substantially where the structures are very old
    yet additionally really important. For establishing a well balanced life, HDPE pipe lining
    has been introduced. In this method, pipes are inserted into the current piping
    system, in other words, slipped into the existing pipes.

  30. That however iis just a cosmetic difference, but none
    tthe less, may be mportant to sopme users. Naturally, the 11-year-old
    company received a number of approaches from hopeful suitors.

    Users are free to choose handset according to
    design, features, looks and usage.

  31. Wow, amazing blog layout! How lengthy have you been running a blog
    for? you made running a blog glance easy. The full look of
    your web site is wonderful, let alone the content material!

  32. Good day I am so grateful I found your blog, I really found you by accident, while I was browsing on Bing for something else, Anyhow I am
    here now and would just like to say thanks for a incredible post and a
    all round thrilling blog (I also love the theme/design),
    I don’t have time to look over it all at the minute but I have saved it and also included your RSS feeds, so when I have time I will be back
    to read much more, Please do keep up the great work.

  34. You actually allow it to become show up not thay hard with your speech nevertheless i to uncover this condition to get seriously something I do believe We would never ever comprehend moreboy teen. It seems as well complex and intensely large for me personally. I will be taking a look to the following post, I’m going to try to get used to that!

  35. I’m impressed, I have to admit. Rarely do I come across a blog that’s equally educative and amusing, and without
    a doubt, you’ve hit the nail on the head. The issue is an issue that
    too few folks are speaking intelligently about. Now i’m very happy I came
    across this during my search for something relating to this.

  38. Wonderful beat ! I wish to apprentice whilst you amend your web site, how can i subscribe for a blog web
    site? The account helped me a acceptable deal. I have been a little
    bit familiar of this your broadcast provided bright clear concept

  40. When it comes to sourcing a mortgage at a later DOAH hearing,
    the work you ought to be held liable for getting a free roof
    inspection unlicensed contractors and replacement work done.

    my blog post: homepage (Tracee)

  41. Fairly aspect to content. I just stumbled upon your web blog and in accession money to talk about i receive really treasured bank account ones blog site content. In whatever way We will be checking in your add to as well as I personally happiness you obtain appropriate of having access to continually easily.

  42. @Krishnan: Yes, I’d agree. Many stock selection models rank the equity universe based on metrics like earning revision, price- to-book-value etc. and feed the output into portfolio construction/optimization algorithms.

  45. Behind ranking actually is just scoring. Potent scoring techniques and technology have been in corporations for at least 30 years. The appeal of the progress in the original blog stems largely from its real-time, ‘all-coverage’ nature. Some sites got to a level of credibility by compromising on predictive accuracy. For example, corrabarative filitering and link analysis can effectively recomend a book you may like or somebodyyou may like to link to, but quick recoomendations often can not afford to build itself on ‘deep drivers’. Often, therefore, the hit comes fast and goes first.

  46. @Al: A fair amount is known about how Google ranks the ads. As you probably know, the ads are chosen based on a real-time auction for the keyword(s) used in that particular search. But they don’t just show the ads in decreasing order of the bid amount; they also take into account the historical click-through-rates for each ad, the quality of the landing page when the ad is clicked on, etc. As a result, even if your bid is way above everyone’e else, you may not make it to the first slot (or even the first page) if your ad/landing-page are deemed to be of low quality.

  47. Page Rank is interesting. But probably more interesting is the algorithm that ranks the ads to display in Google’s search results. From the inventory they must have, to finding the best 5 to show you. Must be interesting. Nice job by them in keeping the focus on the other rank 🙂

    Met the founder of Aardvark a few weeks back. He built a product focused on subjective search, e.g. “what’s the best restaurant in Boston”, and used the social graph to help rank the results. Google and others are not particularly good at subjective search. Google bought Aardvark.

  48. @narayan: I agree with your take on the skewing potential of the “most popular” lists (as well as the benefit of non-behavioral approaches like Pandora). In fact, the effect is so pronounced that it is being exploited.

    My favorite example of this is the Apple Appstore: if an app makes it to the top 10 list (and the list is generated based on recent download volume), it gets a huge boost in the ensuing days. As a result, publishers of new apps take out ads on other popular apps as soon as they launch; if the ads drive volume for the new app, then the new app may make it to the top 10 list and from that point on, they can sit back and watch sales go through the roof, without incremental spending on ads.

    @badri: Fully agree that FICO is a great example of ranking. I omitted it since it is very well-known and “old” at this point. And, yes, there are numerous examples of ranking out there (“top B-schools”, “top cities to live in”, …) but I was looking for non-trivially algorithmic and/or new examples.

  49. One can quite comfortably posit the equivalence between “scoring” and “ranking”, with the former clearly being more granular and the latter trivially derived from the former. If you accept this, then one of the more ubiquitous applications of ranking comes from the credit scoring applications pioneered by FICO (www.fico.com) in the 1980’s. In a very McDonalds-ish statement the company states that to date “more than 100 billion FICO scores have been generated and delivered” to rank – yes, people. I have personally been involved in developing and deploying a very powerful and currently industry dominant credit card fraud risk scoring (ranking) application.

    As you say, there are probably hundreds (if not thousands) of such examples of algorithms out there for ranking everything from wine to women (and men). Of course, some are cleverer than others.

  50. now if only you could only rank ranking algorithms … 🙂

    seriously, isn’t there a danger that ranking algorithms skew their data? take the “MOST POPULAR” list at nytimes.com. the very fact that this list displays the top 10 stories attracts people to read those stories (i know i do) thereby amplifying the popularity of the stories who make the list relative to those that barely miss the cut.

    is this where the pandora approach to establishing conceptual links rather than links discovered by analyzing behaviour comes in?

Leave a Reply

Your email address will not be published. Required fields are marked *