Chapter 16 - Lensrank (a mystery)
An important subject matter that deserves great scrutiny.
Chapter 16 - Lensrank
This is a very touchy subject matter that has been discussed and debated in the SquidU forum for quite a while. I decided to take a closer look because I am usually curious about technical matters and it is my nature to try and find out how things work. Lensrank (LR) has been a mystery to me. I will use this lens to dissect the problem and offer some insight and possible solutions.
Two main issues
The LR controversy can be separated into two issues. First, how important is it? And second, is it fairly implemented?
I believe LR is important on so many levels. It is the basis for financial revenues and for recognition and prestige. Each lensmaster can decide for him or her self how imported it is for him or her. They can choose to focus their energy on other parts of Squidoo such as content creation or publicizing their lens. However, that will not change the fact that LR is important in the eyes of many. Whether it should or should not be the case is irrelevant, perception rules the day. It makes no sense to deny its importance. Even if you disagree with me on this subject, I think you will agree that the LR system should be properly implemented. By properly, I don't mean perfection but I do mean fairly.
I believe LR is important on so many levels. It is the basis for financial revenues and for recognition and prestige. Each lensmaster can decide for him or her self how imported it is for him or her. They can choose to focus their energy on other parts of Squidoo such as content creation or publicizing their lens. However, that will not change the fact that LR is important in the eyes of many. Whether it should or should not be the case is irrelevant, perception rules the day. It makes no sense to deny its importance. Even if you disagree with me on this subject, I think you will agree that the LR system should be properly implemented. By properly, I don't mean perfection but I do mean fairly.
Three elements and a simple analogy
To have a LR that is implemented correctly and fairly, I believe there are 3 elements. First, the system should have transparency. Second, the system should be efficient. And finally, the system must exhibit validity. I will address each element shortly.
First, it is necessary to describe LR with some basic attributes. I would like to think of LR as a "black box". It takes in data and generates a ranked order listing. I like to draw an analogy and compare this LR to a bestseller list of books. A bestseller list changes weekly and the best selling books for that week is tallied and the top 10 by category (fiction / non-fiction) is published in newspapers and on websites of Amazon or Barnes and Noble. The books that appear on this list get the attention. They may be reviewed. The authors may be asked to appear on interviews. They will help sell books and the added sales will mean higher profits for the authors. If you would examine how the bestseller listing behaves over a period of time, you will notice certain books appearing on the list for a week and then drop off the list. Some will appear for a few weeks and then fall off. Once in a while, there may be a book that will stay on the list for quite a long time because it has captured the imagination of many people. However, eventually, it will drop off that list and new books will take their place. That is not to say the book is no longer any good but just that it has become stale. This simple analogy is a very good fit for what LR should be. Not knowing anything about the underlying algorithm, I would expect a very similar behavior on the lens over time.
Since there are more lenses created everyday, I would expect the LR to behave very similarly overtime. That being the best lenses will appear on the top 100 list and some will drop off over a short period, some will drop off over a longer period and a few may stay on much longer but eventually will be replaced.
First, it is necessary to describe LR with some basic attributes. I would like to think of LR as a "black box". It takes in data and generates a ranked order listing. I like to draw an analogy and compare this LR to a bestseller list of books. A bestseller list changes weekly and the best selling books for that week is tallied and the top 10 by category (fiction / non-fiction) is published in newspapers and on websites of Amazon or Barnes and Noble. The books that appear on this list get the attention. They may be reviewed. The authors may be asked to appear on interviews. They will help sell books and the added sales will mean higher profits for the authors. If you would examine how the bestseller listing behaves over a period of time, you will notice certain books appearing on the list for a week and then drop off the list. Some will appear for a few weeks and then fall off. Once in a while, there may be a book that will stay on the list for quite a long time because it has captured the imagination of many people. However, eventually, it will drop off that list and new books will take their place. That is not to say the book is no longer any good but just that it has become stale. This simple analogy is a very good fit for what LR should be. Not knowing anything about the underlying algorithm, I would expect a very similar behavior on the lens over time.
Since there are more lenses created everyday, I would expect the LR to behave very similarly overtime. That being the best lenses will appear on the top 100 list and some will drop off over a short period, some will drop off over a longer period and a few may stay on much longer but eventually will be replaced.
The three elements
Let's now address each of the 3 elements.
- Transparency. I believe LR should be transparent. That means we should be told what metrics are counted in the computation of LR. On the forum discussion, some have raised the analogy that telling people what the algorithm is like publishing the winning numbers of a lottery. That is a false comparison. The real comparison would be telling the odds of winning and the basic rules that appear on the back of every lottery ticket. Let's take a simple example. Suppose LR was to be based on web traffic count only. By stating this fact up front, all lensmaster will know exactly how to get to the top of the rank list. They need to generate traffic to their lens. Knowing this fact is not sufficient to guarantee getting on the list.
- Efficiency. Another key element is computational efficiency. When the number of lens is small, it matters little how long it takes to compute the LR. However, as the number of lens grow, that computation becomes a critical factor. Any LR algorithm must be efficient in that computation or else the system will not be able to sustain the anticipated growth. When the time to compute LR increases to surpass the total number of hours in a day, you are left with only two options. Increase the CPU power or reduce the complexity of the LR algorithm.
- Validity. When validating any given algorithm, there may exist 3 types of faults. The first is a design flaw. There may be something in the basic logic of the LR that does not work in real cases. This can be easily tested and corrected. Second, there may be a bug in the code. A system may have been incorrectly implemented so that some cases will cause a false data to be created that will invalidate the ranking results. This is a harder problem to solve and may require extensive testing in real life cases. Third, the final fault could be the result of intentional "tinkering" by the implementor. This can show up at various times and cause drastic swings in ranking.
Three flaws of the current system.
In trying to learn more about the LR of Squidoo, I created some simple test cases and monitor the results over time. Here is what I discovered and anyone can go and repeat the same experiment and will get the same results. That is called the "scientific method". Where the results can be repeated and verified independently.
I documented this in my lens of Squidoo features wish list.
I will summarize 3 simple flaws of the current design.
I documented this in my lens of Squidoo features wish list.
I will summarize 3 simple flaws of the current design.
- An "empty" content blank lens will have its rank assigned higher after it is created.
- A re-published lens even if no changes were made to the content will get a small boost to the ranking.
- A new lens that has not been published, that is still under construction, will get treated equally as a published lens.
System bug - an averge lensrank
For the second fault of a bug, I decided to implement an average ranking formula to estimate the average ranking of lensmasters. This is not a perfect system but it will alert when some bugs are "acting" up as we have seen on various occasions. It is an approximation and it is more accurate with lensmasters having more lens.
The lens I created is in the link list below - and I have also created a Lotus 123 spread sheet to help with this calculation. When I first proposed this, Gil claims that it is inaccurate and that he had implemented something similar in the past but was removed due to complaints from some lensmasters claiming it was unfair because it favors someone with fewer lens. My response to Gil is that he should have left it in place. The math behind this average is solid. To address the concerns, he should just include the number along with the number of lens for each lensmaster. The combined number will be a good estimate as to the "ranking" of that lensmaster. This is a relative number that only has meaning when compared to other similar lensmasters. The main use would be to track the stability of the LR system. Once a basic baseline number is established for a given lensmaster, it should not vary by too much. Any large swing in this number will mean a bug has appeared in the LR system and must be closely examined.
The lens I created is in the link list below - and I have also created a Lotus 123 spread sheet to help with this calculation. When I first proposed this, Gil claims that it is inaccurate and that he had implemented something similar in the past but was removed due to complaints from some lensmasters claiming it was unfair because it favors someone with fewer lens. My response to Gil is that he should have left it in place. The math behind this average is solid. To address the concerns, he should just include the number along with the number of lens for each lensmaster. The combined number will be a good estimate as to the "ranking" of that lensmaster. This is a relative number that only has meaning when compared to other similar lensmasters. The main use would be to track the stability of the LR system. Once a basic baseline number is established for a given lensmaster, it should not vary by too much. Any large swing in this number will mean a bug has appeared in the LR system and must be closely examined.
Final fault - "tinkering"
The final fault is the most disturbing to me. The "intentional tinkering" either by the Squidoo HQ or by individual lensmasters (who may have inside knowledge), causing the system to create an "unfair" ranking. I have no definitive proof of this but only sample observations. What is most disturbing is the timing of these events. They seem to occur just a few days prior to payday at Squidoo. This has occurred prominently 3 of the past 4 months. The other month showed a smaller change but still noticeable. How do I know this? It is not so hard to demonstrate. I only needed to go to the stat page of a few of my lens. The plot provided by Squidoo showing the ranking history of the lens. Here is one prime example.
Notice the glitches in the plot on 4 occasions over the past 4 months. Each has occurred just prior to PAYDAY. This is very suspicious behavior. At this point, I am not ready to indict anyone but I am keeping a close eye on this going forward. Let everyone be forewarned. Any future "tinkering" should be clearly explained.
Notice the glitches in the plot on 4 occasions over the past 4 months. Each has occurred just prior to PAYDAY. This is very suspicious behavior. At this point, I am not ready to indict anyone but I am keeping a close eye on this going forward. Let everyone be forewarned. Any future "tinkering" should be clearly explained.
What Do I want from Squidoo HQ?
My goal with this lens is to help improve the overall quality of Squidoo. With regard to lensrank, here are my wish list to Squidoo HQ:
- Acknowledge that there is a perceived problem with the current lensrank.
- A promise to look into it and possibly fix it with baby steps. It doesn't have to be tomorrow but certainly near the top of the priority.
- Make a summary statement about what the metrics are that goes into the ranking algorithm. They don't need to publish the glory details of the algorithm.
- Stop "tinkering" with the lensrank just prior to PAYDAY. If they need to implement a fix or an improvement, do it after PAYDAY.
- Notify the lensmasters when changes are being made so that we are not surprised when ranking changes abruptly.
Conclusion
In conclusion, I hope my detail explanation here will convince some of you that LR is a work in progress. It is not perfect and needs to be examined and possibly fixed in baby steps. Taking the analogy to the lottery system once again, I hope whether you play the game or not, I would think everyone should be interested in keeping the game fair and honest and no one should be allowed to cheat. That's the kind of publicity that will lead to front page stories in the newspapers.
My final word is LR should be fair and it should be transparent and it is IMPORTANT for all of us.
Table of Content
Previous Chapter
Next Chapter
My final word is LR should be fair and it should be transparent and it is IMPORTANT for all of us.
Table of Content
Previous Chapter
Next Chapter
Some related links...
- Squidoo features wish list...
- My suggestions on features and list of bugs...
- My lensmasters average ranking
- An easy calculation to rank lensmasters.
- Another detail analysis of lensrank
- A lens by zuzanna that goes into great detail about the importance of lensrank.
Reader Feedback
submit
-
Reply
- jackclee jackclee Aug 5, 2008 @ 1:13 pm
- triathlontraining, You don't need an account with Photobucket to view. However, some companies do disable computer access to image servers such as Photobucket.
-
Reply
- triathlontraining triathlontraining Aug 5, 2008 @ 8:00 am
- Unfortunately, I couldn't see the image under tinkering. It seems it requires an account with Photobucket.
Great info! Keep up the fine work on explaining things. :)
-
Reply
- poddys poddys Jul 27, 2008 @ 8:37 pm
- Just came across this today, having spent much of the day debating over the drastic lensrank plummeting, following it seems changes in the formula to take into account clickouts.
It concerns me that the number of lenses and average lensrank might be taken into consideration. Does that imply that if I have a number of lenses with low rankings and they go down, that it will automatically drag down a perfectly good lens at the top as well? I don't like that idea.
Well 5***** for this lens anyhow before I forget...
-
Reply
- The_Homeopath The_Homeopath Jul 27, 2008 @ 1:40 pm
- So here I sit on another Black Sunday. Thanks for your work trying to figure it all out.
-
Reply
- Monteath Monteath Apr 11, 2008 @ 1:49 pm
- Great information! Thanks for sharing this, it's great food for thought.
- Load More
