New Ranking System (Feedback)

[miodrag.milenkovic]

I'm sorry I've been so obnoxious, but I also can't say I'm not glad that we managed to sway you. I for one appreciate a great deal your efforts to improve an already awesome site. I'll be very happy with any option that would let me remain a civilian.

I also use this site to impress girls, and I feel that any complication from the plain number ranking subtracts from the punch of being able to say "i'm number so and so!", but that's a minor point, please feel free to disregard it.

p.s. i'm only half-joking on the last thing...

[ed_r]

So there's your ranking system, Mio: a list of the 25 most eligible bachelors. "Hi, I'm George Clooney"

[stalepretzel]

I would love to have the "Programmer who does math" or "Mathematician who programs" option. (Mathematicians FTW )
I don't really like the military style rankings either. I don't like war, but that's not the real reason. I just feel like math and military are completely unrelated. I'm totally fine with a numerical-style ranking system.

[EDIT]Wow... looks like someone already came up with this! (Maybe I should read all posts before posting...) Since I, and Euler, and someone else all came up with this, apparently separately, it seems that it's a pretty logical system.[/EDIT]
However, if we were to switch to a new system, I think it would be cool to map the amount of problems solved to different mathematicians/programmers, based on brilliance. Since most famous mathematicians/computer scientists are quite brilliant, we'd need to find some less intelligent ones for the lower ranks.
Also, there could be awards, such as the Gauss award, for solving a difficult problem as a rather new member. Or, the Wiles award (Andrew wiles, mathematician who spent about 9 years, mostly on his own, trying to solve Fermat's Last Theorem), for people who work for a long time on a certain problem (but how would we measure that). Those in the elite ranks could be members of the Pythagorean Brotherhood. (Sorry, I don't know much Computer Science history... wasn't there some guy named Turing? )

In other words:
Yes to Math vs Comp. Sci.
No to military ranking.
Yes to math/programing rankings.

[miodrag.milenkovic]

So there's your ranking system, Mio: a list of the 25 most eligible bachelors. "Hi, I'm George Clooney"

"... and keep your eyes closed"

[rlindley]

The military aspect does not bother me at all. But I think the new system is
overly complicated. After looking at all the lists, I still don't really
know where I stand. And no easy way for me to answer someone who asks me
how I'm doing with the problems. I would like a simple system such as the
following:
1) Rank the problems by ascending difficulty (number of solvers) as you do now.
2) The worth of each problem is it's rank.
3) Total worth of all problems is 1+2+3...+n = n(n+1)/2
4) Each user's score is the sum of the values of the problems he has solved.
5) Rank all user's by their score (use time stamps to break ties).
6) Then show a user's rank, his total points and his percentile.
7) Also show total possible points and total users and maybe other
pertinent statistics.

All this could be based on active users (solved a problem in last x months).
Update hourly or daily or whatever time interval is appropriate.

Wasn't the scoring system something like this way back when?

This would encourage users to attempt the new problems immediately since
initially they would be worth the most. And the really difficult ones would
always have a high value and the really easy ones would become worth less and
less.

I have no idea what impact such a system would have on the server and I'm sure
there are other inherent problems. Just seems to me there ought to be a simple
way to assign a rank which is meaningful to everyone, including outsiders.

[euler]

Part of the reason for change was to give newer members a chance to "compete" with the best. If you've been a member for some time then you were able to solve the initial small collection of problems and then stay on top of them as we released one per week. But we felt that the database of problems had grown so large, and the average difficulty had also increased, such that it is almost unreasonable to expect someone to solve them all and have any chance of getting into the top scorers table. Equally, those that have solved them all (or most of them) deserve recognition, which is why the ranks were introduced.

But you're right: an alternative scoring system could be considered at this point. However, I think we need to build into it the consideration of newer members, which is why we cannot exclusively use the entire problem set.

[neverforget]

If I understand the problem correctly, the P.E. community has reached the point (with enough old members) where it must be careful not to intimidate the newer members with all the higher rankings and dissuade them from exploration. Sometimes this is solved by creating different divisions, either in members (so there will be promotion) or in the problem (so there will be gates that can be passed only when completing a certain criterion). I think divisions are particularly limiting, and can create a "stuck" situation at times. I think one way to reward the newcomers is to simply create milestones that do not need to be fulfilled before continuing on, such as new rankings upon solving certain sets of problems. Another facilitation that should not be too hard to implement is to have a "Show me the 10 easiest problems I have not solved" button.

I have somewhat mixed feelings about the military raking theme. While I may be proud of achieving a certain "figurative military rank" I still feel put off by its more (albeit unintentional) grim implications. Perhaps the scientist/researcher ranking euler mentioned may be better. It still gives a sense of struggle and fight, since scientists and researchers often have to fight long and hard before a worthy discovery. Academic rankings such as "bachelor" or "college graduate" to "professor emeritus" or "Nobel laureate" would be a little less on the violent side than "private" and "colonel", while still retaining a sense of achievement.

[ansrivas]

One way to give higher ranks to newer members would be to partition the set of problems into a small number of classes and make a separate list in addition to the overall score list. The list of easiest problems could have a much larger number of problems. If a new member solves a small number of problems in the hardest list he could climb to the top of that list fairly quickly.

The hardest list may be called "List Euler" this being Project Euler. Other lists may have the name of other mathematicians/(computer??:))scientists.

[miodrag.milenkovic]

How about keeping two scores, with the user being able to switch between the views? One the old plain (total, time when arrived at total) lexicographical ordering,
and the other one an exponential moving average a1*Sum_{k in set of solved}(exp(-a2*(N - k))) where N is the number of the latest problem, and constants a1 and a2
conveniently chosen. This second ranking could be augmented by the time component too. Exponential moving average is famously cheap to update, so it wouldn't
cause a lot of additional load. With the a2 big enough the newcomer would be able to get ahead of an oldtimer who missed the latest problem or two.

[miodrag.milenkovic]

Yet another score could be the "hotness score", e.g. again the time weighted EMA of the differences between the times of the consecutive solutions. To wit, if t1, t2, ..., tk is the sequence of times when the problems were solved, in the chronological order in which the user solved them, not in the order they were posted, the score could be a1*Sum_{k}exp(-a2*t_k*(t_k - t_{k-1})). This would be the (pretty much) instant gratification score where a newcomer would certainly be able to beat an old timer.

[ansrivas]

BTW I can still see remnants of the new scoring style on my problems page. The badge at the top right.

[ed_r]

How about a "most improved" list as well? That would be a good incentive for newbies.

I think we need to build into it the consideration of newer members

... which makes Ed wonder if Colin actually read his suggestion

[JohnMorris]

(3) We use entirely different ranks; for example, fish, periodic table, to mention as few suggestions already made.

Academic ranks?

Freshman, Sophomore, Junior, Senior, Alumni, Lecturer, Associate professor, Professor, Dean, Bursar...

Needs some work - and could end up coming over all Pratchett. Not that that's a bad thing

eta: Finding a more acceptable hierarchy would be easier if a smaller number of wider bands were used, instead of (if I recall correctly - it's not visible now) 20 or so bands each only 10 problems wide as the military system has.

[euler]

... which makes Ed wonder if Colin actually read his suggestion

Sorry, Ed. I did see it, but I forgot to reply. I must admit that I couldn't think how that could be done. Unfortunately we don't keep track of when users register, only when they solved the last problem.

And some good suggestions for FAQ. Perhaps you could remind me to address them after we've (semi-)resolved the current quandary.

[BjornEdstrom]

Here's an idea that could work: Use the new system, but simply replace the names of the ranks and the pictures with a sports theme instead of the military theme. There are medals and cups/goblets in the high ranks, and these small sports rosettes in the lower ranks. Perhaps "belts" too, such as in the martial arts. Some ideas:

White belt to black belt.
White rosette to black rosette
Bronze, silver, gold, platina medal
And then the mythical statuettes, such as Gauss, Newton...
Lots of possibilities here.

By the way, here's a wikipedia article "list of prizes". It wasn't very useful, but I'll write it here anyhow: http://en.wikipedia.org/wiki/List_of_pr ... and_awards

[rayfil]

I'm still amazed at how many people have indicated strong feelings against military ranks all of a sudden, when this forum has been using them ever since inception, i.e. Cadet, Corporal, Sergeant, Wing Commander, etc., etc. without ever the slightest hint of a complaint.

[neverforget]

I'm still amazed at how many people have indicated strong feelings against military ranks all of a sudden, when this forum has been using them ever since inception, i.e. Cadet, Corporal, Sergeant, Wing Commander, etc., etc. without ever the slightest hint of a complaint.

It has to do with visibility. It's like right there in your face when you log in and view your problems. Nice big shiny killer badge to go with my daily problems. I don't think the forum rankings make it quite that obvious. Instead of being 100% genius, you get to be commander in chief. I'm not sure if that would make people feel smarter (though arguably, neither will "5% genius").

Academic ranks?

Freshman, Sophomore, Junior, Senior, Alumni, Lecturer, Associate professor, Professor, Dean, Bursar...

Needs some work - and could end up coming over all Pratchett. Not that that's a bad thing

eta: Finding a more acceptable hierarchy would be easier if a smaller number of wider bands were used, instead of (if I recall correctly - it's not visible now) 20 or so bands each only 10 problems wide as the military system has.

Yeah I think academic ranks are good, and I don't expect many to be upset about them. In any case, adding the option to switch between academic, scientific, or whatever ranking they please should solve all controversy.

We might even give awards for things such as most accurate (fewer wrong submissions), most active (more problems solved recently), etc.

In terms of scoring however, I would choose the exponential decay weighted average over a moving average. After all, solving problem 1 shouldn't be worth nothing, given that the earlier problems are usually then ones new members will likely attempt first. Also, I'm still in favor of the "easiest 10 unsolved problems" button.

[Polytope]

I was very disappointed by the new ranking system. When I joined Project Euler, I had the goal of reaching the top 1000. It was a lot of work, but I knew exactly how many problems I had to solve in order to achieve it, and I could measure my progress. Once I made it to the top 1000, I made a goal to reach the top 500.

With the new system, it is difficult to find my current ranking, and it is hard to estimate my progress. Previously, I knew that I could reach the top 500 by solving 8 more problems, but now I don't know where I stand. Moreover, it seems rather cruel to change the rules so abruptly, after we have invested so much time playing by the old rules. If a new ranking system is created, it should coexist with the old system instead of replacing it.

I also took great offense to the military insignia on my profile, but I trust that this will be removed soon.

[thizzymarley]

Whatever "rankings" or whatever you decide on, I hope it is like the last system in that it is not displayed prominently in the profile.

The element of competition may be a good way to interest people, particularly younger people, in mathematics and computer programming. But I solve problems for fun, and to learn things; I don't want to compete, and I don't want my profile to remind me that I am part of a hierarchy. (For that matter, the military hierarchy is as offensive to me as an "American grad student" [lowest] to "Romanian 12-year-old" [highest] hierarchy would probably be to other problem solvers. Frankly, I'd prefer to attain the status of a Romanian 12-year-old than a commander-in-chief.)

Polytope also raises a good point. Whatever one might say about the old top 1000 list, it was easy to understand and provides continuity for old users. (If overhead is an issue, you could update it once a day, or once a week; the point being that it is still there somewhere.)

[alexk]

I suggest to calculate the rank as following:

For each problem solved, sum the inverse of the number of people that solved it. This way, the more people solve the problem, the less influence it has on your score. The final score can be a ratio of this sum to the sum for all problems.

This should be pretty newbie-friendly, as we ( ) can concentrate on the latest problems that are not solved by many people yet.