Page 1 of 1
Suggestion.. About ratings
Posted: Tue Jan 15, 2008 1:11 pm
by nelapsi
At the moment rating of the is user is just a number of solved problems which is a bit unfair. For example, if user A solved Problem75 (which have been solved by about 540 users) and user B solved Problem1 (solved by more than 15000 users and it can be solved even without pen andd paper). Now these 2 users will have the same rating.
So I think it might be a good idea to create alternative rating (and alternative top1000) which will take in account difficulty of the solved tasks.
For example let
1) U - number of users who solved at least one task
2) ui - number of users who solved i-th task.
2) pi = U - ui
Then we can calculate user's rating like R =[sum]log(pi) (sum over all solved tasks).
Also we can calculate "month" rating (in this case U will be a number of users who solved at least one task during last month - i.e. "active" users)
PS or there might be several standard ratings systems or even some general engine wich will allow users to create own rating systems
Re: Suggestion.. About ratings
Posted: Tue Jan 15, 2008 1:55 pm
by WoLpH
The problem with something like this is mainly the period I guess.
The problems at the top of the list are considered to be easyer and tried more, does that make them easyer? For some yes, for others no.
The second point however is directly related to the amount of time the problem has been online and the people that have solved it, but that makes the rating a bit inacurate, the newest problems have nearly no users that have solved it, although that doesn't mean it's really hard.
So... when calculating the difficulty of a problem it would probably be a good idea to take the date it was placed into consideration aswell

For example, problem 171 has 99 people that solved it, while problem 176 only has already has 78 people that solved it, this probably indicates that 176 is considered to be easyer than 171.
Re: Suggestion.. About ratings
Posted: Tue Jan 15, 2008 2:24 pm
by hk
Until about a year ago we had a dynamic rating system for users ranks and problem difficulty.
About that time the number of solvers already became increasing faster and faster and the dynamic system began to influence server speed so with pain in our heart we disabled it.
Since then we have discussed several semi static rating systems but until now we did not find one that covered our situation well enough.
In the year that followed the speed of increase of the number of members has only been growing every month until about onr or two weeks ago the whole database began to bog and we had to introduce other restraints on the database use.
So I don't know, even when we can think up a valid rating system, we can afford to burden the server with the increased load.
Currently, to be on the lowest page of the top thousand one has had to solve 56 problems. I assume that by then the easiest problems will have been solved by that users.
Re: Suggestion.. About ratings
Posted: Tue Jan 15, 2008 2:36 pm
by nelapsi
well, it might be a good idea to take in account task's "age"...
But just as I said there might be several rating systems and user can choose which of them is "better"...
Regarding "age" - as well we can take in account, for example, number of wrong answers for this task, or number of users, who entered answer (right or wrong), or smth else...
Anyway
Our 1st aim, I think, is to show what other rating systems are needed (in addition to the existing one)
Then, if we get administrations consent, we can discuss how to calculate rating
Re: Suggestion.. About ratings
Posted: Tue Jan 15, 2008 2:44 pm
by nelapsi
about server loading..
I can see at least 2 ways:
1) calculate our ratings once per day (yep, this can hard enough too)
2) once per day download all required information to the separate database, which is used only for ratings
PS Existing rating is
really unfair, because for me most of tasks don't seem to be interest but I want to be in the top1000

Re: Suggestion.. About ratings
Posted: Tue Jan 15, 2008 2:48 pm
by hk
More than one rating system?
Did I not make it clear enough that we had severe problems with the serverload in the current situation?
Using more than one system would make that only worse.
Currently we are recovering from the fact that our provider disabled our entire database due to excess queries to it.
I'm afraid we will have to do with the current system until we are
1) sure that the load on the database stays manageable
2) we are sure that what we can think up one is better
3) what we can think up needs only updating once a week or once a month.
4) what we can think up does not increase the serverload in any way.
Re: Suggestion.. About ratings
Posted: Tue Jan 15, 2008 2:49 pm
by hk
nelapsi wrote:PS Existing rating is
really, because for me most of tasks don't seem to be interest but I want to be in the top1000

Just solve at least 56 problems.
Re: Suggestion.. About ratings
Posted: Tue Jan 15, 2008 2:53 pm
by nelapsi
hk wrote:Just solve at least 56 problems.
I'm too lazy to solve uninteresting tasks

Re: Suggestion.. About ratings
Posted: Tue Jan 15, 2008 2:56 pm
by nelapsi
But with current number of active users even with "fair" rating system I probably won't have enough free time to solve enough interesting tasks to get to the top1000

Re: Suggestion.. About ratings
Posted: Tue Jan 15, 2008 5:17 pm
by euler
I understand your concerns entirely, nelapsi, and one of our senior team members, hk, has already explained the dilemmas we face. Certainly as we add more and more problems to the database it will become increasingly difficult for new members to rise into the top 1000 scorers. We are constantly discussing better ways to score problems and rank members, but for the moment at least we need to ensure that the massive overhaul of the scripts have managed to tame the server load. Once we have established the the new scripts present a consistent and reasonable demand on the servers we will be in a better position to try out a new system.
However, in the meantime there is no harm in discussing ideas here so that when/if we establish the servers can cope with additional demand we will be better placed to implement a new system.