New wait time system.

[albert]

Dear all,

I fear that the current measures against cheating will be utterly ineffective.
This may become the end of the projecteuler.net as we know it.

These are my idea's about how to counter this:

- mark problems whose solution is currently listed. Good solutions to those are marked, but only
lead to a display in the scores list if accompagnied with at least one other solution.
- white list. Those people in the eulerians list are an elite, and it should not be difficult
to find out whether someone belongs there.
There are a few ways to get on the whitelist:
- any publication in the threads list that exhibit original thoughts would do
- "speed points"
- previous merits.

Persons not on the whitelist would be required to submit their programs.
Even if the checking of the programs is not thorough this would be a great deterrent.

Speed points is a ranking based on the speed with which a solution is send.
This would be 1/2 point for the first 1/3 for the second and so on.
Furthermore it would be cumulative.
It goes without saying that a person that scored two times in 20th, and 1 time in the 10th
is not a cheater.
This score I would think is independantly interesting.

Previous merits: people who have been in the eulerians list for more than 10 consequitive weeks.
(although it is hard to see they would not have gathered sufficient speed points.)

I think introducing speed points would already be a great way to identify cheaters anyway.
There is no way a person that has to wait for a solution to appear in cheat list can gather substantial
speed points. Being tenth gives 10 times the number of points as being hundredth.
(And it can be worthwhile to solve that one problem that is still at a 300 solutions level.)

Groetjes Albert

[kops]

Hello all.

I am an on and off problem solver on Project Euler, just about to reach level 2 currently, and while I fully understand the necessity of the new wait time system, I think it is a bit excessive, or implemented too early.

As I mentioned, I return to Project Euler on and off, maybe only one day every so often, but when I do return, I enjoy solving several problems in one sitting. However, a seven hour wait time makes this completely impossible!

I am sure you will not find it hard to believe that several problems on the first one or two pages can easily be solved within a couple hours by a low ranking member such as myself. For instance, after having some troubles on Problem 44, I finally solved it today and moved on to Problem 47 which took mere minutes; yet now I find myself unable to submit my solution. Furthermore, I really don't understand what a cheater might benefit from being able to reach level 3, or even level 4 quickly, and consequently question the necessity of this wait time at those levels. Additionally, this wait time forces users who wish to rise up the ranks from scratch to forgo harder problems for easier ones simply so that they can keep their submissions continually on "cool down".

Again, I fully understand and support the overall concept of a wait time such as what has been implemented, but I do think that it's activation as soon as one reaches level 1 is a bit excessive and contrary to the sustained intellectual challenge Project Euler is supposed to provide.

[euler]

It shouldn't be 7 hours if you're only on level 2. I think the problem is that the date/time it's displaying is not adjusting to your time zone. I've just modified the script, but I am thinking it might be better to display something like: "You may not submit an answer to this problem for another __ hours/minutes."

I must also point out that this is not a final decision and we are open to discussion, welcoming constructive suggestions and alternative ideas.

As mentioned in the News feed we are not happy to have been forced into this situation, and in all of our discussions we have always kept the need to ensure that we do not punish to many for the few as a priority. However, we must also consider the main reason for taking this action. Earning a high score at Project Euler has always been, and always should be, an incredible achievement that is worthy of recognition. To allow cheating half-wits an opportunity to sit alongside some of the cleverest people on the planet without doing something to stop them would be beyond negligent on our part. So please understand that the future and integrity of Project Euler is our utmost priority.

[hk]

I merged the two topics and made it sticky.
Please do not start separate topics.

[Coincoin]

I too come occasionally on euler to solve a couple problems at a time and I also agree it is a bit excessive at lower levels. I'm only level two and most problems at that level still easily take less than an hour to solve, yet, it seems I have to wait a couple of hours before I can even try answering again.

More possible solutions :

- Maybe we could only have the wait time on the last 25 problems since they are the ones that count for the eulerian ranking? Anyone cheating her way up before those last problems is only doing a diservice to herself by not enjoying the challenge.
- Accumulate unused time. If someone solves 4 problems one day and don't come for a week, it's obvious she's not cheating her way up. It would be like accumulating unused cell phone minutes for the next month.
- A white list for accounts with obvious non cheating activity.

[borso]

Sorry guys, but this is a terrible idea. It's too troublesome for normal users but we gain nothing, this system won't stop cheaters from cheating. Why don't you just look for users who solved multiple problem in few minutes and ban them?

[yashkochar]

My two cents,

From the news feed,

This is fair because as you continue to solve more problems then the remaining problems will become increasingly difficult, requiring more time to solve, and this should be reflected quite naturally in the increasing wait time.

This is not entirely true. There are people higher up the lists who have skipped the "easier" problems and have solved the "harder" ones. Also if someone wants to posts in the forum, you have to attempt the "harder" problem early on. And then if they solve the easy one after it they will have wait longer.

I like the idea of "accumulating unused time". However it is also easy to beat. All you need to do is to create an dummy account and leave it for "long" duration and then come back and use the "accumulated unused time". Though the wait by itself can deter some of the cheaters. Many cheaters are impatient to get the "fame" .

Creating a whitelist is okay, but doesn't help much. It is excessive work for the administrators and the pay-off is minimal. Also I don't think the current system affects most of the people at higher levels ( these will be the one to make it to the whitelist ).

How about if the wait time is scaled by the difficulty of the problem instead of number of problems solved?

Also what about problems like 18 and 67 which are similar. If you solve one of them, they you can easily solve the next one, consequently you should not be required to wait too long to be able to submit a solution to that problem.

All this apart,
I am not sure why the ranking system is so important, unless it has some "real world" importance. As already mentioned on the About page,

If you photocopy a crossword solution then what have you achieved?

But then that is an all together different discussion.

Edit: Just to make it clear, I am not against the wait system.

[Antic_Hay]

Hmm...

Am definitely not enjoying this. The wait time is far too long for the level I'm on (Level 2) and it's really irritating to think you have solved a problem and having to wait around before you can even test your solution. It also destroys the way I go about solving the problems, often I'll be stuck on several problems one night and then go sleep on them, and come back the next day refreshed and find I can solve them easily when my head is clearer.

What's more it's fun to submit your solution, then refresh the statistics page to see how you have gone up in the rankings, then immediately be invigorated to go off and solve another puzzle. Obviously this fun is curbed if you're stuck hanging around for your window to open before you can submit your solution.

Also, depending on the problems, some you can go away and have a hack at the theory behind it on pen and paper, maybe on a bus or train commuting, and come home with the solution fresh in your head, ready to code up your solutions and fire them off one after another. Again, this is impossible with this new time system...

Can I think of a better solution? I don't know, but I'll posit that this one is worse than no solution, I'd rather have a few cheaters taking some of the top slots than be stuck as I am right now without being able to advance...(It's nearly 4 am where I am and I'm only awake because I'm anxious to see if I'm right about one of the problems that I'm pretty sure I completed an hour ago but can't confirm)

[quilan]

Just as a bit of a devil's advocate, I remember absolutely plowing through solutions for at least ~100 or so problems in trivial time when I was just starting the site. It wasn't until I got into the real cleverly designed stuff of 150+ where things began to slow down dramatically. While there were outlier problems, my gut instinct says that most everything from problems ~150 onward should probably have the wait system in effect, but stuff before that (especially <100) doesn't need any sort of wait.

As I don't know the exact lengths for each level, I couldn't posit how annoying this currently is, but I just felt I might as well offer up what my experiences were.

Edit: oh, and I fully 100% support a wait system in general. Cheating DEFINITELY needs to be reigned in. I'm very proud of the work that myself & all the other 100%'ers I've discussed things with in the threads over the past years have gone through. There's something immensely satisfying with seeing others finally manage to crack the secrets of problems like 177, 198, 241, 245 etc, so it's a shame to see people instantly bypassing that fun bit of self-discovery. I realize that anti-cheating will never solve the problem (just delay it), but if it'll help then go for it.

[remy72]

Some problems may also be personalized so that the solution for user A would usually differ from the solution for user B.
Concretely, a personalized version of Problem 10 could be :

Find the sum of all the primes below "2000000+f(user)".

f(user) being a function of some internal numeric userid.

[genious999]

The problem with that is people could still just find the code that gives the answer and plug their personal input into it.

[stijn263]

The main reason for this new wait time system is the "automatic problem submitter" that appeared on a certain website ?

If that's the case, add a new answer box with a random question that's tough to solve for a computer, but trivial for a human. Like:

What is the sum of six and four? --> ten
What is the fifth letter in the word microwave? --> o

etc. Or just ask for the letters in a picture, like hotmail and gmail use when you want register an account.

[unc]

Hi All,

I'm a college student with previous experience in both mathematics and programming/algorithmic competitions and have just found Project Euler recently. As such when I sit down and start working on these I can solve a typical problem in my current range (currently problems 30-50) in about 5 minutes of thinking and implementation. I have thoroughly enjoyed Project Euler up until now because it focused more on the mathematics side than the algorithmic side as these types of online sites often do. Thus I was frustrated when I found out about the new timer system implemented as my queue of pending solutions is rapidly outgrowing the pace at which I can submit (currently 30 minutes).

I understand the need to deter cheaters and keep the rankings mostly legit, and it may be the case that a timer system is the only effective way to do this. However do keep in mind that at the end of the day any honest solver can take pride in the problems they have solved and shouldn't need a position on a ranking list to validate their accomplishments. To seriously degrade the experience of the majority of users to deal with a handful of cheaters is a rather harsh step to take. At the minimum, please decrease the time between solutions between levels 1 and 2, and possibly other levels if it is a problem there too.

[hk]

I suggest that you alternate between some of the harder problems and those easy starters.
I assure you some of the harder problems will take you more than your current wait time restriction.

[unc]

Yes, that is what I'll probably have to do, but part of the fun for me was going through all the problems sequentially. I'm a little bit OCD like that I guess . Still though, a good system shouldn't have to force a user to do this in order to keep pace with the submission timer.

[elr]

how about changing the problems from some time to time ?

for example many problems ask "find all ... in range 1 < x < 10000.... " etc
so when the project euler team find that some site published a solution just change the problem for like 1 < x < 10001 and so
its surely eliminate most of the lamers

[hk]

Yes, that is what I'll probably have to do, but part of the fun for me was going through all the problems sequentially. I'm a little bit OCD like that I guess . Still though, a good system shouldn't have to force a user to do this in order to keep pace with the submission timer.

Currently we are evaluating reactions.
The system has not been in use long enough to make decisions about the final wait time restrictions.
That was one of the reasons we kept the news section vague.

Another suggestion could be: start reviewing solved problems. Was your solution the most efficient one?
Is there a more efficient one in the forum??
I must confess that when I started on Project Euler years ago I wasn't taking much care of that.
And for the limits given that wasn't necessary either.
With great joy I have been reviewing problems as low as the range in with you are working now every now and then when there was not much problem development to do.
I just redid one of them yesterday night. With red cheeks of shame I looked at what I had been doing then.

[Antic_Hay]

All these suggestions are all well and good for what to do while waiting, but quite often what I want to do is simply get as many problems done as fast as possible, for me some of the most fun I get out of Project Euler is picking some random target above me in the score rankings and saying "I bet I can get a score above his tonight". With the system in place, I know no matter how fast I am, there is always a lower bound on how fast I can reach my goal, and this I think is terribly dissuading. And fun though it is to go back to old problems and write better solutions (I've written more than my fair share of turkeys as I've gone through the problems) when I don't feel like doing that and feeling like 'racing' instead, it will be irritating being held back.

I really can't think of any other solution beyond vetting source code for high ranking users (which would require significant community effort and probably isn't viable) that works, but I really would suggest erring on the side of caution with regards to the time limits chosen. Consider, you shouldn't be picking a time period that you think a reasonable user should need at least, but rather a period that is more suited to an exceptional problem solver, as they are the people who are most likely to get frustrated and leave and not come back, especially if they've built up a large backlog...

[dnl]

While I'm very sad that a wait system needs to be implemented, I do understand that some people will cheat. Dishonest people exist in all fields, not just politicians, lawyers, sales people, and journalists.

So, I'm OK with the concept. However, basing the wait time on the problem number isn't correct. Problem #48, for example, is dead easy, yet I still need to wait another 1h15m after I solved it! The wait period should be based on the complexity of the problem. Perhaps a wait time based on the problem's complexity ranking (as you use for that view) or inversely related to the number of people that solved it?

[Antic_Hay]

Perhaps a wait time based on the problem's complexity ranking (as you use for that view) or inversely related to the number of people that solved it?

Inverse relationship to number who have solved it is probably a good idea.
I solved Problem 205 just now, compared to others in the 200+ range it was a doddle. I didn't manage to solve it in 2 hours, but I'm sure there's more than a few people who looked at it and coded it up in 5 minutes flat.