Difficulty deflation and open ended difficulty grades (suggestion)

[Yly]

I'm a relative newcomer to Project Euler, and I've noticed that more recent problems of the same nominal difficulty appear to be much harder compared to older problems. For instance, both problem 109 and problem 927 are assessed at 45% difficulty. However, the former I found extremely straightforward and took only about 20 minutes to solve, whereas the latter required at least three non-trivial insights and took me the better part of a weekend.

This is not surprising. In any sport or hobby, the ability level of enthusiasts tends to rise over time. I would like to suggest that Project Euler consider implementing a subjective, open ended grading system for difficulties, akin to systems used in rock climbing. This could allow for greater uniformity across the entire archive, and would have the additional benefit of allowing for future problems of greater difficulty without needing to recalibrate the entire scale. (I also feel that it would be more fun to speak of an "E12" difficulty problem rather than an 80% difficulty problem, but maybe that's just me.)

[byhill]

I assume you know that for problems with a fastest solve table, the difficulty is based on the times on that table? Since many users on the fastest solve table have solved many of the prior problems, you should assume the assigned difficulty is based on the assumption that you have solved many previous problems.

Also, while difficulty deflation is a thing on Project Euler, I think it becomes less pronounced around problem 400 onward.

While in theory, I like the idea of a grading system akin to rock climbing, I think it could lead to its own set of issues. If I recall other .chat posts correctly, the problem setting team have vastly misjudged problem difficulties before. For example, problems 499 and 589 were considered easy before being released.

In addition, with Project Euler, what's hard for on person is trivial for another, even for two people with similar skills in math, computer science, and programming. With rock climbing, it's essentially impossible to complete a V7 if you never topped anything higher than a V3 before, and a V7 climber can almost certainly top any V3 with relative ease. Not so with Project Euler! So I'm skeptical that a subjective grading system would be an improvement over the current system.

Of course, those are just my thoughts

[Yly]

While in theory, I like the idea of a grading system akin to rock climbing, I think it could lead to its own set of issues. If I recall other .chat posts correctly, the problem setting team have vastly misjudged problem difficulties before. For example, problems 499 and 589 were considered easy before being released.

In addition, with Project Euler, what's hard for on person is trivial for another, even for two people with similar skills in math, computer science, and programming. With rock climbing, it's essentially impossible to complete a V7 if you never topped anything higher than a V3 before, and a V7 climber can almost certainly top any V3 with relative ease. Not so with Project Euler! So I'm skeptical that a subjective grading system would be an improvement over the current system.

To clarify, I would propose to let everyone who solves a problem suggest their own personal difficulty for it. This is how it's done on certain climbing sites, e.g. MountainProject.com. In climbing also there can be significant differences in opinion on grades, but I think such differences of opinion themselves provide useful information (e.g. maybe that the problem is "easy once you see the trick", which could apply equally to a boulder or a coding challenge). So I would consider that a feature, rather than a bug.

[Swistakk]

I also do see the difficulty deflation, but I think that tampering with the system at this point would be too intrusive and I wouldn't have liked it, even if they became somewhat more accurate in whatever measure. I think that after you solve problems from various periods of the site, you get a better feel of how their difficulty should be adjusted for your own perception

[KING-OLE]

I've mentioned this several years ago in a different thread, but I think the problem difficulty has increased dramatically, more or less serving the "few" hardcore solvers mostly. Looking at the recent 10 problems, 3 of them have less than 100 solvers, another 3 between 100 and 200, one about 200, one about 300, and only 2 of the 10 have more.

This limits the fun of solving most of these weekly problems to the "math experts". I'm above average when it comes to math, and I'm pretty creative and I'm good at recognizing number patterns, but my better skill is the programming. However, lots of these ends up being brute force, and a lot of those ends up with a CTRL-C after I realize how long it will take to solve it.

Just my 0b10 cents.

[rexcirus]

Nice idea, I support it.

Another possibility is to add more measures of difficulty. I suspect one of the best measurements to be n_of_solvers / time_since_problem_was_published.

[yoni83]

Nice idea, I support it.

Another possibility is to add more measures of difficulty. I suspect one of the best measurements to be n_of_solvers / time_since_problem_was_published.

Solve rates are very non-uniform. Most questions get a lot more solves near the time of publication than when they are old. This is very unsurprising. People try new problems and just move on if they haven't solved them, so the solve rate drops significantly.

For example, I randomly looked at question 841 that was solved by 203 people in that last 2 years. If you look at the fast solver list, half of them solved the question within four days since publication. You will see the same behaviour with pretty much every problem.

The current measure of difficulty takes this into account, as it is based on the solve times of the first 100 solvers. Your suggestion will have a huge bias based on recency. It would rate 841 as harder than 910, a problem that has been solved by less than 80 people in over 6 months.

[shaiephraim]

Maybe let community members try to give their own ranking (1-5), or let user vote on difficulty comparsion between problems(like chess ELO)

[GNFS]

I completely agree with the sentiment that newer problems are harder per difficulty rating, along with the fact that reweighting previous difficulties would be highly disruptive. But I am still in favor of reducing the difficulty rating given to earlier problems because of how big the differences are. Problems like 196 and 199 are at 65% and 70% difficulty, and nowadays they would be like 10-20%.

[00gogo00]

Isn't a big factor in this that the difficulty rating was added several years into the site's lifespan, so the problems before and after that threshold are rated pretty differently because the former had to be determined post-hoc?

[GNFS]

The current difficulty rating is based on the fastest 100 solvers' solve times, but that leaderboard apparently doesn't exist until Problem 277 or so, and a lot fewer people were participating in 2010.

[thedoctar]

Here's my subjective reply.

I think difficulty deflation is only 5-15%.

Actually I think difficulty is a misnomer.

None of the problems are actually difficult, once you know the tricks. And that is (I surmise) by design. Problems are designed so that you only need some (relatively rudimentary) undergrad maths techniques.

Once you've seen a trick, you can easily solve other problems that use the same trick. That's the "AHA" moment.

I consider the difficulty score to mean the “length” of a problem, or how many “tricks” you need.

Most 100% problems have many layers and tricks. But it's not that the trick is super hard, but the problem is composed of say several 40-60% difficulty problems.

I came back to Project Euler ~ a year ago. I solved the most recent problems at the time, 902-930. Going back to earlier problems > 200, there are still many I still struggle on.