Comments (5)
So yeah, fundamentally this is not a bug, just a consequence of the fact that chess engines are not perfect players/evaluators.
Capping at 0, or even reporting negative ACPL scores seems fine.
If absolutely nescessary, each position could be evaluated with sufficiently large MultiPV. Then calculate the loss as the difference between the picked move and the best move as seen from the current position (rather than the difference between the positions). This would always produce consistent results, but is rather inefficient.
from python-chess-annotator.
Try analyzing the game from last ply to first ply & see if the issue persists?
from python-chess-annotator.
If I'm understanding your suggestion, I think this is already the existing behavior. python-chess-annotator starts at the end of the game and works its way backward.
The idea is to populate the engine's hash table with analysis about future positions as we go, hopefully improving the quality of the analysis as we reach more complicated middlegame positions. I haven't done any testing to verify that this actually works as intended, however - we don't set, nor provide any options for the user to set, the engine's hash table size (though we probably should), and I don't even know if python-chess' UCI implementation would preserve the hash between engine calls.
Regardless, can you maybe expand on your thinking here? How do you think/suspect reversing the processing order might affect ACPL calculations?
from python-chess-annotator.
Also, this happens rarely enough that it's very difficult to reproduce. Very often, if I get a negative ACPL, I'll re-run the analysis immediately after and it will turn up positive the second time around.
from python-chess-annotator.
Oops, you're correct that was my idea (to populate the hash table such that already-searched positions and evaluates may be re-used)... hm.
Of course theoretically (with "accurate" evaluations) there should never be an evaluation gain (negative CP loss) between positions. But the default hash size should be adequate, and Lichess analyses have a slightly less tight budget (4 CPU-seconds/move, so ~5 CPU-minutes/game) without major quality problems.
It would be useful to see (rare) examples of the issue in order to measure to what extent any of the following ideas help:
- ML-based (with the cost function being some measure of analysis quality) time budgeting using all available data as input (including numbers produced by Stockfish
eval
command) - Derive a formula from Stockfish
timeman.cpp
and/or find some way to use it to auto-budget CPU (time, threads, memory) to use - Upon detecting an "inaccurate" evaluation, use some heuristic to gracefully recover and produce more "accurate" evaluations.
This all presumes that evaluations can be "accurate"; of course, every legal chess position falls in one of three categories:
- won
- drawn
- lost
and evaluations are simply approximations in cases where a mate (or forced draw) cannot be detected.
from python-chess-annotator.
Related Issues (20)
- -h does not work HOT 2
- Insert human readable commentary HOT 9
- add ?!, ? and ?? HOT 2
- Always annotate the final position HOT 2
- Logging messages should not use string formatting HOT 1
- disregard
- Setting ACPL headers on a root node fails due to new validation in python-chess
- Use reversed() instead of this weird inverted manual for loop stuff HOT 3
- Use reversed(node.mainline()) instead of manually looping backward all the time HOT 1
- Close the eco json file
- Leftover gnuchess process
- Don't use "assert" for failure mode detection
- Broken with recent versions of python-chess
- Zero division error in ACPL calculations
- Division by zero if ply_count = 0 HOT 4
- notice of new project HOT 2
- A few questions about how this works HOT 6
- ZeroDivisionError: float division by zero -- seems to be a different one from reported previously HOT 12
- How to save the result? HOT 4
- Constantly opens a new Stockfish.exe. Because of this, in the task manager, a lot of open stockfish.exe. The program apparently on each batch opens the Stockfish anew. And the old process does not close. How to solve this problem? Because of this, the RAM is overflowed.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from python-chess-annotator.