Improve MCTS #17
Improve MCTS #17
Conversation
| var minTurnTillWin = rootNode.getChildren().stream().map(Node::getTurnsTillWin).min(Integer::compare).orElse(Integer.MAX_VALUE); | ||
| var bestChild = rootNode.getChildren().stream().filter(x -> x.getTurnsTillWin() <= minTurnTillWin) | ||
| .max((a,b) -> Float.compare(a.getReward(), b.getReward())).orElseThrow(); |
There was a problem hiding this comment.
Come to think of it, this choosing method has a fatal flaw. Consider the following scenario:
MCTS has two monsters and the opponent has two monsters.
Let's say that MCTS's active monster can win in 6 turns by doing continuous small amounts of damage, and MCTS's non-active monster has a move that will cause MCTS to win first turn it's used.. And let's say the opponent's active monster can't really deal much damage to MCTS's active monster, but would deal massive damage to MCTS's non-active monster.
In this case, this new method will be flawed.
MCTS will have the following options:
- Keep attacking with current monster (win in 6 turns, reward=10)
- Change to other monster (win in 2 turns, reward=0)
If you expand the "change into other monster", you'll see the following scenarios:
- The opponent uses the deadly move and MCTS loses (reward -100, win in turns=MAX_INT)
- The opponent switches to it's other monster (reward : +100, win in turns=1)
This would be a terrible choice, because of course the opponent isn't just going to give you a free win. It will try to kill you immediately.
Even through random playthroughs, it'll probably figure out that going down this path is pretty stupid as it looses in most of the time. but since this new system prioritizes "win_in_turns", it will choose this path anyway. So if you purely looked at rewards it would (in this case) do the right thing. But that dons't go for other cases like two gods on a single team.
Back to the drawing boards for new ideas I guess.
This reverts commit d63dc5a.
Improvements include:
Infinityif the move is too goodKin the custom-battle screen to randomize your monsterWow! That's a lot. And even without dirty hacks to make it work™.