机器学习助力游戏开发：训练模型充当游戏测试人员，改善在线多人游戏的均衡性

Leveraging Machine Learning for Game Development | Google AI Blog

As an imperfect information card game with a large state space, we expected

Chimera to be a difficult game for an ML model to learn, especially as we were

aiming for a relatively simple model. We used an approach inspired by those used

by earlier game-playing agents like AlphaGo, in which a convolutional neural

network (CNN) is trained to predict the probability of a win when given an

arbitrary game state. After training an initial model on games where random

moves were chosen, we set the agent to play against itself, iteratively

collecting game data, that was then used to train a new agent. With each

iteration, the quality of the training data improved, as did the agent’s ability

to play the game.

For the actual game state representation that the model would receive as input,

we found that passing an **"image" encoding** to the CNN resulted in the best

performance, beating all benchmark procedural agents and other types of networks

(e.g. fully connected). The chosen model architecture is small enough to run on

a CPU in reasonable time, which allowed us to download the model weights and run

the agent live in a Chimera game client using Unity Barracuda.

This approach enabled us to simulate millions more games than real players would

be capable of playing in the same time span. After collecting data from the

games played by the best-performing agents, we analyzed the results to find

imbalances between the two of the player decks we had designed.

Normally, identifying imbalances in a newly prototyped game can take months of playtesting. With this approach, we were able to not only discover potential imbalances but also introduce tweaks to mitigate them in a span of days. We found that a relatively simple neural network was sufficient to reach high level performance against humans and traditional game AI. These agents could be leveraged in further ways, such as for coaching new players or discovering unexpected strategies.