Q [4] 2. [3] [5] ϵ- Q Q CO CO [4] Q Q [1] i = X ln n i + C (1) n i i n n i i i n i = n X i i C exploration exploitation [4] Q Q Q ϵ 1 ϵ 3. [3] [5] [4]

Size: px

Start display at page:

Download "Q [4] 2. [3] [5] ϵ- Q Q CO CO [4] Q Q [1] i = X ln n i + C (1) n i i n n i i i n i = n X i i C exploration exploitation [4] Q Q Q ϵ 1 ϵ 3. [3] [5] [4]"

みさきたておか
5 years ago
Views:

1 1,a) 2,3,b) Q ϵ- 3 4 Q greedy 3 ϵ- 4 ϵ- Comparation of Methods for Choosing Actions in Werewolf Game Agents Tianhe Wang 1,a) Tomoyuki Kaneko 2,3,b) Abstract: Werewolf, also known as Mafia, is a kind of game with imperfect information that features information asymmetry. Recently, artificial intelligence achieved excellent performance in the world of perfectinformation games, however, there are still room for development of imperfect-information games. In this paper, we implemented the agents of Villager, Seer and Werewolfin which we applied our model on the basis of Q-learning, a reinforcement-learning technique, and compared ϵ-greedy,, and methods for choosing actions. For each method, we evaluated it in 3-to-4-player games by the win ratio of agents after learning multiple cases. The experimental results by evalutaing win ratio in greedy method showed that the agent learned in ϵ-greedy performed best in 3-player games while in 4-player games, the Villager agent used method, the Werewolf agent used ϵ-greedy method and the Seer agent used method in stage of learning performed best after learning respectively Graduate School of Interdisciplinary Information Studies, The University of Tokyo 2 Interfaculty Initiative in Information Studies, The University of Tokyo 3 JST, PRESTO a) wangtianhe@g.ecc.u-tokyo.ac.jp b) kaneko@acm.org [2] [6] 2017 Information Processing Society of Japan

2 Q [4] 2. [3] [5] ϵ- Q Q CO CO [4] Q Q [1] i = X ln n i + C (1) n i i n n i i i n i = n X i i C exploration exploitation [4] Q Q Q ϵ 1 ϵ 3. [3] [5] [4] Q [3] ϵ- softmax Q Q 3.1 CO A B B C C A B B C CO CO 1 N 3 N N CO i j i j 3.2 Q Information Processing Society of Japan

3 CO CO CO ( 1 ) ( 2 ) ( 3 ) ( 4 ) 8 ( 5 ) ( 6 ) 3.3 Q [3] Q [5] Q Sample Player 5 ϵ ϵ ϵ- ϵ CO CO CO 4. [3] [5] Information Processing Society of Japan

4 ϵ ϵ ϵ ϵ ϵ- 13 ϵ- ϵ- ϵ Information Processing Society of Japan

5 ϵ ϵ Sample Player Sample Player CO Q ϵ- Q ϵ- 70.0% 37.2% 39.6% 60.8% 37.1% 36.0% 66.7% 35.8% 38.7% 66.4% 36.0% 39.6% ϵ- ϵ- 5 4 ϵ- 66.8% 52.7% 28.1% 57.3% 50.1% 21.3% 53.8% 51.8% 52.2% 7% 52.2% 31.2% ϵ- Q Q Q 5. Q ϵ- 3 ϵ- 4 ϵ Information Processing Society of Japan

6 JSPS 16H02927 JST [1] Sutton, R. S. and Barto, A. G.: Reinforcement learning: An introduction, Vol. 1, No. 1, MIT press Cambridge (1998). [2] Technologies, D.: DeepMind, Accessed July 24, [3] Vol. 29, pp. 1 3 (2015). [4] 30 pp (2014). [5] Vol (2015). [6] (2016) Information Processing Society of Japan

2797 4 5 6 7 2. 2.1 COM COM 4) 5) COM COM 3 4) 5) 2 2.2 COM COM 6) 7) 10) COM Bonanza 6) Bonanza 6 10 20 Hearts COM 7) 10) 52 4 3 Hearts 3 2,000 4,000

2797 4 5 6 7 2. 2.1 COM COM 4) 5) COM COM 3 4) 5) 2 2.2 COM COM 6) 7) 10) COM Bonanza 6) Bonanza 6 10 20 Hearts COM 7) 10) 52 4 3 Hearts 3 2,000 4,000 Vol. 50 No. 12 2796 2806 (Dec. 2009) 1 1, 2 COM TCG COM TCG COM TCG Strategy-acquisition System for Video Trading Card Game Nobuto Fujii 1 and Haruhiro Katayose 1, 2 Behavior and strategy of computers