Skip to content

Commit 708cb55

Browse files
sriramsowmithri9807sriramsowmithri9807
committed
Add parameterized social dilemma game with N-player support, dynamic payoffs, and Axelrod-style bots
- Implemented flexible N-player simultaneous-move game (N >= 2) - Added dynamic payoff matrices with configurable change probability - Implemented stochastic rewards with Gaussian noise - Created 7 Axelrod-style bots (TitForTat, GrimTrigger, Pavlov, etc.) - Added comprehensive unit tests (21 test cases total) - Included example scripts for MARL usage and bot tournaments - Added documentation (README and implementation summaries) Features: - Variable number of agents (default: 3) - Dynamic and non-stationary payoff structures - Optional reward noise for robustness studies - Full Python API exposure for all parameters - Compatible with OpenSpiel's simultaneous-move game API Files added: - Core game implementation and tests - Axelrod-style bots and bot tests - Usage examples and tournament demonstrations - Documentation files
1 parent 8048a4a commit 708cb55

10 files changed

+1667
-0
lines changed

‎IMPLEMENTATION_SUMMARY.md‎

Lines changed: 191 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,191 @@
1+
# Implementation Summary: Parameterized Social Dilemma
2+
3+
## Overview
4+
5+
This implementation addresses GitHub issue for adding a parameterized social dilemma game to OpenSpiel. The implementation provides a flexible N-player simultaneous-move game that supports variable agent counts, dynamic payoff matrices, and stochastic rewards.
6+
7+
## Files Created/Modified
8+
9+
### Core Implementation
10+
11+
1. **`open_spiel/python/games/param_social_dilemma.py`** (Already exists)
12+
- Main game implementation
13+
- Inherits from `pyspiel.Game` and `pyspiel.State`
14+
- Supports N-player games (N ≥ 2)
15+
- Dynamic payoff matrices with optional noise
16+
- Stochastic rewards via configurable noise parameter
17+
18+
2. **`open_spiel/python/games/param_social_dilemma_test.py`** (Already exists)
19+
- Comprehensive unit tests
20+
- Tests default parameters, custom payoff matrices
21+
- Tests stochastic rewards and dynamic payoffs
22+
- Tests game progression and returns accumulation
23+
24+
### New Files Created
25+
26+
3. **`open_spiel/python/games/param_social_dilemma_bots.py`** (NEW)
27+
- Axelrod-style bots implementation
28+
- Bots included:
29+
- `AlwaysCooperateBot`
30+
- `AlwaysDefectBot`
31+
- `TitForTatBot`
32+
- `GrimTriggerBot`
33+
- `PavlovBot` (win-stay, lose-shift)
34+
- `TitForTwoTatsBot`
35+
- `GradualBot`
36+
37+
4. **`open_spiel/python/games/param_social_dilemma_bots_test.py`** (NEW)
38+
- Unit tests for all bot strategies
39+
- Tests bot behavior in various scenarios
40+
- Tests bot interactions in games
41+
42+
5. **`open_spiel/python/games/param_social_dilemma_README.md`** (NEW)
43+
- Comprehensive documentation
44+
- Usage examples
45+
- Configuration parameter reference
46+
- Bot descriptions
47+
48+
### Examples
49+
50+
6. **`open_spiel/python/examples/param_social_dilemma_example.py`** (Already exists)
51+
- Demonstrates basic game usage
52+
- Shows stochastic rewards
53+
- Shows dynamic payoffs
54+
- Custom payoff matrix examples
55+
- Scalability tests with varying player counts
56+
57+
7. **`open_spiel/python/examples/param_social_dilemma_bots_example.py`** (NEW)
58+
- Axelrod-style tournament implementation
59+
- Bot-vs-bot competition with scoring matrix
60+
- N-player tournament scenarios
61+
- Demonstrates bot strategies in practice
62+
63+
### Registration
64+
65+
8. **`open_spiel/python/games/__init__.py`** (Already updated)
66+
- Game is already registered in the imports
67+
68+
## Features Implemented
69+
70+
### ✅ Variable Number of Agents (N-Player)
71+
- Configurable agent count (N ≥ 2)
72+
- Default: 3 players
73+
- Tested with 2, 3, 5, and 8 players
74+
- Compatible with `SimultaneousMoveGame` API
75+
76+
### ✅ Dynamic Payoff Matrices
77+
- Payoffs can change across timesteps
78+
- Controlled by `dynamic_payoffs` and `payoff_change_prob` parameters
79+
- Enables non-stationary environment experiments
80+
81+
### ✅ Stochastic Rewards
82+
- Optional Gaussian noise with configurable standard deviation
83+
- Controlled by `reward_noise_std` parameter
84+
- Useful for robustness studies
85+
86+
### ✅ Python API Exposure
87+
All parameters exposed via Python interface:
88+
- `num_players`: Number of agents
89+
- `num_actions`: Actions per agent
90+
- `max_game_length`: Maximum timesteps
91+
- `payoff_matrix`: Custom payoff structure
92+
- `reward_noise_std`: Reward noise level
93+
- `dynamic_payoffs`: Enable dynamic payoffs
94+
- `payoff_change_prob`: Probability of payoff changes
95+
96+
### ✅ Axelrod-Style Bots
97+
Seven classic strategies implemented:
98+
1. Always Cooperate
99+
2. Always Defect
100+
3. Tit-for-Tat
101+
4. Grim Trigger
102+
5. Pavlov (Win-Stay, Lose-Shift)
103+
6. Tit-for-Two-Tats
104+
7. Gradual
105+
106+
## Implementation Details
107+
108+
### Game Type
109+
- **Dynamics**: SIMULTANEOUS
110+
- **Chance Mode**: DETERMINISTIC (or EXPLICIT_STOCHASTIC with noise)
111+
- **Information**: PERFECT_INFORMATION
112+
- **Utility**: GENERAL_SUM
113+
- **Reward Model**: REWARDS
114+
115+
### Default Payoff Structure
116+
Public goods game formulation:
117+
- Cooperators receive: 3.0 × (cooperators / total_players)
118+
- Defectors receive: 5.0 × (cooperators / total_players)
119+
120+
This creates a social dilemma where individual incentive conflicts with collective benefit.
121+
122+
### Testing
123+
- Core game tests: 13 test cases
124+
- Bot tests: 8 test cases
125+
- All tests use `absltest` framework
126+
- Tests cover edge cases and various configurations
127+
128+
## Usage Examples
129+
130+
### Basic Game
131+
```python
132+
import pyspiel
133+
134+
game = pyspiel.load_game("python_param_social_dilemma", {
135+
"num_players": 3,
136+
"max_game_length": 10
137+
})
138+
139+
state = game.new_initial_state()
140+
state.apply_actions([0, 1, 0])
141+
rewards = state.rewards()
142+
```
143+
144+
### With Bots
145+
```python
146+
from open_spiel.python.games import param_social_dilemma_bots
147+
148+
bot = param_social_dilemma_bots.TitForTatBot(player_id=0, num_players=2)
149+
action = bot.step(state)
150+
```
151+
152+
### Tournament
153+
```python
154+
python3 open_spiel/python/examples/param_social_dilemma_bots_example.py
155+
```
156+
157+
## Compliance with OpenSpiel Standards
158+
159+
✅ Follows OpenSpiel game structure
160+
✅ Uses `pyspiel.Game` and `pyspiel.State` base classes
161+
✅ Implements required methods: `current_player()`, `_legal_actions()`, `_apply_actions()`, etc.
162+
✅ Proper game registration with `pyspiel.register_game()`
163+
✅ Observer implementation for game state observation
164+
✅ Comprehensive test coverage
165+
✅ Example scripts for demonstration
166+
✅ Documentation provided
167+
168+
## Differences from Original Plan
169+
170+
The original issue suggested placing implementation in `open_spiel/games/param_social_dilemma` (C++), but following the maintainer's guidance, this was implemented entirely in Python under `open_spiel/python/games/` for:
171+
- Faster iteration
172+
- Easier parameter configuration
173+
- Better integration with Python-based MARL experiments
174+
- Simpler maintenance
175+
176+
## Future Extensions
177+
178+
Possible enhancements:
179+
1. Add more sophisticated bot strategies
180+
2. Implement tournament ranking systems
181+
3. Add visualization tools for game dynamics
182+
4. Support for asymmetric payoff matrices
183+
5. Integration with learning algorithms
184+
6. Additional social dilemma variants (e.g., public goods with punishment)
185+
186+
## References
187+
188+
- OpenSpiel Developer Guide
189+
- Axelrod's Evolution of Cooperation
190+
- Iterated Prisoner's Dilemma implementation in OpenSpiel
191+
- Multi-agent reinforcement learning benchmarks

0 commit comments

Comments
 (0)