Prisoner's Dilemma Tournament update: So, I have results that I could share!
In the end, 1,468 of the total 1,615 strategies passed all four requirements: 1) was a .py file, 2) ran against the default 9 strats in under 10 seconds, 3) didn't do anything nefarious, and 4) never crashed when paired up against any other strategy. Including the original 9 strats means the entire working roster has 1,477 contenders.
However, due to a 1477*1477 matrix having over 2 million entries (1 million bc of symmetry), and some of those entries taking >1 second to run (most are faster), each single "full-pass" takes 8-10 hours to run. I've done 2 "full-passes" so far, and the top strategies on the leaderboard move around n spots, where n is maybe 20% of their rank (so if they're 30th, they might end up 24th the next round.) #2 moves, but #1 does not. This is also because hundreds of strategies scored very close together. Yes, I'm aware there are many ways to optimize, including multi-threading, caching, and more. However, since I'm already confident the top spot won't ever move, I'm wondering how necessary it is to run hundreds of full-passes.
My main dilemma (haha) is this: If I tried to get the results out as fast as possible, I could get the video and the prize money out in a day or two! It would have to be an unedited, OBS-livestream style discussion. On the other hand, if I were to make a more polished video with more polished data, it might take months. (I envisioned displaying the data in a 3D environment with fancy transitions and animations to make the dataset feel like a new continent being explored. Also, I planned on calling every participant who did anything interesting, to include that in the video, but that would be a big logistical task!) I know many people would side with "quality over quantity" (meaning wait for a polished video), but I do think quick-n-easy argument is valid. It would suck to enter a tournament and not know the results for months, just so Cary could make his pretty scatter plot to pulsate... (like, does anybody even care if a graph pulsates, versus being a still image?) So, what do you guys think?
Also, if you're wondering why I posted this as a YouTube community post instead of a Github issue, my last GitHub issue only got 2 comments. So, in the interest of this post reaching as many participant's eyes as possible, I'm posting it here! (If I ever do future tournaments, I'll agree on a singular announcement location beforehand lol)
carykh
Prisoner's Dilemma Tournament update: So, I have results that I could share!
In the end, 1,468 of the total 1,615 strategies passed all four requirements: 1) was a .py file, 2) ran against the default 9 strats in under 10 seconds, 3) didn't do anything nefarious, and 4) never crashed when paired up against any other strategy. Including the original 9 strats means the entire working roster has 1,477 contenders.
However, due to a 1477*1477 matrix having over 2 million entries (1 million bc of symmetry), and some of those entries taking >1 second to run (most are faster), each single "full-pass" takes 8-10 hours to run. I've done 2 "full-passes" so far, and the top strategies on the leaderboard move around n spots, where n is maybe 20% of their rank (so if they're 30th, they might end up 24th the next round.) #2 moves, but #1 does not. This is also because hundreds of strategies scored very close together. Yes, I'm aware there are many ways to optimize, including multi-threading, caching, and more. However, since I'm already confident the top spot won't ever move, I'm wondering how necessary it is to run hundreds of full-passes.
My main dilemma (haha) is this:
If I tried to get the results out as fast as possible, I could get the video and the prize money out in a day or two! It would have to be an unedited, OBS-livestream style discussion. On the other hand, if I were to make a more polished video with more polished data, it might take months. (I envisioned displaying the data in a 3D environment with fancy transitions and animations to make the dataset feel like a new continent being explored. Also, I planned on calling every participant who did anything interesting, to include that in the video, but that would be a big logistical task!) I know many people would side with "quality over quantity" (meaning wait for a polished video), but I do think quick-n-easy argument is valid. It would suck to enter a tournament and not know the results for months, just so Cary could make his pretty scatter plot to pulsate... (like, does anybody even care if a graph pulsates, versus being a still image?) So, what do you guys think?
Also, if you're wondering why I posted this as a YouTube community post instead of a Github issue, my last GitHub issue only got 2 comments. So, in the interest of this post reaching as many participant's eyes as possible, I'm posting it here! (If I ever do future tournaments, I'll agree on a singular announcement location beforehand lol)
4 years ago | [YT] | 1,035