Machine learning as a game mechanic. - TL;DR: A video game where the AI progressively learns to react to the player's behavior: good idea yes or no?

[Penis Drager]

Imagine a game, let's say some kind of zombie horde shit for example, where something akin to Q-learning or a genetic algorithm is used to progressively find the optimal strategy to defeat the player. With the zombie example the zombies wouldn't necessarily increase in number or health or any metric like that. The zombies would just get progressively "smarter" and waves would be comprised of more and more of the kinds of zombies or combination thereof that particular player has the biggest problems with. It would slowly "learn" to spawn zombies in locations in which the player tends not to look or has a harder time dealing with in general. Overall: the AI will react to a given player's preferred playstyle starting with what tactics get zombies closer to the player and progressing to what ends up dealing more and more damage or expending more and more of his supplies.
An obvious drawback to this is that genetic algorithms in particular take a lot of iterations to get anywhere so the player might just get bored of beasting through dozens of waves of retarded ass zombies so some level of "hard coding" will be required to make the early game fun before any actual "learning" happens. Another is that, while this sort of scheme is basically designed to prevent the player from cheesing quirks in a hardcoded AI, an alternative cheese strategy opens up where the player could just do things a little different every wave or even acting dumb on purpose in an effort to intentionally make the AI dumber.

Note that a zombie game is just a sort of obvious use-case for this sort of thing, an RTS or even fighting games could benefit greatly from it if done competently. Also this is basically just mental masturbation as I have no experience or aspirations in game design.
Thoughts?

Edit: added poll for shits and gigs.

 

[Some JERK]

From what I've seen of machine learning AI in video games, that would be a pretty boring and frustrating game for a very long time before it got interesting or fun.

 

[Stephanie Bustcakes]

player might just get bored of beasting through dozens of waves of retarded ass zombies so some level of "hard coding" will be required to make the early game fun before any actual "learning" happens.

Or they could have been pre-trained on play-testers before shipping. So instead of starting at iteration 0, they start on iteration 100, and your playing brings it to 101, 102 etc. It'll not be tailored to you specifically, but would still be smart enough to play until it figures you out.

 

[Penis Drager]

From what I've seen of machine learning AI in video games, that would be a pretty boring and frustrating game for a very long time before it got interesting or fun.

Yeah I addressed this. If you did "pure" machine learning, the first dozen waves or so would be acting completely randomly: running in circles, running into walls, running away from the player (a particularly frustrating scenario), etc. So you'd need to start with a halfway competent AI. This could take the form of starting them off with a basic "attempt to merge your hitbox with player's hitbox" command or something like what @Stephanie Bustcakes is suggesting.

 

[wtfNeedSignUp]

It won't work, not because it will take too much time, but rather because the AI will quickly rape the player, leading to an unfun experience overall.
This in general has always been the problem in video games, the vast majority of players are fucking idiots who use the same working strategy they found with no deviation, or the games themselves are built on the AI being retarded.

 

[Penis Drager]

It won't work, not because it will take too much time, but rather because the AI will quickly rape the player, leading to an unfun experience overall.
This in general has always been the problem in video games, the vast majority of players are fucking idiots who use the same working strategy they found with no deviation, or the games themselves are built on the AI being retarded.

I actually had a discussion related to this about Crysis (I never played it myself so this is all coming second hand). I guess a lot of people hate the game because the AI was actually competent and knew how to use cover and "complex" (as far as video games are concerned) strategies like flanking.
But I feel like there will be a niche set of players that would be into a game that learns from them for the gimmick alone.
It absolutely wouldn't be a AAA title. But an indie game on Steam that used this as its main selling point would pull in a few bucks, I think.

 

[libRT]

I've thought of this before, but there are a few problems.
1. Deep learning is CPU/Memory expensive (i think?) not very ideal to run whilst the game loop is running. Maybe iterate whilst in menus but then you won't get much evolution between.
2. Difficulty. If the mobs are too easy (player advantage), machine learning isn't really going to change difficulty much. If the mobs have natural advantage, machine learning will eventually cream most players. Adaptive difficulty might be fun, especially if the player can choose paths that show difficulty and rewards it more. (Like L4D director, though that was just an AI and no ML iirc)
3. Updates, if you were to add new systems/items/traps etc, the AI would need to adapt to these making the dataset previously gathered either broken or shit.
Sure I had more points but can't recall them.

 

[mr.moon1488]

I think it would be fun if done right because it would keep the game from getting too repetitive.

 

[Str8Bustah]

This game already exists, it's called armored core 3

 

[Wintersun]

It's been done before, Hello Neighbor had AI learned predictive pathing, which was pretty neat for a little indie game (before they fucked it up).

Metal Gear Solid 5 also had something similar, though not exactly what you're talking about, in the enemy units adopting different gear and tactics depending on what you did. Lots of headshots would result in them getting helmets, missions mostly at night had them get night vision goggles, decoys you use would eventually be used on you, etc.

I think a combination of learning AI with certain parameters designed to counter whatever tactics you use would be lauded as some of the most groundbreaking programming of all time.

 

[gaystoner]

Elden Ring.

 

[Kosher Dill]

"Zanac" originated this premise, though of course they weren't really running AI on a NES. Great game.

 

[Gender: Xenomorph]

Keep in mind that, to complete a playthrough of Mario Lvl 1 it takes 1000s of itterations.

The first iterrations have trouble getting past the first screen or enemy, and often wait out the timer or die to the first enemy. The player will have a really bad couple of 100s of hours before getting a competent AI.

The AI is really bad at adapting to change and new strats. It may be really easy to cheese the AI for the first couple of encounters (meat shields or artillery harassment). It adapts after it faces these encounters. IMO the biggest drawback.

If it's turn based, the AI will analyze possible moves and pick up the one that it thinks is the most favorable. In a RTS game, you will need to make these sort of checks for 30 seconds, every 1 second, and it may end up very expensive to be practical.

Making a server-based, constantly evolving RTS AI that takes input from players playing against it may be an interesting experiment though. You could have the server constantly revise the AI with each player and may be an interesting experience, but in the first 100s of days, the AI may have trouble buildings its first builder, and in the end it will cheese the engine and ruin the game for players really late on.

 

[Kovic's Work Buttplug]

Didn't MGS5 enemies adapt to your playstyle? If you went for headshots, they eventually all started wearing bulletproof helmets. Were there any other ways they adapted I'm not remembering?

 

[Mr. Brown]

This is something that has existed since PS3.

F.E.A.R was a game that did this in 2014.

 

[ellroy]

This is something that has existed since PS3.

F.E.A.R was a game that did this in 2014.

I think you mean 2005.

The bots in F.E.A.R. were interesting and there hasn't been anything like them since. White paper from MIT concerning them.

However those Bots didn't use machine learning. Machine learning would be terribly boring unless it was already trained. EA apparently had machine learning bots fight each over on a special private server in Battlefield 1 but I haven't heard anything more from their work with it.

 

[Frail Snail]

I'd like to see a very primitive (and therefore not resource intensive) form of machine learning, or just some sort of advanced genetic algorithm, integrated into a game's AI codebase without being the main part of it. Things like navigation paths, manually specified hints and preset core behaviors are solid ideas. Adding depth to the little steps AI takes out of its predefined box based on how you play would be cool. To me, really great computer enemies are like magic tricks. Everybody knows it's sleight of hand and trickery but, when executed perfectly, it's enough to make you feel immersed.

 

[Manul Otocolobus]

I've thought of this before, but there are a few problems.
1. Deep learning is CPU/Memory expensive (i think?) not very ideal to run whilst the game loop is running. Maybe iterate whilst in menus but then you won't get much evolution between.
2. Difficulty. If the mobs are too easy (player advantage), machine learning isn't really going to change difficulty much. If the mobs have natural advantage, machine learning will eventually cream most players. Adaptive difficulty might be fun, especially if the player can choose paths that show difficulty and rewards it more. (Like L4D director, though that was just an AI and no ML iirc)
3. Updates, if you were to add new systems/items/traps etc, the AI would need to adapt to these making the dataset previously gathered either broken or shit.
Sure I had more points but can't recall them.

The first issue has been dealt with. Almost all processors these days have machine learning dedicated cores or integrated algorithms to accelerate machine learning.

The main issue with using machine learning is that eventually the only way to defeat it is to be creative, but most games impose limitations on what is possible given a combination of environment, player, and items. If the game truly allowed complete freedom for the player to act however they want, only bound by the basic laws of the environment and the form of the player, then you would have an awesome game.

 

[BroccoliBrain]

L4D already did that the Director (the name of the AI). It controlled zombie spawns but also item spawns too. The zombie AI itself isn't changed but the Director can really fuck you over by putting something you can't handle in exactly the wrong spot or doubling up spawns for later if it couldn't find a place to put one in the present.

 

[Merried Senior Comic]

If I remember right, the multiplayer bots in Killzone 2 and 3 would adapt to player strategies.