Machine learning as a game mechanic. - TL;DR: A video game where the AI progressively learns to react to the player's behavior: good idea yes or no?

[Krokodil Overdose]

Three words: "Soul Calibur IV." That was a game from nearly two decades ago that accomplished basically what you're talking about not by machine learning or a super-deep AI or whatever, but by having the computer read the human's controller inputs. The single-player experience was as crushingly unfun as you'd suppose. That's the problem with this idea: making a perfect play AI isn't that hard for many games, and after a certain level the AI is effectively cheating through optimized, instantaneous decision making that a human can never do- functionally identical to reading the controller. Deep Learning or whatever just adds extra steps to that process.

 

[Glowie Hunter Art Bell]

I'd like to see this utilized in an open world, GTA-esque + sims + simulator game where the world itself reacts to your antics, which then has an effect on how NPCs interact with you, each other, and the game world, as well as how the game world's design changes in response.

Keep committing crimes in a neighborhood? Higher police presence. Houses in the area devalue and can be purchased for cheaper.

Blow up a major bridge? Traffic in alternate routes is now heavier. Businesses you own in the area such as gas stations profit more.

Shut down a major in game company/facility? The game world loses that company, industries shift. Npcs lose jobs and are more prone to hostile behavior, etc. You see where I'm going with this.

 

[Rich Evans Ayypologist]

You'd need to first make a proof of concept for this in something like a fighting game. Computer has the same options as the player character, a set amount of possible choices and decisions to make. This is a simplified version of what you'd have to do in other games, where the computer has more options than the player (or vice-versa), as in this case you really can use player behavior to very easily train a play pattern. DOTA2 did this to some extent with bots that were fed the inputs and behavior of some of the pros to emulate their style.

If you train some kind of AI based on all players' inputs and behaviors across the world at different levels of skill across different characters, you could train up the exposure. Reference sets between characters in aggregate to inform how to behave - IE, know which dashes will get caught by an SPD and which won't.

I think there's potential if you use the mass of information that's being crunched by thousands to millions of players. But using ML to train it on an individual level is something of a pipe dream, where you're gonna get bamboozled by smoke and mirrors.

 

[libRT]

The first issue has been dealt with. Almost all processors these days have machine learning dedicated cores or integrated algorithms to accelerate machine learning.

Not sure about that processor comment, maybe Xeons have dedicated ML cores, but consumer cpus I doubt it, though they are increasing the amount of cores. GPUs are often used for ML, but you would still need it to run alongside the game. It would be nice if you could use the processor in isolation, but it still would need a fuckton of memory and bandwidth. Running it alongside a game doesn't seem viable even with specialised architecture, it doesn't magically reduce the complexity of the algorithm, training the AI would still take significant time.

The main issue with using machine learning is that eventually the only way to defeat it is to be creative, but most games impose limitations on what is possible given a combination of environment, player, and items. If the game truly allowed complete freedom for the player to act however they want, only bound by the basic laws of the environment and the form of the player, then you would have an awesome game.

Agreed, but eventually the AI would outclass the player unless it was significantly crippled. It would eventually have access to cheesing strategies too, but with the speed and precision of a TASbot

L4D already did that the Director (the name of the AI). It controlled zombie spawns but also item spawns too. The zombie AI itself isn't changed but the Director can really fuck you over by putting something you can't handle in exactly the wrong spot or doubling up spawns for later if it couldn't find a place to put one in the present.

I think it was AI but didn't use ML, though I may be wrong.

 

[Dysnomia]

The only way it's going to have much appeal is if it's some kind of waifu game.

 

[Spud]

MGSV did similar to what you describe, it reacted to how you played the game and over time started to counter your play-style, head-shot kills over time will get the guards wearing helmets, if you do everything at night the guards start equipping NVGs, sneaking leads to them setting mines around the perimeter of the outposts, they get better gear and armor over time but you could also counter this, it's not a mission you play but send your combat team to sabotage it can work but to make it effective, you need to let the player counter it in someway, give a variety of ways to complete the goal, level design that both the AI and player can use to their advantage (The FEAR AI is over-hyped in reality the level designer and the AI coordinator held hands all the way through development which results in the AI using the level environment to it's advantage and communicated information among themselves to function as proper squads)

 

[wtfNeedSignUp]

Though about it some more, I'll go full autistic on this idea and what's the potential problem with it:
First of all the question of what kind of learning we use here, you can have raw input (basically a map with the units on screen), or have someone make that input into some hyper parameters (like have enemy clustering or number of small formations as a dedicated number).
Considering you want to have a player-dedicated AI (rather than one made from aggregated multiple users), learning raw input is completely infeasable unless you want the player to play thousands of games (since more free elements -> higher amount of things to optimize). And that's without going into the fact that we still struggle to make a car AI that doesn't randomly run over people. Things like formation is far harder to put into programming than the average person thinks.

So with this hyper parameters you now need a way to optimize them, which means a value function of win/tie/lose. Now the vast majority of players will not want to play a game they lose too much, so you can't start the game in a position the pc can curbstomp the player. So the best way to optimize it is by some heuristic like the amount of damage inflicted to the player. There are ton of heuristics and you can fuck yourself hard if you do it wrong. For example, if the game has a single point the AI needs to destroy to win the game, but the heuristic is amount of damage to building, then you'd have an AI that ignores the target goal and waste time destroying barricades (since more damage -> higher score). This also has another problem which I'll go into it later.

So a potential game can have multiple waves of zombies attacking the player, each wave with its own AI. This will give you some amount of sample to optimize on (which would take a millisecond between rounds), but here's another problem, if one wave wrecked the player's shit and the next one beats what's left of his forcees, then you'll have the latter wave be more important than the former, which is incorrect and will lead to wrong optimization. You also want the game to still be fun and functional, so you need to watch out of cases where the AI basically fucks off and the player needs to wait for something to happen.

Finally there's a user problem, which is summarized with this clip:

The player can basically cheese the entire process and have the shittiest AI be considered as the best one by the heuristic and completely break the game.

 

[THEY STOLE MY FORESKIN]

The Amiibo fighters you could scan in Smash Bros 4 were marketed as being able to adapt to your play style and get better at fighting each time they leveled up, they could up to level 50. I remember back then some people doing tests to see how they would learn, if you used the shield too much they would try to grab you and if you made a custom stage with only a small passage through which they could knock you out they would try to aim you in that direction when going for a kill. I don't remember hearing too much about it after that, maybe because it wasn't that useful, I'm really not sure, and I think Ultimate doesn't have this feature.

It would be very cool if a game decided to train its AI based on the gameplay of the top players on the leaderboards for racing games and FPSes, the training could be done server-side and the AI could be changed through updates periodically, for example every month. I'm uncertain how feasible this is, but it would be really cool and I'm sure the technology is there for it.

 

[Ughubughughughughughghlug]

This was what Hello Neighbor promised to be (machine learning stealth game) and then it turned out to be a stinky turd (Matpat bait puzzle-platformer game).

 

[Involuntary Celebrity]

I've thought of this before, but there are a few problems.
1. Deep learning is CPU/Memory expensive (i think?) not very ideal to run whilst the game loop is running. Maybe iterate whilst in menus but then you won't get much evolution between.

If what you're testing against is the player themselves then you can't be pushing a shitload of iterations a second anyway. It's gotta be a live system in real-time, so what's being described here is really more of a life simulation than ML (although you can certainly do those with a genetic algorithm-style evolution of little neural networks for each agent for example, which works great in real-time from what I've seen).

It would be ideal for a zombie game. There's no risk of the system becoming superhuman or anything and ruining the fun because (a) the player and zombies have different affordances: the fact that the player is faster, sneakier and/or can use guns and shit is already the reason such games are playable despite the zombies having looming advantages. That asymmetry is what drives the game in the first place, it's not like you and the AI are both playing chess.
And (b), there's a lot of inertia in those simulations; the player will be able to notice and react to trends before they're entirely optimised. In fact depending how exactly it works, players could play gardener and possibly manipulate the shit out of it. How you account for that on a design level would be getting down into the weeds of a specific game, though.

 

[Hijo de Dang Woodchucks!]

That's the problem with this idea: making a perfect play AI isn't that hard for many games, and after a certain level the AI is effectively cheating through optimized, instantaneous decision making that a human can never do- functionally identical to reading the controller. Deep Learning or whatever just adds extra steps to that process.

This, I fucking hate it when games start fucking with you because it reads your inputs (Soul Calibur 4, most King Of Fighters, Elden fucking Ring) or in strategy games starts throwing buckets of enemies with top of the line equipment and stats that are fucking impossible (Play a Fire Emblem game on the upper difficulties and pay very close attention to your enemies. Tell me how long it takes you to realize RNG on its own will not save you.)

 

[Gender: Xenomorph]

3. Updates, if you were to add new systems/items/traps etc, the AI would need to adapt to these making the dataset previously gathered either broken or shit.
Sure I had more points but can't recall them.

Or you could do like 90s games where AI ignores some mechanics.

 

[std::string]

This, I fucking hate it when games start fucking with you because it reads your inputs (Soul Calibur 4, most King Of Fighters, Elden fucking Ring)

Button reading was a lazy way to increase difficulty. Samurai Shodown 4's final bosses were the most hilarious examples of this 'cause you couldn't even hit them on the hardest difficulty.

 

[Smaug's Smokey Hole 2]

Using machine learning in games would be really cool if it wasn't just used to try to smash the player. That's bad game design. The player is playing the game and a good game will play with the player. An ML AI/opponent should maybe work by settings up goals that are unknown to the player and then try to achieve them. In the zombie horde scenario the AI might set up the goal to drive the player up on top of the roof of the shack they're defending, or push them into the nearby caves, or the shoreline etc. Not to defeat the player, just to play with them. It will try to make this happen using the finite units/zombies it has at its disposal.

The player doesn't know what the game is up to, the player might defeat the AI before this happens, but it will appear as if each round/scenario is different.

 

[Involuntary Celebrity]

Elden Ring doesn't "read your inputs" and people sound retarded bitching about it. Enemy AI is based on entity states (spoiler, enemies in games don't have fucking cameras for eyes), they just tuned the reaction times really aggressively low in order to punish certain habits they didn't want players bringing over from earlier in the series. They could have done this more believably with a few more milliseconds of delay, or raycast occlusion checks for state information along the decision tree etc, but having good-feeling reactive enemies in games is always about limiting the omniscient information available to the system.

Learning enemies'd have to take that to an extreme implicitly because you want the input they use to be extremely specific and situational anyway.

Or you could do like 90s games where AI ignores some mechanics.

Depending how they learn they might be really agnostic to new mechanics anyway. Like in some defence-type games where enemies learn, it's basically just a (possibly decaying) heatmap of where critters die so they'll try to send new waves on paths that avoid your killboxes, which has the effect of them testing your perimeter for weaknesses.
In that case it doesn't matter what kills them. They don't care if it's a new feature, just whether it has a more or less effective on the heatmap. And you can do basically the same with reward maps if that's a thing.

Generally you want the learning part of your system to be as simple as possible so this principle would probably apply to many other kinds of games; like in a Souls example again you have humanoid enemies fundamentally using the same rig and a lot of the same affordances as the player, so they get access to most of what the player does implicitly and improving their tactical behaviour with ML/whatever might just require trivial logic to incorporate common resource/projectile/hitbox data.

Like I said earlier, the "learning dataset" is kind of a non-factor if what you're training against is the player in realtime. But for general competitive symmetrical strategic AIs (like in Chess or Starcraft), they use self-adversarial training with often very little base knowledge. So maybe if you want some pre-training like the OP suggested, you can basically add whatever you want and it's just affecting time for continued or re-training (on the dev side, pre-deployment); not such a big deal if the idea is to give them a fundamental baseline and not optimise the shit out of their strategy beforehand.

 

[Gender: Xenomorph]

I mean this all could work if AI is stored on a server and the info is accessed when playing online.

Think playing a RTS game, but instead of playing against human opponents, you play against a machine learning AI.

That way you don't have to worry about new players playing against potato AI and having to play to 'build up' their AI, and also AI thinking 10 iterations at a time will not have to drawn out the user's computer player.

 

[Accidental Racism]

It's a good idea, which means that unfortunately, it will probably never be done.

 

[Involuntary Celebrity]

It's a good idea, which means that unfortunately, it will probably never be done.

It can legit be way easier from a dev perspective than compelling decision-based AI so I guarantee that it will. I can already think of some games/mods using simpler evolutionary approaches.

What I'm really excited about though are language model-based characters.
Remember how old RPGs/adventure games let you ask characters about anything, but it was really cumbersome and pointless because they'd each only respond to a handful of prompts from maybe a hundred possible topics?
People are already playing around with using waifuGPTs trained on game world topics for this, but there's research on extending language agents to bound actions. For example, an AI that plays old console games by treating controller inputs as verbs so that their understanding of a particular game on a particular platform gains a grammatical relationship which can inform their uptake of a new game. That might sound weird but I think I remember reading somewhere that our language centres are evolved from motor neurons, so maybe it's actually logical and it's an interesting approach to robots being able to link speech to their physical capabilities.

Characters in games are basically robots without any of the overhead or limitations of physical robots, so we might see applications there even sooner. NPCs that can actually talk, react to your words/actions, and take their own actions in response.

 

[Illuminati Order Official]

Echo is based on a very simplified version of this idea: your enemies begin as very simple and unskilled crash test dummies and then "learn" from you - if you use guns, they learn to use guns, if you run a lot they become faster, if you use magic they copy your magic and so on.

 

[John Titor]

I vaguely remember a video where they showed the AI in TES: Oblivion to be too 'smart'. Someone casted a sleep spell on a dog and it caused a chain reaction where the entire town killed each other. I imagine something like that happening.