The AI thread

[belphegor]

Regarding the effectiveness of the B-29 defensive guns, does anyone know how many MiG-15s were shot down by B-29s in the Korean War?  As far as I can tell, in the April 12, 1951 battle, no MiG-15s were lost, or at least not to the B-29s' guns.  The B-29 had a pretty advanced remote control and lead computing gun system, but it was designed with prop plane speeds and ranges in mind; it wouldn't surprise me if it just wasn't very effective once jets came onto the scene.

[KillerBee]

I offer this link concerning B-29s in Korea: https://nuclearcompanion.com/data/b-29-superfortress-chronology-korean-war/#june-1950

This might provide some insight.  There are many more Korean War resources out there for sure that may confirm or deny.
 

[Edit] If you’re a mission builder, I suggest you explore the compass direction those B-29s are flying to and from targets on and the effective combat ranges of the various intercept aircraft involved before committing to a mission design.  Remember, there are no B-29s in Korea![/Edit]

Edited June 21 by KillerBee

[Juri_JS]

5 hours ago, belphegor said:

Regarding the effectiveness of the B-29 defensive guns, does anyone know how many MiG-15s were shot down by B-29s in the Korean War?  As far as I can tell, in the April 12, 1951 battle, no MiG-15s were lost, or at least not to the B-29s' guns.  The B-29 had a pretty advanced remote control and lead computing gun system, but it was designed with prop plane speeds and ranges in mind; it wouldn't surprise me if it just wasn't very effective once jets came onto the scene.

Maybe we shouldn't overestimate the B-29s guns effectiveness, especially when not flown in tight formations. Even the North Korean 56th Fighter Regiment with their Yak-9s managed to shoot down a B-29 and damaged several others. Apparently the unit suffered few losses in these missions.

[gecko]

I think for the dev teams in the past, the answer is not how easy or difficult the AI was to improve, but what importance they attached to it, which was apparently not very much. Pretty much all combat flight sim development for the past two decades has heavily leaned into the multiplayer side of things at the expense of making a single player experience that was actually good.

BSR seems to not to value AI development as much either since it isn't as good as a human. To each his own, I do understand the draw of multiplayer. But let's not pretend this is impossible to improve, or hide from the widely recognized success smaller developers have had in this area just because you watched one video (in which the AI was in fact doing something a lot better than in other sims) and for some reason decided that's all there is to see.

It's also worth pointing out, that the intelligence of a human opponent is not going to be matched in flight sim AI in the foreseeable future, and that is an attractive challenge for multiplayer. On the other hand, a guy who's been playing flight sims in his gym shorts for 20 years with the benefit of 80 years of historical hindsight, a comfy chair, and a bag of chips, but who has never flown a real airplane of any kind, and mainly wants to increase his stats, is not going to act the same as an actual pilot in a real cockpit who *is* 20 years, and knows a rash decision has a good chance of resulting in him dying a horrible death within the next five minutes. 

Getting modern players to discipline themselves en mass to fly from a more historical perspective, and operate in some semblance of following procedures used at the time in order to recreate the experience of an actual mission is going to be vastly more difficult than getting the AI to do those things. Kudos to a few groups who have made impressive efforts, but even they still rely on AI aircraft to fill in the gaps, and most of them still lack the training, discipline, and incentives of a real pilot. The shift towards multiplayer has made those of us who preferred the big picture mission experience be dependent on whether a bunch of other people decide to fly together in a way they have neither the training, discipline, nor motivation to do. I'm not saying that's even the point of multiplayer or that it should be, but we have lost something we used to have. That's why some of us prefer single-player, and are frustrated that 20 years later, sims are not doing any better at this, and are arguably a fair bit worse, because it hasn't been something developers have cared about in any meaningful way.

A lot of what's been shared indicates the developers now are trying to correct some of that imbalance with Korea. We're now at the point where we find out if they made meaningful progress and if they have the commitment to iron out the issues that will inevitably pop up and make it work really well. Gritting my teeth and hoping for the best.

Edited June 21 by gecko
Typo

[BraveSirRobin]

3 hours ago, AEthelraedUnraed said:

Easy: spaghetti code, technical debt, feature creep and a mismatch between the initial design choice and the desires of the customers. Not saying that all of these are necessarily the case here (except for the last one, which was confirmed by the devs in one of their blogs), but all of these are easily solved by starting from scratch.

That’s not “details”, that’s a bunch of generic buzzwords.  And “desires of customers” has absolutely nothing to do with whether more human spotting/awareness is now easier.

 

3 hours ago, AEthelraedUnraed said:

For my part, I'd love to hear from you why a believable spotting/situational AI is an extremely difficult problem, beyond your personal opinions and your earlier non-sequitur.

Obviously the AI actually knows where everything is even if the AI pilot in a particular plane can not see something.   How do you decide if an aircraft in a shadow over a forest is visible?  It might be visible against open sky, but what about forest?  Or a city?  Or water?  Does it change with range?  How much?  Does a spotted aircraft become invisible if it flies over water?  Over a city?  If it’s suddenly not visible, how long does the pilot keep looking for it?  Does that change with range?  How much?  Does the field of view for spotting new targets become smaller if the pilot is already “tunnel locked” on an existing target?  Do AI even get tunnel locked?  How often do you refresh which aircraft can be seen?  Should certain circumstances change that time?  I could go on and on for the rest of the morning.  And most importantly, how do you process all those things for every aircraft without slowing the game to a crawl.

I’m also a little surprised to see a programmer who thinks AI programming might be easy.  During my 28 years of programming I’ve never talked to a person who thought AI programming was easy under any circumstances.

[Avimimus]

3 hours ago, AEthelraedUnraed said:

For my part, I'd love to hear from you why a believable spotting/situational AI is an extremely difficult problem, beyond your personal opinions and your earlier non-sequitur.

@AEthelraedUnraed - The spotting AI problem is probably much more solvable than other AI issues - so you are right about that. Which is one reason why I highlight it! One basically needs to calculate a field of view, taking into account obstructions from the cockpit, and have the crew pan back-and-forth and checking if an object falls within that box, and is within range (and possibly applying a probability for the pilot failing to notice it or what would be called academically 'inattentional blindness'. I believe DCS released a white paper on their solution. 

AI is an extremely difficult problem overall though. Just take a look at something like pathfinding issues in games (including ones where the AI is a person - represented as a rectangle that is about 20 cm deep and 40 cm wide, with linear acceleration, on a flat plane)... and look at all of the pathfinding failures in a lot of different games, as well as the different algorithms used over the years.

Now consider that the programming has to be very efficient. One could have dozens of aircraft updating their decisions almost constantly. To make a human level decision the pilot has to consider where the plane will be at a future point in time... because some manoeuvres lose energy and some maintain or gain energy the actual possible future states for a single time increment would look like a horizontal fountain plume, with shallower turns on ones which are level or downward leading to a further distance for that time increment. Now, to make a good decision one has to do this for not just your own airplane, but for all of the relevant opponent aircraft (and every possible choice they could make), as well as chain-together multiple future manoeuvres (with both pilots being able to switch to interrupt their current manoeuvre at any time to start a new one - which could then influence when the opponent switches its maneouvres)... now to be realistic, this also needs to be done with a delay to represent the pilot needing to assess/perceive the opponent's manoeuvres... and also the pilots need to plan ahead if they are to strategise the best response... imagining multiple possible responses and counter-responses... if you are good, you are doing all of this, all of the time - but programming this into a simulation in a way which is computationally efficient is nearly impossible.

 

 

[Avimimus]

2 minutes ago, BraveSirRobin said:

Obviously the AI actually knows where everything is even if the AI pilot in a particular plane can not see something.   How do you decide if an aircraft in a shadow over a forest is visible?  It might be visible against open sky, but what about forest?  Or a city?  Or water?  Does it change with range?  How much?  Does a spotted aircraft become invisible if it flies over water?  Over a city?  If it’s suddenly not visible, how long does the pilot keep looking for it?  Does that change with range?  How much?  Does the field of view for spotting new targets become smaller if the pilot is already “tunnel locked” on an existing target?  Do AI even get tunnel locked?  How often do you refresh which aircraft can be seen?  Should certain circumstances change that time?  I could go on and on for the rest of the morning.  And most importantly, how do you process all those things for every aircraft without slowing the game to a crawl.

I’m also a little surprised to see a programmer who thinks AI programming might be easy.  During my 28 years of programming I’ve never talked to a person who thought AI programming was easy under any circumstances.

You make some good points here - particularly regarding lighting and contrast (which are real issues discussed a lot in historical accounts regarding air combat). One might also add in settings for the mission builder to simulate pilot fatigue and whether the crew is alert (e.g. in the combat area) or relaxed (e.g. over allied territory).

I think some of these could be added - but I'm pretty sure that simulating at least a probability of not being spotted at all (e.g. while diving on a formation) or being spotted early could be an improvement.

Hmm... I might even add that one could probably get by without fully simulating the panning of the field of view:

Just having a set number of second until the next scan of the sky (perhaps with some randomisation to make it less predictable), and a probability of detection per scan (inversely correlated to the distance to the enemy...), and perhaps another modifier that can be controlled by an MCU so the mission builder can adjust alertness...

P.S: Simulating the field of view and obstructions might be good though - as it would also cause the AI to periodically temporarily lose sight of the player and have to re-acquire the player during the dogfight - which would make things more realistic.

[BraveSirRobin]

27 minutes ago, Avimimus said:

P.S: Simulating the field of view and obstructions might be good though - as it would also cause the AI to periodically temporarily lose sight of the player and have to re-acquire the player during the dogfight - which would make things more realistic.

Leave out too many of the things I listed (and dozens of things I didn’t list) and people will notice.    And complain.  Millions of years of evolution have created some complex problems for combat sim developers.

[Aapje]

2 hours ago, Avimimus said:

One basically needs to calculate a field of view, taking into account obstructions from the cockpit, and have the crew pan back-and-forth and checking if an object falls within that box, and is within range (and possibly applying a probability for the pilot failing to notice it or what would be called academically 'inattentional blindness'. I believe DCS released a white paper on their solution. 

Doing it properly would require taking more into account like the contrast against the background, lighting (spotting into the sun should be very hard, but lighting can also impact observability ion other situations), relative movement (a plane flying sideways relative to the observer would be easier to spot than one flying straight towards or away from the observer). Also, range should not be a binary where a plane suddenly becomes visible, but it should impact the chance of observing it.

As for inattentional blindness, one can expect a real pilot to be better at spotting in the combat zone, and better when having already having had contact recently.

Then to mimic humans well, losing visibility in some way should result in a guesstimation of the position of the enemy and the observing plane should act on that guess, not the real position of the enemy. And then there should also be a chance-based calculation on reacquiring the enemy, which again should depend on the factors mentioning in my first paragraph.

Edited June 21 by Aapje

[Avimimus]

Yes, very much @Aapje! I see you've also got the dragonfly hunting technique down around relative motion!

A few of these things could be approximated through (a sufficient model rather than a perfect model...) with the use of a few probabilities in a fairly simple equation.

I have developed the suspicion that a lot of the reason why people complain about difficulty spotting targets in the sim isn't because spotting is too hard for the player, but rather it is because spotting is too easy for the AI.

On the old forum there was that thread discussing research from WWII showing that something like 80% of fighters that were shot down were unaware that they were even under attack (until it was too late)... so dogfights might only be happening for 20% of kills, with the rest happening in a single pass.

It'd be pretty need to see probabilities for AI responses reflecting this (with some engagements involving being spotted early and the entire formation reacting as information is spread through radio) and other engagements allowing ambushing entire formations.

[BlitzPig_EL]

6 minutes ago, Avimimus said:

I have developed the suspicion that a lot of the reason why people complain about difficulty spotting targets in the sim isn't because spotting is too hard for the player, but rather it is because spotting is too easy for the AI.

In my personal case it's both of the reasons you cite.  It's obvious to anyone that has played the sim for any length of time that the AI verge on the omniscient, and have shooting skill all out of proportion to human players.  Then there is my personal struggle with spotting.  If enhanced visibility is not enabled I am pretty much unable to spot aircraft, especially when viewed from above.  I have been watching the early access videos and am perplexed at how players (Krupinski) are spotting aircraft like they are painted hi viz red.  Kudos to them, but I just can't see these aircraft.

[Avimimus]

My frustration is often difficulty spotting road vehicles. Which might be an issue with contrast or motion, or a lack of glinting off windscreens - or it could be just the fact that there was often a lot more road traffic historically to find (so missing the occasional truck wouldn't mean missing all the trucks). I also suspect that wingmen might help with spotting ground targets.

Anyway, one thing improved in Korea seems to be that there are more ground vehicles in a convoy (of course, there should also be pack animals if we wanted to get the 1950s DPRK accurate).

[Dash,Polder]

I hope Laubox had shadows on low in that cockpit demonstrator, well worth the watch for me, haven't seen much of internals for the Yak or IL-10, devastatingly claustrophobic on some of those rides.

My only wish for the AI is they use some teamwork and provide each other with mutual support, that would move the goal posts more than anything.  You pick on one and you get a gang fight coming your direction. 

[KillerBee]

32 minutes ago, Dash,Polder said:

My only wish for the AI is they use some teamwork and provide each other with mutual support, that would move the goal posts more than anything.  You pick on one and you get a gang fight coming your direction. 

At this point, I expect my wingmen to do as they’re told, but only so far as keeping themselves alive and not doing stupid stuff both they and I know will get them killed!  Dying on one’s sword is not approved!

[KC23]

Thanks for the videos that included lots of CAS missions. Hate to say it, but the AI dogfighting looks kinda dull to me. Very uninspired dogfighting AI if you ask me. I saw one video where he set the bad guys to Ace and went 4 Vs 4. That was decent with bad guys getting on his six if he was target fixated. Other than that. Boring.

[Avimimus]

27 minutes ago, KC23 said:

Hate to say it, but the AI dogfighting looks kinda dull to me. Very uninspired dogfighting AI if you ask me. I saw one video where he set the bad guys to Ace and went 4 Vs 4. That was decent with bad guys getting on his six if he was target fixated. Other than that. Boring.

One thing that is worth keeping in mind is that combat in the early jet era looked quite different from combat in WWII.

From https://lend-lease.net/articles-en/interview-with-porfiriy-borisovich-ovsyannikov-part-2/:

"Was the F-84 Thunderjet capable of flying with the MiG-15 in a maneuver fight?

I did not dogfight with them. They came in, and I attacked them when they were bunched up. Like us during WW II, they were escorting their bombers. We had two groups: one group of immediate escort, and another so-called maneuver group, which engaged in combat. They did the same thing, used the same tactics. Those that were flying immediate escort were tied to the bombers; they were required to protect them. They only fended off attacks. And those that were covering this group could freely engage in combat.

In your opinion, what was the better airplane — the MiG or the Saber?

For me it was the MiG.

In what way was it superior to the Saber?

I don’t know. But I fought with them. And I got out of difficult situations. Does this mean anything? Either I was better, or the other pilot was not as good, or…

Well, how did you get out of it? Did you always use the same methods, or various methods?

I got out, as a rule, by climbing. We did not engage in pure turns. They taught us that in the regiment. In the first place, it’s difficult to hit someone while turning because it’s hard to stay on his tail. The bullets fly behind him."

 

So most historically accurate Korean War flights they'd mostly be:

'Mig-15Bis dives on the target and fires if it manages to get an aim' followed by 'Mig-15Bis climbs away from the target'.

What'd keep it exciting is there would sometimes be a second Mig-15Bis lining up to dive on you next.

The Migs were in a relatively target rich environment, being used primarily as interceptors in hit and fade attacks... and they were often outnumbered but had an advantage at high altitudes and in climb rate (but in few other areas)... so it is somewhat unsurprising that dogfights and complex air combat maneouvres were rare.

Even if a dogfight happens, the turn rates of the aircraft... the wing loadings and impacts of gee loading at the relatively higher speeds, the limited acceleration of jet engines... all of this spreads out the air combat and makes it less cinematic (or cinematic in a different sense perhaps). If you want really want superbly elegant short range fights - you need to look before 1942 (and maybe as far back as WWI).

[kodamcity]

7 hours ago, Avimimus said:

I got out, as a rule, by climbing. We did not engage in pure turns. They taught us that in the regiment. In the first place, it’s difficult to hit someone while turning because it’s hard to stay on his tail. The bullets fly behind him."

Well, the AI enemy aircraft appearing in the current Korea video all seem to insist on turning in only one direction. In that case, it looks like Korea AI is completely wrongly made right now. I really hope it is improved in the official version released in August.

[Reisen]

Enigma's review had him shooting down a Sabre from an IL-10.

 

Come on, guys. This AI needs to be fixed  

[Aapje]

@kodamcity This is not going to change significantly. 1CGS has already told us that they can't find an AI expert and told us what parts they did work on (comms, landings, taxxing). Better accept that this is what it is.

[kodamcity]

@AapjeI think you are right. It seems the development team lacks the ability to improve the AI. With only two months left until the official release, I don't think the situation can improve dramatically.

If I hadn't liked this game in the first place, I wouldn't have written such excessive complaints about the AI, nor would I have uploaded RoF and GB gameplay videos to YouTube for over a decade, nor would I have written such a vast guide to help South Korean users get started.

https://namu.wiki/w/IL-2 Sturmovik: Great Battles

Do you see this link?

Although I received help from other users, I wrote almost every section myself. That is how much affection I have for this series.

South Korean users who watch the YouTube videos I uploaded sometimes ask, "Is this game worth playing?" Then, here is what I can say: The AI level is so low that you cannot properly enjoy it in single-player mode. Also, since there are many users in the US time zone on the multiplayer servers, it is difficult to enjoy multiplayer in South Korean time zones. The points I mentioned are things I heard from users to whom I recommended the game. So, to ensure that new players wouldn't be disappointed, I relayed their stories exactly as they were.

I have been repeating this for over a decade. And when IL-2 Korea was announced, I thought I could finally recommend the game to people after hearing that the Career Mode would be enhanced. I expected that if the Career Mode—a single-player element—advanced, the AI level would naturally improve as well. However, I was extremely disappointed by the level of AI shown in the Early Access videos.

That is why I wrote the post somewhat sensitively regarding the AI issues. I apologize if you guys felt uncomfortable.

[Avimimus]

20 minutes ago, kodamcity said:

@AapjeI think you are right. It seems the development team lacks the ability to improve the AI. With only two months left until the official release, I don't think the situation can improve dramatically.

They do tend to support the product post-release, so it is possible that AI work will continue for sometime.

 

6 hours ago, kodamcity said:

Well, the AI enemy aircraft appearing in the current Korea video all seem to insist on turning in only one direction. In that case, it looks like Korea AI is completely wrongly made right now. I really hope it is improved in the official version released in August.

This is inaccurate. The AI can evade in different directions and has at least nine evasive maneouvres (based on the wingman commands menu shown in some of the videos).

P.S. There was an issue in Great Battles which goes back to Rise of Flight, where it is possible for AI to try to turn after running out of energy and altitude (at which point it could get locked into a right-hand turn)... but that is a different issue which can only really be fixed by keeping the AI from trying to do minimum radius turn fights at low altitude. But that isn't really even an evasive manoeuvre - just a loss in flexibility that happens once you've exhausted your kinetic energy.

[kodamcity]

5 minutes ago, Avimimus said:

This is inaccurate. The AI can evade in different directions and has at least nine evasive maneouvres (based on the wingman commands menu shown in some of the videos).

P.S. There was an issue in Great Battles which goes back to Rise of Flight, where it is possible for AI to try to turn after running out of energy and altitude (at which point it could get locked into a right-hand turn)... but that is a different issue which can only really be fixed by keeping the AI from trying to do minimum radius turn fights at low altitude. But that isn't really even an evasive manoeuvre - just a loss in flexibility that happens once you've exhausted your kinetic energy.

If, in Korea, the AI exhibits behavior of attempting to turn in only one direction like in previous games when it loses energy and altitude...

The same behavior will likely occur after performing the 9 types of evasive maneuvers, so those must be the videos I saw. I don't mean to be sarcastic, but honestly, it doesn't look like there is much difference. 

The MiG-15 AI pilots will not be able to shake off the F-86 like Borisovich in the interview. Because eventually, behavior that turns in only one direction will appear. 

[Aapje]

2 hours ago, kodamcity said:

Then, here is what I can say: The AI level is so low that you cannot properly enjoy it in single-player mode.

That is subjective. Plenty of people seem to consider the AI to be good enough to have fun.

2 hours ago, kodamcity said:

The MiG-15 AI pilots will not be able to shake off the F-86 like Borisovich in the interview. Because eventually, behavior that turns in only one direction will appear. 

I imagine that there would be a lot of complaints if the AI would fight like that. I also wonder if Borisovich actually knew more than the one trick that they taught him, since he doesn't even seem to understand leading your shots.

[Avimimus]

2 hours ago, kodamcity said:

If, in Korea, the AI exhibits behavior of attempting to turn in only one direction like in previous games when it loses energy and altitude...

The same behavior will likely occur after performing the 9 types of evasive maneuvers, so those must be the videos I saw. I don't mean to be sarcastic, but honestly, it doesn't look like there is much difference. 

The MiG-15 AI pilots will not be able to shake off the F-86 like Borisovich in the interview. Because eventually, behavior that turns in only one direction will appear. 

Yes, that analysis is quite logical - definitely a possible issue which a simulation would need to solve.

P.S. This has also been an issue in Rise of Flight/Flying Circus and even FS-WWI back in the day - it becomes very pronounced in earlier eras where the power-to-drag ratios are even lower.

[kodamcity]

17 minutes ago, Aapje said:

That is subjective. Plenty of people seem to consider the AI to be good enough to have fun.

That is correct. I saw people here as well who had no particular complaints about AI.

On the other hand, I would like to let you know that there are quite a few users who did not enjoy the game due to AI issues, as I have heard from users to whom I recommended RoF and GB that the AI is terrible.

 

17 minutes ago, Aapje said:

I imagine that there would be a lot of complaints if the AI would fight like that. I also wonder if Borisovich actually knew more than the one trick that they taught him, since he doesn't even seem to understand leading your shots.

 

I agree. For users who want to achieve a lot of shootdowns, an AI that acts intelligently will not be popular. Only users who value historical accuracy will likely respond positively. In my opinion, simply slightly modifying the AI's behavior of trying to turn in only one direction would be positive for most users.