If we want to marry narrative play and competitive play we need to resolve this conflict. We need narrative play to be easy. Anyone should be able to pick up a game and effortlessly create a story together through play.
People can, they're just lazy and want GW to do it for them. But then they look at something like crusade, and whine that it's too much bookkeeping. Ok, fine play the missions in white dwarf and shit, no they won't play that either. Make up their own missions? Nope. GW used to do narrative series of books, people bitched endlessly about it(because of the rules and adding to the list of shitty books to need to buy) so GW quit doing that. These people don't know wtf they want.
Likewise competitive play needs to be more than numbers on paper deciding whether the roll of a dice is successful or not.
I'm not sure what it would be at that point. We're talking about a game revolving around resolving conflict with a diceroll. The dice roll is going to be a determining factor.
Sometimes competitive needs to be ok with things being unbalanced.
Absofuckinglutely not. The entire reason why competitive 40k has taken off while casual players who rarely play demanding a narrative(but can't shit or get off the pot), is due to having balanced map layouts where there's an opportunity for combined arms, shooting, or melee armies to work. That updates happen so there isn't just a hope that you bought into the right set of lucky models to have a broken mechanic for the next 3-6 years, and so on.
The flavor should come from the rules. Unfortunately 10th edition has standardized so many rules that armies aren't able to shine. They are so afraid to make anything exceptional that they also ground away any edge. There are a few minor instances where I can see the rules reflecting the lore. Space Wolves have Krom Dragongaze that makes someone visible and in range take a battle shock test. I know it's also not unique but I can see what this rule is trying to represent. The character glares at someone and his stare alone can cause them to shake in fear. It's simple and adds flavor to a character. But what flavor is Space Marine Lieutenants give lethal hits? He gives lethal hits because that's what the rules say. This rule could and does go on absolutely anything and we've just arbitrarily decided that's what the Lieutenant do.
Sure, but what did we have previously to this? every army having one of 16 variations of deep strike? How many different ways do we need to list that something gets an extra hit on a 6? Let's look at necron tesla weapons.
10th: Sustained hits 2
9th: Each time an attack is made with this weapon, an unmodified hit roll of 6 scores 2 additional hits.
Now you could argue that you would need to know what sustained hits means. Sure. But once you know that, you don't need a full sentence on every weapon profile in the game that has a variation of it being 1, 2, or 3. Here's another example, power fists (captain in terminator armor)
10th: WS 2
9th: When resolving an attack made with this weapon, subtract 1 from the hit roll.
I'd rather just see that the thing hits on a 3 or whatever it's supposed to be rather than a sentence of text to tell me that it's different than the model's normal WS. That isn't "flavor".
You mentioned lieutenants giving lethal hits. Ok, that alone you're right it isn't any kind of flavor. But what else do they do?
10th: gives lethals to the unit they're attached to, and "
Target Priority: This model’s unit is eligible to shoot and
declare a charge in a turn in which it Fell Back."
9th: can't even lead a unit. "
Angels of DeathCompany Heroes
: If your army is
Battle-forged, then for each LIEUTENANT unit included in a
Detachment, a second LIEUTENANT unit can be included in that Detachment without taking up an additional
Battlefield Role slot." Whoopdeedoo, there's no flavor in that either. "Tactical Precision (Aura)
: While a friendly <CHAPTER> CORE unit is within 6" of this model, each time a model in that unit makes an attack, re-roll a wound roll of 1." That's just a reroll wounds of 1 aura, so not even reroll all wounds.
8th: "Tactical Precision
: Re-roll wound rolls of 1 for attacks made by models in friendly
<CHAPTER> units whilst their unit is within 6" of this model." Ok, so that's all they get unless you buy them a jump pack, because the other rule is just that they don't need to be in coherency with anything because they still can't lead a unit. "Company Heroes
: During deployment, every model in this unit must be set up at the same time, though they do not need to be set up in unit coherency. From that point onwards, each model is treated as a separate unit."
And before that, they weren't a thing. But my point still stands. There was nothing flavorful with the rules for lieutenants any moreso than what they currently do in 10th. If anything I'd argue that having the ability to screw up your positioning and letting your lieutenant get sniped or taken out with precision in melee, removing the ability for the unit to fall back and charge actually deals more of a blow to the unit's effectiveness narratively by losing it's officer, than having a lieutenant get taken off the battlefield in previous editions where even an aura could affect multiple units in range.