Sound travels through all sectors unless they are closed, like a door. To prevent this you edit a line and check the "Block Sound" flag. So to block sound from alerting all monsters you have to set this flag on all lines between 2 solid lines. You usually do this at narrow parts and by doors.
The tricky part is though is that you need 2 layers of sound blocking lines since sound travel through the first blocker and stops and the second one it encounters. This is because that's how you segment your monster encounters.
There's a button on the left of the editor that's called "Sound Propagation Mode". Click it and then point at a sector and the editor shows which sectors monsters will be alerted if you make a sound at that point.