For most of the players it would be the hitters, no question. However if you were a 'character' player, or HW , or goon you'd be more on edge because of the fighters on the other team. Just like Schultz mentioned in his book ..looking ahead in the schedule to when the Flyers played the Bruins - when and how he'd fight O'Reilly, Cashman, etc. It kept him up nights..not when is will
Ken Hodge, or
Don Marcotte coming to town but the fighters. Of course Schultz is in the minority , when your job is to intimidate and fight -the gm is a little different...a bad or embarrassing loss could cause you to lose your job.
So for 99% of players .......the hitters are more fearful.