Most people measure goodness and badness by how they feel about it, and for anyone who feels a significant amount of badness (grief, anger, whatever) at the death of a single person, it is physiologically impossible for them to feel a million times worse about the death of a million people.
It's intuitively obvious to most people, therefore, that suffering, annoyance, life-saving, and so on, are not additive: they just have to check how they feel about the situation to know that.
In order to suggest that they are additive, and that N "people annoyed" can outweigh M "people suffering", you have to first convince someone that how their own internal measurement of goodness (how they feel about it) is not as accurate as some external measurement.
