Unfortunately, very few countries have any idea how many cases there's really are.
I think you're right. How many cases are out there where the person never really got sick, and nobody knows they had it (not just in the US.) I think there is probably a pretty good amount of people out there in the world like that, and it would mean it isn't as bad as its being made out to be.
I'm not saying it's not a concern, but what happened to the bird flu, pig flu, H1N1, SARS, etc? There were what, 60 million cases of H1N1 in the US, and 25k people died? I don't remember fights over toilet paper during that? Schools closing? The general stupidity?
Oh well. Here we are.