Squirrels and chipmunks just harbour the micro-organisms that cause plague, hantavirus, leptospirosis, salmonella, Lyme disease, tularaemia, and (possibly) rabies.
To answer agony's question, I think rats and mice are just as cute - I used to have them as pets.