I don't think you need to worry about this. Colds are caught through close contact like being sneezed or coughed on directly, or touching dirty tissues then touching nose or eyes. One or two random virus particles in the air won't infect a normal person, even an older man. As with most airborne viruses, infection comes from exposure to a quantity of virus particles over time. For example, sitting next to someone sick for an hour is FAR more risky than walking past someone on the sidewalk who coughs or sneezes as the pass by (and not right in your face.)
When you are outdoors, the enormous volume of air dilutes the particles and carries them away. Please set your mind at ease, walking outdoors with a cold, or opening your windows will not infect your neighbors.