I'm not suggesting anything remotely resembling 'sterile' living. I'm telling you primarily that if you allow contact between your pets, your food or your food prep/storage surfaces, you are likely eating their crap.
Fecal matter in particular is absolutely notorious for causing minor food poisoning. Cats in particular track it everywhere they go. They track it far away from their litter boxes. They do so everyday without exception. Dogs transfer fecal matter on occasion.
Eating crap will make you sick. Even trace amounts will likely trigger a reaction. Trace amounts are not necessarily visible. This is why we wash our hands after wiping our butts. This is why I suggest you dry your hands only on clean towels. It's not 'sterile living'. It's basic hygiene. The same level of care should be taken to avoid eating your pet's crap.