Yes, it is unfair that you have osteo with such an active lifestyle and plenty of calcium. I am sure your doctor is measuring how much calcium you ingest is absorbed (a blood or urine test). I know my specialist likes her patients to get their calcium from food.

I haven't fractured yet. All I can add is that biking isn't weight bearing and neither of your current favorite sports are a good idea. I walk fast (not stroll) and lift 8 lb weights, the latter amount on my doctor's advice. Be sure and engage in balance exercises and keep flexible, but avoid twisty exercises (or strong twists).

Pill medication isn't bad. The risk of side effects are there, but what isn't often shared is their frequency. Plus, you have to weigh it all against the worse case scenario of a fall on your rear that nails your spine -- the risk is being bedridden and in chronic pain. I wouldn't dare risk a fall and neither should you.