Taking Loved Ones from Memory Facility for a Holiday

I have moved from physical caregiver to an overseer, an advocate, a visitor now that my husband is in a Memory Care Facility. Each day feels like a learning curve. The book 36 Hour Day has a chapter about taking residents out, but it is clear, that every resident is different. My daughter and I took my husband to a nearby doctor's appointment. It was the first time out of the building in 3 weeks. Returning to the facility spun him around, and he had great difficulty reorienting himself again after acclimating very well initially. I would welcome comments on how to make this transition easier on the family and the loved one. Holidays are coming up and bringing my husband out of the facility to home with many family members whom he no longer recognizes may be too much. I'm looking for advice from people who have done this. My daughter asked," Are we bringing him over for a holiday celebration for him or for ourselves?" I'm very concerned that it would cause more harm than good.