That feels like exactly what happens to me! I went to an ENT at Mass General Hospital last week and they had no idea what was happening, which was disappointing. They advised avoiding my triggers which for me is yawning while talking or with my head turned to the side. This has happened to me once or twice a year since adolescence even though I of course try very hard to avoid the triggers.
Incidentally, I have only managed to "pop it back into place" once (quite accidentally). Good for you for being brave enough to push it back in. I hope I can get comfortable with that as well.