I had a hemorrhagic stroke caused by a (currently removed) meningioma. And six months after the surgery and stroke, I started having seizures. I later learned that epilepsy can be a side effect of having a stroke. And epilepsy doesn’t necessarily develop immediately; it can take months before the first seizure. So my (survivor, but non-expert) opinion is to continue with the neurologist who would be better at treating epilepsy for maybe a year and if everything goes well, then consider switching.