We need to know a bit more about you, your cancer, and treatment thus far. It sounds like you might be on some form of ADT. If not, and your prostate has been removed and you are otherwise cancer-free, are you taking any additional "new" medicines? You may have an allergy to that medication.
If all of the above is not applicable, then I suggest going to your physician for a routine CBC, CRP, Comprehensive Metabolic Panel ("Chem panel"), and a Urinalysis that is collected sterile so it can be cultured in case there are bacteria present in your urine = UTI = fever). The CBC will show whether you have a high or low White Blood Cell Count (WBC), and what the WBC "Differential" reveals. The Differential enumerates the different types of White Blood Cells, and would include the presence of any immature cells, which could be indicative of a far more serious concern. I won't go there yet...no need to worry prematurely. The CRP (C-Reactive Protein) is a non-specific indicator of an immune/inflammatory response "somewhere" in your body. Also...
Have you been on a vacation lately to central California or the mid-West? There are certain febrile diseases that are endemic to those areas (carried/contacted by dust) that you can contract and take home with you. If you were a mountainous area, did you drink any river, creek, or lake water that you did not boil or otherwise treat/disinfect? Do your symptoms include stomach ache-like distress? Do you have diarrhea? You can pickup a common parasite (Giardia) from such water sources. If you have Giardia or any parasite, your CBC Differential would reveal a higher than normal percentage of Eosinophils which are cells that respond to parasitic infections and any allergic/immune problems.
Please follow up and let me know. Most of all...let your doctor know, and ask for the tests that I mention above. I was a former Director of Clinical and Anatomical Pathology services (a.k.a. "the Lab") in hospitals, so I dealt with this on a daily basis.