Here is an excerpt from a review article in The Open Neurology Journal published in 2017: “[Musical hallucinations] are prevalent above all in patients with neurological pathologies, whether these be cerebral lesions (vascular cerebral diseases, tumors, demyelinating disease...), epilepsy of the temporal lobe, Parkinson's, infectious processes of the central nervous system, etc. […] They can be produced as a side effect of some medications (pentoxifylline, tramadol, Bromocriptina) and have also been observed in hepatic transplants.”
Reference: https://doi.org/10.2174%2F1874205X01711010011

Covid was obviously not around when this article came out, but it does sound like the musical hallucinations you are experiencing are likely neurological in nature, and Covid definitely can wreak havoc on the nervous system (i.e., the loss of taste or smell).

I think letting your doctor know about your symptoms is appropriate, and hopefully they will be able to refer you to a neurologist who can help you. Best of luck with your recovery!