Even if you remove it all you are likely to find it growing in a jar of cured buds, it's spread by mold spores and if you disturb it, it sprays mold spores all over everything in the room. I would chop the affected stuff off and make concentrate from the rest.
Then I would thoroughly clean the area with a fungicide to prevent further infections. Good airflow is key.