Seed productions takes weeks from pollination to fully formed seed. While there may be some minor localized impact from a few pollinations, it shouldn't be enough to have any significant impact on the potency or production of your lady. I'd keep after the nanners and remove any that you find to minimize any pollination, but you should not see any seeds as a result of these nanners this late into flowering.