the best thing to do with tap water is to fill up some jugs and let it sit for at least 24 hours, doing this allows the chlorine to evap making it safer and better for the plants, i leave my water standing till the next time i water, so i fill them up and let them sit tll i water then fill them up and let them sit till i next water, this also raises the water temp to room temp meaning it wont shock the plants, id try and do what ilove420mag said and dont water for a few days keep checking the wight of the pot and water when it feels light, i good thing to do would be to fill up the same size pot with compost and then compare the dry weight with no plant to the pots with the plants in then you will know when it needs watering, when the top 2 inches of soil are bone fry then thats the best time to water, the roots need oxygen as well as water so keeping the roots wet stops the plant taking up the nutes and causes root rot, so id fill up some jugs or a bucket with water and let it sit till the next time you water the plants, this will make the water safer for the plants and should balance out the ph in the tap water, the water company puts chemicals in the water to make it safe to drink so letting it sit for a couple of days lets the chlorine and anything else to evap, but then this water has been sitting and has lost its oxygen content so either place a air pump stone in the bucket to replace the oxygen or pour the water into a pop bottle and give it a good shake to introduce oxygen back into the water.
so dont flush again leave it for a couple of days and see what happens, plants grown in soil are less affected by nute problems compared to hydro so you should be ok, just give the plants and roots time to recover, so let the pots dry out and the plants will benefit, the roots need oxygen as well so having the roots wet all the time dont help the plant,
just keep an eye on it, you will know if your plants are thirst as the leaves will droop and look limp so if you notice them signs then water till you have a small amount of run off and stop watering, also with the nutes add them every 2 or 3 times you water the plants and not every time you water, using nutes that much causes salts to stop the roots taking any nutes and this affects the plants, someone else may be able to cover this a bit better as i am no expert and dont have the experience, most of my info is stuff ive learned over my time on this site, hope its been some help to you