Q: Brown Sludge in Radiator Reservoir and Very little coolant

brown sludge in radiator reservoir. I have a 2002 passat 1.8t. Nothing is mixed in with the oil. My father flushed the coolant system, first thing this morning and I drove my other car to work. We let the car run for a few minutes and the radiator reservoir was clear. I came back after work ( approx 8 hours: don't know if it matters), started the car and the check coolant light came on immediately. Checked the reservoir and there is was again. Coolant overflowed everywhere the first time the brown sludge happened and got on a lot of stuff, so I couldn't tell if the white smoke was coming from my tailpipe or if it was burning off because things were hot. No sign of oil leakage either. I did refill coolant the first time the coolant came on but I did not take in consideration the type of coolant and mixed coolants. Also my dad drained the radiator and poured water in the reservoir to flush. I figured you would have to let the water run through the system to completely get all the sludge

My car has 169000 miles.
My car has an automatic transmission.