"Colored" says one of two things to me: early to mid 20th century terminology that was pc back then, or an official race designation in South Africa before apartheid was abolished. I'd never think to use "colored" in reference to a person's skin. (Clothes, sure :-)

(Someone actually called Pata colored?! To his face?! Some people are really stupid.)