Is it the color of rust or actually yellow? If it's the former, it's most likely rail dust. Almost inevitable if the vehicle is shipped by rail.

As LeMans stated, try a detailing clay bar.