OBAMA said it best on 60 min last Sunday when asked about the same topic:

"There will those who will not vote for because I'm black, but most likely they will even out by those who will vote for me because I'm black.” It shouldn't be an issue!