These secrets cannot be kept anyway, so there is no point in you doing so. I have a cousin who is furious with me because when her daughter and grandsons wanted to know how we were related (we were matched by our DNA across two different platforms), I told the truth.
Your family is hiding what they need to hide, the racists are hating who they need to hate, so go ahead, and tell the truth you need to tell.