Can this be considered racist...?

You are visiting a church and all members of the congregation are of a different race than you. You ume before you even get there that you'll be looked down upon, just because you are 'different'. Is that racist thinking, on your part?