I think the black man was somewhat prejudiced, and of course, against the white woman/man. (The second person is mentioned as a man in a version and a woman in another version of the poem.)

His prejudice is regarding the white people. They are on the opposite end of that of the white people and their enmity is almost universal. But it is very unlikely that a black person would hate a white for his/her skin colour, as white is universally seen as superior. So, his main prejudice is that the white people always hate them for their skin colour and so they need to take revenge. It can actually be no prejudice at all as this is true in many cases, and earlier in the poem we see the white woman show her hatred against the black man.

Therefore, it seems, his prejudice is only partial. He was rather more concerned with revenge than with his prejudice against the white woman. As the white woman rejected to share her log as it would help a black man, it was an insult to him. That is why he wanted to take revenge against the white woman for all the insults and hatred their race had suffered in the hands of the white people at large by not sharing his log of wood. He took it as a rare chance to irritate the white.