“Or so the story’s told” means the poet is unsure about the authenticity or accuracy of the incident he is narrating. It is a story which floats on the oral level and he has only heard it. Though the above quoted line follows the line “Each one possessed a stick of wood”, I think, it essentially is not concerned with that mere information of each person possessing a log. Rather on a general level, it indicates to the lack of authenticity of the story.