Going inside to take shelter from the rain is indeed a good idea.
It's better than being exposed to the wind and rain outside; getting the cotton-padded jacket wet would be too uncomfortable. Besides, this is due to unexpected weather changes, an objective reason. Even if the village chief asks later, they would have a reasonable answer.
The villagers under the willow tree agreed with this suggestion, saying, "Alright then, while the rain hasn't gotten heavy, let's hurry inside."
He was truly fed up and did not want to face the cold wind and rain anymore.
The little house would be warmer after all, and they could even light a stove to warm their hands. The brothers could comfortably sleep through the night. Although it's not exactly cozy or pleasant, getting through the night is good enough.
The villagers have all faced hardships; they don't have grander expectations and only wish to stay away from the wind and rain, completing their night of tomb-guarding duty.
