Idle words

I find it interesting that Jesus specified "idle" words.  We will have to account for our idle words.  Why do we have to account for idle words but not others? 

Matthew 12:36-37(NKJV)

But I say to you that for every idle word men may speak, they will give account of it in the day of judgment. 

For by your words you will be justified, and by your words you will be condemned.”

My guess is that idle words is a way of saying that we will have to account for the time that we chose not to serve God.  Just a guess.