Jesus will judge the living and the dead?

I have always found this a bit confusing. 

2 Timothy 4:1(NKJV)

I charge you therefore before God and the Lord Jesus Christ, who will judge the living and the dead at His appearing and His kingdom: 

Here is where it confuses me.  On the one hand we are taught that when we die, if we have accepted Jesus, we go to Heaven.  But the wording above makes it seem like when we die we all wait in the ground for Jesus to return and only then do we go to Heaven. 

So which is it?  And more importantly does it really matter?  For me at least, as long as I get there eventually that is all that matters! 

And based on the totality of what I have read in Scripture I still think we go right to Heaven. 

We shall see!  :)