Why does he bother, if we never go anywhere?

Maybe he doesn't know how to, or he is waiting for you to make the next move. Maybe he thinks since you pull away you are not that interested. Do you send him clear signals you like him? Maybe he is not sure how you feel about him.