How can I be closer to my hamster?

I've had my Chinese dwarf hamster for about a week now. I adopted him from another family. He is about 5 months old. I realized that he doesn't respond well with me. I'm not sure if he misses his old but he's always running away and hiding in his cage. I try to pick him up to hold him but he moves so fast. He likes to bury himself in his bedding and spends a great majority of time in his tunnels. I talk to him every time I pass his cage (which is a lot of times), hoping he would get used to me. But all he does is hide. Is this just his personality? Are Chinese hamsters just naturally timid? Is there a way that I become closer to him and possibly be able to hold him without him being scared?