First off... I think your expectations are too high. Just because someone leaves their phone off or doesn't contact you does not mean anything. Unless this has been going on for some time... (A week or two) It does sound like he is genuinely interested in you, otherwise he would not be talking about you. I would be cautious though, usually when things don't work out the first time they don't work out at all. Just go with the flow, and don't fret over anything. Just because he is not talking to you every day does not mean he is not interested.
Ask him how he feels, have him tell you... Then bring up your concerns. Just be open about everything, that's the best approach to take.
Guy's do tend to forget things, or he may have gotten busy... Wait it out, and good luck.