My husband is hardly ever interested in sex.

I have been married 5 years and my husband has lost interest in sex for the past 3 years. He is a good person and very caring. I am otherwise compatible with him.Should I ignore this problem and focus on the nicer aspects of our marriage? I am worried because I am in my late 30s and he is in his early 40s and I want a family.