Who said the infertility concerns only women? The man may well have the problem, and this has been caused by specific causes. A man can biologically have the ability to have children even in older age, but that does not mean that the quality and strength of his sperm remain unchanged after the age of 40.
Smoking and alcohol definitely affect the male’s sperm to some extent. And it is not at all certain that it will make any difference if you limit the period when your partner attempts to conceive.
Two other factors responsible for male infertility are the use of prohibited substances (drugs, anabolics, etc.) and obesity.
Any serious problem that has occurred in the genitals of the man may be responsible for his subsequent infertility.