Feeling cold can vary on our perception, age, race and hormonal concentrations.
This has been shown in Gend Med study from 2007:
More women than men had sensitivity to cold. Whereas the percentage of men who had sensitivity to cold significantly increased with aging (P < 0.05), the percentage of women who had sensitivity to cold was already high (23.7%) at 50 to 60 years of age and did not change with aging.
Therefore based on the numbers we can say that sensitivity increases with aging and it is different in men and women. And this was not not associated with circulating hormonal concentrations.
These studies were tested only on Japanese men and postmenopausal women aged >=50 years, so if it's the evidence based on the numbers, it's up to you.