Former cabinet minister criticized for attending climate change skepticism conference: ‘Sparking at least one call for his resignation’