Carbon Monoxide.....safe if gas is switched off?

Alarm was going off, had all windows and doors open for about an hour and gas man been to check on it but is unsure if it is or not and has left the gas switched off and said it should be safe? does switching the gas off and leaving it off stop further carbon monoxide been kicked out of the boiler?