According to the US Food and Drug Administration, refrigerated foods, including milk, should never be out of the fridge at room temperature for longer than two hours.
If cold milk is warmed but untouched, it can be returned to the fridge for a later feeding. Some mothers keep the leftover milk at room temperature to use within an hour if the baby appears hungry after a short sleep. Others refrigerate and reheat the milk left from a previous feeding.