As soon as a fish dies, their body starts to break down and the meat becomes bad. You NEVER want to leave fish out at room temperature because you run the risk of contracting a food borne illness. Leave the fish in refrigeration until you are ready to prepare and eat it.
Fresh fish just a few hours and for frozen you don't really want to leave it out for more than half an hour, just to thaw it and have it not be hard as a rock. Best to thaw in the frig.