Remove Default Checked Option on Radio Button when Another Option is Selected

If i'm using radio buttons in HTML and would like a default option to show as selected, I've used the 'checked' attribute to achieves this. How do I make it so when I check on another option on the radio buttons this default option is removed. In the code below, when you check another option, the original option remains and you can't uncheck anything?

In the option below the fish option is the one with the 'checked' attribute added.

http://ift.tt/2yUtqGg

HTML

<input type="radio" id="dog"name="dog"value="dog"><label for="dog">Dog</label>
<input type="radio"  id="cat" name="cat" value="cat"><label for="cat">Cat</label>
<input type="radio"  id="fish" name="fish" value="fish" checked><label for="fish">Fish</label>
<input type="submit">