Allow checking to a checkbox even after adding attr("checked", false)

I am not sure if a formulated the title correctly but the logic is this.

I have 4 checkboxes, 3 Yes and 1 No. Selecting 3 Yes will uncheck No and selecting No will uncheck Yes.

My jQuery as per below:

var yes = $("input[type=checkbox].yes");

if (yes.length == yes.filter(":checked").length) {
    $("#no").attr("checked", false);
} else if($("#no").is(":checked")) {
    yes.attr("checked", false);
} else {
    // do nothing
}

It works. However, if I checked all Yes, I won't be able to select No. Instead, I need to uncheck 1 Yes to select No. Same thing for No. If I check No, all Yes will be unchecked. However I need to uncheck No to be able to select Yes again.

Will there be a way that I can still select No if all Yes is checked?

Thanks!