resolve checking one box and unchecking the other

I am trying to have my two checkboxes have separate values, when one is checked the other is unchecked. I am able to accomplish this using:

 final CheckBox checkYes = (CheckBox) findViewById(R.id.visual_yes);
        final CheckBox checkNo = (CheckBox) findViewById(R.id.visual_no);

        CompoundButton.OnCheckedChangeListener checker = new CompoundButton.OnCheckedChangeListener() {

            @Override
            public void onCheckedChanged(CompoundButton buttonView, boolean isChecked) {
                if (checkYes.isChecked() ) {

                    checkNo.setChecked(false);
//                    finish();
                }
                else if (checkNo.isChecked()) {
                    checkYes.setChecked(false);

//                    finish();
                }
                savePrefs("checkYesVisual", checkYes.isChecked());
                savePrefs("checkNoVisual",checkNo.isChecked());
            }
        };
        checkYes.setOnCheckedChangeListener(checker);
        checkNo.setOnCheckedChangeListener(checker);
    }

but my problem is that the first button that I press is the only one that works how it should. For example, if I press yes first and then press no after it doesn't switch. But if I press no and then yes the check switches (as I want it to).

I believe my problem is with setting the checkNo textbox to false when yes is clicked and that value just saving until the yes box is unchecked.

Any suggestions?