Checkbox state for single item only inside recyclerview

I have recyclerview containing items with checkboxes. RV list is ArrayList of custom objects. Each of this object has his state inside as variable and I change this state according to checkbox state.

I want this specific behaviour --> Only one item can be checked at the same time. By default, first item is set checked so i set check state of my checkbox according to item state inside onBindViewHolder. Another important rule (and this is where is the issue) is, that one item has to be checked all the time. There cant be all items unchecked. So I made setOnCheckedChangeListener to solve this.

If item is set checked, you will set check variable inside this item to true, then you have to iterate the list and uncheck every item in a list except actual item.

Then if you want to uncheck item, state is changed to false, then you have to iterate over the list again and check if some of the item is checked. If this is true, you can safely uncheck the item and refresh data. If its false, you cant uncheck this item, because all of the items would be unchecked, so you will set check state back and refresh data.

Everytime you refresh data it will copy state inside item to the checkbox state.

Issue: The issue is, that if I click on checked item once, it works --> item remains checked, but as I click on that item again, it will uncheck and idk why.

Code sample of my listener:

//uncheck all variants if this is checked - one variant has to be checked everytime
                holder.foodVariantCheckBox.setOnCheckedChangeListener { buttonView, isChecked ->
                    if (isChecked){
                        for (variantPos in 0 until foodVariants.size){
                            if (variantID != foodVariants[variantPos].foodVariant.getInt(ctx.getString(R.string.food_variant_id))){
                                foodVariants[variantPos].isChecked = false
                            } else {
                                it.isChecked = true
                            }
                        }
                        notifyDataSetChanged()

                    } else {
                        it.isChecked = false
                        var isSomeVariantChecked = false
                        for (variantPos in 0 until foodVariants.size){
                            isSomeVariantChecked = foodVariants[variantPos].isChecked
                            if (isSomeVariantChecked){
                                break
                            }
                        }
                        if (!isSomeVariantChecked){
                            it.isChecked = true
                        }
                        notifyDataSetChanged()
                    }
                }