Need help removing Android ListView items using CheckBoxes

I have a list of tasks that I am displaying inside of a ListView. Inside of each ListView item is a CheckBox with the task title and a TextView displaying the due date. What I am trying having trouble with is removing the item from the list when the Checkbox inside the ListView item is checked.

What I have at the moment is an OnCheckChangedListener that is set for the CheckBox, inside of the Adapter for the listview. This is set in the getView() method. When the Checkbox is clicked the task is removed from the ArrayList, and a new CustomAdapter is made and applied to the listview using the new taskList. This did not work so I added a Handler to make these changes on the UI thread. Still doesn't work. From the logs I can tell that the listener is getting called but the items are not removed.

ArrayList<Task> taskList;
Context mContext;
ListView listView;
Handler handler;

public CustomAdapter(Context mContext, ArrayList<Task> taskList){
    this.mContext = mContext;
    this.taskList = taskList;
    this.handler = new Handler(mContext.getMainLooper());
}

@Override
public int getCount() {
    return taskList.size();
}

@Override
public Object getItem(int position) {
    return taskList.get(position);
}

@Override
public long getItemId(int position) {
    return position;
}

@Override
public View getView(int position, View convertView, ViewGroup parent) {
    if(convertView == null){
        convertView = LayoutInflater.from(mContext).inflate(R.layout.listview_item,parent,false);
    }

    final View row = convertView;

    CheckBox checkBox = (CheckBox) convertView.findViewById(R.id.checkBoxTask);
    TextView textTaskDueDate = (TextView)convertView.findViewById(R.id.textTaskDueDate);

    checkBox.setText(taskList.get(position).getTask());
    textTaskDueDate.setText("Due Date: " + taskList.get(position).getDueDate());

    checkBox.setTag(position);

    listView = parent.findViewById(R.id.listViewTasks);

    checkBox.setOnCheckedChangeListener(new CompoundButton.OnCheckedChangeListener() {
        @Override
        public void onCheckedChanged(CompoundButton buttonView, boolean isChecked) {
            if(isChecked){
                Log.i("CheckBox Listener", "Check heard");
                handler.post(new Runnable() {
                    @Override
                    public void run() {
                        taskList.remove(buttonView.getTag());
                        listView.setAdapter(new CustomAdapter(mContext, taskList));
                    }
                });
            }
        }
    });

    return convertView;
}