Post checked array to complex viewmodel

I have some trouble getting the checked values in my form model to be posted correctly.

I have this Product class

public class Product
{
    public int Id { get; set; }
    public string Name { get; set; }
    public byte[] Photo { get; set; }
    public int InStock { get; set; }
    public decimal Price { get; set; }
    public decimal Discount { get; set; }
    public decimal TotalPrice => Price - Discount;
    public int SupplierId { get; set; }
    public string SupplierName { get; set; }
    public IEnumerable<Category> Categories { get; set; }//Contains only the categories the product is bound to
}

And this Category class

public class Category
{
    public int Id { get; set; }
    public string Name { get; set; }
    public string Description { get; set; }
}

And this EditVM class

public class EditVM
{
    public Product Product { get; set; }
    public IEnumerable<Category> Categories {get; set;}//Contains all the categories in the db
}

In my ProductController I have the Edit action method.

public async Task<IActionResult> Edit(int id)
{
    var product = await _productRepo.Get(id);
    if (product == null) return RedirectToAction(nameof(Index));
    var categories = await _categoryRepo.Get();
    var editVM = new EditVM()
    {
        Product = product,
        Categories = categories
    };
    return View(editVM);
}

This is the HTML in my view where I loop through the categories

@{var categoryIdList = @Model.Product.Categories.Select(x => x.Id); //Contains only the categories the Product is bound to (And gets the id's)
int counter = 0;}
@foreach (Category category in Model.Categories)//Loop through all the categories in the db
{
    if (categoryIdList.Contains(category.Id))
    {
        <input type="checkbox" name="Product.Categories[@counter].Id" value="@category.Id" checked />
    }
    else
    {
        <input type="checkbox" name="Product.Categories[@counter].Id" value="@category.Id" />
    }
    <input type="hidden" name="Product.Categories[@counter].Id" value="@category.Id" />
    <label>@category.Name</label>
    { counter++;}
}

So far, so good, everything works till here. If I have 5 categories in the db and the product is bound to 2 of them, 5 checkboxes are shown and 2 are checked. So this is perfect.

However, the problem is when I select or deselect checkboxes. Now every Categories.Id in the db (Checked or unchecked) is posted to the HttpPost Edit action method and I want only the Ids posted whom are selected.

[HttpPost]
public async Task<IActionResult> Edit(Product Product)
{
    var EditedProduct = await _productRepo.Update(Product);
    return RedirectToAction(nameof(Index));
}

I could chose to make an EditPostVM but this has nót my preference.