MouseDown/Up events not woking properly in WPF

I have a custom control based on ToggleButton. In its constructor, I create a few elements of type FrameworkElementFactory and to one of them I assign an EventHandler like this:

button.AddHandler(Ellipse.MouseEnterEvent, new MouseEventHandler(Toggle));

And here's my event handler:

void Toggle(object sender, EventArgs e)
    {
        DependencyObject active = GetTemplateChild("Active");
        DependencyObject button = GetTemplateChild("ToggleButton");
        Storyboard sb = new Storyboard();

        ThicknessAnimation ta = new ThicknessAnimation();
        Storyboard.SetTargetProperty(ta, new PropertyPath("Margin"));
        Storyboard.SetTarget(ta, button);
        ta.Duration = TimeSpan.FromSeconds(0.2);


        DoubleAnimation da = new DoubleAnimation();
        Storyboard.SetTargetProperty(da, new PropertyPath("Width"));
        Storyboard.SetTarget(da, active);
        da.Duration = TimeSpan.FromSeconds(0.2);

        if ((bool)IsChecked)
        {
            ta.To = new Thickness(0);
            da.To = (double)button.GetValue(Ellipse.WidthProperty);
            IsChecked = !IsChecked;
        }
        else
        {
            ta.To = new Thickness(this.Width - (double)button.GetValue(Ellipse.WidthProperty), 0, 0, 0);
            da.To = this.Width;
            IsChecked = !IsChecked;
        }

        sb.Children.Add(ta);
        sb.Children.Add(da);
        sb.Begin();
    }

But it doesn't work. Doesn't work in the sense that it works only once. I mean it is supposed to set a slider's width based on the IsChecked property. Initially, it is set to false. When I click it, it expands to the right as expected. But when I click again, it does not move left as expected.

The more surprising thing is that it works perfectly with the MouseEnter event. All other events like MouseDown/Up/LeftMouseButtonUp/PreviewMouseDown..... are not working either.

If you want to reproduce the problem without much editing use the code here. Then just use it in a WPF app and you'll see.

Any help much appreciated.