When removing silences, use the [separate] option instead of [delete]. This will move clips detected as silences to another track.
You can then select the clips you want to remove and use the alt+del (Windows) or opt+del (Mac) shortcut to delete them and ripple the timeline.