First time in more than a century that a government has won a seat from the opposition at a by-election.