I doubt Boris will be in charge for much longer, I just think it was a 'warning ' and normal business will be resumed when General Election time comes.
I believe it will send out a couple of messages that should be concerning to Johnson and the Conservatives.
1) This was a huge swing against the Tories which may not have been Tory voters staying at home but Tory voters voting for someone else.
2) The Tories tried the usual "he's a remainer" schtick and it didn't work. Brexit, with many voters, has been and gone. A key tactic used to divide and conquer the opposition now looks like it might be ineffective.
The incumbent party losing by elections is not uncommon. But the level of swing has to be of concern.
I think Johnson is on his last chance with the backbenchers. It is incredible that despite an 80 seat majority Johnson is having to act as if he has a few seat majority. He has no discipline from the rank-and-file. He had to rely on the opposition to get critical public health legislation through the Commons during a pandemic FFS. He jettisoned the entire moderate wing of the party in 2019 and by doing so has given the rag-bag of hard right wingers power. The nuttier end of the conservative party always existed, but before Brexit they were generally ignored. They have been taking over the Tory party and Johnson turbo-charged the process.