Georgia put in the performance of their lives, scoring after 93 seconds through Kvaratskhelia before Georges Mikautadze’s second-half penalty gave them room to breathe.
England have topped Group C at Euro 2024 and will play Slovakia, who finished third in Group E, in the last 16 on Sunday in Gelsenkirchen (17:00 BST).
Gareth Southgate’s side have a path through the knockouts which avoids many of the major teams.
They had looked on course to play the Netherlands in the last 16, but Georgia gained a shock 2-0 win over Portugal to leave Slovakia as England’s next opponents.
Spain, Germany, Portugal, France and Belgium are all on the other side of the draw and England cannot play any of them before the final.
The Three Lions failed to beat Slovenia on Tuesday, but won the group despite their 0-0 draw as Serbia and Denmark also played out a goalless stalemate.
They have topped the group, but now need to win three knockout games – the last 16, quarter-final and semi-final to try to earn a place in the final in Berlin on 14 July.