The Poppies took the lead after just seven minutes as Marcus Kelly prodded home from close range to hand the Heed an uphill task.
Toby Lees’ first Gateshead goal hauled the visitors level midway through the second half, before Joshua Kayode secured a huge three points for the Heed late on.
Mike Williamson opted for three changes from Tuesday’s 1-1 draw against Bradford (Park Avenue) as Michael Nelson, Elliot Forbes and Jordan Preston came into the starting 11 at Latimer Park.
Kettering went ahead through the first real chance of the afternoon as Kelly latched onto a cross towards the far post and poked the ball past the on-rushing Brad James to hand the hosts the lead.
JJ O’Donnell came close to restoring parity almost instantly as he met Preston’s cross at the near post, sending the ball just wide of Paul White’s goal.
The Heed’s early opportunities fell O’Donnell’s way and it was the midfielder who threatened to grab an equaliser on the half-hour mark, shooting on the turn from 20 yards only to see the ball whistle just wide.
James was called into action twice to keep Kettering’s advantage at just the one, producing an impressive stop to keep out Michael Richens’ close-range header before making another stop moments later to halt Kelly.
The second half began in relatively uneventful fashion as Gateshead looked for a way back into the game, and after 65 minutes they did just that via Lees.
Scott Barrow’s free-kick from the centre of the pitch found the centre-back’s run at the back post, and he made no mistake from close range to side-foot home and draw the Heed level.
The visitors wrestled control of the game from Kettering having drawn level, enjoying more possession before striking the post with just over 10 minutes to go as Greg Olley turned and struck the woodwork from 18 yards.
The game’s defining moment arrived one minute into injury time, as Kayode prodded home a wicked cross from Nicky Deverdics to send the Heed Army into raptures and secure the first three points of the season.
Kettering Town XI: White; Stohrer, Skarz, Graham, Cartwright, Richens (c), Meikle (Solkhon (85’), Kennedy, Nti, Hodge (Tchuimen-Nimely, 72’), Kelly.
Unused subs: O’Connor, Carta, Brighton
Goals: Kelly (7’)
Bookings: Nti (65’), Kennedy (83’)
Gateshead XI: James; Nicholson, Lees, Nelson, Barrow (c); Agnew, Forbes (Oliver, 69’); Kayode, Olley, O’Donnell (Blackett, 83’); Preston (Deverdics, 50’).
Unused subs: Guthrie, Tear.
Goals: Lees (65’), Kayode (90+1’)
Bookings: Agnew (49’), Nicholson (59’), Olley (90+2')
Referee: Ben Atkinson
MOTM: Nicky Deverdics