The Terriers missed out at the semi-final stage last time around and Peter Jackson's side, who were hit by injuries and illness, suggested that the hangover is continuing.
Cheltenham, meanwhile, a team who nobody gave much of a chance to back in the summer, are playing with their heads held high and currently occupy the most lofty league position in their entire history.
The Robins won even though they were made to sweat after Terriers substitute Pawel Abbott sprung the offside trap and raced onto a pass from Gary Taylor-Fletcher before rounding goalkeeper Shane Higgs four minutes from time.
And Huddersfield midfielder Mark Hudson had a great chance to level the game in injury time but screwed his shot wide from eight yards out with only Higgs to beat.
Earlier, Huddersfield showed some promise going forward in the opening stages with Taylor-Fletcher collecting a pass from Nathan Clarke and then seeing a dangerous cross-cum-shot intercepted by diving goalkeeper Higgs at the feet of Andy Booth.
Moments later, a quick breakaway from defence ended with Taylor-Fletcher drawing a fine save from Higgs, who dived full-length to his right to block the shot.
However, Cheltenham began the game at a quick tempo with their forwards and midfield players combining well across the pitch.
With 10 minutes gone, a move of several passes ended with Steve Guinan shuffling a pass from Brian Wilson into the path of Kayode Odejayi. The big striker knocked a ball between the central defenders for Grant McCann to run through and score with a firm drive from 10 yards.
Cheltenham went on to have the best of it until half-time, although Huddersfield's Luke Beckett had one sharp turn and shot held by Higgs.
McCann then went close with a long-range shot and, as the half drew to a close, the travelling fans voiced their displeasure by chanting "sort it out" at manager Jackson.
The visitors did raise their game for the second half following the introduction of Danny Schofield on the left as Taylor-Fletcher was moved out to the right.
Higgs had to make another fine save from Beckett but it was Cheltenham that increased their lead with 71 minutes played after some fine persistence from former Huddersfield player Craig Armstrong.
The left-sided player cut infield and looked to have lost the ball on at least one occasion but kept going before passing to JJ Melligan on the right-hand corner of the penalty area. The Irishman fired a low, angled drive in off the far post to set Cheltenham up for three points, albeit with that piece of late fortune.