The Millers are now 12 games without a win after twice falling behind to goals from Luke Beckett and the winner from Michael Collins.
Chris O'Grady scored from the penalty spot to level the scores in the first half after Andy Holdsworth had been sent off for handball and Delroy Facey headed a late consolation.
Martin Woods tested Terriers goalkeeper Matt Glennon early on with a rasping 20-yard drive, but it was the West Yorkshire side who swept into the lead on three minutes.
Chris Brandon robbed Craig Fleming of possession on the edge of the area and his cut-back was rifled home by Beckett.
The visitors, hoping to reduce a ten-point gap to the top six, were slow to build on the advantage and had to wait another 20 minutes before Danny Schofield warmed Gary Montgomery's hands with a long-range shot.
O'Grady brought a good save from Glennon moments later, but on 31 minutes he equalised for the Millers from the penalty spot after Holdsworth handled Ian Sharps' goal-bound effort. Holdsworth was sent off for the offence.
Buoyed by this Rotherham took control and Ian Henderson brought a good save from Glennon with an audacious 30-yard lob. Moments earlier Rotherham thought they had a second penalty when Henderson appeared to be bundled over by David Mirfin, but the referee waved play on.
Huddersfield were desperate for the half-time whistle and after getting his team back in the dressing room Terriers' boss Peter Jackson was rewarded with a second goal just 50 seconds into the second half.
Brandon skipped past Stephen Brogan and crossed for Beckett to head home at the far post.
The ten men put the game beyond doubt on 58 minutes when Collins' long-range effort beat the outstretched arm of Montgomery and crept into the corner.
Rotherham substitute Facey pulled a goal back within minutes of coming onto the field on 70 minutes with a downward header from Henderson's cross.
Seconds later he almost equalised with an audacious lob. Henderson brought a smart save from Glennon on 82 minutes, but with just three minutes left he was sent off for a second bookable offence for a lunge on Schofield.