Republic of Ireland Under-21 midfielder Collins struck his seventh goal of the campaign to make him Town's second highest scorer.
Collins' goal condemned Posh to only their third defeat in 24 matches, but the visitors dominated the second half without making the breakthrough.
Collins, out of the side under previous boss Stan Ternent, is blossoming under new manager Lee Clark and the goals keep coming.
The midfielder kept his cool to slot home the only goal after a superb defence-splitting pass from Danny Cadamarteri after 35 minutes.
Huddersfield, with new signings Lionel Ainsworth and Anthony Pilkington starting on the wings, had the better chances in the first half.
Phil Jevons curled a free-kick wide of the post and Ainsworth showed a fantastic touch to control a difficult ball but then missed the target with his shot.
A Robbie Williams piledriver went high over the bar and Posh's best chance fell to Chris Whelpdale, but young keeper Alex Smithies proved equal to it.
Peterborough boss Darren Ferguson was far from pleased with his side's first-half showing and sent his players out early for the second half.
He pushed Aaron McLean further forward with George Boyd and Craig Mackail-Smith just behind and Posh had more than enough chances to equalise.
Sub Shaun Batt screwed wide when through on goal and Mackail-Smith was also clear but Smithies stood up well to block.
Smithies again denied Peterborough with an acrobatic tip-over from McLean before Charlie Lee was wayward with a free header from a corner.
Huddersfield survived a late scare when 18-year-old Smithies, a reputed target for Premier League clubs, blotted his copybook by fumbling Boyd's high ball.
McLean reacted first but stabbed wide and Huddersfield held out.