Despite getting himself into trouble by sometimes speaking off the cuff, Newt has a very conservative record

From the American Conservative Union, Newt has a lifetime average of 90. The Republican average at that time was 85.
From the NTU 1979-98, Gingrich's score averaged 61% (100% is a perfect score on supporting lower taxes and limited government). The average Republican scored 56%.

From the Club for Growth:

Voted YES on the Reagan tax cut of 1981
Voted YES on the Reagan tax reform bill of 1986
Voted NO on the George H.W. Bush “Read My Lips” tax hike in 1990.
Voted NO on the Clinton tax hike in 1993.
Voted YES on the capital gains tax cut in 1997.

Other positions:

an immediate and permanent repeal of the Death Tax;
elimination of all capital gains taxes;
reduction of the corporate tax rate to 12.5 percent;
a 50 percent payroll tax cut for both employers and employees;
a 100 percent tax write off for businesses’ equipment purchases.