A series of stress relaxation experiments were conducted on glassy PETG at various temperatures and held strains. In this work, a new fractional model of stress relaxation based on the Scott-Blair’s model was develop. The model can capture the relaxation behavior of PETG at both large and small strains as well as with varying temperatures. Furthermore, the discussion of fractional order shows the ability of order to capture the mechanical property evolution of polymers in the whole scope of the experiment, which is consistent with the observation from experimental results. The method of least squares was employed to determine the model parameters by fitting the fractional constitutive model to the experimental data. It can be found that the yield and approach glass transition temperature show similar effect to the stress relaxation behavior, and the fractional order is a key parameter in determining the stress of entire stress relaxation process. Then, the parameter sensitivity analysis result explains mathematically why the changes of order have close relation to stress relaxation behavior.