After losing a 12-point lead, DeRozen helped Toronto hold on down the stretch. His step-back jumper put the Raptors back up 89-85 with 4 minutes left.
Then after missing two free throws, DeRozen was fouled again after corralling the rebound, making both this time with 2:09 left for a five-point lead.
O.J. Mayo had 16 points off the bench for the Bucks, including a 3 that tied the game at 85 with 5:53 left.
The Raptors controlled the paint early and outrebounded the Bucks 60-38.
Good ball movement on the perimeter led to a couple of key open shots, including a 3 by DeRozen from the corner to help build a 77-65 lead to start the fourth quarter.
But that deficit is nothing for the Bucks, who fell behind by 20 in each of their first two games on the road, including a rousing 105-98 win earlier in the week in Boston.
True to early season form, Milwaukee stormed back behind frontcourt reserves John Henson and Khris Middleton, and a 3 from Mayo that tied it at 85 with 5:53 to go.
They just couldn't convert on late opportunities, however, including a stretch of 1-of-6 shooting from the free throw line in a 4-minute stretch after tying the game. They also missed two 3s in that span.
The Raptors, and DeRozen specifically, took advantage. He had six rebounds, and Amir Johnson added 11 points and eight boards.
Ersan Ilyasova had 14 points on 6-of-12 shooting for Milwaukee, which lost a home opener for the first time since 2008. Caron Butler, a Wisconsin native playing his first home since being traded to Milwaukee in the offseason, added 12 points and six rebounds.
The game featured all the typical theatrics of a home opener, with a pulsing bass rattling the arena stands and the Bucks walking individually out of the tunnel with the lights dimmed.
But all that fun was spoiled by the Raptors.
It was a new coach and 11 new players for the Bucks but it was still the old court for the home opener.
The teams met for an exhibition game the previous the week in Milwaukee that was called off in the first quarter because the court that was to be newly installed for the 2013 season turned out to be too slick for players to get good footing. That court is being refinished.
The Raptors had no problems early on the court Saturday night, especially in the lane.
Outrebounding the Bucks 31-21 in the first half, Toronto showed little fear driving against Bucks shot-blocking center Larry Sanders. Gay had 11 points at halftime, while Kyle Lowry finished off the half by hitting a 3 just before the halftime buzzer for a 51-46 lead.
Nate Wolters, a second-round draft pick making his first career start at the point because of a hamstring injury to Brandon Knight, had his moments, like when he hit a floater in the lane to the game at 41 with 2:45 left in the second quarter.
His team mired by injuries since the preseason, coach Larry Drew was looking for more consistency in the lineup after falling behind by double digits on both stops of Milwaukee's season-opening two-game trip.
They split the trip with a loss to the Knicks and a win in Boston, with Wolters making key contributions each time of the bench.
Wolters finished with seven points and 10 assists.
NOTES: Besides Knight, the Bucks were also without PG Luke Ridnour (back spasms). ... Raptors G Steve Novak missed his homecoming with a sore back. The seventh-year sharpshooter from the Milwaukee suburb of Brown Deer went to Marquette. He was a freshman on the 2003 team that went to the Final Four.