Duffy drove in Mallex Smith, who doubled with one out and moved to third base on a ground ball.
Diego Castillo (1-0) got his first major league win after pitching two innings of relief.
Ryan Tepera (3-2) pitched the final 1 2/3 innings for the Jays, who were swept in the three-game series and have lost five of six to the Rays this season. It was the 12th time the Blue Jays have scored one run or none this season.
Curtis Granderson had the first three of Torontos five hits, leadoff singles in the first, third and fifth innings and was the only runner to reach third base until Smith in the ninth.
With two outs in the Toronto ninth, Aldemys Diaz hit a line drive that was caught by left fielder Johnny Field near the top of the wall.
Toronto starter J.A. Happ, who was 3-0 in his five previous starts, gave up one hit and three walks in five innings.
Rays starter Wilmer Font gave up two hits in 3 1/3 innings.
Blue Jays: 3B Josh Donaldson (left calf tightness) will run the bases Thursday and if all goes well could join Triple-A Buffalo on Friday.
Rays: OF Johnny Field was back in the lineup after missing two games with a sprained knee.
SIXTEEN NOT-SO SWEET
Thursdays game at New York marks the start of a 16-game stretch in which the Rays will play only the Yankees, Astros and Nationals. “Were going to really see what kind of team we have here against the best in the league,” Blake Snell said.
Blue Jays: RHP Aaron Sanchez (3-5, 4.33 ERA) will pitch Friday night against the Washington Nationals in the opening game of a five-game homestand.
Rays: LHP Blake Snell (8-3, 2.30 ERA) will start in Thursday nights opener of a four-game series at Yankee Stadium against Domingo German (0-4, 5.32). Snell is 3-0 with a 0.38 ERA in his past four starts.