Caicedo turned and fired past Wojciech Szczesny in the 95th minute after great wing-play from Joaquin Correa.
Cristiano Ronaldo scored his sixth goal in four league games when he converted Juan Cuadrado’s cross to give Juve the lead in the 15th minute.
They remain third, three points behind AC Milan who play Verona at 19:45 GMT.
The result will be a bitter blow for coach Andrea Pirlo and his men, who were rarely troubled by Lazio. Simone Inzaghi’s team had plenty of possession but had failed to test former Arsenal keeper Szczesny up until the goal.
Juve looked far more potent each time they surged forward. Ronaldo was again their standout player in the final third. Aside from the goal, the former Real Madrid player struck the woodwork with a shot, produced a drive that missed the post by a foot and also saw his thumping free-kick punched over the bar by Pepe Reina.
The forward was substituted with 15 minutes of the match remaining after appearing to suffer a knock in a challenge.
Lazio move up to ninth in the Serie A table with 11 points.