You are not talking about a tensioner, are you?
By the way, the 2002 - 2006 Camry tensioner is a pain. This is how Shouty and everyone else says to change the serpentine belt:
http://i.imgur.com/xrf0IDt.gif
He complained the original was made of "pot metal," but the new one was high-quality. Well, here is Mom's original:
Here is the high-quality new one on Mom's car with the same part broken:
She towed it to her mechanic who used a special tool. I do not have any idea what it was, all of the tools that I see are basically sockets on breaker bars.