Yes, this sounds like it would be a great idea as long as there's not a lot of play between the hitch and the adapter, otherwise your cammback may have a bit of sway to it.
Just as a side note, if anyone's interested in trying this out, I still have the trailer hitch I took off my Prius sitting in my garage
