Well, I think I figured out the problem. Possibly.
I'm going to share my screw up with everyone because I learned a valuable lesson, and I'm pretty sure this is the source of my problem.
The first and second time I desoldered my LCD I used a common screw driver to pry apart the boards, because I didn't desolder them correctly.
And when I did that prying I put divets in the board...
Now, I didn't know that the signal line from the atmega runs along the outside of the board so I didn't know my screw driver was going to cause any real damage besides superficial...
So, here's a picture of the damage... For anyone who doesn't know - DO NOT use a screw driver to pry apart circuit cards!
For everyone else, - please feel free to laugh
So, now I guess a fix would be to solder in jumper wires to send the signals over to bypass the damage I did... Time to study the schematics I suppose..