Does this question need an engineer?
The only degrees I have are my temperature, and a buddy tried to fly this idea past me. I raised the same questions you did: how in the world would you balance it? It doesn't take advanced edumecationing to know this would be a dog's breakfast.
I put it to my friend: why not just make all the cylinders work as well as you can possibly make them? Dropping out a couple of jugs when you don't need them is well and good, but mixing cylinder sizes in one engine feels like creating a problem for the intellectual challenge of then fixing it.
SAAB had a design a long time ago for a variable compression engine; one side of the top end was hinged from the bottom end, and a hydraulic servo would move the top up and down to bigger- and smallerize the combustion chamber as necessary.
You'll notice that SAAB didn't follow up on it, instead spending their time developing their brilliant turbo engines, a similarly effective and much more achievable method of making an engine behave as if it were two different mills. Those were the days.
You're not in front of me...
I'm 25,000 miles in front of you.