The "voltage" you are concerned with is the dielectric breakdown voltage. If you put capacitors in series, the voltage drop is split between the capacitors. So if you put identical capacitors in series, the new peak allowable voltage is the number of capacitors times the breakdown voltage of a single capacitor.
The capacitance is the harmonic mean of them, which in the case of identical capacitors is just 1/(number of capacitors).
It's analogous to resistors in parallel, except instead of adding current you add voltage, and instead of taking the harmonic mean of resistances you take the harmonic mean of capacitances
