Yes, the voltage slowly changes across the capacitor when it is connected to a simple circuit. This happens due to a continuous accumulation of electrons across metallic plates of the capacitor.
Due to the assembly of charges, the capacitor starts to become charged. This gradual increment of charge directly causes the voltage to drop across the capacitor. The voltage drop across the capacitor is directly proportional to its charge, hence the voltage increases across the capacitor in a steady manner.

How Does Voltage Change Across a Capacitor
The voltage V across the capacitor is directly proportional to the number of charges which are assembled across the capacitor. For a given capacitance, it is known that, the greater the charge, the greater the voltage that is dropped across the capacitor. Hence, Q = CV.
Due to the accumulation of charges across the metallic plates of the capacitor, the charges tend to attract each other. But due to the dielectric medium between them, the charges are stuck on the plates. The stationary charges on these plates create an electric field, which influences electrical potential energy and voltage.
Voltage Change During Charge

Initial Stage
In a simple RC circuit, the total voltage provided by the source is initially dropped. As the voltage drop across the capacitor is proportional to its charge, thus being an uncharged capacitor, it works as a short circuit right when the electrons begin to circulate the circuit. The resistor’s voltage is proportional to the current, due to electron circulation, the current flows through the resistor and ideally, the total provided voltage given by the source is consumed by it.
Change of voltage during charge
A metallic plate that is placed at one end of the capacitor, starts to be negatively charged as it faces the negative pole of the voltage source. Similarly, the other end of the capacitor becomes positively charged.
Gradually the charges start to build up around the capacitor, so portions of source voltage start to drop across the capacitor. Due to the increment of charge around the metallic plates of the capacitor, the voltage starts to increase proportionally.
On the other hand, the voltage dropped across the resistor starts to lessen. This results in the amount of current being decreased more and more. This is because, due to electrostatic repulsion, it gets more challenging for the charges that are being drifted towards the capacitor, to join the stationary electrons present in the negatively charged metallic plate of the capacitor.
Thus, the rate of charge starts to decrease around the RC circuit. In mathematical terms, the charge as a function of time Q (t) increases but the slope decreases. The theory says the charge obeys the formula Q(t)= CVs (1-e-t/τ).

Final Stage
At this point, the capacitor plates will be so full of charges that they cannot accept anymore. As the capacitor ideally consumes the total voltage from the source, hence there is no voltage drop across the rest of the resistive circuit.
Ideally, the total voltage provided from the source is directly stored by the capacitor for discharging purposes. Thus there is ideally no conduction of current, and so we can say that the capacitor works as an open circuit during DC generation. The corresponding voltage across the capacitor is V (t) = VS(1-e-t/ τ).

Discharging
After disconnecting the voltage source from the RC circuit, the capacitor works as a voltage source for a given specific time. Initially, the charges will start to flow towards the resistor, hence current conduction would be ignited at the start.
Gradually, the amount of charge across the capacitor will be decreased more and more, hence the amount of current will slowly decrease. The voltage across the capacitor will decrease correspondingly. In the end, there will be no voltage across the capacitor. Thus, the conduction of the current will ideally become zero.
Frequently Asked Questions
What happens when capacitors are connected in parallel?
When capacitors are connected in parallel, they are all connected directly to the voltage source. Each capacitor has the same voltage as the voltage source and each capacitor has a separate charge.
Therefore, the resultant capacitance of the circuit increases and will be able to store more energy as the equivalent capacitance is the sum of individual capacitances of all capacitors involved.
Conclusion
A capacitor can be slowly charged to the necessary voltage and then discharged quickly to provide the energy needed. It is even possible to charge several capacitors to a certain voltage and then release them in such a way as to get more voltage (but not more energy) out of the system than was put in.
Do capacitors connected in parallel continually, later drain the dc voltage on which it is connected to?
Capacitors connected in parallel with lower capacitance can drain the DC voltage on which they are connected to. It is important to choose capacitors with similar capacitance and low leakage current when connecting them in parallel to prevent voltage drainage.