The move drew condemnation from Kyiv and Western nations, who dismissed the votes as a sham and pledged not to recognize their results.
Ukrainian officials said people were banned from leaving some occupied areas until the four-day vote was over.
Adding that armed groups were going into homes, and employees were threatened with the sack if they did not participate.
Reuters could not immediately verify reports of coercion.
In Kyiv on Saturday, internally displaced people from Mariupol, like Oleh Sukhov, protested the vote.
"Today a referendum is taking place in Mariupol. I am strongly against it. It is not needed. How, say, I, a native Mariupol resident, am now able to say 'no' in this ballot? I have no such right. I considered myself, I consider myself a Mariupol resident and I want Mariupol to be Ukraine."
But back in his hometown, some, like Alexei are grateful for the vote... telling Reuters "we are only hoping for the best."
The votes on becoming part of Russia were hastily-organized after Ukraine recaptured large swathes of the northeast in a counter-offensive earlier this month.
In his nightly address on Friday, Ukrainian President Volodymyr Zelenskiy condemned the referendums.
"The world will react absolutely appropriately to the pseudo-referendums - they will be unequivocally condemned, and to the criminal mobilization which the occupiers are trying to conduct now in Crimea and other parts of Ukraine which they are temporarily in control of. These are not just crimes against the international law and Ukrainian law. These are crimes against specific people, against a nation."
By incorporating the four areas, Moscow could portray attacks to retake them as an attack on Russia itself.
And potentially use that to justify even a nuclear response - a terrifying prospect in a war that has already killed tens of thousands of people, uprooted millions and pummeled the global economy.
Polling stations were also set up in Moscow for residents of those regions now living in Russia.
Flag-waving government supporters attended rallies in Moscow and St. Petersburg in favor of the referendums and the war effort.
Ukraine, Western leaders and the United Nations have slammed the votes as an illegitimate precursor to illegal annexation.
There are no independent observers, and much of the pre-war population has fled.
Moscow says the referendums offer an opportunity for people in the region to express their view.
Russian President Vladimir Putin maintains Russia's actions in Ukraine are aimed at carrying out a "special military operation" to demilitarize the country, rid it of dangerous nationalists and defend Russia from transatlantic alliance NATO.