Voice Cloning: How It Works, and Why Consent Matters

A person's voice is one of the most personal things about them. We recognize family and friends from a single word on the phone, and we trust what we hear when a familiar voice speaks.

Voice cloning changes that. With the right software and a sample of someone's speech, it's now possible to create a synthetic version of their voice that can say anything. The technology has genuinely valuable uses, from giving people back a voice they've lost to helping businesses create consistent spoken content. It has also been used to deceive, defraud, and mislead.

This article explains how voice cloning works, where it's used, the risks it creates, and why consent sits at the heart of using it responsibly.

What Is Voice Cloning?

Voice cloning is the use of AI to create a synthetic voice that sounds like a specific real person. Once a voice has been cloned, text-to-speech technology can use it to say any words, in that person's voice, even words they never actually spoke.

It differs from ordinary text-to-speech, which uses generic voices that don't belong to any identifiable individual. A cloned voice is meant to be recognizable as a particular person.

How Voice Cloning Works

Learning what makes a voice unique

Every voice has distinctive qualities: its pitch, tone, accent, rhythm, the way certain sounds are pronounced, and subtler characteristics shaped by the size and shape of the person's vocal tract. Voice cloning systems analyze recordings of a person speaking and learn these characteristics.

Many systems create a compact digital representation of the voice, sometimes called a voice embedding or voice profile. The speech synthesis system then uses this profile to generate new speech that carries the same characteristics.

Two main approaches

High-quality or professional cloning typically uses longer recordings, often from around half an hour to several hours of clear, studio-quality speech. The speech model is trained, or fine-tuned, specifically on that voice. The result is usually very close to the original speaker, with natural expression and consistency across long passages.

Rapid or instant cloning works from much shorter samples, sometimes just a few seconds to a minute of audio. Instead of training a new model, it uses a large model that has already learned from many voices and adapts to the new voice from the sample. The results can be convincing, especially in short clips, though they may be less consistent and less expressive than professional clones.

The speed and ease of rapid cloning are what make it both useful and concerning. A short clip from a video, a voicemail greeting, or a social media post may be enough to create a recognizable imitation.

Legitimate Uses of Voice Cloning

Used with proper consent, voice cloning has many valuable applications.

Restoring and preserving voices

For people facing conditions that may take away their ability to speak, such as motor neurone disease (also known as ALS), certain cancers, or the aftermath of a stroke, voice cloning can preserve their own voice. Through a process often called voice banking, they record their speech while they still can, and a synthetic version is created for use with communication devices. Speaking in their own voice, rather than a generic one, can mean a great deal to them and their families.

Media and entertainment

Voice cloning is used to dub films and videos into other languages while keeping an actor's voice, to make small corrections to recorded dialogue without bringing an actor back to the studio, and to narrate audiobooks in an author's own voice.

Brand and business voices

Businesses sometimes create a consistent branded voice by working with a voice actor who agrees to have their voice cloned. This allows the business to produce spoken content, phone messages, or AI voice agent conversations in that recognizable voice, under a clear agreement with the actor.

Personal use

Some people create clones of their own voice to read their written messages aloud, produce content more efficiently, or communicate in situations where speaking is difficult.

The Risks of Voice Cloning

The same technology that makes these uses possible can also be misused, and in ways that are particularly convincing because people instinctively trust familiar voices.

Impersonation scams

One of the most common forms of misuse is the family emergency scam. A person receives a call that sounds like a relative, often a child or grandchild, saying they're in trouble, perhaps arrested or in an accident, and urgently need money. The panic and the familiar voice make it hard to stop and think.

Businesses are targets too. Criminals have used cloned voices of executives to instruct employees to transfer funds urgently. Reports of such frauds have appeared in several countries, sometimes involving very large sums.

Political and public deception

In January 2024, voters in New Hampshire received robocalls featuring an AI-generated imitation of President Joe Biden's voice, discouraging them from voting in the state's primary election. The US Federal Communications Commission later fined the political consultant responsible $6 million, and the phone company that carried the calls also reached a settlement with the regulator. The incident was a prominent example of how cloned voices can be used to mislead the public.

Harm to the person whose voice is copied

Voice cloning without permission can damage a person's reputation by putting words in their mouth, take away professional work from voice actors and performers, and cause real distress to people who discover their voice is being used without their knowledge.

Undermining trust in what we hear

More broadly, as cloned voices become more common, people may begin to doubt genuine recordings and calls, making it harder to trust any voice heard over the phone or online.

Why Consent Matters

Because a voice is so closely tied to a person's identity, consent is the foundation of responsible voice cloning. Meaningful consent generally means that it is:

• Informed. The person understands what voice cloning is, how their voice will be used, and what the synthetic voice will be able to say.
• Specific. The consent covers particular uses, such as a business's phone messages or an audiobook, rather than unlimited use for anything.
• Documented. It's recorded clearly, often in a written agreement, so there's no dispute later about what was agreed.
• Given by the right person. The person consenting is genuinely the owner of the voice, not someone with access to their recordings.
• Fairly compensated, where the voice is used commercially, particularly for professional voice actors and performers.
• Revocable or time-limited, where appropriate, with clear terms about what happens if the person withdraws or the agreement ends.

How Responsible Providers Protect Against Misuse

Many reputable voice cloning providers build safeguards into their services. These can include requiring the person whose voice is being cloned to record a spoken consent statement, verifying that this statement matches the voice in the sample, checking that recordings come from a live speaker rather than existing media, restricting the cloning of well-known public figures, adding inaudible markers, often called watermarks, to synthetic audio so it can be identified later, and monitoring how cloned voices are used and removing ones that break the rules.

None of these measures is perfect, but together they make misuse harder and easier to trace.

The Law Is Catching Up

Laws governing voice cloning are developing quickly and vary widely between countries.

In the United States, the FCC ruled in 2024 that AI-generated voices count as "artificial" voices under the Telephone Consumer Protection Act, which generally means calls using them require prior consent. The Federal Trade Commission has also taken steps against impersonation fraud, including a rule targeting the impersonation of businesses and government agencies. Some states have passed their own protections. Tennessee's ELVIS Act, enacted in 2024, specifically protects people's voices from unauthorized use, and longstanding right of publicity laws in many states protect people from having their likeness or identity used commercially without permission. Federal legislation addressing unauthorized digital replicas has also been proposed.

In the European Union, the AI Act includes transparency obligations, requiring that AI-generated or manipulated audio, often called deepfakes, be disclosed as such in many circumstances.

Because the rules are changing, anyone using voice cloning commercially should check the current requirements wherever they operate and wherever their audio will be heard.

Protecting Yourself From Voice Cloning Scams

As cloned voices become more convincing, a few simple habits help protect people and businesses:

• Be wary of urgency. Scams rely on pressure. A request to act immediately, keep it secret, or not tell anyone else is a warning sign.
• Verify independently. Hang up and call the person back on a number you already know is theirs, rather than the number that called you.
• Agree on a family code word. A word or phrase only your family knows can quickly confirm whether a caller is genuine.
• Watch for unusual payment requests. Requests for gift cards, cryptocurrency, wire transfers, or cash are common signs of fraud.
• Set business verification rules. Businesses can require that any payment or sensitive request made by phone is confirmed through a separate, established channel, however senior the person asking appears to be.
• Be mindful of what you share. Publicly posted videos and voice recordings can provide material for cloning.

Responsible Use in Business Calling

For businesses considering cloned voices in their calls, the principles are straightforward. Only clone a voice with the clear, documented consent of the person it belongs to. Set out in writing exactly how it will be used. Never use a cloned voice to impersonate someone or to suggest a real person is speaking when they aren't. And be open with the people being called that they're hearing an AI or synthetic voice.

A Voice Is Part of Who Someone Is

Voice cloning is a remarkable technology. It can give a person back their own voice, carry a performance across languages, and allow businesses to communicate consistently. But because a voice is so deeply tied to identity and trust, using it without permission isn't just a technical shortcut. It's taking something personal. Consent, transparency, and safeguards are what separate voice cloning that helps people from voice cloning that harms them.

Related articles

• What to Do If You Receive, or Fall for, a Scam Call