Audio deepfake, songs made by artificial intelligence, is in its infancy, but it’s already getting interesting. A pretty creditable deepfake Sinatra has been getting a lot of attention, and it’s very close to Ole Blue Eyes vocally. The deepfakes are often called “in the style of” the artists. That’s a ballpark description.
The AI machine-learns the songs from millions of tracks by any number of artists. The “style” is configured using these tracks. Interestingly, the A.I. finds some good hooks, and even bizarre lyrics, some of which even word blacksmith John Lennon wouldn’t have minded.
The sound varies, a lot. Some of it is very much “digital garage”. Doesn’t sound balanced, stuff gets buried in the signal. It’s OK often enough, though, to be worth hearing just to find out what it can do.
What it can do is put together some very interesting stuff and good hooks. It’s not as two-dimensional as the idea might seem. It creates hybrid songs, and some of them really do deserve a bit of respect as musical ideas.
Open AI Jukebox is one of the primary drivers of deepfakes. You can make your own, but there’s a lot to learn about how these things are put together. Jukebox is very open about the limitations of deepfakes, and their site is well worth seeing in terms of what it can and doesn’t do.
I have to say I like the sheer amount of effort that’s gone into making deepfakes work. It’s quite a technical achievement. The sheer volume of data required for any music track is appalling in normal formats, let alone synthesizing new tracks from millions of other tracks.
Be warned – There’s a lot of back end to this idea. You need to check out how it’s done to really understand what it can do and why. There’s obviously lots more tech and evolution to come.
Comparing deepfakes to the revolting, stagnant, formulaic slop now claiming to be pop music, it’s all in favor of the deepfakes. (Face it, bozos; you’re dead and buried, but still making noises.) The deepfakes are comparatively honest attempts to make music, not just recycle recipes for high fructose, allegedly musical nothings.
Copyright issues? Maybe, maybe not.
There’s a naturally sensitive question of whether you can copyright a voice built in to deepfake audio. In theory, yes. In practice, maybe not. A voiceprint can be copyrighted, in theory. Whether or not you can make that stick to an AI which will have a few tonal or pre-programmed differences is another matter.
Musical copyright is a different and sometimes quite nasty, ballgame. Using similar structures can be a breach of copyright, but that could be easy to find and fix. A fairly similar AI could do a “Copyscape for music” quite easily. It’d put to rest the endless copyright tussles, simply reading and comparing the two pieces of music.
You could run your music through a musical Copyscape and see if anything else affects your copyright. Easy, and above all, quick. Unlike the mindless rabid assaults on people sharing a few files, you could resolve the copyright instantly, without the melodrama.
What to do with deepfake audio
There are many possibilities. If you’re a bit of a musical hobbyist like me, someone who occasionally makes unprovoked musical noises, there are many.
Musicians should also note that they can use things like Jukebox for reviewing their own music. The Beatles’ deepfakes are quite interesting, and I’m pretty sure Lennon-McCartney wouldn’t have been slow to see the value of having a few extra options. (They both said, regularly, that songwriting could be a grind. This way of adding options to songs could de-grind some of those issues.)
I was thinking of doing a deepfake Dark Side of the Moon. I’d call it Smug Snide of the Spoon. Somewhere between a sneer and a genuine interest to see what any sort of intelligence could do with the unstated musical assets on most Floyd albums. A track called Great Pig in the Pie, for example, could dovetail Animals with Dark Side, perhaps groping for that ultra-mundanely-lived-in sound. “Obscured by Clods”, perhaps?
On the more practical side of deepfake audio, any musical track is a series of decisions. Most musicians don’t have the time, the patience, or the space to do deeply insightful other options. As a creative asset, deepfake audio may be the AI version of George Martin so many people need.
Anyway – Don’t ignore this. It’s potentially extremely useful and interesting. This is AI’s early steps in musical composition with character rather than just music scores to work with. I see a lot more happening, and much of it is good.
Check out this YouTube playlist of deepfakes and deepfake patches. You’ll hear at least a few things you didn’t expect to hear, and that’s a very good sign for the future.
