electro-music.com   Dedicated to experimental electro-acoustic
and electronic music
 
    Front Page  |  Radio
 |  Media  |  Forum  |  Wiki  |  Links
Forum with support of Syndicator RSS
 FAQFAQ   CalendarCalendar   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   LinksLinks
 RegisterRegister   ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in  Chat RoomChat Room 
 Forum index » Clavia Nord Modular » G2 Patches - Completed » Audio In
Spectral Vocoder
Post new topic   Reply to topic Moderators: Nord Modular Editors
Page 1 of 2 [30 Posts]
View unread posts
View new posts in the last week
Mark the topic unread :: View previous topic :: View next topic
Goto page: 1, 2 Next
Author Message
Tim Kleinert



Joined: Mar 12, 2004
Posts: 1148
Location: Zürich, Switzerland
Audio files: 7
G2 patch files: 236

PostPosted: Wed Feb 03, 2010 5:10 pm    Post subject: Spectral Vocoder
Subject description: Realtime spectral analysis and additive resynthesis. G2 goes frequency-domain. :)
Reply with quote  Mark this post and the followings unread

Hi,

I've been reading up on DFT and FFT and pondered how much of it could be applied to the G2. It was obvious pretty quickly that a complete frequency-domain representation of a given audio signal and consecutive processing was totally out of the question. However, by severely limiting the amount of frequency bins, simple stuff could be achieved.

My idea was that a monophonic, harmonic and highly periodic signal can be represented by the sum of it's partials and their magnitudes. So, let's say, if I want to graft the spectral information of a source signal onto this monophonic pitch, I only need to extract from the source signal the magnitudes of the sinusoids that directly correspond to the frequencies of these partials. This limits the required frequency bins down to a very small amount -which the G2 can handle. Smile

The result of that idea is this patch, which I call a spectral vocoder. In some ways it is similar to the "classic" vocoder, in other ways not at all. The classic vocoder is based on analysis and resynthesis bandpass filter banks, which arbitrarily filter and process the source and carrier signals without knowing much about them. But in this case I know that the result shall be a pitched monophonic signal, so therefore I know a) exactly which information I want from the source signal (the magnitudes of the sinusoids that correspond to the partials of my monophonic pitched signal), and b)how to best represent this signal: by the sum and magnitudes of it's partials. In other words: by additive synthesis.

Since the standard computer DSP programming approach to measuring the magnitudes of sinusoids in a source signal is impossible to realise on the G2, I opted for an old "analog" technique, which is to heterodyne the source signal with the sine and cosine of the frequency to be measured, and taking the square root of the quadratures of the results. (This actually is nothing else but transforming rectangular coordinates into polar coordinates, with the polar radius being the magnitude of the sinusoid). Quadratures are easy on the G2, and thanks to a building block by Mike Estlick, square roots are possible too.

Patched in the most economical way possible, the process described above uses close to 25% of a DSP. And since I always wanted to build a patch that completely maxes out an expanded G2 Laughing, here finally was the opportunity. Smile But instead of tediously filling up all VA and FX slots with these analysis stages one by one, I used the G2s polyphony feature, each voice conveniently acting as a macro for a single analysis/resynthesis band. These bands are controlled by the FX area via interslot busses. So I get 30 bands in total from an expanded G2, which is as good as it gets. Of course you can use less bands by turning down the voice count (or using an unexpanded G2 Laughing), but the resulting sound will be increasingly dull.

So, at last, a single monophonic voice that maxes out your expanded G2! Laughing

The controls:
INPUT: Select channel for the source audio
INPUT COMP: Compress the source audio to taste
Resp1 and Resp2: Response characteristics of the analysis stages. Controls the sensitivity, speed and immunity to inharmonic frequency content.
FORMANT: Shift formants up and down (simply by offsetting the analysis and resynthesis frequency bands)
HI THRU Freq/Level: Just like classic vocoders, let's you add some high frequency content of the source signal, especially good for sibilants and "air".
KEY PLAY: Attack and Release times of the keyboard envelope.
PRE-ANALYSIS EQ: Tweak the source audio before it goes to the analysis stage.
POST-RESYNTHESIS EQ: Tweak the resynthesized result.

...and there was DSP power left for a delay module (yay! Laughing).

There's a short demo mp3 of this patch too, with me talking into the microphone and playing some keys. G2 goes Antares. Laughing

This is probably as close as you can get the G2 into frequency-domain territory. (But never say never Wink )

cheers,
tim


SpectraVocoderTK.pch2
 Description:
Monophonic vocoding by realtime spectral analysis and additive resynthesis. Monophonic monster patch that completely maxes out an expanded G2.

Download
 Filename:  SpectraVocoderTK.pch2
 Filesize:  3.08 KB
 Downloaded:  2515 Time(s)


SpectraVocoder_demo.mp3
 Description:
Demo of the spectal vocoder patch. G2 goes Antares. :o)

Download
 Filename:  SpectraVocoder_demo.mp3
 Filesize:  657.18 KB
 Downloaded:  2000 Time(s)

Back to top
View user's profile Send private message
jamos



Joined: Jun 01, 2004
Posts: 514
Location: Eugene, Oregon, USA
Audio files: 4
G2 patch files: 41

PostPosted: Wed Feb 03, 2010 9:07 pm    Post subject: Reply with quote  Mark this post and the followings unread

Cool, cool. Very Happy I use on of my G2's primarily for vocal processing, so I'll throw this at my singer and see what he thinks.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Roland Kuit



Joined: Sep 29, 2003
Posts: 1090
Location: The Netherlands/Sweden
Audio files: 8
G2 patch files: 127

PostPosted: Wed Feb 03, 2010 10:05 pm    Post subject: Reply with quote  Mark this post and the followings unread

wow....nice 1 tim. thanks for charing
Back to top
View user's profile Send private message
dorremifasol



Joined: Sep 28, 2006
Posts: 814
Location: Barcelona, Spain
Audio files: 7
G2 patch files: 49

PostPosted: Thu Feb 04, 2010 9:34 am    Post subject: Reply with quote  Mark this post and the followings unread

Impressive!!!
_________________
Cheers,
Albert
Back to top
View user's profile Send private message Visit poster's website
Wan



Joined: Mar 31, 2004
Posts: 259
Location: Netherlands, Ugchelen
Audio files: 4
G2 patch files: 46

PostPosted: Fri Feb 05, 2010 12:53 am    Post subject: Reply with quote  Mark this post and the followings unread

Amazing patch again! Shocked
_________________
Grtz Wan
Back to top
View user's profile Send private message Visit poster's website
Tim Kleinert



Joined: Mar 12, 2004
Posts: 1148
Location: Zürich, Switzerland
Audio files: 7
G2 patch files: 236

PostPosted: Fri Feb 05, 2010 5:35 am    Post subject: Reply with quote  Mark this post and the followings unread

Glad you like it. Smile It was quite an adventure. In the beginning I had a pitch-tracker as well to track the pitch of the incoming signal to extract the partials thereof and port those to the pitch of choice in the additive resynthesis stage. This resulted in the formants being shifted as well, producing the "mickey mouse" effect. However, we all know that the pitch-tracker is so-so, so it caused occasional warbles in the spectral distribution, and anyway -who want's mickey mouse? Wink.

It was so obvious to extract the sinusoid magnitudes from the input signal directly (without pitch-detection) that I didn't think of it doh -hence porting to the desired pitch only those magnitudes that correspond to it's partials, and therefore retaining formants. Easier this way, no pitch-track warbles, and no mickey-mouse.

You can hear the high-frequency partials ring a little bit, which makes it sound more artificial than it should. This is due to the square root function being based on feedback, which is noticeable especially in the higher partials which have high frequency and low amplitude. There's no way around this unfortunately. It would sound better otherwise.

cheers,t

EDIT: If you want some fun, try twiddling the partial quantizer module, which defines the partial spacing. You can tweak it eg. to only odd harmonics, sounds pretty weird. Laughing
Back to top
View user's profile Send private message
fairplay



Joined: Apr 13, 2009
Posts: 11
Location: Germany, near Munich

PostPosted: Fri Feb 05, 2010 5:47 am    Post subject: Reply with quote  Mark this post and the followings unread

...that's embarrasing! - how do you make things like this? - all the time?? - again and again???...i feel pretty depressed now {being unable to do things like this myself}...

Wink

...thank you for sharing!...
Back to top
View user's profile Send private message
Tim Kleinert



Joined: Mar 12, 2004
Posts: 1148
Location: Zürich, Switzerland
Audio files: 7
G2 patch files: 236

PostPosted: Fri Feb 05, 2010 8:05 am    Post subject: Reply with quote  Mark this post and the followings unread

Too much spare time I guess Rolling Eyes Embarassed ...Laughing Wink
Back to top
View user's profile Send private message
klangumsetzer



Joined: Jan 23, 2006
Posts: 513
Location: Sweden
Audio files: 32

PostPosted: Fri Feb 05, 2010 12:34 pm    Post subject: Reply with quote  Mark this post and the followings unread

tim wrote:
(...) It was so obvious to extract the sinusoid magnitudes from the input signal directly (without pitch-detection) that I didn't think of it doh -hence porting to the desired pitch only those magnitudes that correspond to it's partials, and therefore retaining formants. (...)
This is due to the square root function being based on feedback, which is noticeable especially in the higher partials which have high frequency and low amplitude. (...)

obvious? Shocked uh, hm, what was that middle part again? Laughing

thanks for sharing, tim. keep up the excellent work!


best regards

eike
---
Back to top
View user's profile Send private message
3phase



Joined: Jul 27, 2004
Posts: 1183
Location: Berlin
Audio files: 13
G2 patch files: 141

PostPosted: Tue Feb 09, 2010 4:12 pm    Post subject: Reply with quote  Mark this post and the followings unread

Shocked
Back to top
View user's profile Send private message Send e-mail Visit poster's website
blue hell
Site Admin


Joined: Apr 03, 2004
Posts: 24079
Location: The Netherlands, Enschede
Audio files: 278
G2 patch files: 320

PostPosted: Tue Feb 09, 2010 5:15 pm    Post subject: Reply with quote  Mark this post and the followings unread

3phase wrote:
Shocked


Well this sounds great, and i don't understand a bit of it either

Shocked

_________________
Jan
also .. could someone please turn down the thermostat a bit.
Posted Image, might have been reduced in size. Click Image to view fullscreen.
Back to top
View user's profile Send private message Visit poster's website
Inventor
Stream Operator


Joined: Oct 13, 2007
Posts: 6221
Location: near Austin, Tx, USA
Audio files: 267

PostPosted: Tue Feb 09, 2010 6:20 pm    Post subject: Reply with quote  Mark this post and the followings unread

The main part that I don't understand is how does the patch determine what the fundamental frequency of the voice is? I don't know the G2, so looking at the patch won't help me there. Anyone understand that part of it?

Les

_________________
"Let's make noise for peace." - Kijjaz
Back to top
View user's profile Send private message Send e-mail
seraph
Editor
Editor


Joined: Jun 21, 2003
Posts: 12398
Location: Firenze, Italy
Audio files: 33
G2 patch files: 2

PostPosted: Wed Feb 10, 2010 12:53 am    Post subject: Reply with quote  Mark this post and the followings unread

Blue Hell wrote:
3phase wrote:
Shocked


Well this sounds great, and i don't understand a bit of it either

Shocked


that's encouraging Shocked anyway I will soon post a song featuring it Wink

_________________
homepage - blog - forum - youtube

Quote:
Don't die with your music still in you - Wayne Dyer
Back to top
View user's profile Send private message Visit poster's website
Tim Kleinert



Joined: Mar 12, 2004
Posts: 1148
Location: Zürich, Switzerland
Audio files: 7
G2 patch files: 236

PostPosted: Wed Feb 10, 2010 5:46 am    Post subject: Reply with quote  Mark this post and the followings unread

Oh, all these "shocked" emoticons... what did I do. Embarassed Laughing

inventor wrote:
The main part that I don't understand is how does the patch determine what the fundamental frequency of the voice is?

It doesn't. Smile That's the whole point. Smile It extracts from the source signal only the sinusoid magnitudes that correspond to the harmonic series of the monophonic destination pitch. This is then synthesized from these magnitude measurements via additive synthesis.

It's a bit expensive for only one resulting voice (to put it mildly Laughing) from an expanded G2, so almost posted it in the "experimental" subsection. Laughing It's just that when I read up on DFT, I thought "Come on, the G2 must be able to do this, if only a very little bit." It worked out. Smile Glad you like it.
Back to top
View user's profile Send private message
blue hell
Site Admin


Joined: Apr 03, 2004
Posts: 24079
Location: The Netherlands, Enschede
Audio files: 278
G2 patch files: 320

PostPosted: Wed Feb 10, 2010 1:09 pm    Post subject: Reply with quote  Mark this post and the followings unread

tim wrote:
what did I do. Embarassed Laughing


Shocked you brought some life here, thats a good thing Exclamation

Cool

_________________
Jan
also .. could someone please turn down the thermostat a bit.
Posted Image, might have been reduced in size. Click Image to view fullscreen.
Back to top
View user's profile Send private message Visit poster's website
drapdap



Joined: Oct 11, 2004
Posts: 204
Location: London
Audio files: 1
G2 patch files: 1

PostPosted: Sat Feb 13, 2010 10:00 am    Post subject: Reply with quote  Mark this post and the followings unread

this one is so much fun, thanks Tim!

i made my engine boot with this one for now... Very Happy
it was just what the doctor ordered, as i added an xlr input recently with a knob, so i can vocode with just a mic and the engine. and a keyboard, erh, you lucky G2 key owners.

thanks again, Tim...
Back to top
View user's profile Send private message
seraph
Editor
Editor


Joined: Jun 21, 2003
Posts: 12398
Location: Firenze, Italy
Audio files: 33
G2 patch files: 2

PostPosted: Sun Feb 14, 2010 6:05 am    Post subject: Reply with quote  Mark this post and the followings unread

seraph wrote:
I will soon post a song featuring it Wink


Posted Image, might have been reduced in size. Click Image to view fullscreen.

arrow http://www.seraph.it/dep/det/TheSingingAlchemist.mp3

read blog for more informations Very Happy

_________________
homepage - blog - forum - youtube

Quote:
Don't die with your music still in you - Wayne Dyer
Back to top
View user's profile Send private message Visit poster's website
Tim Kleinert



Joined: Mar 12, 2004
Posts: 1148
Location: Zürich, Switzerland
Audio files: 7
G2 patch files: 236

PostPosted: Mon Feb 15, 2010 4:50 pm    Post subject: Reply with quote  Mark this post and the followings unread

Thank you Carlo. Seeing/hearing ones patches used for artistic expression is the ultimate compliment.

cheers,
t
Back to top
View user's profile Send private message
seraph
Editor
Editor


Joined: Jun 21, 2003
Posts: 12398
Location: Firenze, Italy
Audio files: 33
G2 patch files: 2

PostPosted: Tue Feb 16, 2010 1:07 am    Post subject: Reply with quote  Mark this post and the followings unread

tim wrote:
Thank you Carlo. Seeing/hearing ones patches used for artistic expression is the ultimate compliment.

cheers,
t


Tim,
I gave credit to you, on my blog entry, for the Spectral Vocoder. It's easy to use a patch like yours "for artistic expression" when they are so beautifully built Exclamation
Thanks to you Very Happy

_________________
homepage - blog - forum - youtube

Quote:
Don't die with your music still in you - Wayne Dyer
Back to top
View user's profile Send private message Visit poster's website
Inventor
Stream Operator


Joined: Oct 13, 2007
Posts: 6221
Location: near Austin, Tx, USA
Audio files: 267

PostPosted: Wed Feb 17, 2010 2:44 am    Post subject: Reply with quote  Mark this post and the followings unread

I played Carlo's song on my radio show and mentioned the g2 patch of tim's after the song. You guys are collaborating so well it makes for great radio!

Les

_________________
"Let's make noise for peace." - Kijjaz
Back to top
View user's profile Send private message Send e-mail
iPassenger



Joined: Jan 27, 2007
Posts: 1067
Location: Sheffield, UK
Audio files: 5
G2 patch files: 78

PostPosted: Wed Feb 17, 2010 6:35 am    Post subject: Reply with quote  Mark this post and the followings unread

seraph wrote:
seraph wrote:
I will soon post a song featuring it Wink


arrow http://www.seraph.it/dep/det/TheSingingAlchemist.mp3

read blog for more informations Very Happy


This is great. Smile Love the oboe-ish sounds and the delicate percussion is sublime. I presume you have some kind of custom tuning set-up.

_________________
iP (Ross)
- http://ipassenger.bandcamp.com
- http://soundcloud.com/ipassenger
Back to top
View user's profile Send private message Visit poster's website
seraph
Editor
Editor


Joined: Jun 21, 2003
Posts: 12398
Location: Firenze, Italy
Audio files: 33
G2 patch files: 2

PostPosted: Wed Feb 17, 2010 6:44 am    Post subject: Reply with quote  Mark this post and the followings unread

iPassenger wrote:
I presume you have some kind of custom tuning set-up.


thanks and yes, you are right Exclamation check out my blog for more informations Wink

_________________
homepage - blog - forum - youtube

Quote:
Don't die with your music still in you - Wayne Dyer
Back to top
View user's profile Send private message Visit poster's website
G2egory



Joined: Nov 19, 2004
Posts: 69
Location: Charlottesville VA
Audio files: 3
G2 patch files: 2

PostPosted: Wed Feb 17, 2010 5:13 pm    Post subject: Reply with quote  Mark this post and the followings unread

The Spectral Vocoder is dynamite. Thanks for sharing the patch.
Back to top
View user's profile Send private message
peterkadar



Joined: Jan 11, 2009
Posts: 152
Location: Toronto, Canada
G2 patch files: 1

PostPosted: Thu Feb 18, 2010 2:30 am    Post subject: Reply with quote  Mark this post and the followings unread

wow, this is incredible! Thanks so much for all your hard work, and for sharing it with the rest of us.

Your Kung Fu is strong!!
Back to top
View user's profile Send private message
seraph
Editor
Editor


Joined: Jun 21, 2003
Posts: 12398
Location: Firenze, Italy
Audio files: 33
G2 patch files: 2

PostPosted: Sun Feb 28, 2010 6:19 am    Post subject: Reply with quote  Mark this post and the followings unread



one more example of microtonal spectral vocoding using Tim's patch Very Happy

_________________
homepage - blog - forum - youtube

Quote:
Don't die with your music still in you - Wayne Dyer
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic Moderators: Nord Modular Editors
Page 1 of 2 [30 Posts]
View unread posts
View new posts in the last week
Goto page: 1, 2 Next
Mark the topic unread :: View previous topic :: View next topic
 Forum index » Clavia Nord Modular » G2 Patches - Completed » Audio In
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Forum with support of Syndicator RSS
Powered by phpBB © 2001, 2005 phpBB Group
Copyright © 2003 through 2009 by electro-music.com - Conditions Of Use