Antescofo and SuperVP for Automatic Accompaniment
Auteur | 4 Utilisateurs souscrits | |
---|---|
gorlist |
Hello and happy new year! I would like to reproduce the effect of this demo video: http://www.youtube.com/watch?v=YkMGtpcAA04 I assume Antescofo tracks the tempo and drives SuperVP which does the realtime stretching of the wav file.
|
Janvier 3, 2014 à 14:44 #7336 | |
arshiacont |
Hi Gorlist, Automatic accompaniment has been thought to be an easy affair. Our extended studies in the past year has shown that it’s tricky especially when the accompaniment is in Audio. It is also one of the main reasons we restructured the Antescofo language in the past months. Describing the entire process requires more space and we will do that in a dedicated article. Meanwhile, here are several insights on how you can get along. Keep in mind that a good accompaniment is neither synchronisation with tempo nor position but both! Think of this as synchronizing to tempo, and also wanting the accompaniment to wait for the solo musician or jump to the beginning of phrases. In our approach, this is left to the artist to program using attributes of Antescofo language. Moreover, our experience has shown that a scientifically correct synchronization will lead to a musical disaster! In live situations, the accompaniment will serve as feedback to the musicians, making both go slow and more… . I won’t get into details of this explicitly but will shed some lights for now in the solutions below. In what follows I am going to base on the following experiments on voice/kareoke accompaniment, Piano Concerto Accompaniment, and Jazz Accompaniment. Ingredients– A working version of Antescofo for Max or Puredata Preparing your Solo ScoreFor most existing pieces, you should be able to find a MIDI or MusicXML version of the score on the internet. Once there, just drag-n-drop them into AscoGraph (which will comeup by double-clicking on Antescofo object in Max, or by launching the application in the Antescofo package for Pd users). It’ll be automatically converted to Antescofo format. It’s always good practice to double-check the consistency of the score since some MIDI an MusicXML files are corrupted by nature. An easy way to check this is to press the [play] button on either Max/Pd or AscoGraph and listen to the MIDI simulation of the score. Choosing your Time Stretch StrategyWith SuperVP for Max, you have two choices represented by two Max objects, both stretching an audio file preloaded in a Buffer object (see each object help patch): Now, simplistically speaking, strategy (1) is a good one if you have macro segmentations in your accompaniment audio (e.g. phrase positions to synchronize); whereas the second strategy is good if you have micro-segmentations of the audio (e.g. beat positions). They can ofcourse be combined. Preparing your Accompaniment ScoreIn general, you put the Accompaniment commands inside your Solo score prepared previously. Based on the chosen strategy, there are several (easy) ways to do this. Here I am going to pass you the basic approaches leading to different musical behavior and will leave the details in a more comprenhensive article. I start with the historical (and easiest) one and go towards slightly harder (but more convenient) solutions: (1) Mark down musical positions in the accompaniment audio that musically correspond to the solo part. Then at each starting position in the solo score, you trigger SuperVP.Play by asking it to play from start-position to end-position (marker i to marker i+i) at a duration corresponding to the beat-duration of that excerpt but translated by a function to milli-seconds. Additionally, update the SuperVP.Play speed parameter using the real-time detected tempo of Antescofo. This is the approach in the video posted 2 years ago for Mozart Piano Concerto No.24. If I had to re-do this example, I would use approach #3 below! (2) Pass the accompaniment audio in AudioSculpt and run IrcamBeat. It’ll put markers on every beat! Export the markers as TEXT from AudioSculpt menu. Each marker address (in milli-seconds) corresponds to a beat position. Rearrange them in binary maps, where the first elements is an increasing integer (beat positions), and the second the buffer address. Put them in a Curve in the begining of your score, with an Action, triggering SuperVP.Scrub. (3) Enhance strategy #2: We do not always synchronize with tempo and position. There are places (called Pivots by composer Marco Stroppa) whose position values are more important for synchronization (e.g. beginning of a phrase). This means that we do not want to be tight on all beat positions but on some! This can be simply done by giving position labels to the @tight attribute. Moreover, you might want your accompaniment to have a different tempo that that of the musician… . You want it (for example) to coverge at the END of a phrase. This can be done by the new @target attribute. Once your score is ready, then you just plug it in, press start and play along just as in the videos! The last approach is somehow new and is being intensively tested as we speak! We hope to publish an article with accompanying examples for all three (and other) approaches mentioned here. Meanwhile, hope this helps… . |
Janvier 5, 2014 à 17:04 #7351 | |
gorlist |
Arshia thanks for the great reply!
Is this because of the input from the microphone not being the solo instrument only? From what I’ve read Antescofo can only follow one instrument at a time. Something like http://www.youtube.com/watch?v=puBSu99ytlk . |
Janvier 5, 2014 à 18:46 #7353 | |
arshiacont |
You said:
Not really. Antescofo (in principle) is intelligent enough not to mix the solo from the accompaniment section. If the speakers (diffusing the accompaniment) and the microphone (for solo performer) are fairly spread apart, and you pass through the Calibration Process (as indicated in the HELP and Tutorial) you should not have any major feedback problems. Just as a side note: Antescofo is usually used for synchronizing live electronics with performers (think of the accompaniment audio replaced by several or many audio effects!). We have done concerts in major halls where the flute solo was surrounded by a huge (and loud) orchestra. This is for example the case in Pierre Boulez’ …Explosante-Fixe… (see video on this link.. this performance actually uses Antescofo.. Note the microphone on flute towards 1:10). HOWEVER: I have experimented feedback problems on iPads, if you use the internal mic.. which is very close to the speakers (genious design:). There, we have problems… . I tried some real-time substraction algorithms (already on ios SDK). The downside of all that is introducing DELAY in recognition. Which is bad. But even in the iPad case, if you use external speakers or a cheap Bluetooth microphone the problem is solved! In short: The tricky aspect of automatic accompaniment is not actually technical but musical! For Info: There will be a dedicated session on Automatic Accompaniment in the next Acoustical Society of America (ASA) meeting in Providence (May 2014). We will be there! |
Janvier 5, 2014 à 19:05 #7354 | |
gorlist |
@Arshia, can you please do a tutorial on the integration of SuperVP and Antescofo? |
Mars 17, 2014 à 17:56 #8670 | |
arshiacont |
Attached to this post is a compressed file containing a small Max patch on creating Automatic Accompaniment using Antescofo and SuperVP. The ZIP file also contains a step-by-step demo, including a score, accompaniment audio file, and a simulation file; on Thelenious Monk’s “Round Midnight” (30 seconds). To download the file, you need to see the post on the web and be logged. To run the demo, you need the Antescofo and Supervp.scrub objects either in your Max path or a copy of them next to the patch. Just follow the steps in the patch to see the demo! You can use the “Simulation_piano.aif” file to simulate a musician playing. Alternatively, you can use a microphone (don’t forget to calibrate). Once you load the score “RoundMidnight_Score.txt”, double-click on Antescofo object (open AscoGraph) to see the score. This is how I produced this demo: Preparing the solo scoreIt is easy to find a MIDI score of the piece online! Once found, just drag and drop it to AscoGraph. Then Save the file to a safe place as Antescofo text score. It’ll only contrain the instrumental part. Next, we’ll prepare the audio accompaniment. Preparing the AccompanimentHere, we attempt to do Audio Accompaniment on “RoundMidnight_Accompaniment.aiff” which contains everything except the solo. To do this, we use the SuperVP.scrub object which basically does (very high quality) real-time phase vocoding controlled by the position on audio buffer. We need to produce the correspondence between the instrumental score and the accompaniment audio. To do this, I extract beat positions on the accompaniment audio using IrcamBeat technology already in AudioSculpt, convert them to a <tt>Curve</tt> and insert it in the score. Here is the procedure to follow:
The best way to test the coherence of your result, is to open the patch, load your score and NOTE: The Curve in the score above interpolates between break-points with a time-grain of 0.05 beats and sends them to SuperVP as buffer positions to synthesize. There is definitely more to synchronization strategies than what described here. We are in process of publishing and preparing documentations on this new feature.
Attachments: |
Mars 18, 2014 à 13:19 #8676 | |
gorlist |
Arshia, thank you for your quick reply. |
Mars 20, 2014 à 16:29 #8743 | |
![]() Jean-Louis Giavitto |
Hi Gorlist. Audiosculpt is used here just to produce the beat markers. You can use Protools or Logic. Audacity is an open source free software you can use to produce the markers. See for instance this help page. You can put the label by hand, and then export it. You have then to edit the exported file to achieve the same syntax as the one used in the example given by Arshia. Alternatively, you can install the Queen Mary Vamp plugins that propose a beat tracker (never used, so I cannot tell you the quality of the produced markers).
|
Mars 20, 2014 à 17:03 #8749 | |
arshiacont |
Gorlist, As Jean-Louis mentioned in his email, the only thing you need is to be able to put MARKERS on your accompaniment audio and translate them to CURVE points in Antescofo. The MARKERS should correspond more or less to your synchronization points. One easy way to do this is to just “tap” in beats by hand on the audio file while being played as Markers. Some software allow that. Once Markers are created then you need to export them somehow as lists in text file. A good hack for this is the sfmarker~ object in Max! As mentioned in my previous message this is rather straight forward with AudioSculpt. We are thinking of integrating this workflow into AscoGraph through a drag-n-drop of Audio in future version. |
Mars 21, 2014 à 14:27 #8751 | |
Grig Burloiu |
Bringing this topic back into focus. A lot of great information in this thread, thank you Arshia. Is everything here current or have there been more advancements in automatic accompaniment techniques? I’ve started using strategy #2 described above. Instead of using AudioSculpt, I just extracted the important sync points from the score manually. I then have a curve driving supervp.scrub~ that looks like this:
This is of course similar to the Round Midnight example you posted. Now, wouldn’t it be nice if instead of writing out / computing all of these curve breakpoints, we could generate them dynamically?
But this doesn’t work, I’m not sure why. Are the curves even being created?
But we get a syntax error, looks like forall is not allowed within a Curve definition. Cheers, |
Avril 1, 2015 à 16:01 #13139 | |
![]() Jean-Louis Giavitto |
Hello Grig. There are some ways to achieve the construct you mention. A I can suggest two approaches to achieve your requirements. The first approach is to consider a Curve with only two breakpoints and to embed this
Notice two things : the index of the current breakpoint must stay the same during the whole Curve. So we update it first at the entry of the loop body. The second thing to note is the The second approach is perhaps more close of what you have in mind. There is an alternative version of the curve construct where, instead of a predefined set of breakpoints, you use a Here is an example of a dynamic
See the reference manual page 84 for defying and managing NIM and page 53 for using a Curve to “play” a NIM. One of the interest of a NIM is that you can extend it dynamically. For example, in the following code :
|
Avril 1, 2015 à 16:36 #13140 | |
Grig Burloiu |
Thanks Jean-Louis, looks like the dynamically constructed NIM is what I was looking for! |
Avril 1, 2015 à 16:48 #13141 | |
![]() Jean-Louis Giavitto |
I forgot to mention that the synchronization strategies apply equally well to a |
Avril 1, 2015 à 17:05 #13142 | |
Grig Burloiu |
Hi guys, back to this old topic One question: is there a way to pause&resume Antescofo’s follower (and, implicitly, the Curve running SuperVP), in-between events? Like say, have an external message trigger a I know we can |
Août 3, 2017 à 09:32 #23361 |
Vous devez être connecté pour répondre à ce sujet.