PORTLAND, OR – After 23 years in development, former Nixon’s Head and Autumn Carousel musician Mike Fingeroff has released a White Paper in support of the pending release of Strat-o-Matic Rock ‘n Roll, a virtual band development, composing, and recording tool.
“Although I respect the work Pete Townshend and his developers have been doing,” said Fingeroff in a recent interview, “their software will allow users to create little more than a musical self-portrait. That’s cool, but what rock ‘n roll fans really want to know is, ‘What would a band with Jimi Hendrix, John Bonham, John Entwistle, and Gary Rossington sound like?'”
What Strat-o-Matic Rock ‘n Roll offers is not only a chance to form these Supergroups but, through the software, to accurately simulate the writing and recording of original music. The possibiliities are endless, but the best way to understand exactly how this works is to study the following White Paper. Fingeroff will be available to field your technical questions.
Strat-o-Matic Rock ‘n Roll is a real-time system level performance modeler. It differs from other “performance engines” in that it allows both the novice and expert musician to create complete musical productions by applying a series of constraints in the design environment. The end result is pristine audio with song qualities that can rival creations that would normally takes years of practice to duplicate.
The key patent-pending technologies involve a combination of Linear Predictive Coding techniques for modeling vocal tract formant frequency acoustics (Fig. 1), Blind Adaptive Filtering analysis (Fig. 2) and reproduction of numerous amps, guitars, acoustics, etc, and Neural Networks for “on the fly” performance creation and modeling of a wide range of rock-n-roll styles.
Because of the compute-intensive nature of the various algorithms used to process a performance, massively parallel hardware is required to run in real time (Fig. 3). These algorithms, described using pure untimed C++, were taken directly to synthesizable RTL by using the industry leading high-level C++ synthesis tool. The auto-generated RTL is then synthesized, placed, and routed, and then it is run on a stand-alone Altera Stratix-II FPGA-based platform. This is what we refer to as the Strat-o-matic Magic Box. The Magic Box can be run as a stand-alone platform for live performances, or can also be interfaced with a PC using either light pipe or Firewire interfaces. The Stratomatic performance engine can run as an independent application or as an AU plug-in under Logic, Pro Tools, etc.
Strat-o-matic Rock ‘n Roll has two primary modes of operation: performance mode and accompaniment mode.
In performance mode the user drives the song creation process by selecting the instruments and instrument styles. Pre-canned templates are provided for quickly creating either general period sounds or specific instrument arrangements, e.g.
Song Template > Beatles > Rubber Soul > I'm Looking Through You
Once the initial instrument tracks have been created the user can drive the song creation by adjusting parameters on the performance engine. The performance engine has been designed to operate on a simplified set of constraints for the beginner or nonmusician. These constraints consist of general purpose constraints, such as Mood, Rhythmic Style, Room, etc. These general-purpose constraints run a number of heuristic algorithms based on the instrument performance settings. Once the initial constraints are set the user simply clicks on the Generate button and the song is created.
For the novice this may be the last step in the song creation process, however there are a number of additional optimizations available to the advanced user. When running in simple mode, the vocals are auto generated using the performance database, vocal tract models, as well as a specialized phrasing engine that mimics vocal inflections of many of rock ‘n roll’s best vocalists. The limitation with this mode is that the lyrics are drawn from a limited database of about 20 songs, leading to a limited number of vocal possibilities and themes. There is some room for customization here, but advanced users will probably wish to cut their own vocals. Users can run their own vocal tracks through the Strat-o-Matic vocal algorithms. This will extract a baseline excitation that is then fed though a user selectable vocal tract model. The phrasing engine will post process the reconstructed vocal applying stylistic vocal transitions. Auto pitch detection and correction algorithms are also run with no artifact insertion. This approach usually leads to more natural sounding vocals.
The other operational mode is accompaniment mode. This is a real-time mode that allows the user to play their instrument of choice with Strat-o-Matic providing the back up. This mode uses many of the same performance settings used in performance mode to allow the specification of instruments sets, tempo, style, etc. The key technology that differentiates accompaniment mode is the “learning engine.” The learning engine processes the user’s input instrument in real-time and performs causality analysis to adapt the instrument set to match the user’s style. Advanced foot pedal control is supported to allow the user to drive the performance engine when things like guitar solos and drum fills are desired at specific places in the performance.
Strat-o-Matic Rock ‘n Roll is currently being beta tested by a number of industry leading music electronics companies, but more beta testers are being accepted currently. The product will not be commercially available until Q3 08.
Mike Fingeroff holds advanced degrees in both Electrical Engineering and Computer Science and is currently working in the field of high-level synthesis and electronic system level design.
Mike,
I’m glad you fianlly put the pen to paper on this. I know how long you have worked on this.
At Princeton, I have access to both a IBM BlueGene Supercomputer (top-100 fastest in world) as well as a number of different “beowulf” clusters. Do you have a preference of which will work better?
Also, will Torque, Maui, LoadLeveler, or Condor work w/ Strato-matic?
That’s and interesting question although it is a little outside of my field. I believe that what your talking about is more distributed computing using software scheduling. I don’t doubt that the type of platform your talking about could meet the computing requirements, but unless you have several million dollars at your disposal it isn’t practical for the average consumer. The stratomatic approach is to take all the software untimed C++ algorithms that make up Stratomatic and to synthesize them to masssively parallel RTL (VHDL, Verilog) using high-level C++ synthesis (This is a more hardware based type of scheduling where operator scheduling, sharing, and resource allocation are done at the operator level.. This can lead to more than a 1000x runtime spped up over C++ running on a processor. FPGAs (Field Programmable Gate Arrays) provide a cost effective way to create a hardware platform that can run in real-time. We chose the Altera StratixII 90nm devices mainly because their architecture provides built-in DSP blocks that can run up to 400MHz. Currently the host software only integrates with Logic, ProTools, etc runninng under OSx.
I hope that answered your questions?
Mike
Definitely. Crytal clear.
Mike — Your system sounds mighty impressive. It’s a shame cross-regional, real-time collaboration would still be impossible due to network latency. No matter how sophisticated TCP/IP acceleration technologies get, the fundamental laws of physics cannot be overcome!
Still, the solitary, pasty, hollow-eyed, basement-dwelling musical accretionists of the world thank you for your hard work, I’m sure. Well done!
Hi Fritz,
the beauty of this system is that there
is no need for collaboration. It essentially allows one person to create the entire
performance. No more arguing over what
drum fill should go where, how to play
the lead, etc. Hopefully in a couple
more years we will have it to the point
where we can even eliminate the one
required person.
Can the program be set to limit developmental stages of a given musician, such as Richards pre-feathered roach clip, or must you work with the whole span of an artist’s work?
From reading your piece and some of the comments, I have a firm understanding of the technical aspects of this program. Now I’m curious to learn more about its practical applications. Thanks.
That type of fine-grained control is probably several years away. Right now the interface is more driven by either selecting a particular song or album, although you can also input general parameters e.g. Lennon > 1965 to 1967.
As Mr Moderator described in his introduction the practical application of this is to be able to combine your favorite musicians in a musical framework. Who wouldn’t love to hear what Lennon and McCartney would have sounded like if John Bonham or Neil Peart were manning the kit.
It also let’s non-musicians, or guys like myself that were never very good, to level the playing field. Now I don’t have to feel self conscious that I’m not playing a solo fast enough, or things like that.
Mike
Good question, Meanstom. I was hoping to select parameters to have both Fat Elvis and Skinny Elvis performing in the same band. Fingeroff’s answer seems to imply that the conflicting drivers would create a feedback loop.
What’s all this computer crap?! I thought it was going to have player cards and dice!
No, but there is an online mode that allows multiple users to login and create a “Jamband”. Each user can either manually pick a band member, or can have it chosen randomly. Once the band has been constructed users can each provided inputs into the performance creation process. After perfromance creation Stratomatic RR will launch a podcast for band members to critique thier creation.