Gestural Controller Software Design and Conclusion

Firstly a video another proof of concept video for my supporting work for my masters:


Construction 4 Edit from TheAudientVoid on Vimeo.
Another bit of proof of concept video from my supporting documentation for my Masters.
The drums at the start were fed into the markov chain drum recorder earlier. Basically this patch takes what you put into it, makes a markov grid and spits permutations of its input out according to whatever method you use to retrieve the data (in this case it uses the recorded midi sequence being played back to create note on/offs which send a bang to the pitch generation patch. These are quantised to 16th notes and output).

You can see how the gestural control works with the gloves pretty clearly at the start as the hand position is used to control a filter over a drum section.

 Around the 3 minute mark i start playing some percussion live instead of using the markov chain recorded section.




and now the final sections of my Dissertation, please look at the annotated patch pictures that accompany the text as they are meant to be seen in conjunction with this section. There are in fact many more annotated patches in the actual maxforlive device but I will post about those another day in a more detailed breakdown of the software design

 

Software Design

Once the hardware has been designed and created the software must be made to produce useful values for control of Ableton Live. As maxforlive offers an almost infinite possibility of functions it is important to decide what you wish to do with the software before you start building it. “By itself , a computer is a tabula rasa , full of potential , but without specific inherent orientation. It is with such a machine that we seek to create instruments with which we can establish a profound musical rapport . ”(Tanaka 2)


It is important that we create a system whereby the software plays to the strengths of the performer and hardware design, these elements must work in tandem to create an innovative and usable ‘playing’ experience. Firstly the Arduino data has to be understood by Max/Msp. I chose Firmata as it uses the robust OSC protocol to transmit data to max/msp and is also provided with pre made max/msp objects for receiving the data, this code proved to be very stable and fast at passing messages. Once this was uploaded to the board and xbees were configured correctly it becomes simple to receive values that can become usable in your software. As we are using a range of analogue sensors it is important to include a calibration stage in the software so that minimum and maximum values can be set, inputs can be smoothed and then also assigned to a function. To this function I used the “Sensor-Tamer” max patch as a basis for creating a calibration system for all the inputs. These are then scaled and sent to a max patch which allows us to choose an effect from the current Ableton live set.

Left Hand Maxuino Input and other modules, annotated

Right Hand Maxuino Input and other modules, annotated

The analogue inputs can be made to produce midi messages as well as directly controlling parameters of effects from Live menus, the advantage of this is that you can then operate with two distinct modes, one for controlling fx parameters and other for passing midi messages to synths. Due to the fact that Ableton Live merges midi from all input channels to one channel of output you have to use the internal routing (S and R objects) functions of Max/Msp to send midi to a number of tracks. Obviously as this is a control system for live performance you need to have a way to control more than one synth/plugin and you want to be able to control various parameters for each synth. Creating small plugin objects for the channels you wish to control makes it easy to do this and as these simply pipe the midi from the input channel to the selected max receive object and because of this it is possible to assign the same physical controller to a different midi assignment on every channel. This again comes back to the watchword of customizability and allows the user to create dynamic performances where many elements can be changed without touching the computer. This also works neatly around the problem of only being able to send information to the active channel in a sequencer as your midi is routed ‘behind the scenes’ and effectively selects the channel you wish to use without any physical selection of the channel (i.e. no mouse click is required).

The footpedal which currently implements record and step through patch features

As the system is to be used to perform live there are a number of utility functions which also need to be created such as freezing and recording loops, stepping through channels, octaves and patches. These are best implemented away from the gloves themselves as the gloves are most intuitive to play when using both hands (8 notes over two hands), as you can only have a fixed number of switches that are easily playable it makes sense to assign these to notes (with sharps of notes being achieved through a foot pedal). Having switches used for playing on both hands also means that you can create polyphony by pressing down finger switches on both hands simultaneously. There is also the practical consideration that you do not want to have to stop playing a pattern to push a button to record a loop or to freeze things, by moving these functions to your feet you can continue playing whilst accessing control functions. For ease of use recording and freezing functions are assigned to all looping plugins from a single switch, as you are only sending midi data to one channel at a time there is no chance of creating a ‘false-positive’ and recording unwanted sounds in the wrong channel and having one switch to operate freeze or record greatly simplifies control for the end user.

I also decided to use a phone mounted on my arm running touchOSC to control some functions of Ableton live as it is useful in some cases to have visual feedback and again this allows the gloves to be freed up for musical functions. Some of these functions echo the footswitch controls to allow the performer to move away from the laptop and into the audience and as touchOSC has two-way midi control it updates the status of a switch or setting to correspond with the footswitch being pressed so there are no crossed signals. With touchOSC it is easy to design your own interface and to assign buttons to Ableton Live functions. As this essentially operates as a midi controller it is only necessary to put the software into midi learn mode, click the function you wish to assign and touch the button on the phone. This again allows for a high level of customizability for the end user and for interfaces to be made and set up according to the type of performance you wish to create. It is for example particularly suited to triggering sounds or prerecorded loops as many buttons are required for this (one button per clip) and this would not be sensibly achievable using the gloves. Although currently using a predesigned interface due to hardware constraints it is my aim to implement a touchOSC system that as well as providing controls for loops and other parameters provides a full set of feedback from the gloves and foot pedal and thus it will be possible to see what instrument, bank and so forth you have chosen in the software. This will become vital to the projects aim of being able to move completely away from the computer when performing.













At the time of writing this I did not have an apple device to create a custom layout so this HUD was used to show data from Max on the laptop .



Algorithmic Variation

“Each artwork becomes a sort of behavioral Tarot pack, presenting coordinates which can be endlessly reshuffled by the spectator, always to produce meaning”(Ascott 1966 3)

The Markov Chain Recorder/Player, Annotated

I decided that I wanted to be able to manipulate midi data within my performance to produce a number of variations to the input. These variations had to sound human and make intelligent choices from the data that was presented. To this end I have used Markov Chains to analyze midi data to create a system whereby a circular causal relationship between the user and the patch is developed. The patch takes midi input and then creates a probability table as to which note will be played next, after each note is generated it is fed back into the system and used to look up the next note from the probability grid. This means that whatever midi data is fed to the patch will be transformed in a way that preserves the most important intervals and melodic structures of your original data but allows for permutation, this in turn means that the performer must react to what the patch outputs and there is the possibility to input more data to change the markov chain that you are using and thus alter the performance further. In essence I wished to create a system of patches that function very much like an improvising live band, a certain set of melodic parameters are agreed upon, by midi input, and then used as a basis for improvisation. The data from these markov chains can be output in two ways, either the computer can be set to automate the output itself or you may use the gloves the push data from the markov chain into a synth, both of these methods yield different but equally valid musical results and allow the performer to create very different types of results. The idea of using markov chains to create predictable but mutating data has much in common with Cybernetic and Conversation theory where the interaction of two agents and the interpretation of these leads to the creating of a third which in turn influences the original agents. If we consider the original midi data in the patch to be the first agent and the person using the controller to be the second the interpretation of data from the computer influences the playing of the person using the controller and in turn this can be fed back into the computer to create another set of data which is again interpreted, permuted and responded to by the performer. This application of disturbing influences to the state of a variable in the environment can be related to Perceptual Control Theory.
“Perceptual control theory currently proposes a hierarchy of 11 levels of perceptions controlled by systems in the human mind and neural architecture. These are: intensity, sensation, configuration, transition, event, relationship, category, sequence, program, principle, and system concept. Diverse perceptual signals at a lower level (e.g. visual perceptions of intensities) are combined in an input function to construct a single perception at the higher level (e.g. visual perception of a color sensation). The perceptions that are constructed and controlled at the lower levels are passed along as the perceptual inputs at the higher levels. The higher levels in turn control by telling the lower levels what to perceive: that is, they adjust the reference levels (goals) of the lower levels.” (Powers 1995)
Despite this being in reference to control systems in the human mind it is easy to see how it is also applicable to computer control systems, the higher level systems that are accessible by the user tell the software what to perceive, this is done in two ways, firstly the input of midi data, this input allows the software to create a lower level abstraction, being the table of probability, which is then called upon to trigger notes.
The changes must be subtle and controlled enough that the performer is reacting to them and responding rather than fighting the computer to maintain control of the system. The process that is used to determine probability of notes is a closed system to the performer (all one needs to do is feed in a midi file) the performer has access to an open system which can be used to alter key characteristics of the processes after this, they also have access to play along with this process through a separate control system linked to an instrument, hence the feel of improvising along with a band is created. In Behaviourist Art and the Cybernetic Vision Roy Ascott states: “We can say that in the past the artist played to win, and so set the conditions that he always dominated the play”(Ascott 1966 2) but that the introduction of cybernetic theory has allowed us to move towards a model whereby “we are moving towards a situation in which the game is never won but remains perpetually in a state of play” (Ascott 1966 2) Although Ascott is concerned with the artist and audience interaction we can easily apply this to the artist/computer/audience interaction whereby the artist has a chance to respond to the computer input and the audience and to use this response to shape future outcomes from the computer, thus creating an ever changing cyclical system that rather than being dependant on the “total involvement of the spectator” is dependent on the total involvement of the performer.

Improvements

Having worked on developing this system for two years there are still improvements to be made, although the idea to use conductive thread would have been very good from a design and comfort point of view, as it allowed components to be mounted on the glove without bulky additional wiring, the technology proved to be too unstable to withstand normal usage and when creating something for live performance it needs to be robust. It was the case with this design that something could be working in one session and not the next, and obviously if a mission critical thread was to come unraveled it had the potential to take power from the whole system rather than causing a single element not to work. Also the thread, being essentially an un-insulated wire, if not stitched carefully created the possibility of short circuits when the glove were bent in a particular way. In addition to this the switches, even when used with resistors (also made of thread) produced a voltage drop in the circuit that changed the values of the analogue sensors. Obviously this change in values will change what happens to a parameter that the sensor controls and therefore can produce very undesirable effects within the music you are making.
Although the accelerometers produce usable results for creating gestural presets and manipulating parameters the method used to work out the position of the hands could be further improved by the use of gyroscopes instead. Gyroscopes are able to accurately tell the position of an object when it is not in motion where as accelerometers work best when in a state of constant motion. With a gyroscope we would be able to introduce an addition value into our gestural system, we would be able to tell the amount of rotation from the starting position, and this would allow us to use very complicated gestures to control parameters within Ableton.
The current ‘on the glove mounting’ of the components works but is in my opinion not robust enough to withstand repeated usage and so it will be important to build the gloves again using a more modular design. Currently the weak point is stress placed on soldered connections when the gloves twist or bend and even though using longer than necessary wiring helps to alleviate this it does not totally solve the problem, therefore it is necessary to create a more modular design which keeps all soldered components contained and does not subject them to any stress. The best way that this could be achieved would be to mount the Xbee, Arduino and power within a wearable box housing and have all soldered connections housed within it as well. To make sure there is no cable stress it is possible to mount screw down cable connectors in the box for two wire input sensors and three pin ribbon cable connectors for analogue sensors, in this way no stress is put on the internal circuitry and the cabling is easily replaceable as none of it is hard soldered. These cables would run between the box and a small circuit board mounted on the glove near the sensor where the other end would plug in. This also increases the durability of the project as it can be disassembled before transport and as such does not risk any cables getting caught or pulled and makes every component easily replaceable, without soldering, in event of a failure.
I would like to introduce a live ‘gesture recording’ system to the software so that it is possible to record a gesture during a live performance that can be assigned to a specific control, this would allow the user to define controls on the fly in response to what movements are appropriate at the time. However this will take considerable work to design and implement effectively as value changes must be recorded and assigned in a way that does not break the flow of the performance and although it is relatively simple to record a gesture from the gloves by measuring a change in values of certain sensors assigning these to a parameter introduces the need to use dropdown boxes within the software to choose a channel, effect and parameter and how to achieve this away from the computer is not immediately apparent. It may be possible to choose this using touchOSC when an editor becomes available for the android version of the software, but as yet this is not possible.
Further to this the touchOSC element of the controller must be improved with a custom interface which collects suitable controls on the same interface page and receives additional feedback from Ableton such as lists of parameters controlled by each sensor, the sensors current value and the names of clips which can be triggered. Using the Livecontrol API it should be possible to pass this information to a touch screen device but again without an editor being available for the Android version of touchOSC this is not yet possible. I have investigated other android based OSC software solutions such as OSCdroid and Kontrolleur but as yet these also do not allow for custom interfaces. OSCdroid however looks promising and having been in touch with the developer the next software revision will include a complex interface design tool that should allow for these features to be implemented. I will be working with the developer to see if suitable Ableton control and feedback can be achieved once this has been released.

Conclusion

In essence the ideas and implementations I have discussed mean that we can create an entire musical world for ourselves informed by both practical considerations and theoretical analysis of the environment in which we wish to perform. We can use technology to collect complex sets of data and map them to any software function we feel is appropriate, we can use generative computer processes to add a controlled level of deviation and permutation to our input data and we can use algorithms to create a situation whereby we must improvise and react to decisions made by the computer during the performance of a piece. We can have both total control of a musical structure and allow a situation whereby we must respond to changes being made without our explicit instruction. It is my hope that through this it is possible to create a huge number of different musical outcomes even if using similar musical data as input. The toolset that I have created hopefully allows the performer to shape their work to the demands of the immediate situation and to the audience they are playing to and opens up live computer composition in a way that allows for ‘happy mistakes’ and moments of inspiration.
As previously stated it is my hope that these new technologies can be used to start breaking down the performer and audience divide. It is possible to realize performances where the performer and audience can enter into a true feedback loop and can both influence the outcome of the work. In the future there is the potential to also use camera sensing and other technologies (when they are more fully matured and suitable for use in ‘less than ideal’ situations) to capture data from the crowd as well as the performer. The performer can remain in control of the overall structure but could conduct the audience in a truly interactive performance. This technology potentially allows us to reach much further from the stage than traditional instruments and to create immersive experiences for both performer and audience. It is this idea and level of connection and interactivity that should move electronic musicians away from traditional instrument or hardware modeling controllers and look for more exciting ways to use technology.

“All in all, it feels like being directly interfaced with sound. An appendage that is simply a voice that speaks a language you didn't know that you knew” Onyx Ashanti

Adaptive Physical Controllers - Part 3 - The Dance Music Ritual

The Dance Music Ritual

As the name suggests Dance music has a specified aim of producing movement in the audience and to create a sense of community amongst them “Your kinaesthetic sense is externalised by being transferred from your own body into the body of the crowd… The room ceases to be occupied by strangers, instead it is filled with the party folk all satisfying their need to be”(Jackson 2004 19). Slogans such as “Peace, Love, Unity and Respect” (PLUR ethos) exemplify the community ideas of dance music audiences and Turners idea of Spontaneous Communitas “the transient personal experience of togetherness” has been taken on by many dance music scholars to explain the feeling of community and connection that the audience may experience in a rave setting.

“If the ecstatic raver is indeed an anonymous body of textless flesh, one that has shed its identity, ideology and language, one that has either divested or radically altered its culturally inscribed body image, then the thematic boundaries that normally delineate our edges are destabilized and perhaps dissolved. Dancing amidst a crowd of ecstatic bodies, the raver is consumed not only by an immediate ‘experience’ of the phenomenal world, but also by his or her body’s subconscious knowledges of unity and alterity (not to mention genderless sexual specificity)—knowledges that are quite different from those of self- reflective thought. Lost in the reflexivity and natural transgressivity of the flesh, in its indeterminacy and interwovenness, the raver is a mute witness to the blurring of once clear demarcations between himself and the crowd, between herself and the rave.” - Landau in (St. John 2004 p.121)

In my past work I have looked at the use of rituals within music in both modern and tribal cultures, in modern society this is seen most clearly on the dancefloor of clubs, there is a tribal and ritual element to dancing together and the musical style that accompanies this “‘repetitive, minimalistic, seamless cyclings of sonic patterns accompanied by a relentless driving or metronomic rhythm” (Fatone 2001) which creates not only community between the dancers but often ecstatic experience. James Landau states “in the psychoanalytical view, ecstacy’s transgressive relationship to binary thought stems from the rave-assemblage propelling its participants into the Real, a cognitive space ‘beyond’ the ego and its organizational structures” (St. John 2004 ) This idea is supported by St John who states, “The party makes possible a kind of collective ego-loss, a sense of communal singularity - a sensation of at-one-ness - is potentiated”(St. John 2004). However often the performer cannot take part in this ecstatic experience due to their physical disconnection from the dancefloor and the movement of the dancers. Is it not strange that a musician can produce music that makes their audience dance but they must remain rooted in place behind their computer? Would it not be more beneficial for the performer to be able to join the dance and become part of the community they are creating music for? Would they not be more fully immersed in their own sonic landscapes if unshackled from the computer screen and became free to roam the space their sound occupies, interacting with the audience and using their whole body to feel their performance in the way the audience does?

One of the huge benefits of electronic instruments is that as they do not have an element which needs a microphone and as such they are not subject to feedback in the same way that a traditional musician would be, this simple fact has seemingly been overlooked in the majority of live performances and the traditional room setup of placing the performer in a position of separation from their audience is adopted, by creating wireless wearable controllers it is possible to move beyond this traditional staging setup. I would assert that the ritual and community aspect that dance music embraces would be furthered immeasurably by the breaking down of the audience/performer divide and creating a situation where no one is placed ‘on a pedestal’ but instead all are intertwined with each other. Within the Electronic Music Scene there are far fewer ‘superstar’ performers than in other musical genres and although some performers break into the mainstream and achieve wide spread acclaim many are much less willing to fulfill the traditional hero archetype. Indeed the community often quickly derides those who are seen to have ‘risen above their station’ or developed an overbearing ego. When talking about this the musician Shackleton says “in rock music you have a projection of the individual, and it’s almost like the extension of a performance art where you have an individual being / doing a very egotistical thing and in that context it’s wonderful… because of course that person is venting something and the crowd can enjoy that, in that context. But I think I’ve never really seen it like that, the artist isn’t so important.”(Brignoli 2011) “I don't need lots and lots of money, I don't need a lot of fame or this sort of thing, I just like doing what I'm doing. That's good for me.”(Keeling 2010) Even very well known DJ’s such as Paul Van Dyke are known for their grounded attitudes towards their work “He is so sincere and is one of the nicest people I've ever met. You don't expect someone that's so well known to be so humble." Jessica Sutta (2001). Some artists and groups take this idea even further, Scot Gresham-Lancaster in his article about ‘the Hub’ (an ‘interactive computer network music group’) states “The Intent to detach ego from the process of music making we inherited directly from Cage. To refine that impulse and make a living machine that both incorporates our participation and lets the breath of these new processes out into the moment”. (Gresham-Lancaster 1998)

If as St John states ‘electronic dance music would be a conduit for experimentation, transgression and liberation, with rave becoming the manifestation of counter-culture continuity”(St. John 2008 156) the this freedom should logically be extended to break down traditional audience performer divides. Onyx Ashanti describes this sensation of using a wireless controller whilst being amongst the audience “I "thought" I would do what I usually do, which was to stand in front of the DJ booth and "perform". Not the case, at all! Before I realized it, I had eased into the crowd and was dancing with a couple of very attractive women, BUT WAIT...I was still creating and playing as well!”(Ashanti 2011) We can see clearly from this quote the excitement of the performer in this situation, that he can interact with the audience whilst creating and feedback into his system the energy from the audience. Within this context it is clear that the gestures one must ascribe to their controller are those of dancing, the performer must be able to dance with the audience and use their gestures to both manipulate the music and interact with others. I believe it is this situation, facilitated by the movement of the performer away from the computer that will truly revolutionize the performance of dance music.

Adaptive Physical Controllers

Well I havnt updated this for a while but its because I have been busy writing my dissertation for my MA in Sonic Arts. There are some videos of my project coming soon when I edit things together but for now I will serialize my dissertation for you all to read if your interested in these topics. It covers both the theory behind alternative controllers and what I have done with my own work. I'll be posting a couple of sections of it up every day, although its very text heavy I hope it will be interesting to some of you who also are interested in these type of projects and will explain a bit about it all. When I've released the whole thing on here I will also put a download link to a printable pdf version which will include all the images, bibliography etc..

So without further ado here is the first section which covers the introduction and also some discussion of existing digital input devices for music

Adaptive Physical Controllers - 

Introduction

In the past input devices for interacting with computers to produce or manipulate sound have largely been modeled on traditional hardware and replicating the features of these devices. However developments in computing technology allow us to interact with computers in much more exciting and dynamic ways. When interacting with computers for performance we must ask ourselves, do we wish the computer to model a traditional instrument? Or do we wish to create our own instruments and modes of interaction based on our specific requirements?  In an age where computer processor technology has moved forward so rapidly we should be looking for new models of human computer interaction whilst taking the best aspects of traditional approaches with us. In the past many physical controllers have been used to generate a sound or change parameters within a synthesizer (DiGenova and Massachusetts Institute of Technology. Dept. of Mechanical Engineering. 2010 19) but it is important to recognize that with the complexity of today’s software and the availability and ease of use of making home build electronics it is possible to use a controller for almost any purpose you can imagine.

I will be looking in particular at the linking of Max/Msp and Ableton Live as it allows us to create complicated controller interfaces and devices whilst allowing us to access the API of a powerful live music system. Harnessing the flexibility of Max/Msp with the more traditional and time domain orientated approach of Ableton Live allows the performer to create an adaptive system whereby a controller can be used to perform multiple tasks and control any parameter, for example the pitch, timbre and velocity of a synth, the tempo of a song, a parameter within an effect, what sounds are playing, and so on. It is this adaptive nature of home built personalized controllers that allow us to explore new ways of interacting with computers and music. Projects such as David Jeffrey Merrills Flexigesture (Merrill and Massachusetts Institute of Technology. Dept. of Architecture. Program In Media Arts and Sciences. 2004) “a personalizable and expressive physical input device” and Onyx Ashantis “Beatjazz” project (Ashanti 2010) move towards this goal and attempt to combine the best aspects of traditional hardware controllers with the possibilities audio programming languages and custom controllers present to us.

In my project I have attempted to make a pair of gloves that can be used to create and manipulate music within Ableton Live. The aim of this is being able to play live improvised electronic dance music without interacting with the computer directly. In many aspects of live electronic music the excitement of performance has been lost, there is often little interaction between the performer and the computer, and even if the music is composed and created in real time there is little for the audience to visually identify with. Unlike a performance with traditional instruments it is almost impossible for the audience member to visualize what the performer is doing. When using a computer the ‘wall of the screen’ separates the performer from the audience and obscures their actions “Conventional sonic display technologies create a “plane-of-separation” between the source/method of sound production and the intended consumer. This creates a musical/social context that is inherently and intentionally presentation (rather than process) orientated”(Bahn, Hahn et al. 2001 1). It is one of the central paradoxes for the electronic performer, although they have a box that is capable of creating almost any sound imaginable the central mechanisms for creating these sounds are obscured from all but the user themselves. I wish to find a way to attempt to overcome this problem by creating a system that allows access to the many features computer music software offers whilst removing the user from a fixed position in front of the computer screen and creating a direct visual feedback for the audience.

Modeling Controllers on Traditional Instruments

There are clearly benefits to modeling controllers on traditional instruments, by doing so you provide a safe reference point for the user and in theory reduce the learning curve required to play it (providing the user has previous instrument training). By working in a familiar framework you play on the existing strengths of the performer, however there are limitations to traditional instruments that I believe make them unsuitable for use as a modern day controller. Traditional use of a computer requires many keys and key combinations to perform specific functions, this is relatively easy using a keyboard and mouse as every key is individually marked and key combinations are easily pressed. However if you translate this idea of a grid of keys to the fretboard of a guitar you can begin to see the problems that may occur. It is very difficult to translate a vast number of controls to a small number of keys and direct midi instrument mapping often yields the problem that software will only allow you to control one parameter at once and many midi sequencers and live performance programs do not allow you to easily switch between channels and instruments.

There is obviously a great difference between an instrument that attempts to simply recreate the analogue in a digital domain and a controller that seeks to redefine performer and computer interaction, for example the Yamaha WX5^[1] seeks to recreate the experience of playing a woodwind instrument but with extra keys for computer control, shifting octaves and so on, if seeking to simply replace a traditional instrument with a digital one instruments like this are an effective choice. However as we are looking to create a new type of computer control interface the mechanics and implementation of these instruments are less relevant to us than something such as the Eigenharp^[2] which bills itself as ‘The most expressive electronic instrument ever made’ and attempts to go further than simply recreating existing instrument designs. Indeed it looks to incorporate aspects of many existing instruments and to allow the user to play VST plugins to create a hybrid design that straddles both traditional and digital instrument designs. Undoubtedly the quality of the keys, their velocity sensing and the ability to move them in both a horizontal and vertical direction goes a long way to allowing the player to perform all the traditional expressions associated with musical instruments^[3] in a way that has not been available in the past and design features such as the inclusion of a breath controller and excellent instrument models allows the player to easily replicate, for example, wind instrument sounds. However it is the more strictly digital interactions with this controller that may leave the end user wanting.

 The Eigenharp, in a desire to remain as traditional as possible, takes the approach of using a complex set of key presses and lights to navigate through un-named menus on the instrument. Whilst usable this requires that the user become familiar with a menu tree that has little visual guide and without the names of the menus appearing and only colored lights to mark where you are or what option is active it is all too easy to choose the wrong option. In addition built in requirements such as having to reselect the instrument you are playing to exit the menu tree add un-necessary complexity. In this case the desire to pretend that the computer the instrument is plugged in to does not exist feels like a denial of the capability of the device and negates much of the goal to present an instrument that can be quickly mastered by the user. Although there is a limited computer interface provided with the Eigenharp this is mainly for choosing sounds, scale modes and VST’s and as such is more of a pre performance configuration tool than something that can be used ‘on the fly’.

I believe that the main fault with the Eigenharp model is that it binds the user to a specific predefined interface. The benefit of creating an alternative controller is that you can create an interface that combines well with your intuitive workflow and techniques. When using a powerful ‘virtual’ instrument that is linked to the computer you have the opportunity to allow the user to reprogram settings to work in a way that suits their needs. This is one of the central tenants of adaptive controller design; the end user can specify how to work with the tool that is used for interaction. For playability the Eigenharp undoubtedly succeeds in creating an instrument that can replicate the experience and sound of playing a ‘real’ instrument with all associated traits but it is in the user interface that stops if from being truly revolutionary and which does not allow the user to access the full capabilities of the instrument in a way that is complementary to their workflow.

“1st law of alternative controllers; adapt to the new reality. 2nd law of alternative controllers; adapt reality.” - Onyx Ashanti

“We feel that the physical gesture and sonic feedback are key to maintaining and extending social and instrumental traditions within a technological context” - (Bahn, Hahn et al. 2001 1)

I believe that a radical approach to instrument control systems is required to get the most from modern computers and audio software. Audio programming languages such as Max/Msp or Puredata and hardware interfaces such as the Arduino make it easy for a musician to design their own instrument and define their interaction with the computer in a way that is most appropriate to their performance. It is possible to create a “dynamic interactive system” (Cornock and Edmonds 1973) where the performer and computer feedback to each other to create ever changing interactive situations. It has become simple to create a system whereby the action of different sensors is easily assignable and changeable from within the software, and it is this flexibility and almost unlimited expandability that makes these tools suitable for creating a truly futuristic control system.

---

^[1] see Appendix A 1.1 for picture

^[2] see Appendix A 2.1 for picture

^[3] Vibrato, Pitch Bend, Slides, etc....