Thursday, 14 July 2011

Updates and more dissertation

Apologies for not updating this for a while, ive been moving house, to Berlin! Also ive been finishing an EP to be released on Planet Terror.

So without further ado, the next section of my dissertation...........

Hardware Design

“Any sufficiently advanced technology is indistinguishable from magic.” - Arthur C Clarke

“One way one can attempt their adepthood in magic is to try weaving a spell without using any of the prescribed tools. Just quiet the mind and slip off into a space within your mind that belongs only to you. Cast your will forward into the universe and see if you get the desired results.” (Nicht 2001 47)

When designing my controller I looked at the idea of openhanded magic as a source of inspiration. Rather than being directly related to card tricks and illusion open handed magic is a form of magic in modern occult systems whereby the practitioner does not use tradition ritual props but uses the focus of their will in the moment to achieve the intended results. The performer must achieve some sense of gnosis and ‘at-one-ness’ for this to succeed and as we have previously explored dancing is one route to this state. As explained by Joshua Wetzel:

“Dancing This method could also be termed “exhaustion gnosis.” The magician engages in continuous movement until a trance-like state of gnosis occurs. Dance gnosis is particularly good for visions and divinatory sorts of workings, or at least that is the history of its use. However, it is apparent how it could be used in any type of magical activity. The effort to maintain continuous motion eventually forces the mind to a single point of concentration, the motions themselves become automatic and there is a feeling of disassociation from the mind. It is at this point that the magician performs rituals, fire sigils and various other magical acts. This is also a great form of “open handed magic.” You can do it in a club full of people, with dozens watching, and no one has a clue.” (Wetzel 2006 21)

As discussed earlier I feel that the dance floor has a strong ritual and tribal element associated with it and I believe that these ideas can be incorporated into the design and usage of an adaptive controller system. If the ultimate aim of the design is to interact with the audience and the “blurring of once clear demarcations between himself and the crowd, between herself and the rave” then it is possible to incorporate the ideas of ritual and ritual magick to inform the creation of your controller. Although the idea of creating something ‘magic’ is certainly in one sense that it should ‘wow’ the audience and create something novel and exciting to draw them into the performance I believe that for the performer/programmer the idea must become more abstracted. If we refer back to the earlier idea of having the space within a performance to have moments of inspiration and the room to experiment, take risks and also possibly fail and couple this with the intended purpose of the music we are focusing on, to make people dance, then surely the optimal state for creation of this is to be in the trance like state of the dancer. In the previous section I asked the question “Would they (the performer) not be more fully immersed in their own sonic landscapes if unshackled from the computer screen and became free to roam the space their sound occupies, interacting with the audience and using their whole body to feel their performance in the way the audience does?” and I believe the answer to this is to allow the performer a system of control that allows them to largely forget the mechanism of creation and to ‘feel’ what they are making by being in the same state as the dancer themselves. When looking at how to design my controller I have tried to think about this question throughout and use it as a reference when trying to ascertain the best way to incorporate a feature into the hardware and software design. The controller must be simple to use, requiring natural hand gestures, and notes must be easy to trigger and record so that the flow of the performer is not interrupted by the technology. It has taken a great amount of trial and error to reach a stage where this is possible and indeed the use and design of a controller to allow such interaction with audience and music is, by necessity, in a constant state of flux where new ideas can always be incorporated and refined to move towards the optimal playing experience. As I have previously stated this idea of a continually evolving and demand responsive controller system is the optimum state for these projects and although temporary goals can be established the performer/designer should always be looking for a way to improve and advance their work and as such it can never be described as truly ‘finished’.

It is relatively easy to build your own controller system and use it to interact with a computer and there a number of advantages in creating your own system over co-opting existing computer interface devices. With a basic knowledge of electronics it is possible to create anything from a simple input device to a whole new instrument. Using an interface such as the Arduino you can simply, and with minimal processor load, send analog and digital signals to your software and there are a huge number of sensors on the market that you cannot find in a pre made solution and making your own controller allows a novel approach to the capture of data. The traditional computer controller model of interface relies on pushing buttons to input data and thus even when using a modern controller such as the Wii-mote we are still tied into this idea of physical buttons as the main input device. Other devices such as the Kinect although allowing gestural input only work under specific lighting and placement conditions which would make it largely unsuitable for use in a live performance environment. If we build our own system it is possible for us to use a vast number of different devices such as bend and pressure sensors or accelerometers to receive input. This approach allows us to fully incorporate the idea of gestures to manipulate music as it does not rely on you tapping a key but rather invites you to use your whole body. As previously stated with the controller I wished to design I did not wish to copy or model traditional instruments but rather to create a unique interface with a distinct playing experience to take advantage of the many controls available to us to manipulate. To get the most from the custom controller experience we must develop our own language to interact with computers and the music being made.

In designing a physical controller it is important to think about what you intend to use it for and what controls you need. Do you just need switches or do you need analog control values that you can use to, for example, turn a knob or move a fader? Do you want your controller to play like a traditional instrument or to have a totally non-traditional input method? With my project it was important to have a number of analog controllers as well as digital switches and also some kind of control for moving through the live interface was required, this meant that I added a touchOSC component to my project for feedback and control of Ableton’s midi map triggered features, this allows you to trigger clips and manipulate controls all without having to look at the computer. In my project only the hands contain sensors and the feet perform basic functional software control, which are also replicated on the touch screen device, allowing the performer total freedom of movement. Being free from the computer allows the performer to more fully enter into the flow of the music and to, for example, dance whilst creating. In this aspect my controller is attempting to remove itself from a more traditional model of playing music where you would have to think about the placement of an instruments, your hands on the keys and so on. As my project is particularly focused on creating electronic ‘dance’ music, which has little link to traditional instruments, it seems counter productive to produce something which models itself upon a traditional instrument as in the setting of a live performance this would look misplaced.

Rather than create a system where the user has to hold a controller my system is built entirely into a set of gloves and as such one simply has to move their hand to affect change in the music. The hardware has gone through a number of revisions to find the best setup to compliment my workflow. Initially I used available ready made sensors to create my gloves, and whilst these made for a relatively simple construction they presented a serious set of problems regarding connections to the gloves, keeping the sensors in place and not putting stress on the weak points of their construction. Many commercially available sensors are designed to be used in a static setup where once mounted they are not moved, however when making something such as a pair of gloves it must be recognized that there will be a large amount of movement and that actions as simple as putting on or removing the gloves may produce unwanted stress on connections that may break or impair the functionality of the system.
Over the development time of my project technology has become available that allows you to make bend sensors, pressure sensors and switches out of conductive material. This creates a distinct advantage over traditional sensors as they are more durable, easier to wear and very simple to fix and replace. Conductive thread has, in theory, made it possible to create a controller with less physical wiring, the wires can be sown into controller, are flexible and do not restrict movement and are more comfortable for the user. I initially remade my project using this technology, however this technology also has drawbacks that only become apparent after a period of usage and have meant that it was unsuitable for this project. A prototype version of the gloves were made using conductive thread rather than wiring and although this initially worked it was found that stretching and compressing the thread in a vertical direction lead to it unraveling. As the wire functions in the same way as a multicore wire when the thread is not tightly wound together you get a loss of signal, initially I sought to counter this problem by covering the conductive thread in latex but as this seeped between the strands of the thread this also lead to a loss of signal. This conductive thread technology is certainly useful in some situations however when used on a pair of gloves the amount of stretching required to get them on and off means that the thread breaks very quickly. However it is still used in the project to connect between circuit boards and the conductive fabric fingertips of the gloves and between circuit boards and the analog sensors in places where there is not a great amount of stress placed on them.

The analogue sensors are also made from conductive material and this has the advantage of making the sensors easily replaceable if broken and easy to fine-tune the output values and sensitivity. The bend sensors on the fingers are made using conductive thread, conductive fabric, velostat and neoprene. By sewing conductive thread into two pieces of material and sandwiching layers of velostat between them you can easily create a sensor which is simple to adjust the sensitivity of as this is determined by the number of layers of velostat between the conductive thread, a sensor made this way also has the advantage that it can easily be mounted on the gloves via stitching. These sensors also can be made to look almost any way you desire, in the case of my project simple black circles, and as such they are in keeping with the idea of open handed magic where the actual method is partially obscured from the audience but easy to use and understand for the performer. The switches in the gloves are also made in a way that removes the need for any wiring, electronics or unwieldy physical switches. Using conductive thread it is possible to create a switch that can be closed by applying a voltage across it and this greatly simplifies the construction of the gloves as only one positive terminal is needed, in this case placed on the thumb, thus the switches are constructed by wiring a ground and input wire to each finger and are closed by touching the finger and thumb together. This natural gesture requires no learning on the part of the user and we can, for example, use each switch to trigger a drum hit or play a key on a synthesizer as well as performing more command based functions if required. I have taken the approach of making the switches on both hands produce midi notes (one for each whole tone in an octave with an extra c of the octave above on the last finger of the right hand and a foot pedal to sharpen/flatten the notes) as this yields the most natural playing experience, but it is possible to program these switches to provide other controls is required.
My controllers use accelerometers in each hand to work out the position of your hands, this allows us to seamlessly change between parameters being controlled. For example if your right hand is held at a 45 degree angle the accelerometer can function to control a cut off filter within your music software, however if you tilt the right hand further to 90 degrees the functionality of the left hand can change and could instead be used to control the volume of a part or the length of a sample. As we can produce accurate results with these sensors we are able to build a huge amount of multifunctionality into a very simple control system. Positioning of the hands is very easy for the performer to feel without the need for constant visual re-assurance and this contributes to the ease of use of the system.
I have also incorporated multi colored LED’s into the hand for visual feedback, by using three color LED’s we have a huge variety of potential colors to choose from which can indicate function, and therefore we also cut down on the amount of wiring needed to manage this and space used on the glove. There are three of these LED’s mounted on the gloves, two represent feedback from the notes played and change color corresponding to the instrument chosen and the third is used as a metronome so it is easy to record sections in time with the computers tempo setting and gives the performer visual feedback for their timing.
By using Xbee radios in conjunction with the Arduino and sensors we are able to unwire ourselves from the computer completely. This of course simplifies the use of the controllers as it does not matter where the performer is in relation to the computer and for my project this is vitally important to my core idea of ‘open handed magic’ and audience interaction. The most obvious disadvantage of using wireless communication is increased complexity of setup. To get the xbees to talk to one another over a meshed wireless network is not a simple task and Arduino code that works when the unit is plugged in via USB does not necessarily work when passed over a serial radio connection. For example the Arduino2Max code, available online, is a simple piece of programming that allows the Arduino to pass results from each of its inputs to max/msp. However this does not work when Xbees are introduced as the data being reported by the serial.print function floods the buffers of the xbees and means that data is only reported once every ten seconds or so. Obviously as we are aiming for a system with as low latency as possible this situation is unacceptable and another means of passing the data must be sought. In the case of my project this meant the Firmata system which can be uploaded to the Arduino and which communicates data to the computer by the use of the OSC protocol. Although the code for this system is much more complex than Arduino2Max the results that it produces are far more accurate and do not result in any appreciable latency. However to get this to work in the way I required demands a greater level of coding knowledge for both the Arduino and Max/MSP and messages are passed to and from the serial port using more complicated OSC messages and must, for some functions, be translated into a format that max understands to create usable data. Using series 2 Xbees also creates an additional problem in that they are designed for more complex tasks than serial cable replacement, as such part of their standard behavior is to continually seek nearby nodes that they can connect and pass information to. Through extensive testing and research I have found that if this mode was utilized the stream of data from the gloves to the computer and visa-versa was often delayed by seconds at a time, as the xbees seem to prioritize data integrity over timing. However it is possible to quickly bypass this by setting the xbees to only look for a specifically addressed endpoint and this seemed to solve inconsistent timing issues. There is a distinct advantage to using the Firmata/OSC based communication and that is that if there is a dropout from the controller the flow of data will resume when the connection is restored. I.e. if the battery runs out and wireless communication is lost when a new battery is used the wireless communication is resumed and the data will also resume being seen in max/msp. This does not occur with more simple codes and therefore using this more complex system provides a level of redundancy to our hardware that allows us to continue performing without the need to reboot the computer or software.

When powering an Arduino over USB you do not need an additional power source as the USB bus can provide what is needed to run your sensors, however when using wireless you must include and external power source, this must be powerful enough to provide the correct voltage for the Arduino, wireless module and sensors and must have a long enough battery life to not run out mid performance. This obviously increases the size and weight of the controller and if you are using conductive thread it is important that the power source is placed in close proximity to the most high voltage mission critical elements of the project. This is because conductive thread has a resistance of 10 ohms per foot (i.e. one foot of conductive thread is equal to a 10 Ohm resistor) and therefore you lose power from your source the more thread is placed between it and your components. However if traditional wiring is used this becomes less of an issue. Li-Po batteries were chosen for this project due to their high power output and quick recharge time, one must be aware though that they must not be discharged under 3 volts and that if the packaging is damaged the batteries are liable to expand and potentially become unstable, therefore care must be taken to ensure that they are looked after properly when used. These batteries clearly offer the most potential for a system like this however as they allow somewhere in the range of 1000 − 3000 mAh to be output, this is more than enough to power the lilypad, xbee, sensors and lights for a long duration. Originally I had looked at using AAA batteries and although these powered the system on they ran down very quickly and with some sensors produced a voltage drop that would reset the Arduino and cause unreliable operation.

No comments: