Abstract
Human feelings, thoughts and behavior are all part of the human emotion that plays an important role in interpersonal human communication. Brainwaves-Based Emotion Estimation is one of the most common used and efficient methods for emotion estimation research. Biosignals are space-time records of biological events such as a contracting muscle, or a heartbeat. During these biological events, the chemical, electrical, and mechanical activity produces signals that can be analyzed and measured.
Biosensor technology reveals a significant role for brain computer interface (BCI) for entertainment, disabilities and other various researches. These neural and muscular activities create measurable electrical signals in the human body. In medical diagnostics, measuring and analyzing these signals is common practice, as is inducing muscular activity through electrical signals. In this project, I will digitize neural and muscular activities
______________________________________________________
Interested in idea of producing multimedia that humans produce besides the regular process for creativity and accessibility purposes, using biosignaling
We’re all biosignal – idea of being exploiting it for specific tasks, it goes back to when biosignal has been compromiosed, need for prostehtics (glasses),
People whove been compromised havnet been allowed to exert themselves creatively
(the perfect dancer, the perfect painter) creative expressions has always been compromised by biosignals, video editing is a vehicle for getting recorded media out,
Eloquence of using hands, and high level of dexterity to edit digital content
“Creativity as extension of humanity”
If you don’t have your hands how to do all this? How do you reach this high level of dexterity?
Incredible progress for artificial limbs, and want to open up multimedia authoring and editing for people who are compromised
Open potential to creative potential
Aesthetics, is it a untapped new area?
Whos worked in bio signal?
Whos used biosignaling for creative use, for sound?
What’s the quality of that? Is it a new aesthetic? Can it conform to the rigor of the current video editing or is it purely experimental?
Extending the dialogue of with smarter and smarter prosthetics, someones gonna want to return the normal life with a high level dexterity, dexterity = (bio)signals
Want to first do experimental media with biosignals so that we can see where it leads to and how it can be used for “regular” media production.
Complexity and emergence –
In the description you want to talk about systems, potential outcomes for some emergence in outcome…
Humans are complex system
Trying to find one signal which you can use as an example to interpret the outcome of it and create(?)
Taking any biosignal, be more specific, have a certain parameter, complex biosignal and enable me to manipulate it in a simple system then into a complex system
Intuition is emergent, video editing is emergent, like choosing a certain type of aesthetic is emergent
Signal processing of biological data implies many different interesting problems dealing with signal acquisition, sampling, and quantization. The noise reduction and similar problems as image enhancement are a fundamental step in order to avoid significant errors in the analysis of data. Feature extraction is the most important part of the analysis of biological signals because of the importance which is clinically given to even the smallest singularity of the image (signal).
The objective of our work is to develop new methods of signal and image processing allowing quantitation and robust qualification based on such data.
Sharing visual is considered part of a communication medium, biosignal sharing can be considererd a new communication medium
For example, newspapers deliver their messages by the modality text, and videoconference uses the modalities audio and video. In the conceptualization of biosignal sharing as a new communication medium, one can consider the biosignals themselves as a new modality.
The types of modalities that are used in a message differ across different media, and the selected modalities determine which verbal and non-verbal cues are being exchanged, or in other words, which (social) information is available in social interaction.
“Beauty and compelling than we can normally create” - Timothy
Since this interface relies on the biosignals originating from the contraction of muscles on the face during particular movements, even individuals with a paralyzed limb can use it with ease.
New accessibility solution for creative expression to increase the inclusion of user with disabilities
The technology reveals a great role for human emotional disorder treatment, brain computer interface for disabilities, entertainment and many other research areas.
Thesis
By utilizing biosensing, I will digitize emotional changes through biosignal analysis to discover new ways to approach multimedia composition in real-time and access a novel visual spectrum to explore alternatives to traditional video/audio editing techniques.
This project will attempt to open new ways to incorporate live content creation for artists to create easily accessible collaboration between them and front of house VJ to improve the disconnect between the DJ and the VJ. (OSC)
Introduction/Description (expanded thesis statement)
Biological signals, or biosignals, are space, time, or space–time records of a biological event such as a beating heart or a contracting muscle. The electrical, chemical, and mechanical activity that occurs during these biological event often produces signals that can be measured and analyzed. Biosignals, therefore, contain useful information that can be used to understand the underlying physiological mechanisms of a specific biological event or system, and which may be useful for medical diagnosis. Biological signals can be acquired in a variety of ways (e.g., by a physician who uses a stethoscope to listen to a patient's heart sounds or with the aid of technologically advanced biomedical instruments). Following data acquisition, biological signals are analyzed in order to retrieve useful information. Basic methods of signal analysis (e.g., amplification, filtering, digitization, processing, and storage) can be applied to many biological signals. These techniques are generally accomplished with simple electronic circuits or with digital computers. In addition to these common procedures, sophisticated digital processing methods are quite common and can significantly improve the quality of the retrieved data. These include signal averaging, wavelet analysis, and artificial intelligence techniques.
Luigi Galvani (1786) “link between muscle activity and electricity in his experiments on dead frogs”
1791 about electricity as a vital force of life.
Since 1960s.
Marco Donnarumma (2007-2010)
Stelarc
End of Fall 2022
Literature Review/prior art
Project Scope & timeline
(Could refer to work that I’ve done that drew me into the topic)
Materials & Methods
Preliminary studies
Tools/methods
EEG, EMG, Adobe, Processing(?), Python(?), TouchDesigner(?)
software you’ve used…
Tools that you’ve used, interacting with existing software and letting know what you are doing beyond what the software lets you do
What version of the software you used…
Development
Intended outcomes
ramification of the future of this project
How ill show this project
Potential dissemination
References
By utilizing Electroencephalography (EEG) and/or electromyography sensor (EMG), we will digitize emotional changes through brainwave (biosensing)biosignal analysis to discover new ways to approach multimedia composition in real-time and access a new visual spectrum to explore traditional video/audio editing techniques. (This project will attempt to open new ways to incorporate live content creation for artists to create easily accessible collaboration between them and front of house VJ to improve the disconnect between the DJ and the VJ.) This experiment will also explore accessibility for people with disabilities to dial into their creative potential of storytelling and editing that do not generally have access to tangible skills of editing.
We will use biosensors like, Electroencephalography (EEG)/ electromyography (EMG), to produce brainwave/muscle analysis data to generate new approaches to multimedia composition that would access new patterns of video/audio editing techniques. The tangible skills of multimedia composition will be questioned and possibly support assistive technologies to broaden accessibilities to visual/audio editing/composition for the disabled.
By utilizing Electroencephalography (EEG)/ electromyography sensor (EMG), produce data from biosensors to discover new ways to approach multimedia composition in real-time and access a new visual spectrum that hasn’t been explored in traditional video/audio editing techniques. We will focus on alpha waves and the attention data to discover hidden signals that occur while we edit to uncover new possibilities of post-production. This experiment will also explorer accessibility for people with disabilities to take on storytelling and editing tasks of video/audio.
We will use biosensing to create data based on neurofeedback received while editing. This brainwave/muscle sound analysis to discover new ways of creating visual narratives. Explore accessibilities for disabled people to create
Thesis
By utilizing biosensing, I will digitize emotional changes through biosignal analysis to discover new ways to approach multimedia composition in real-time and access a novel visual spectrum to explore alternatives to traditional video/audio editing techniques.
This project will attempt to open new ways to incorporate live content creation for artists to create easily accessible collaboration between them and front of house VJ to improve the disconnect between the DJ and the VJ. (OSC)
This experiment will also explore accessibility for people with disabilities to dial into their creative potential of storytelling and editing that do not have access to tangible skills of editing.
Description or abstract -> collaborative use and studio use
Analog -> digitize (Arduino) -> td
Machine learning model as you think about different things, machine learning models shifts visual
Machine learning tools that now you could basically hook the sensor up, train it with whole bunch of people, hit a button to show the machine this person is happy, and machine would know what happy would look like,
Noise of muscles flexed and stretched, motion and intention
Equipment on arm that picks up and Bluetooth it
Different kinds of biosensors (Research this)
How easy is it to connected to MaxMSP or TouchDesigner
Has anyone made a converter tools? Or is it something I have to write
Facial expression to catch emotions – facial emotion detection on machine learning
During the drop – are they feeling it or performing it?
Consumer accessible?
Brainwave – relaxed, focused
That ring picks up position
Heartbeat orchestra – look them up
Asian lady that does mediation to make bowels sing
stelarc
spirit photography? How film composition came to be in this weird style
night science vs day science
_________________________________________________________________________
Abstract
Human feelings, thoughts and behavior are all part of the human emotion that plays an important role in interpersonal human communication. Brainwaves-Based Emotion Estimation is one of the most common used and efficient methods for emotion estimation research. Biosignals are space-time records of biological events such as a contracting muscle, or a heartbeat. During these biological events, the chemical, electrical, and mechanical activity produces signals that can be analyzed and measured.
Biosensor technology reveals a significant role for brain computer interface (BCI) for entertainment, disabilities and other various researches. These neural and muscular activities create measurable electrical signals in the human body. In medical diagnostics, measuring and analyzing these signals is common practice, as is inducing muscular activity through electrical signals. In this project, I will digitize neural and muscular activities through biosignal analysis to discover new ways to approach multimedia composition in real-time and access a novel visual spectrum to explore alternatives to tradition video and audio editing techniques. These alternative methods could be difficult when compared to traditional storyline editing, thus provide a stronger case for the unknown experimental aesthetics. This project will attempt to open new ways to incorporate live content creation for artists to create easily accessible collaboration between them and FOH VJ, which would improve the disconnect between the two parties. This experiment will also explore accessibility solution for creative expression to increase the inclusion of users with disabilities to have access to producing multimedia that wee make besides the regular tangible process for creativity.
Prehistories and Imaginary media
- before brainwave happened
imaginary media – back in 1990s brainwave games, imagine an older internet version of super brainwave sensors – something from the past – technology from the past that could be imagined in different ways
Biosensing technology
Watch prosthetics documentary and letting people interact without hands.
Idea of light organ – (composer starts with a S) – synthesizer connection to brain
Prehistories and Imaginary media
Throughout history humans have created prosthetic devices to enhance the basic qualities, functions, and independence of human life. Prothesis generally aid, replace, and correct a part of the body that is either missing or to help make it work better. It could aid a weak part of the body, replace a deformed part of the body, or correct a physiological malfunction of the body. Prosthetic devices are there to improve the quality of life and restore the users to their sense of completeness. Some of these devices are crucial in restoring and improving the quality of life by replacing or supporting people who have endured the loss of limbs or tissues from diseases, trauma or a condition from birth. There are other prosthetic devices that we generally don’t think about, which are things like glasses, braces, and hearing aids. The history of prosthetics isn’t just about a story of medical advancements in the field of science and technology, but also a story of human beings suffering through hardships to retake part of their missing self and to empower their “sense of wholeness”.
About 3000 years ago, the history of the first prosthetic device was discovered to be an Egyptian noblewoman’s toe (Bender). This is an interesting story because one would expect the first piece of prosthetic device to be something more important like an arm or a leg. In that era in the Egyptian society, the big toe had a significant meaning because it allowed the person to wear the traditional Egyptian sandals. Although the function of a toe is important to the human body, this prosthetic device displayed that the toe represented the person’s identity as much as the function it provided. The prothesis helped complete the woman’s sense of wholeness, but it also helped her represent herself as an Egyptian. From ancient history of prosthetics, it shows that prosthesis not only provided an important function to the human body, but highly valued the identity it could provide to the user.
In the times of ancient Rome, of the Second Punic War, marks the next big revolution in prosthetics. General Marcus Sergius suffered the loss of his right hand during battle, which was crucial in holding a sword and a shield. Unable to hold the two tools, he was unequipped for battle. He became the first person to be given a prosthetic limb, which was an iron hand that was fashioned to fit him to hold his sword and shield. The prothesis allowed the general to keep his identity as a general. A prothesis was a practical tool that supported a specific function, but also a cultural and personal tool that allowed to people to retain their identity (Bender).
Creativity has always been an extension of humanity. One might identify themselves with the creative work that they output, and it creates a sense of identity for individuals.
The identity that prothesis provided
“Biosignal is any signal in living beings that can be continually measured and monitored.” Humans are biosignals; every little movement, thought, and sound that we make are all biosignals. Since biosignals also originate “from even the contraction of muscles on the face during particular movements, even individuals with paralyzed limb can use it with ease.”
Humans are complex systems as we are full of biosignals
In the 1990s, a company called The Other 90% Technologies had invented a product called the MindDrive and it was marketed as a device that could command machines by thinking.
Bibliography
Liu, Jingxin. “Brainwave-Based Human Emotion Estimation Using Deep Neural Network Models for Biofeedback.” Brunel University Research Archive: Home, Brunel University London, 1 Jan. 1970, http://bura.brunel.ac.uk/handle/2438/18141.
Bender, E. (2015, September 21). The history of prosthetics. UNYQ The History of Prosthetics Comments. Retrieved October 10, 2021, from https://unyq.com/the-history-of-prosthetics/.