Notes & Lines discusses the
intersection of math, science, and technology with performing and visual arts.
Topics include bizarre instruments, technically-minded musicians, and cross-pollination of science and art.
The musical world lost an icon on January 10. David Robert Jones, a.k.a.
David Bowie, passed away from liver cancer, two days after his 69th
birthday coincided with the release of his 25th studio album.
Needless to say, artists from all over will mourn his
passing, as so
many already have. Yet David Bowie also helped shape our technologies in
both overt and subtle ways.
Let's begin with his early career. After almost 10 years of
seeking fame as the lead singer of fledgling London blues and rock bands in the
1960s, Bowie began to promote himself as a solo artist. He found some initial
success, and in 1969 published Space Oddity, a five-minute-long track that he
wrote after watching Stanley Kubrick's 1968 sci-fi film, 2001: a Space Odyssey.
The film wasn't an immediate success, but nonetheless influenced many from the
first space generation, among them Bowie.
Yet Bowie earned his stripes as a true technologist with his
love of computers and the internet. He encouraged fans to cut fan videos for
his 1994 single Jump by including software in the package. In 1997 he cybercast
one of his concerts, even if most internet speeds meant most people
couldn't watch. In 1998, Bowie was the first artist to use the web to
distribute his work when he
offered the single Telling Lies for download from his official website,
accompanied by an online chat session. Two years later, Bowie would start
his own internet service provider, BowieNet. For $20 a month, U.S. and U.K.
residents would get online access, an @DavidBowie.com mailing address, a homepage
that they could build and customize, as well as exclusive media and chats with
Bowie and other artists. In a 2000 interview with FACT Magazine, Bowie
described his vision for an internet where the artist is "demystified" to the
audience by platforms on the internet-akin to modern social media. BowieNet
lived until 2012 and most of its content was lost by its shutter.
As an entertainer, Bowie was extravagant and garish. He
challenged preconceptions and wasn't afraid to take risks. It turns out these
attributes also help entrepreneurs in the tech industry.
Those unfortunate enough to visit most areas of Buffalo will
likely notice its other defining feature: the Great Lakes dialect, or Inland
North English. This accent is common to cities like Buffalo, Rochester,
Cleveland, Chicago, and Detroit; most Americans are familiar with it through
SNL's exaggerated "Bill
Swerski" sketches. But linguists believe that the Inland North accent is
less a punchline and more an innovative breeding ground for a fundamental shift
in how American English is spoken.
The dialect's defining feature, and one that's immediately
noticeable, is the raising, tensing, and lengthening (sometimes referred to as
"flattening") of the short a. This
effect makes words like cat and that sound like kyet and thyet. The
dialect has lots of other defining features, too, like the "working-class" reduction
of th- to t- or d- (hence, "da
Bears" and "da Cubs"), using "pop" for soda, and saying "teeter-totter" to
refer to a seesaw. But a group of linguists led by William Labov have taken an
interest in the Great Lakes vowels and now believe that their flattened
qualities have started a major shift in American pronunciation.
As late as fifty years ago, Great Lakes pronunciation was
the broadcast standard for General American English, but it's been radically
diverging. Phonetic changes have a tendency to create a sort of domino effect
as they spread, so as people in the Great Lakes cities begin to speak with
tensed a sounds, their other vowel
pronunciations shift as well due to the simple lack of possible vowel sounds.
This is why those in Toledo and Buffalo commute to their jabs, not jobs, and in
many cities but is starting to sound
like bought and bet like but. Linguists
refer to this "Northern
Cities vowel shift" as a chain shift, proving that in linguistics, as in
many other disciplines, changes don't occur in a vacuum: every phonetic change
elicits a corresponding reaction. In the case of the Northern Cities shift, the
final link in the chain would be the shift from bit to bet, although it
appears this one hasn't yet taken effect.
Researchers are making a big deal over this phenomenon
because major vowel shifts are few and far between in the history of the
English language. The last major change, sometimes known as the Great Vowel
Shift, occurred between 1350 and 1700 and involved the evolution of our
currently known long vowels. This shift also bridged the divide between Middle
English and our modern language, so it's kind of a big deal. What's more,
English short vowels have remained relatively stable since Anglo-Saxon times;
if the Northern Cities shift continues to spread and topple other vowel sounds,
it could be the biggest change to English in 1000 years.
Sometimes speaking to a blue-blooded Chicagoan or Detroiter
is the best way to realize the radical nature of the shift. Asking them to
identify the "busses with the antennas on top," for example, you'd hear bosses with the antennas on tap and
could easily misunderstand the meaning. I never noticed the accent until I
moved 250 miles away, and according to Labov this is standard procedure: almost
all Great Lakes speakers insist they speak General American English. Now if
you'll excuse me, my kyet is
filthy...I should problem give him a byeth.
Inventors and other persons of genius have long had a
reputation for being a bit eccentric. The "mad scientist" stereotype brings to
mind the fictional Henry Jekyll, Julius Kelp, and Emmett Brown, as well as
real-life eccentrics such as Einstein, Edison, and Tesla. And for all these
well-known social outliers, it's likely that hundreds of their peers were completely
derided as raving madmen for sharing ideas that bucked contemporary paradigms.
The history of music has seen its share of eccentrics as
well. While the music of Stravinsky and other outliers is now commonly accepted
by public ears, the short-lived musical career of early-20th-century
painter and amateur musician Luigi Russolo is remembered as one of the most
radical of all time.
Russolo was a key figure in the avant-garde Futurist
artistic movement, which emphasized industrialization, speed, and the triumph
of technology over nature. Futurism arose in Italy and exerted an influence on
that country's art and architecture; it also spread to Russian literature and
art as well. One gets a pretty good idea of Futurism by looking at even a few
Futurist paintings: they're splashy, colorful, and seem to be moving fast
despite being static on the canvas.
Russolo and the handful of other Futurist composers felt
that traditional music as they knew it had run its course, and the newly
industrialized world was craving a, well, more industrialized form of music. In
1913 he wrote a manifesto entitled The Art of Noises, in which he
posited that 20th-century musicians "find far more enjoyment in the
combination of the noises of trams, backfiring motors, carriages, and bawling
crowds than in rehearsing [Beethoven]." In the same text he proposed
re-classifying musical sounds into six different categories:
*Group 6: voices of animals and men, including shouts,
screams, groans, howls, etc.
Realizing that contemporary "normal" instruments were
incapable of producing such sounds, Russolo invented a group of instruments he
called intonarumori, Italian for
"noise makers." These were relatively simple boxes containing a crank- or
electric-motor-driven wheel that vibrated a string made of catgut or metal.
Many models were equipped with a lever that tautened or loosened the string,
changing the "pitch" of the noise. Despite the fact that each box also had a
metal speaker or some other type of resonator, the intonarumori were relatively quiet acoustic instruments.
Russolo's vision of noise music was roundly dismissed by the
established music scene, and most of the original intonarumori were either misplaced or destroyed during the 1940 bombing
of Paris. Futurist music more or less began and ended with Russolo, and the
entire Futurist movement had died out by the end of World War II. Russolo's non-traditional
views on noise influenced other composers, however. The Swiss composer Arthur
Honegger, for example, cited Russolo as an influence on his relatively
well-known 1923 orchestral work Pacific
231, which uses traditional instruments to simulate the sounds of a steam
locomotive. And while most of Russolo's original instruments are lost, some modern avant-garde
musical groups have reconstructed the intonarumori
using original design sketches and occasionally perform Russolo's music.
Apart from these niches, though, Russolo and his
"futuristic" music--however strange and un-musical--remain forgotten by modern
A few weeks ago I bought my first guitar--which I need for a
new part-time music gig-- in about 15 years. As I removed its packaging I
thought about how simple most musical instruments are-for example, the guitar
and most other stringed instruments are just a set of tensioned cords suspended
between a nut and bridge. Stopping them with your fingers shortens the string's
vibrating portion and changes the pitch, and the instrument's body naturally
amplifies the vibrations caused by plucking or bowing.
My gig involves a lot of strumming and singing (folksy
stuff), so I had to order a capo along with my guitar. Most seasoned guitarists
are familiar with these little devices: they're used to raise the pitch of the
entire guitar to play in many different keys using the same basic chord shapes.
My capo's packaging spouted on and on about how it's "precision engineered" and
meticulously designed, so I got to wondering about its history and design.
In a technical sense a capo is really an artificial nut (the
little white notched thing close to the headstock) that's clamped to different
points on the neck to change the vibrating length of all the strings at once.
Because the capo can't be placed any farther up than the guitar's actual nut, it
can only raise, not lower, the pitch from standard tuning. Capos are most
frequently used in folk and folk-influenced music, and might have become popular
in these styles because folk musicians were keen on the idea of using familiar
chords in more difficult keys that better suited their voices.
All designs since the first capo came about in the mid-1700s
involve some type of thin bar that depresses the strings at a preselected place
and remains there by clamping onto the guitar's neck. The first device was a
simple brass clip that relied on its own tension to grip the neck, but this
quickly fell out of favor because the clip back scratched the heck of the neck
itself. A few different late-18th century designs that maintained
tension with a screw in conjunction with a piece of wood (cejilla-style) or metal (yoke-style) solved this problem.
Three of the many capo designs (left to right): Ashborn's 1850 patent | Dunlop trigger | Planet Waves ratchet
Capo engineering really took off after the first capo patent
was issued in the US in 1850, and dozens of designs have emerged since.
Most of these relied on a spring or some type of screw until the development of
the now-common trigger capo in 1979. This one uses a strong integral spring
along with a simple clamp design (supposedly inspired by a typical clothespin)
that facilitates quick one-handed changes to its position. The trigger model
and a 1978 Shubb design--which uses a screw to custom-adjust the clamp to the
instrument's neck width--seem to be the most popular modern devices.
A few strange capos also cropped up in the '80s. The Third Hand "partial capo" from 1980 consists of six individual "fingers" that can
clamp any combination of strings, rather than all of them at once. And the
ratchet capo, patented in 1989, works like a cable tie for the guitar's neck,
with a lever-release for quickly changing the position.
For an instrument that's barely changed over the course of 300
years, who knew its little $15 accessory could be so complicated?
(And for those who crave more capo design info, Sterner's Capo Museum has info and images for 243 different capo types.)
Image credits: Creative Tools / CC BY 2.0 | Sterner
Artificially intelligent systems are capable of many
human-like behaviors. At least on a surface-level, they can think, learn,
speak, read, and simulate emotions. The convergence of human and machine
behavior has led to the well-worn ideas of the Singularity and possible machine
Those uncomfortable sharing their headspace with a machine
purport that computers will likely never be capable of one of our most human
traits: creativity. Creativity seems random, messy, and subjective, three attributes
not easily grasped by computers. Creative AI research, while interesting and
promising, has provided more questions than answers; namely, "What is
creativity, and who's capable of it?"
Creativity is "the ability to transcend traditional ideas,
rules, patterns, relationships, or the like, and to create meaningful new
ideas, forms, methods, [and] interpretations," according to Random House
Dictionary. Given that computers essentially analyze fairly standard rules,
patterns, and relationships, a truly creative one seems unlikely. Creative
works are often described as "novel" and somehow valuable or useful; without
direct human input, this is a monumental challenge for AI.
These hurdles haven't stopped researchers from trying,
though. Lior Shamir, a computer science professor at Lawrence Technological
University, developed algorithms that identified
similarities between works of Jackson Pollock and Van Gogh and correctly
ordered all thirteen Beatles LPs chronologically based on audio and visual
samples. Because computers aren't adept at handling discrete pixel and
frequency data, Shamir converted each visual or audio sample into thousands of
numerical values that were fed into a pattern recognition system. Shamir's
music understanding programs are based on an earlier project that analyzed
15,000 whale songs to identify that whales communicate using different dialects
depending on their geographical origins.
While analyzing creative works might seem much simpler than
actually producing them, automated painting and composing software programs
have been used for decades. AARON,
a program created by Harold Cohen in the mid-'70s,
is hand-coded to produce artworks (like the one below) in a specific style. Even Cohen is quick to
point out that AARON isn't creative, though: it simply follows procedures
outlined by its programmer, who is the true artist. Returning to the definition
of creativity, AARON follows the rules rather than transcending them. More complex painting
robots are becoming common, although these still seem to be merely going
through the motions.
If true artistic creativity seems outside the reach of AI,
what about the experimental kind? Assuming Pasteur's assertion that "in the
experimental fields, chance favors the prepared mind," a data-loaded processor
would be much more likely to stumble upon a scientific breakthrough than a
messy human brain.
AI research seems peculiarly unconcerned with the philosophy
of the work, with practical solutions trumping abstract ideas: an application
that works will naturally form the basis for an abstract theory. Many in the
field point to aviation as a good example: it took building a working flying
machine to answer the century-long debate asking "Can we, and should we, fly?"
So is there a useful application for creative AI? Music
recommendation engines used by iTunes and streaming music services would
certainly benefit from creativity breakthroughs. Although in a subjective
sense, I wouldn't find an hour of identical-sounding music very appealing.
Pocket-sized instruments have always appealed to musicians. Pulling
out a harmonica, xaphoon,
or ocarina and
playing on the go seems like the pinnacle of convenience for the musically
minded. But imagine the ability to transform any surface you come across into a
musical instrument, using only your smartphone and a small device named after a
Last year, four RPI students formed startup ToneTree and began work on such a device, which
they've dubbed Birch. Birch consists of a very-high-frame-rate infrared camera
that allows users to draw a sonic interface on a table or other flat surface. It's
technologically similar to an infrared QWERTY keyboard, albeit with a much
faster camera for real-time sound production.
ToneTree co-founder and CEO Brian Cook, who's pursuing an
MFA in Integrated Electronic Arts, has been using his interest in music and
technology to come up with innovative instruments since his undergrad days. At
the University of Hartford he wrote a piece for the commemoration of a new
building and invented a wired mallet device with integrated sensors and
switches as well as remote cameras to enable performers to "play" the new
building. Cook and his three ToneTree compatriots started work on the Birch
prototype after he realized that doing away with the wires and creating a
free-form instrument would heighten the experience.
The new instrument allows users to draw shapes on a flat
surface and program them to correspond with waveforms like notes or percussive
sounds. (Check out this demo.) This free-form interface makes Birch's expressive possibilities almost
endless. The team has also talked about developing mats printed with a piano
keyboard or guitar fretboard in the future.
The group has made rapid advances on Birch since undertaking
the prototype in November 2014. They've just recently achieved data processing
fast enough to eliminate most of the lag between finger movement and sound
production and are still experimenting with faster optics. Birch renders a
user's table or wall touch-sensitive, like a piano, by tracking finger velocity
and producing tones that are proportionately forceful or gentle. The backend is
powered by a combination of OpenCV for computer vision, MIDI for the musical
portion, and Qt for the user interface.
The Birch is a patent-pending design and will be sold commercially
in the coming year. Cook and software designer Ronald Sardarian discussed
expanding research to a future device that would enable users to transform any
flat surface into a touchscreen using a simple mobile device. It might be a
useful addition to other promising augmented reality (AR) technologies such as
Classroom application of Birch seems like a natural
progression, and the inventors are already partnering with local school STEAM
(STEM+the arts) programs. Thinking back to my own school days, I imagine that
most would rather play with this neat new technology than study the recorder. And
if its future AR applications allow tapping walls rather than waving my hands in the air,
well, sign me up for that too.
"Twenty years from now you will be more disappointed by the things that you didn't do than by the ones you did do. So throw off the bowlines. Sail away from the safe harbor. Catch the trade winds in your sails. Explore. Dream. Discover." -- Mark Twain