Notes & Lines discusses the
intersection of math, science, and technology with performing and visual arts.
Topics include bizarre instruments, technically-minded musicians, and cross-pollination of science and art.
Music is a no-brainer when it comes to AI research. It has a finite set of rules, a relatively limited scale, and pretty strict limits as to what “sounds good,” at least based on a researcher’s subjectivity. The concept of musical AI is also at least 50 years old: well-known futurist Ray Kurzweil appeared with a music-writing computer of his own invention on I’ve Got a Secret in 1965. However outlandish a music-writing computer might’ve been in the 1960s, Ray unfortunately stumped only one panelist.
This summer has seen a bumper crop of headlines about musical AI. The most notable example is probably IBM’s Watson, the Jeopardy-winning supercomputer. IBM is leveraging Watson to create Watson Beat, a new app designed to boost a musician’s creativity. A user feeds the app a musical snippet and a list of desired instrument sounds, and Watson more or less remixes and alters the sample, choosing its own tempo and doing its own orchestration. Richard Daskas, a composer working on the Beat project, says the app could be helpful for a DJ or composer experiencing “musician’s block” in that it “generate[s] ideas to create something new and different.” An IBM researcher working on the project says Watson Beat should be available commercially by the end of the year.
If there’s a developing tech-related area, how can we not expect Google to be in the ring? A few months ago the tech giant released a 90-second melody created by its Magenta program. Google is using Magenta, a project they first announced in May, to apply its machine learning systems to creating art and music. Similar to Watson Beat, a user feeds Magenta a series of notes, and the program expands them into a longer, more complex sample. Magenta relies on a trained neural network, an AI technique inspired by the brain, to remix its inputs. Google’s efforts at neural networks have already tackled visual artistic development: its DeepDream algorithm was the basis for a visual gallery show early in 2016.
Recent research from Baidu, the Chinese tech giant, takes a different tack and combines AI, visual art, and music. The company’s experimental machine learning algorithms analyze patterns in visual artworks and map them to musical patterns, creating a kind of “soundtrack.” (Check out this video, in which the AI Composer tackles Van Gogh’s Starry Night and other images.) Baidu says the program first attempts to identify known objects such as animals or human faces within the image, and analyzes colors for perceived moods (red=passion, yellow=warmth, etc.). AI Composer contains a large musical library categorized by “feel,” and it draws upon these musical samples to piece together an original composition in the mood of the image.
A large grain of salt is necessary when evaluating AI developments, at least in my opinion. It’s exciting to see artificial neural networks doing their thing, but even considering the subjective nature of art and music, it’s hard to see how Watson, Magenta, or AI Composer have produced anything “good” or worth listening to. Granted, they’re all in the early stages, so who knows? Maybe we’ll see a day when composers come up with basic ideas and let computers do the rest. I for one hope that day’s far off over the horizon.
Jonathan Fuller describes how the digital organ grew from the desire of a father to give his son a decent instrument to play at home. The desire of a 21st century organ virtuoso for a virtual pipe organ, a personal instrument that would travel with him, led to the development of the International Touring Organ, perhaps the most sophisticated application of digital organ technology – and organ-building ingenuity – to come to concert venues yet. The ITO story, like that of the Allen digital organ, combines the vision of a performer with pioneering technology to produce a stupendous, if sometimes controversial, musical instrument.
Traditional pipe organ. Image credit: Wikimedia Commons.
Cameron Carpenter, a prodigiously-talented, Julliard-educated organist, is out to change the conception that pipe organs are just for sacred music (with exceptions for baseball parks and movie theaters). His repertoire ranges from Bach to Bernstein to Leonard Cohen. Carpenter began campaigning for a portable pipe organ ten years or so ago. "Unlike a violinist who plays at 28 the instrument they played at eight, I have to go every night and play an instrument I don't know," he said. “What one doesn't see is the countless hours of rehearsals which are spent, not on the music, they're spent on (the organ).” Pipe organs aren’t portable. And they are intricately complex; no two are alike, and a performer typically needs hours to get acquainted with each instrument prior to a performance. Moreover, an instrument fixed in place limits concerts to locations with organs. Carpenter understood that applying the digital technology powering smaller organs to instruments with a grander scale would give him the portable instrument he craved.
The organ builders
Marshall and Ogletree Organ Builders (M and O) is the brainchild of two highly-respected concert organists, Douglas Marshall and David Ogletree. Founded in 2002, the company’s purpose is to build digital organs that meet – or surpass – the sound of the greatest pipe organs in the world. Since its inception the company has built a dozen organs. Each organ’s sounds are based on sound samples from pipe organs, using M and O’s patented process based on the one described in Fuller’s article. The company works in close cooperation with each instrument’s commissioners. Marshall and Ogletree undertook a daring proof of concept: the company built an electronic organ based solely on the sound of one pipe organ, an 1897 George Hutchings organ at Boston's Basilica of Our Lady of Perpetual Help. “Blind” comparisons of the sound of the original organ with the sound of the M and O organ demonstrated the fidelity of the new instrument to the original. Follow the link to do your own comparison.
In 2013, Carpenter commissioned M and O to build his touring organ, an instrument equal in sound to the finest pipe organs in the world that is transportable to concert venues. To create the sounds the artist wanted, M and O synthesized sounds from Carpenter’s favorite organs worldwide, including at least one Wurlitzer and organs he knew from his childhood.
The complexity of this commission goes beyond creating the organ sound; M and O developed the sound system and all of the internal components that make this organ work. Eighteen cases of components fill the truck that accompanies Carpenter to his concerts:
The six-manual console
Ten cases of speakers
Eight cases of subwoofers
A supercomputer/amplifier system
Supercomputers – three of them – with extremely fast processing speed manage the conversation between the organ’s manuals and the stored sounds. The flexible sound system can operate on two channels, for a set of headphones, up through 48 channels to fill a large concert venue.
The ingenuity involved in making this whole system portable is just about as impressive as engineering the digital organ sounds. The console breaks down into six pieces. The speakers, subwoofers, and console pieces fit into purpose-designed cases that protect them from the rigors of travel. According to M and O, “the entire organ, including console, racks and 48 channel audio system can be set up from dock to operational in about two hours, and broken down in about the same.” This video shows a greatly speeded-up version of organ set–up.
Carpenter debuted his new International Touring Organ (ITO) March 9, 2014, in New York City’s Alice Tully Hall. The verdict of New York Times music critic Anthony Tommasini on the organ’s sound: “quite terrific.” Later in 2014, Mark Swed in the Los Angeles Times dubbed the ITO “… a genuine dream organ, a fantastically versatile electronic instrument with magnificent sounds…” A more recent Boston Classical review (July 11, 2015) characterizes the ITO as “an orchestra unto itself, capable of shimmering wah-wah effects.” Writing when the ITO made its debut, the artist himself pronounced that “… [The ITO] dares me to make good on my talent …”
Perhaps the most significant technical breakthrough for M and O organ technology is the firm’s ability to build instruments finely tuned to the commissioner’s liking, to fit both the venue(s) and the performer’s repertoire. The ITO’s portability, while authentic, is still limited to locations that its truck can access – and with enough power to pump into its circuits. I’m personally thankful that this somewhat lumpy portability brought Campbell and the ITO to Tanglewood, the Boston Symphony Orchestra’s summer home, last July. I can confirm that Campbell and the ITO do indeed sound quite terrific.
This coming summer marks the 200th anniversary of one of
the most severe weather anomalies in modern history. The Year Without a Summer,
as it's now commonly known, wreaked havoc on much of the Northern Hemisphere. In
upstate New York and New England, in the vicinity of CR4 headquarters, snow
fell in June, frosts were common from May through August, and temperatures
sometimes swung violently between normal summer highs of 90° F or more to
near-freezing in a matter of hours.
The climatic conditions of 1816 also resulted in
unseasonably low temperatures and heavy rains as far east as China. In Europe famine
was widespread and riots, looting, arson and demonstrations were common
occurrences. Throughout the hemisphere, farming became nearly impossible, and
grain prices increased exponentially. In an age when subsistence farming was
the norm and commoners worked their hands to the bone to feed their families,
crop failures often meant the possibility for starvation.
Contemporary observers were almost completely perplexed as
to the disappearance of the summer of 1816, but scientists now believe it was
the result of a few interrelated factors. The most significant of these was the
April 1815 eruption of Mount Tambora on the Indonesian island of Sumbawa. Tambora
was likely the most powerful volcanic eruption in recorded history, with a
column height of over 26 miles and a tephra volume of over 38 cubic miles. Over
70,000 Indonesians were killed following the blast. The enormous amount of
volcanic ash that spewed into the atmosphere reflected large quantities of
sunlight and lowered Northern Hemisphere temperatures.
To compound the effects of the ash, modern scientists also
believe that solar magnetic activity was at a historic low in 1816, the
midpoint of a 25-year solar period known as the Dalton Minimum. By studying the presence of carbon-14 in tree rings, solar astronomers have concluded that
sunspot activity was abnormally low, reducing the transmission of solar radiation
to Earth. Ironically, the Tambora eruption often caused a dry fog to settle
over the Northern Hemisphere, producing a reddened and dimmed Sun and causing
sunspots to become visible to the naked eye. With little knowledge of the
eruption, 19th-century Americans and Europeans often blamed the red,
spotty Sun alone for the abnormal weather conditions, while in reality
Tambora's ash played a much more significant role.
A third less-studied factor is the possibility of a solar
inertial shift. These shifts, occurring every 180 years or so due to the
gravitational pull of the largest planets in the Solar System, cause the Sun to
wobble on its axis and possibly affect Earth's climate. Scientists point to
three of these shifts--in 1632, 1811, and 1990--that correspond to major climatic
events: the solar Maunder Minimum from 1645-1715, the Dalton Minimum discussed
above, and the eruption of Mount Pinatubo with corresponding global cooling in
1991. This association remains largely hypothetical, however.
The Year Without a Summer produced some interesting and
long-lasting cultural effects. Thousands left the American Northeast and
settled in the Midwest to escape the frigid summer; Latter-day Saints founder
Joseph Smith was forced move from Vermont and settle in Western New York, the
first in a series of events that culminated in his writing The Book of Mormon. German
inventor Karl Drais may have invited the Laufmaschine,
the predecessor of the bicycle, in 1818 in response to the shortage of horses
caused by the 1816 crop failure.
That summer may have influenced contemporary art as well.
The high concentrations of tephra in the atmosphere led to spectacular yellow
and red sunsets, which were captured by J.M.W. Turner's paintings of the 1820s.
(If you've ever wondered about the vivid red sky in the more widely known
painting The Scream, some modern
scholars believe Edvard Munch may have viewed a similarly vivid sunset as a
result of the 1883 eruption of Krakatoa.) Trapped inside their Swiss villa due
to the excessive rains in June 1816, a group of English writers on holiday
passed the time by seeing who could write the most frightening ghost story.
Mary Shelley came up with the now-famous Frankenstein
which she would finish and publish in 1818, while Lord Byron's unfinished fragment
The Burial inspired John William
Polidori to write The Vampyre in
1819, effectively launching the still-healthy field of romantic vampire
The advancement of agricultural technology more or less
ensures that we'll never have a comparable subsistence crisis to that of 1816,
despite any further severe weather anomalies. Even so, it's chilling to examine
that year's events and attitudes toward them, as expressed by surviving
journals and works of art.
There are few more polarizing issues than environmental
ones: climate change, the feasibility of alternative energy, and, in recent
news, Earth Day. My 2016 Earth Week was spent scrounging for recyclables for a school
project in which my son built a robot statue out of milk jugs, cereal boxes,
and empty yogurt cups. From where I sit Earth Day is a good thing, a message of
"don't be so anthropocentric that you think you can just throw your crap
anywhere and let Mother Nature take care of it."
The original 1970 Earth Day was by contrast marked by
apocalyptic predictions and fear-based thinking. Americans were surrounded by
grim reminders of industrial pollution, such as the 1969 Cuyahoga River fire,
the labeling of Lake Erie as a "gigantic cesspool," and heavy smog in urban
areas. The original Earth Day saw the prediction of mass starvations, worldwide
famines, the extinction of 80% of all living animals, the reduction of ambient
sunlight by 50%, 45-year lifespans, and the elimination of all crude oil, all
by the year 2000.
All of these predictions fell well short, of course. The
1970 prognosticators were concerned about pollution and fossil fuels but were
mostly anxious about overpopulation, a topic that arouses little fear 46 years
later. If anything, our world is looking to be in better shape. We're seeing
significantly higher crop yields using the same amount of land, lower staple
food prices, and no more DDT; we have at least as much fossil fuel to last us
for about another century, maybe; and population looks to level off within the
next few decades. The Earth Day doomsters, particularly Paul Ehrlich and John
Holdren, exercised faulty and outdated logic in assuming that Negative Impact = (Population)(Affluence)(Technology).
What they didn't plan for is that technology has the power to boost positive
inputs like food production and medicine, and we've learned effective strategies
to reduce pollution along the way.
The problem with predictions is that for every correct one,
there are countless more that are dead wrong, even those based on rigorous data
and scientific extrapolation. So-called futurists predict like it's their job,
often assigning a target date to the year, and are typically wrong. Consider,
for example, Arthur C. Clarke's
predictions for the 21st century. The occurrence of a
technological singularity is still a hot topic in AI communities, and most
thinkers--building on the assumption that Moore's Law will continue unabated--agree
that self-improving artificial general intelligence will occur within the next
50 years or so. At the 2012 Singularity Summit, Stuart Armstrong acknowledged
the uncertainty in predicting advanced AI by stating that his "current 80%
estimate [for the singularity] is something like 5 to 100 years." Now that's how to make a prediction...
This isn't to say that long-term thinking isn't valuable or
honorable. In grad school I became intrigued by the Long Now Foundation, a non-profit working to
foster slower/better thinking rather than the prevailing faster/cheaper
activities of modern times. Aside from hosting seminars and advocating a five-digit
date structure (ie, 02016 for 2016) to anticipate the Year 10,000 problem,
the group keeps a record of long-term
predictions and bets made by its members for fun and accountability. They're
also constructing a clock designed to run for 10,000 years with minimal
maintenance using simple tools, and are working on a publicly accessible
digital library of all known human languages for posterity.
While the Long Now's activities may seem radical, most
examples of true long-term thinking are, given our blindly rushing existence. Shortly
before his death, Kurt Vonnegut proposed a
presidentially appointed US Secretary of the Future, whose sole duty is to
determine an activity's impact on future generations. The Great Law of the
Iroquois famously mandated that current generations make decisions that would
benefit their descendants seven generations (about 140 years) into the future. The
difficulty, of course, is satisfying ourselves in the now as well as in the
future. As environmental skeptics point out, it's nearly impossible to plan out
our fossil fuel dependency and use for the next decade, let alone for the next
Perhaps our best bet is to view the future in terms of
possibility. To paraphrase Clarke's first law of prediction: "When a distinguished but elderly scientist
states that something is possible,
he's almost certainly correct."
From 2008 to 2009 I worked for HSBC Bank, one of the more
interesting workplaces to be in during the Great Recession. A disgruntled
non-customer, whom I believe was teetering on the edge of financial oblivion
like so many of us, once pointedly asked me what the hell HSBC stood for,
anyway. I told him that it was just an acronym*...that the actual name of the
business was HSBC Bank--nothing more, nothing less. "But you're like, a Chinese
bank, right? Isn't the 'H' for Hong Kong?" I assured him it was not, making him
even angrier at the situation.
While I may have been coy about (playfully) screwing with
this man, HSBC is a British company; it was
originally based in Hong Kong and the acronym once stood for Hong Kong and
Shanghai Banking Corporation. In 1991 it reorganized and from then on legally
existed as HSBC Holdings plc. I can't say I know why the company disassociated
from its Far Eastern roots, but the point is that language and acronyms change
for various reasons: falling in and out of common usage, to avoid certain
stereotypes or associations, or just because they become too verbose or
[*As a CR4 editor
pointed out to me after reading this post, HSBC might be more accurately termed
an initialism before 1991, and a pseudo-initialism since. Acronyms are "pronounced,"
like NATO and JPEG, while initialisms are strings of initials.]
A few weeks ago the Associated Press made a major announcement: the 2016 AP Stylebook will lowercase both "internet" and "web." In
line with past stylebook changes, it's safe to assume that the AP believes that
these two terms are now generic enough to merit lowercased usage. Pro-lowercase
activists look to the origin of the word to make their point: the "internet" of
old was simply an internetwork of smaller networks using the same protocol. So
when we speak about the modern Internet--the one I'm using to research this blog
post and connect remotely to my office computer--we're referring to the largest
and best-known example of an internet. Also, they say lowercasing is more
efficient, saving thousands of Shift-key strokes, and that capitalized nouns
are a strain on the eyes, introducing roadblocks into neatly flowing text.
The other side of the battle, on which I sometimes side, takes
issue with the word "the." Think about the star at the center of our solar
system. A star at the center of some other distant solar system could be called
its "sun," but we call the most local and best-known example to us on Earth the Sun, capitalized and all, for
clarity. I know of no other significant internets other than THE Internet--if
you know of one feel free to comment and enlighten me. And regarding the web,
what if we're trying to describe researching spider webs online? Would we look
up webs on the web? Isn't the Web
clearer? Call me antiquated (my wife
does on a daily basis, so I'm used to it), but I like my Internet and Web, even
if I'm too lazy to click Shift and actually capitalize them most of the time.
These technologically related style changes happen pretty
frequently. For example, AP changed their usage of Web site to website in 2010,
and e-mail to email in 2011. These make more sense as generic terms, in my
opinion: we surely no longer think of email as "electronic mail." With the slow
demise of postal mail, perhaps email will one day be referred to as just
"mail," and postal mail will become oldmail or cismail, maybe.
The fluidity of technical terminology is also easily seen in
anacronyms, or words that were formerly acronyms but have fallen into common
usage. Lasers were originally "light amplification by stimulated emission of
radiation," for example. Treating "laser" as a common noun allowed us to back-form the verb "to
lase," meaning to produce laser light. Ironically for me as a technical writer
and editor, even the verb "to edit" was back-formed from "editor," the original
The possibility for confusing variation and evolution in the
English language is endless. Who knows? Maybe in 50 years our descendants will
just switch on their computers and internet.