CR4 - The Engineer's Place for News and Discussion®
Login | Register for Engineering Community (CR4)

Notes & Lines

Notes & Lines discusses the intersection of math, science, and technology with performing and visual arts. Topics include bizarre instruments, technically-minded musicians, and cross-pollination of science and art.

Creative AI: Can Computers Create?

Posted July 28, 2015 9:01 AM by Jonathan Fuller

Artificially intelligent systems are capable of many human-like behaviors. At least on a surface-level, they can think, learn, speak, read, and simulate emotions. The convergence of human and machine behavior has led to the well-worn ideas of the Singularity and possible machine superintelligence.

Those uncomfortable sharing their headspace with a machine purport that computers will likely never be capable of one of our most human traits: creativity. Creativity seems random, messy, and subjective, three attributes not easily grasped by computers. Creative AI research, while interesting and promising, has provided more questions than answers; namely, "What is creativity, and who's capable of it?"

Creativity is "the ability to transcend traditional ideas, rules, patterns, relationships, or the like, and to create meaningful new ideas, forms, methods, [and] interpretations," according to Random House Dictionary. Given that computers essentially analyze fairly standard rules, patterns, and relationships, a truly creative one seems unlikely. Creative works are often described as "novel" and somehow valuable or useful; without direct human input, this is a monumental challenge for AI.

These hurdles haven't stopped researchers from trying, though. Lior Shamir, a computer science professor at Lawrence Technological University, developed algorithms that identified similarities between works of Jackson Pollock and Van Gogh and correctly ordered all thirteen Beatles LPs chronologically based on audio and visual samples. Because computers aren't adept at handling discrete pixel and frequency data, Shamir converted each visual or audio sample into thousands of numerical values that were fed into a pattern recognition system. Shamir's music understanding programs are based on an earlier project that analyzed 15,000 whale songs to identify that whales communicate using different dialects depending on their geographical origins.

While analyzing creative works might seem much simpler than actually producing them, automated painting and composing software programs have been used for decades. AARON, a program created by Harold Cohen in the mid-'70s, is hand-coded to produce artworks (like the one below) in a specific style. Even Cohen is quick to point out that AARON isn't creative, though: it simply follows procedures outlined by its programmer, who is the true artist. Returning to the definition of creativity, AARON follows the rules rather than transcending them. More complex painting robots are becoming common, although these still seem to be merely going through the motions.

If true artistic creativity seems outside the reach of AI, what about the experimental kind? Assuming Pasteur's assertion that "in the experimental fields, chance favors the prepared mind," a data-loaded processor would be much more likely to stumble upon a scientific breakthrough than a messy human brain.

AI research seems peculiarly unconcerned with the philosophy of the work, with practical solutions trumping abstract ideas: an application that works will naturally form the basis for an abstract theory. Many in the field point to aviation as a good example: it took building a working flying machine to answer the century-long debate asking "Can we, and should we, fly?"

So is there a useful application for creative AI? Music recommendation engines used by iTunes and streaming music services would certainly benefit from creativity breakthroughs. Although in a subjective sense, I wouldn't find an hour of identical-sounding music very appealing.

Image credits: The Academic Minute | Stanford University

22 comments; last comment on 07/31/2015
View/add comments

Your Kitchen Table Is Now A Musical Instrument

Posted June 25, 2015 4:26 PM by Jonathan Fuller
Pathfinder Tags: electronic music tonetree

Pocket-sized instruments have always appealed to musicians. Pulling out a harmonica, xaphoon, or ocarina and playing on the go seems like the pinnacle of convenience for the musically minded. But imagine the ability to transform any surface you come across into a musical instrument, using only your smartphone and a small device named after a tree.

Last year, four RPI students formed startup ToneTree and began work on such a device, which they've dubbed Birch. Birch consists of a very-high-frame-rate infrared camera that allows users to draw a sonic interface on a table or other flat surface. It's technologically similar to an infrared QWERTY keyboard, albeit with a much faster camera for real-time sound production.

ToneTree co-founder and CEO Brian Cook, who's pursuing an MFA in Integrated Electronic Arts, has been using his interest in music and technology to come up with innovative instruments since his undergrad days. At the University of Hartford he wrote a piece for the commemoration of a new building and invented a wired mallet device with integrated sensors and switches as well as remote cameras to enable performers to "play" the new building. Cook and his three ToneTree compatriots started work on the Birch prototype after he realized that doing away with the wires and creating a free-form instrument would heighten the experience.

The new instrument allows users to draw shapes on a flat surface and program them to correspond with waveforms like notes or percussive sounds. (Check out this demo.) This free-form interface makes Birch's expressive possibilities almost endless. The team has also talked about developing mats printed with a piano keyboard or guitar fretboard in the future.

The group has made rapid advances on Birch since undertaking the prototype in November 2014. They've just recently achieved data processing fast enough to eliminate most of the lag between finger movement and sound production and are still experimenting with faster optics. Birch renders a user's table or wall touch-sensitive, like a piano, by tracking finger velocity and producing tones that are proportionately forceful or gentle. The backend is powered by a combination of OpenCV for computer vision, MIDI for the musical portion, and Qt for the user interface.

The Birch is a patent-pending design and will be sold commercially in the coming year. Cook and software designer Ronald Sardarian discussed expanding research to a future device that would enable users to transform any flat surface into a touchscreen using a simple mobile device. It might be a useful addition to other promising augmented reality (AR) technologies such as Microsoft's HoloLens.

Classroom application of Birch seems like a natural progression, and the inventors are already partnering with local school STEAM (STEM+the arts) programs. Thinking back to my own school days, I imagine that most would rather play with this neat new technology than study the recorder. And if its future AR applications allow tapping walls rather than waving my hands in the air, well, sign me up for that too.

Image credit: Quickmeme

5 comments; last comment on 06/30/2015
View/add comments

RIP Grooveshark

Posted May 11, 2015 12:00 AM by Jonathan Fuller

Many people of a certain age (mine, specifically) will likely tell tales about the brief Golden Age of Napster to their children and grandchildren. True peer-to-peer sharing. Hard-to-find songs obtained within minutes. No meddling from evil behemoth record companies. From 1999 to its 2001 shutdown, Napster was the original pioneering P2P music-sharing service until it was sued by members of the Recording Industry Association of America (RIAA) for trading copyrighted music.

Most free P2P sites have likewise been shut down or converted to subscription models, and dozens of music streaming services have cropped up over the past decade, but the legality of such services remains nebulous at best. This was apparent at the end of last month, when the popular streaming website Grooveshark shut down after reaching a settlement with Universal Music. The company was found guilty of copyright infringement and faced up to $736 million in damages for uploading Universal's copyrighted tracks but escaped that payout by voluntarily shutting down. Grooveshark's homepage now includes a relatively contrite message describing the shutdown.

The controversial Grooveshark has drawn the ire of record companies since its founding in 2007. Despite pioneering legit licensing deals with large publishers such as Sony/ATV, ASCAP, and BMI, they had been in death throes since last year, when an early internal email was revealed as part of the legal case. In it, CTO and co-founder Joshua Greenberg urged all employees to upload their own MP3s to the service (nearly 6000 in all) or face his corporate wrath. The email was found to negate protection under the "safe harbor" provision of the Digital Millennium Copyright Act (DMCA) and doomed the company to big damages or complete failure.

Digital copyright discussions bring up interesting issues around technology, usership and ownership. For example, if a user illegally uploads or downloads copyrighted content to or from a file-sharing service, who's truly responsible? The DMCA's safe harbor provision, which mandates that online user-centric services like Grooveshark and Google's YouTube take down offending content when notified by the copyright holder, gives users a carte blanche for infringing behavior, at least until the service is caught in the act. Since popular sharing sites have hundreds of millions of users, many of whom wittingly or unwittingly upload copyrighted content on a regular basis, illegal content may stay active for years before a takedown notice occurs.

Even more nebulous in terms of DMCA are sites like Vid to MP3, which allows users to extract audio from any YouTube link in a matter of minutes. Extracting audio from, say, a recorded talk or lecture is perfectly legal, but because YouTube is slow to remove the millions of infringing videos on their site, a user could easily extract audio from a copyrighted music video and effectively pirate the song itself. Vid to MP3 attempts to do its due diligence by including an anti-piracy notice on its homepage, but how many users realistically follow this directive with zero chance of personal backlash?

DMCA's message to on-demand service users is that copyrighted content showing up free on the Web is probably at least partially illegal. Some music services such as Pandora and Spotify have evolved to comply with the elevated royalties required under DMCA, legally acquiring content from large publishers and generating revenue through ads or premium subscriptions. As a former Grooveshark user I will admit that I'm sad to see albums-worth of free music down the drain. But the law's the law and musicians still have to have to eat.

Image credit: Nisha A / CC BY 2.0

6 comments; last comment on 09/15/2015
View/add comments

Vinyl: No Longer Just for Audiophiles?

Posted March 12, 2015 12:00 AM by Jonathan Fuller

Music biz writers scrambled to their blogs in January 2015, when Nielsen SoundScan announced that 2014 vinyl sales had come in at 9.2 million units sold, a whopping 52% more than 2013 sales. That figure is also the highest mark for vinyl sales since SoundScan began surveying in 1991. A question is likely lingering in the minds of both musicians and analysts: was 2014 the peak of the vinyl fad, or does it represent a legitimate new/old music technology trend?

It goes without saying that the music business landscape has changed dramatically in the last decade or so. The rise of mp3 vendors like iTunes and Amazon Digital Services more or less killed album sales instantly, as listeners now download one or two favorite tunes for $2 instead of shelling out $11.99 for an album. Streaming services and piracy seem to be doing the same to digital downloads as we speak: Nielsen reported that 2014 downloading fell 13% compared to 2013, while streaming-per-song shot up 53%. The state of consumer music listening has changed as well: the days of group listening in your friend's living room are long gone. Ubiquitous portable music players of all kinds have rendered consumer listening a passive, near-constant experience.

Cooler-than-you hipsters and hi-fi enthusiasts have long touted vinyl's chicness and unique sonic textures (see below), but the business side is smelling a possible boon hidden within those acetate grooves. If the buyers are buying (which they are), records give music suppliers exactly what they've been looking for since the birth of the mp3: a degradable product that's very difficult to copy and redistribute. Plus, an artist might sleep a little easier when selling vinyl, as its technological tracking limitations (ie, it's relatively difficult to find single songs) ensure that a listener will hear most if not all of your album as it's intended it to sound, rather than hearing only a single track downloaded off the same album.

The downside is that pressing vinyl is expensive and time-consuming-records are still about five times more expensive to manufacture than CDs. United Record Pressing, currently the largest producer in America, recently acquired a second plant and doubled their capacity, but still advertises a 4-6 month lead time for new albums despite working 24 hours a day, six days per week. For comparison, a batch of 1000 CDs typically takes a few weeks to produce.

So is the commonly held belief that vinyl "sounds better" really true? The answer is a subjective "maybe." First, it's important to note that the physics of turntable belts, trays, and cartridges affects playback more than a CD player, so a record played on one machine may sound significantly different than on another. And the suspect audio quality of early mp3s that required loads of compression to allow eight songs to fit into your pocket may have started and propagated the "digital audio sounds awful next to vinyl" argument.

The supposed "warmth" of vinyl is caused by defects related to analog recording. Each time a master tape is played back in-studio, its iron filings are slightly degraded, and since high frequencies are represented as faster waves on the tape, they're typically the first to be reduced. A vinyl record therefore has a natural lack of high frequencies, resulting in a warmer or "bassier" sound. In addition, low frequencies are difficult to reproduce on vinyl, so almost all vinyl albums have undergone some type of low-frequency signal processing. This processing, when added to the vibrations and noise inherent in all but the best home turntables, results in a distorted yet warmer sound that's probably drastically different than the sound heard by engineers at the mixing desk.

CDs and digital audio, on the other hand, may lack the characteristic warmth but far outshine vinyl in terms of dynamic range. Vinyl's noise issues result in a dynamic range of about 60 dB, while the standard for CDs is 96 dB with less playback noise (scratching and whirring motors) to boot. Newer digital recording techniques are capable of 118-120 dB ranges with comparable perceived playback. Aside from its use for authentic AM-radio special effects, analog recording has generally taken a backseat to digital, which requires little to no signal conversion and has no degradation effects. In general, your digital recording sounds on your mp3 player like it did in the mixing booth.

A related issue here is the recent improvement in audio signal conversion. Many casual listeners immediately noticed the flat aural quality for classic vinyl albums reissued on CD in the early '90s; this was due to primitive ADC technology, and many older groups are choosing to re-re-release their classics again in the 2010s on both CD and vinyl to take advantage of better ADCs and DACs. On the flip side, the suspicion that most modern vinyl reissues are sourced from CDs and not master tapes has some in the audiophile community up in arms.

Regardless of whether it sounds "better," vinyl's sales surge may or may not save a small part of the music industry's bacon. Those who champion analog aural characteristics will be delighted to know that supply for their turntable is plentiful and growing, and those who don't will be putting their earbuds back in and casually walking away.

Special thanks: Chris Reed Jr., engineer at Glens Falls Music Academy in Glens Falls, NY, assisted with this post.

Image credit: Donnie Ozone / Creative Commons

14 comments; last comment on 05/13/2015
View/add comments

Acoustic Guitar Tonewoods

Posted July 26, 2014 12:00 AM by cheme_wordsmithy

I remember the day I received my acoustic guitar, and played it for the first time. In the moment, it seemed produced the most beautiful sound I had ever heard from an instrument of its kind. As my ears became attuned to more guitar music, I came to realize that the tonal quality of mine was nothing special. That realization drove me to delve deeper into the realm of acoustic guitar sound, and would help me find the right build for my next guitar a few years later.

As an amateur whittler and wood-worker, the beauty of wood is one of the reasons I think the acoustic guitar is such a neat instrument. Apart from maybe the body shape, the wood choice is the single most critical factor in determining the guitar's sound. What's more, different wood types can be chosen for each of the guitar's main body parts (top, back, sides, and bracing) to create a unique blends of look and tone.

Because wood comes from trees, which are living organisms, no two pieces are identical - each cut will have its own unique characteristics based on grain pattern, size, etc. A luthier (guitar-maker) must assess these characteristics to determine the musical quality of the wood piece for guitar building. Velocity of sound is perhaps the most important, and is a measure of how fast the wood transmits received energy. In other words, when you pluck a string, how will the wood respond to the energy of its vibration. Wood with a higher velocity of sound will generally have more colorful, lively, and accurate tones.

Characteristics like velocity of sound will vary by cut, but are also largely inherent in the wood based on its type (stereotypes of a sort, if you will). Different types of guitar woods (tonewoods as they are called) have different tonal qualities. The best tonewood for each musician will thus vary based on their playstyle. Here is an overview of some of the woods commonly used in guitar-making:

Spruce - Spruce is the acoustic guitar standard for soundboards, and comes in a number of species (Sitka being most commonly used). Spruce is a rigid, lightweight softwood with a high velocity of sound, and strikes a good balance of response for lighter and heavier playing styles.

Cedar - Cedar is a softwood with a balanced warm sound, and has a rich and quick response favorable to lighter playing styles.

Mahogany - As a topwood, mahogany (a hardwood) has a relatively low velocity of sound and high density, giving it a very mellow, warm, somewhat "punchy" sound suitable to country-blues music.

Koa - Koa is a hardwood similar to mahogany, with a solid tone throughout, with heavier emphasis on the mid-range.

Maple - Maple has a high degree of internal dampening and lower velocity of sound. This gives a heavy emphasis on the high range and a very bright tone, that contrasts well with the warm nature of finger-picking.

Rosewood - Rosewood is known for its high response rate, with a darker tone throughout most of its range, and a strong bottom end. It's broad range gives it presence in any style of music, finger-picking or pick-style playing.

The complexity of tonewoods goes well beyond this overview, as their are many more in this list I did not mention. In addition, the back, sides, top, and bracing on a guitar can all be made of different woods to create unique varieties of sound.

I'd be interested to hear what you fellow musician and/or woodworker CR4ers have to say. Do you own or have you ever made a wooden instrument? By sound or by application, do you have a favorite type of wood for your instrument?


Tonewood Data Source

15 comments; last comment on 01/09/2015
View/add comments

For Love of the Index

Posted May 09, 2014 12:00 AM by Jonathan Fuller
Pathfinder Tags: books index indexing

If you're a literate human who's a (diligent) knowledge worker or (diligent) student, chances are you've used an index this week. Or at least you did, back when you had routine physical contact with printed books. You might not think of it on a regular basis, but most of those indexes (not indices; those are the financial tools) were created by a freelancer who specializes in reading book manuscripts really closely and mapping their contents. Book indexing is an old-school craft: some professionals still use a pencil and paper to organize headings and topics before working them into index format.

I considered starting a freelance indexing firm myself during the dark days between grad school and professional success. That is, until I envisioned the sleepless nights over 36-hour turnarounds and the increasing eyeglass prescriptions and headaches caused by daily manuscript reviews. I'm retrospectively happy I avoided this career path and its idiosyncrasies, beginning with the big one: the unreasonable, nearly unmanageable deadlines.

In an industry known for tight deadlines, indexing is the last step in the publishing process: the author completes the manuscript, the editor corrects it, the author rewrites, the editor edits again, and off it goes for an index. All an indexer really needs is a completed manuscript to do his or her job, but that 600-page manuscript is probably two weeks delayed already, resulting in a 48-hour deadline (and two sleepless nights). The index might be complete when the publisher announces that pages have been cut or added; in that case, it's now useless and has to be rewritten on an even tighter deadline. I'm imagining stuff like this might occasionally happen on engineering projects too, yes?

Technological development stresses this industry out, as it's done for many antiquated professions. E-books and digital prints are becoming more common, with most reader software incorporating some form of full-text search. Heck, everybody knows how to CTRL+F these days. Authors and publishers are becoming dubious of paying freelancers (typically $3 to $6 per indexed page) and instead are using proprietary software to scan their manuscripts and generate an instant index in a matter of minutes; and it's more or less free. By all indications the profession may be imploding: the U.S. government-backed basic indexing course (which was curiously run by the Department of Agriculture for over 35 years) will cease to exist after September 30th of this year, although in fairness this seems to be part of a larger scale-back of older USDA distance education programs.

Manual indexing is still an extremely useful and necessary practice despite its faults. As hashed to death in previous blog posts, indexers (and humans in general) can do what automated systems can't: they can really and truly think. Indexing is a precise, strategic craft which involves planting markers - Hansel-and-Gretel-like breadcrumbs - where readers are likely to find them. Indexers use tools such as cross-references, see's, and see also's to anticipate user needs and point queries in the right direction. Since the vast majority of freelance indexers are library school-trained individuals who longed for an entirely different form of tedium, indexers are pretty adept at (and usually enjoy) making stuff findable and helping users find it.

Indexes help users find correct topical information at high speeds; you're not just CTRL+F'ing (this post has now officially graduated from PG to PG-13) for individual terms, but instead are finding the knowledge you need. Sure, you may have to turn a few pages (or, in the case of e-readers, scroll a bit), but if the index is good and you have a vague idea of what you're looking for, you'll pick it out. Even if you don't know what you're looking for or at, there's still hope: indexes were the original semantic search engine. Confused about references to Eric Arthur Blair in a text about 20th-century British writers? The index can quickly assure you that you're in fact reading about Orwell, George. Finding references to Cristobal Colon in Spanish primary sources? See also Columbus, Christopher.

While it briefly attracted my attention as a career, indexing wound up in the junk bin alongside my aspirations to become a musicology professor and indiepop star. These days some consider it the rotting branch of the publishing industry. (Well, magazines…and books…but…OK, there are several rotting branches of the publishing industry.) Some indexers (namely the ones doing good business) insist all is well in their world, others believe their ship has taken on water but is still afloat, and many wail that the profession is hopelessly outmoded has been dying a slow, agonizing death for decades. Regardless of its current state, indexing is a largely anonymous, specialized profession that gets little notice or consideration from the world at large. So, next time you use a book (!) instead of Google to nail a research paper or win a bet over some useless factoid, think of the indexer who toiled to help you out.

Image credits: Chronicle of Higher Education | ShirtWoot!

Add a comment

Previous in Blog: Wait...What'd I Just Say?  
Show all Blog Entries in this Blog