Jesse M. Heines | Daniel A. Walzer – Teaching a Computer to Sing
As technology permeates more and more aspects of our daily lives, computer literacy and computer programming skills are more valuable than ever in the workplace. Professors Jesse Heines and Daniel Walzer of the University of Massachusetts Lowell are working at the intersection of music and technology to engage middle school students in learning computer programming by ‘Teaching a Computer to Sing’.
Computer literacy has become a critical component of modern life, as computerised ‘smart’ technology continues to become a part of many everyday tools. The ability to write computer code is highly valuable in the workforce and the demand for programmers is expected to continue to rise over the next few decades. Despite a worldwide need for skilled programmers, many students are not introduced to computer coding concepts until college.
Early exposure to computer programming principles can help students gain confidence in working with technology and prepare them to compete in the job market of an increasingly technology-driven society. This jump start can be particularly important for young women and minority students, who are underrepresented in science, technology, engineering and mathematics (STEM). Diversity in technology leads to more unique innovations to solve the problems humanity faces – helping students see themselves as ‘technical’ early in their educations can greatly impact how they perceive and pursue STEM careers as adults.
In an effort to broaden understanding of STEM concepts, many educational researchers have taken interest in the intersection of STEM and the arts. Recent efforts have sought to connect visual art, theatre, music and literature with modern technology and mathematics to increase our understanding of the arts and to identify potential educational avenues between subjects that often seem unrelated. These efforts have demonstrated that music and computer programming complement one another well and are particularly amiable to adaptation for the classroom.
Music holds almost universal appeal to students of all ages, and its underlying structure often translates well into a variety of computer programming languages. Heines and Walzer are developing the innovative program, ‘Teaching a Computer to Sing’ or TACTS, that is engaging middle school students in both technology and the arts by teaching them to translate music into computer code.
‘Our work focuses on teaching basic computer science concepts to students who might never take a formal course in computer science or computer programming. We do this through music, showing students connections between the structure of music and the structure of computer programs.’
Walzer describes how, ‘our work focuses on teaching basic computer science concepts to students who might never take a formal course in computer science or computer programming. We do this through music, showing students connections between the structure of music and the structure of computer programs.’ They have spent the past two years piloting TACTS as a middle school after-school program, and their results show promise for combining STEM education with the arts.
Overcoming Educational Obstacles
The overarching goal of the TACTS curriculum is to increase students’ understanding and confidence in both music and computer programming. The concept was originally designed and implemented with undergraduate students, and the course has enjoyed great popularity at the college level. Heines and Walzer hoped to adapt the program to meet the needs of younger pupils and to accommodate the often more limited resources available within a typical public middle school classroom.
One of the major aims of the pilot study was to identify potential challenges of teaching in the school environment and to modify the structure of the program to provide the best learning experience possible for younger pupils. The TACTS program ran for 2.5 hours twice a week after school, offering long blocks of learning time, but at a time of day that might prove difficult for student attention span. On a typical afternoon in the program, students spend the first hour singing and learning music, take a short break, and then spend the rest of the day on the computer coding.
The team recognised that getting children to pay attention to any kind of learning exercises after they had already spent seven hours in school was going to be a challenge. They partnered with a seasoned middle school music teacher, who was already well-liked by the students and experienced in getting distracted kids involved in group activities.
Even with the music teacher’s help, they still found that it was a struggle to keep the kids engaged in large group activities. They eventually recruited multiple university students to act as teaching assistants, creating a high student-teacher ratio that allowed for teachers to interact with students in pairs.
The college age teaching assistants contributed to a more clubhouse feel for the program, where younger students were able to bond with college students as role models while receiving more individual attention. In the first year, with mostly male teaching assistants, the TACTS team quickly recognised that the group of primarily female students responded most strongly to female teachers, and that their female teaching assistants were able to provide valuable insights into what middle school girls were going through. In the second year they hired more female teaching assistants to better replicate the gender ratio of students in the program.
Access to technology also proved to be a major problem that the TACTS team had to overcome. While their undergraduate course typically consisted of students with their own laptops, many of their middle schoolers did not have access to a computer at home and were reliant on the machines provided by the school.
This was further complicated by the fact that the school-owned computers were equipped with security measures to prevent students from installing software on the computer. To circumvent these issues, Heines and Walzer identified music and coding programs that could be used as cloud-based software from any computer with Internet access. This vastly increases the flexibility of the program and means that any school can easily implement the program within a tight budget.
Learning the Harmony
Previous research has suggested that education focusing on the intersection between STEM and the arts could help students learn more about both subjects. In the initial pilot of this program, Heines and Walzer wanted to determine if an interest in understanding music could facilitate learning to translate a song into computer code and if working with the individual parts of a song could also help students learn how to sing multi-part harmonies.
Music is usually more familiar to students than the language of computer programming. The TACTS team expected that it would be easier for the students to initially focus on learning the song before transferring it into code after they had a firm understanding of the music. Walzer summarises the philosophy underlying starting with music: ‘This approach tries to make the abstract concepts of computer science more concrete and thus easier for students to understand.’
Before students can begin translating songs into code, they must first understand the musical components of a song. Initially the TACTS team started with popular pop songs, working with a music arranger to simplify the complex harmonies into more manageable compositions.
Early into their pilot they found that despite their best efforts, students were struggling with the complex rhythms of the pop songs. At the suggestion of the TACTS music teacher, they shifted to partner songs, which are simpler songs designed to help students learn how to sing in harmony. As part of their music education, the students get to enjoy demonstrations of people using the musical concepts they are working on in the program. Many of the undergraduate teaching assistants are music majors and occasionally performed for the class.
TACTS has also brought in an a cappella group to demonstrate the possibilities of multipart singing. These visits served to both educate the students about music and to inspire them to keep working on their own compositions in the program.
Programming a Song
Perhaps the biggest challenge of the TACTS program was selecting an appropriate coding language for middle school pupils. Heines and Walzer started with the Scratch programming language and MIDI music notation. This required students to first identify a musical note’s alphabetical name and then convert it into a corresponding MIDI numerical code.
In this notation, the code bears little resemblance to sheet music, and it is harder to identify on paper. The students were struggling with this conversion, and the TACTS team felt that they would make more progress with a simpler language. They switched to EasyABC notation, which more closely resembles common musical score format, and started using the Pencil Code environment to play the resulting sequences.
Students had a much easier time connecting the dots with these programs. These tools are available online and can be used on a website without installing anything on the computer, making them ideal for use on school machines with tight restrictions on software installation.
One of the greatest benefits of writing code for music is that students can immediately play back their coded score and observe tangible results of their programming work. Unlike many coding applications where mistakes can be difficult to identify, students can simply listen to the output of their programs to find the places where the code is not working. They are in effect learning to code through play, mistakes in the code are easy to hear, and successes are instantly rewarded.
For trickier song segments, the high student-teacher ratio allowed the TACTS team to be readily available to provide support and help students work through coding challenges. In the second year of the pilot, TACTS introduced a third program called Soundtrap that permitted students to incorporate a broader range of recording tools and sounds and gave them the opportunity to learn a second coding language format. This allowed second year students to take more creative license with the digital music they were creating while also letting them apply their coding skills to a more advanced project.
A Bright Future for TACTS
The ultimate goal of the TACTS program was to facilitate student learning in both music and computer programming. The results of the initial pilot study show promise for implementing teaching programs at the intersection of art and science. All of the participating students learned basic coding skills and demonstrated an understanding of the foundations of computer programming.
Students also learned to sing in three-part harmonies, surpassing the music teacher’s expectations for what was possible in their age group. When asked about their perception of music and their own music abilities, students that participated in the program indicated that they both perceived music more positively and saw themselves as more capable than when the program began.
When asked the same questions about programming, most students perceived programming about the same as at the start of the program, but the majority felt much more confident about their own ability to write computer code.
Walzer notes, ‘just as our work has built on that of others, our work has been picked up by other educators and adapted and extended for their own school and college environments.’ They are excited to see the flexible program sing out in classrooms across the nation.
Meet the researchers
Jesse M. Heines, Ed.D.
Department of Computer Science
University of Massachusetts Lowell
Jesse M. Heines began his journey in academia with a Bachelor of Science in Earth Sciences from the Massachusetts Institute of Technology in 1970. He continued on to earn a Doctorate in Education specialising in Educational Media and Technology from Boston University in 1979. He has over 30 years of practical experience working in computer-based instruction development. In 1985 he joined the Computer Science faculty at the University of Massachusetts Lowell and retired a Professor Emeritus in 2016. He has won awards for excellence in computer science teaching and published over 40 peer-reviewed papers on computer-based learning. For more information and resource materials on the TACTS program, please see https://jheines. github.io/tacts/Workshops/.
Daniel A. Walzer, Ph.D.
Assistant Professor of Composition for New Media
Department of Music
University of Massachusetts Lowell
Daniel A. Walzer began his career in music with a Bachelor of Music in Jazz Studies from Bowling Green State University in 2000. He went on to complete a Masters of Music in Jazz Studies at the University of Cincinnati’s College-Conservatory of Music in 2005 and in 2014 earned his Master of Fine Arts in Music Production and Sound Design for Visual Media from the Academy of Art University, SF. He has recently completed his Doctorate in Leadership from the University of the Cumberlands, KY. Walzer’s writings on music technology and education appear in more than a dozen peer-reviewed journal articles, book chapters, and refereed conference proceedings. Originally trained as a percussionist, Walzer also maintains an active career as a composer in a variety of settings. In 2014 he joined the faculty of the University of Massachusetts Lowell, where he currently serves as an Assistant Professor of Composition for New Media.
Gena R. Greher, Ed.D., Professor, Music and Music Education, Department of Music, University of Massachusetts Lowell
Rachel R. M. Crawford, Music Teacher, Bartlett Community Partnership School, Lowell (Massachusetts) Public Schools
Firas AL-Rekabi, Math Teacher, Bartlett Community Partnership School, Lowell (Massachusetts) Public Schools
Peter B. Holtz, Principal, Bartlett Community Partnership School, Lowell (Massachusetts) Public Schools
U.S. National Science Foundation