CS 472: Digital Music Processing

Ursinus College, Spring 2021

Course Evolving: Site Last Updated 5/16/2021

Table Of Contents

Overview Technology Logistics Homework

Grading Classroom Environment Participation

Collaboration Policy Other Resources / Policies

---

Overview

Class Times / Locations

• Monday/Wednesday/Friday, 2:30-3:20PM in Helfferich Hall, Dance Studio (Remote Students on Zoom)

Virtual Drop-In Conversation Hours over Discord

• Monday 4PM - 5PM
• Tuesday 3PM - 4PM
• Wednesday 5:30PM - 6:30PM
• Friday 4PM - 6PM

Prerequisites/Requirements

Math 111 or equivalent, CS 173 or equivalent

Hybrid Instruction

This course will be conducted in a hybrid fashion, where class time will occur in person following socially distanced protocols, but where some students may be attending remotely. Please review to the logistics section for more info. In spite of the complexity of this arrangement, I will make every effort to make sure that all students welcome and valued, that we can still build a community, and that you get all of the help you need (this is part of the reason we are using so many different technologies).

Instructor

Christopher J. Tralie

I grew up right around the corner in the Montgomery County and attended Upper Dublin High school (class of 2007). I then did my undergraduate degree in Electrical Engineering at Princeton University and my master's and Ph.D. degrees in Electrical And Computer Engineering at Duke University (heavily studying math and CS along the way). I finally started my dream job at Ursinus College in Fall of 2019! You can read more about my interests on my professional web site. Looking forward to getting to know everyone as we work through this course together!

Course Description

This hands-on course will take a broad overview of how to represent, analyze, and morph/transform digital musical audio with a computer. My goal at a high level is by the end of the semester, students should be able to walk into a poster session at a conference like ISMIR or DAFX and understand roughly what's going on. On the synthesis/transformative end, students will create digital musical instruments, they will learn how to filter and denoise audio, they will create programs such as vocoders (e.g. "talking guitars") and autotuners (i.e. "I Am T-Pain"), and they will implement algorithms to automatically time warp and align different versions, or "cover songs," so that they can be compared side by side, even when the timing is different. On the analysis end, students will train a computer to automatically figure out what notes a person is playing, to "tap a virtual foot" to the beat, to discover repeated sections such as verse/chorus, to identify a song from small clips of audio (the so-called "Shazam algorithm").

In the process of implementing the above algorithms, students will gain familiarity with a fundamental tool known as the Discrete Fourier Transform, which allows us to break audio down into its fundamental components: a bunch of pure sine waves added together. This amazing way of looking at audio will allow us to "put on a different set of glasses" to find patterns and filter out parts of audio more easily.

There has been a long tradition of implementing multimedia algorithms in Matlab. The downside of Matlab is that it is proprietary and quite expensive, and aspects of the language beyond matrix multiplications are not very elegant. Thankfully, in recent years, a library in Python known as numpy has matured and allowed the Python language to accomplish everything that Matlab could with digital audio. So though the focus is more on the algorithms themselves, we will learn python and numpy as we go along in this class, and these are valued skills in data science well beyond audio processing.

Learning Goals

1. Develop broad knowledge of analysis and synthesis techniques for digital audio, with an eye towards musical applications.
2. Learn how to incrementally design, implement, and test signal processing pipelines in code.
3. Use mathematical and physical models to synthesize audio and to process messy real-world data.

Learning Objectives

1. Practice patient problem solving by developing comfort with the edit -> compile -> run loop, along with intermediate debugging skills.
2. Implement algorithms based on the Short-Time Fourier Transform for a variety of analysis and synthesis tasks in audio.
3. Implement several basic machine learning pipelines for different audio tasks, such as artist identification, cover song identification, and beat tracking.

---

Technology Logistics

Since we will be working in a hybrid learning scenario with the pandemic, there are an array of technologies we will be using in concert to stay in touch. Unfortunately, there's just not a one stop shop for anything technological these days (much like there isn't one social media platform that's used for all kinds of interactions...).

Below is a table summarizing what kinds of communications/activities occur via each technology, and below that there are more details on everything. This is admittedly complex, and it will take some getting used to, but it will be worth it once we get it nailed down.

NOTE: I will repeat the same announcements across e-mail and Discord, so you don't have to check all both for announcements.

Class web site	Schedule (Topics, Study Resource Links, Assignment Deadlines) Pre-class modules
Canvas	Assignment Deadlines, Submission, and Feedback
Zoom	Synchronous class sessions with breakout rooms
Discord	Chat/polls during synchronous class sessions Announcements Office Hours Class-wise questions and discussions Individual chat with Prof. Tralie of a less sensitive nature (e.g. tech support, assignment questions not appropriate for the entire class)
Microsoft Teams	Buddy Collaborations/Screen Sharing Individual chat with Prof. Tralie of a less sensitive nature (e.g. tech support, assignment questions not appropriate for the entire class) Individual chat with Prof. Tralie of a more sensitive/private nature (e.g. questions about grades, academic planning, personal issues) (*)
E-mail	Announcements Individual chat with Prof. Tralie of a more sensitive/private nature (e.g. questions about grades, academic planning, personal issues), though I prefer Microsoft Teams for this and will respond more quickly over Teams Individual chat with Prof. Tralie of a less sensitive nature (e.g. tech support, assignment questions not appropriate for the entire class), though I prefer Microsoft Teams or Discord for this and will respond more quickly over Teams

*: For privacy reasons, anything of a personal nature, and particularly things that have to with educational records (e.g. grades), need to be kept within Ursinus sanctioned platforms like Outlook e-mail and Microsoft Teams.

Canvas

We will be using Canvas, but only to submit assignments and to store all of the grades. I will also keep all of the due dates current on the calendar there, as students have appreciated this common space for all of their classes in the past.

Zoom And Class Time

Ursinus now has a professional license for Zoom. During class time, we will meet on Zoom in synchronous, virtual sessions. The main reason we're using Zoom instead of Teams for class is because of its breakout room functionality. Students will also have discord open during class for more advanced polling. Due to the hybrid classroom during COVID-19, please note the important points below:

• We will all be logged into Zoom during class time. This means that students who are in person are expected to bring laptops and headphones
• In the absence of accommodations, students will keep their cameras on at all times. This includes students in class. Surveys at Ursinus last semester showed that remote students felt better in hybrid learning environments when everyone, including students in class, had their cameras on.
• Students who attend class in person are expected to wear a mask at all times, to practice social distancing (6 feet distance to nearest neighbor), and to wipe down their places before they leave. This is all common courtesy during a global pandemic, and it will help to keep our workplace safe. Students who violate this will be asked to move online.

We will do the following activities during class:

• At the beginning of each session, students will have an opportunity to ask questions on the pre-class modules
• During most classes, students will work together in groups in virtual break-out rooms on slightly more challenging problems than the modules to hone skills for the assignments.
• We may also reserve some of the class sessions for working on assignments so that students have a chance to get a head start to ask me questions.

Discord

To facilitate informal, class-wide discussions about the class, we will have a Discord channel for the class. My goal is for this to turn into a flourishing area to work through confusion and to share ideas as a group. All questions are welcome! We will also keep the CS 371 Discord channel open during class so we can do more advanced polling than Zoom allows.

In addition to the regular channel, we will be holding drop-in office hours on the Ursinus ACM Discord channel, since office hours are joint between my classes. You can find it under the "Chit Chat" Category, as shown below

Please do not send me direct messages or anything of a sensitive nature (e.g. grades) over Discord. Instead, use Microsoft Teams or e-mail for that, since those transactions are locked down better under Ursinus control.

Microsoft Teams

For one on one direct messages with me, and for buddy group coding with screen sharing, we will be using Microsoft Teams, which is linked to your Office suite through Ursinus, so you are automatically enrolled. This is an easy platform for students to initiate video sharing, so I highly recommend it for group work.

---

Readings

There is no official textbook for the course. Knowledge will be imparted via the pre-class modules and links on the schedule.

---

Deliverables

Pre-Class Modules

In addition to the readings, the class will revolve around a set of online modules that you complete before class on your own time (click here for an example from last fall in CS 174). This will be more active than reading the textbook alone, and you will get direct feedback as you're learning and writing code directly in the web browser.

Programming Assignments

Since the goal is to give you a broad overview of digital music processing, the bulk of the grade in the course will be earned by completing 8 medium scale programming assignments throughout six units in the course. Be sure to start them early, since debug time can often be unpredictable! Please refer to the collaboration and sharing rules for these assignments.

A word on patience and debugging

If you're taking this course, then you've certainly had experience with debugging, but it is a skill you will still need to work on, so you should expect to hit some roadblocks. In fact, it is time consuming and difficult even for very experienced programmers. So do not be hard on yourself if your programs don't work the first time around (they rarely do, even if you've been programming for decades!). But be sure you leave yourself adequate time to work on the assignments, because the amount of time it takes to resolve issues can be unpredictable.

---

Grading

Breakdown

Pre-Class Modules	15%
Attendance / Participation	5%
Programming Assignments	80%

Flexible Submission Policy

In the absence of accommodations or communication with me, all assignments are due at 11:59PM EST on the date(s) stated on the schedule. Students can turn in those assignments past the deadlines, and the scores will be adjusted as follows:

• -5% for work submitted between 1 minute - 6 hours late
• -10% for work submitted up to 12 hours late
• -15% for work submitted up to 24 hours late
• -25% for work submitted up to 48 hours late
• -40% for work submitted up to 96 hours late
• -50% for work submitted more than 96 hours late

Letter Grades

Letter grades will be assigned on the scale below at the end of the course.

A+: 99+ A: 93-98.9 A-: 89.5-92.9

B+: 87-89.49 B: 83-86.9 B-: 79.5-82.9

C+: 77-79.49 C: 73-76.9 C-: 69.5-72.9

D+: 67-69.49 D: 63-66.9 D-: 59.5-62.9

F: 59.49 and below

---

Classroom Environment

Inclusive Environment

Computer science is a field that has historically been and continues to be steeped in inequalities. We will do our best to put the topics we're working on into the appropriate historical context, and to address broader societal issues that are related to the code that we write. We will strive to do better in our course, with an honest look at where we have been in the field. To that end, my goal is to foster a environment in which students across all axes of diversity feel welcome and valued, both by me and by their peers. Axes of diversity include, but are not limited to, age, background, beliefs, race, ethnicity, gender/gender identity/gender expression (please feel free to tell me in person or over e-mail which pronouns I should use), national origin, religious affiliation, and sexual orientation. Discrimination of any form will not be tolerated.

.Furthermore, I want all students to feel comfortable expressing their opinions or confusion at any point in the course, as long as they do so respectfully. As I will stress over and over, being confused is an important part of the process of learning computer science. Learning computer science and struggling to grow is not always comfortable, but I want it to feel safe. In other words, I will regularly keep you at the boundary of your comfort zone with challenging, real-world assignments, but I want you to feel comfortable with me and your peers and respected as a learner during the process.

Finally, I am aware that, particularly during the pandemic, there are a variety of factors that may make it difficult to perform at your best level in class. At Ursinus, we are fortunate to have quite a mix of students from different backgrounds, many of whom need to work part time, and an increasing number of whom are commuters and have family obligations. If you find yourself having difficulty performing at the level that you want and/or turning assignments in on time because of any of these issues, please communicate with me, and we can come up with a solution together (I will gently reach out if I notice any slips even if you don't communicate). This is a foundational course for the CS major, and I want to work to keep your excitement alive, regardless of your personal circumstances. You belong in CS!

---

Participation

Overall Participation Score / Classroom Etiquette

Students are expected to attend class either in person or via the Zoom sessions during class time with their camera on, and they will be graded for attendance.

• Points will be evenly divided among all classes.
• Students with an unexcused absence from a class will lose all points for that class.
• It is imperative that students show up on time, because important announcements may happen at the beginning of every class.

Furthermore, we're shooting for engagement over mere attendance; students are expected to be active in class exercises and to be fully invested in the class (i.e. no internet browsing). This is especially important during the pandemic. Students who are not engaged risk losing points from participation on that particular day.

Maximizing Your Communal Experience

Here are ways students can maximize their experience as a class community, and which could lead to extra credit in certain situations.

• Helping to teach a student a topic during office hours.
• Certain calls for participation in class
• Particularly helpful or insightful messages on Discord
• Finding mistakes in the book or on the assigned homework

Discord Communication Policy

Since this is a class-wide communication, the following rules apply to online communication

• Students are expected to be respectful and mindful of the classroom environment and inclusivity standards. They are equally applicable to a virtual environment as they are in class.
• Students are not permitted to publicly share direct answers or questions which might completely give away answers to any homework problems. When in doubt, please send me a direct message on Microsoft Teams.
• I will attempt to answer questions real time during my virtual office hours. Otherwise, I will make every attempt to respond within 24 hours on weekdays. I cannot be expected to respond at all on Saturdays or Sundays or outside of 10AM-8PM on weekdays, so please plan accordingly. (Of course, students can and should still respond to each other outside of these intervals, when appropriate).

The points above are part of a more general term referred to as "netiquette." Please refer to the chart below, provided by Touro College

Collaboration Policy

Overall Philosophy

The collaboration policy for this class walks the line between encouraging openness and collaboration during a challenging learning process, while also making sure that each students is progressing technically at an individual level without relying on 100% on other classmates. Communication between students is allowed (and encouraged!) on most assignments, but it is expected that every student's code or writeups will be completely distinct. Please do not copy code off of the Internet. Please cite any sources in addition to materials linked from the course website that you used to help in crafting your code and completing the assignment.

Assignment Buddies

To encourage collaboration, students will be allowed (not required) to choose one or more "buddies" to work "near" during the programming assignments. Students are still expected to submit their own solutions, but they are allowed to provide substantial help to each other, and even to look at each others' code during the process. Students should indicate their buddies in the README upon assignment submission. Please let me know if you would like a buddy but are having trouble finding one.

Collaboration Scenarios Table

Below is a table spelling out in more detail when and how you are allowed to share code with people (table style cribbed from Princeton CS 126).

* You may view and copy code from class exercises and class resources without citing them, but you should not copy solutions from previous semesters that the instructor may have provided

NOTE: The terms "exposing" and "viewing" exclude sending or ingesting electronically, which would be considered copying. Exposing and viewing are normally done in the context of in-person working or in the help room. Since we are working remotely, what this means is that buddies can screen share as they are working through things, but they should not send code directly.

NOTE ALSO: "Other people" includes internet sources.

If the collaboration policy has been violated in any way, regardless of intent, then it may be an academic dishonesty case, and it will be referred to the Associate Dean for Academic Affairs. I am required to make this report in every occurrence, so it is best to speak with me first if there are any questions about the policy or expectations. You should feel free to have these conversations with me anytime prior to making your submission without fear of penalty.

On a more personal note, though a willful violation of academic honesty may seem merely transactional to a student, faculty take violations very personally, as they are disrespectful to the time and effort we put into our courses. I would also like to emphasize that your reputation is much more important than your grades. The recommendations we as faculty write go a long way, and we are much happier to write positive recommendations for students with lower grades who show grit and growth than we are to write recommendations for students with higher grades who have cheated.

---

Other Resources / Policies

Accommodations

In addition to our general awareness diversity, Ursinus College is also committed to providing reasonable accommodations to students with disabilities. Students with a disability should contact the Directory of Disability Services ASAP. Dolly Singley is located in the Center for Academic Support in the lower level of Myrin Library. Please visit this link for more information on the process. I will do my best to accommodate your requests, and they will be kept completely confidential.

One on One Tutoring

One on one tutoring for up to two hours per week is available through the institute for student success. Please click here to fill out a Qualtrics survey if you'd like to take advantage of this.

Let's Talk

Mental health care is increasingly recognized as a crucial service for the undergraduate population. Please visit this link for more information about complementary counseling services provided by the college. The Wellness Center has a virtual drop-in crisis hour at 2-3 pm each weekday, which is available for students in crisis who need to be seen immediately by a clinician. If you are still hesitant to go, take me (Professor Tralie) as an example of someone who has benefited greatly from talk therapy in the past. I am happy to discuss this in office hours in more detail.

Title IX

Title IX is a federal law, under which it is prohibited to discriminate on the basis of gender. The Title IX Coordinator is available to receive inquiries and to investigate allegations in this regard.

Inclement Weather Policy

In the event that the College closes due to inclement weather or other circumstances, our in-person class sessions, drop-in office hours, or other meetings will not be held. I will contact you regarding our plan with regard to rescheduling the class or the material, any assignments that are outstanding, and how we can move forward with the material (for example, any readings or remote discussions that we can apply). If necessary, I may schedule online virtual sessions in lieu of class sessions, and will contact you with information about how to participate in those. I will communicate this plan to the department so that it can be posted on my office door if it is feasible to do so. This policy and procedure will also apply in the event that the College remains open but travel conditions are hazardous or not otherwise conducive to holding class as normal. Should another exigent circumstance arise (for example, illness), I will follow this policy and procedure as well.