Author: Branchek

Eric Branchek – Prototype 3 and Elevator Pitch

In the past week, I was finally able to get my program to generate a song of more than one note. Using the AC/DC songs Hell’s Bells and Back in Black, it generated this song: https://clyp.it/k32srox2?token=fbd7d03bf3eacbb19e0a2b459dd2cf52

I noticed that in this case (and in the case of a few other songs I generated) the generated song seems to start off a bit random, and then settle into one riff. I think I have an idea of why, so I will look into that.

My next goal is to continue transcribing AC/DC songs, since having a sample size of two songs obviously makes it sound almost the same as the two songs. However, I am thinking about taking a different approach to it. Instead of feeding it whole songs, I’m thinking about feeding it riffs or melodies, and then allowing the program to piece these riffs or melodies together to create a song, since I’ve been using modern day music with more repetition than other styles.

I also gave an early version of my elevator pitch in class, which was along the lines of this:

“My project is a program that is able to create music based off influence from music scores available to it.”

Based off of feedback from my peers in class and from other friends outside of class, I realized that I needed to be more specific on what it does and how it works. Here is my new and improved elevator pitch:

“My project is a machine learning program that takes songs of a certain style as input, learns what makes them unique, and generates a song based off of the input. For example, you can give the program some scores of Beatles songs. It will figure out what features make all of those Beatles songs similar using an algorithmic process, and it will then generate a song that utilizes these features. The result is a what a Beatles song would sound like if they were still together.”

Eric Branchek – Prototype 2

For the second week, I had two main goals for my prototype: try to make it output songs with more than one note, and get a dedicated system to run the program and train the model.

Unfortunately, I was not able to get an output song that utilized more than one note, but I was able to get my hands on a Raspberry Pi to use as a dedicated system. However, I ran into some issues while trying to set up the SD card with the operating system and libraries I would need to run my program.

For the last week of prototyping, I intend to stick with my plan of transcribing some songs for my program to use (once it starts working correctly).

Eric Branchek – Industry Event

On November 17th, I attended a seminar at Princeton University called “Diving into TensorFlow 2.0”. TensorFlow is a library available for Python (and other programming languages) that makes machine learning coding much easier. In addition, I was introduced to some important resources that will assist me in bringing music into machine learning. Ultimately, I learned a lot about the basics of machine learning and certain key concepts that will be useful in my thesis.

After the presentation, I talked to the presenter, Josh Gordon, who works for Google and teaches deep learning at Columbia University. He said he personally has not done much music and machine learning work, but he did send me a library and a bunch of projects that use it: https://magenta.tensorflow.org/

There were many cool projects I found on this website, but there was a project similar to mine that stood out right here: https://midi-me.glitch.me/

After playing with it for a bit, I noticed that its attempt at making songs similar to the inputted songs was far from perfect. I think the reasoning for this is that it does not take direct user feedback on how accurate the result was. I think that aspect will help make such program more accurate, since it would be impossible to make it accurate with a limited data set and a lack of user input. Another way I thought of that would make a program more accurate is giving it some basic music theory; for example, knowledge of song structure and maybe commonly used scales.

I won’t lie: seeing “MidiMe” did make my project seem a lot more daunting. But if I go about developing it differently, I think I can still come up with a good result.

Eric Branchek – Prototype 1

For my first prototype, I was able to retrieve several MIDI files containing music from some old video games, write the base code for the program, and feed it the MIDIs to generate my first piece. The end result was a MIDI file containing 500 notes containing the same pitch, which was not what I intended. However, this allows me to continue with my plan this week of tweaking the code to make it work.

One thing I did notice that would help me would be having a dedicated machine to train my program, because training it on my laptop was painfully slow. I plan to set that up either this week or the following one, just so that training will go much quicker.

Eric Branchek – Research Update 2

I made the decision to use Python this week, due to its similarities to Java and C++ (versus LISP and Nyquist) and a large amount of applicable libraries. In particular, I found a library called Keras that was written to handle code for neural networks, making my job to program one much easier. As a introduction, I watched a two hour video that explained neural networks and showed basic applications of them with Keras. In addition to Keras, I came across a library called Music21 through an article on generating music with Keras. Music21 appears to make inputting MIDI scores into neural networks much easier.

I also thought a bit more on the data I plan to use. At the earlier prototyping stages, I figured it would be a good idea to write a bunch of short, simple songs myself, just to keep things more basic for scratching the surface of Keras. However, once I get past that stage, I will need some MIDI scores of well known songs so that I will have a clearer goal to test the output’s accuracy with. I have been thinking of albums I could use as test sets for this stage; preferably albums that have a consistent sound. Two albums that have come to mind are AC/DC’s Back in Black and Linkin Park’s Hybrid Theory. A part of my mind wonders if I will need bigger test sets, since albums tend to have only around 10 to 15 songs, but I don’t think that will be a glaring issue.

One simple idea for testing the accuracy of songs generated by the program involves asking a simple question: “What band does this sound like?” I would have to ask other people this question instead of myself, since their answer would be unbiased. The biggest issue with this method is that it is a lot easier to make a neural network if the accuracy is given in a percentage, rather taking in one of two possible answers. Therefore, I think the best way to going about testing for accuracy is by using this question: “From a scale of 1 to 10, 10 being most accurate, how much does this sound like *band name*?”

Here’s a list of the links I referenced in this post:

https://towardsdatascience.com/how-to-generate-music-using-a-lstm-neural-network-in-keras-68786834d4c5


https://www.youtube.com/watch?v=6g4O5UOH304


https://keras.io/


https://web.mit.edu/music21/

Eric Branchek – Research Update 1

My main research this week was looking at the different programming languages I specified last week and learning a bit about them. Since I already knew a decent amount of Java and C++, there wasn’t much to look into with them other than sound and MIDI libraries. If I were to pick between the two, I think I would pick C++, as I found a lot of useful C++ MIDI/sound libraries for my purposes.

The main language I worked with was Nyquist, due to the main textbook in my research field utilizing it. I found that the biggest upside with Nyquist was that making sound with it was very simple. Nyquist allowed me to define instruments in the code and directly make sound with them. However, in terms of coding the algorithm, I feel that a different language personally may be a bit easier for me to use.

One language I didn’t initially list that I think could be useful is Python. Python is becoming a very popular language nowadays, and like C++, it has a lot of useful libraries available.

In addition to checking out programming languages, I learned a little more about neural networks. I had previously learned a little bit about them, but I think using them in my code would be very useful in making the end result more accurate. I would use them as sort of an extension to my original plan using Markov chains.

Basically, the program would compose a few songs, and human input would give feedback on how accurately the song depicts the end goal. For example, on a scale of 1-10, I could say a song composed by the program is a 7 (10 being the most accurate, 1 being very inaccurate), another would be a 3, etc. The program would use this data to find the patterns that make a certain song “accurate”, and therefore be able to compose songs more intelligently. I was able to find a Python library for this purpose.

Eric Branchek – Final Concept

Algorithmic/automated composition is an idea that has been around for a very long time now, but has only had major advances in the last 30 years or so, thanks the the advances of computers. Even then, I haven’t found as much research on this topic as other more popular topics. I want to take this research and extend it into the entertainment field.

My project involves a program that takes music from an already existing band or genre, and creates its own music that will sound similar to the music categorized in the already existing band or genre. I have a heavy interest in music and computer science, and I wanted to combine these interests with this project because I feel they go very well together. In the future, I want to expand upon this project by incorporating machine learning elements in order to make this idea more free-form.

I feel that this field could have a great impact on entertainment. For example, it could be used to help musicians with writer’s block by giving them a base to go off of when writing music. It could also be used to aid with incorporating elements of a different, less familiar genre in one’s music. There are tons of possibilities that could be offered by my type of project.

The way I see it working starts by transcribing songs of a specific artist into a MIDI score. This MIDI score will contain the notes played by each instrument in the song as data. The program then takes in this data as input. With multiple MIDI scores, the program will analyze the song to figure out certain characteristics of the song, such as the key. Using the individual notes from each instrument, the program will construct instrument parts for a new song.

One way of doing this last step would be with Markov chains. The Markov chain could store data about the note choices in a song. Along with the individual note choices, it would store the probability of the next note being a certain note. For example, if an A note comes up in the song, it could store the chance of the next note being C as 50% (based on what happens when an A occurs in the rest of the song), the chance of the next note being E as 25%, and so on.

I plan to do some research on other methods that could help me with this step, as I see this being the most difficult part of my project. In addition to that, I will need to spend a decent amount of time transcribing songs to MIDI so that I have data to go off of.

Eric Branchek – Research Plan

I have written up a bunch of resources that I would like to check out.


People
Professor Nakra
Dan Shiffman
David Cope


Books
Algorithmic Composition: Paradigms of Automated Music Generation – Very detailed
Algorithmic Composition – A Guide to Composing Music with Nyquist – Goes over many concepts, but seems to focus on Nyquist
The Algorithmic Composer – An older book, but appears to be a good introduction


Resources
MIDI – For containing the data of existing music scores
LISP – A programming language
Nyquist – A programming language based on LISP
MaxMSP – A programming language created for music applications
Java – A programming language that I am familiar with
Python – A programming language that I am familiar with
C++ – A programming language that I am familiar with


Articles
https://ccrma.stanford.edu/~blackrse/algorithm.html
https://quod.lib.umich.edu/s/spobooks/bbv9810.0001.001

Eric Branchek – Revised Concept

Revised Concept – Algorithmic Composition

I am fairly set on my first idea with algorithmic composition. I feel like the scope can be small or widened, depending on the progress I make.

Here are some of the responses I have to comments that came up:

Parsing the music will be hard – My idea for this is to transcribe the songs into MIDI sheet music, so that my program can take that as input. While it would be cool to have actual audio parsed, I agree that that would take too much work.

Possibly beyond the scope of time – I feel confident that I can start small and build up as much as I can. I have already started a bit of work on this idea in my free time, so I have a feel for the challenge and how long it might take.

Incorporating aspects of Idea 2 into this idea – I think this is definitely a cool idea! I could use vaporwave songs to generate new vaporwave songs. The only tricky bit I see with this is that some aspects of vaporwave can be glitchy sounding, which is more of an audio thing that can’t be transcribed into audio. For example, it might pitch shift a semitone off a normal key.

Instead of that, maybe I could use the visual aspect from Idea 2. The only question I see with that is whether or not the visuals would be generated on-the-spot, too. If they would be, I would have to see how much work would go into that, because I do not have the background to really know right now.