instrument-classification/music21_generating_pitches.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generating simple audio samples with music21\n",
    "\n",
    "We'd like to synthesize simple audio samples containing a single note or a chord. The samples, however, should be parameterized by several attributes. We'd like to modify:\n",
    "\n",
    "- pitch\n",
    "- [precise frequency]\n",
    "- instrument\n",
    "- volume\n",
    "- duration\n",
    "- location in the sample\n",
    "\n",
    "The main output should be MIDI which can be synthesized to audio. Besides that we'd like to store the parameters and metadata like the the pitch class, frequency, etc.\n",
    "\n",
    "In this notebook we'll explore how to acomplish this with the `music21` library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import music21\n",
    "\n",
    "from music21.chord import Chord\n",
    "from music21.duration import Duration\n",
    "from music21.instrument import Instrument\n",
    "from music21.note import Note, Rest\n",
    "from music21.stream import Stream\n",
    "from music21.tempo import MetronomeMark\n",
    "from music21.volume import Volume"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "data_dir = 'data/working/example-parametric-note'\n",
    "os.makedirs(data_dir, exist_ok=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We're about to create a Note object which represents a single note and both its pitch and duration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<music21.note.Note C>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Note('C')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "s = Stream([Note('C')])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we have MuseScore installed, we can we the music sheet representation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<music21.ipython21.objects.IPythonPNGObject at 0x1046ac8d0>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "                <div id='midiPlayerDiv145'></div>\n",
       "                <link rel=\"stylesheet\" href=\"http://artusi.xyz/music21j/css/m21.css\" \n",
       "                    type=\"text/css\" />\n",
       "                <script>\n",
       "                require.config({\n",
       "                    paths: {'music21': 'http://artusi.xyz/music21j/src/music21'} \n",
       "                }); \n",
       "                require(['music21'], function() { \n",
       "                               mp = new music21.miditools.MidiPlayer();\n",
       "                               mp.addPlayer('#midiPlayerDiv145'); \n",
       "                               mp.base64Load('data:audio/midi;base64,TVRoZAAAAAYAAQABBABNVHJrAAAAFwD/AwAA4ABAiACQPFqIAIA8AIgA/y8A'); \n",
       "                        });\n",
       "                </script>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "s.show('midi')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that there's some rest at the beginning and end of the MIDI file. It looks like a quarter-note rest. The reason is that \"MIDI controllers may not be able to play notes at deltaTime=0\" See: https://groups.google.com/d/msg/music21list/ne_P_ZUvRNk/XynaiODzAgAJ\n",
    "\n",
    "Anyway this might be good in the audio processing, since there will be a space for the windowing function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'data/working/example-parametric-note/c.midi'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.write('midi', data_dir + '/c.midi')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Properties of the Note"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<music21.note.Note C>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n = Note('C')\n",
    "n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def describe_note(note):\n",
    "    p = note.pitch\n",
    "    print(note)\n",
    "    print('pitch:', note.pitch)\n",
    "    print('duration:', note.duration)\n",
    "    print('name:', p.name)\n",
    "    print('full name:', p.fullName)\n",
    "    print('pitch class:', p.pitchClass)\n",
    "    print('octave:', p.octave)\n",
    "    print('frequency', p.frequency, 'Hz')\n",
    "    print('midi:', p.midi)\n",
    "    print('pitch space:', p.ps) # like MIDI, but floating point"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<music21.note.Note C>\n",
      "pitch: C\n",
      "duration: <music21.duration.Duration 1.0>\n",
      "name: C\n",
      "full name: C\n",
      "pitch class: 0\n",
      "octave: None\n",
      "frequency 261.6255653005985 Hz\n",
      "midi: 60\n",
      "pitch space: 60.0\n"
     ]
    }
   ],
   "source": [
    "describe_note(n)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating Note with parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<music21.note.Note E>\n",
      "pitch: E\n",
      "duration: <music21.duration.Duration 1.0>\n",
      "name: E\n",
      "full name: E\n",
      "pitch class: 4\n",
      "octave: None\n",
      "frequency 329.62755691286986 Hz\n",
      "midi: 64\n",
      "pitch space: 64.0\n"
     ]
    }
   ],
   "source": [
    "# different note in the default octave\n",
    "describe_note(Note('E'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<music21.note.Note G#>\n",
      "pitch: G#3\n",
      "duration: <music21.duration.Duration 1.0>\n",
      "name: G#\n",
      "full name: G-sharp in octave 3\n",
      "pitch class: 8\n",
      "octave: 3\n",
      "frequency 207.65234878997245 Hz\n",
      "midi: 56\n",
      "pitch space: 56.0\n"
     ]
    }
   ],
   "source": [
    "# a note in the specific octave\n",
    "describe_note(Note('G#3'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<music21.note.Note E->\n",
      "pitch: E-2\n",
      "duration: <music21.duration.Duration 1.0>\n",
      "name: E-\n",
      "full name: E-flat in octave 2\n",
      "pitch class: 3\n",
      "octave: 2\n",
      "frequency 77.78174593052012 Hz\n",
      "midi: 39\n",
      "pitch space: 39.0\n"
     ]
    }
   ],
   "source": [
    "# note specified by its octave and pitch class within an octave\n",
    "describe_note(Note(octave=2, pitchClass=3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<music21.note.Note A>\n",
      "pitch: A0\n",
      "duration: <music21.duration.Duration 1.0>\n",
      "name: A\n",
      "full name: A in octave 0\n",
      "pitch class: 9\n",
      "octave: 0\n",
      "frequency 27.499999999999947 Hz\n",
      "midi: 21\n",
      "pitch space: 21.0\n"
     ]
    }
   ],
   "source": [
    "# note specified by its integer MIDI number\n",
    "describe_note(Note(midi=21))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<music21.note.Note A>\n",
      "pitch: A0(+25c)\n",
      "duration: <music21.duration.Duration 1.0>\n",
      "name: A\n",
      "full name: A in octave 0 (+25c)\n",
      "pitch class: 9\n",
      "octave: 0\n",
      "frequency 27.899996710781842 Hz\n",
      "midi: 21\n",
      "pitch space: 21.25\n"
     ]
    }
   ],
   "source": [
    "# microtonal pitch using the pitch space attribute (like MIDI but floating point)\n",
    "describe_note(Note(ps=21.25))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Changing duration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<music21.note.Note A>\n",
      "pitch: A0\n",
      "duration: <music21.duration.Duration 0.5>\n",
      "name: A\n",
      "full name: A in octave 0\n",
      "pitch class: 9\n",
      "octave: 0\n",
      "frequency 27.499999999999947 Hz\n",
      "midi: 21\n",
      "pitch space: 21.0\n"
     ]
    }
   ],
   "source": [
    "# note with duration of half of a quarter note\n",
    "note = Note(midi=21, duration=Duration(0.5))\n",
    "describe_note(note)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<music21.note.Note A>\n",
      "pitch: A0\n",
      "duration: <music21.duration.Duration 2.5>\n",
      "name: A\n",
      "full name: A in octave 0\n",
      "pitch class: 9\n",
      "octave: 0\n",
      "frequency 27.499999999999947 Hz\n",
      "midi: 21\n",
      "pitch space: 21.0\n"
     ]
    }
   ],
   "source": [
    "# note with duration of half of a quarter note\n",
    "note = Note(midi=21, duration=Duration(2.5))\n",
    "describe_note(note)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Changing volume\n",
    "\n",
    "Volume can be specified by parameters:\n",
    "\n",
    "- `velocity` with range from 0 to 127 or by\n",
    "- `velocityScalar` with range from 0.0 to 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<music21.volume.Volume realized=0.0>\n",
      "<music21.volume.Volume realized=0.25>\n",
      "<music21.volume.Volume realized=0.5>\n",
      "<music21.volume.Volume realized=1.0>\n"
     ]
    }
   ],
   "source": [
    "for v in [0, 32, 64, 127]:\n",
    "    print(Volume(velocity=v))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<music21.volume.Volume realized=0.0>\n",
      "<music21.volume.Volume realized=0.25>\n",
      "<music21.volume.Volume realized=0.5>\n",
      "<music21.volume.Volume realized=1.0>\n"
     ]
    }
   ],
   "source": [
    "for v in [0, 0.25, 0.5, 1.0]:\n",
    "    print(Volume(velocityScalar=v))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<music21.volume.Volume realized=0.71>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Chord(['C']).volume"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<music21.volume.Volume realized=0.25>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c = Chord([Note('C')])\n",
    "c.volume = Volume(velocityScalar=0.25)\n",
    "c.volume"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How to set tempo?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metronome = MetronomeMark(number=60)\n",
    "metronome.durationToSeconds(Duration(1.0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just add a metronome mark at the beginning of the stream."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<music21.ipython21.objects.IPythonPNGObject at 0x10406e940>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Stream([MetronomeMark(number=60), Note('C')]).show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sequence of notes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def make_instrument(id):\n",
    "    i = Instrument()\n",
    "    i.midiProgram = id\n",
    "    return i\n",
    "\n",
    "def chord_with_volume(chord, volume):\n",
    "    chord.volume = Volume(velocityScalar=volume)\n",
    "    return chord\n",
    "\n",
    "def generate_single_note(midi_number, midi_instrument=0, volume=1.0, duration=1.0, tempo=120):\n",
    "    \"\"\"\n",
    "    Generates a stream containing a single note with given parameters.\n",
    "    midi_number - MIDI note number, 0 to 127\n",
    "    midi_instrument - MIDI intrument number, 0 to 127\n",
    "    duration - floating point number (in quarter note lengths)\n",
    "    volume - 0.0 to 1.0\n",
    "    tempo - number of quarter notes per minute (eg. 120)\n",
    "\n",
    "    Note that there's a quarter note rest at the beginning and at the end.\n",
    "    \"\"\"\n",
    "    return Stream([\n",
    "        MetronomeMark(number=tempo),\n",
    "        make_instrument(midi_instrument),\n",
    "        chord_with_volume(Chord([\n",
    "            Note(midi=midi_number, duration=Duration(duration))\n",
    "        ]), volume)\n",
    "    ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "                <div id='midiPlayerDiv345'></div>\n",
       "                <link rel=\"stylesheet\" href=\"http://artusi.xyz/music21j/css/m21.css\" \n",
       "                    type=\"text/css\" />\n",
       "                <script>\n",
       "                require.config({\n",
       "                    paths: {'music21': 'http://artusi.xyz/music21j/src/music21'} \n",
       "                }); \n",
       "                require(['music21'], function() { \n",
       "                               mp = new music21.miditools.MidiPlayer();\n",
       "                               mp.addPlayer('#midiPlayerDiv345'); \n",
       "                               mp.base64Load('data:audio/midi;base64,TVRoZAAAAAYAAQABBABNVHJrAAAAJAD/AwAAwAAA4ABAAMAAAP9RAwehIIgAkDx/iACAPACIAP8vAA=='); \n",
       "                        });\n",
       "                </script>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "generate_single_note(60).show('midi')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's make a sequence. Note that by just passing a list of notes to the Stream we get a chord, not a sequence, so we must append each note separately."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "s = Stream()\n",
    "s.append(make_instrument(50))\n",
    "s.append(Note(midi=60))\n",
    "s.append(Note(midi=64))\n",
    "s.append(Note(midi=67))\n",
    "s.write('midi', data_dir + '/sequence_separated.midi')\n",
    "s.show('midi')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the previous example we see, that notes may overlap. So let's add some rests to make better separation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "                <div id='midiPlayerDiv809'></div>\n",
       "                <link rel=\"stylesheet\" href=\"http://artusi.xyz/music21j/css/m21.css\" \n",
       "                    type=\"text/css\" />\n",
       "                <script>\n",
       "                require.config({\n",
       "                    paths: {'music21': 'http://artusi.xyz/music21j/src/music21'} \n",
       "                }); \n",
       "                require(['music21'], function() { \n",
       "                               mp = new music21.miditools.MidiPlayer();\n",
       "                               mp.addPlayer('#midiPlayerDiv809'); \n",
       "                               mp.base64Load('data:audio/midi;base64,TVRoZAAAAAYAAQABBABNVHJrAAAAMQD/AwAAwDIA4ABAAMAyiACQPFqIAIA8AJAAkEBaiACAQACQAJBDWogAgEMAiAD/LwA='); \n",
       "                        });\n",
       "                </script>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "s = Stream()\n",
    "s.append(make_instrument(50))\n",
    "s.append(Note(midi=60))\n",
    "s.append(Rest(duration=Duration(2.0)))\n",
    "s.append(Note(midi=64))\n",
    "s.append(Rest(duration=Duration(2.0)))\n",
    "s.append(Note(midi=67))\n",
    "s.write('midi', data_dir + '/sequence_separated.midi')\n",
    "s.show('midi')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It might be useful to generate a big sequence into a single MIDI file with notes being separated and then cutting the audio to pieces, rather than producing a lot of tiny MIDI files and calling FluidSynth for each. However, there's a risk of spilling sound across the samples."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}