Topics in Audio Processing & Music Technology

Content

• 01 DDSP: Differentiable Digital Signal Processing

  • 00:00:00 Introduction
  • 00:00:30 Magenta Research Project
  • 00:01:14 DDSP Library Overview
  • 00:01:36 DDSP Modules
  • 00:02:29 DDSP Processors Module
  • 00:04:16 DDSP Core Module
  • 00:06:41 DDSP Spectral Ops Module
  • 00:07:46 DDSP Original Paper

• 02 Differentiable Digital Signal Processing: Timbre Transfer - Deep Learning

  • 00:00:00 Timbre Transfer Demo
  • 00:02:55 Installing and Importing the DDSP Library
  • 00:03:10 Recording or Uploading Audio
  • 00:05:08 Loading a Model: Violin
  • 00:06:56 Resynthesizing Audio
  • 00:08:01 Modifying Conditioning
  • 00:09:06 Pre-trained Trumpet Model
  • 00:10:22 Pre-trained Flute2 Model
  • 00:11:18 Pre-trained Tenor Saxophone Model
  • 00:12:56 Pitch Shifting
  • 00:13:50 Demo Application Conclusions
  • 00:15:19 DDSP ICLR Paper: The International Conference on Learning Representations
  • 00:17:18 Install and Imports
  • 00:18:16 Gin Config Framework
  • 00:22:35 Recording Audio
  • 00:26:23 base64: Decode the Base64 encoded bytes-like object or ASCII string
  • 00:26:43 Audio bytes to Numpy
  • 00:27:44 Uploading an Audio File
  • 00:29:29 Colab Files Upload
  • 00:33:43 Log Magnitude Spectrogram
  • 00:36:40 DDSP Spectral Operations: Compute Magnitude
  • 00:37:11 DDSP Spectral Operations: Short-time Fourier Transform (STFT)
  • 00:38:15 Tensorflow Signal: Short-time Fourier Transform (STFT)
  • 00:39:37 DDSP Safe Log
  • 00:43:14 Numpy Rotate 90
  • 00:44:56 Pyplot Matshow
  • 00:46:33 DDSP Colab Utils: Play Audio
  • 00:49:52 Scipy IO Wavfile Write
  • 00:51:37 IPython Display Audio
  • 00:55:29 Computing Audio Features
  • 00:56:28 CREPE: A Convolutional Representation for Pitch Estimation
  • 00:57:39 CREPE ICASSP 2018 Paper
  • 01:06:16 DDSP Timbre Transfer Audio Features
  • 01:09:28 Computing F0
  • 01:10:23 Viterbi Algorithm
  • 01:13:38 Computing Loudness
  • 01:17:50 Dynamics, Intensity and Loudness
  • 01:21:12 A-weighting Function
  • 01:24:59 Plotting the Audio Features
  • 01:25:28 Loading a Pre-Trained Model
  • 01:27:05 Gsutil Tool
  • 01:28:22 tf.io.gfile
  • 01:29:19 Parsing Gin Config
  • 01:31:03 Gin Query Parameters
  • 01:35:57 Understanding the Pre-Trained Model
  • 01:36:16 DDSP Autoencoder Model
  • 01:37:13 DDSP Model Base Class
  • 01:39:01 DDSP ICLR Paper: Autoencoder Architecture
  • 01:39:43 Gin Config - Model Parameters
  • 01:42:17 DDSP Encoders
  • 01:44:12 DDSP Decoders
  • 01:45:47 Recurrent Neural Network (RNN)
  • 01:46:18 Gated Recurrent Units (GRUs)
  • 01:47:28 Harmonic Oscillator / Additive Synthesizer
  • 01:48:54 Filtered Noise / Subtractive Synthesizer
  • 01:50:08 Multi-Scale Spectral Loss
  • 01:51:52 DDSP Reverb
  • 01:52:21 Conclusion

• 03 Tensorflow + Tensorboard + Scikit-learn: Simple audio recognition: Recognizing keywords

  • 00:00:00 Introduction
  • 00:00:34 Simple audio recognition: Recognizing keywords Example
  • 00:03:53 Tensorflow Keras
  • 00:05:38 Tensorflow
  • 00:07:44 Seed for Experiment Reproducibility
  • 00:09:56 Speech Commands Dataset
  • 00:11:33 Downloading files using 'tf.keras.utils.get_file'
  • 00:15:07 Returning a list of files that match the given pattern(s) using 'tf.io.gfile.glob'
  • 00:16:19 Splitting the dataset into training, validation and testing subsets
  • 00:19:29 Scikit-learn
  • 00:20:14 Stratified ShuffleSplit
  • 00:24:38 Writing descriptive and efficient input pipelines using 'tf.data.Dataset'
  • 00:26:20 Reading the contents of a file using 'tf.io.read_file'
  • 00:27:14 Applying transformations across the elements of a dataset using '.map(...)'
  • 00:31:18 Spectrograms using 'tf.signal.stft'
  • 00:31:32 Zero-padding in Tensorflow
  • 00:34:00 Listening to audio files
  • 00:36:10 Plotting Spectrograms
  • 00:40:29 Batching the training and validation subsets using '.batch(...)'
  • 00:41:48 Caching elements of a dataset using '.cache(...)'
  • 00:42:20 Prefetching elements of a dataset unsing 'prefetch(...)'
  • 00:43:13 Preprocessing Layers: Resizing and Normalization
  • 00:43:48 Building a Sequential Model in Tensorflow Keras
  • 00:46:10 Printing a string summary of the network using '.summary()'
  • 00:46:46 Configuring the model for training using '.compile(...)'
  • 00:48:12 Configuring Tensorboard
  • 00:50:18 Training the Model using '.fit(...)'
  • 00:50:33 Visualization in Tensorboard
  • 00:52:20 Evaluating the Test set Performance
  • 00:53:39 Displaying a Confusion Matrix using '.tf.math.confusion_matrix(...)'
  • 00:54:40 Running Inference on an Audio File

YouTube Playlist