For the last few months, I have been fascinated by the strides possible in deep-learning. Deep-learning has far outperformed many traditional techniques in denoising. The question remains: can this be used for science? A reliable and robust denoising techique based on deep-learning could be useful in improving parameter estimation and also holds the key in potential breakthrough science i.e. fundamentally new discovery. I come across various kinds of spectra as part of my daily work. How to approach this problem?
Denoising: supervised vs unsupervised
For now, let’s be abstract without focussing on a particular problem. Consider an observed signal whose power spectrum is denotedby \(P(\omega)\). Let’s say we have some model, \(M(\omega)\) which can explain the broad features of the spectra fairly well. However, no model is perfect and we always need some additional terms which would take \(M(\omega)\) closer to \(P(\omega)\). $$ P(\omega) := M(\omega) + S(\omega, {p_i}) $$ There is, of course, some noise in the measurment which could be modelled as \(\mathcal{N}(\omega)\). For now, let’s only consider a simple additive noise. The statistics of noise isn’t important at the moment. $$ P^{\mathrm{model}}(\omega) := M(\omega, {\alpha_j}) + S(\omega, {p_i}) + \mathcal{N}(\omega) $$ One approach to denoising is to setup a deep network to learn to extract \(M(\omega)\) from \(P^{\mathrm{model}}\). This is a really bad idea in many cases because of the following reasons:
- The systematics \(S(\omega)\) may not be well understood or more complicated to model than a simple additive one
- The noise model may not be good. Especially in the cases of limited data, it may not always be easy to test noise models robustly
Hence, using \(P^\mathrm{model}(\omega)\) as a proxy for \(P(\omega)\) may not always result in a good denoised solution – a supervised learning technique learns the model really well and only that! What we need is an unsupervised technique, which can do model-independent denoising.