Rendered at 22:24:40 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
abetusk 4 hours ago [-]
Power law distributions are specializations of the more general Levy stable distributions [0] [1]. Levy stable distributions answer the following question:
Given that the sum of independent and identically distributed random variables that converge to a distribution, what is the distribution they converge to?
If you answered Gaussian, you'd be wrong. The correct answer is Levy stable. There was no condition on finite variance. When variance can be infinite, Levy stable, or power law tail distributions, is the result. When the variance is finite, a Gaussian is the limiting distribution and, consequently, a Gaussian distribution is part of the family of Levy stable distributions.
The stability quality is the reason why the Levy stable (aka power law tail) distributions show up all over the place. If you've ever heard that the reason why the Normal distribution is called "normal", because the sums of (finite variance) random variables converges to a Gaussian, the same reasoning applies to the Levy stable. In some sense, Levy stable distributions are more normal than the normal distribution. My opinion is that infinite variance is hard for people to wrap their heads around, so they reject the premise.
Unfortunately I don't have a good answer for what the article brings up about the Fourier transform, but I'm almost positive that this can be answered with Levy stable distributions in mind. I will say that the distribution is often characterized by it's characteristic function. A short perusal of Wikipedia talks about Levy stable distributions being closed under Fourier transforms, which is what the article is talking about.
When I was learning DSP, I was surprised by the fact that generating pink (1/f) noise, sample by sample, is not mathematically easy at all.
One practical approach is passing white noise (every sample independent random) through a "pinking filter", which is a sum of a bunch of lowpass filters at different frequencies, so that the sum of their cutoff "knees" approximates the frequency curve of pink noise. Another approach is generating in chunks using Fourier transform from a desired shape. Given the ubiquity of 1/f noise, you'd think there would be a simpler and more direct algorithm, but no.
stagger87 5 hours ago [-]
If you change your Fourier transform approach into a time domain convolution, then you can generate it sample by sample. White noise -> FIR filter, this is a pretty simple and direct algorithm as far as DSP goes. The wiki on pink noise mentions this. I've done the FIR filter approach to simulate 1/f phase noise.
5 hours ago [-]
threatripper 12 hours ago [-]
1/f noise basically kills averaging. You collect more signal but at the same time equally more noise.
DoctorOetker 7 hours ago [-]
1/f noise does not truly kill averaging, observe how for increasing f, the noise spectrum looks "white" locally, but with a decreasing noise power for ever higher frequencies.
the 1/f noise at significant power levels is restricted to the lower and lower frequencies, effectively a slowly varying reference ("0") voltage of the amplifier, measuring ADC ground every other sample effectively recalibrates the offset voltage of the amplifier, think of "correlated double sampling".
Effectively measuring in sequence 0V, signal, 0V, signal, ... moves the signal of interest to a higher frequency band, where the 1/f noise is more tame.
When a cliff blocks the way of a vehicle, we don't say "cliffs kill vehicle travel", insteas we just drive around it...
rcxdude 12 hours ago [-]
Yup, it has the fun property that you'll get the same error in your estimate of the mean regardless of averaging time (though since it's not stationary this isn't even really that well defined).
(though of course, a random walk means you'll get even worse as you measure for longer...)
mturmon 5 hours ago [-]
Yes. My PhD advisor had a research interest in axiom systems for probability that are weaker than the familiar Kolmogorov axioms, which are sometimes abbreviated "CMP" for "conventional mathematical probability".
I'll try to remember the setup. The CMP axioms imply that, in a shift-invariant system X(t) (which is a different class than "stationary" -- not necessarily implying existence of second moments), if the mean of X(t) exists finite, then a long-term average of X(t) must converge.
However, you can observe time-invariant physical systems (such as a noisy resistor in a static environment) with spectra that obey the 1/f law down to very low frequencies (i.e., over very long time baselines) -- the time average does not converge. My advisor had a stack of magnetic tapes on his bookshelf with such samples.
These systems would seem to be disobeying the axioms of CMP, thereby motivating searches for alternative formulations that are more general.
plus 8 hours ago [-]
This makes me think of the Cauchy distribution, a probability distribution whose average follows the distribution itself rather than converging (hence, the distribution has no "mean" despite being symmetric). Is there any connection here, or is that just a coincidental similarity?
TeMPOraL 15 hours ago [-]
Just as interesting and surprising to me is that this 0-100Hz line is unexplained. Given that signal processing is the foundation of pretty much all digital technology, as well as the analog technologies that came before, I've kind of assumed every segment of a frequency plot be well studied and understood, with half a dozen of names to choose for (doublesine quefrency this, Kowalski-Shannon that...) and a heap of decades old textbooks about it.
I confirm, the low frequency range looks weird on every DFT plot I ever saw, particularly the audio ones. I just assumed it has something to do with ADC and is probably explained on Wikipedia. It's literally one of the last place on Earth when I'd expect to find unsolved mysteries.
rcxdude 14 hours ago [-]
It's not completely unexplained. Roughly speaking you can get that power spectrum in the limit when you are adding up many different events where the magnitude of the event is inversely proportional to its likelihood (and in practice, there is a limit to the magnitude of the events that will cause it to level off at some point, but for some processes this is not measurable even over decades). The main mystery in most cases is what exactly is the physical process that is causing it. For some electronics it looks like it is due to trapped charges sometimes tunneling around, but it doesn't explain every case of it in electronics let alone everything else. Convective thermal effects can also be a good candidate in a lot of systems, since turbulence also has 1/f noise properties.
(Also, the low frequency range on a DFT can look weird for reasons other than noise: it'll also tend to rise up if there's any longer-term structure to the signal as well, so you need to be careful interpreting them blindly if you're trying to measure noise)
tiazumdove 13 hours ago [-]
Low frequencies are studied quite extensively, especially the mHz-10Hz region for noise characterization of solid state materials. 1/f is quite well studied depending on your field. In semiconductor physics, for example, one possible explanation is electrons trapped in defects or on charged islands and then slowly trickling down. Of course this explanation cannot be used in other fields where 1/f noise shows up as well. The problem is that no model gives a satisfying answer as to why it occurs therefore its unexplained. Lack of model doesn't mean that you can't engineer your way around 1/f noise for example the chopper amp works so well because it shifts away towards frequencies above the 1/f cutoff.
analog31 10 hours ago [-]
Compounding the mystery is that it crops up everywhere, not just electronic noise plots. But outside of a few systems such as electronic noise in a laboratory setting, it's phenomenally hard to measure. For one thing, to get into the 1/f domain, you have to measure things for a long time. And the noise measurement itself is noisy. And the number of things that you need to control, such as environmental conditions, increases.
So its existence is often largely treated as an empirical rule of thumb rather than having a specific physical cause.
The other thing to note is that the noise plot in a dataset is probably a curve fit.
jhallenworld 6 hours ago [-]
1/f noise means that if you wait long enough an asteroid will hit the earth or the sun will go nova, etc.
Surely there is some connection to entropy.
lars 14 hours ago [-]
The comment section at the bottom of the article is pretty interesting. 20 years of people thinking about this.
AnthonBerg 12 hours ago [-]
As noise, its 1/f is 0.748544393 years per comment, haha.
(The frequency is 42.3338481 nanohertz.)
mturmon 16 hours ago [-]
The piece ends with the observation that maybe the fact that 1/f noise is its own Fourier transform is a clue.
Turns out this property is not unusual. There are many such pairs - there’s a reasonably well-known journal paper with a construction technique.
fch42 15 hours ago [-]
The paper linked at the "see also" section ?
(thx)
shiandow 10 hours ago [-]
I think 1/f is the uniform measure for scaling rather than translations. It's to multiplication what the standard uniform measure is for addition.
If you want to be boring you could call it the uniform distribution for log frequency.
Wikipedia has an article with some other information on pink noise (a more common name), including a random generator: https://en.wikipedia.org/wiki/Pink_noise. Some music generation algorithms use pink noise, as it (supposedly) strikes a better balance between randomness and predictability.
rcxdude 13 hours ago [-]
Pink noise is also perceptually flat noise, because it contains the same energy in each octave (or decade). 'truly' white noise (equal energy for equal bandwidth) tends to sound quite tinny/hissy in comparison.
kadoban 13 hours ago [-]
I went through a phase of using ~whitenoise while working to block out distractions.
Actual white noise indeed sounds really bad and grating. The best for me was a mix of pink and brown noise, pink for an ~equal baseline and brown to make it sound a little more mellow.
I suspect most/all generators meant to block out noise do something similar. It really sounds pretty bad without that, especially for anything more than a few seconds.
rcxdude 12 hours ago [-]
Yeah. pink noise is often confused with white noise in audio because it looks flat on a lot of equaliser displayers (because they show energy per octave instead of energy per Hz).
peter_d_sherman 5 hours ago [-]
My favorite part of the article:
>"Here is something even more interesting. As you approach α = -1, the time domain approaches a shape of t-1, and the frequency domain approaches a flat magnitude with a zero phase. However, a flat magnitude and zero phase corresponds to a delta function, δ(t), in the time domain."
>"Indeed, Heaviside introduced the δ-function in his work on electromagnetism and electrical engineering.[14] In a 1963 interview, Dirac stated, "All electrical engineers are familiar with the idea of a pulse, and the δ-function is just a way of expressing a pulse mathematically."[15]"
Given that the sum of independent and identically distributed random variables that converge to a distribution, what is the distribution they converge to?
If you answered Gaussian, you'd be wrong. The correct answer is Levy stable. There was no condition on finite variance. When variance can be infinite, Levy stable, or power law tail distributions, is the result. When the variance is finite, a Gaussian is the limiting distribution and, consequently, a Gaussian distribution is part of the family of Levy stable distributions.
The stability quality is the reason why the Levy stable (aka power law tail) distributions show up all over the place. If you've ever heard that the reason why the Normal distribution is called "normal", because the sums of (finite variance) random variables converges to a Gaussian, the same reasoning applies to the Levy stable. In some sense, Levy stable distributions are more normal than the normal distribution. My opinion is that infinite variance is hard for people to wrap their heads around, so they reject the premise.
Unfortunately I don't have a good answer for what the article brings up about the Fourier transform, but I'm almost positive that this can be answered with Levy stable distributions in mind. I will say that the distribution is often characterized by it's characteristic function. A short perusal of Wikipedia talks about Levy stable distributions being closed under Fourier transforms, which is what the article is talking about.
[0] https://en.wikipedia.org/wiki/L%C3%A9vy_distribution
[1] https://en.wikipedia.org/wiki/Stable_distribution
One practical approach is passing white noise (every sample independent random) through a "pinking filter", which is a sum of a bunch of lowpass filters at different frequencies, so that the sum of their cutoff "knees" approximates the frequency curve of pink noise. Another approach is generating in chunks using Fourier transform from a desired shape. Given the ubiquity of 1/f noise, you'd think there would be a simpler and more direct algorithm, but no.
the 1/f noise at significant power levels is restricted to the lower and lower frequencies, effectively a slowly varying reference ("0") voltage of the amplifier, measuring ADC ground every other sample effectively recalibrates the offset voltage of the amplifier, think of "correlated double sampling".
Effectively measuring in sequence 0V, signal, 0V, signal, ... moves the signal of interest to a higher frequency band, where the 1/f noise is more tame.
When a cliff blocks the way of a vehicle, we don't say "cliffs kill vehicle travel", insteas we just drive around it...
(though of course, a random walk means you'll get even worse as you measure for longer...)
I'll try to remember the setup. The CMP axioms imply that, in a shift-invariant system X(t) (which is a different class than "stationary" -- not necessarily implying existence of second moments), if the mean of X(t) exists finite, then a long-term average of X(t) must converge.
However, you can observe time-invariant physical systems (such as a noisy resistor in a static environment) with spectra that obey the 1/f law down to very low frequencies (i.e., over very long time baselines) -- the time average does not converge. My advisor had a stack of magnetic tapes on his bookshelf with such samples.
These systems would seem to be disobeying the axioms of CMP, thereby motivating searches for alternative formulations that are more general.
I confirm, the low frequency range looks weird on every DFT plot I ever saw, particularly the audio ones. I just assumed it has something to do with ADC and is probably explained on Wikipedia. It's literally one of the last place on Earth when I'd expect to find unsolved mysteries.
(Also, the low frequency range on a DFT can look weird for reasons other than noise: it'll also tend to rise up if there's any longer-term structure to the signal as well, so you need to be careful interpreting them blindly if you're trying to measure noise)
So its existence is often largely treated as an empirical rule of thumb rather than having a specific physical cause.
The other thing to note is that the noise plot in a dataset is probably a curve fit.
Surely there is some connection to entropy.
(The frequency is 42.3338481 nanohertz.)
Turns out this property is not unusual. There are many such pairs - there’s a reasonably well-known journal paper with a construction technique.
If you want to be boring you could call it the uniform distribution for log frequency.
Actual white noise indeed sounds really bad and grating. The best for me was a mix of pink and brown noise, pink for an ~equal baseline and brown to make it sound a little more mellow.
I suspect most/all generators meant to block out noise do something similar. It really sounds pretty bad without that, especially for anything more than a few seconds.
>"Here is something even more interesting. As you approach α = -1, the time domain approaches a shape of t-1, and the frequency domain approaches a flat magnitude with a zero phase. However, a flat magnitude and zero phase corresponds to a delta function, δ(t), in the time domain."
Related:
https://en.wikipedia.org/wiki/Dirac_delta_function
>"Indeed, Heaviside introduced the δ-function in his work on electromagnetism and electrical engineering.[14] In a 1963 interview, Dirac stated, "All electrical engineers are familiar with the idea of a pulse, and the δ-function is just a way of expressing a pulse mathematically."[15]"