Transmitting information in noise: A practical tutorial

William H Payne

First posted
Monday January 10, 2005 07:57

Wednesday April 20, 2005 17:50


One purpose of this tutorial is to show by simple example how an n-bit information subchannel can be constructed in a channel transmitting both content and some noise or pure noise.

Intelligent Noise by Alfred Pfanstiehl published in 1962 raised awareness that information might be transmitted in noise. Is Pfahsniehl's article science fiction and total malarkey?

A second surfacing of the possibility of transmitting information in noise is the PhaserPhone.

A method to transmit a mux [multiplexer] channel number with data and noise is shown by practical example.

And the method doesn't require use of pseudorandom numbers! But pseudorandom numbers can be used if desired. And even Vernam ciphering can be used too.

The example method is related to accuracy of measurement and distribution sampling and why digitization should nearly always occur at a sensor.

Sensor value accuracy

Several thermometers of the same make and type are usually in a store.

One of the first things noticed is that they usually don't read exactly the same value. There will one or more thermometer[s] with the highest value and one or more with the lowest value.

Measurements of sensor values are usually most accurate for the high-order digits and less accurate for low-order digits.

Low-order digits may be, in fact, random numbers.

Binary real numbers

Let's represent the digital value, V, of a sensor reading be represented as a real number in binary.

For example

V = 1011.01

That's the binary point separating the integer portion of the value from the factional portion

is 11.25 decimal.

Here's a tutorial on converting integer and decimal portions of decimal numbers to binary. Spanish version.

Sampling from continuous distributions

Terminal bits [to the far right] of numbers sampled from continuous distribution become uniformly distributed. That is, these bits have have probability of 1/2 of being a one or zero.

Significant bits [leading bits], on the other hand, may not have a a 1/2 probability of being a one or zero.

The probability depends on the height and slope of the of the probability density.

Here's the 1970 article describing the "conditional bit" distribution sampling technique.

Basic idea is that the probability of the leading bit being a one is computed. Then conditional probability of the next less significant bit is computed. And so forth from left to right.

Depending on the shape of the density, at some point a uniform number may be appended to the previous computed bits since the distribution of terminal bits is becoming unform.

Our work in digit-by-digit computation from left to right is a direct descendant of the work of Henry Briggs [Born: Feb 1561 in Warleywood, Yorkshire, England Died: 26 Jan 1630 in Oxford, England].

Briggs worked in decimal. We work in binary.

Briggs predates Isaac Newton [1642 - 1727].

Henry Briggs U Adelaide.

Here's a tutorial on digit-by-digit computation of elementary functions.

The real world of measurement

Analog to digital converters claim to accurately convert anywhere from about 8 bits to 24 bits.

In the real world, getting 16 bits accuracy out of a measurement system is very difficult. Ten bits accuracy is tough.

Electrical noise, magnetic fields, power supply variations, temperature and humidity variation, vibration, ... and a host of other

external conditions upset any converter.

The more careful the design, then usually the more valid leading bits.

Double wall board shielding, optosiolation, separate analog and digital on-board power supplies have been shown to improve performance.

Temperature and other physical disturbance compensation helps too.

So in reality, what one usually has is the leading bits good data and the trailing bits random data.

A multiplexed analog to digital converter as only one a/d converter in the chip.

But the chip can measure, say, an example 4 separate input voltages. Eight channels are usually used. But we're using a simple Moe-type example.

Here a possible test set up.

Diagram 1

Rather than use an actual sensor, we'll use a 16 bit digital to analog converter to test the a/d converter.

We'll put an op amp buffer at the output of the d/a converter.

Buffering is a very good idea

You can adjust the output swing with a feedback resistor.

The same output goes into each of the four input channels. Setting the bits m1 and m2 determine which channel is being input into the a/d converter.

Many, if not most, a/d converters have an integral sample and hold amplified. When the sample and hold pin goes low, then the the voltage is "frozen" for the a/d converter. Otherwise the a/d converter would be trying to measure a changing voltage.

Our test is to set a value into the d/a converter.

Then we will read the a/d converter on each of its four channels.

Theoretically, if we put the binary number, specified in hexadecimal. 51C2 into the d/a converter with a unit gain to the buffer amplified, then we should read 51C2 out from all four channels.

But, alas, we live in the real world.

What we might see is agree on the leading ten bits and not too much agreement on the trailing six bits. The least significant bit will show zero correlation between the four channels on.

Let's admit it. The last six bits are, in reality, noise!

Since the last six bits are acknowledge noise, and noise is noise, any random trailing six bits can be substituted for an observed value.

Assume that bit 15 is the most significant and bit zero is the least significant bit.

Let's AND 1111 1111 1111 1100 with the values read from mux channels 0, 1, 2 and 3.

AND is defined

1 1 1
1 0 0
0 1 0
0 0 0

AND with 1 leaves unchanged all of the signal and noise bits. AND with 0 creates a noise-free channel containing zeros.

ANDing with a zero and one mask makes a neat digital filter.

For a 1 in the mask, all values are passed through the filter.

For a 0 in the mask, the filter perfectly removes all noise!

AND 1111 1111 1111 1100 means that bits 1 and 0 are always going to be zeros.

Now OR the mux channel number into bit positions 0 and 1.

OR is defined

1 1 1
1 0 1
0 1 1
0 0 0

Here's an example
1 The signal plus noise read from the a/d converter is 0110 1101 0010 0011.
2 ANDing with the mask gives 0110 1101 0010 0000
3 Suppose mux channel 2 is to be transmitted with the signal plus noise. Channel 2 is encoded 0000 0000 0000 00010.
4 So the signal plus noise plus channel number is 0110 1101 0010 0000 OR 0000 0000 0000 00010 = 0110 1101 0010 0010.

This way when a value is read from the converter and passed along to a calling computer, like a Windows PC machine, both the a/d value, some noise and mux channel are being transmitted.

Including the mux channel number in the low order bits of an a/d value and associated noise is not contrived idea.

Bill wrote dll, vxd, and wdm drivers for a Measurement Computing DAS 1400 series ISA bus a/d converter card.

The board controller stepping the mux. Assumption is that the mux stepped up one channel each time an a/d value was read.

HOWEVER, in some cases noise would cause the mux channel to skip. And it was impossible to read from the board which channel the mux was on! So it would be nice to have the mux channel number included with the signal and associated noise.

We have just created a 2 bit subchannel in noise.

----- Original Message -----
From: bill payne
Sent: Wednesday, January 12, 2005 6:40 PM

Subject: successive samples

Am I correct in my thinking that this method of using the lower digits can be expanded upon in a fairly simple manner, ie the final two digits of 8 consecutive samples reconstructed into bytes, a source of assorted possibilities, such as more unique identifiers or embedded commands?


But one has to have a way to synch sucessive samples so as not to get wrong answers.

One can even swich columns which is similar to changing communications channels.


----- Original Message -----
From: bill payne
Sent: Thursday, January 13, 2005 8:20 AM
Subject: synchronization

A marker pattern to simulate a pulse, ie 16 1s followed by 16 0s... suitable for synchronizing?

I haven't thought about the problem.

Digitizing at the senson, synchronous serial communications from the a/d to an 8051 family microcontroller, then over USB 2.0 to a PC occupy my thoughts. In addition to our legal project and fixing grey rabbit, of course.

I'm also thinking that for large productions and/or environments where the signal could possibly be momentarily blocked or lost, relay devices.

Resynchronoization is a problem. But I think there may lots of workable alternatives.

A well thought out plan to use relay devices could also provide new reverb options. The prominence of a single relay amidst several in proximity to the source, if adjusted at the controller/console, could be used to reproduce reverb.

There looks to be some really good business implementing some of these ideas.

One, of course, can encrypt bit positions 0 and 1 so that it would be really difficult for an observer to detect that an information subchannel in noise were present.

So you get the idea, we hope that information can be communicated on a noise subchannel, or even the valid data channel ... if one is able to sacrifice accuracy.

Others know about techniques for hiding information in noise.

Mason: It seems the Iranians where convinced someone was reading their supposedly secure and secret communications. In the Swiss television documentary, a former Crypto engineer made a sensational allegation. For his own safety he wonted to remain anonymous and the program makers reconstructed the intervju. [xxx check spelling of "intervju]

[A voice in German says a few meanings, then the voice fades out. A ghost voice says:]

The Iranians had probably long suspected they had been sold manipulating enciphering machines although i don't know if Mr Buehler was aware of what was going on. Of course nobody at Crypto was official supposed to know about the manipulation, although many people suspected it, and a few actually knew about it. I know proof exists that foreign intelligence services, that is; the Germans and Americans, fixed Crypto's machines in such a way that they could read the traffic being sent. I was actually connected with such manipulation programs. If I haven't been naturally there was no way I got to hear about it. People came down from Bad Godesberg in Germany with so called enciphering programs that determined the coding process in cipher machines, or people came over from the states from the government electronic divisions in Arizona. That's the Americans secret services counterpart to the Bad Godesberg outfit.

It took some time before I was 100 % certain about what was going on. I deposited the proof; that is the technical data and the manipulated enciphering programs in a bank safe. Then I alerted the federal authorities in Bern. I had many meetings with them but suddenly contacts where broken of, and the whole business came to a halt.

[The German voice continues, tells a few meanings, and fades out]

Feeding a digital to analog converter

Look at the blue star in Diagram 1.

What we want to do is to feed numbers into the d/a converter and see whether the same numbers are obtained by the a/d converter!

Interest lies in accurate measurement of analog signals, NOT hiding information in noise! But the two subjects are related.

Let's feed a d/a converter.