Reflections on Langlet’s APL Theory of Human VisionStuart Smith

stuart.smith3@comcast.net

AbstractIn the APL Theory of Human Vision [1,2], Langlet introduced the “neurobit system.” This system, it is claimed, is able to simultaneously (1) detect periodic replications in boolean vectors and (2) compress them, reducing their lengths without loss of information. In this paper, the neurobit network is shown to be capable of detecting certain kinds of replicated patterns, but that this capability depends critically on the data to be processed having dimensions that are powers of two. The success of data compression is also shown to depend similarly on the dimensions of the data to be compressed; however, the key functions of the neurobit-system give surprisingly simple spectral analyses of a whole class of fractal patterns. The appendices contain the APL functions needed to allow experimentation with the ideas presented in this paper. The workspace is also available from www.cs.uml.edu/~stu

IntroductionFig. 1 is one of the more intriguing ones in Langlet’s APL Theory of Human Vision presentation [1].

Fig. 1 Langlet’s neurobit-system

What this figure seems to promise is almost too good to be true: an extremely simple system that simultaneously detects periodic replications in boolean vectors and compresses them. The triangular structure shown in fig. 1 is a “fanion.” [2,8] Given a length n boolean vector, S, the fanion function generates successive rows of a fanion with n-1 iterations of S←(1↓S)≠ ¯1↓S, the initial value of S being retained as the top row of the fanion. The top row of the fanion in this case is the character string ‘’’'AA'’ rendered as the corresponding binary character codes. Reading down the left edge of the fanion we have Langlet’s Helical transform (“helicon”) of the vector. Reading up the right edge of the fanion, we have Langlet’s Cognitive transform (“cogniton”) of the vector [3,4,5,6,7]. The value of each element in a given row is the exclusive-or (i.e., APL ≠) of the two elements immediately above it.1 As Langlet pointed out [2], all of these relationships are retained when the fanion is rotated 120º in either direction, that is, the left edge

1 The neurobit system is thus a modulo-2 difference table, which, according to [9, p.1091], is equivalent to the evolution of Wolfram’s simple cellular automaton using either Rule 60 or Rule 102. This connection is discussed in [11] and [12].

is the Helical transform of the top row, the right edge is the Cognitive transform of the top row, and each element in a given row is the exclusive-or of the two elements immediately above it.

Data CompressionThe data compression claim will be considered first. The given vector of 16 items in fig. 1, representing the binary codes for the string 'AA', is compressed to just the six topmost elements of both the left and right edges of the fanion. All the 1’s are at the upper end of the result, followed by a tail consisting solely of 0’s. To uncompress the result, the 0’s tail must be concatenated back onto one of the compressed segments—left or right—and then the appropriate transform must be applied (Helical if the left edge of the fanion was used, Cognitive if the right edge was used). It is therefore necessary to remember how many zeroes there are below the compressed segment along each edge. The original string 'AA' requires two bytes, but the compressed string also requires two bytes: one to hold the (encoded) replicated element 'A' and another to hold a count of the zeroes that need to be tacked onto it in order to uncompress the string. Thus, no compression is achieved; however, the transforms of strings of up to 255 identical characters still require only two bytes, thus achieving a degree of compression that increases with the length of the string. This is good, but the compression observed in fig. 1 disappears if only the seven low-order bits of the ASCII code for 'A' are used to form the input vector, seven bits being all that are necessary to select any one of the 128 ASCII characters. The input to fanion in this case is a length 14 vector consisting of two identical 7-bit codes. As can be seen, the output has no all-0’s tail; thus, no compression has been achieved.

fanion 1 0 0 0 0 0 1 1 0 0 0 0 0 1

1 0 0 0 0 0 1 1 0 0 0 0 0 1 1 0 0 0 0 1 0 1 0 0 0 0 1 1 0 0 0 1 1 1 1 0 0 0 1 1 0 0 1 0 0 0 1 0 0 1 1 0 1 1 0 0 1 1 0 1 1 1 0 1 0 1 0 1 1 0 1 1 1 1 1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 1 0 0 0 1 1 0 0 1 1 0 1 1 1 0

Another, perhaps more serious, problem with the Helical and Cognitive transforms is their extreme sensitivity to noise in the data they operate on. A single corrupt bit will force the use of another analysis method, from which only an approximation of the repeated unit can be extracted.

Detection of Periodic ReplicationFor any theory of human vision, we will certainly want to know how effective the operations of the theory are in detecting important features of two-dimensional scenes. The ability of the Helical and Cognitive transforms to detect periodic replications is particularly well demonstrated when they are applied to highly structured binary images. A very simple test case is a square image consisting of equal-width black and white stripes. Fig. 2 (left), for example, is a 16×16 image consisting of sixteen stripes, each one unit wide, and fig. 2 (right) is its Helical transform.

Fig. 2 Stripes pattern and its Helical transform

The 16×16 Helical transform of this image consists of a single black element (1) in an otherwise white (0’s) matrix. The black element is located at (0,1), that is, in position 1 in row 0. If the striped pattern of Fig.2 (left) is transposed, as in fig. 3 (left), the Helical transform is also transposed, as in fig. 3 (right):

Fig. 3 Horizontal stripes pattern and its Helical transform

Combining the vertical and horizontal stripes patterns element-wise with exclusive-or produces the checkerboard pattern shown in fig. 4 (left). Because the combining operation is exclusive-or, the intersections of black (1) stripes form white (0) squares:

Fig. 4 Checkerboard pattern and its Helical transform

Fig. 4 (right) is the Helical transform of the checkerboard pattern. The black square at (0,1) represents the contribution of the vertical stripes, the black square at (1,0) the contribution of the horizontal stripes.

Because the Helical transform is its own inverse, applying it to the transformed image will reproduce the original striped or checked image. Further investigation shows that the Helical transform of any binary image that (a) is square, (b) has a side whose length is a power of two, and (c) is built only from alternating equal-width black and white stripes will have a single 1 in the first row if the stripes are vertical, in the first column if the stripes are horizontal, and in both the first row and first column for the checkerboard pattern. Each 1 will be located in a row or column position whose index is a power of two, which will also be the relative width of each stripe. We conclude, then, that the two-dimensional Helical transform can detect simple cases of periodic replication.

A more searching test of the Helical transform’s ability to detect periodic replication is provided by 2×2 block-replacement L-system patterns. These fractal patterns, some quite pretty, are all built in the same way: a 2×2 matrix of 1’s and 0’s is given as a start pattern, and then successively larger square matrices are built by replacing each 1 with a copy of the start pattern and each 0 with a copy of another, usually different, 2×2 matrix. Successively higher orders of Langlet’s G (“geniton”) matrix, for example, can be built in exactly this way. The start pattern is G:

1 11 0

To generate the next higher order matrix, replace each 1 with G and each 0 with a 2×2 all-zeroes matrix. The result is this 4×4 matrix:

1 1 1 11 0 1 01 1 0 01 0 0 0

This process can be continued indefinitely, producing larger and larger matrices, all of which will be square with side a power of two. All 256 possible combinations of two 2×2 boolean matrices can be generated with the tile function. Thus we can know in every case exactly how a given tiling pattern was generated.

Thue-Morse PlaneThe next test case is an image that has a relatively complex visual appearance but, as it turns out, a very simple Helical transform: the Thue-Morse plane, a familiar fractal image. Fig. 5 (left), for example, is the 16×16 Thue-Morse plane, and fig. 5 (right) is its Helical transform.

Fig. 5 Thue-Morse Plane and its Helical transform

As can be seen in the transformed image, 1’s are at positions 0, 1, 2, 4, and 8 in both the first row and first column. This analysis of the plane yields a set of nine 16×16 “basic forms,” each corresponding to a specific 1 in the transformed image. A particular basic form is obtained simply by applying the Helical transform to a matrix containing 1 in the selected position and 0’s everywhere else. When the basic forms are combined element-wise with ≠ the result is the original Thue-Morse plane. The first basic form, corresponding to the 1 at (0,0) in the transformed matrix, is an all-1’s (i.e., completely black) square. The effect of the all-1’s basic form is to complement an image, that is, reverse black and white; thus, the bit at (0,0) functions as a sort of “sign bit.” The remaining basic forms correspond to 1’s in the first row and first column. As expected, these images consist of black and white stripes whose relative widths are 1, 2, 4, and 8. The relative width of the stripes in a given basic form is the same as the row or column index of that image’s 1 in the transform. Fig. 6 shows the set of basic forms of the 16×16 Thue-Morse plane:

Fig. 6 Decomposition of the 16×16 Thue-Morse plane into nine basic forms

The Helical transform reveals replication of both horizontal and vertical stripes at four different scales. This is a global picture of the large-scale replicated elements from which the original image is built. This may be a somewhat surprising result given that the construction of the Thue-Morse plane is usually described as a process that gradually builds the plane out of smaller pieces.

Hadamard matrixThe last simple test case, the Hadamard matrix, is interesting because it has a relatively complex visual appearance but a very simple Helical transform. It is also of interest because it has applications in image and signal processing and in cryptography, among other areas. Fig. 7 (left) is a 16×16 Hadamard matrix represented as a binary image, and fig. 7 (right) is its Helical transform.

Fig. 7 Hadamard matrix and its Helical transform

In the transformed matrix, the 1’s lie on the main diagonal at positions (0,0), (1,1), (2,2), (4,4), and (8,8). This analysis yields a set of five basic forms, each corresponding to a 1 in the transformed image. The first basic form, corresponding to the 1 in the “sign bit” position (0,0) in the transformed matrix, is again an all-1’s (i.e., completely black) square. Fig. 8 shows the basic forms corresponding to the 1’s on the diagonal. The transform reveals replication of a squares-within-a-square pattern at four different scales.

Fig. 8 Decomposition of the 16×16 Hadamard matrix into five basic forms

Most of the other 2×2 tilings have Helical transforms in which the replication patterns are obvious but involve a large number of basic forms, far too many to permit showing all of them as was done above with the Thue-Morse plane and the Hadamard matrix. These patterns can be generated with the tile function listed in Appendix A. Many of the patterns and their transforms produce a kind of visually appealing “computer art.”

Beyond 2×2 Block Replacement ImagesThe Helical and Cognitive transforms appear to be exquisitely designed to detect multiple levels of replication in the kinds of binary images represented by 2×2 block replacement images. This is not surprising because both transforms were originally based on Langlet’s G matrix, which is itself a member of that collection. However, we immediately encounter difficulties when we want to apply the transforms to images that are not square with a side whose length is a power of 2. The solution in these cases is to pad the image with zeroes and/or use a truncated transform matrix.2 The downsides are that (1) these versions of the transforms are generally slower and (2) the results are typically much less informative.

2 These possibilities are discussed in Langlet transforms at www.cs.uml.edu/~stu

To illustrate the problem, consider fig. 9 (left), a 27×27 Sierpinski “carpet,” and fig.9 (right), its Helical transform.

Fig. 9 27×27 Sierpinski Carpet and its Helical transform

The replication pattern is easily visible in the Carpet image, but the transformed image appears to reveal nothing beyond mirror symmetry around the main diagonal. This situation arises out of the mismatch between the Langlet transform matrices, which consist of elements that are replicated at scales that are powers of two, and the 3×3 tiling patterns, which consist of elements that are replicated at scales that are powers of three. While the rows and columns of a 2×2 tiling pattern are precisely aligned with the rows and columns of Langlet transform matrices (which are themselves 2×2 tiling patterns), this can never occur with 3×3 tiling patterns.

DiscussionAs we already saw in the data compression example at the beginning of this paper, many of the nice properties of the Helical and Cognitive transforms depend on the dimensions and periodicities of their arguments being powers of two. In Langlet’s system the problems that arise when the dimensions and/or periodicities are not powers of two cannot be dealt with through conventional image or signal processing techniques (e.g., windowing, smoothing, filtering, etc.). The reason is that Langlet insisted that essentially all operations in his system have exact inverses so that no information could ever be lost in the course of a computation. To achieve the goal of completely reversible computation, it is necessary to follow Langlet’s droll mini-manifesto and scrupulously respect the “the rights of a bit:” [6]

Déclaration des Droits du Bit (Antwerp, 1994)

0. All bits contain information, so they deserve respect.1. Every small bit is a quantum of information.

Nobody has the right to crunch it.Nobody has the right to smooth it.Nobody has the right to truncate it.Nobody has the right to average it with its neighbour(s).Nobody has the right to sum it, except modulo 2.   

Given the severity of these requirements and the nature of the problems encountered, it is not clear how one would proceed with the work described here. One of the practical problems Langlet sought to solve was to somehow connect his system to the domains of image and signal processing. I had hoped that he had gone further in this direction than what I knew from his publications; however, in conversations after his passing with people who knew him I learned that he had never made the connection. By contrast, Wolfram’s magisterial A New Kind of Science [9] has pointed in directions in which progress might be made in a style of computing reminiscent of Langlet’s. One will recognize in this book many of the themes that Langlet was developing, although there is no mention of Langlet in it. Meanwhile, however, the game has changed significantly. The appearance of CNN (Cellular Neural Network) chips [10], which combine large numbers of digital and analog processors on a single chip, has enabled real-time processing of large quantities of optical sensor data. The main computational burden is carried by the analog units operating in parallel, while the digital units are used principally for control and memory. The two-dimensional neurobit network envisioned by Langlet resembles a CNN chip in some respects, but CNN’s constitute a more powerful and comprehensive approach to machine vision, pattern recognition, and image processing.

References[1] Gérard Langlet. The APL Theory of Human Vision (foils). Vector, 11:3.

[2] Gérard Langlet. The APL Theory of Human Vision. APL ‘94 - 9/94 Antwerp, BelgiumACM 0-89791 -675-1 /94.

[3] Gérard Langlet. Towards the Ultimate APL-TOE. ACM SIGAPL Quote Quad, 23:1, July 1992.

[4] Gérard Langlet. The Axiom Waltz - or When 1+1 make Zero. VECTOR, 11:3.

[5] Gérard Langlet. Paritons and Cognitons: Towards a New Theory of Information. VECTOR, 19:3.

[6] Gérard Langlet. Binary Algebra Workshop. Vector, 11:2.

[7] Michael Zaus. Crisp and Soft Computing with Hypercubical Calculus: New Approaches to Modeling in Cognitive Science and Technology with Parity Logic, Fuzzy Logic, and Evolutionary Computing: Volume 27 of Studies in Fuzziness and Soft Computing. Physica-Verlag HD, 1999.

[8] Gérard   Langlet . The Fractal MAGIC Universe. Vector, 10:1, p.137, 1993.

[9] Stephen Wolfram. A New Kind of Science. Wolfram Media, 2002.

[10] Leon Chua and Tamás Roska. Cellular Neural Networks and Visual Computing.Cambridge University Press, 2002.

[11] Stuart Smith. Wolfram’s Rule 60 and Extended Additivity.www.cs.uml.edu/~stu.

[12] Stuart Smith. In Search of the World Formula. www.cs.uml.edu/~stu.

Appendix A: Langlet Function DefinitionsThis section contains definitions of the functions needed to explore the ideas presented in this paper. Appendix B contains definitions of the functions used to create figs. 2-8 above. Langlet’s Helical and Cognitive transforms can be implemented in several different ways. The implementations here were designed to avoid WS FULL when applied to big images (e.g., 512×512 and larger). The functions defined in Appendices A and B are available in the Langlet tools workspace at www.cs.uml.edu/~stu.

Helical transform of square boolean matrix ⍵, whose side is a power of 2

HEL2←{⍉↑HEL¨↓⍉↑HEL¨↓⍵}

Cognitive transform of square boolean matrix ⍵ , whose side is power of 2

COG2←{⍉↑COG¨↓⍉↑COG¨↓⍵}

Helical transform of boolean vector S, whose length is a power of 2

z←HEL S :If (⍴S)=1 z←S :Else z←(HEL lhalf S),HEL(lhalf S)≠rhalf S :EndIf

Cognitive transform of boolean vector S, whose length is a power of 2

z←COG S :If (⍴S)=1 z←S :Else z←(COG(rhalf S)≠lhalf S),COG rhalf S :EndIf

Auxiliary functions required by the transforms lhalf←{((⍴y)÷2)↑⍵}

rhalf←{((⍴y)÷2)↓⍵}

Helical transform for general vector or matrix argument

y←hel x :If 1=⍴⍴x :If (2⍟⍴x)≠⌊2⍟⍴x y←thel x :Else y←HEL x :EndIf :ElseIf 2=⍴⍴x :If ∨/(2⍟⍴x)≠⌊2⍟⍴x y←thel2 x :Else y←HEL2 x :EndIf :Else ⎕←'argument must be a vector or matrix' z←⍳0 :EndIf

z←thel v;rv;lg;rm;mat rv←⍴v lg←⌈(⍟rv)÷⍟2 rm←(2*lg)-rv mat←rm trunc P 2*lg z←⌽v bmp mat

thel2←{⍉↑hel¨↓⍉↑thel¨↓⍵}

Cognitive transform for general vector or matrix argument

y←cog x :If 1=⍴⍴x :If (2⍟⍴x)≠⌊2⍟⍴x y←tcog x :Else y←COG x :EndIf :ElseIf 2=⍴⍴x :If ∨/(2⍟⍴x)≠⌊2⍟⍴x y←tcog2 x :Else y←COG2 x :EndIf :Else

⎕←'argument must be a vector or matrix' z←⍳0 :EndIf

z←tcog x z←⌽thel⌽x

tcog2←{⍉↑cog¨↓⍉↑cog¨↓⍵}

z←n trunc mat :If ((⍴⍴mat)≠2)∨((1↑⍴mat)≠(¯1↑⍴mat)) ⎕←'argument must be a square matrix' :EndIf z←((-n),n)↓mat

Functions to view bitmap bits and write it out to .bmp file file

disp bits 'bm'⎕WC'bitmap'('Bits'bits)('Cmap'(2 3⍴255 255 255 0 0 0)) 'f'⎕WC'form'('Picture' 'bm')

bits writebmp file;bm 'bm'⎕WC'bitmap'('Bits'bits)('Cmap'(2 3⍴255 255 255 0 0 0)) bm.File←mypath,file 2 ⎕NQ'bm' 'FileWrite'

Function to read .bmp file file into variable argbargb←readbmp file;bmp 'bmp'⎕WC'Bitmap'(mypath,file,'.bmp') argb←bmp.Bits

Appendix B: Functions to create the figures in the textIn each definition, n is the number of iterations of block replacement to be performed. The side of the resulting matrix will be 2n+1. blk and wht specify the two 2×2 binary patterns used to tile the image.

Sierpinski triangle (Langlet’s G matrix)

y←sierpinski n;blk;wht;⎕IO⎕IO←0blk←2 2⍴1 1 1 0wht←2 2⍴0 0 0 0tile_step←{⍎⍤1⍕(wht blk)[⍵]}y←n{(tile_step⍣⍺)⍵}blk

Hadamard matrix

y←hadamard n;blk;wht;⎕IO⎕IO←0blk←2 2⍴1 1 1 0wht←2 2⍴0 0 0 1tile_step←{⍎⍤1⍕(wht blk)[⍵]}y←n{(tile_step⍣⍺)⍵}blk

Thue-Morse plane

y←TMplane n;blk;wht;⎕IO⎕IO←0blk←2 2⍴1 0 0 1wht←2 2⍴0 1 1 0tile_step←{⍎⍤1⍕(wht blk)[⍵]}y←n{(tile_step⍣⍺)⍵}blk

stripes

y←stripes n;blk;wht;⎕IO⎕IO←0blk←2 2⍴1 0 1 0wht←2 2⍴1 0 1 0tile_step←{⍎⍤1⍕(wht blk)[⍵]}y←n{(tile_step⍣⍺)⍵}blk

checkerboard

y←checkerboard n;x;⎕IO⎕IO←0x←stripes ny←x≠⍉x

Sierpinski carpet

y←carpet n;wht;blk;⎕IO ⎕IO←0 blk←3 3⍴1 1 1 1 0 1 1 1 1 wht←3 3⍴0 tile_step←{⍎⍤1⍕(wht blk)[⍵]} y←n{(tile_step⍣⍺)⍵}blk