United States Patent: 4,541,012

[US Patent & Trademark Office, Patent Full Text and Image Database]

( 1 of 1 )

United States Patent	*4,541,012*
Tescher	September 10, 1985

Video bandwidth reduction system employing interframe block differencing and transform domain coding

Abstract

Video type information signals are compressed for transmission and reproduction by comparing corresponding blocks of time domain information signals from successive fields, converting a block of the time domain information signals to a transform domain signal represented by discrete cosine transform coefficients when the difference between the corresponding blocks exceeds a block difference threshold, and encoding the transform domain coefficients for transmission to a decoding site. Corresponding blocks of time domain information signals from successive fields are compared by storing the successive fields in memory on a pixel by pixel basis, retrieving each block on a pixel by pixel basis, forming the difference between corresponding pixels from the successive blocks, squaring the resulting difference signal, summing the squares and dividing by the number of pixels per block. Successive fields are merged by weighted summing of corresponding pixels.

Inventors:	Tescher; Andrew G. (Claremont, CA)
Assignee:	Compression Labs, Inc. (San Jose, CA)
Appl. No.:	336984
Filed:	January 4, 1982

Current U.S. Class: 348/400.1

Intern'l Class: H04N 007/12

Field of Search: 358/133,135,136,138,13 382/43 364/725,726,727,826,827

References Cited [Referenced By]

U.S. Patent Documents

3366739	Jan., 1968	Parkinson	358/133.
4179709	Dec., 1979	Workman	358/133.
4202011	May., 1980	Koga	358/136.
4216354	Aug., 1980	Esteban et al.	381/31.
4224678	Sep., 1980	Lynch et al.	364/724.
4225885	Sep., 1980	Lux et al.	340/146.
4237494	Dec., 1980	Yamaguchi	358/280.
4254438	Mar., 1981	Yamazaki et al.	358/261.
4302775	Nov., 1981	Widergren et al.	358/136.

Other References

Haskell-Frame to Frame Coding-IEEE Trans. on Infor. Theory, vol. IT-20, No. 1, pp. 119-120, Jan. 1974.
Kangko-Digital Television Transmission-IEEE Comm.-Jul. 1980, pp. 14-22.
Mounts, F. W., "A Video Encoding System with Conditional Picture-Element Replenishment", The Bell System Technical Journal, Sep. 1969.
Murray, G. Graham, Staley, Patrick A. and Thoma, George R., "Motion-adaptive TV data compression", SPIE, vol. 241, pp. 51-60, Real-Time Signal Processing III(1980).
Haskell, B. G. and Schmidt, R. L., "A Low-Bit-Rate Interframe Coder for Videotelephone", The Bell System Technical Journal, vol. 54, No. 8, Oct. 1975.
Jones, Jr., Harry W., "A Conditional Replenishment Hadamard Video Compressor", SPIE, vol. 119, Applications of Digital Image Processing, (IOCC 1977).
Hein, David N. and Jones, Jr., Harry W., "Conditional replenishment using motion prediction", SPIE, vol. 207, Applications of Digital Image Processing III (1979).

Primary Examiner: Orsino, Jr.; Joseph A.
Attorney, Agent or Firm: Fliesler, Dubb, Meyer & Lovejoy

Claims

What is claimed is:

1. A method for processing time domain information signals having a successive field format to effect substantial compression of said signals, said method comprising the steps of:

comparing corresponding blocks of time domain information signals from successive fields to form a block difference signal;

converting a block of said time domain information signals to transform domain signals including a D.C. coefficient representing the average intensity of a converted block and a plurality of discrete transform coefficients when said block difference signal exceeds a first variable parametric value;

varying said first variable parametric value in accordance with the number of blocks selected for conversion; and

encoding said transform domain coefficients for subsequent utilization.

2. The method of claim 1 wherein said step of comparing includes the steps of storing successive fields of said time domain information signals in a first memory device on a pixel by pixel basis, and retrieving said corresponding blocks from said memory device on a pixel by pixel basis.

3. The method of claim 2 further including the step of replacing a block pf pixel elements in said memory device with the corresponding block in a successive field when said difference exceeds said first variable parametric value.

4. The method of claim 2 wherein said step of storing includes the step of merging successive fields on a pixel by pixel basis.

5. The method of claim 4 wherein said step of merging is performed by summing corresponding pixels from successive fields in accordance with a predetermined weighting factor.

6. The method of claim 5 wherein the pixels from the first appearing merged field are weighted by a factor of 3/4 and the pixels from the later appearing field are weighted by a factor of 1/4.

7. The method of claim 1 wherein said step of converting is performed by transforming said block along a first direction and subsequently transforming said block along the orthogonal direction.

8. The method of claim 7 wherein said first direction corresponds to a horizontal line of raster scan information and said orthogonal direction corresponds to a vertical column of raster scan information.

9. The method of claim 7 wherein said step of converting includes the step of storing said transform coefficients in diagonal format in a second memory device.

10. The method of claim 9 further including the step of storing the field address of each transformed block in said second memory device.

11. A method for processing time domain information signals having a successive field format to effect substantial compression of said signals, said method comprising the steps of:

comparing corresponding blocks of time domain information signals from successive fields to form a block difference signal, said step of comparing including the steps of storing successive fields of said time domain information signals in a first memory device on a pixel by pixel basis, retrieving said corresponding blocks from said memory device on a pixel by pixel basis, forming the difference between corresponding pixels from said successive blocks, squaring the resulting pixel difference signals, summing the squares of said resulting pixel difference signals, and dividing the resulting sum by a number of pixels per block to form said block difference signal;

converting a block of said time domain information signals to transform domain signals including a D.C. coefficient representing the average intensity of a converted block and a plurality of discrete transform coefficients when said block difference signal exceeds a first variable parametric value; and

encoding said transform domain coefficients for subsequent utilization.

12. The method of claim 11 wherein each block comprises a total of 64 pixels grouped in an 8 by 8 array.

13. A method for processing time domain information signals having a successive field format to effect substantial compression of said signals, said method comprising the steps of:

comparing corresponding blocks of time domain information signals from successive fields to form a block difference signal;

converting a block of said time domain information signals to transform domain signals including a D.C. coefficient representing the average intensity of a converted block and a plurality of discrete transform coefficients when said block difference signal exceeds a first variable parametric value; and

encoding said transform domain coefficients for subsequent utilization, including the steps of providing a plurality of different code tables, dividing each of said plurality of discrete transform coefficients by a second variable parametric value to obtain a corresponding quantized coefficient, calculating the predictive value of each quantized coefficient, selecting one of said plurality of code tables in accordance with the predictive value, and generating a code representing said quantized coefficient from said selected table.

14. The method of claim 13 wherein said step of calculating is performed in accordance with the formula PM.sub.K =1/4C.sub.K +3/4PM.sub.K-1, where PM.sub.K is the predictive mean value of the K.sup.th quantized coefficient, C.sub.K is the value of the K.sup.th quantized coefficient and PM.sub.K-1 is the predictive mean value of the K-1.sup.th quantized coefficient.

15. The method of claim 13 wherein said step of encoding includes the steps of providing a dedicated code table for encoding each DC coefficient and selecting said dedicated table for each said DC coefficient.

16. The method of claim 13 wherein said time domain information signals are color video signals having quadrature components, and wherein said step of encoding includes the steps of providing individual code tables for said quadrature components, calculating the average value of each quadrature component for the converted block, and selecting a code value representing said average value from the corresponding individual quadrature component table.

17. The method of claim 13 wherein said step of encoding further includes the step of generating an address code specifying the field address of the converted block.

18. The method of claim 17 wherein said step of generating includes the steps of providing an address code table having code values arranged in accordance with the following formula:

If .DELTA..sub.k =1, code 1 bit (.DELTA..sub.k)

If .DELTA..sub.k <32, code 1 bit zero+5 bits (.DELTA..sub.k)

If .DELTA..sub.k .gtoreq.32, code 6 bits zero+10 bits (.DELTA..sub.k)

where

.DELTA..sub.k =A.sub.k -A.sub.k-1

A.sub.k =numerical address of current block

A.sub.k-1 =numerical address of most recently encoded block,

and selecting said address code table for each said address code.

19. The method of claim 13 wherein said step of encoding further includes the steps of comparing said predictive value with a preselected fixed threshold value, and generating a run length code specifying the total number of successive quantized coefficients of zero value when said predictive value lies below said preselected fixed threshold value.

20. The method of claim 19 wherein said step of encoding further includes the step of generating an end of block code when the predictive values for successive remaining quantized coefficients in the converted block lie below said preselected fixed threshold value.

21. The method of claim 19 wherein said codes are multi-bit binary codes, and wherein said method further includes the steps of transferring said codes to a buffer in the order of generation, monitoring the number of bits transferred to said buffer, and varying said first and second parametric values in accordance with said number of transferred bits in order to minimize buffer overflow.

22. The method of claim 21 wherein said second parametric value is varied in accordance with the formula:

D=D'.sub.K +K.sub.D .multidot.BFN(B.sub.K -N/2)

where

BFN(X)=(X/N-.vertline.X.vertline.)

D.sub.K =Distortion parameter for block K

D'.sub.K =Filtered distortion parameter

D'.sub.K =T.multidot.D'.sub.K-1 +(1-T)D.sub.K-1

where

T=a constant (close to 1)

K.sub.D =a constant

B.sub.K =# of bits in buffer for block K

N=Max. number of bits.

23. The method of claim 22 wherein said first parametric value is varied in accordance with the alternate formula:

T.sub.K =T.sub.INIT

for B.sub.LOW.ltoreq.B.sub.K .ltoreq.B.sub.HIGH

T.sub.K =T.sub.INIT +K.sub.R .multidot.BFN(B.sub.LOW -B.sub.K)

for B.sub.K <B.sub.LOW

T.sub.K =T.sub.INIT +K.sub.R .multidot.BFN(B.sub.K -B.sub.HIGH)

for B.sub.K >B.sub.HIGH

where

T.sub.K =replenishment threshold for block K

T.sub.INIT =initial threshold (about 5 for 8-bit input data)

K.sub.R =multiplier constant (about 25-75)

B.sub.LOW =low cutoff (about 0.1 of buffer)

B.sub.HIGH =high cutoff (about 0.75 of buffer).

24. The method of claim 13 wherein each of said code tables comprises a Huffman code table.

25. A method of encoding transform coefficients representing time domain information signals having a successive field format prior to transmission over a communication link in order to effect substantial compression of said signals, said transform coefficients being arranged in a plurality of groups, each group representing an N by N block of field information signals, said method comprising the steps of:

(a) providing a plurality of code tables:

(b) generating a block address code from a first dedicated one of said code tables, said block address code representing the field address of the block represented by a group of said transform coefficients;

(c) generating a DC coefficient code representing the average intensity of said block from a second dedicated one of said plurality of code tables; and

(d) generating a succession of codes representing the remaining transform coefficients corresponding to said block by calculating the predictive value of each said remaining transform coefficient, selecting one of said plurality of code tables in accordance with said predictive value, and generating a code representing the corresponding transform coefficient from said selected table.

26. The method of claim 25 wherein said step of calculating is performed in accordance with the formula PM.sub.K =1/4 C.sub.K +3/4 PM.sub.K-1, where PM.sub.K is the predictive mean value of the K.sup.th coefficient, C.sub.K is the actual value of the K.sup.th coefficient and PM.sub.K-1 is the predictive mean value of the K-1.sup.th coefficient.

27. The method of claim 25 wherein said time domain information signals are color video signals having quadrature components, and wherein said method of encoding includes the steps of providing individual code tables for said quadrature components, calculating the average value of each quadrature component for said block, and selecting a code value representing said average value from the corresponding individual quadrature component table.

28. The method of claim 25 wherein said method of encoding further includes the steps of comparing each said predictive value with a preselected threshold value, and generating a zero run length code specifying the total number of successive predictive values lying below said preselected threshold value.

29. The method of claim 28 wherein said codes are multi-bit binary codes, wherein said method of encoding further includes dividing each of said remaining transform coefficients by a variable parametric value to obtain corresponding quantized remaining coefficients for use in forming the predictive values, and wherein said method further includes the steps of transferring said codes to a buffer in the order of generation, monitoring the number of bits transferred to said buffer, and varying said variable parametric value in accordance with the following formula:

D=D'.sub.K +K.sub.D .multidot.BFN(B.sub.K -N/2)

where

BFN(X)=X/(N-.vertline.X.vertline.)

D.sub.K =Distortion parameter for block K

D'.sub.K =Filtered distortion parameter

D'.sub.K =T.multidot.D'.sub.K-1 +(1-T)D.sub.K-1

where

T=a constant (close to 1)

K.sub.D =a constant

B.sub.K =# of bits in buffer for block K

N=Max. number of bits.

30. The method of claim 28 further including the step of generating an end of block code when the predictive values for successive remaining transform coefficients in said block lie below said preselected threshold value.

31. The method of claim 25 wherein said transform coefficients comprise discrete consine transform coefficients.

32. A method for processing time domain information signals for transmission over a communication link, said time domain information signals having a successive field format, said method comprising the steps of:

comparing corresponding sub-field blocks of time domain information signals from successive fields to form a block difference signal;

converting a sub-field block of said time domain information signals to a transform domain signal represented by a D.C. coefficient representing the average intensity of the converted block and a pluality of discrete transform coefficients when said block difference signal between the corresponding blocks exceeds a first variable parametric value; and

generating a frame sync code signal indicating the beginning of a frame;

generating a first control code signal B.sub.K representative of the fullness of a transmission rate buffer at the beginning of said frame;

generating a second control code signal D.sub.K representative of a second variable parametric value at the beginning of said frame; and

generating a plurality of block replenishment code symbols each representative of the value of transform coefficients corresponding to individual sub-field blocks having interfield block differences greater than said first variable parametric value, each said block replenishment code symbol including a block address code specifying the field address of the corresponding block, a D.C. code term representative of said D.C. coefficient of the corresponding block, and a plurality of coefficient code terms representative of the value of said plurality of discrete transform coefficients as quantified according to said second control code signal D.sub.K for the corresponding block.

33. The method of claim 32 wherein said time domain information signals are color video signals having quadrature components, and wherein said step of generating block replenishment code symbols includes the step of providing first and second color code terms in each of said plurality of said block replenishment code symbols representing the average value of each quadrature component for said corresponding block between said block address code and said DC code term.

34. The method of claim 32 wherein said step of generating block replenishment code symbols includes the step of providing a run length code term specifying the total number of successive transform coefficient zero values having a predictive mean value less than a preselected fixed threshold value.

35. The method of claim 32 wherein said step of generating block replenishment code symbols includes a step of providing an end of block code term when the predictive values for successive remaining transform coefficients in said corresponding block lie below a preselected fixed threshold value.

Description

BACKGROUND OF THE INVENTION

This invention relates to information signal processing in general, and in particular to the field of processing time sequential information signals (such as video signals) for the purpose of compressing the amount of information to be transferred from an encoding site to a decoding site.

In recent years, increasing efforts have been directed toward providing more efficient information signal encoding techniques used to process time sequential information signals prior to their transmission from a transmitting station to a receiving station. The requirement for more efficient encoding techniques has been prompted by two major factors: firstly, a substantial increase in the quantity of information required to be transferred via communication links and, secondly, maximum occupancy of the communication frequency bands available for voice and data transmission. An early technique employed to reduce the amount of information required to be transferred without substantial degradation is the signal processing technique known as conditional replenishment, described in U.S. Patent No. 3,984,626 to Mounts et al., the disclosure of which is hereby incorporated by reference. Briefly, in the conditional replenishment signal processing technique, individual line element sample signals from a successive field of information are compared with the corresponding line elements in the previous field, and the difference therebetween is tested against a fixed threshold. If the difference exceeds the threshold value, the new value is encoded and transmitted to a receiving station, along with an appropriate address code specifying the line location of the sample to be updated in the field memory of the receiving station. Thus, rather than transmitting each and every line sample for every field, only those samples which differ by a significant threshold amount are transmitted, which substantially reduces the number of samples in the communication channel pipeline. Although this saving in the amount of actual data flowing through the communication pipeline is somewhat offset by the necessity of simultaneously transmitting the address information, this disadvantage is more than overcome by the substantial reduction in the total number of samples which must be transmitted in order to maintain the information current at the decoding site. When used to process video type information signals, an even greater reduction in the required number of transmitted samples is achieved due to the inherent nature of video signals, which possess intrinsic interfield correlation (e.g. abrupt interfield changes for background portions of video images occur relatively infrequently).

Another compression technique known in the art is the use of transform domain encoding, in which each field of information signals is divided into a number of rectangular or square arrays of individual picture elements (for example a 16 pixel by 16 pixel array) termed blocks, and each block is converted to the transform domain. For each converted block, the individual transform coefficients are then encoded and transmitted along with appropriate address codes, as well as additional overhead information (e.g. field start signals, frame start signals and the like). One such transform domain processing system is disclosed in U.S. Pat. No. 4,189,748 to Reis, the disclosure of which is hereby incorporated by reference.

Although many types of mathematical transform functions have been proposed for implementation in a transform domain signal processing system, in reality most transform functions are inappropriate for implementation due to the complexity of the required logic circuitry. This disadvantage is exacerbated in applications requiring real time signal processing by virtue of the minimum time period required to perform the signal processing necessary to generate the values of the transform coefficients. For a general discussion of the advantages and disadvantages of the different types of transform functions, reference should be had to the collection of technical publications entitled "Image Transmission Techniques, Advances in Electronics and Electron Physics, Supplement 12", Pratt, Academic Press, 1979, particularly the section entitled "Transform Image Coding".

SUMMARY OF THE INVENTION

The invention comprises a method and system for processing time domain information signals which combines the advantages of conditional replenishment and transform domain coding in such a manner that information signal compression of a magnitude substantially greater than that available in known systems is achieved while affording real time information signal processing.

In its broadest aspect, the invention provides a method of processing time domain information signals having a successive field format to effect substantial compression of the signals, the method including the steps of comparing corresponding blocks of time domain information signals from successive fields, converting a block of the time domain information signals to a transform domain signal represented by discrete cosine transform coefficients when the difference between the corresponding blocks exceeds a first variable parametric value, and encoding the transform domain coefficients for subsequent utilization, e.g. transmission from a transmitting station to a receiving station, recording on video tape or other magnetic media, etc. The corresponding blocks of time domain information signals from successive fields are compared by storing the successive fields in memory on a pixel by pixel basis, retrieving the corresponding blocks from memory also on a pixel by pixel basis, forming the difference between corresponding pixels from the successive blocks, squaring the resulting difference signal, summing the squares of the resulting difference signals, and dividing the resulting sum by the number of pixels per block. In the preferred embodiment, the method is optimized by employing a total of 64 pixels per block arranged in an 8 by 8 array and by merging successive fields on a pixel by pixel basis, the merging being performed by summing corresponding pixels from successive fields in accordance with a predetermined weighting factor of 3/4 for the earlier appearing (previously merged) field and 1/4 for the later appearing field.

The conversion of a block of the time domain information signals to the transform domain is accomplished by first transforming the individual block samples along a first direction, which is the horizontal line direction in the preferred embodiment, and subsequently transforming the same block samples along the orthogonal direction, which is the vertical direction in the preferred embodiment. For each transformed block, the individual block samples corresponding to the previous field are replaced with the updated block information, and the transformed coefficients for the converted block are stored in diagonal format in a diagonal memory unit. In addition, an address code indicating the field address of a transformed block is also stored in the diagonal memory for subsequent encoding.

The transform coefficients for each converted block stored in the diagonal memory are encoded using a plurality of different code tables, one of the tables being dedicated to the first coefficient in each diagonal group, corresponding to the DC term and representing the average signal intensity of the converted block, and the remaining tables being selected on a coefficient by coefficient basis. Specifically, each transform coefficient (other than the first or DC coefficient) is first quantized by digitally dividing the coefficient by a varible parametric value D.sub.K, after which the predictive mean value of each quantized coefficient is calculated by summing a weighted portion of the actual value of that quantized coefficient with the predictive mean value of the previous quantized coefficient weighted by a different factor, and the newly calculated predictive mean value is used to select that one of the several available individual code tables capable of encoding the quantized coefficient value with a minimum number of binary bits. In addition to the tables noted above, separate tables are provided for encoding the block address of the encoded transform coefficients, for directly encoding the D.C coefficient, and for run length coding certain coefficient values. In the case of time domain information signals comprising color video signals with quadrature components two preselected quantized coefficient code tables are used to represent the average value of each color quadrature component of the corresponding converted block.

To further compress the amount of information encoded prior to utilization, those successive transform coefficients with zero value whose predictive mean lies below the value of a preselected fixed threshold are transmitted as a run length code. In addition, when the predictive values for successive remaining cosine coefficients in the converted block lie below the preselected fixed threshold, a single end-of-block code is generated.

The codes corresponding to a given converted block are transferred at a availablef rate to a rate buffer in the order of generation prior to utilization, and the number of binary bits transferred to the buffer is monitored in order to gauge the buffer fullness. The dynamic occupancy of the buffer is used to control the value of the variable parametric value D.sub.K in order in minimize the possibility of buffer overflow, utilizing a special algorithm. The buffer fullness state is also used to control the first variable parametric value--termed the block difference threshold T.sub.K --also by employing a special algorithm. Thus, as the rate buffer approaches the completely filled state, the magnitude of D.sub.K is increased, which increases the minimum quantizing interval employed in sampling the transform coefficients during the encoding process. In addition, the block difference threshold T.sub.K is similarly increased to reduce the number of blocks selected for conversion to the transform domain and subsequent encoding. Similarly, as the state of the buffer fullness decreases, both D.sub.K and T.sub.K are lowered in value in accordance with the special algorithms employed in order to increase the number of blocks selected for conversion to the transform domain and to decrease the minimum quantization interval used in the encoding process.

The codes representing the converted blocks are formatted in the rate buffer in the following fashion. The start of each frame is denoted by a frame sync code signal, which is followed by a first control code signal representative of the buffer fullness at the beginning of the frame and a second control code signal representative of the quantizing interval D.sub.K value at the beginning of the same frame. The control code signals are followed by individual block replenishment code symbols which include a block address code specifying the field address of the corresponding block, the DC code term representative of the average intensity of the corresponding block, and the plurality of coefficient code terms reprentative of the predictive mean value of the transform coefficients for the corresponding block. For color video signal processing, the quadrature component code terms are included between the block address code and the DC code term. The termination of the last block is signified by the subsequent appearance of the frame sync code signal for the next succeeding frame.

The decoding process is essentially the inverse of the encoding process. For each frame of encoded information, the first and second control code signals are used to establish the initial minimum quantization interval to be employed for inverse quantizing the block replenishment code symbols. The received replenishment code symbols are decoded using a parallel set of inverse code tables, which are selected using the same predictive mean algorithm as that employed in the encoding process. The block address, quadrature chrominance and D.C. term codes are coupled directly to a diagonal memory unit, while the coefficient code terms are inverse quantized by multiplying each code term by D.sub.k, using the transmitted initial value of D.sub.K for the first block of data, and the resulting coefficients are stored in the diagonal memory unit. After the first block has been decoded, the distortion constant D.sub.k is recalculated and the newly calculated value of D.sub.k is used to inverse quantize the next block of data.

The coefficients stored in the diagonal memory unit are then transformed to time domain digital samples using an inverse discrete cosine transform, and the resulting samples are stored in an output memory unit, replacing previous samples representing the same block. The merged field samples stored in the output memory unit, which replicate the merged field samples stored in a corresponding reference memory unit at the encoder stie, are finally processed to provide video output signals.

Further compression is achieved according to the invention by special initial processing of the luminance and chrominance samples. The luminance signals are sub-sampled at less than the standard rate (which is 512 lines/frame and 512 samples/line for NTSC video), the preferred embodiment employing 256 lines/frame and 256 samples/line. Each quadrature chrominance component is sub-sampled at less than the standard rate and averaged over a given block. In the preferred embodiment, each quadrature component is sub-sampled at one-half the standard rate for each block line and the sub-samples for each block line are averaged, after which each block line average is combined to obtain a block average. Prior to averaging, each chrominance component sample is further modififed by discarding the two least significant bits of the sample. After transmission from an encoding site to a decoding site, the full range of luminance and chrominance samples is recovered by individual interpolative processing of the received luminance and chrominance samples.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an encoder incorporating the invention;

FIG. 2 is a block diagram illustrating a decoder incorporating the invention;

FIG. 3 is a schematic view of a portion of a display screen illustrating the replenishment block size;

FIG. 4 is a trellis diagram illustrating the cosine transform alogorithm employed in the preferred embodiment;

FIG. 5 is a schematic diagram illustrating the manner in which transform coefficients are stored in a diagonal memory unit;

FIG. 6 is set of probability distribution curves illustrating the manner in which the quantized coefficient encoding tables are constructed;

FIG. 7 is a schematic diagram illustrating typical predictive mean values for a single block; and

FIG. 8 is a schematic diagram illustrating the code formatting for one frame of replenishment information.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Turning now to the drawings, FIG. 1 is a block diagram illustrating a preferred embodiment of the encoded portion of the invention. As seen in this FIG., analog video signals are coupled to the input of an analog processor unit 11 in which composite video input signals are separated into the standard luminance and quadrature chrominance components and converted to multi-bit digital samples at a predetermined sampling rate. In the preferred embodiment eight bit digital samples are taken at a 10.7 MHz sampling rate. The equivalent digital samples produced in analog processor unit 11 are coupled to the input of an input digital processor 12 in which incoming field samples are merged with the corresponding samples from the previous field in the manner described below. The resulting individual merged field samples from input digital processor 12 are stored in an input memory unit 13 having a sufficient capacity to contain one field of digital information. An additional memory unit 14, termed a reference memory, is coupled to the data output of input memory unit 13. Reference memory unit 14 stores a reference field of information for comparison with a newly merged field stored in input memory unit 13, and has the same capacity as input memory unit 13.

In operation, the individual digital samples from an incoming field supplied to input digital processor 12 are added to the corresponding digital samples of the previously merged field stored in input memory unit 13 on a weighted basis, and the resulting weighted sums are stored in input memory unit 13, replacing the previously stored samples on a pixel by pixel basis. In the preferred embodiment, the samples are weighted by a factor of 3 to 1 between the older samples stored in the input memory unit 13 and the incoming field samples, i.e. the earlier samples are multiplied by a factor of 3/4, the later samples are multiplied by the factor of 1/4 and the resulting weighted samples are added together. The weighting multiplication and the addition are accomplished with conventional digital multipliers and adders, in combination with appropriate conventional addressing logic.

Each newly merged field stored in input memory unit 13 is compared on a block by block basis with the reference field stored in reference memory unit 14 by means of a block difference and decision unit 16. As illustrated in FIG. 3, each block element consists of a rectangular array of 8 pixels by 8 pixels, and the difference between each block is obtained by digitally subtracting corresponding pixel samples read from input memory unit 13 and reference memory unit 14 in block difference and decision unit 16, squaring the resulting difference signals, summing the squares of the resulting difference signals, and dividing the resulting sum by the number of pixels per block (64). Each block difference value so obtained is tested against a threshold T.sub.K supplied from a distortion calculation unit 18. If the block difference exceeds the threshold, the corresponding block in input memory unit 13 is converted to a set of transform coefficients by means of a one dimension cosine transform unit 20, and the transform coefficients are stored in a diagonal memory unit 21 along with a corresponding block address code specifying the field block to which the transform coefficients correspond. In addition, whenever a block is selected for conversion to the transform domain, the reference memory unit 14 is updated by replacing the corresponding block in reference memory unit 14 with the newly selected block.

The conversion of each selected block to the transform domain is done by a one dimensional cosine transform unit 20 in two steps: a first transformation along the horizontal direction, followed by a second transformation along the vertical direction. The cosine transform unit 20 implements the well known discrete cosine transform function: ##EQU1## where ##EQU2## at zero, 1 for k (1, N-1) and zero elsewhere and comprises a plurality of conventional digital multiplying accumulators configured to implement the 8 point cosine transform algorithm shown in the trellis diagram of FIG. 4. During the transformation along the horizontal direction, developing coefficients are stored in diagonal memory unit 21, and are subsequently recalled during the transformation in the vertical direction. After the selected block has been completely converted to the transform domain, the resulting series of coefficients is stored in diagonal memory unit 21 along with a multi-bit digital word specifying the block address of the block corresponding to the series of coefficients and two multi-bit digital words specifying the average value of the chrominance quatradure components for the corresponding block, the coefficients being arranged in the diagonal form illustrated schematically in FIG. 5.

The transform coefficients and corresponding block address and chrominance quadrature digitial characters are next encoded for subsequent transmission in the following manner. The block address digital character corresponding to a series of transform coefficients is coupled directly to a coder unit 22 which contains in the preferred embodiment nine separate code tables, eight tables containing a set of code characters arranged according to the Huffman code technique, in which the number of bits per specific character depends upon the probability of occurrence of that character, and one table containing a set of code characters arranged according to a special variable length coding technique specified below. The special variable length code table is dedicated for use with the block address code, and the application of a new block address code to the dedicated code table results in the generation of a block address transmission code. The block address is actually encoded by forming the numerical difference between the current block address and the address of the most recent previously encoded block address, and generating a code in accordance with the following algorithm:

If .DELTA..sub.k =1, code 1 bit (.DELTA..sub.k)

If .DELTA..sub.k <32, code 1 bit zero+5 bits (.DELTA..sub.k)

If .DELTA..sub.k .gtoreq.32, code 6 bits zero+10 bits (.DELTA..sub.k)

where

.DELTA..sub.k =A.sub.k -A.sub.k-1

A.sub.k =numerical address of current block

A.sub.k-1 =numerical address of most recently encoded block.

The color quadrature components are encoded using dedicated Huffman code tables in coder unit 22. The tables listed as table number 2 and table number 3 in appendix A are used for the Q and I component values, respectively. After the block address and the color quadrature components have been encoded in the manner noted above, the first coefficient in diagonal memory unit 21 corresponding to the block, and which respresents the average luminance of the block, is encoded using dedicated Huffman code table number 7 shown in appendix A. Thereafter, the cosine coefficients are processed for encoding by passing each cosine coefficient through a quantizer unit 23 in which the individual coefficients are divided by a distortion constant D.sub.k supplied by distortion calculator unit 18. More particularly, each coefficient is multiplied by the quantity 1/D.sub.k using a digital multiplier, and the resulting rounded product, designated as a quantized cosine coefficient, is coupled to coder unit 22. In the preferred embodiment, each quantized cosine coefficient comprises a 12 bit digital character having 1 sign bit and 11 bits of magnitude. For each quantized cosine coefficient received in coder unit 22, the predictive mean value is calculated using the following relationship:

PM.sub.K =1/4C.sub.K +3/4PM.sub.K-1

where PM.sub.K is the predictive mean value of the K.sup.th quantized coefficient, C.sub.K is the value of the K.sup.th quantized coefficient and PM.sub.K-1 is the predictive mean value of the K-1.sup.th quantized coefficient. The predictive mean value PM.sub.K is used to select one of six of the Huffman code tables 1-6 listed in Appendix A to be used to encode the next appearing quantized coefficient, in the manner described below. Thus, PM.sub.K is used to select the Huffman code table for quantized coefficient K+1, PM.sub.K+1 is used for quantized coefficient K+2, etc.

The four most significant magnitude bits of each quantized coefficient are next examined in coder unit 22 using conventional logic circuitry and, if the most significant four bits are zero, the quantized coefficient is Huffman coded using one of tables 1-6 listed in appendix A. Each table 1-6 is constructed using a different one of six probability distribution curves illustrated in FIG. 6. Each curve comprises an exponential function, with different curves having different mean values ranging from 1 to 32. The calculated predictive mean value PM.sub.K measures the steepness of the probability curve for a given quantized coefficient, and thus each Huffman code table is selected for a particular quantized coefficient by converting the value of PM.sub.K to the log (base 2) equivalent value, and using the converted value to specify the appropriate table. The table selected is ideally that table capable of encoding the quantized coefficient with the least number of bits.

When the four most significant bits of a quantized coefficient are non-zero, a Huffman coded special escape symbol from the appropriate table and the actual twelve-bit quantized coefficient are transmitted. The escape symbol is the last symbol found in the Appendix A tables.

Coding of the transform coefficients proceeds as described until the predictive mean falls below a preselected fixed threshold, termed the run length threshold. When this occurs, a run length code corresponding to the number of successive quantized coefficients having value zero is generated by coder unit 22 using table number 8 from appendix A. If the zero run extends to the end of the block, a special end of block code is generated by coder unit 22 from table number 8. The above described encoding process is graphically illustrated in FIG. 7 in which the trend of the predictive mean values is illustrated by the solid curve labelled PM.sub.i. The run length threshold is designated by the horizontal broken line, and the run length and end of block segments are designated with the legends RL and EOB, respectively. In the preferred embodiment, the numerical value of the run length threshold is one.

The code characters generated in coder unit 22 are stored in their order of generation in a rate buffer unit 25 having a predetermined maximum capacity N. Rate buffer 25 is a conventional unit capable of accepting binary input bits at a variable rate and generating bits at the output thereof at a constant rate of 2.39.times.10.sup.5 bits/sec. in the preferred embodiment. Since the rate at which coder unit 22 supplies binary bits to the input of rate buffer 25 can vary widely, while the buffer output bit rate is constant, a rate feedback technique is incorporated into the encoder of FIG. 1 to minimize the probability of buffer overflow. For this purpose, a signal representative of the number of bits actually transferred from coder unit 22 to buffer unit 25 is coupled to distortion calculator unit 18 for each replenishment block K, and the value of the distortion constant D.sub.k, which establishes the magnitude of the minimum quantization interval for quantizer unit 23, is recalculated. The calculation is performed in accordance with the following relationship:

DISTORTION CALCULATION

D=D'.sub.K +K.sub.D .multidot.BFN(B.sub.K -N/2)

where:

BFN(X)=(X/N-.vertline.X.vertline.)

D.sub.K =Distortion parameter for block K

D'.sub.K =Filtered distortion parameter

D'.sub.K =T.multidot.D'.sub.K-1 +(1-T)D.sub.K-1

where

T=a constant (close to 1)

K.sub.D =a constant

B.sub.K =# of bits in buffer for block K

N=Max. number of bits

In addition, the distortion calculator 18 updates the value of the block difference threshold T.sub.k for each encoded replenishment block in accordance with the following relationship:

REPLENISHMENT CALCULATION

T.sub.K =T.sub.INIT

for B.sub.LOW .ltoreq.B.sub.K .ltoreq.B.sub.HIGH

T.sub.K =T.sub.INIT+K.sub.R .multidot.BFN(B.sub.LOW -B.sub.K)

for B.sub.K <B.sub.LOW

T.sub.L =T.sub.INIT +K.sub.R .multidot.BFN(B.sub.K -B.sub.HIGH)

for B.sub.K >B.sub.HIGH

where

T.sub.K =replenishment threshold for block K

T.sub.INIT =initial threshold (about 5 for 8-bit input data)

K.sub.R =multiplier constant (about 25-75)

B.sub.LOW =low cutoff (about 0.1 of buffer)

B.sub.HIGH =high cutoff (about 0.75 of buffer)

Thus, as the buffer unit 25 fullness increases, the quantizer unit 23 provides coarser quantization intervals for the encoding of the transform coefficients, which tends to reduce the number of bits per symbol generated by coder unit 22, and thus tends to reduce the buffer fullness. In addition, the block difference threshold T.sub.k is raised, which tends to reduce the number of blocks selected for replenishment transformation, which also tends to reduce the buffer fullness. Similarly, when the buffer fullness decreases, the distortion constant D.sub.k provides finer quantization intervals for processing the transform coefficients, which tends to increase the number of bits per symbol generated by coder unit 22; and the block difference threshold T.sub.k is lowered, tending to select more blocks for replenishment processing, both of which tend to increase the buffer fullness.

The manner in which the serially generated code symbols representing the block replenishment information are arranged for transmission from buffer unit 25 to a decoder site is shown in FIG. 8, which illustrates one entire frame of information. As seen in this FIG., a frame of information commences with a frame sync code signal indicating the beginning of the frame, followed by a first control code signal B.sub.K which specifies the state of buffer fullness at the beginning of the frame. This control code signal is followed by the second control code signal D.sub.k, which is the actual value of the distortion constant at the beginning of the frame. Following this header information, which is used to reset the decoder shown in FIG. 2 at the beginning of each frame, are groups of block symbols containing the block replenishment information. After the last such group, a new frame sync code signal indicates the beginning of the succeeding frame of information.

Each group of block replenishment code symbols commences with the block address code, is followed by the two color quadrature component code symbols and continues with the coefficient code symbols, as indicated in FIG. 8. The arrangement of the coefficient codes indicated in FIG. 8 for the first block to be updated corresponds to the representative plot shown in FIG. 7.

The block replenishment symbols encoded in the manner described above are transmitted over a suitable communication link to the decoder system shown in FIG. 2, which provides inverse processing for the received information code symbols. Thus, after receipt of a frame sync code signal, the initial value of the distortion constant D.sub.k is coupled to an inverse quantizer unit 23' and the first block of replenishment information is initially decoded in decoder unit 22'. Decoder unit 22' contains the inverse code tables illustrated in appendix B which generate digital values from the received code symbols applied to the input thereof. The tables are arranged in a manner similar to that employed in coder unit 22, so that the block address codes, the color quadrature component codes and the DC coefficient code are all applied to their respective dedicated tables, while the cosine coefficient codes are applied on an individual basis to a selected one of six tables, depending on the value of the predictive mean calculated for each received quantized coefficient code. The emerging twelve-bit digital characters representing the quantized coefficients are inverse quantized in unit 23' by simply digitally multiplying each twelve-bit character with the value of D.sub.k, and the resulting inversely quantized cosine coefficients are stored in diagonal memory unit 21', along with the corresponding block address digital character, the digital character representing the DC term and the quadrature chrominance characters. The coefficients are transformed to the time domain by subjecting the coefficients to a two-step inverse cosine transformation in unit 20', with the intermediate resulting values being stored back in diagonal memory unit 21'. After the inverse cosine transformation process has been completed, the resulting pixel samples are stored in output memory unit 13, replacing the former eight by eight block of pixel information. It should be noted that the field of information stored in output memory unit 13' comprises a replica of the field of information stored in reference memory unit 14 of the FIG. 1 encoder.

The replenished field information contained in output memory unit 13' is coupled to an output digital processor unit 12', and thence to analog processor unit 11' which converts the digital video data to analog form. The emerging analog video signals are coupled to a suitable utilization device, such as a raster scan monitor.

As binary bits are transferred from buffer unit 25' to decoder unit 22', a signal representing the number of bits transferred is coupled to distortion calculator unit 18', which updates the value of the distortion constant D.sub.k for each block of replenishment information. The distortion calculator 18' employs the same algorithm as that noted above for distortion calculator 18.

Further compression of the input video is obtained by two additional signal processing techniques. Firstly, the luminance portion of a field of input video is subsampled to provide 256 lines/frame and 256 samples/line (65,536 pixels/frame), as opposed to the normal standard of 512 lines/frame and 512 samples/line (262,144 pixels/frame). After decoding, the full luminance field is recovered in output digital processor 12' by interpolative processing of the actual luminance samples transmitted through the system. Deleted luminance samples are reconstructed by summing adjacent samples and dividing the result by two. Thus, if A and B are adjacent luminance samples in a line, the intermediate sample is reconstructed by forming the sum A/2+B/2, and inserting this result between sample A and sample B.

The quadrature chrominance components for each 8.times.8 block of input video are compressed by first discarding the 2 least significant bits of each 8-bit digital chrominance quadrature component sample to form 6-bit digital characters. For each 8.times.8 block, the separate quadrature component samples are averaged over the block by summing every other component sample in a given line, dividing the result by four, storing the result obtained for each line, summing the result for the eight lines in a block and dividing the result by eight. Specifically, for the I quadrature component for block K, the initial sample array is: ##EQU3## The first row average is:

I.sub.AV1 .+-.(I.sub.11 +I.sub.13 +I.sub.15 +I.sub.17)/4

The last row average is:

I.sub.AV8 =(I.sub.81 +I.sub.83 +I.sub.85 +I.sub.87)/4

The block average is:

I.sub.AV BLOCK K =(I.sub.AV1 +I.sub.AV2 + . . . I.sub.AV8)/8

For the Q quadrature component, the first row average is:

Q.sub.AV1 =(Q.sub.12 +Q.sub.14 +Q.sub.16 +Q.sub.18)/4

The last row average is:

Q.sub.AV8 =(Q.sub.82 +Q.sub.84 +Q.sub.86 +Q.sub.88)/4

The block average is:

Q.sub.AV BLOCK K =(A.sub.AV1 +Q.sub.AV2 + . . . Q.sub.AV8)/8

The resulting block average for each component is stored as a 6-bit character in input memory 13. The full chrominance field is recovered in output digital processor 12' by inverse interpolative processing of the average chrominance samples transmitted through the system.

While suitable for many applications requiring information signal compression, the invention is especially adapted for use in a video teleconferencing system in which the prime criterion is bandwith reduction with minimum degradation in the subjective quality of the video images. At a typical sampling rate of 10.7 MHz for digital video transmission using eight-bits per digital sample character, the required bit rate to reliably transmit video information without compression is 8.56.times.10.sup.7 bits per second. By processing video signals according to the invention, using the same sample rate and the same size digital characters (i.e. eight-bits) in the analog to digital converter section of the encoder, and the digital to analog section of the decoder, compressed digital video can be transmitted from the encoder buffer unit 25 to the decoder buffer unit 25' at a rate of 2.39.times.10.sup.5 bits per second, which is 0.25 percent of the standard uncompressed digital bit rate. As will be appreciated by those skilled in the art, such a substantial reduction in the bit rate enables video information of good picture quality to be transmitted over a communication link having a substantially narrower bandwith, for example four conventional digital voice channels, with the result that substantially more information traffic can be routed over available communication links.

While the above provides a full and complete disclosure of the preferred embodiments of the invention, various modifications, alternate constructions and equivalents may be employed without departing from the true spirit and scope of the invention. For example, while eight by eight pixel blocks are employed in the preferred embodiment of the invention, blocks of other sizes may be employed, if desired. The relevant criteria for selecting appropriate block sizes are the processing time required by the block difference and decision unit 16, the cosine transform unit 20, the quantizer unit 23, the coder unit 22 and the distortion calculator 18. In general, larger blocks require more processing time, and the speed of currently available digital circuitry provides a practical limitation of about 32 pixels by 32 pixels on the maximum block size. In addition, for applications in which the amount of interframe image motion is excessive (i.e. greater than that normally present in video conferencing applications), a smaller block size may be necessary in order to provide decoded video signals of good subjective quality. Selection of smaller block sizes, however, increases the required minimum bit rate for the buffer units 25, 25'. In addition, different weighting factors may be employed for field merging, if desired; however, in the development of the preferred embodiment it has been discovered that a ratio of seven to one results in decoded video signals which are quite blurry, while a ratio approaching one to one results in a substantially increased number of blocks selected for replenishment, requiring a higher minimum bit rate for reliable transmission and decoding. The above description and illustrations, therefore, should not be construed as limiting the scope of the invention, which is defined by the appended claims. ##SPC1## ##SPC2## ##SPC3## ##SPC4##

* * * * *

Current U.S. Class:	348/400.1
Intern'l Class:	H04N 007/12
Field of Search:	358/133,135,136,138,13 382/43 364/725,726,727,826,827