JPEG

From Wikipedia, the free encyclopedia

**Joint Photographic Experts Group**
A photo of a flower compressed with successively more lossy compression ratios from left to right.
File extension:	`.jpeg, .jpg, .jpe .jfif, .jfi, .jif (containers)`
MIME type:	`image/jpeg`
Type code:	`JPEG`
Uniform Type Identifier:	public.jpeg
Developed by:	Joint Photographic Experts Group

In computing, JPEG (pronounced JAY-peg; IPA: [ˈdʒeɪpɛg]) is a commonly used standard method of compression for photographic images. The name JPEG stands for Joint Photographic Experts Group, the name of the joint ISO/CCITT committee which created the standard. The group was organized in 1986, issuing a standard in 1992 which was approved in 1994 as ISO 10918-1. JPEG should not be confused with MPEG, the Moving Picture Experts Group, which produces compression schemes for motion pictures.

JPEG provides for lossy compression of images (although there are variations on the standard baseline JPEG which are lossless). The file format which employs this compression is commonly also called JPEG; the most common file extension for this format is .jpg, though .jpeg, .jpe, .jfif and .jif are also used.

JPEG itself specifies both the codec defining how an image is transformed into a stream of bytes, and the file format used to contain that stream. The file format is known as 'JPEG Interchange Format' (Often confused with the JPEG File Interchange Format), and is specified in Annex B of the standard. It is possible for JPEG data to be embedded in other file types, such as TIFF format images.

JPEG/JFIF is the format most used for storing and transmitting photographs on the World Wide Web. For this application, it is preferred to formats such as GIF, which has a limit of 256 distinct colors that is insufficient for colour photographs, and PNG, which produces much larger image files for this type of image. The compression algorithm is not as well suited for line drawings and other textual or iconic graphics, and thus the PNG and GIF formats are preferred for these types of images.

The MIME media type for JPEG is image/jpeg (defined in RFC 1341).

[edit] JPEG Codec

[edit] Encoding

Many of the options in the JPEG standard are not commonly used. Here is a brief description of one of the more common methods of encoding when applied to an input that has 24 bits per pixel (eight each of red, green, and blue). This particular option is a lossy data compression method.

[edit] Color space transformation

First, the image is converted from RGB into a different color space called YCbCr. This is the same as the color space used by PAL, MAC and Digital color television transmission but not by NTSC which uses IQ, a totally different color space based on different axes and asymmetrical bandwidth.

The Y component represents the brightness of a pixel
The Cb and Cr components together represent the chrominance

[edit] Block splitting

After subsampling each channel must be split into 8x8 blocks, if the data for a channel does not represent an integer number of blocks then the encoder must fill the remaining area of the incomplete blocks with some form of dummy data (repeating the edge pixels is a common but probablly non-optimal technique)

[edit] Downsampling

The human eye can see more detail in the Y component than in Cb and Cr. Using this knowledge, encoders can be designed to compress images more efficiently.

The above transformation enables the next step, which is to reduce the Cb and Cr components (called "downsampling" or "chroma subsampling"). The ratios at which the downsampling can be done on JPEG are 4:4:4 (no downsampling), 4:2:2 (reduce by factor of 2 in horizontal direction), and most commonly 4:2:0 (reduce by factor of 2 in horizontal and vertical directions). For the rest of the compression process, Y, Cb and Cr are processed separately and in a very similar manner.

[edit] Discrete cosine transform

The DCT transforms 64 pixels to a linear combination of these 64 squares

The 8x8 subimage shown in 8-bit greyscale

The 8x8 subimage shown scaled to full 8-bit range greyscale

The compressed 8x8-squares are visible in the scaled up picture, as well as the visual artifacts of the lossy compression

Next, each component (Y, Cb, Cr) of the image is "tiled" into sections of eight by eight pixels each, then each tile is converted to frequency space using a two-dimensional forward discrete cosine transform (DCT, type II).

If one such 8×8 8-bit subimage is:

$\begin{bmatrix} 52 & 55 & 61 & 66 & 70 & 61 & 64 & 73 \\ 63 & 59 & 55 & 90 & 109 & 85 & 69 & 72 \\ 62 & 59 & 68 & 113 & 144 & 104 & 66 & 73 \\ 63 & 58 & 71 & 122 & 154 & 106 & 70 & 69 \\ 67 & 61 & 68 & 104 & 126 & 88 & 68 & 70 \\ 79 & 65 & 60 & 70 & 77 & 68 & 58 & 75 \\ 85 & 71 & 64 & 59 & 55 & 61 & 65 & 83 \\ 87 & 79 & 69 & 68 & 65 & 76 & 78 & 94 \end{bmatrix}$

which, if we now subtract 128 from each element, results in

$\begin{bmatrix} -76 & -73 & -67 & -62 & -58 & -67 & -64 & -55 \\ -65 & -69 & -73 & -38 & -19 & -43 & -59 & -56 \\ -66 & -69 & -60 & -15 & 16 & -24 & -62 & -55 \\ -65 & -70 & -57 & -6 & 26 & -22 & -58 & -59 \\ -61 & -67 & -60 & -24 & -2 & -40 & -60 & -58 \\ -49 & -63 & -68 & -58 & -51 & -60 & -70 & -53 \\ -43 & -57 & -64 & -69 & -73 & -67 & -63 & -45 \\ -41 & -49 & -59 & -60 & -63 & -52 & -50 & -34 \end{bmatrix}$

and then taking the DCT and rounding to the nearest integer results in

$\begin{bmatrix} -415 & -30 & -61 & 27 & 56 & -20 & -2 & 0 \\ 4 & -22 & -61 & 10 & 13 & -7 & -9 & 5 \\ -47 & 7 & 77 & -25 & -29 & 10 & 5 & -6 \\ -49 & 12 & 34 & -15 & -10 & 6 & 2 & 2 \\ 12 & -7 & -13 & -4 & -2 & 2 & -3 & 3 \\ -8 & 3 & 2 & -6 & -2 & 1 & 4 & 2 \\ -1 & 0 & 0 & -2 & -1 & -3 & 4 & -1 \\ 0 & 0 & -1 & -4 & -1 & 0 & 1 & 2 \end{bmatrix}$

Note the rather large value of the top-left corner. This is the DC coefficient. The remaining 63 coefficients are called the AC coefficients.

[edit] Quantization

The human eye is good at seeing small differences in brightness over a relatively large area, but not so good at distinguishing the exact strength of a high frequency brightness variation. This fact allows one to get away with greatly reducing the amount of information in the high frequency components. This is done by simply dividing each component in the frequency domain by a constant for that component, and then rounding to the nearest integer. This is the main lossy operation in the whole process. As a result of this, it is typically the case that many of the higher frequency components are rounded to zero, and many of the rest become small positive or negative numbers.

A common quantization matrix is:

$\begin{bmatrix} 16 & 11 & 10 & 16 & 24 & 40 & 51 & 61 \\ 12 & 12 & 14 & 19 & 26 & 58 & 60 & 55 \\ 14 & 13 & 16 & 24 & 40 & 57 & 69 & 56 \\ 14 & 17 & 22 & 29 & 51 & 87 & 80 & 62 \\ 18 & 22 & 37 & 56 & 68 & 109 & 103 & 77 \\ 24 & 35 & 55 & 64 & 81 & 104 & 113 & 92 \\ 49 & 64 & 78 & 87 & 103 & 121 & 120 & 101 \\ 72 & 92 & 95 & 98 & 112 & 100 & 103 & 99 \end{bmatrix}$

Using this quantization matrix with the DCT coefficient matrix from above results in:

$\begin{bmatrix} -26 & -3 & -6 & 2 & 2 & -1 & 0 & 0 \\ 0 & -2 & -4 & 1 & 1 & 0 & 0 & 0 \\ -3 & 1 & 5 & -1 & -1 & 0 & 0 & 0 \\ -4 & 1 & 2 & -1 & 0 & 0 & 0 & 0 \\ 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{bmatrix}$

For example, using −415 (the DC coefficient) and rounding to the nearest integer

$\mathrm{round} \left( \frac{-415}{16} \right) = \mathrm{round} \left( -25.9375 \right) = -26$

[edit] Entropy coding

Entropy coding is a special form of lossless data compression. It involves arranging the image components in a "zigzag" order employing run-length encoding (RLE) algorithm that groups similar frequencies together, inserting length coding zeros, and then using Huffman coding on what is left.

The JPEG standard also allows, but does not require, the use of arithmetic coding which is mathematically superior to Huffman coding. However, this feature is rarely used as it is covered by patents and because it is much slower to encode and decode compared to Huffman coding. Arithmetic coding typically makes files about 5% smaller.

The zig-zag sequence for the above quantized coefficients would be:

−26,
−3, 0,
−3, −2, −6,
2, −4, 1, −4,
1, 1, 5, 1, 2,
−1, 1, −1, 2, 0, 0,
0, 0, 0, −1, −1, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0,
0, 0, 0, 0,
0, 0, 0,
0, 0,
0

JPEG has a special Huffman code word for ending the sequence prematurely when the remaining coefficients are zero.

Using this special code word: "EOB", the sequence becomes:

−26,
−3, 0,
−3, −2, −6,
2, −4, 1, −4,
1, 1, 5, 1, 2,
−1, 1, −1, 2, 0, 0,
0, 0, 0, −1, −1, EOB

JPEG's other code words represent combinations of (a) the number of significant bits of a coefficient, including sign, and (b) the number of consecutive zero coefficients that follow it. (Once you know how many bits to expect, it takes 1 bit to represent the choices {-1, +1}, 2 bits to represent the choices {-3, -2, +2, +3}, and so forth.) In our example block, most of the quantized coefficients are small numbers that are not followed immediately by a zero coefficient. These more-frequent cases will be represented by shorter code words.

The JPEG standard provides general-purpose huffman tables; encoders may also choose to generate huffman tables optimized for the actual frequency distributions in images being encoded.

[edit] Compression ratio and artifacts

This image shows the (accentuated) difference between a jpg-saved image and the original. Note especially the changes occurring near sharp edges.

The resulting compression ratio can be varied according to need by being more or less aggressive in the divisors used in the quantization phase. Ten to one compression usually results in an image that cannot be distinguished by eye from the original. 100 to one compression is usually possible, but will look distinctly artifacted compared to the original. The appropriate level of compression depends on the use to which the image will be put.

Those who use the World Wide Web may be familiar with the irregularities known as compression artifacts that appear in JPEG images. These are due to the quantization step of the JPEG algorithm. They are especially noticeable around eyes in pictures of faces. They can be reduced by choosing a lower level of compression; they may be eliminated by saving an image using a lossless file format, though for photographic images this will usually result in a larger file size. Compression artifacts make low-quality JPEGs unacceptable for storing heightmaps. The images created with ray-tracing programs have noticeable blocky shapes on the terrain.

Some programs allow the user to vary the amount by which individual blocks are compressed. Stronger compression is applied to areas of the image that show fewer artifacts. This way it is possible to make a JPEG file smaller and prettier by hand.

[edit] Decoding

Decoding to display the image consists of doing all the above in reverse.

Taking the DCT coefficient matrix (after adding the difference of the DC coefficient back in)

and multiplying it by the quantization matrix from above results in

$\begin{bmatrix} -416 & -33 & -60 & 32 & 48 & -40 & 0 & 0 \\ 0 & -24 & -56 & 19 & 26 & 0 & 0 & 0 \\ -42 & 13 & 80 & -24 & -40 & 0 & 0 & 0 \\ -56 & 17 & 44 & -29 & 0 & 0 & 0 & 0 \\ 18 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{bmatrix}$

which closely resembles the original DCT coefficient matrix for the top-left portion. Taking the inverse DCT (type-III DCT) results in an image with values (still shifted down by 128)

Notice the slight differences between the original (top) and decompressed image (bottom) which is most readily seen in the bottom-left corner.

$\begin{bmatrix} -68 & -65 & -73 & -70 & -58 & -67 & -70 & -48 \\ -70 & -72 & -72 & -45 & -20 & -40 & -65 & -57 \\ -68 & -76 & -66 & -15 & 22 & -12 & -58 & -61 \\ -62 & -72 & -60 & -6 & 28 & -12 & -59 & -56 \\ -59 & -66 & -63 & -28 & -8 & -42 & -69 & -52 \\ -60 & -60 & -67 & -60 & -50 & -68 & -75 & -50 \\ -54 & -46 & -61 & -74 & -65 & -64 & -63 & -45 \\ -45 & -32 & -51 & -72 & -58 & -45 & -45 & -39 \end{bmatrix}$

and adding 128 to each entry

$\begin{bmatrix} 60 & 63 & 55 & 58 & 70 & 61 & 58 & 80 \\ 58 & 56 & 56 & 83 & 108 & 88 & 63 & 71 \\ 60 & 52 & 62 & 113 & 150 & 116 & 70 & 67 \\ 66 & 56 & 68 & 122 & 156 & 116 & 69 & 72 \\ 69 & 62 & 65 & 100 & 120 & 86 & 59 & 76 \\ 68 & 68 & 61 & 68 & 78 & 60 & 53 & 78 \\ 74 & 82 & 67 & 54 & 63 & 64 & 65 & 83 \\ 83 & 96 & 77 & 56 & 70 & 83 & 83 & 89 \end{bmatrix}$

This is the uncompressed subimage and can be compared to the original subimage (also see images to the right) by taking the difference (original - uncompressed) results in error values

$\begin{bmatrix} -8 & -8 & 6 & 8 & 0 & 0 & 6 & -7 \\ 5 & 3 & -1 & 7 & 1 & -3 & 6 & 1 \\ 2 & 7 & 6 & 0 & -6 & -12 & -4 & 6 \\ -3 & 2 & 3 & 0 & -2 & -10 & 1 & -3 \\ -2 & -1 & 3 & 4 & 6 & 2 & 9 & -6 \\ 11 & -3 & -1 & 2 & -1 & 8 & 5 & -3 \\ 11 & -11 & -3 & 5 & -8 & -3 & 0 & 0 \\ 4 & -17 & -8 & 12 & -5 & -7 & -5 & 5 \end{bmatrix}$

with an average absolute error of about 5 values per pixels (i.e., $\frac{1}{64} \sum_{x=1}^8 \sum_{y=1}^8 |e(x,y)| = 4.8125$ ).

The error is most noticeable in the bottom-left corner where the bottom-left pixel becomes darker than the pixel to its immediate right.

[edit] Lossless editing

A number of alterations can be performed to a JPEG image without any quality loss. Blocks can be rotated in 90 degree increments, flipped in the horizontal vertical and diagonal axes and moved about in the image. Not all blocks from the original image need to be used in the modified one.

The top and left of a JPEG image must lie on a block boundry but the bottom and right need not do so, This limits the possible losless crop operations and also what flips and rotates can be performed on an image whose edges do not lie on a block boundry for all channels.

It is also possible to transform between baseline and progressive formats without any loss of quality since the only difference is the order in which the coefficiants are placed in the file.

Furthermore if a jpeg is decompressed edited and then recompressed using the same chroma subsampling and quantisation tables then unedited areas of the image should have minimal quality loss.

When using lossless cropping it is important to realise that if the crop region has its bottom or left side is not on a block boundry then the rest of the data from the partially used blocks will still be present in the cropped file and can be recovered relatively easilly by anyone with a hex editor and an understanding of the format.

[edit] JPEG Interchange Format (the file format)

Please help improve this article by expanding this section.
Further information might be found on the talk page or at requests for expansion. Please remove this message once the section has been expanded.

[edit] Shortcomings

There are three things lacking from a JPEG Interchange Format file which prevent it from being universally understood:

Colour Space definition
Component Sub-Sampling Registration definition
Pixel Aspect Ratio definition

Several further standards have attempted to rectify this situation. These include, 'JPEG File Interchange Format', 'Exchangeable image file format' and ICC color profiles

[edit] Color profile

Many JPEG files embed an ICC color profile (color space). Commonly used color profiles are sRGB color space and Adobe RGB color space. Because these color spaces use a non-linear transformation, the dynamic range of an 8-bit JPEG file is about 11 stops.

[edit] Usage

JPEG is at its best on photographs and paintings of realistic scenes with smooth variations of tone and color. In this case it usually performs much better than purely lossless methods while still giving a good looking image. In fact, it will usually produce much better results for such images than, for example, GIF, which can be lossless as long as the image contains 256 or fewer unique colors but requires severe quantization for full-color images.

Some operations on JPEG images, such as rotation by multiples of 90°, can be performed losslessly as long as the image size is a multiple of eight pixels in both directions. One program which can do this is the jpegtran utility which comes with the reference implementation.

[edit] Photographs

JPEG compression artifacts blend well into photographs with detailed non-uniform textures, allowing higher compression ratios.

Full quality (Q = 100), filesize 83261 B.

High quality (Q = 50), filesize 15138 B.

Medium quality (Q = 25), filesize 9553 B.

Low quality (Q = 10), filesize 4787 B.

Lowest quality (Q = 1), filesize 1523 B.

NOTE: The above images are not IEEE / CCIR / EBU test images, and the encoder settings are not specified or available.

The mid-quality photo uses only one sixth the storage space but has little noticeable loss of detail or visible artifacts. However, once a certain threshold of compression is passed, compressed images show increasingly visible defects. See the article on rate distortion theory for a mathematical explanation of this threshold effect.

[edit] Medical Imaging: JPEG's 12-bit mode

There are many medical imaging systems that create and process 12-bit JPEG images. The 12-bit JPEG format has been part of the JPEG specification for some time, but very few consumer programs (including web browsers) support this rarely used JPEG format.

[edit] Potential patent issues

In 2002 Forgent Networks asserted that it owned and would enforce patent rights on the JPEG technology, arising from a patent that had been filed on October 27, 1986, and granted on October 6, 1987 (U.S. Patent 4,698,672 ). The announcement created a furor reminiscent of Unisys' attempts to assert its rights over the GIF image compression standard.

The JPEG committee investigated the patent claims in 2002 and were of the opinion that they were invalidated by prior art.^[1] Others also concluded that Forgent did not have a patent that covered JPEG.^[2] Nevertheless, between 2002 and 2004 Forgent was able to obtain about US$90 million by licensing their patent to some 30 companies. In April 2004, Forgent sued 31 other companies to enforce further license payments. In July of the same year, a consortium of 21 large computer companies filed a countersuit, with the goal of invalidating the patent. Surprisingly, in contrast to the other major computer companies such as Sony and Philips, Microsoft launched a major lawsuit against Forgent. In February 2006, the United States Patent and Trademark Office agreed to re-examine Forgent's JPEG patent at the request of the Public Patent Foundation.^[3] On May 26, 2006 the USPTO found the patent invalid based on prior art. The USPTO also found that Forgent knew about the prior art, and did not tell the Patent Office, making any appeal to reinstate the patent highly unlikely to succeed.^[4]

Forgent also possesses a similar patent granted by the European Patent Office in 1994, though it is unclear how enforceable it is.^[5]

As of October 6, 2006, the U.S. patent's 20-year term appears to have expired, and in November 2006, Forgent agreed to abandon enforcement of patent claims against use of the JPEG standard.^[6]

The JPEG committee has as one of its explicit goals that their standards (in particular their baseline methods) be implementable without payment of license fees, and they have secured appropriate license rights for their upcoming JPEG 2000 standard from over 20 large organizations.