Guide to FM-94 BUFR (Chapters 1-3)

3.1.1.2 Section 0 - Indicator Section

Structure

SECTION 0		Section 1	Section 2	Section 3	Section 4	Section 5
Octet No.	Contents
1 – 4	"BUFR" (coded according to the CCITT International Alphabet No. 5)
5 – 7	Total length of BUFR message, in octets (including Section 0)
8	BUFR edition number (currently 3)

Total message length (octets 5 – 7): The earlier editions of BUFR did not include the total message length. Thus, in decoding BUFR Edition 0 and 1 messages, there was no way of determining the entire length of the message without scanning ahead to find the individual lengths of each of the sections. Edition 2 eliminated this problem by including the total message length in octets 5 – 7.

Edition Number (octet 8): By design, BUFR Edition 2 contained the BUFR Edition number in octet 8, the same octet position relative to the start of the message as it was in Editions 0 and 1. By keeping the relative position fixed, a decoder program can determine at the outset which BUFR version was used for a particular message and then behave accordingly. This meant that archives of records in BUFR Editions 0 or 1 did not need to be updated.

Edition number changes: The Edition number will change only if there is a structural change to the data representation system such that an existing and functioning BUFR decoder would fail to work properly if given a "new" record to decode. Edition changes can come about in three main ways. First, if the basic bit or octet structure of the BUFR record were changed, for example by the addition of something new in one of the "fixed format" portions of the record, computer program changes would obviously be required for the programs to work properly. The addition of total BUFR message length to octets 5 – 7 of the Indicator Section fell in this category – it caused the Edition number to change from 1 to 2. The WMO community expects these changes to be kept to a bare minimum.

The second way is if the data description operators in Table C (Data description operators) are augmented. These operator descriptors are qualitatively different from simple data descriptors: where the data descriptors just passively describe the data in the record, the operator descriptors are, in effect, instructions to the decoding program to undertake some particular action. Table C defines what actions are possible. Descriptors of type 1 (F=1), the replication operators, are also in this category since they too tell the computer program to do something. Unfortunately, not all of the "operator" type descriptors are collected in Table C. Some of the nominal data descriptors, in particular the "increment" descriptors found in Table B, Classes 4, 5, 6, and 7, take on the character of operators in conjunction with data replication, as well as the operator qualifiers in Table B, Class 31. These topics will be expanded on further later in Chapter 3.1.

A third change that would require a new Edition would be a change to the Regulations and/or the many notes scattered through the documentation (The "notes", by the way, are as important as the "Regulations" in formally defining BUFR - they contain many of the details that flesh out the rather sparse regulations. Ignore them at your peril.). This is not particularly likely to happen - more likely will be clarifications to the Regulations or notes that will serve to make the rules more precise in (currently) possibly ambiguous cases. Whether these cases should be considered as requiring an Edition number change is a matter of some judgment. The WMO will be the final arbiter.

Sample message decomposition (Indicator Section): The Indicator Section of the sample BUFR Message shown in Figure 3.1.1-1 is decomposed in detail below. The hexadecimal equivalent of the first four octets is shown to clarify the representation of the four characters “B”, “U”, “F”, and “R”. Note also that the value of the bits in octet 7 is 52 and the value of the bits in octet 8 is 3.

octet number:

        1      |       2      |        3       |       4       |       5       |       6       |       7        |       8

binary string:

01000010 01010101 01000110 01010010 00000000 00000000 00110100 0000011

hexadecimal:

   4      2       5      5       4       6      5       2       0      0       0      0      3      4       0     3

decoded:

        B             U               F               R                                               52                 3

                                                          Length of message in octets ----+-------¦

                                                                                                     BUFR Edition ----+

Figure 3.1.1-2. Section 0

3.1.1.3 Section 1 - Identification Section

Structure

C O N T I N U O U S B I N A R Y S T R E A M
Section 0		SECTION 1	Section 2	Section 3	Section 4	Section 5
Octet No.	Contents
1 – 3	Length of section, in octets
4	BUFR master table number – this provides for BUFR to be used to represent data from other disciplines, with their own versions of master tables and local tables. For example, this octet is zero for standard WMO FM 94 BUFR tables, but ten for standard IOC FM 94 BUFR Tables whose, use is focused on oceanographic data.
5	Originating/generating sub-centre (defined by Originating/generating centre)
6	Originating/generating centre (Common Code tableC-1)
7	Update sequence number (zero for original BUFR messages; incremented for updates)
8	Bit 1= 0 No optional section
	= 1 Optional section included
	Bits 2 – 8 set to zero (reserved)
9	Data category (BUFR Table A)
10	Data sub-category (defined by local ADP centres)
11	Version number of master tables used (currently 9 for WMO FM 94 BUFR tables)
12	Version number of local tables used to augment the master table in use
13	Year of century
14	Month
15	Day
16	Hour
17	Minute
18 -	Reserved for local use by ADP centres

Length of section (octets 1 – 3): The length of Section 1 can vary between BUFR messages. Beginning with Octet 18, a data processing center may add any type of information they choose. A decoding program need not know what that information may be. Knowing what the length of the Section is, as indicated in octets 1-3, a decoder program can skip over the information that begins at octet 18 and position itself at the next section, either Section 2, if included, or Section 3. Bit 1 of octet 8 indicates if Section 2 is included. If there is no information beginning at octet 18, one octet must still be included (and set to 0) in order to have an even number of octets within the section.

Originating/generating sub-centre (octet 5) and Originating/generating center (octet 6): Octet 6 is used to identify the national (or international) originating/generating centres, using the same Common Code table (C – 1) as is in use for GRIB. This table is coordinated and maintained by the WMO and published as part of the codes Manual. Any national sub-center numbers that may be required are generated by the national (or international) center in question and that number is to be placed in octet 5. List of sub-centres numbers should be passed to the WMO Secretariat for publication in the Manual.

Update sequence number (octet 7): This feature is not widely used, but it is a powerful one. Note that the rule does require one to re-send an entire message if even only one element in the message is a correction of a previous message element. The "associated field" (see Section 3.2.6) is used to indicate which element(s) is(are) the corrected one(s) within the total message.

Optional Section 2 (octet 8): This section is not usually sent in international messages but it is put to use in some computer centers that use BUFR frequently in a data base context. Some samples are given in Section 3.1.1.4. If it is present, the flag in octet 8 must be set to 1.

Data category (octet 9): The data category (taken from BUFR Table A) provides a quick check of the type of data in the BUFR message. Processing centres can use this information in their observational data ingest processing suite.

Data sub-category: This is purely a local option, useful in processing the observational data after it has been decoded from BUFR. By adding this information to the BUFR files in which the ingested data are placed, a processing centre knows in considerable detail just what sort of data is in a BUFR message. This can make the choice of subsequent processors that much easier. It also makes it possible to search through a collection of various data types, encoded in BUFR, and select out only those for which there is a special interest. This has obvious applications in a data base context. As an example here are the sub-types currently in use at the National Centers for Environmental Prediction, Washington, DC, USA:

BUFR Data Category 0: Surface data – land
Data Sub-type	Description
1	Synoptic – manual and Automatic
7	Aviation – METAR
11	SHEF
12	Aviation – SCD
20	MESONET – Denver, urban
21	MESONET – RAWS (NIFC)
22	MESONET – MesoWest
23	MESONET – APRS Weather
24	MESONET – Kansas DOT
25	MESONET – Florida
30	MESONET – Other
BUFR Data Category 1: Surface data – sea
Data Sub-type	Description
1	Ship – manual and automatic
2	Drifting buoy
3	Moored buoy
4	Land based C-MAN station
5	Tide gage
6	Sea level pressure bogus
7	Coast guard
8	Moisture bogus
9	SSMI
BUFR Data Category 2: Vertical soundings (other than satellite)
Data Sub-type	Description
0	Unassigned
1	Rawinsonde - fixed land
2	Rawinsonde - mobile land
3	Rawinsonde – ship
4	Dropwinsonde
5	Pibal
7	Wind Profiler (from NOAA)
8	NEXRAD winds
9	Wind profiler (from PILOT)
BUFR Data Category 3: Vertical soundings (satellite)
Data Sub-type	Description
1	Geostationary
2	Polar orbiting
3	Sun synchronous
BUFR Data Category 4: Single level upper-air (other than satellite)
Data Sub-type	Description
0	Unassigned
1	AIREP
2	PIREP
3	AMDAR
4	ACARS (from ARINC)
5	RECCO – flight level
6	E-ADAS
BUFR Data Category 5: Single level upper-air (satellite)
Data Sub-type	Description
10	NESDIS SATWIND: GOES – High Density IR
11	NESDIS SATWIND: GOES – High Density WV Imagery
12	NESDIS SATWIND: GOES – Hi Density Visible
13	NESDIS SATWIND: GOES – Picture Triplet
14	NESDIS SATWIND: GOES – Hi Density WV Sounding
21	INDIA SATWIND: INSAT – IR
22	INDIA SATWIND: INSAT – Visible
23	INDIA SATWIND: INSAT – WV Imagery
41	JMA SATWIND: GMS – IR
42	JMA SATWIND: GMS – Visible
43	JMA SATWIND: GMS – WV Imagery
50	NESDIS SATWIND: GMS – IR
51	NESDIS SATWIND: GMS – WV Imagery
64	EUMETSAT SATWIND: METEOSAT – IR
65	EUMETSAT SATWIND: METEOSAT – VIS
66	EUMETSAT SATWIND: METEOSAT – WV Imagery
BUFR Data Category 12: Surface data (satellite)
1	SSM/I – Brightness temperatures
2	SSM/I – Derived products
3	GPS – Integrated precipitable water
5	ERS – SAR
9	ERS – Radar altimeter data
10	Navy sea surface temperatures
11	NESDIS sea surface temperatures
12	Navy high resolution sea surface temperatures
103	SSM/I – Neural net 3 products
137	QUIKSCAT data
BUFR Data Category 31: Oceanographic data
1	BATHY
2	TESAC
3	TRACKOB
11	NLSA ERS2: Altimeter – high resolution
12	NLSA TOPEX: Altimeter – high resolution
13	NLSA TOPEX: Altimeter – low resolution
14	NLSA GFO: Altimeter – high resolution

Date/time (octets 13 – 17): The Manual suggests placing the date/time "most typical for the BUFR message content" (whatever that may mean) in the appropriate octets. For synoptic observations, the nominal synoptic time is obviously appropriate. But the exact time of the observation can be placed in the body of the message if this is of interest or value to the users of the data. Collections of satellite observations, which are inherently asynoptic, by convention (at least as NOAA does) have the time of the first observation of the collection in the date/time octets. The exact times for each satellite observation will, of course, be in the body of the message.

As the Year 2000 rollover period approached, it was realized the Year of century was not being encoded uniformly because the regulations specifying the values to use for Year of century were not clearly stated. To that end, a new note was added to the Identification Section. The new note reads: “To specify the year 2000, octet 13 (Year of century) must contain a value of 100. To specify the year 2001, octet 13 must contain a value of 1 (by International Convention, the date of 1 January 2000 was the first day of the hundredth year of the twentieth century and the date of 1 January 2001 was the first day of the first year of the twenty-first century). One should also note that year 2000 was a leap year, and February 29, 2000 exists.” Lack of specification of the Century in BUFR was also felt to be a deficiency, and some processing centres have begun the practice of using octet 18 (see below) of this section for that value.

Reserved for use ..." (octets 18 - ): It is not expected that international BUFR messages will contain anything past octet 18. However, octet 18 itself, which is also reserved for local use, must be present in order to maintain an even number of octets in the Identification Section. Traditionally, octet 18 was set to zero. However, as noted above, some centres now use this octet for the Century. Nevertheless, there is no real damage if Section 1 is "extended" past octet 18, because the "Length of section" in octets 1-3 indicates the full size of Section 1. Any operational decoding program worthy of the name will check the number in octets 1-3 and respond accordingly, presumably by skipping the extra material.

Sample message decomposition (Identification Section): The Identification Section of the sample BUFR Message shown in Figure 3.1.1-1 is decomposed in detail below:

Figure 3.1.1-3. Section 1

octet number:

        1      |       2       |        3      |       4       |       5       |       6       |       7       |       8

binary string:

00000000 00000000 00010010 00000000 00000000 00111010 00000000 00000000

decoded:

        0              0              18              0                0              58            0                0

      length of section -----+-------¦

              standard BUFR tables ------+-------¦

                         originating center (US Navy - FNMOC) ---+------¦

                                                             flag indicating Section 2 not included -----+------

octet number:

        9      |      10       |      11     |     12       |     13       |      14      |      15      |      16

binary string:

00000000 00000000 00001001 00000001 00000001 00000100 00011101 00001100

decoded:

        0              0                 9              1              1               4               29              12

-------+----|

data category

data sub-category-+--|

version of master tables ---+----¦

               version of local tables ---+--------¦

                                           year of century -----+------|

                                                                           month ---+-------¦

                                                                                               day ----+------¦

                                                                                                              hour ---+--------|

octet number:          17              18

binary string:      00000000 00000000

decoded:                    0                0

               minute -----+-----¦

                            local use ----+-------|

3.1.1.4 Section 2 - Optional Section

Structure

C O N T I N U O U S B I N A R Y S T R E A M
Section 0		Section 1	SECTION 2	Section 3	Section 4	Section 5
Octet No.	Contents
1 – 3	Length of section, in octets
4	Set to zero (reserved)
5 -	Reserved for use by ADP centres

Use

Section 2 may or may not be included in any BUFR message. When it is contained within a BUFR message, bit 1 of octet 8 in Section 1 is set to 1. If Section 2 is not included in a message then bit 1 of octet 8 in Section 1 is set to 0. Section 2 may be used for any purpose by an originating center. The only restrictions on the use of Section 2 are that octets 1 - 3 are set to the length of the section, octet 4 is set to zero and the section contains an even number of octets.

A typical use of this optional section could be in a data base context. The section might contain pointers into the data section of the message, pointers that indicate the relative location of the start of individual sets of observations (one station's worth, for example) in the data. There could also be some sort of index term included, such as the WMO block and station number. This would make it quite easy to find a particular observation quickly and avoid decoding the whole message just to find one or two specific data elements.

Note the Optional Section was not present in the sample BUFR Message shown in Figure 3.1.1-1.

3.1.1.5 Section 3 - Data description Section

Structure

C O N T I N U O U S B I N A R Y S T R E A M
Section 0		Section 1	Section 2	SECTION 3	Section 4	Section 5
Octet No.	Contents
1 – 3	Length of section, in octets
4	Set to zero (reserved)
5 – 6	Number of data subsets
7	Bit 1 = 1 observed data
	= 0 other data
	Bit 2 = 1 compressed data
	= 0 non-compressed data
	Bit 3 - 8 set to zero (reserved)
8 -	A collection of descriptors which define the form and content of individual data elements comprising one data subset in the data section (Section 4)

Number of data subsets (octets 5 – 6): BUFR Regulation 94.5.2 states “ … Octet 8 and subsequent octets shall contain a collection of descriptors which define the form and content of individual data elements in the Data Section. A “data subset” shall be defined as the subset of data described by one single application of this collection of descriptors.” In this context, the "collection of descriptors" means ALL the descriptors included in Section 3 of the BUFR message. In other words, one pass through the complete collection of descriptors will allow one to decode one data subset from Section 4. One then loops back in the descriptor list for as many times as indicated by the Number of data subsets in octets 5 – 6. All the data in Section 4 are properly described by repeated use of the same set of descriptors from Section 3.

This does not imply that the data subsets are themselves identical in format. The use of delayed replication, as in a collection of TEMPs with varying numbers of significant levels, could cause variations in format (octet count) among data subsets. But they are still considered "subsets" in that the same set of descriptors will properly describe each individual set. The use of the delayed replication descriptor is what makes this possible, and is what delayed replication was designed for.

As we will see in Chapter 3.1.6, certain descriptor operators, from Table C can be used to redefine reference values, data lengths, scale factors, and add associated fields. There is also a group of descriptors that "remain in effect until superseded by redefinition". However, Regulation 94.5.3.9 states, “If a BUFR message is made up of more than one subset, each subset shall be treated as though it was the first subset encountered. This Regulation means that ALL of these redefinitions or "remain in effect" properties are canceled when one cycles back to reuse a set of descriptors for a new data subset. You wipe the slate clean and start as though it was the first time.

Even though data subsets may be compressed and, as a result, the individual elements in each data subset are all reordered, the data subset concept still holds. The data subset count must be included in the correct location, and must be correct. It is impossible to decompress a message without that information; and even if the data are not compressed the count is necessary to retrieve all the data subsets in a given message.

If octets 5-6 indicate that there is more than one data subset in the message, with the total number of the subsets given in those octets, then multiple sets of observations, all with the same format (as described by the data descriptors) will be found in Section 4. This is, for example, a means of building "collectives" of observations. Doing so realizes a large portion of the potential efficiency of BUFR.

Flag Bit 1 (octet 7): Conceptually, one subset is a collection of related meteorological data. For observational data (Flag bit 1 = 1), each subset usually corresponds to one “observation”, where "observation", in this context, could mean one surface synoptic report, one rawinsonde ascent, one profiler sounding, one satellite derived sounding with radiances, etc. No examples of non-observational data subsets (Flag bit 1 = 0) are given in the BUFR specifications in the Manual on Codes, but a typical one would be a message consisting of a collection of numerical model forecasts of "soundings" at grid-points or other specific locations. Each forecast sounding (pressure, temperature, wind, relative humidity, whatever, at the many levels of the model) would then be one data subset.

Flag Bit 2 (octet 7): If the data in Section 4 is compressed, bit 2 of octet 7 is set to one. If the data is not compressed, it is set to zero. The nature of "data compression" will be described in Chapter 3.1.5.

Sample message decomposition (Data Description Section): The Data Description Section of the sample BUFR Message shown in Figure 3.1.1-1 is decomposed in detail below. The data descriptors are given in octets 8 – 13. Note that octet 14 has been added and set to zero to ensure the Data Description Section contains an even number of octets.

Figure 3.1.1-4. Section 3

octet number:

        1      |       2       |        3      |       4       |       5       |       6       |       7       |

binary string:

00000000 00000000 00001110 00000000 00000000 00000001 10000000

decoded:                           14           0                0              1         ||

length of section -+----------------¦

                                    reserved -----+------¦

                               number of data subsets -----------+-------------¦

                                                        flag indicating observed data+

                                              flag indicating non-compressed data+

octet number:

        8      |       9       |      10      |     11       |     12       |      13      |      14

binary string:

00000001 00000001 00000001 00000010 00001100 00000100 00000000

decoded:

  0        01           001   0        01           002   0        12           004        0

descriptors in F X Y format:

     0 01 001              |       0 01 002            |      0 12 004            ¦

      needed to complete section with an even number of octets ----+-------

Recall from Layer 2 that descriptors are composed of thee parts - F (2 bits), X (6 bits), and Y (8 bits).  Figure 3.1.1-5 describes the decoding of the three descriptors contained in octets 8 – 13 in more detail.  Descriptors themselves are discussed at length in Section 3.1.2.

octet number            8                         9                      10                  11

binary string    0 0 0 0 0 0 0 1  0 0 0 0 0 0 0 1  0 0 0 0 0 0 0 1  0 0 0 0 0 0 1 0

                       |     ¦                ¦                         ¦      ¦                 ¦                        ¦

decoded         +- 0+---01-----+ +------001------++-0+-----01----+ +-----002-------+

meaning            F        X                    Y               F         X                   Y

                 +---------------descriptor 1------------++--------------descriptor 2---------+

octet number               12                    13

binary string    0 0 0 0 1 1 0 0  0 0 0 0 0 1 0 0

                       |     ¦                 ¦                        ¦

decoded         +-0+----12------++------004------+

meaning           F          X                   Y

                       +------------descriptor 3----------+

Figure 3.1.1-5. Decoding of Octets 8 – 13 of Section 3

3.1.1.6 Section 4 - Data Section

Structure

C O N T I N U O U S B I N A R Y S T R E A M
Section 0		Section 1	Section 2	Section 3	SECTION 4	Section 5
Octet No.	Contents
1 – 3	Length of section, in octets
4	Set to zero (reserved)
5 -	Binary data, as defined by the descriptors that begin at octet 8 of Section 3.

Sample message decomposition (Data Section)

The Data Section of the sample BUFR Message shown in Figure 3.1.1-1 is decomposed in detail below. The length of the Data Section, given in octets 1 – 3, is 8, and octet 4 is reserved. The remaining octets 5-8 contain the data itself:

octet number:

        1      |       2       |        3      |       4        |       5       |       6      |       7        |        8

binary string:

01000000 00000000 00001000 00000000 10010000 11110101 11011100 01000000

                                                  ¦                ¦

decoded                               8   |                |

---------------length of section ---+ reserved+------Data as described by descriptors-------¦

                                                                                in Section 3 (Figure 1-6)

 ----+

Figure 3.1.1-6. Section 4

Now, consider octets 5 – 8 in more detail.  First, recall from octets 8 – 13 of the Data Description Section that the data is described by the three descriptors 0 01 001, 0 01 002, and 0 12 004.  How to determine the characteristics of the data indicated by these three descriptors will be discussed later in Layer 3.  For now, suffice it to note the following:

DESCRIPTOR			NAME	UNIT	SCALE	REFERENCE VALUE	DATA WIDTH (Bits)
F	X	Y	NAME	UNIT	SCALE	REFERENCE VALUE	DATA WIDTH (Bits)
0	01	000	WMO block number	Numeric	0	0	7
0	01	002	WMO station number	Numeric	0	0	10
0	12	004	Dry-bulb temperature at 3 m	K	1	0	12

Thus, the WMO block number occupies the first 7 bits of octets 5 – 8 in the Data Section, or 1001000.  This binary string of 7 bits has a value of 72, so the WMO block number is 72.  The WMO station number occupies the next 10 bits of octets 5 – 8 in the Data Section, or 0111101011.  This binary string of 10 bits has a value of 491, so the WMO station number is 491.  The temperature, in degrees Kelvin, occupies the next 12 bits of octets 5 – 8 in the Data Section, or 101110001000.  This binary string of 12 bits has a value of 2952.  Since the scale for this temperature descriptor is 1, we must divide by 10 to retrieve the original value of 295.2 degrees Kelvin.  This accounts for 29 of the 32 bits in octets 5 – 8 of the Data Section.  Since all sections of a BUFR message must have an even number of octets and end on an octet boundary, the last three bits of octet 8 are set to zero.  This decomposition is depicted pictorially in Figure 3.2.1-7 below.

octet number:                5                     6                         7                     8

Binary string:  1 0 0 1 0 0 0 0  1 1 1 1 0 1 0 1  1 1 0 1 1 1 0 0  0 1 0 0 0 0 0 0

                       ¦                  ¦ ¦                              ¦ ¦                                 ¦ ¦       ¦

decoded:        +---- 72 ----+ +--------- 491 --------++---------- 2952 --------++-----+

                                                                         3 bits of zero to end octet+-- --+

Figure 3.1.1-7. Decoding of Octets 5 – 8 of Section 4

3.1.1.7 Section 5 - End Section

Structure

C O N T I N U O U S B I N A R Y S T R E A M
Section 0		Section 1	Section 2	Section 3	Section 4	SECTION 5
Octet No.	Contents
1 – 4	"7777" (coded according to the CCITT International Alphabet No. 5)

Sample message decomposition (End Section)

The End Section of the sample BUFR Message shown in Figure 3.1.1-1 (and the End Section of all BUFR messages) is decomposed in detail below. The hexadecimal equivalent of the four octets is shown to clarify the representation of the four characters 7”, “7”, “7”, “7”:

octet number:         1              2              3                4

binary string:  00110111 00110111 00110111 00110111

hexadecimal:      3   7          3   7         3   7          3   7

decoded:               7                7             7               7

Figure 3.1.1-8. Section 5

3.1.1.8 Required Entries

In any BUFR message there are required entries, and there will be a minimum number of bits to represent even the smallest amount of data. Therefore, there will be a minimum length for any BUFR message. The required and the minimum number of octets in each section are:

Section 0, octets 1 – 8 required

Section 0 will always contain 8 octets.

Section 1, octets 1 – 18 required

Section 1 will contain a minimum of 18 octets.

Section 3, octets 1 – 7 required

Section 3 will contain a minimum of 10 octets. The data descriptors begin in octet 8. A single data descriptor occupies 16 bits, or 2 octets. Since the Section must contain at least one descriptor and have an even number of octets, there will be a minimum of 10 octets in Section 3. Note that Section 3 will always conclude with 8 bits set to zero since all descriptors are 16 bits in length and the first descriptor begins in octet 8.

Section 4, octets 1 – 4 required

Section 4 will contain a minimum of 6 octets. The data is in bits 5 and beyond. However, since the Section must contain an even number of octets there must be at least 2 octets after octet 4, so there will be a minimum of 6 octets in Section 4.

Section 5 - octets 1 – 4 required

Section 5 will always contain 4 octets.

There will thus be a minimum of 46 octets, or 368 bits, in any BUFR message. For each section, the minimum number of bits is:

C O N T I N U O U S B I N A R Y S T R E A M

Section 0

(8 octets)

64 bits

Section 1

(18 octets)

144 bits

Section 2

(optional)

Section 3

(10 0ctets)

80 bits

Section 4

(6 octets)

48 bits

Section 5

(4 octets)

32 bits

Figure 3.1.1-9 is the same BUFR message used in Figures 3.1.1-1 to 3.1.1-8. However, in Figure 3.1.1-9, those octets that are required in any BUFR message, as described above, are shown in bold. Not included in the bold areas are descriptors contained in octets 8 - 14 of Section 3 and the data in Octets 5 - 8 of section 4.

Class 01 - Identification

TABLE REFERENCE					TABLE ELEMENT NAME	BUFR				CREX
						UNIT	SCALE	REFERENCE VALUE	DATA WIDTH (Bits)	UNIT	SCALE	DATA WIDTH (Characters)
F		X	Y
0	01			001	WMO block number	Numeric	0	0	7	Numeric	0	2
0	01			002	WMO station number	Numeric	0	0	10	Numeric	0	3
0	01			003	WMO Region number/geographical area	Code table	0	0	3	Code table	0	1
0	01			004	WMO Region sub-area (see Note 9)	Numeric	0	0	3	Numeric	0	1
0	01			005	Buoy/platform identifier	Numeric	0	0	17	Numeric	0	5
0	01			006	Aircraft flight number	CCITT IA5	0	0	64	Character	0	8
0	01			007	Satellite identifier	Code table	0	0	10	Code table	0	4
0	01			008	Aircraft registration number	CCITT IA5	0	0	64	Character	0	8
0	01			009	Type of commercial aircraft	CCITT IA5	0	0	64	Character	0	8
0	01			010	Stationary buoy platform identifier; e.g. C-MAN buoys	CCITT IA5	0	0	64	Character	0	8
0	01			011	Ship or mobile land station identifier	CCITT IA5	0	0	72	Character	0	9
0	01			012	Direction of motion of moving observing platform	Degree true	0	0	9	Degree true	0	3
0	01			013	Speed of motion of moving observing platform	m s–1	0	0	10	m s^-1	0	3
0	01			014	Platform drift speed (high precision)	m s–1	2	0	10	m s^-1	2	4
0	01			015	Station or site name	CCITT IA5	0	0	160	Character	0	20
0	01			018	Short station or site name	CCITT IA5	0	0	40	Character	0	5
0	01			019	Long Station or site name	CCITT IA5	0	0	256	Character	0	32
0	01			020	WMO Region sub-area	Numeric	0	0	4	Numeric	0	2
0	01			021	Synoptic feature identifier	Numeric	0	0	14	Numeric	0	4
0	01			022	Name of feature (see Note 11)	CCITT IA5	0	0	224	Character	0	28
0	01			025	Storm identifier	CCITT IA5	0	0	24	Character	0	3
0	01			026	WMO storm name*	CCITT IA5	0	0	64	Character	0	8
0	01			027	WMO long storm name	CCITT IA5	0	0	80	Character	0	10
0	01			031	Identification of originating/generating centre (see Note 10)	Code table	0	0	16	Code table	0	5
0	01			032	Generating application	Code table defined by originating/ generating centre (Notes (3), (4) and (5))	0	0	8	Code table	0	3
0	01			033	Identification of originating/generating centre	Code table	0	0	8	Code table	0	3
0	01			034	Identification of originating/generating sub-centre	Code table	0	0	8	Code table	0	3
0	01			036	Agency in charge of operating the Observing platform	Code table	0	0	20	Code table	0	7
0	01			041	Absolute platform velocity – first component (see Note 6)	m s-1	5	–1073741824	31	m s^-1	5	10
0	01			042	Absolute platform velocity – second component (see Note 6)	m s-1	5	–1073741824	31	m s^-1	5	10
0	01			043	Absolute platform velocity – third component (see Note 6)	m s-1	5	–1073741824	31	m s^-1	5	10
0	01			050	Platform transmitter ID number	Numeric	0	0	17	Numeric	0	6
0	01			051	Platform transmitter ID number	CCITT IA5	0	0	96	Character	0	12
0	01			060	Aircraft reporting point (Beacon identifier)	CCITT IA5	0	0	64	Character	0	8
0	01			062	Short ICAO location indicator	CCITT IA5	0	0	32	Character	0	4
0	01			063	ICAO location indicator	CCITT IA5	0	0	64	Character	0	8
0	01			064	Runway designator	CCITT IA5	0	0	32	Character	0	4
0	01			075	Tide station identification	CCITT IA5	0	0	40	Character	0	5
0	01			080	Ship line number according to SOOP	CCITT IA5	0	0	32	Character	0	4
0	01			085	Observing platform manufacturer's model	CCITT IA5	0	0	160	Character	0	20
0	01			086	Observing platform manufacturer's serial number	CCITT IA5	0	0	256	Character	0	32

Notes:

(1) The storm identifier (descriptor 0 01 025) has the following meaning: the first two characters shall be a numeric sequence number assigned by the originator of the message; the third character is a letter indicating the ocean basin where the storm is located, as follows:

W NW Pacific Ocean

E NE Pacific Ocean to 140°W

C NE Pacific Ocean 140°W – 180°W

L N Atlantic Ocean, including Caribbean and Gulf of Mexico

A N Arabian Sea

B Bay of Bengal

S S Indian Ocean

P S Pacific Ocean

F RSMC Nadi's zone in South Pacific

U Australia

O South China Sea

T East China Sea

There is no requirement that differing observers coordinate sequence numbers even though they both may be reporting the same storm.

(2) WMO storm name (descriptor 0 01 027): the storm name NAMELESS shall be used in those cases where an identifiable tropical disturbance has not reached tropical storm strength and has not been assigned an official name.

(3) Where a centre other than the originating centre generates quality information, replacement or substitute values, and/or statistical information, the centre may be indicated by using 0 01 033.

(4) A generating centre may wish to indicate a reference to the application that generated quality information, etc.; it may use descriptor 0 01 032 for this purpose. However, the corresponding code tables will vary from centre to centre.

(5) Code table 0 01 032 is to be generated by each centre.

(6) The components of absolute platform velocity (0 01 041, 0 01 042, 0 01 043) are defined as follows:

– First component: From the Earth’s centre to 0 degree longitude at the Equator: velocity of the platform along this line relative to the Earth’s centre.

– Second component: From the Earth’s centre to 90 degrees East longitude at the Equator: velocity of the platform along this line relative to the Earth’s centre.

– Third component: From the Earth’s centre to the North Pole: velocity of the platform along this line relative to the Earth’s centre.

(7) The values for descriptors 0 01 041, 0 01 042 and 0 01 043 have been chosen to be suitable for polar orbiting satellites in approximately Sun-synchronous orbits. Geostationary orbits would require greater data widths for distance and slightly less for speed.

(8) Left handed xyz axes have been chosen for descriptors 0 01 041, 0 01 042 and 0 01 043.

(9) Descriptor 0 01 020 should be used instead of 0 01 004 for encoding this element.

(10) Descriptor 0 01 033 shall be used instead of descriptor 0 01 031 for encoding originating/generating centre. Code table 0 01 034 is to be established by the associated originating/generating centre identified by descriptor 0 01 033 and provided to the Secretariat for publication.

(11) For 0 01 022, the character string representing the “Name of feature” should be of the form: “Type of phenomenon” – “Location or geographical name” (e.g.: “volcano – Popocatepetl”, “oil fire – Kuwait”)

Class 12 - Temperature

TABLE REFERENCE			TABLE ELEMENT NAME	BUFR				CREX
				UNIT	SCALE	REFERENCE VALUE	DATA WIDTH (Bits)	UNIT	SCALE	DATA WIDTH (Characters)
F	X	Y
0	12	001	Temperature/dry-bulb temperature	K	1	0	12	°C	1	3
0	12	002	Wet-bulb temperature	K	1	0	12	°C	1	3
0	12	003	Dew-point temperature	K	1	0	12	°C	1	3
0	12	004	Dry-bulb temperature at 2 m	K	1	0	12	°C	1	3
0	12	005	Wet-bulb temperature at 2 m	K	1	0	12	°C	1	3
0	12	006	Dew-point temperature at 2 m	K	1	0	12	°C	1	3
0	12	007	Virtual temperature	K	1	0	12	°C	1	3
0	12	011	Maximum temperature, at height and over period specified	K	1	0	12	°C	1	3
0	12	012	Minimum temperature, at height and over period specified	K	1	0	12	°C	1	3
0	12	013	Ground minimum temperature, past 12 hours	K	1	0	12	°C	1	3
0	12	014	Maximum temperature at 2 m, past 12 hours	K	1	0	12	°C	1	3
0	12	015	Minimum temperature at 2 m, past 12 hours	K	1	0	12	°C	1	3
0	12	016	Maximum temperature at 2 m, past 24 hours	K	1	0	12	°C	1	3
0	12	017	Minimum temperature at 2 m, past 24 hours	K	1	0	12	°C	1	3
0	12	021	Maximum temperature at 2m	K	2	0	16	°C	2	4
0	12	022	Minimum temperature at 2m	K	2	0	16	°C	2	4
0	12	030	Soil temperature	K	1	0	12	°C	1	3
0	12	051	Standard deviation temperature	K	1	0	10	°C	1	3
0	12	052	Highest daily mean temperature	K	1	0	12	°C	1	3
0	12	053	Lowest daily mean temperature	K	1	0	12	°C	1	3
0	12	061	Skin temperature	K	1	0	12	°C	1	3
0	12	062	Equivalent black body temperature	K	1	0	12	°C	1	3
0	12	063	Brightness temperature	K	1	0	12	°C	1	3
0	12	064	Instrument temperature	K	1	0	12	K	1	4
0	12	065	Standard deviation brightness temperature	K	1	0	12	K	1	4
0	12	071	Coldest cluster temperature	K	1	0	12	K	1	4
0	12	072	Radiance	W m^-2 sr^-1	6	0	31	W m^-2sr^-1	6	9
0	12	075	Spectral radiance	W m^-3 sr^-1	-3	0	16	W m^-3sr^-1	-3	5
0	12	076	Radiance	W m^-2 sr^-1	3	0	16	W m^-2sr^-1	3	5
0	12	101	Temperature/dry-bulb temperature	K	2	0	16	°C	2	4
0	12	102	Wet-bulb temperature	K	2	0	16	°C	2	4
0	12	103	Dew-point temperature	K	2	0	16	°C	2	4
0	12	104	Dry-bulb temperature at 2m	K	2	0	16	°C	2	4
0	12	105	Web-bulb temperature at 2m	K	2	0	16	°C	2	4
0	12	106	Dew-point temperature at 2m	K	2	0	16	°C	2	4

Consider now the sample BUFR message discussed in Section 3.1.1 of Layer 3 and illustrated in Figures 3.1.1-1 through 3.1.11-9.  That BUFR message had three descriptors in the Data Description Section that described three quantities in the Data Section.  The three descriptors were 0 01 001, 0 01 002, and 0 12 004.  The first two have an F value of 0 and a Y value of 01, so they refer to Table B parameters in class 01 – Identification.  In the table for Class 01 (above), note the rows for table references 0 01 001 (WMO block number) and 0 01 002 (WMO station number).  In those rows, under the part of the table labeled BUFR, are the values of Unit, Scale, Reference Value, and Data Width (bits) needed for encoding and decoding WMO Block Number and WMO station number in BUFR messages.  Likewise, descriptor 0 12 004 has a Y value of 12, and hence refers to a Table B parameter in Class 12.  In that table (also above), in the row for table reference 0 12 004 (Dry-bulb temperature at 3 m), are the values of Unit, Scale, Reference Value, and Data Width (bits) needed for encoding and decoding Dry-bulb temperature at 3 m in BUFR messages.  These are the values noted earlier in the discussion of the BUFR Data Section, and are reproduced here:

DESCRIPTOR			NAME	UNIT	SCALE	REFERENCE VALUE	DATA WIDTH (Bits)
F	X	Y	NAME	UNIT	SCALE	REFERENCE VALUE	DATA WIDTH (Bits)
0	01	000	WMO block number	Numeric	0	0	7
0	01	002	WMO station number	Numeric	0	0	10
0	12	004	Dry-bulb temperature at 3 m	K	1	0	12

BUFR messages usually have many more parameters than three, and therefore have many more descriptors than this.  More complicated BUFR messages will be discussed presently.  However, the procedure for finding the needed values of Unit, Scale, Reference Value, and Data Width (bits) is the same whether there are three descriptors or thirty three.

3.1.3.4 Table C - Data Description Operators

Descriptors with F = 2 refer to BUFR Table C - Table C data description operators. These are used when there is a need to redefine Table B attributes temporarily, such as the need to change data width, scale or reference value of a Table B entry. Table C is also used to add associated fields such as quality control information, indicate characters as data items, and signify data width of local descriptors. Some of these operators are somewhat complex. Table C data description operators are described at length later in Chapter 3.1.6.

3.1.3.5 Table D - Lists of Common Sequences

Table D contains descriptors that describe additional descriptors. A single descriptor used in Section 3 with F = 3 is a pointer to a Table D entry. Similarly to Table B, BUFR Table D is organized into various classes, or categories, of sequences that have common characteristics. The X value of the sequence descriptor identifies the category to which that particular sequence descriptor belongs. The Y value of the sequence descriptor is the entry in that category for that particular sequence descriptor. There are currently defined 19 categories of common sequences in Table D (Table 3-3).

Table 3-3. BUFR Table D list of common sequences

F   X    CATEGORY OF SEQUENCES

3  00     BUFR table entries sequences

3  01     Location and identification sequences

3  02     Meteorological sequences common to surface data

3  03     Meteorological sequences common to vertical sounding data

3  04     Meteorological sequences common to satellite observations

3  05     Reserved

3  06     Meteorological or oceanographic sequences common to oceanographic

             observations

3  07     Surface report sequences (land)

3  08     Surface report sequences (sea)

3  09     Vertical sounding sequences (conventional data)

3  10     Vertical sounding sequences (satellite data)

3  11     Single level report sequences (conventional data)

3  12     Single level report sequences (satellite data)

3  13     Sequences common to image data

3  14     Reserved

3  15     Oceanographic report sequences

3  16     Synoptic feature sequences

3  18     Radiological report sequences

3  21     Radar report sequences

As an example, if the Table D descriptor 3 01 001 were used in Section 3, the expansion of that descriptor is two Table B descriptors, 0 01 001 and 0 01 002.

                       + 0 01 001 ---WMO block number

3 01 001---->¦

                       + 0 01 002 ---WMO station number

Table D descriptors may also refer to an expansion list of descriptors that contain additional Table D descriptors.  The descriptor 3 01 025 expands to 3 01 023, 0 04 003 and 3 01 012.  Furthermore, 3 01 023 additionally expands to 0 05 002 and 0 06 002 and 3 01 012 expands to 0 04 004 and 0 04 005.  Thus, the single Table D descriptor 3 01 025 expands to a total of 5 separate Table B entries:

                                             + 0 05 002 ---Latitude

                       + 3 01 023--->¦

                       ¦                     + 0 06 002 ---Longitude

¦

¦

  3 01 025---->¦------------------- 0 04 003----Day

¦

¦

¦

                       ¦                     + 0 04 004 ---Hour

                       + 3 01 012--->¦

                                             + 0 04 005 ---Minute

The order of the data in Section 4 is then according to the following sequence of Table B entries: 0 05 002  0 06 002      0 04 003      0 04 004  0 04 005.

Figure 3.2.3-1 illustrates a more complex example. The single low altitude surface observation sequence descriptor 3 07 002 expands into 2 more Table D descriptors, 3 01 032 and 3 02 011. The descriptor 3 01 032 further expands into 5 more descriptors – four from Table D and one from Table B – 3 01 001, 0 02 001, 3 01 011, 3 01 012 and 3 01 024. The descriptor 3 02 011 further expands into 3 more Table D descriptors – 3 02 001, 3 02 003 and 3 02 004. Note that the expansion of 3 01 032 ultimately provides information on the location in time and space and the type of station, because Class 01 of Table D contains Location and Identification sequences. The expansion of 3 02 011, on the other hand, ultimately provides the meteorological parameters, since Class 02 of Table D contains Meteorological sequences common to surface observations.

SECTION 4

                                                                                                                     WIDTH IN BITS

3 07 002       +3 01 032  +3 01 001           +0 01 001        >WMO BLOCK NO.              7

                                                                  +0 01 002        >WMO STATION NO.        10

                                       +0 02 001                                   >TYPE OF STATION           2

                                       +3 01 011           +0 04 001        >YEAR                                12

                                                                  +0 04 002        >MONTH                              4

                                                                  +0 04 003        >DAY                                     6

                                       +3 01 012           +0 04 004        >HOUR                                 5

                                                                  +0 04 005        >MINUTE                              6

                                       +3 01 024           +0 05 002        >LATITUDE (COARSE

                                                                                            ACCURACY)                    15

                                                                  +0 06 002        >LONGITUDE(COARSE

                                                                                            ACCURACY)                    16

                                                                  +0 07 001        >HEIGHT OF STATION     15

                     +3 02 011  +3 02 001           +0 10 004        >PRESSURE                      14

                                                                  +0 10 051        >PRESSURE REDUCED

                                                                                            TO MSL                            14

                                                                  +0 10 061        >3 HR PRESSURE

                                                                                            CHANGE                          10

                                                                  +0 10 063        >CHARACTERISTIC OF

                                                                                            PRESSURE                        4

                                       +3 02 003           +0 11 011        >WIND DIRECTION             9

                                                                  +0 11 012        >WIND SPEED AT 10m      12

                                                                  +0 12 004        >DRY BULB TEMP AT

                                                                                            2m                                     12

                                                                  +0 12 006        >DEW POINT TEMP AT

                                                                                            2m                                     12

                                                                  +0 13 003        >RELATIVE HUMIDITY        7

                                                                  +0 20 001        >HORIZONTAL

                                                                                            VISIBILITY                        13

                                                                  +0 20 003        >PRESENT WEATHER        8

                                                                  +0 20 004        >PAST WEATHER (1)          4

                                                                  +0 20 005        >PAST WEATHER (2)          4

                                       +3 02 004           +0 20 010        >CLOUD COVER

                                                                                            (TOTAL)                             7

                                                                  +0 08 002        >VERTICAL

                                                                                            SIGNIFICANCE

                                                                                            SURFACE OBS                  6

                                                                  +0 20 011        >CLOUD AMOUNT              4

                                                                  +0 20 013        >HEIGHT OF BASE OF

                                                                                            CLOUD                             11

                                                                  +0 20 012        >CLOUD TYPE Cl                6

                                                                  +0 20 012        >CLOUD TYPE Cm              6

                                                                  +0 20 012        >CLOUD TYPE Ch               6

                                                                                                                                    -----

                                                                                          TOTAL BITS:                   267

Figure 3.1.3-1.  Example of a surface observation sequence using Table D sequence descriptor 3 07 002

As is shown in Figure 3.1.3-1, descriptors in Table D may themselves refer to Table D, provided no circularity results on repeated expansion. Completion of the expansion process leads to a total of 31 Table B descriptors. The 16 bits in Section 3 taken by the single sequence descriptor 3 07 002 results in a savings of 480 bits (30 x 16 bits) over what the 31 Table B descriptors would occupy in bits.

A complete layout of a BUFR message containing just 1 surface observation is illustrated in Figure 3.1.3-2. As indicated in octets 5-7 of Section 1, there are a total of 78 octets in the message, or 624 bits. Of the 624 bits, 267 are for the actual parameters of data (Figure 3.1.3-1) and the remaining 357 bits are BUFR overhead. BUFR overhead in this context is the number of bits that are not actual surface data. In this example there are more bits used for the overhead than for the surface data.

	Section Octet No.	Octet in Message	Encoded Value	Description
Section 0 (indicator section)	1-4	1-4	BUFR	Encoded international CCITT
				Alphabet No. 5
	5-7	5-7	78	Total length of message (octets)
	8	8	3	BUFR edition number
Section 1 (identification section)	1-3	9-11	18	Length of section (octets)
	4	12	0	BUFR master table
	5-6	13-14	58	Originating center (U.S. Navy - FNMOC)
	7	15	0	Update sequence number
	8	16	0	Indicator that Section 2 not included
	9	17	0	Table A - surface land data
	10	18	0	BUFR message sub-type
	11	19	9	Version number of master tables
	12	20	0	Version number of local tables
	13	21	92	Year of century
	14	22	4	Month
	15	23	18	Day
	16	24	0	Hour
	17	25	0	Minute
	18	26	0	Reserved for local use by ADP centers (also needed to complete even number of octets for section
Section 3 (Data description section)	1-3	27-29	10	Length of section (octets)
	30	0	Reserved
	5-6	31-32	1	Number of data subsets
	7	33	bit 1=1	Flag indicating observed data
	8-9	34-35	3 07 002	Table D descriptor for surface land in F X Y format
	10	36	0	Need to complete section with an even number of octets
Section 4 (Data section)	1-3	37-39	38	Length of section (octets)
	4	40	0	Reserved
	5-38	41-74	data	Continuous bit stream of data for 1 observations, 267 bits plus 5 bits to end on even octet (see Figure 3.1.3-1 for expansion)
Section 5 (End section)	1-4	75-78	7777	Encoded CCITT International Alphabet No. 5

Figure 3.1.3-2. BUFR message of 1 surface observation using Table D descriptor 3 07 002

Figure 3.1.3-3 is a complete layout of a BUFR message containing the maximum number of 448 subsets to fit within the 15000-octet limit. This message would contain 14996 octets or 119968 bits. Of these 119968 bits, 119616 are data and 352 bits are BUFR overhead. The 5 bit difference in overhead from Figure 3.1.3-2 (357 bits) and Figure 3.1.3-3 (352 bits) is due to the number of bits set to 0 at the end of Section 4 in order to complete the section at the end of an even numbered octet. For 1 subset of 267 bits, 5 additional bits are needed to complete the octet. For 448 subsets, or 119616 bits, no additional bits are needed to complete the last octet.

	Section Octet No.	Octet in Message	Encoded Value	Description
Section 0 (Indicator section)	1-4	1-4	BUFR	Encoded international CCITT section)Alphabet No. 5
	5-7	5-7	14996	Total length of message (octets)
	8	8	2	BUFR edition number

Section 1 (identification section)	1-3	9-11	18	Length of section (octets)
	4	12	0	BUFR master table
	5-6	13-14	58	Originating center (U.S. Navy - FNMOC)
	7	15	0	Update sequence number
	8	16	0	Indicator that Section 2 not included
	9	17	0	Table A - surface land data
	10	18	0	BUFR message sub-type
	11	19	2	Version number of master table
	12	20	0	Version number of local tables
	13	21	92	Year of century
	14	22	4	Month
	15	23	18	Day
	16	24	0	Hour
	17	25	0	Minute
	18	26	0	Reserved for local use by ADP centers (also needed to complete even number of octets for section
Section 3 (Data Description Section)	1-3	27-29	10	Length of section (octets)
	4	30	0	Reserved
	5-6	31-32	448	Number of data subsets
	7	33	bit 1=1	flag indicating observed data
	8-9	34-35	3 07 002	Table D descriptor for surface land in F X Y format
	10	36	0	need to complete section with an even number of octets
Section 4 (Data section)	1-3	37-39	14956	length of section (octets)
	4	40	0	Reserved
	5-14956	41-14992	data	continuous bit stream of data for 448 observations, 267 bits per observation with no added bits to end on an even octet
Section 5 (End section)	1-4	14993- 14996	7777	encoded CCITT International Alphabet No. 5

Figure 3.1.3-3. BUFR message of 448 surface observations using Table D descriptor
3 07 002

Table D should be limited to lists of descriptors likely to be most frequently used. Table D was not designed to be a comprehensive list of all sequences likely to be encountered. To do so would require an excessively large Table D and would reduce considerably flexibility when encoding minor differences in reporting practices. More flexibility is retained if the Data Description Section contains several descriptors.

Most BUFR message may be encoded without using Table D. The data description contained within Section 3 can be accomplished entirely by using only element descriptors of Table B and operator descriptors of Table C (the only exception is when a replication of more than 63 values is needed, for the XX part – 6 bits - of the replication can describe only to 63 quantities. In this case, special procedures involving Table D may be needed). To do so, however would involve considerable overhead in terms of the length of the Section 3 data description. The use of sequence descriptors from Table D is particularly effective in reducing the size of BUFR messages containing single observations almost to that of the original traditional alphanumeric code form size.

The use of Table D is another major contributor to the efficiency of BUFR.

3.1.3.6 Comparison of BUFR and Character Code Bit Counts

The surface observations illustrated in Figures 3.2.3-1 to 3.2.3-3 are the equivalent of the following parameters in the WMO code form FM 12 SYNOP:

YYGGi_w IIiii i_Ri_xhVV Nddff 1s_nTTT 2s_nT_dT_dT_d 3P_oP_oP_oP_o4PPPP 5appp 7wwW₁W₂ 8N_hC_LC_MC_H

Data encoded in this form would consist of 55 characters plus 10 spaces between each group of 5 characters for a total of 65 characters. For transmission purposes these 65 characters would require a total number of 520 bits (65 X 8 bits per character).

A complete BUFR message with 1 observation (Figure 3.2.3-2) requires 78 octets or 624 bits, 104 more than the corresponding character representation. However, 69 of the extra 104 bits are a result of including the latitude, longitude, station height and year, month, and minute in BUFR. For the same information, a BUFR message with one surface observation would be only 35 bits longer (about 7%) than the traditional character version. On the other hand, the systematic passing of geographical co-ordinates, easily performed with the table driven codes, would alleviate the well-known WMO Volume A problems. The 46 extra bits it takes to include this information is truly a price worth paying.

Of the 624 bits in the BUFR message, 267 are taken by the surface observation and 357 are BUFR overhead. If, however, a collective of 448 observations in character form were transmitted, the total number of bits would be 232960 (520 X 448). The corresponding BUFR representation (Figure 3.1.3-3) would require only 14996 octets, or 119968 bits, about half the length of the character representation. Moreover, this figure does not include the effect of using the BUFR compression capability. Use of compression, discussed in Chapter 3.1.5, would make the BUFR message even more compact.

3.1.3.7 Code Tables and Flag Tables

Since some meteorological parameters are qualitative or semi-qualitative, they are best represented with reference to a code table or a flag table. BUFR code tables and flag tables refer to elements defined within BUFR Table B. They are numbered according to the X and Y values of the corresponding Table B reference. For example, the Table B entry 0 01 003, WMO Region number, geographical area, indicates in the Unit column that this is a BUFR code table, and the number of that code table is 0 01 003.

Code Tables.

A code table lists a number of possibilities the parameter to which the code table applies can use. Only one of the possibilities can be chosen, and one of the possibilities is always “missing value”. Many of the code tables that have been included in the BUFR specification are similar to existing WMO code tables for representing character data. Attachment II of the WMO Manual on Codes, Volume 1, Part B is a list of the code tables associated with BUFR Table B and the existing specifications and code tables of the WMO Manual on Codes, Volume 1, Part A. However, there is not a one-to-one BUFR code table relationship to the character code tables. The character Code Table 3333, Quadrant of the Globe, for example, has no meaning in BUFR, as all points on the globe in BUFR are completely expressed as latitude and longitude values.

Flag tables.

A flag table also lists a number of possibilities the parameter to which the flag table applies can use. However, in a flag table, any combination of the possibilities can be chosen. The flag table accomplishes this with a string of bits, where each bit indicates an item of significance. A bit set to 1 indicates an item is included, or is true, while a bit set to 0 indicates omission, or false. In any flag table, when all bits are set it is an indication of a missing value. However, in order to allow the option of all the possibilities being chosen and still allow for the possibility of a missing value, flag tables always have one more bit than the number of items of significance. In all flag tables within the BUFR specification, bits are numbered from 1 to N from most significant to least significant within a data width of N bits, i.e., from left (bit 1) to right (bit N).

3.1.3.8 Local Tables

Since a data processing center may need to represent data conforming to a local requirement where this data is not defined within Table B, specific areas of Table B and D are reserved for local use (Figure 3.1.3-4). These areas are defined as entries 192 to 255 inclusive of all classes. Centers defining classes or categories for local use should restrict their use to the range 48 to 63 inclusive.

The Local portions of the Tables can be updated, changed, augmented, etc. at will by the local group concerned. No international notice is required or expected. It is presumed that bulletins containing local descriptors will not be sent out internationally (but see the discussion of operator descriptor 2 06 YYY for an exception). "Local", although not defined in the BUFR documentation, is generally taken to mean "within the processing center that is generating the BUFR messages", and not necessarily one country. The U. S. has a number of processing centers (the civilian weather service, Air Force, Navy, and other groups as well, each potentially identified by a unique processing center number and sub-number) each one of which is free to use the "local" portions of the BUFR tables as they see fit.

CLASS

FOR INTERNATIONAL USE

FOR LOCAL USE

RESERVED FOR FUTURE USE

FOR LOCAL USE

(IF NEEDED)

FOR LOCAL USE

(IF NEEDED)

FOR LOCAL USE

         0                                                                                     192                              255

ENTRY WITHIN CLASS

Figure 3.1.3-4. Table reservations

If a data processing center had multiple sources of data receipt, for example, it may be necessary to indicate the source of an observation by the circuit from which the data was received. A local Table B descriptor such as 0 54 192 could be used which may be a code table specifying circuits of transmission. The Table B entry could be:

Table Reference	Element Name	Units	Scale	Reference Value	Data Width (Bits)
0 54 192	Circuit	Code table	0	0	3

The corresponding local code table could be:

Code Table 0 54 192: Circuit designators for data receipt

Code

Figure Circuit

0 GTS

1 AWN

2 AUTODIN

3 ANTARCTIC

4-6 Reserved

7 Missing value

Using the same Table D descriptor, 3 07 002, as in Figure 3.1.3-1, adding the local

descriptor 0 54 192 would produce the expansion as in Figure 3.1.3-5.

SECTION 4

                                                                                                                     WIDTH IN BITS

0 54 192                                                                            >LOCAL DESCRIPTOR       3

3 07 002       +3 01 032  +3 01 001           +0 01 001        >WMO BLOCK NO.              7

                                                                  +0 01 002        >WMO STATION NO.        10

                                       +0 02 001                                   >TYPE OF STATION           2

                                       +3 01 011           +0 04 001        >YEAR                                12

                                                                  +0 04 002        >MONTH                              4

                                                                  +0 04 003        >DAY                                     6

                                       +3 01 012           +0 04 004        >HOUR                                 5

                                                                  +0 04 005        >MINUTE                              6

                                       +3 01 024           +0 05 002        >LATITUDE (COARSE

                                                                                            ACCURACY)                    15

                                                                  +0 06 002        >LONGITUDE(COARSE

                                                                                            ACCURACY)                    16

                                                                  +0 07 001        >HEIGHT OF STATION     15

                     +3 02 011  +3 02 001           +0 10 004        >PRESSURE                      14

                                                                  +0 10 051        >PRESSURE REDUCED

                                                                                            TO MSL                            14

                                                                  +0 10 061        >3 HR PRESSURE

                                                                                            CHANGE                          10

                                                                  +0 10 063        >CHARACTERISTIC OF

                                                                                            PRESSURE                        4

                                       +3 02 003           +0 11 011        >WIND DIRECTION             9

                                                                  +0 11 012        >WIND SPEED AT 10m      12

                                                                  +0 12 004        >DRY BULB TEMP AT

                                                                                            2m                                     12

                                                                  +0 12 006        >DEW POINT TEMP AT

                                                                                            2m                                     12

                                                                  +0 13 003        >RELATIVE HUMIDITY        7

                                                                  +0 20 001        >HORIZONTAL

                                                                                            VISIBILITY                        13

                                                                  +0 20 003        >PRESENT WEATHER        8

                                                                  +0 20 004        >PAST WEATHER (1)          4

                                                                  +0 20 005        >PAST WEATHER (2)          4

                                       +3 02 004           +0 20 010        >CLOUD COVER

                                                                                            (TOTAL)                             7

                                                                  +0 08 002        >VERTICAL

                                                                                            SIGNIFICANCE

                                                                                            SURFACE OBS                  6

                                                                  +0 20 011        >CLOUD AMOUNT              4

                                                                  +0 20 013        >HEIGHT OF BASE OF

                                                                                            CLOUD                             11

                                                                  +0 20 012        >CLOUD TYPE Cl                6

                                                                  +0 20 012        >CLOUD TYPE Cm              6

                                                                  +0 20 012        >CLOUD TYPE Ch               6

                                                                                                                                    -----

                                                                                          TOTAL BITS:                   270

Figure 3.1.3-5.  Example of a surface observation sequence using Table D sequence descriptor 3 07 002 and local descriptor

The following modifications would have to be made to the BUFR message if the local descriptor 0 54 192 were to be included in a message (Figure 3.1.3-6):

In Section 0, octets 5-7, the total length of the message, increases from 14996 octets to 14998 octets.

In Section 1, octet no. 12 (octet 20 within the message) would have the version number of the local tables in use.

In Section 3, octets 1-3, the encoded value would increase from 10 octets to 12 octets. If one descriptor were being added, the length of the section increases by 2 in order to keep the section an even number of octets. In octets 5-6, the number of data subsets decreases from 448 to 443. The number of data subsets has been reduced to keep the total message length under the 15000 octet maximum.

Also in Section 3, the descriptors will occupy octets 8-11 vice octets 8-9 to accommodate the added descriptor.

Note that in Section 4, octets 1-3, the encoded value for length of section remains the same at 14956 octets. The number of bits needed for 448 subsets without a local descriptor is 119616 (448 X 267), or exactly 14952 octets. For 443 subsets with 3 bits added to each subset for the local information, 119610 bits are needed (443 X 270). Adding 6 bits to complete the octet brings the total bit count for all 443 subsets to 119616, the same number of bits as 448 subsets without the added local information.

	Section Octet No.	Octet in Message	Encoded Value	Description
Section 0 (indicator section)	1-4	1-4	BUFR	Encoded international CCITT
				Alphabet No. 5
	5-7	5-7	14998	total length of message (octets)
	8	8	2	BUFR edition number
Section 1 (identification section)	1-3	9-11	18	length of section (octets)
	4	12	0	BUFR master table
	5-6	13-14	58	originating center (U.S. Navy - FNMOC)
	7	15	0	Update sequence number
	8	16	0	indicator that Section 2 not included
	9	17	0	Table A - surface land data
	10	18	0	BUFR message sub-type
	11	19	2	Version number of master tables
	12	20	1	Version number of local tables
	13	21	92	year of century
	14	22	4	Month
	15	23	18	Day
	16	24	0	Hour
	17	25	0	Minute
	18	26	0	reserved for local use by ADP centers (also need to complete even number of octets for Section 3
Section 3 (Data description section)	1-3	27-29	12	length of section (octets)
	4	30	0	reserved
	5-6	31-32	443	number of data subsets
	7	33	BIT 1=1	flag indicating observed data
	8-11	34-37	0 54 192	local and Table D descriptors
			3 07 002	In F X Y format
	10	38	0	need to complete section with an even number of octets
Section 4 (Data section)	1-3	39-41	14956	length of section (octets)
	4	42	0	Reserved
	5-14956	43-14994	data	continuous bit stream of data for 443 observations, 270 bits per observation plus 6 bits to end on even octet
Section 5 (End section)	1-4	14995- 14998	7777	encoded CCITT international Alphabet No. 5

Figure 3.1.3-6. BUFR message of 443 surface observations using 2 descriptors, local descriptor 0 54 192 and Table B descriptor 3 07 002.

3.1.4 Data Replication

3.1.4.1        Introduction.

The replication operator (F = 1) is used to define a sequence of descriptors to be repeated, or replicated, together with the number to times the sequence is to be repeated (the replication factor).  Replication provides a flexible and efficient means to describe repetition of a sequence of parameters in the Data Section.

3.1.4.2 Simple Replication

In simple replication, it is desired to encode a series of parameters a fixed number of times in all reports in Section 4. To illustrate, we will assume a sequence of four descriptors – Vertical significance (surface observations), Cloud amount, Cloud Type, and Height of base of cloud – is to be repeated four times in all reports in the Data Section. In this case, the replication sequence would be the following:

1 04 004 0 08 002 0 20 011 0 20 012 0 20 013

In this example, the descriptors have the following meanings:

1 04 004	F = 1	Replication operator
	X = 04	The following 4 descriptors are replicated
	Y = 004	The 4 descriptors will be replicated 4 times in every report in the Data Section
0 08 002	F = 0	First Table B descriptor in the sequence to be replicated
	X = 08	Class 08 – Significance qualifiers
	Y = 002	Entry 002 – Vertical significance (surface observations)
0 20 011	F = 0	Second Table B descriptor in the sequence to be replicated
	X = 20	Class 20 – Observed phenomena
	Y = 011	Entry 011 – Cloud amount
0 20 012	F = 0	Third Table B descriptor in the sequence to be replicated.
	X = 20	Class 20 – Observed phenomena
	Y = 012	Entry 012 – Cloud type
0 20 013	F = 0	Fourth Table B descriptor in the sequence to be replicated
	X = 20	Class 20 – Observed phenomena
	Y = 013	Entry 013 – Height of base of cloud

Thus the above sequence of five descriptors is equivalent to the following sequence of 16 descriptors:

0 08 002 0 20 011 0 20 012 0 20 013 0 08 002 0 20 011 0 20 012 0 20 013

In this case, every report in the Data Section will contain four sets of the four parameters “Vertical significance (surface observations), Cloud amount, Cloud type, and Height of base of cloud”. However, by using replication, there only need to be five descriptors in the Data Description Section instead of 16, and this will occupy only 10 octets (2 x 5) rather than 32 (2 x 16). The ability to describe a repeated pattern in the data by a single set of descriptors contributes to the efficiency of BUFR.

3.1.4.3 Delayed Replication

A special form of the replication operator allows the replication factor to be stored with the data in Section 4, rather than with the descriptors in Section 3. This special form is called delayed replication. It is indicated by Y = 0, and allows the data to be described in a general way, with the number of replications being different from subset to subset. Since the data in Section 4 now contains an additional data element (the actual replication count), a descriptor must be added to Section 3 to account for and describe this data element. The appropriate descriptor is found in Class 31 – Data description operator qualifiers. There are three descriptors available to use with delayed replication:

                                                                                                                     refer.      data

Descriptor                                name                                units       scale   value      width

0 31 000       Short delayed descriptor replication factor  Numeric 0          0             1

0 31 001       Delayed descriptor replication factor           Numeric 0          0             8

0 31 002       Extended delayed descriptor replication

                     factor                                                            Numeric 1          0             16

Special note: When Y is 0, the Class 31 delayed replication descriptor is placed immediately after the replication operator and before the sequence of descriptors to be replicated. HOWEVER, the delayed replication descriptor is NOT included in the count (X) of the number of descriptors to be replicated.

To illustrate this, suppose the example just discussed were to contain a delayed replication descriptor. The sequence would then be as follows:

1 04 000 0 31 001 0 08 002 0 20 011 0 20 012 0 20 013

Now, the descriptors have the following meanings:

1 04 000	F = 1	Replication operator
	X = 04	The four descriptors following the Delayed description replication factor (0 31 001) are replicated
	Y = 000	Delayed replication
0 31 001	F = 0	Table B descriptor
	X = 31	Class 31 - Data description operator qualifiers
	Y = 001	Entry 001 – Delayed description replication factor, occupying 8 bits in Section 4
0 08 002	F = 0	First Table B descriptor in the sequence to be replicated
	X = 08	Class 08 – Significance qualifiers
	Y = 002	Entry 002 – Vertical significance (surface observations)
0 20 011	F = 0	Second Table B descriptor in the sequence to be replicated
	X = 20	Class 20 – Observed phenomena
	Y = 011	Entry 011 – Cloud amount
0 20 012	F = 0	Second Table B descriptor in the sequence to be replicated.
	X = 20	Class 20 – Observed phenomena
	Y = 012	Entry 012 – Cloud type
0 20 013	F = 0	Third Table B descriptor in the sequence to be replicated
	X = 20	Class 20 – Observed phenomena
	Y = 013	Entry 013 – Height of base of cloud

First, note the Y value of the replication operator is 000, indicating this is a case of delayed replication. Second, note the X value of the replication operator is still 4. The portion of the Data Section corresponding to this sequence of six descriptors will first contain a value, occupying 8 bits, corresponding to 0 31 00. It indicates the number of times the next four descriptors will be replicated. After this value, the Data Section will contain the sequence of the four parameters “Vertical significance (surface observations), Cloud amount, Cloud type, and Height of base of cloud” repeated this number of times.

Descriptor 0 31 001, Delayed descriptor replication factor has a Data width of 8 bits and can therefore describe up to 255 replications, while 0 31 002, Extended delayed descriptor replication factor, has a Data width of 16 bits and can describe up to 65536 replications. If it is known that there will be more than 255 replications, 0 31 002, must be used instead of 0 31 001.

0 31 000: Special Case

0 31 000, Short delayed descriptor replication factor, has a data width of only 1 bit. Thus it can describe only two possibilities, no replications (bit turned off – value of zero) or one replication (bit turned on – value of 1). In order to understand the use of this descriptor, recall that if the replication count is zero for any sequence, there will be no corresponding values in the Data Section following the replication factor. 0 31 000 therefore provides an efficient method of saving space in the data section by not including missing data at a cost of only one bit. Recall, for example, that 0 12 001, Temperature/dry bulb temperature, has a Data width of 12 bits. If there are data sets in which the temperature is frequently missing, it may be more efficient to use 0 31 000 than setting all 12 bits on each time it is missing. This can be an effective way of saving space when BUFR is used to represent data in a database, particularly with data types in which some of the parameters are seldom observed.

3.1.4.4 Delayed Replication Using a Sequence Descriptor

Table D descriptor 3 02 005 expands to the four descriptors 0 08 002 0 20 011 0 20 012 0 20 013 we have been using as an example. If we use this sequence descriptor, the delayed replication sequence described in the previous Section becomes:

1 01 000 0 31 001 3 02 005

In this case, the meanings of the descriptors in the replication sequence are:

1 01 000	F = 1	Replication operator
	X = 01	The single descriptor following the Delayed description replication factor (0 31 001) is replicated
	Y = 000	Delayed replication
0 31 001	F = 0	Table B descriptor
	X = 31	Class 31 - Data description operator qualifiers
	Y = 001	Entry 031 - Delayed descriptor replication factor occupying 8 bits in Section 4
3 02 005	F = 3	Table D descriptor
	X = 02	Category 03 - Meteorological sequences common to surface data
	Y = 005	Entry 005 of Category 02

In this example, note the count of the number of descriptors to be replicated (X = 01) applies to the single Table D descriptor that is actually in the message, and NOT to the set of possibly very many descriptors that the single Table D descriptor represents (this rule applies when a sequence descriptor is used in conjunction with simple replication as well). As in the previous example, Y = 000 in the replication operator indicates that the number of times the sequence of parameters in the Data Section represented by 3 02 005 is replicated is indicated by the value of the Delayed descriptor replication factor just preceding the replicated sequence.

Figure 3.1.4 -1 provides an example of a TEMP observation sequence using a single Table D descriptor that expands to include delayed replication. In this example, the replication factor indicates how many levels are contained within the observation. The bit count of 245 bits is for 1 level, each additional level would require 83 bits.

SECTION 4

                                                                                                                             WIDTH IN

                                                                                                                             BITS

3 09 008     +3 01 038       +3 01 001     +0 01 001     WMO BLOCK NO.                7

                                                               +0 01 002     WMO STATION NO.          10

                                          +0 02 011                          RADIOSONDE TYPE           8

                                          +0 02 012                          RADIOSONDE COMP.

                                                                                    METHOD                              4

                                          +3 01 011     +0 04 001     YEAR                                   12

                                                               +0 04 002     MONTH                                 4

                                                               +0 04 003     DAY                                       6

                                          +3 01 012     +0 04 004     HOUR                                   5

                                                               +0 04 005     MINUTE                                6

                                          +3 01 024     +0 05 002     LATITUDE

                                                                                    (COARSE ACCURACY)     15

                                                               +0 06 002     LONGITUDE

                                                                                    (COARSE ACCURACY)     16

                                                               + 0 07 001    HEIGHT OF STATION       15

                   +3 02 004       +0 20 010                          CLOUD COVER (TOTAL)    7

                                          +0 08 002                          VERTICAL

                                                                                    SIGNIFICANCE                    6

                                          +0 20 011                          CLOUD AMOUNT                4

                                          +0 20 013                          HEIGHT OF BASE OF

                                                                                    CLOUD                               11

                                          +0 20 012                          CLOUD TYPE Cl                   6

                                          +0 20 012                          CLOUD TYPE Cm                6

                                          +0 20 012                          CLOUD TYPE Ch                 6

                   +1 01 000                            DELAYED REP. 1 DESCRIPTOR             0

                   +0 31 001                                                 REPLICATION COUNT        8

                   +3 03 014       +0 07 004                          PRESSURE                         14

                                          +0 08 001                          VERTICAL

                                                                                    SOUNDING SIG                   7

                                          +0 10 003                          GEOPOTENTIAL                17

                                          +0 12 001                          TEMPERATURE                 12

                                          +0 12 003                          DEW POINT                         12

                                          +0 11 001                          WIND DIRECTION                9

                                          +0 11 002                          WIND SPEED                       12

                                                               Total bits one replication                           83 bits

                   TOTAL BITS: 162 + n (replication number x 83), if n = 1, TOTAL = 245

Figure 3.1.4-1. Example of TEMP observations sequence using delayed replication

3.1.4.5 Delayed Repetition

Whereas delayed replication means the descriptor sequence is to be expanded to match the data, delayed repetition means the data is to be expanded to match the descriptor count. There are two delayed repetition descriptors available:

                                                                                                                     refer.      data

Descriptor                                name                                units       scale   value      width

0 31 011       Delayed descriptor and

                     data repetition factor                                    Numeric 0          0             8

0 31 012       Extended delayed descriptor and

                     data repetition factor                                    Numeric 0          0             16

In a delayed repetition sequence, the delayed replication operator is followed by either 0 31 011 or 0 31 012, depending of the number of repetitions expected. The data will contain only one value corresponding to the delayed repetition factor. However, the output of a decoder must produce the number of repetitions of that parameter, all with the same value, indicated by the delayed repetition factor. Delayed repetition is designed for run-length encoding consisting of a fixed number of values of a given element, the precision being such that many successive values may be the same.

For example, any line of a radar image can be broken up into segments consisting of identical pixel values and segments where the values vary. The first kind of segment calls for delayed repetition, a descriptor and a value both encoded once, but the value to be repeated in the output the number of times indicated by the delayed repetition factor. The second requires delayed replication, N values to be coded in the message and one descriptor repeated N times in the output to correspond.

3.1.5 Data Compression

Even though BUFR makes efficient use of space by virtue of binary numbers that take only as many bits as are necessary to hold the largest expected value, a further compression may be possible. The method employed by BUFR for data compression is similar to that used in the WMO Code FM 92 GRIB (GRidded Binary fields). Like elements from the full set of observations are collected together, their minimum values subtracted out, and the differences from the minimums are then encoded with a bit length selected to hold the largest difference from the minimum value. This is repeated for all the elements.

Consider the following group of identically defined data subsets:

	station	Station	pressure	temperature	dew point
	number	Height
subset 1	101	296	10132	122	110
subset 2	103	291	10122	121	110
subset 3	107	310	10050	105	099
subset 4	112	295	missing	110	102
subset 5	114	350	10055	095	089
subset 6	116	325	10075	101	091

The minimum value of each element, noted in bold above, is:

101 291 10050 095 089

For each element, subtract the minimum value of that element from each of the original values. Then the table becomes:

	Station number	Station height	pressure	temperature	dew point
subset 1	0	5	82	27	21
subset 2	2	0	72	26	21
subset 3	5	19	0	10	10
subset 4	11	4	missing	15	13
subset 5	13	59	5	0	0
subset 6	15	34	25	6	2

After each difference from the minimum value has been determined for each element, the number of bits necessary to store the largest difference value for each element is determined. For the station number the largest difference is 15, which is equivalent to 1111 in binary and therefore occupies 4 bits. This presents a small problem. All four bits set on, as is the case for the number 15, is properly interpreted as "missing", not as a numeric value of 15. What is done is to simply add one bit to the number needed to store the largest difference value; thus 15 gets stored in 5 bits, as 01111. It is not technically necessary to add one bit to the bit lengths for all the elements, only for those elements whose maximum differences to be encoded "fills" the available space; that is, if the number is 3 to be stored in 2 bits, 7 in 3 bits, 15 in 4 bits, 31 in 5 bits, etc. However, a convenient way to accomplish this and assure that there is always room for missing values (if needed) is to always add 1 to the largest difference value and figure the number of bits based on this larger-by-one value.

In the example, the station height would be placed in 6 bits, the pressure in 7 (with the "missing" indicated as 1111111), etc., as in the following table:

	Station number	Station height	pressure	temperature	dew point
Largest Difference Value + 1	16	60	83	28	22
Number of bits	5	6	7	5	5

Whereas in the non-compressed storage of data in Section 4 there is a continuous bit stream for all parameters for an entire observation, in the compressed form all elements of the same parameter from each observation form a continuous stream (see Figure 3.1.5-1). In order to decode compressed data, two additional items - the minimum value and the bit count - appear in the compressed form of storage in Section 4 preceding each set of element values. The Section 4 representation for compressed data for each parameter used in the example above is then:

Station number minimum value (101) occupying 10 bits as specified by the Table B data width for entry 0 01 002, followed by;

6 bits containing the count in bits (5) that each of the station number differences from the minimum value will occupy, followed by;

The 6 station number differences from the minimum values (0, 2, 5, 11, 13 and 15), where each value occupies 5 bits;

and etc for the other elements. Schematically, the layout of Section 4 for both non-compressed data and compressed data is:

Section 4 data non-compressed

+----------------------------------------------------------------------------------------------------------------------+

parameter 1, parameter 2,……parameter n          parameter 1,parameter 2,…...parameter n

+------------------------------------------------------+    +-----------------------------------------------------+

                  observation 1                                                             observation 2

Section 4 data compressed

+----------------------------------------------------------------------------------------------------------------------+

minimum value, bit count, parameter 1                  minimum value, bit count, parameter 2

+------------------------------------------------------+    +-----------------------------------------------------+

observation 1, obs. 2,…,…....observation n          observation 1,.obs. 2,…,…...observation n

Figure 3.1.5-1. Comparison of non-compressed and compressed data in Section 4

After the last station number difference (15), the next 15 bits (Table B data width for entry 0 07 001) will be taken by the minimum value for station height (291) followed by the count of bits to represent the differences (6) and then each of the elements occupying 6 bits apiece (5, 0, 19, 4, 59, 34). Continuing the process for all 5 parameters would produce within Section 4 the following bit counts:

                                       Station          station

                                       Number        height          pressure     temperature       dew point

Table B descriptor         0 01 002       0 07 001      0 10 004     0 12 004            0 12 006

Data width to contain

minimum value                   10              15                14                12                      12

6 bits containing the

bit counts for differences    6                6                  6                  6                        6

actual bit count contained

in the 6 bits                          5                6                  7                  5                        5

compressed data

representation for

6 subsets                          30               36                42               30                      30

total bit count for 6

subsets including

compression bit counts     46             +57              +62             +48                    +48

                                                                                                                          total bits =261

Thus, 261 bits are necessary to represent all 6 subsets in compressed form in Section 4. Using the same set of values for the 6 subsets in non-compressed form there would be bit counts in Section 4 as follows:

                                       Station          station

                                       Number        height          pressure     temperature       dew point

Table B descriptor

data width                           10              15                14                12                      12

Total bit count

for 6 subsets                       60            +90              +84             +72                    +72

                                                                                                                           = 378

A total of 378 bits are necessary to represent all 6 subsets in non-compressed form.

Special Case 1: All values of a single parameter are missing

There are other conditions that can occur when encoding compressed data. If all observed values of one parameter, or element, are missing, the minimum value occupying the specified Table B data width in Section 4 is set to all 1's, the 6 bits specifying how many bits are used for each value is set to 0, but the difference values are omitted. If, for example all the dew points were missing from the 6 subsets, then the number of bits to represent the dew points would still include 12 bits for the minimum value (all set to 1) and 6 bits to specify the number of bits used for each value. However, the dew point difference values themselves would be omitted – in this case reducing the number of bits to represent the dew point values by 30:

                                       Station          station

                                       Number        height          pressure     temperature       dew point

Table B descriptor         0 01 002       0 07 001      0 10 004     0 12 004            0 12 006

Data width to contain

minimum value                   10              15                14                12                      12

6 bits containing the

bit counts for differences    6                6                  6                  6                        6

actual bit count contained

in the 6 bits                          5                6                  7                  5                        0

compressed data

representation for

6 subsets                          30               36                42               30                        0

total bit count for 6

subsets including

compression bit counts     46             +57              +62             +48                    +18

                                                                                                                          total bits =231

In the non-compressed form, however, storage of the missing dew point values would still occupy 12 bits each, with all bits set to 1, and the size of the Data Section is not reduced:

                                       Station          station

                                       Number        height          pressure     temperature       dew point

Table B descriptor

data width                           10              15                14                12                      12

Total bit count

for 6 subsets                       60            +90              +84             +72                    +72

                                                                                                                           = 378

To summarize, if all values for a single parameter are missing when BUFR compression is used:

· The minimum value occupies the number of bits as indicated in Table B, with all bits set to 1.

· The 6 bits specifying how many bits are used for each value is set to 0.

· The difference values are omitted.

Special Case 2: All values of a single parameter are equal

The other condition that may occur is if all the difference values are identical. In this case, the minimum value occupying the specified Table B data width in Section 4 is set to the actual minimum value, the 6 bits specifying the count of bits for each difference value is set to 0, and difference values will be omitted. This condition would produce the same bit count as if all elements were missing.

To summarize, if all values for a single parameter are equal when BUFR compression is used:

· The minimum value occupies the number of bits as indicated in Table B and is set to the value (it is the minimum)

· The 6 bits specifying how many bits are used for each value is set to 0.

· The difference values are omitted.

Data compression is most effective when the range of values for the parameters is small. In the example of the 6 subsets, the number of bits needed to represent the difference of all values of each parameter from their minimum value is half, or less than half, the number of bits required for storage of the original values (as indicated by the Table B entry data widths). If the 6 subsets were put into a message where compression was not applied, the length of the message would be 100 octets (Figure 3.2.5-2). By applying compression, the length of the message would be reduced to 86 octets (Figure 3.2.5-3).

Using the range of values for the same 6 subsets, not realistic, but to show the effect of compression for a large data set, a total of 4267 subsets could be put into a BUFR message not exceeding 15000 octets (Figure 3.2.5-5) when compression is used. In non-compressed form there would only be 1898 subsets within the 15000-octet limit (Figure 3.2.5-4).

SECTION        OCTET NO.    OCTET IN       ENCODED

                        IN SECTION   MESSAGE      VALUE                     DESCRIPTION

Section 0

(indicator            1-4                   1-4                   BUFR              encoded CCITT international

section)                                                                                      Alphabet No. 5

                           5-7                   5-7                   100                  total message length (octets)

                           8                      8                         3                   BUFR edition number

 Section 1

(identification      1-3                   9-11                 18                    length of section (octets)

section)              4                      12                    0                      BUFR master table

                           5-6                   13-14               58                    originator (U.S. Navy - FNMOC)

                           7                      15                    0                      update sequence number

                           8                      16                    0                      indicator for no Section 2

                           9                      17                    0                      Table A - surface land data

                           10                    18                    0                      BUFR message sub-type

                           11                    19                    9                      version number of master table

                           12                    20                    0                      version number of local tables

                           13                    21                    92                    year of century

                           14                    22                    4                      month

                           15                    23                    18                    day

                           16                    24                    0                      hour

                           17                    25                    0                      minute

                           18                    26                    0                      reserved for local use by ADP centers (also needed to complete even number octets for section)

Section 3

(Data description 1-3                 27-29               18                    length of section (octets)

section)

                           4                      30                    0                      reserved

                           5-6                   31-32               6                      number of data subsets

                           7                      33                    bit 1=1             flag indicating observed data

                                                                           bit 2=0             flag indicating no compression

                           8-17                 34-43               0 01 002          WMO station no.

                                                                           0 07 001          height of station

                                                                           0 10 004          pressure

                                                                           0 12 004          temperature

                                                                           0 12 006          dew point

                           18                    44                    0                      needed to complete section with an even number of octets

Section 4

(Data section)    1-3                   45-47               52                    length of section (octets)

                           4                      48                    0                      reserved

                           5-52                 49-96               data                 continuous bit stream of data for 6 subsets, 63 bits per subset plus 6 bits to end on even octet

Section 5

(End section)     1-4                   97-100             7777                encoded CCITT international Alphabet No. 5

Figure 3.1.5-2.    BUFR message of 6 subsets in non-compressed form

SECTION        OCTET NO.    OCTET IN       ENCODED

                        IN SECTION   MESSAGE      VALUE                     DESCRIPTION

Section 0

(indicator            1-4                   1-4                   BUFR              encoded CCITT international

section)                                                                                      Alphabet No. 5

                           5-7                   5-7                   86                    total message length (octets)

                           8                      8                        3                    BUFR edition number

 Section 1

(identification      1-3                   9-11                 18                    length of section (octets)

section)              4                      12                    0                      BUFR master table

                           5-6                   13-14               58                    originator (U.S. Navy - FNMOC)

                           7                      15                    0                      update sequence number

                           8                      16                    0                      indicator for no Section 2

                           9                      17                    0                      Table A - surface land data

                           10                    18                    0                      BUFR message sub-type

                           11                    19                    9                      version number of master table

                           12                    20                    0                      version number of local tables

                           13                    21                    92                    year of century

                           14                    22                    4                      month

                           15                    23                    18                    day

                           16                    24                    0                      hour

                           17                    25                    0                      minute

                           18                    26                    0                      reserved for local use by ADP centers (also needed to complete even number octets for section)

Section 3

(Data description 1-3                 27-29               18                    length of section (octets)

section)

                           4                      30                    0                      reserved

                           5-6                   31-32               6                      number of data subsets

                           7                      33                    bit 1=1             flag indicating observed data

                                                                           bit 2=1             flag indicating compression

                           8-17                 34-43               0 01 002          WMO station no.

                                                                           0 07 001          height of station

                                                                           0 10 004          pressure

                                                                           0 12 004          temperature

                                                                           0 12 006          dew point

                           18                    44                    0                      needed to complete section with an even number of octets

Section 4

(Data section)    1-3                   45-47               52                    length of section (octets)

                           4                      48                    0                      reserved

                           5-52                 49-82               data                 261 continuous bits of compressed data plus 11 bits to end on even octet

Section 5

(End section)     1-4                   83-86               7777                encoded CCITT international Alphabet No. 5

Figure 3.1.5-3.         BUFR message of 6 subsets in compressed form

SECTION        OCTET NO.    OCTET IN       ENCODED

                        IN SECTION   MESSAGE      VALUE                     DESCRIPTION

Section 0

(indicator            1-4                   1-4                   BUFR              encoded CCITT international

section)                                                                                      Alphabet No. 5

                           5-7                   5-7                   15000              total message length (octets)

                           8                      8                         3                   BUFR edition number

 Section 1

(identification      1-3                   9-11                 18                    length of section (octets)

section)              4                      12                    0                      BUFR master table

                           5-6                   13-14               58                    originator (U.S. Navy - FNMOC)

                           7                      15                    0                      update sequence number

                           8                      16                    0                      indicator for no Section 2

                           9                      17                    0                      Table A - surface land data

                           10                    18                    0                      BUFR message sub-type

                           11                    19                    9                      version number of master table

                           12                    20                    0                      version number of local tables

                           13                    21                    92                    year of century

                           14                    22                    4                      month

                           15                    23                    18                    day

                           16                    24                    0                      hour

                           17                    25                    0                      minute

                           18                    26                    0                      reserved for local use by ADP centers (also needed to complete even number octets for section)

Section 3

(Data description 1-3                 27-29               18                    length of section (octets)

section)

                           4                      30                    0                      reserved

                           5-6                   31-32               1898                number of data subsets

                           7                      33                    bit 1=1             flag indicating observed data

                                                                           bit 2=0             flag indicating no compression

                           8-17                 34-43               0 01 002          WMO station no.

                                                                           0 07 001          height of station

                                                                           0 10 004          pressure

                                                                           0 12 004          temperature

                                                                           0 12 006          dew point

                           18                    44                    0                      needed to complete section with an even number of octets

Section 4

(Data section)    1-3                   45-47               14952              length of section (octets)

                           4                      48                    0                      reserved

                           5-52                 49-14996         data                 continuous bit stream of data for 1898 subsets, 63 bits per subset plus 10 bits to end on even octet

Section 5

(End section)     1-4                   14997-15000   7777                encoded CCITT international Alphabet No. 5

Figure 3.1.5-4.         BUFR message of 1898 subsets in non-compressed form

SECTION        OCTET NO.    OCTET IN       ENCODED

                        IN SECTION   MESSAGE      VALUE                     DESCRIPTION

Section 0

(indicator            1-4                   1-4                   BUFR              encoded CCITT international

section)                                                                                      Alphabet No. 5

                           5-7                   5-7                   15000              total message length (octets)

                           8                      8                        3                    BUFR edition number

 Section 1

(identification      1-3                   9-11                 18                    length of section (octets)

section)              4                      12                    0                      BUFR master table

                           5-6                   13-14               58                    originator (U.S. Navy - FNMOC)

                           7                      15                    0                      update sequence number

                           8                      16                    0                      indicator for no Section 2

                           9                      17                    0                      Table A - surface land data

                           10                    18                    0                      BUFR message sub-type

                           11                    19                    9                      version number of master table

                           12                    20                    0                      version number of local tables

                           13                    21                    92                    year of century

                           14                    22                    4                      month

                           15                    23                    18                    day

                           16                    24                    0                      hour

                           17                    25                    0                      minute

                           18                    26                    0                      reserved for local use by ADP centers (also needed to complete even number octets for section)

Section 3

(Data description 1-3                 27-29               18                    length of section (octets)

section)

                           4                      30                    0                      reserved

                           5-6                   31-32               6                      number of data subsets

                           7                      33                    bit 1=1             flag indicating observed data

                                                                           bit 2=1             flag indicating compression

                           8-17                 34-43               0 01 002          WMO station no.

                                                                           0 07 001          height of station

                                                                           0 10 004          pressure

                                                                           0 12 004          temperature

                                                                           0 12 006          dew point

                           18                    44                    0                      needed to complete section with an even number of octets

Section 4

(Data section)    1-3                   45-47               14952              length of section (octets)

                           4                      48                    0                      reserved

                           5-52                 49-14996         data                 119569 continuous bits of compressed data plus 15 bits to end on even octet

Section 5

(End section)     1-4                   14997-15000   7777                encoded CCITT international Alphabet No. 5

Figure 3.1.5-5. BUFR message of 4267 subsets in compressed form

3.1.6 Table C Data Description Operators

3.1.6.1 Changing Data Width, Scale and Reference Value.

Table C data description operators are used for a number of purposes. Perhaps the most common is to refine Table B attributes temporarily, such as changing the data width, scale or reference value of a Table B entry.

If data from a BUOY observation (FM 18, Report of a buoy observation) were being encoded into BUFR, there are no Table B entries corresponding to latitude and longitude in thousandths of degrees. The Table B entries for latitude and longitude are high accuracy (hundred thousandths of a degree) and coarse accuracy (hundredths of a degree). There are several possible methods to handle the encoding of latitude and longitude for BUOY in thousandths of degrees. One method would be to choose the high accuracy Table B entries for latitude and longitude in hundred thousandths of degrees. There would be no loss of accuracy, but a lot of unused bits for each observation would be encoded in Section 4. The high accuracy latitude requires 25 bits for representation, high accuracy longitude 26 bits. To represent latitude and longitude to thousandths of degrees would require 18 and 19 bits respectively. If the extra bits from using high accuracy were not deemed a concern, this would be the easiest method, but if it were desirable to use only the bits required to represent latitude and longitude in thousandths of degrees, there are two ways for this to be accomplished. First, and the least desirable of any method, would be to create local descriptors for Table B with the appropriate scale and reference values for thousandths of degrees. This is the least desirable method because if the BUFR message were to be transmitted to another center, then the receiving center would have to have the correct definition of the local descriptors available to their BUFR decoder program. The other method would be to use the Table C data description operators 2 01 Y to change the data width of the Table B descriptor for latitude and longitude, 2 02 Y to change the scale, and 2 03 Y to change the reference values.

Table 3.1.6-1. BUFR Table C - Data Description Operators

Reference	Operand	Operator Name	Operation Definition
F X
2 01	Y	Change data width	Add (Y-128) bits to the data width given for each data element in Table B, other than CCITT IA5 (character) data, code or flag tables
2 02	Y	Change scale	Add Y – 128 to Scale in Table B for elements that are not code or flag tables.
2 03	Y	Change reference values	Subsequent element values descriptors define new reference values for corresponding Table B entries. Y bits in the Data Section represent each new reference value. Definition of new reference values in concluded by encoding this operator with Y=255. Negative reference values shall be represented by a positive integer with the left-most bit (bit 1) set to 1.

There is now a choice to be made between temporarily changing latitude and longitude from hundredths of degrees to thousandths, or, from changing them from hundred thousandths to thousandths. It doesn't matter which is done, as the only difference between the choices will be the Y operand entries of the data description operators. For this example, we will choose to change latitude and longitude from hundredths of degrees (coarse accuracy) to thousandths of degrees. Before beginning, however, it will be useful to recall the Table B values of Scale, Reference Value, and Data width for 0 05 002 (latitude - coarse accuracy), and 0 06 002 (longitude - coarse accuracy):

Latitude (coarse accuracy): Scale = 2, Reference value = -9000, Data width = 15 bits

Longitude (coarse accuracy): Scale = 2, Reference value = -18000, Data width = 16 bits

Then review the steps to encode any value into BUFR:

1. Multiply the value in SI units by 10^SCALE in order to retain the desired precision.

2. Subtract the Reference value from the scaled value to ensure positive integers.

3. The value, now a positive integer, must fit within the bit width defined for this element.

Latitude – Change data width

The largest negative latitude, the latitude at 90.000 ^o South, is –90.000. Since we want to retain thousandths of degrees, we must multiply by 10³ (Scale of 3), which will produce -90000. In order to ensure positive integers, the Reference value for latitude will therefore have to be –90000.

The largest encoded value of latitude will be for 90 ^o North, or 90.000. Multiplying by 10³ (Scale of 3) produces 90000, and subtracting the Reference value of –90000 produces 180000. This is the largest value for Latitude that must be encoded, and the number of bits needed Change data width to accommodate 180000 is 18.

Table C entry 2 01 Y – Change data width - is used to change the data width of the Table B entry for latitude (coarse accuracy) from 15 bits to 18 bits. The Y operand is determined by the Operator definition of adding Y-128 bits to the data width given for the element 0 05 002. The number 128 is the midpoint between 1 and 255 which is the range of values for the 8 bits of Y. Numbers between 1 and 127 will produce a negative value for changing data width, 129 to 255 a positive value. Since the bit width must be increased by 3, a Y value of 131 is needed.

Latitude – Change Scale

The next step is to change the value of Scale from 2 (scaling by10²⁾ to 3 (scaling by 10³) in order to properly decode the reported latitude that will be encoded in Section 4 with 18 bits. This is accomplished with Table C entry 2 02 Y – Change scale. The definition for change scale is “Add Y – 128 to Scale in Table B for elements which are not code or flag tables”. Since the value of Scale must be changed from 2 to 3, a Y value of 129 is needed.

Latitude – Change Reference value

To complete the necessary changes for Table B, the reference value also needs to be modified. It was found above that the Reference value must be changed from –9000 to –90000. Here again it must be determined how many bits are necessary to accommodate the new value, as the new reference value itself is encoded into Section 4. The number of bits to accommodate 90000 (positive value) is 17. It is, however, necessary to indicate this is to be a negative value that will require an additional bit. To indicate a new reference value as negative, the left most bit of the reference value encoded into Section 4 is set to 1. The sequence of operators needed to change the reference value for 0 05 002 is:

1) The 2 03 018 "Change reference values” operator, which announces a change and states how many bits are set aside for the new reference value in the data section (18 in this example)

2) The Table B descriptor 0 05 002, which indicated that the Reference value for Latitude (coarse accuracy) is to be changed. There can be more than one Table B descriptors here if the Reference value of more than one Table B descriptor is to be changed, provided they all can be described in 18 bits. There are, of course, as many 18-bit values in the data as there are data descriptors following the 2 03 018 descriptor.

3) 2 03 255 to terminate the reference value definition

Longitude – Change data width, scale, and reference value

In this particular case it will not be necessary to have separate Data Description operators to modify longitude data width and change of scale. The increase in number of bits for data width to accommodate longitude to thousandths of degrees is also 3. The change of scale is also the same. There will, however, be a required change of reference value from -18000 to -180000. By following the same steps as when changing the latitude Table reference value, the Data Description operator for changing the longitude reference value would be 2 03 019 followed by the data descriptor 0 06 002, followed by the descriptor 2 03 255 to indicate the end of the list of descriptors for which reference values are being changed.

Once Data Description operators 2 01 Y, 2 02 Y and 2 03 Y have been used in Section 3, they remain in effect for the rest of whatever follows in the Section 3 data descriptions. To cancel operator 2 01, and 2 02, the additional entries must 2 01 000 and 2 02 000 must be included in Section 3. To cancel the reference value change indicated by the operator 2 03 018, there must be included in Section 3 an operator 2 03 000.

Coding example

The data description operators encoded into Section 3 for BUOY observations would then be:

                  0 01 005    buoy/platform identifier

                  0 02 001    type of station

                  3 01 011    Table D descriptor which expands to descriptors for year, month and day

                  3 01 012    Table D descriptor which expands to descriptors for hour and minute

    +----------2 01 131    increase data width by 3

¦

    ¦    +------2 02 129    increase scale by 1

    ¦    ¦

    ¦    ¦   +-- 2 03 018    change reference value - new value contained in 18 bits in Section 4

    ¦    ¦   ¦

    ¦    ¦   ¦     0 05 002    new reference value applies to latitude - coarse accuracy

    ¦    ¦   ¦

    ¦    ¦   +--  2 03 255    terminate reference value definition 203018

    ¦    ¦

    ¦    ¦   +--  2 03 019    change reference value - new value contained in 19 bits in Section 4

    ¦    ¦   ¦

    ¦    ¦   ¦      0 06 002    new reference value applies to longitude - coarse accuracy

    ¦    ¦   ¦

    ¦    ¦   +--  2 03 255    terminate reference value definition 203019

    ¦    ¦

    ¦    ¦          3 01 023    Table D descriptor which expands to descriptors 0 05 002 and 0 06 002

    ¦    ¦

    ¦    +------ 2 02 000    cancel change scale

¦

    +-----------2 01 000    cancel change data width

                    2 03 000    cause all redefined reference values to revert back to standard Table B values

            OTHER ADDITIONAL DATA DESCRIPTORS

            TO COMPLETE BUOY DESCRIPTION

Figure 3.1.6-1. Example of changing scale, Reference value and Data width for longitude and latitude, coarse accuracy

The order for cancellation of nested Data Description Operators follows the above pattern where the last defined is the first canceled.  Also note that the latitude and longitude must be encoded prior to the 2 03 000 descriptor or the reference values would have reverted back to the original Table B values.

If instead of changing latitude and longitude from hundredths to thousandths, it were to be changed from hundred thousandths to thousandths the following descriptions would be used:

                  0 01 005    buoy/platform identifier

                  0 02 001    type of station

                  3 01 011    Table D descriptor which expands to descriptors for year, month and day

                  3 01 012    Table D descriptor which expands to descriptors for hour and minute

    +----------2 01 121    decrease data width by 7

¦

    ¦    +------2 02 127    multiply scale by -1

    ¦    ¦

    ¦    ¦   +-- 2 03 018    change reference value - new value contained in 18 bits in Section 4

    ¦    ¦   ¦

    ¦    ¦   ¦     0 05 001    new reference value applies to latitude - high accuracy

    ¦    ¦   ¦

    ¦    ¦   +-- 2 03 255    terminate reference value definition 203018

    ¦    ¦

    ¦    ¦   +-- 2 03 019    change reference value - new value contained in 19 bits in Section 4

    ¦    ¦   ¦

    ¦    ¦   ¦    0 06 001    new reference value applies to longitude - high accuracy

    ¦    ¦   ¦

    ¦    ¦   +--2 03 255    terminate reference value definition 203019

    ¦    ¦

    ¦    ¦        3 01 021   Table D descriptor which expands to descriptors 0 05 001 and 0 06 001

    ¦    ¦

    ¦    +-----2 02 000    cancel change scale

¦

    +---------2 01 000    cancel change data width

                  2 03 000    Cause all redefined reference values to revert back to standard Table B values

            OTHER ADDITIONAL DATA DESCRIPTORS

            TO COMPLETE BUOY DESCRIPTION

Figure 3.1.6-2. Example of changing scale, Reference value and Data width for longitude and latitude, high accuracy

Which would be the better of the methods? Using high accuracy latitude and longitude, or using Data Description operators to change latitude and longitude definitions to thousandths of degrees will each produce the same results. In terms of number of bits saved by changing to thousandths of degrees over high accuracy, a BUOY observation containing data equivalent to the BUOY code (FM 18 Sections 0 through Section 2) would require 214 bits per observation using high accuracy latitude and longitude. If Data Description operators changed latitude and longitude to thousandths of degrees then the observation would require 200 bits per observation, or a saving of 14 bits per observation, hardly worth the effort!

The preceding example does not imply that changing data width, scale and reference values should not be done, but it does point out that to do so to lower the number of bits within the data section for a given parameter is probably not that beneficial. Rather, it is to be used in those instances where the Table B entries do not provide enough significance for new technologies. These operator descriptors provide the flexibility within BUFR to handle those situations. If, for example, satellites were to measure latitude and longitude to millionths of degrees, to maintain significance of those measurements would require changing data width, scale and reference values, at least until (or if) there is a new Table B entry.

This example also shows that when changing data width, scale and reference values, a single Table D descriptor cannot be used in Section 3. The reason is that changing data width and scale apply to all descriptors in Table B until the change data width and/or change scale is canceled. Since the descriptor to be affected may be deep within the Table D expansion process, there is no way to include the Data Descriptor operators in that expansion. A change in reference value, however, can be accomplished while still using a single Table D entry. This is possible because after the entry for change reference value, 2 03 YYY, there must also be included the Table B descriptor or multiple descriptors that are to have new reference values.

3.1.6.2 Changing Reference Value Only.

The Table B entries for geopotential, 0 07 003 and 0 10 003 have a reference value of -400, too restrictive for very low pressure systems. The Table C Data Description operator 2 03 YYY can be placed as the first descriptor in Section 3, followed by the Table B descriptor(s) to which it applies. Placing 2 03 010, followed by 0 10 003 before the Table D descriptor means that each time data is encountered in Section 4 for 0 10 003, the new reference value indicated by the count of 10 bits specified by YYY applies. Within 10 bits the limit of the new reference value as a negative number is -511. The descriptor to conclude the list of descriptors for which new reference values are supplied follows immediately, followed in turn by the Table D descriptor (Figure 3.1.6-3). In Figure 3.1.6-3, the order of the Section 3 descriptors is:

2 03 010 0 10 003 2 03 255 3 09 008

The Section 4 data will be in the order as indicated by Figure 3.1.6-3.

SECTION 4

                                                                                                                             WIDTH IN

                                                                                                                             BITS

2 03 010                                                                      CHANGE REFERENCE

                                                                                    VALUE (ACTUAL

                                                                                     REFERENCE VALUE

                                                                                    IN SECTION 4)                    0

0 10 003                                                                      REFERENCE VALUE

                                                                                    TO CHANGE FOR

                                                                                    GEOPOTENTIAL                10

2 03 255                                                                      TERMINATE CHANGE

                                                                                    REFERENCE VALUE           0

3 09 008     +3 01 038       +3 01 001     +0 01 001     WMO BLOCK NO.                7

                                                               +0 01 002     WMO STATION NO.          10

                                          +0 02 011                          RADIOSONDE TYPE           8

                                          +0 02 012                          RADIOSONDE COMP.

                                                                                    METHOD                              4

                                          +3 01 011     +0 04 001     YEAR                                   12

                                                               +0 04 002     MONTH                                 4

                                                               +0 04 003     DAY                                       6

                                          +3 01 012     +0 04 004     HOUR                                   5

                                                               +0 04 005     MINUTE                                6

                                          +3 01 024     +0 05 002     LATITUDE

                                                                                    (COARSE ACCURACY)     15

                                                               +0 06 002     LONGITUDE

                                                                                    (COARSE ACCURACY)     16

                                                               + 0 07 001    HEIGHT OF STATION       15

                   +3 02 004       +0 20 010                          CLOUD COVER (TOTAL)    7

                                          +0 08 002                          VERTICAL

                                                                                    SIGNIFICANCE                    6

                                          +0 20 011                          CLOUD AMOUNT                4

                                          +0 20 013                          HEIGHT OF BASE OF

                                                                                    CLOUD                               11

                                          +0 20 012                          CLOUD TYPE Cl                   6

                                          +0 20 012                          CLOUD TYPE Cm                6

                                          +0 20 012                          CLOUD TYPE Ch                 6

                   +1 01 000                            DELAYED REP. 1 DESCRIPTOR             0

                   +0 31 001                                                 REPLICATION COUNT        8

                   +3 03 014       +0 07 004                          PRESSURE                         14

                                          +0 08 001                          VERTICAL

                                                                                    SOUNDING SIG                   7

                                          +0 10 003                          GEOPOTENTIAL                17

                                          +0 12 001                          TEMPERATURE                 12

                                          +0 12 003                          DEW POINT                        12

                                          +0 11 001                          WIND DIRECTION               9

                                          +0 11 002                          WIND SPEED                      12

2 03 000                                                                      CAUSE REDEFINED

                                                                                    REFERENCE VALUE

                                                                                    TO REVERT BACK TO

                                                                                    STANDARD TABLE B

                                                                                    VALUE                                   0

                                                                                                      TOTAL BITS: = 255

Figure 3.1.6-3.      Example of changing the reference value for geopotential when embedded within a Table D descriptor

3.1.6.3 Add Associated Field.

The Data Description operator 2 04 Y permits the inclusion of quality control information of Y bits attached to each following data element. The additional Y bits of the associated field appear in the data section as prefixes to the actual data elements. The Add Associated Field operator, whenever used, must be immediately followed by the Class 31 Data description operator qualifier 0 31 021 to indicate the meaning of the associated fields.

2 04 Y is defined in Table C as:

Table
Reference	Operand	Operator Name	Operation Definition
F X
2 04	Y	Add associated field	Precede each data element field with Y bits of information. This operation associates a data field (e.g. quality control information) of Y bits with each data element.

Also of value in this case are several notes that apply to this Table C entry. They are:

(5) Nesting of the operator descriptor 2 04 Y is defined such that:

(a) Each new definition adds to the currently defined associated field:

(b) Each cancellation (2 04 0) cancels only the most recently defined addition to the associated field.

(6) When the descriptor 2 04 Y is to be used, it shall precede the first of the data descriptors to which it applies.

(7) The data descriptor operator 2 04 Y shall be followed immediately by the descriptor 0 31 021 to indicate the meaning of the associated fields.

(8) In the data stream, the 6 bits described by 0 31 021 shall precede the Y bits.

(9) Once an associated field has been established and given meaning, the meaning may be changed by a re-application of descriptor 0 31 021. The associated field needs not to be cancelled in order to change the meaning. Further, if an associated field is cancelled, and then re-established, it must be given a meaning by a proper application of the 0 31 021 descriptor, as described in Notes (5) to (8), i.e., a previous assignment of meaning does not remain in force when the associated filed is cancelled.

Keep these notes in mind, and refer back to them as necessary, during the ensuing description of the Add associated field operator descriptor. Hopefully, the meaning of these notes will be clear upon completion of the discussion.

The Data description operator qualifier 0 31 021 takes its meaning from a code table:

Code Table 0 31 021: Associated field significance

Code Figure Meaning

               0                            Reserved

               1                            1 bit indicator of quality:        0 = good

                                                                                            1 = suspect or bad

               2                            2 bit indicator of quality:        0 = good

                                                                                            1 = slightly suspect

                                                                                            2 = highly suspect

                                                                                            3 = bad

               3-6                         Reserved

               7                            Percentage confidence

               8-20                       Reserved

               21                          1 bit indicator of correction:  0 = original value

                                                                                            1 = substituted/corrected value

               22-62                     Reserved for local use

               63                          Missing value

Associated fields are generally for the purpose of providing additional information about the particular data element with which they are associated. The most common use is in the arena of "quality assessment", where some sort of "confidence" indication is given. Other applications are possible and can be established by additions to Code Table 0 31 021.

Creating (or dealing with) an associated field in a message is a two-step process. The first is to establish the field and set the number of bits that will precede all the data elements following the appearance of the associated field operator (2 04 YYY). YYY is that number of bits. If 255 bits is not enough, you can keep adding more bits by repeating the operator. You can also generate compound associated fields by repeating the operator if the additional information is complicated.

The second step is to define the meaning of those bits, i.e., how they are to be interpreted by a user of the data. This is done by immediately following each 2 04 YYY descriptor with Class 31 descriptor, 0 31 021, which, by reference to the Code table 0 31 021, establishes that meaning. A little care is required here. Code Table 0 31 021 gives a (small) number of significance code figures (all taking up 6 bits in the data) for different size associated fields; one must be consistent in setting an associated field length and identifying the meaning of the bits in the field.

Once an associated field is established, those extra bits must be (are assumed to be) prefixed to every following data element, until the associated field is canceled.  If the quality information has no meaning for some of those following elements, but the field is still there, there is at present no explicit way to indicate "no meaning" within the currently defined meanings.  One must either redefine the meaning of the associated field in its entirety (by including 0 31 021 in the message with a data value of 63 - "missing value") or remove the associated field bits by the "cancel" operator: 2 04 000.  If multiple or compound associated fields have been defined, each must be canceled separately

If quality control information were to be added to a single parameter such as pressure (Table B descriptor 0 07 004), the following sequence would appear in Section 3:

                               2 04 007  0 31 021  0 07 004  2 04 000

The meanings of the descriptors in this sequence are:

                        2 04 007 - indicator that 7 bits of data precede all following Table B entries

                        0 31 021 - code table entry for the meaning of the 7 bits preceding the Table B entry

                        0 07 004 - Table B entry for pressure

                        2 04 000 - cancellation of the Add associated field operator

The Section 4 data width for this sequence is 27 bits.  The operators 2 04 007 and 2 04 000 do not occupy any bits within Section  4.  The 27 bits are taken by 0 31 021 (6 bits) and 0 07 004 (21 bits - 7 bits of associated field plus 14 bits of pressure value).

               When multiple Table B entries are preceded by 2 04 YYY as in

                               2 04 007  0 31 021  0 07 004  0 31 021  0 10 003  2 04 000,

the Add associated field operator 2 04 007 and the Associated field significance descriptor 0 31 021 both apply to the two Table B descriptors 0 07 004 and 0 10 003.  The Section 4 data width for the sequence in this case is:

                               2 04 007    0 bits

                               0 31 021    6     (establish meaning of associated field)

                               0 07 004    21   (7 associated bits plus bits 14 data)

                               0 31 021    6     (change meaning of associated field)

                               0 10 003    24   (7 associated bits plus 17 bits data)

                               2 04 000    0

Note that the associated fields are not prefixed onto the data described by 0 31 YYY descriptor.  This is a general rule: none of the Table C operators are applied to any of the Table B, Class 31 Data description operator qualifiers.

Now consider the following sequence of parameters as described by the Table D descriptor 3 03 014:

Section 4

                                                                                                   Width in bits

                      +0 07 004----------------> Pressure -------------------   14

                      ¦0 08 001----------------> Vertical sounding sig -----    7

                      ¦0 10 003----------------> Geopotential ---------------   17

3 03 014 ---> ¦0 12 001----------------> Temperature ----------------  12

                      ¦0 12 003----------------> Dew point ------------------    12

                      ¦0 11 001----------------> Wind direction -------------     9

                      +0 11 002----------------> Wind speed --------------    12

---

                                                                                    Total bits     83

If quality control information were to be added to this sequence by placing in Section 3 the operators 2 04 YYY and 0 31 021 immediately preceding 3 03 014, and the cancellation operator 2 04 000 following 3 03 014, the following sequence would be produced:

Section 4

                                                                                                   Width in bits

2 04 007------------------------------------> Add associated field--------   0

0 31 021------------------------------------> Associated field sig.---------  6

                        ¦-----------------------------Associated field ------------     7

                        |0 07 004----------------> Pressure -------------------    14

                        ¦------------------------------Associated field ------------    7

                        ¦0 08 001----------------> Vertical sounding sig -----    7

                        ¦-----------------------------Associated field ------------     7

                        ¦0 10 003----------------> Geopotential ---------------   17

                        ¦-----------------------------Associated field ------------     7

3 03 014 -----> ¦0 12 001---------------> Temperature ---------------   12

                        ¦-----------------------------Associated field ------------     7

                        ¦0 12 003----------------> Dew point ------------------    12

                        ¦-----------------------------Associated field ------------     7

                        ¦0 11 001----------------> Wind direction -------------     9

                        ¦-----------------------------Associated field ------------     7

                        |0 11 002----------------> Wind speed --------------      12

2 04 000------------------------------------> Cancel Add associated

                                                              field----------------------------    0

---

                                                                                    Total bits       138

Adding associated fields to a data sequence that is described by a Table D descriptor means the associated fields are placed before all data items in the sequence.  If quality control information were to be applied only to the pressure and geopotential parameters, the Table D descriptor could not be used but instead each individual parameter would have to be listed in Section 3.

                   +-- 2 04 007----------------> Add associated field -------    0

                   ¦     0 31 021----------------> Associated field sig -------     6

                   ¦                                          Associated field -----------     7

                   ¦     0 07 004----------------> Pressure -------------------     14

                   +-- 2 04 000----------------> Cancel add associated field- 0

                       0 08 001----------------> Vertical sounding sig --------    7

                   +-- 2 04 007----------------> Add associated field--------    0

                   ¦     0 31 021----------------> Associated field sig ---------   6

                   ¦                                         Associated field--------------    7

                   ¦     0 10 003----------------> Geopotential ----------------   17

                   +-- 2 04 000----------------> Cancel add associated field- 0

                       0 12 001----------------> Temperature ------------------   12

                       0 12 003----------------> Dew point ----------------------   12

                       0 11 001----------------> Wind direction -----------------    9

                       0 11 002----------------> Wind speed --------------------- 12

---

                                                 Total bits -------------------------------  109

If quality control information were to be added to the pressure and geopotential parameters of the TEMP observations as described in Figure 3.1.4-1 the following adjustments would have to be made.  The single Table D descriptor 3 09 008 could no longer be used as the expansion includes the additional Table D descriptor 3 03 014 which further expands to those parameters where quality control information would need to be inserted.  The actual order of the Section 3 descriptors would now be:

                               3 01 038   3 02 004   1 13 000   0 31 001   2 04 007   0 31 021

                               0 07 004   2 04 000   0 08 001   2 04 007   0 31 021   0 10 003

                               2 04 000   0 12 001   0 12 003   0 11 001   0 11 002

               The expansion of this descriptor sequence, and the order of items in Section 4, is illustrated in Figure 3.1.6-4:

SECTION 4

                                                                                                                             WIDTH IN

                                                                                                                             BITS

                   +3 01 038       +3 01 001     +0 01 001     WMO BLOCK NO.                7

                                                               +0 01 002     WMO STATION NO.          10

                                          +0 02 011                          RADIOSONDE TYPE           8

                                          +0 02 012                          RADIOSONDE COMP.

                                                                                    METHOD                              4

                                          +3 01 011     +0 04 001     YEAR                                   12

                                                               +0 04 002     MONTH                                 4

                                                               +0 04 003     DAY                                       6

                                          +3 01 012     +0 04 004     HOUR                                   5

                                                               +0 04 005     MINUTE                                6

                                          +3 01 024     +0 05 002     LATITUDE

                                                                                    (COARSE ACCURACY)     15

                                                               +0 06 002     LONGITUDE

                                                                                    (COARSE ACCURACY)     16

                                                               + 0 07 001    HEIGHT OF STATION       15

                   +3 02 004       +0 20 010                          CLOUD COVER (TOTAL)    7

                                          +0 08 002                          VERTICAL

                                                                                    SIGNIFICANCE                    6

                                          +0 20 011                          CLOUD AMOUNT                4

                                          +0 20 013                          HEIGHT OF BASE OF

                                                                                    CLOUD                               11

                                          +0 20 012                          CLOUD TYPE Cl                   6

                                          +0 20 012                          CLOUD TYPE Cm                6

                                          +0 20 012                          CLOUD TYPE Ch                 6

                   +1 13 000                                                 DELAYED REP.

                                                                                    13 DESCRIPTORS               0

                   +0 31 001                                                 REPLICATION FACTOR      8

                   +2 04 007                                                 ADD ASSOCIATED FIELD   0

                   +0 31 021                                                 ASSOCIATED FIELD

                                                                                    SIGNIFICANCE                    6

                                                                                    ASSOCIATED FIELD            7

                   +3 03 014       +0 07 004                          PRESSURE                         14

                   +2 04 000                                                 CANCEL ADD

                                                                                    ASSOCIATED FIELD            0

                                          +0 08 001                          VERTICAL

                                                                                    SOUNDING SIG                   7

                   +2 04 007                                                 ADD ASSOCIATED FIELD   0

                   +0 31 021                                                 ASSOCIATED FIELD

                                                                                    SIGNIFICANCE                    6

                                                                                    ASSOCIATED FIELD            7

                                          +0 10 003                          GEOPOTENTIAL                17

                   +2 04 000                                                 CANCEL ADD

                                                                                    ASSOCIATED FIELD            0

                                          +0 12 001                          TEMPERATURE                 12

                                          +0 12 003                          DEW POINT                         12

                                          +0 11 001                          WIND DIRECTION                9

                                          +0 11 002                          WIND SPEED                       12

                                                                                    TOTAL BITS:                     277

Figure 3.1.6-4. Example of TEMP observations sequence using delayed replication and quality control information

3.1.6.4 Encoding Character Data.

There may be occasions when it is necessary to encode character data into BUFR. An observation encoded into BUFR that originated from the character code FM 13 SHIP, for example, has within that code form the optional inclusion of plain language. If this character information were carried over for encoding into BUFR, the Data description operator 2 05 Y would be used in Section 3 to indicate the inclusion of character data in Section 4 of the BUFR message. Table C defines 2 05 Y as follows:

Table
Reference	Operand	Operator Name	Operation Definition
F X
2 05	Y	Signify character	Y characters (CCITT international Alphabet No. 5) are inserted as a data field of Y x 8 bits in length

The following parameters from the FM 13 SHIP code form:

            +    6IsEsEsRs       +

            ¦                              ¦

         (  ¦    or ICING +        ¦  )

            ¦                              ¦

            +    plain language +

described by BUFR descriptors would be:

Is ------->         0 20 033          Cause of ice accretion
EsEs ----->      0 20 031          Ice deposit (thickness)
Rs ------->       0 20 032          Rate of ice accretion

It would have to be determined in advance how many characters would be allowed for the plain language. If only the word ICING were to be placed in Section 4, the Data Descriptor 2 05 005 would be used. If it were determined that ICING plus 25 additional characters, including spaces, were to be described then the descriptor would be 2 05 030. The data descriptors and data width in Section 4 would then be:

                                                                 data width

                                                                       in bits

              0 20 033   cause of ice accretion          4

              0 20 031   ice deposit (thickness)         7

              0 20 032   rate of ice accretion             3

              2 05 030   character information        240

Since an observation in FM 13 SHIP code would have either the parameters for ice reported, or ICING + plain language, but not both, then if there were no plain language the character information would be set to spaces. If the ICING + plain language were reported then the data for descriptors 0 20 033, 0 20 031 and 0 20 032 would be set to missing (all bits set to “1”). Since Section 3 indicates a count of how many subsets (observations) are included in Section 4, the above descriptors apply to all subsets, even if an individual observation does not contain any icing information. In that case the entire set of icing data for an observation would be set to missing and spaces.

3.1.6.5 Signifying Length of Local Descriptors.

Local Descriptors were provided in BUFR to enable a data processing center the capability of describing information of any type within BUFR for the center's internal use. There does exist, however, the possibility that once data is described in BUFR it may be necessary to transmit a BUFR message to another center, where the BUFR message would contain local information. Since a receiver of the BUFR message may or not know the meaning of the local descriptor, it could be impossible to decode the message, as the receiver would not know the data width in Section 4 of the local information (Figure 3.1.3-5). While it could be argued that BUFR messages containing local information should never be transmitted to another center, it may require a separate set of software to remove local information before the message is ready for transmission. To overcome this situation the Data Description operator 2 06 Y was developed to allow local information to be contained within a transmitted message and to give information to the receiver that indicates the length in bits of the local data. Table C defines 2 06 Y as:

Table
Reference	Operand	Operator Name	Operation Definition
F X
2 06	Y	Signify data width for the immediately following local descriptor	Y bits of data are described by the immediately following descriptor

The meaning of the Data Description operator 2 06 Y is that the following local descriptor is describing Y bits of data in Section 4. Figure 3.1.6-5 is the same as Figure 3.1.3-5, but with the proper 2 06 Y descriptor added. Knowing the width in bits of data in Section 4 then allows the receiver of the message to bypass that number of bits and allow proper decoding of Section 4.

The operator 2 06 Y can only be used when it precedes a local descriptor with F = 0. While it is within the rules of BUFR to create local descriptors with F = 3 (sequence descriptor), the Data Description operator 2 06 Y cannot be used to bypass whatever number of bits are being described by a sequence descriptor. Since a sequence descriptor expands to other descriptors and in the expansion process other local descriptors or delayed replication may be encountered, there is no way of knowing in advance how many total bits are covered by a sequence descriptor.

SECTION 4 WIDTH

                                                                                                                                 IN BITS

2 06 003                                                                            > 3 BITS ARE DESCRIBED

                                                                                          BY THE FOLLOWING LOCAL DESCRIPTOR           0

0 54 192                                                                            > Local descriptor                 3

3 07 002       +3 01 032  +3 01 001           +0 01 001        >WMO BLOCK NO.              7

                                                                  +0 01 002        >WMO STATION NO.        10

                                       +0 02 001                                   >TYPE OF STATION           2

                                       +3 01 011           +0 04 001        >YEAR                                12

                                                                  +0 04 002        >MONTH                              4

                                                                  +0 04 003        >DAY                                     6

                                       +3 01 012           +0 04 004        >HOUR                                 5

                                                                  +0 04 005        >MINUTE                              6

                                       +3 01 024           +0 05 002        >LATITUDE (COARSE

                                                                                            ACCURACY)                    15

                                                                  +0 06 002        >LONGITUDE(COARSE

                                                                                            ACCURACY)                    16

                                                                  +0 07 001        >HEIGHT OF STATION     15

                     +3 02 011  +3 02 001           +0 10 004        >PRESSURE                      14

                                                                  +0 10 051        >PRESSURE REDUCED

                                                                                            TO MSL                            14

                                                                  +0 10 061        >3 HR PRESSURE

                                                                                            CHANGE                          10

                                                                  +0 10 063        >CHARACTERISTIC OF

                                                                                            PRESSURE                        4

                                       +3 02 003           +0 11 011        >WIND DIRECTION             9

                                                                  +0 11 012        >WIND SPEED AT 10m      12

                                                                  +0 12 004        >DRY BULB TEMP AT

                                                                                            2m                                     12

                                                                  +0 12 006        >DEW POINT TEMP AT

                                                                                            2m                                     12

                                                                  +0 13 003        >RELATIVE HUMIDITY        7

                                                                  +0 20 001        >HORIZONTAL

                                                                                            VISIBILITY                        13

                                                                  +0 20 003        >PRESENT WEATHER        8

                                                                  +0 20 004        >PAST WEATHER (1)          4

                                                                  +0 20 005        >PAST WEATHER (2)          4

                                       +3 02 004           +0 20 010        >CLOUD COVER

                                                                                            (TOTAL)                             7

                                                                  +0 08 002        >VERTICAL

                                                                                            SIGNIFICANCE

                                                                                            SURFACE OBS                  6

                                                                  +0 20 011        >CLOUD AMOUNT              4

                                                                  +0 20 013        >HEIGHT OF BASE OF

                                                                                            CLOUD                             11

                                                                  +0 20 012        >CLOUD TYPE Cl                6

                                                                  +0 20 012        >CLOUD TYPE Cm              6

                                                                  +0 20 012        >CLOUD TYPE Ch               6

                                                                                          TOTAL BITS:                   270

Figure 3.1.6-5. Example of surface observations with local descriptor and data descriptor operator 2 06 Y

3.1.6.6 Data Not Present.

Table C defines the Data not present operator (2 21 YYY) as follows:

Table
Reference	Operand	Operator Name	Operation Definition
F X
2 21	YYY	Data not present	Data values present in Section 4 (Data section) corresponding to the following YYY descriptors shall be limited to data from Classes 1 – 9 and Class 31

2 21 YYY permits the construction of a BUFR message containing only coordinate (classes 1 – 9) and delayed replication and quality control information (Class 31). Recall that delayed replication in conjunction with Associated field significance (0 31 021) permits specification of (somewhat limited) quality assessment information. Combining this information with the coordinate information could then be linked back to the original data-containing message by comparison of the coordinate information in the two messages or, in a local context, through database information in Section 2.

3.1.6.7 Quality Assessment Information

The remaining data description operators currently in Table C (Operator descriptors 2 22 000 through 2 37 255) provide a more sophisticated method of including quality assessment information in BUFR than does use of the Add associated field operator. Table C defines these descriptors as:

Table
Reference	Operand	Operator Name	Operation Definition
2 22	000	Quality information follows	The values of Class 33 elements which follow relate to the data defined by the data present bit-map.
2 23	000	Substituted values operator	The substituted values which follow relate to the data defined by the data present bit-map.
2 23	255	Substituted values marker operator	This operator shall signify a data item containing a substituted value; the element descriptor for the substituted value is obtained by the application of the data present bit-map associated with the substituted values operator.
2 24	000	First order statistical values follow	The statistical values which follow relate to the data defined by the data present bit map.
2 24	255	First order statistical values marker operator	This operator shall signify a data item containing a first order statistical value of the type indicated by the preceding 0 08 023 element descriptor; the element descriptor to which the first order statistic relates is obtained by the application of the data present bit-map associated with the first order statistical values follow operator; first order statistical values shall be represented as defined by this element descriptor.
2 25	000	Difference statistical values follow	The statistical values which follow relate to the data defined by the data present bit-map.
2 25	255	Difference statistical values marker operator	This operator shall signify a data item containing a difference statistical value of the type indicated by the preceding 0 08 024 element descriptor; the element descriptor to which the first order statistic relates is obtained by the application of the data present bit-map associated with the difference statistical values follow operator; difference statistical values shall be represented as defined by this element descriptor, but with a reference value of –2ⁿ and a data width of (n+1), where n is the data width given by the original descriptor. This special reference value allows the statistical difference values to be centered around zero.
2 32	000	Replaced/retained values follow	The replaced/retained values which follow relate to the data defined by the data present bit-map.
2 32	255	Replaced/retained value marker operator	This operator shall signify a data item containing the original of an element which has been replaced by a substituted value. The element descriptor for the retained value is obtained by the application of the data present bit-map associated with the substituted values operator.
2 35	000	Cancel backward data reference	This operator terminates all previously defined backward references and cancels any previously defined data present bit-map; it causes the next data present bit-map to refer to the data descriptors which immediately precede the operator to which it relates.
2 36	000	Define data present bit-map	This operator defines the data present bit-map which follows for possible re-use; only one data present bit-map may be defined between this operator and the cancel use defined data present bit-map operator.
2 37	000	Use defined data present bit-map	This operator causes the defined data present bit-map to be used again.
2 37	255	Cancel use defined data present bit-map	This operator cancels the re-use of the defined data present bit-map.

Since these operator descriptors are more sophisticated, however, they are also more difficult to understand (and challenging to program). Consequently, the explanation is lengthy. Furthermore, many may not use them and therefore not need to understand them. Therefore, the explanation of the use of operator descriptors 2 22 000 through 2 37 255 is contained in the Appendix to Chapter 3.1.6.7.

3.2 CREX

3.2.1 SECTIONS OF A CREX MESSAGE

3.2.1.1 Overview of a CREX Message

The term "message" refers to FM 95 CREX (referred throughout the remainder of Chapter 3.2 as simply “CREX”) being used as a data transmission format, although CREX could be used for on-line storage or data archiving as well. For transmission of data, each CREX message consists of a string of alphanumeric characters, comprising five sections:

Section 0		Section 1		Section 2	Section 3	Section 4
Section Number	Name		Contents
0	Indicator Section		"CREX" (four alphanumeric letters)
1	Data Description Section		CREX Master Table number, edition, number, and table version number, data category, a collection of data descriptors which define the form and content of data subsets in the Data Section, and optional check digit indicator “E”
2	Data Section		A set of data subsets defined in Section 1
3	Optional Section		“SUPP” (four alphanumeric letters), followed by additional items for local use
4	End Section		"7777" (four alphanumeric figures)

All sections of a CREX message except the last terminate with the two contiguous characters “++”, which are referred to as the “Section terminator”. Within the Data Description and Data sections, CREX messages contain a series of “groups”. Regulation 95.1.3 defines a “group” as follows: “A group is a sequence of one or more contiguous characters corresponding to a single data descriptor or data value. Groups shall be separated from each other by one or more space characters. Multiple space characters shall be used when needed to improve human readability.” Theoretically there is no upper limit to the size of a CREX message but, by convention, CREX messages are restricted to 15000 octets or 120000 bits.

Figure 3.2.1-1 is an example of a complete CREX message consisting of one land surface report. This example will be used to describe the structure of a CREX message. CREX uses many of the principles of FM 94 BUFR. These will be pointed our as we proceed through Chapter 3.2. Sample CREX Report:

CREX++

T000103 A000 D07004++

03 075 1 1989 01 09 09 00 5845 –00308 09962 10001 0019 03 240 0013 –073 /// ///

3000 015 07 02 075 07 06 0120 38 20 10 0001 07 05 08 0120++

Figure 3.2.1-1: Sample CREX Report

3.2.1.2 Section 0 – Indicator Section

Structure

SECTION 0		Section 1		Section 2	Section 3	Section 4
Group Number	Contents		Meaning
1	CREX		Beginning of a CREX Message

Example

The sample CREX message in Figure 3.2.1-1 is reproduced below, with the Indicator Section in bold.

CREX++

T000103 A000 D07004++

03 075 1 1989 01 09 09 00 5845 –00308 09962 10001 0019 03 240 0013 –073 /// ///

3000 015 07 02 075 07 06 0120 38 20 10 0001 07 05 08 0120++

Figure 3.2.1-2: Sample CREX Report, highlighting the Indicator Section

3.2.1.3 Section 1 – Data Description Section

Structure

Section 0		SECTION 1		Section 2	Section 3	Section 4
Group Number	Contents		Meaning
1	Ttteevv		T: Indicator for CREX Tables tt: CREX Master Table (00 for Standard WMO FM 95 CREX Tables) ee: CREX Edition Number (currently 01) vv: CREX Table Version Number (currently 03)
2	Annn		A: Indicator for CREX Table A nnn: Data Category from CREX Table A
3 to n	Bxxyyy, Cxxyyy, Dxxyyy, And/or Rxxyyy		B, C, D: Indicators for CREX Table B, C, or D entries xx: xxyyy indicates references from CREX tables B, C, or D yyy, and/or R: Indicator for CREX Replication Operator xx: number of replicated descriptors yyy: number of replications of the xx descriptors. yyy = 0 indicates delayed replication, where the number of repetitions is found in the Data Section.
n + 1	E		E: Optional check digit indicator

The Data Description Section describes the form and content of the data in the Data Section. There are four basic sets of information in the Data Description Section: the edition/table description, the data category, the data description itself, and an optional check digit indicator. The data described by the set of descriptors in Section 1 is collectively referred to as a data subset. For observational data, one data subset corresponds to one report.

Example

The sample CREX message in Figure 3.2.1-1 is reproduced below, this time with the Data Description Section in bold:

CREX++

T000103 A000 D07004++

03 075 1 1989 01 09 09 00 5845 –00308 09962 10001 0019 03 240 0013 –073 /// ///

3000 015 07 02 075 07 06 0120 38 20 10 0001 07 05 08 0120++

Figure 3.2.1-3: Sample CREX Report, highlighting the Data Description Section

Edition/Table Group

This group of information always begins the Data Description Section. The group has the form Ttteevv, where:

Letter        Meaning                                                        Use in the Sample CREX Message

   T      ---> Indicator for the Edition/Table Group

   tt      ---> CREX Master Table used -------------------> 00 for Standard WMO FM 95 CREX Tables

   ee    ---> CREX Edition Number used ----------------> 01 for current CREX edition Number

   vv    ---> CREX Table version number used --------> 03 for current CREX Master Tables version

Data Category Group

This is always the second group in the Data Description Section. The group has the form Annn, where:

Letter        Meaning                                                        Use in the Sample CREX Message

   A      ---> Indicator for the Data Category Group

   nnn   ---> Data category from CREX Table A -------> 000 for Surface data - land

Data Description Groups

After the CREX edition/table descriptor group and data category group, Section 1 has one or more data descriptors. Each data descriptor is considered one group. These groups are the heart of a CREX message. As with BUFR, a descriptor is of the form F xx yyy, where the type of group depends on F:

F = B è Element descriptor, and refers to Table B entries

F = C è Operator descriptor, and refers to Table C entries

F = D è Sequence descriptor, and refers to Table D entries

F = R è Replication operator

The meanings of and uses for X and Y depend on the value of F. CREX descriptors are discussed in detail in Chapter 3.2.2. The descriptor sequence in the sample CREX message is decomposed in detail in Chapter 3.2.4.

Optional Check Digit Indicator

A check digit indicator is optional at the end of Section 1. If present, it takes the form of the single character “E”, and signifies the presence of a check digit added in front of each data value in the Data Section.

3.2.1.4 Section 2 – Data Section

Structure

Section 0		Section 1		SECTION 2	Section 3	Section 4
Group Number	Contents		Meaning
1 to n	(d) data values		d: Optional check digit data values: Data values corresponding to the descriptors in Section 1

Organization of the Data Values

The Data Section is comprised of one or more groups, where each group represents one data value. The sequence of data values corresponds to the list of descriptors defined in the Data Description Section. The set of data values corresponding to a single application of the descriptors in the Data Description Section comprises one data subset. There may be many data subsets in the Data Section. In that case, each data subset in the Data Section is terminated by the character “+” (the “subset terminator”). However the subset terminator is not present following the last data subset in the Data Section – rather the section terminator (“++”) serves that purpose. The data values in Section 2 are separated by at least one space character. Additional space characters may be inserted between groups to improve alignment and readability.

Each numeric data value includes leading zeroes when the number of digits required to represent the value is smaller than the number of characters defined in the corresponding CREX Table B entry, and likewise for the delayed replication number if present. Each character value is left justified and includes trailing blanks when the number of characters required to represent the data value is smaller than the number of characters defined in the corresponding CREX Table B entry. Keeping the number of characters representing the data value always equal to the original data width defined in CREX Table B or in the regulations facilitates both the encoding and the interpretation of a CREX message.

Only negative numbers are signed. The number of characters allowed for a group, found in CREX Table B, does not include the negative sign if it is present. A missing value in Section 2 is represented by a string of solidi (“/”) characters equal in number to the number of characters allowed for that group in CREX Table B

Example Without Optional Check Digit

The sample CREX message in Figure 3.2.1-1 is reproduced below, this time with the Data Section in bold. The Data Section of this sample message will be decomposed in detail in Chapter 3.2.4.

CREX++

T000103 A000 D07004++

03 075 1 1989 01 09 09 00 5845 –00308 09962 10001 0019 03 240 0013 –073 /// ///

3000 015 07 02 075 07 06 0120 38 20 10 0001 07 05 08 0120++

Figure 3.2.1-4: Sample CREX Report, highlighting the Data Section

Example With Optional Check Digit

Figure 3.2.1-5 is exactly like Figure 3.2.1-4, except the optional check digit indicator (“E”) is present at the end of Section 1, and a check digit is therefore added in front of each data value in Section 2. The check digit is a numeric character from “0” to “9“, and takes the value of the unit digit of the ordered number of the data value counting from left to right.. The check digit immediately precedes the first character of each data value. However, note that the check digit immediately precedes the negative sign if a data value is negative.

CREX++

T000103 A000 D07004 E++

003 1075 21 31989 401 509 609 700 85845 9–00308 009962 110001 20019 303 4240 50013 6–073 7/// 8/// 93000 0015 107 202 3075 407 506 60120 738 820 910 00001 107 205 308 40120++

Figure 3.2.1-5: Sample CREX Report, highlighting the Data Section

3.2.1.5 Optional Section

Structure

Section 0		Section 1		Section 2	SECTION 3	Section 4
Group Number	Contents		Meaning
1	SUPP		The supplementary Optional Section is present
2 to p	Items for local use		Additional items for local use

Section 3 is optional. If it is present, it will contain additional items defined by each Centre for their own specific use. For example, a data processing Centre might add quality control information here. The sample CREX message in Figures 3.2.1-1 through 3.2.1-5 does not have an Optional Section.

3.2.1.6 End Section

Structure

Section 0		Section 1		Section 2	Section 3	SECTION 4
Group Number	Contents		Meaning
1	7777		End of a CREX Message

Note that Section 4 does not have a section terminator.

Example

The sample CREX message in Figure 3.2.1-1 is reproduced below with the End Section in bold.

CREX++

T000103 A000 D07004++

03 075 1 1989 01 09 09 00 5845 –00308 09962 10001 0019 03 240 0013 –073 /// ///

3000 015 07 02 075 07 06 0120 38 20 10 0001 07 05 08 0120++

Figure 3.2.1-6: Sample CREX Report, highlighting the End Section

3.2.2 CREX Descriptors

3.2.2.1 Fundamentals of CREX Descriptors

A CREX descriptor is a set of 6 alphanumeric characters. The 6 characters are divided into 3 parts, - F (1 letter), xx (2 digits), and yyy (3 digits, or a – (minus sign) followed by 2 digits for C02yyy data description operators for negative scales – more on this in the Chapter on CREX Table C). A space character always precedes a data descriptor. The F xx yyy descriptors in CREX Section 1 describe the form and content of the data in Section 2.

Schematically, a CREX descriptor can be visualized as follows:

xx

yyy

1 letter

2 digits

3 digits

F (1 letter) denotes the type of descriptor, and can be B, C, D, or R. The four possibilities for F have the following meanings:

F = B è Element descriptor, and refers to Table B entries

F = C è Operator descriptor, and refers to Table C entries

F = D è Sequence descriptor, and refers to Table D entries

F = R è Replication operator

Case 1: F = B

When F = B, the descriptor functions as an element descriptor, and defines a single data item by reference to CREX Table B entry B xx yyy.

Case 2: F = C

When F = C, the descriptor functions as an operator descriptor, and defines an operation by reference to CREX Table C entry C xx yyy. CREX Table C is provided to address cases when no option other than use of an operator descriptor to change the unit, scale or data width is possible. However, the most basic attribute of CREX is its human readability, and use of operator descriptors always reduce the readability of a CREX message. Therefore, operator descriptors from CREX Table C should only be used as a last resort.

Case 3: F = D

When F = D, the descriptor functions as a sequence descriptor, and defines a list of element descriptors, replication descriptors, operator descriptors, and/or sequence descriptors by reference to CREX Table D entry D xx yyy.

Case 4: F = R

If F = R, the descriptor functions as a replication descriptor. The 2 digits “xx” define the number of following descriptors to be repeated, and the 3 digits “yyy” give the number of times the sequence of “xx” descriptors is to be repeated. As with BUFR, if “yyy” = “000”, the descriptor defines a delayed replication. In this case, the number of replications of the sequence of data values is given in the Data Section (for example, the number of levels in a sounding). The number of replications is contained in a four-digit number in the data section corresponding to the position of the replication descriptor in the Data Description Section.

Descriptor Class or Category

When F = B or D, xx (2 digits) indicates the class or category of descriptor. With 2 digits, there are 100 possibilities, classes 00 to 99. Thus far, 29 classes have been defined.

Descriptor Entry in the Class or Category

When F = B or D, yyy (3 digits) indicates the entry within a class xx. With 3 digits, there are 1000 possibilities (000 too 999) within each of the 100 classes. There are a varying number of entries within each of the 29 classes that are currently defined.

3.2.2.2 Coordinate Descriptors

The descriptors in Classes 00 through 09 (with 03 and 09 at present reserved for future use) have a special meaning added to them over and above the specific data elements that they describe. They (or the data they represent) "remain in effect until superseded by redefinition" (see Regulation 95.3.5). By this is meant that the data in these classes serve as coordinates (in a general sense) for all the following observations. Once you encounter a B 04 004 descriptor (which describes the "hour"), one must assume that the hour (a time coordinate) applies to all the following observed parameters, until either another B 04 004 descriptor is encountered or you reach the end of the data subset.

Obviously the familiar coordinates (the two horizontal dimensions - Classes 05 and 06, the vertical dimension – Class 07, and time – Class 04) are in this sub-category of descriptors. However, some features that one might not think of as "coordinates", other than in a general sense, are in this sub-category as well. Forms of "identification" of the observing platform (block and station number, aircraft tail number, etc.) are "coordinates" in this sense, in that they most certainly apply to all the observed parameters taken from that platform and they "remain in effect until superseded by redefinition". The instrumentation that is used to take the measurements (Class 02) also falls in the same category - it applies to all the actual observed values of a particular parameter because all those observed values were measured with that particular instrument. However, some observations (like SYNOPs) involve several instruments, and therefore the instrumentation would need to be redefined a number of times in an individual SYNOP report. Nevertheless, the “coordinate” philosophy still applies for an individual observed quantity.

A source of confusion can arise by noting that some parameters (height and pressure, for example) appear twice in the Tables: in Class 07 (for values used as coordinates, or the independent variable) and again in Class 10 (for reported values, or the dependent variable). Which table descriptor is appropriate depends on the nature of the measurement that involves these parameters. A Radiosonde, which measures wind, temperature, and humidity (and geopotential height by calculation) as a function of pressure, would report the pressure values using Class 07 (the vertical coordinate or independent variable) and the other parameters from the non-coordinate classes (10 for geopotential, 11 for wind, 12 for temperature, and 13 for humidity). An aircraft radar altimeter, on the other hand, might calculate pressure (and use Class 10 to report the value) as a function of its height measurement (Class 07).

There is an exception to the "remain in effect until redefined" rule: when two identical descriptors from Classes 04 to 07 are placed back to back, that is to be interpreted as defining a range of coordinates. In this way a layer, an area, a volume, or a span of time can be defined as needed. If the same descriptor shows up later on in the message, then that appearance does indeed redefine that particular coordinate value even if the original coordinates defined a range. The others still remain in effect.

3.2.2.3 Increment Descriptors

Increment descriptors are those descriptors in Classes 04 – 07 with the word “increment” in the element name. As an example, consider Class 04 of Table B:

3.2.3.3        CREX Table C – Data Description Operators

Descriptors with F = 2 refer to CREX Table C - Data Description Operators. One of the crucial attributes of CREX is its human readability. Because use of Data Description Operators inevitably interferes with this human readability, the number of CREX Data Description Operators has been kept more limited and the functions they perform more simple than was the case with BUFR. Furthermore, it is strongly recommended that the available CREX Data Description Operators be used only as a last resort.

In all cases, the action specified by the CREX operator descriptor applies only to the data value corresponding to the following element descriptor. The original Table B value is back in force again for that element when subsequently referenced in the data description section until a new change is specified.

C 01 YYY: Data Width Replacement

When this operator is used, YYY characters (from 000 to 999) replace the specified Table B data width for the following descriptor.

C 02 YYY:  Scale Factor Replacement

When this operator is used, YYY (from –99 to 999) replaces the specified Table B Scale Factor for the following descriptor.  This is the case (for Scale Factors from – 99 to –01) referenced in Chapter 3.2.2.1 when the yyy part of the descriptor consists not of three digits but of a minus sign followed by 2 digits.

C 05 YYY:  Character Insertion

When this operator is used, YYY characters (from 001 to 999), including spaces, are inserted as a data field.

C 07 YYY:  Units Replacement

When this operator is used, the unit specified by code figure YYY in Common code table C-6 replaces the specified Table B unit for the following descriptor.  For example, YYY = 201 changes unit to knot and YYY = 741 changes unit to km h^-1.

C 60 YYY:  National Letters Insertion

When this operator is used, YYY national letters, including spaces, are inserted as a data field.  One should note, however, that only the characters from the International Telegraphic Alphabet No. 2 (ITA2) are likely to be transmitted accurately to all recipients.

3.2.3.4 Table D - Lists of Common Sequences

From a conceptual point of view, CREX Table D is not necessary. The Data Description Section of a CREX message can fully and completely describe the data using only element descriptors, operator descriptors, and the rules of description. However, such a means of defining the data would involve considerable overhead in terms of the length of the Data Description Section. Table D is a device to reduce this overhead.

Table D contains descriptors that describe additional descriptors. A single descriptor used in Section 2 with F = 3 is a pointer to a Table D entry. As with Table B, CREX Table D is organized into various classes, or categories, of sequences that have common characteristics. The xx value of the sequence descriptor identifies the category to which that particular sequence descriptor belongs. The yyy value of the sequence descriptor is the entry in that category for that particular sequence descriptor. There are currently defined 19 categories of common sequences in Table D:

                   CREX Table D list of common sequences

F XX           CATEGORY OF SEQUENCES

3 00            BUFR table entries sequences

3 01            Location and identification sequences

3 02            Meteorological sequences common to surface data

3 03            Meteorological sequences common to vertical sounding data

3 04            Meteorological sequences common to satellite observations

3 05            Reserved

3 06            Meteorological or oceanographic sequences common to oceanographic

                   observations

3 07            Surface report sequences (land)

3 08            Surface report sequences (sea)

3 09            Vertical sounding sequences (conventional data)

3 10            Vertical sounding sequences (satellite data)

3 11            Single level report sequences (conventional data)

3 12            Single level report sequences (satellite data)

3 13            Sequences common to image data

3 14            Reserved

3 15            Oceanographic report sequences

3 16            Synoptic feature sequences

3 18            Radiological report sequences

3 21            Radar report sequences

3 35            Monitoring Information

As with Table B, BUFR and CREX sequence descriptors share many similarities. Because of these, common sequences are not defined in both CREX Table D and BUFR Table D unless the conversion between both Table D entries is not completed by a simple replacement of the “F” part of each descriptor. If a new CREX Table D descriptor is not defined in BUFR Table D, it is assigned a number not used by any BUFR sequence. Similarly, a new BUFR Table D sequence is assigned a number not used by any CREX Table D sequence.

As a simple example of this relationship, consider the CREX Table D descriptor D 07 004 used in the sample CREX report in Figure 3.2.1-1. CREX Table D, defines D 07 004 as:

                                   +D 07 002

¦

   D 07 004 -      ----> ¦R 01 000

¦

                                   +D 02 005

However, D 02 005 and D 07 002 are not given in CREX Table D. Applying the above rule, since these are not given in CREX Table D, it follows that the sequences are in BUFR Table D with only the F part of the descriptors different. Thus, to find the composition of D 02 005 and D 07 002, one looks for 3 02 005 and 3 07 002 in BUFR Table D. This relationship between BUFR and CREX Tables A, B, and D helps maintain the close relationship between the BUFR and CREX code forms that was the intention of their developers. It also substantially eases the task of maintaining the Tables, a task that grows ever greater for the WMO Secretariat as the tables expand.

The sample CREX report illustrated in Figure 3.2.1-1 will be decomposed in detail in Chapter 3.2.4. Hopefully, that example will clarify any remaining questions about the relationship between the CREX and BUFR data description tables.

3.2.3.5 Code Tables and Flag Tables

Since some meteorological parameters are qualitative or semi-qualitative, they are best represented with reference to a code table or a flag table.

Code Tables

A code table lists a number of possibilities the parameter to which the code table applies can use. Only one of the possibilities can be chosen, and one of the possibilities is always “missing value”. Many of the code tables that have been included in the CREX specification are similar to existing WMO code tables for representing character data. However, there is not a one-to-one CREX code table relationship to the character code tables. The character Code Table 3333, Quadrant of the Globe, for example, has no meaning in CREX, as all points on the globe in CREX are completely expressed as latitude and longitude values.

Relationship Between CREX and BUFR Code Tables

CREX code tables have the same code figures as BUFR code tables. However, since CREX Code Tables are generally longer than the corresponding BUFR Code Tables (for example 99 entries rather than 63), the values in a CREX code table corresponding to code figures larger than the code figure for “Missing” in the BUFR Code Table are declared “Not Used” within the CREX table.

Flag tables

A flag table also lists a number of possibilities the parameter to which the flag table applies can use. However, in a flag table, any combination of the possibilities can be chosen. The flag table accomplishes this with a string of bits, where each bit indicates an item of significance. A bit set to 1 indicates an item is included, or is true, while a bit set to 0 indicates omission, or false. In any flag table, when all bits are set it is an indication of a missing value. However, in order to allow the possibility of all the possibilities being chosen and still allow for the possibility of a missing value, flag tables always have one more bit than the number of items of significance.

Relationship Between CREX and BUFR Flag Tables

CREX flag tables are the same as BUFR flag tables. However, CREX is character-oriented, not bit oriented, and this means a special procedure is needed to express the flag table values in a CREX message. Flag Table values in CREX are expressed using an octal representation. In the octal representation, a set of 3 bits is represented by a figure from 0 to 7, with zeroes added on the left when the number of flags is not a multiple of 3. Thus:

000 = 0 (no bit set)

                                                                001 = 1  (bit 3 set)

                                                                010 = 2  (bit 2 set)

                                                                011 = 3  (bits 2 and 3 set)

                                                                100 = 4  (bit 1 set)

                                                                101 = 5  (bits 1 and 3 set)

                                                                110 = 6  (bits 1 and 2 set)

                                                                111 = 7  (bits 1, 2, and 3 set)

For example, the seven flags “1100110” are first augmented by adding two zeroes on the left, which produces “001100110”.  Using the above table, this would be translated to the character string “146” (since 001 in bits 1, 2, 3 è 1, 100 in bits 4, 5, 6 è 4, and 110 in bits 7, 8, 9 è 6).  The character string “146” would then appear in the CREX message.  In CREX, missing value for flag table shall be indicated by a set of solidi “/” covering the data width.

3.2.3.6 Local Tables

Since a data processing center may need to represent data conforming to a local requirement, and this data may not be defined within Table B, specific areas of Table B and D are reserved for local use (Figure 3.2.3-4). These areas are defined as entries 192 to 255 inclusive of all classes. Centers defining classes or categories for local use should restrict their use to the range 48 to 63 inclusive. The Local portions of the Tables can be updated, changed, augmented, etc. at will by the local group concerned. No international notice is required or expected. Some of these local use entries may require local tables as well.

CLASS

FOR INTERNATIONAL USE

FOR LOCAL USE

RESERVED FOR FUTURE USE

FOR LOCAL USE

(IF NEEDED)

FOR LOCAL USE

(IF NEEDED)

FOR LOCAL USE

         0                                                                                    192                               255

ENTRY WITHIN CLASS

Figure 3.2.3-4. Table reservations

Local descriptor entries and local tables to support them present a problem for users of CREX. Whereas there is an operator descriptor within BUFR to note the presence of a local descriptor that allows a software program to skip that local descriptor, no corresponding feature is available in CREX. Because of the human readability factor, it is best to simply not disseminate CREX messages with local descriptors.

3.2.4 Decomposition of the Sample CREX Message

3.2.4.1 Decomposition of the Descriptor Sequence in the Sample CREX Message

Let us recall once again the sample CREX message given in figure 3.2.1-1:

CREX++

T000103 A000 D07004++

03 075 1 1989 01 09 09 00 5845 –00308 09962 10001 0019 03 240 0013 –073 /// ///

3000 015 07 02 075 07 06 0120 38 20 10 0001 07 05 08 0120++

Figure 3.2.4-1: Sample CREX Report

The sequence descriptor in the sample report, D07004, refers to an entry in CREX Table D. CREX Table D defines D 07 004 as:

                        +D 07 002

¦

D 07 004 -----> ¦R 01 000

¦

                        +D 02 005

Figure 3.2.4-2: Level 1 Decomposition of the Basic Sequence Descriptor from the Sample CREX Message

As we noted during the discussion of CREX Table D (Chapter 3.2.3.4), the definitions of D 02 005 and D 07 001 are not to be found in CREX Table D. Rather, they are found under descriptors 3 02 005 and 3 07 001 in BUFR Table D. Remembering that in this case the BUFR Table D definitions convert to CREX definitions by substituting F = D for F = 3 and F = B for F = 0, the second level decomposition of the basic sequence descriptor is:

                                           +D 01 032

¦

                +D 07 002 -----> ¦

                ¦                          ¦

                ¦                         +D 02 011

¦

D 07 004 ¦R 01 000

¦

                ¦                         +B 08 002

                +D 02 005 -----> ¦B 20 011

                                           ¦B 20 012

                                          +B 20 013

Figure 3.2.4-3: Level 2 Decomposition of the Basic Sequence Descriptor from the Sample CREX Message

In the second level decomposition, there are two more sequence descriptors that must be decomposed, D 01 032 and D 02 011. Once again, they are found in BUFR Table D, under 3 01 032 and 3 02 011. Again by substituting F = D for F = 3 and F = B for F = 0, the third level decomposition of the basic sequence descriptor becomes:

                                                  +D 01 001

¦

                                                  +B 02 001

¦

                                 +D 01 032¦D 01 011

                                 ¦                ¦

                                 ¦                ¦D 01 012

                                 ¦                ¦

                                 ¦                +D 01 024

                +D 07 002¦

                ¦                ¦

                ¦                ¦                +D 02 001

                ¦                ¦                ¦

                ¦                +3 02 011 ¦D 02 003

D 07 004 ¦                                  ¦

                ¦                                 +D 02 004

¦

                ¦R 01 000

¦

                ¦                                +B 08 002

                +D 02 005 -----------> ¦B 20 011

                                                  ¦B 20 012

                                                 +B 20 013

Figure 3.2.4-4: Level 3 Decomposition of the Basic Sequence Descriptor from the Sample CREX Message

In the third level decomposition, there are now 7 Table D descriptors to be decomposed, D 01 001, D 01 011, D 01 012, D 01 024, D 02 001, D 02 003, and D 02 004.  As before, they are all found in BUFR Table B, under 3 01 001, 3 01 011, 3 01 012, 3 01 024, 3 02 001, 3 02 003, and 3 02 004, respectively.  Substituting F = D for F = 3 and F = B for F = 0, the fourth level decomposition of the basic sequence descriptor becomes:

                                                                             +B 01 001

                                                  +D 01 001 -----> +B 01 002

¦

                                                  +B 02 001

¦

                                                  ¦                         +B 04 001

                                 +D 01 032| D 01 011 -----> ¦ B 04 002

                                 ¦                ¦                         +B 04 003

                                 ¦                ¦

                                 ¦                ¦                         +B 04 004

                                 ¦                ¦D 01 012 ------> ¦B 04 005

                                 ¦                ¦

                                 ¦                ¦                         +B 05 002

                                 |                +D 01 024 ----->¦ B 06 002

                                 ¦                                          +B 07 001

¦

                                 ¦                                          +B 10 004

                +D 07 002¦                +D 02 001 -----> ¦ B 10 051

                ¦                ¦                ¦                          ¦ B 10 061

                ¦                ¦                ¦                          +B 10 063

                ¦                ¦                ¦

                ¦                ¦                ¦                         +B 11 011

                ¦                ¦                ¦                          ¦B 11 012

                ¦                ¦                ¦                          ¦B 12 004

                ¦                ¦                ¦D 02 003-------->¦B 12 006

                ¦                ¦                ¦                          ¦B 13 003

                ¦                ¦                ¦                          ¦B 20 001

                ¦                +D 02 011¦                          ¦B 20 003

                ¦                                 ¦                          ¦B 20 004

                ¦                                 ¦                         +B 20 005

D 07 004 ¦                                 ¦

                ¦                                 ¦                         +B 20 010

                ¦                                 ¦                          ¦B 08 002

                ¦                                 ¦                          ¦B 20 011

                ¦                                 +D 02 004--------¦B 20 013

                ¦                                                            ¦B 20 012

                ¦                                                            ¦B 20 012

                ¦                                                           +B 20 012

¦

                ¦R 01 000

¦

                ¦                                                           +B 08 002

                +D 02 005 ---------------------------------> ¦B 20 011

                                                                             ¦B 20 012

                                                                            +B 20 013

Figure 3.2.4-5: Level 4 Decomposition of the Basic Sequence Descriptor from the Sample CREX Message

There are no longer any Table D descriptors in the level 4 decomposition of the basic descriptor sequence. Therefore, we now look up the meaning and data width (in characters) of each CREX Table B descriptor. Now we look under the CREX columns of the combined BUFR/CREX Table B and obtain the fifth (and, gratefully, final) level decomposition of the basic descriptor sequence from the sample CREX message:

SECTION 2                                                                                                           Data Width

                                                                                                                             (characters)

                                       +B 01 001 WMO Block Number --------- 2

                                                  +D 01 001 -----> +B 01 002 WMO Station Number ------- 3

¦

                                                  +B 02 001 ----------------------Type of Station  ---------------- 1

¦

                                                  ¦                         +B 04 001 Year ------------------------------- 4

                                 +D 01 032| D 01 011 -----> ¦ B 04 002 Month ----------------------------- 2

                                 ¦                ¦                         +B 04 003 Day -------------------------------- 2

                                 ¦                ¦

                                 ¦                ¦                         +B 04 004 Hour ------------------------------- 2

                                 ¦                ¦D 01 012 ------> ¦B 04 005 Minute ----------------------------- 2

                                 ¦                ¦

                                 ¦                ¦                         +B 05 002 Latitude (coarse accuracy) ---4

                                 |                +D 01 024 ----->¦ B 06 002 Longitude (coarse accuracy)  5

                                 ¦                                          +B 07 001 Height of Station ----------------5

¦

                                 ¦                                          +B 10 004 Station pressure  --------------- 5

                +D 07 002¦                +D 02 001 -----> ¦ B 10 051 Pressure (MSL) -----------------5

                ¦                ¦                ¦                          ¦ B 10 061 3-hour pressure change------ 4

                ¦                ¦                ¦                          +B 10 063 Characteristic of pressure

                |                |                |                                                                     tendency ---- 2

                ¦                ¦                ¦

                ¦                ¦                ¦                          +B 11 011---> Wind direction (10 m) ---- 3

                ¦                ¦                ¦                           ¦B 11 012---> Wind speed (10 m) --------4

                ¦                ¦                ¦                           ¦B 12 004---> Temperature (2 m) --------3

                ¦                ¦                ¦D 02 003 --------- ¦B 12 006---> Dew point (2 m) ----------- 3

                ¦                ¦                ¦                            ¦B 13 003---> Relative humidity --------- 3

                ¦                ¦                ¦                            ¦B 20 001---> Horizontal visibility ------- 4

                ¦                +D 02 011¦                            ¦B 20 003---> Present weather - ---------3

                ¦                                 ¦                            ¦B 20 004---> Past weather (1) ----------2

                ¦                                 ¦                           +B 20 005---> Past weather (2) ----------2

D 07 004 ¦                                 ¦

                ¦                                 ¦                           +B 20 010---> Cloud cover (total) ------- 3

                ¦                                 ¦                            ¦B 08 002---> Vertical significance ----- 2

                ¦                                 ¦                            ¦B 20 011---> Cloud amount ------------- 2

                ¦                                 +D 02 004----------¦B 20 013---> Height of base of cloud --4

                ¦                                                              ¦B 20 012---> Cloud type ------------------2

                ¦                                                              ¦B 20 012---> Cloud type ------------------2

                ¦                                                             +B 20 012---> Cloud type ------------------2

¦

                ¦R 01 000------------------------------------------->Delayed replication of 1 descriptor-4

¦

                ¦                                                             +B 08 002---> Vertical significance ----- 2

                +D 02 005 -----------------------------------> ¦B 20 011---> Cloud amount ------------- 2

                                                                               ¦B 20 012---> Cloud type ------------------2

                                                                              +B 20 013---> Height of base of cloud - 4

                                                                                    Total number of characters ---------106

Figure 3.2.4-6: Level 5 Decomposition of the Basic Sequence Descriptor from the Sample CREX Message

Note that there are 35 element descriptors and one replication descriptor in the final decomposition of the original sequence descriptor.  If the fully expanded form had been in the sample CREX message, the message would have been 245 characters longer (36 descriptors at 6 characters each (216) + 35 space separators (216 + 35 = 251) less 6 characters for the one sequence descriptor).  This is a quite substantial saving of space.  The cost in readability is the lack of a complete “map” to the contents of the Data Section.  However, by having the CREX Tables on hand, or prepared lists of complete expansions of the more common sequence descriptors, one can have readily available the contents of a CREX message using the single Table descriptor D 07 004.  In practice, as is the case with the current character code forms, frequent usage would probably soon lead to memorization of the contents of standard messages like this one.

3.2.4.2        Decomposition of the Data Section in the Sample CREX Message

The Data Section of the sample CREX message shown in Figure 3.2.4-1 is:

03 075 1 1989 01 09 09 00 5845 –00308 09962 10001 0019 03 240 0013 –073 /// ///

3000 015 07 02 075 07 06 0120 38 20 10 0001 07 05 08 0120++

Figure 3.2.4-7 Data Section from the Sample CREX Message

With the complete expansion of the descriptor sequence now available, we are ready to decompose the Data Section of the sample CREX message. There is a one-to-one correspondence between each element descriptor in Section 1 and each data value in Section 2. With the units, scale values, and data widths and units from CREX Table B, it is therefore very simple to interpret the CREX message Data Section. This one-to-one correspondence is one of the strengths of the CREX code form, for it facilitates human encoding and interpretation. The result of this exercise is given in Figure 3.2.4-8.

SECTION 2              CREX

Data                                                        Data

Descriptor    Element Name                  Width        Decoded value

                                                            (characters)

B 01 001 ---> WMO Block Number ------ 2            03

B 01 002 ---> WMO Station Number -----3            075          Station 03975

B 02 001 ---> Type of Station --------------1           1              Manned station

B 04 001 ---> Year --------------------------- 4          1989

B 04 002 ---> Month ------------------------- 2          01

B 04 003 ---> Day ---------------------------- 2         09            Date: January 9, 1989

B 04 004 ---> Hour --------------------------- 2         09

B 04 005 ---> Minute ------------------------ 2           00            Time: Hour 9, minute 0

B 05 002 ---> Latitude (coarse accuracy)4           5845        Latitude: 58.45 deg. East

B 06 002 ---> Longitude (coarse accur.)-5            -00308     Longitude: 3.08 deg. South

B 10 004 ---> Station pressure ------------5            09962      Station Pressure: 996.2 hPa

B 10 051 ---> Pressure (MSL) ------------ 5           10001      MSL Pressure: 1000.1 hPa

B 10 061 ---> 3-hour pressure change-- 4            0019        Pressure Change: + 1.9 hPa

B 10 063 ---> Characteristic of pressure

                         tendency ------------------- 2           03            Pressure tend.: higher than 3 hours ago.

B 11 011 ---> Wind direction (10 m) ----- 3            240

B 11 012 ---> Wind speed (10 m) -------- 4            0013        Wind: from 240^o at 13 m s^-1

B 12 004 ---> Temperature (2 m) -------- 3            -073         Temperature: – 7.3 ^oC

B 12 006 ---> Dew point (2 m) ------------ 3            ///             Dew point: missing

B 13 003 ---> Relative humidity -----------3            ///             Relative humidity: missing

B 20 001 ---> Horizontal visibility ---------4             3000        Visibility: 3,000 m

B 20 003 ---> Present weather ------------3            015          Precipitation within sight

B 20 004 ---> Past weather (1) ------------2           07            Past weather (1): Snow

B 20 005 ---> Past weather (2) ----------- 2            02            Past weather (2): > than ½ of sky covered

B 20 010 ---> Cloud cover (total) --------- 3           075          Sky: 75% covered

B 08 002 ---> Vertical significance ------- 2            07            Low cloud

B 20 011 ---> Cloud amount --------------- 2           06            Cloud cover: 6/8

B 20 013 ---> Height of base of cloud --- 4            0120        Cloud base: 1200 m (120 decameters)

B 20 012 ---> Cloud type ------------------- 2           38            Clouds type: Cu and Sc

B 20 012 ---> Cloud type ------------------- 2           20            Clouds: no C_M

B 20 012 ---> Cloud type ------------------- 2           10            Clouds: no C_H

R 01 000 ---> Delayed replication of

                        1 descriptor----------------- 4           0001        Delayed replication of the following 1 descriptor (D 02 005) 1 time

1 sequence only:

B 08 002 ---> Vertical significance ------ 2             07            Low cloud

B 20 011 ---> Cloud amount -------------- 2            05            Cloud cover 5/8

B 20 012 ---> Cloud type ------------------ 2            08            Cloud type: Cu

B 20 013 ---> Height of base of cloud -- 4              0120        Cloud base: 1200 m (120 decameters)

Figure 3.2.4-8          Decomposition of the Data Section from the Sample CREX Message

Note in this report the dew point is missing and is therefore encoded as three solidi (“///”), since the data width for dew point is three characters. Since the dew point is missing, the relative humidity is also missing. This is likewise indicated by the three solidi, since the data width for relative humidity is also three characters. The final cloud group is replicated only once. Had the delayed replication number been greater than one, that group would have been repeated a corresponding number of times.

* Descriptor 0 01 027 should be used instead of 0 01 026 to encode this element.

Descriptor	Interpretation
0 04 004	Sets the value of the hour at one increment LESS than the "starting" value.
0 04 014	Sets the value of the increment in hours and increments the hour
1 XX 000	Set up (delayed) replication of "next" XX descriptors
0 31 001	Replication count (not included in the span of replication XX)
	XX descriptors to be replicated

Code Figure
0	Reserved
1	Reserved
2	Maximum value
3	Minimum value
4	Mean value
5	Median value
6	Modal value
7	Mean absolute error
8	Reserved
9	Best estimate of standard deviation (N-1)
10	Standard deviation (N)
11	Harmonic mean
12	Root-mean-square vector error
13-31	Reserved
32	Vector mean
33-62	Reserved for local use
63	Missing value

Code Figure
0	Data not suspect
1	Data slightly suspect
2	Data highly suspect
3	Data considered unfit for use
4-6	Reserved
7	Quality information not given

Table of Contents

Initiation of the Descriptor Sequence

Construction of a Data Present Bit Map

Specification of the Type of First Order Statistic

Specification of the Statistical Values

Initiation of the Descriptor Sequence

Construction of a Data Present Bit Map

Specification of the Type of First Order Statistic

Difference Statistics

Specification of the Statistical Values

Initiation of the Descriptor Sequence

Construction of a Data Present Bit Map

Specification of the Type of Quality Information

Initiation of the Descriptor Sequence

Construction of a Data Present Bit Map

Initiation of the Descriptor Sequence

Construction of a Data Present Bit Map

Code Figure	Meaning
0	Surface data – land
1	Surface data – sea
2	Vertical soundings (other than satellite)
3	Vertical soundings (satellite)
4	Single level upper-air data (other than satellite)
5	Single level upper-air data (satellite)
6	Radar data
7	Synoptic features
8	Physical/chemical constituents
9	Dispersal and transport
10	Radiological data
11	BUFR tables, complete replacement or update
12	Surface data (satellite)
13 – 19	Reserved
20	Status information
21	Radiances (satellite measured)
22 – 30	Reserved
31	Oceanographic data
32 – 100	Reserved
101	Image data
102 – 239	Reserved
240 – 254	For experimental use
255	Indicator for local use, with sub-category

Class Number	Class Name	Comments
00	BUFR table entries
01	Identification	Identifies origin and type of data
02	Instrumentation	Defines instrument types used
03	Reserved
04	Location (time)	Defines time and time derivatives
05	Location (horizontal – 1)	Defines geographical position, including horizontal derivatives, in association with class 06 (first dimension of horizontal space)
06	Location (horizontal – 2)	Defines geographical position, including horizontal derivatives, in association with class 05 (second dimension of horizontal space)
07	Location (vertical)	Defines height, altitude, pressure level, including vertical derivatives of position
08	Significance qualifiers	Defines special character of data
09	Reserved
10	Vertical elements and pressure	Height, altitude, pressure and derivatives observed or measured, not defined as a vertical location
11	Wind and turbulence	Wind speed, direction, etc.
12	Temperature
13	Hydrographic and hydrological elements	Humidity, rainfall, snowfall, etc.
14	Radiation and radiance
15	Physical/chemical constituents
19	Synoptic features
20	Observed phenomena	Defines present/past weather, special phenomena, etc.
21	Radar data
22	Oceanographic elements
23	Dispersal and transport
24	Radiological elements
25	Processing information
26	Non-coordinate location (time)	Defines time and time derivatives that are not coordinates
27	Non-coordinate location (horizontal – 1)	Defines geographical positions, in conjunction with class 28, that are not coordinates
28	Non-coordinate location (horizontal – 2)	Defines geographical positions, in conjunction with class 27, that are not coordinates
29	Map data
30	Image
31	Data description operator qualifiers	Elements used in conjunction with data description operators
33	Quality information
35	Data monitoring