4.5 Byte Strings

Bytes and Byte Strings in The Racket Guide introduces byte strings.

A byte string is a fixed-length array of bytes. A byte is an exact integer between 0 and 255 inclusive.

A byte string can be mutable or immutable. When an immutable byte string is provided to a procedure like bytes-set!, the exn:fail:contract exception is raised. Byte-string constants generated by the default reader (see Reading Strings) are immutable, and they are interned in read-syntax mode. Use immutable? to check whether a byte string is immutable.

Two byte strings are equal? when they have the same length and contain the same sequence of bytes.

A byte string can be used as a single-valued sequence (see Sequences). The bytes of the string serve as elements of the sequence. See also in-bytes.

See Reading Strings for information on reading byte strings and Printing Strings for information on printing byte strings.

See also: immutable?.

4.5.1 Byte String Constructors, Selectors, and Mutators

4.5.2 Byte String Comparisons

Changed in version 7.0.0.13 of package base: Allow one argument, in addition to allowing two or more.

Changed in version 7.0.0.13 of package base: Allow one argument, in addition to allowing two or more.

Changed in version 7.0.0.13 of package base: Allow one argument, in addition to allowing two or more.

4.5.3 Bytes to/from Characters, Decoding and Encoding

4.5.4 Bytes to Bytes Encoding Conversion

Certain encoding combinations are always available:

• (bytes-open-converter "UTF-8" "UTF-8") — the identity conversion, except that encoding errors in the input lead to a decoding failure.

• (bytes-open-converter "UTF-8-permissive" "UTF-8") — the identity conversion, except that any input byte that is not part of a valid encoding sequence is effectively replaced by the UTF-8 encoding sequence for #\uFFFD. (This handling of invalid sequences is consistent with the interpretation of port bytes streams into characters; see Ports.)

• (bytes-open-converter "" "UTF-8") — converts from the current locale’s default encoding (see Encodings and Locales) to UTF-8.

• (bytes-open-converter "UTF-8" "") — converts from UTF-8 to the current locale’s default encoding (see Encodings and Locales).

• (bytes-open-converter "platform-UTF-8" "platform-UTF-16") — converts UTF-8 to UTF-16 on Unix and Mac OS, where each UTF-16 code unit is a sequence of two bytes ordered by the current platform’s endianness. On Windows, the input can include encodings that are not valid UTF-8, but which naturally extend the UTF-8 encoding to support unpaired surrogate code units, and the output is a sequence of UTF-16 code units (as little-endian byte pairs), potentially including unpaired surrogates.

• (bytes-open-converter "platform-UTF-8-permissive" "platform-UTF-16") — like (bytes-open-converter "platform-UTF-8" "platform-UTF-16"), but an input byte that is not part of a valid UTF-8 encoding sequence (or valid for the unpaired-surrogate extension on Windows) is effectively replaced with (char->integer #\?).

• (bytes-open-converter "platform-UTF-16" "platform-UTF-8") — converts UTF-16 (bytes ordered by the current platform’s endianness) to UTF-8 on Unix and Mac OS. On Windows, the input can include UTF-16 code units that are unpaired surrogates, and the corresponding output includes an encoding of each surrogate in a natural extension of UTF-8. On Unix and Mac OS, surrogates are assumed to be paired: a pair of bytes with the bits #xD800 starts a surrogate pair, and the #x03FF bits are used from the pair and following pair (independent of the value of the #xDC00 bits). On all platforms, performance may be poor when decoding from an odd offset within an input byte string.

A newly opened byte converter is registered with the current custodian (see Custodians), so that the converter is closed when the custodian is shut down. A converter is not registered with a custodian (and does not need to be closed) if it is one of the guaranteed combinations not involving "" on Unix, or if it is any of the guaranteed combinations (including "") on Windows and Mac OS.

In the Racket software distributions for Windows, a suitable "iconv.dll" is included with "libmzschVERS.dll".

The set of available encodings and combinations varies by platform, depending on the iconv library that is installed; the from-name and to-name arguments are passed on to iconv_open. On Windows, "iconv.dll" or "libiconv.dll" must be in the same directory as "libmzschVERS.dll" (where VERS is a version number), in the user’s path, in the system directory, or in the current executable’s directory at run time, and the DLL must either supply _errno or link to "msvcrt.dll" for _errno; otherwise, only the guaranteed combinations are available.

Use bytes-convert with the result to convert byte strings.

If dest-bstr is not #f, the converted bytes are written into dest-bstr from dest-start-pos to dest-end-pos. If dest-bstr is #f, then a newly allocated byte string holds the conversion results, and if dest-end-pos is not #f, the size of the result byte string is no more than (- dest-end-pos dest-start-pos).

The result of bytes-convert is three values:

• result-bstr or dest-wrote-amt — a byte string if dest-bstr is #f or not provided, or the number of bytes written into dest-bstr otherwise.

• src-read-amt — the number of bytes successfully converted from src-bstr.

• 'complete, 'continues, 'aborts, or 'error — indicates how conversion terminated:

  • 'complete: The entire input was processed, and src-read-amt will be equal to (- src-end-pos src-start-pos).

  • 'continues: Conversion stopped due to the limit on the result size or the space in dest-bstr; in this case, fewer than (- dest-end-pos dest-start-pos) bytes may be returned if more space is needed to process the next complete encoding sequence in src-bstr.

  • 'aborts: The input stopped part-way through an encoding sequence, and more input bytes are necessary to continue. For example, if the last byte of input is 195 for a "UTF-8-permissive" decoding, the result is 'aborts, because another byte is needed to determine how to use the 195 byte.

  • 'error: The bytes starting at (+ src-start-pos src-read-amt) bytes in src-bstr do not form a legal encoding sequence. This result is never produced for some encodings, where all byte sequences are valid encodings. For example, since "UTF-8-permissive" handles an invalid UTF-8 sequence by dropping characters or generating “?,” every byte sequence is effectively valid.

Applying a converter accumulates state in the converter (even when the third result of bytes-convert is 'complete). This state can affect both further processing of input and further generation of output, but only for conversions that involve “shift sequences” to change modes within a stream. To terminate an input sequence and reset the converter, use bytes-convert-end.

The result of bytes-convert-end is two values:

• result-bstr or dest-wrote-amt — a byte string if dest-bstr is #f or not provided, or the number of bytes written into dest-bstr otherwise.

• 'complete or 'continues — indicates whether conversion completed. If 'complete, then an entire ending sequence was produced. If 'continues, then the conversion could not complete due to the limit on the result size or the space in dest-bstr, and the first result is either an empty byte string or 0.

4.5.5 Additional Byte String Functions