The recently released Allegro CL version 10.1 has a number of enhancements relating to characters and to external formats. These enhancements include:
- Support for the GB 18030 external format. The GB 18030 external format is a Chinese government standard which defines the required language and character support necessary for software in China. See this Wikipedia article for more information on GB 18030. See See Basic External-Format Types in iacl.htm for a list of external formats. Support is provided for ensuring writing and reading characters with the :gb18030 external format are valid. See the variable *gb-18030-transcoding-error-action* for more information.
- Better support for UTF-16 characters. 16-bit character Allegro CL (that is, the alisp, mlisp, and allegro executables, but not alisp8, mlisp8, and allegro8) stores characters in UTF-16. New operators facilitate the creation of UTF-16 octets and characters. These are particularly useful when using characters outside the ASCII range. The operators include code-to-utf16-char, compose-octet-array, and compose-string.
- Better support for BOMs (Byte Order Marks) in Unicode. A new function sniff-for-unicode applied to a stream returns information on BOMs. The value of the variable *unicode-version* is a string containing the version number for the Unicode Character Database used to build the Allegro CL character name table and collation rules. And new unicode external formats handle BOMs. See The unicode and fat External-Format Types; the unicode BOM in iacl.htm.
Coming soon via a patch is support for a strict UTF-8 external format and tools for ensuring all bytes are valid UTF-8 (rejecting, for example, invalid multi-byte sequences), Keep an eye on the patch list and the release-notes.htm for this upcoming patch.
https://franz.com/support/tech_corner/
The recently released Allegro CL version 10.1 has a number of enhancements relating to characters and to external formats. These enhancements include:
- Support for the GB 18030 external format. The GB 18030 external format is a Chinese government standard which defines the required language and character support necessary for software in China. See this Wikipedia article for more information on GB 18030. See See Basic External-Format Types in iacl.htm for a list of external formats. Support is provided for ensuring writing and reading characters with the :gb18030 external format are valid. See the variable *gb-18030-transcoding-error-action* for more information.
- Better support for UTF-16 characters. 16-bit character Allegro CL (that is, the alisp, mlisp, and allegro executables, but not alisp8, mlisp8, and allegro8) stores characters in UTF-16. New operators facilitate the creation of UTF-16 octets and characters. These are particularly useful when using characters outside the ASCII range. The operators include code-to-utf16-char, compose-octet-array, and compose-string.
- Better support for BOMs (Byte Order Marks) in Unicode. A new function sniff-for-unicode applied to a stream returns information on BOMs. The value of the variable *unicode-version* is a string containing the version number for the Unicode Character Database used to build the Allegro CL character name table and collation rules. And new unicode external formats handle BOMs. See The unicode and fat External-Format Types; the unicode BOM in iacl.htm.
Coming soon via a patch is support for a strict UTF-8 external format and tools for ensuring all bytes are valid UTF-8 (rejecting, for example, invalid multi-byte sequences), Keep an eye on the patch list and the release-notes.htm for this upcoming patch.
https://franz.com/support/tech_corner/
