The recently released Allegro CL version 10.1 has a number of enhancements relating to characters and to external formats. These enhancements include:
Support for the GB 18030 external format. The GB 18030 external format is a Chinese government standard which defines the required language and character support necessary for software in China. See this Wikipedia article for more information on GB 18030. See See Basic External-Format Types in iacl.htm for a list of external formats. Support is provided for ensuring writing and reading characters with the :gb18030 external format are valid. See the variable *gb-18030-transcoding-error-action* for more information.
Better support for UTF-16 characters. 16-bit character Allegro CL (that is, the alisp, mlisp, and allegro executables, but not alisp8, mlisp8, and allegro8) stores characters in UTF-16. New operators facilitate the creation of UTF-16 octets and characters. These are particularly useful when using characters outside the ASCII range. The operators include code-to-utf16-char, compose-octet-array, and compose-string.
Coming soon via a patch is support for a strict UTF-8 external format and tools for ensuring all bytes are valid UTF-8 (rejecting, for example, invalid multi-byte sequences), Keep an eye on the patch list and the release-notes.htm for this upcoming patch.