文章来自《Python cookbook》.

翻译仅仅是为了个人学习,其它商业版权纠纷与此无关!

-- 0.706 [2004-09-27 18:34:42]

Converting Between Unicode and Plain Strings 在Unicode和普通字符串之间转换

Credit: David Ascher, Paul Prescod

问题 Problem

You need to deal with data that doesn't fit in the ASCII character set.

你需要处理不适合用ASCII字符集表示的数据.

解决 Solution

Unicode strings can be encoded in plain strings in a variety of ways, according to whichever encoding you choose:

Unicode字符串可以用多种方式编码为普通字符串, 依照你所选择的编码(encoding):

   1 #将Unicode转换成普通的Python字符串:"编码(encode)"
   2 unicodestring = u"Hello world"
   3 utf8string = unicodestring.encode("utf-8")
   4 asciistring = unicodestring.encode("ascii")
   5 isostring = unicodestring.encode("ISO-8859-1")
   6 utf16string = unicodestring.encode("utf-16")
   7 
   8 
   9 #将普通的Python字符串转换成Unicode: "解码(decode)"
  10 plainstring1 = unicode(utf8string, "utf-8")
  11 plainstring2 = unicode(asciistring, "ascii")
  12 plainstring3 = unicode(isostring, "ISO-8859-1")
  13 plainstring4 = unicode(utf16string, "utf-16")
  14 
  15 assert plainstring1==plainstring2==plainstring3==plainstring4

讨论 Discussion

If you find yourself dealing with text that contains non-ASCII characters, you have to learn about Unicode梬hat it is, how it works, and how Python uses it.

如果你发现自己正在处理包含非ASCII码字符的文本, 你必须学习Unicode,关于它是什么,如何工作,而且Python如何使用它。

Unicode is a big topic.Luckily, you don't need to know everything about Unicode to be able to solve real-world problems with it: a few basic bits of knowledge are enough.First, you must understand the difference between bytes and characters.In older, ASCII-centric languages and environments, bytes and characters are treated as the same thing.Since a byte can hold up to 256 values, these environments are limited to 256 characters.Unicode, on the other hand, has tens of thousands of characters.That means that each Unicode character takes more than one byte, so you need to make the distinction between characters and bytes.

Unicode是一个大的主题。幸运地,你并不需要知道关于Unicode码的每件事,就能够用它解决真实世界的问题: 一些基本知识就够了。首先,你得了解在字节和字符之间的不同。原先,在以ASCII码为中心的语言和环境中,字节和字符被当做相同的事物。由于一个字节只能有256个值,这些环境就受限为只支持 256个字符。Unicode码,另一方面,有数万个字符,那意谓着每个Unicode字符占用多个字节,因此,你需要在字符和字节之间作出区别。

Standard Python strings are really byte strings, and a Python character is really a byte.Other terms for the standard Python type are "8-bit string" and "plain string.",In this recipe we will call them byte strings, to remind you of their byte-orientedness.

标准的Python字符串确实是字节字符串,而且一个Python字符真的是一个字节。换个术语,标准的Python字符串类型的是 "8位字符串(8-bit string)"和"普通字符串(plain string)". 在这一份配方中我们把它们称作是字节串(byte strings), 并记住它们是基于字节的。

Conversely, a Python Unicode character is an abstract object big enough to hold the character, analogous to Python's long integers.You don't have to worry about the internal representation;the representation of Unicode characters becomes an issue only when you are trying to send them to some byte-oriented function, such as the write method for files or the send method for network sockets.At that point, you must choose how to represent the characters as bytes.Converting from Unicode to a byte string is called encoding the string.Similarly, when you load Unicode strings from a file, socket, or other byte-oriented object, you need to decode the strings from bytes to characters.

相反地,一个Python Unicode码字符是一个大到足够支持(Unicode)字符的一个抽象对象, 类似于Python中的长整数。 你不必要为内在的表示担忧; 只有当你正在尝试把它们传递给给一些基于字节的函数的时候,Unicode字符的表示变成一个议题, 比如文件的write方法或网络套接字的send 方法。那时,你必须要选择该如何表示这些(Unicode)字符为字节。从Unicode码到字节串的转换被叫做编码。同样地,当你从文件,套接字或其他的基于字节的对象中装入一个Unicode字符串的时候,你需要把字节串解码为(Unicode)字符。

There are many ways of converting Unicode objects to byte strings, each of which is called an encoding.For a variety of historical, political, and technical reasons, there is no one "right" encoding.Every encoding has a case-insensitive name, and that name is passed to the decode method as a parameter. Here are a few you should know about:

将Unicode码对象转换成字节串有许多方法, 每个被称为一个编码(encoding)。由于多种历史的,政治上的,和技术上的原因,没有一个 "正确的"编码。每个编码有一个大小写无关的名字,而且那一个名字被作为一个叁数传给解码方法。这里是一些你应该知道的:

If you want to be able to encode all Unicode characters, you probably want to use UTF-8.You will probably need to deal with the other encodings only when you are handed data in those encodings created by some other application.

如果你想要能够编码所有的Unicode码字符,你或许想要使用UTF-8。只有当你需要处理那些由其他应用产生的其它编码的数据时,你或许才需要处理其他编码。

参考 See Also

Unicode is a huge topic, but a recommended book is Unicode:A Primer, by Tony Graham (Hungry Minds, Inc.)梔etails are available at http://www.menteith.com/unicode/primer/.

Unicode是一个极大的主题,但是关于Unicode,一本被推荐的书:A Primer, by Tony Graham (Hungry Minds, Inc.),其细节可从http:// www.menteith.com/icode/primer/获得.