First write a utf-16 file with its signature:
>>> f1 = open('utf16.txt', 'w', encoding='utf-16')
>>> f1.write('0123456789')
>>> f1.close()
Then read it twice:
>>> f2 = open('utf16.txt', 'r', encoding='utf-16')
>>> print('read1', ascii(f2.read()))
read1 '0123456789'
>>> f2.seek(0)
0
>>> print('read2', ascii(f2.read()))
read2 '\ufeff0123456789'
The second read returns the BOM!
This is because the zero in seek(0) is a "cookie" which contains both the position
and the decoder state. Unfortunately, state=0 means 'endianness has been determined:
native order'.
maybe a suggestion: handle seek(0) as a special value which calls decoder.reset().
The patch implement this idea. |