Python 2 pdb: a statement behaves differently when run at the pdb prompt

author

2018-05-01 16:51

Problem This question may turn out to be really stupid, but here it is. The following statement triggers an exception on a particular email message:

  File "/Users/me/tools/maildir-deduplicate/maildir_deduplicate/mail.py", line 104, in body_lines
_, _, body = self.message.as_string().partition("\n\n")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd7 in position 621: ordinal not in range(128)

If I run under PDB and manually test it at the prompt, no exception thrown and body correctly set.

> /Users/me/tools/maildir-deduplicate/maildir_deduplicate/mail.py(105)body_lines()
-> _, _, body = self.message.as_string().partition("\n\n")
(Pdb) _, _, body = self.message.as_string().partition("\n\n")

But if I hit next line, it still throws the exception:

(Pdb) n
UnicodeDecodeError: UnicodeD...ge(128)')
> /Users/me/tools/maildir-deduplicate/maildir_deduplicate/mail.py(105)body_lines()
-> _, _, body = self.message.as_string().partition("\n\n")

If I break the statement, the exception is thrown on the partition() call.

  File "/Users/me/tools/maildir-deduplicate/maildir_deduplicate/mail.py", line 106, in body_lines
body = self.message.as_string()
_, _, body = body.partition("\n\n")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd7 in position 621: ordinal not in range(128)

Same story running under pdb: exception will be thrown if I hit n, but not if I enter _, _, body = body.partition("\n\n") at the prompt.

Any ideas what might be causing this?

Solution by Mark Tolonen: I suspect you have a from __future__ import unicode_literals in your code:

Test code:

	`from __future__ import unicode_literals`
	`body = b'abc\n\ndef\xd7ghi'`
	`_,_,body = body.partition('\n\n')`

When run directly (no pdb):

Traceback (most recent call last):
  File "C:\test.py", line 4, in <module>
    _,_,body = body.partition('\n\n')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd7 in position 8: ordinal not in range(128)

When stepped through in pdb it gets the UnicodeDecode error:

> c:\test.py(2)<module>()
-> from __future__ import unicode_literals
(Pdb) n
> c:\test.py(3)<module>()
-> body = b'abc\n\ndef\xd7ghi'
(Pdb) n
> c:\test.py(4)<module>()
-> _,_,body = body.partition('\n\n')
(Pdb) n
UnicodeDecodeError: UnicodeD...ge(128)')      <<<<<<<<<<<<<<<<
> c:\test.py(4)<module>()
-> _,_,body = body.partition('\n\n')

When manually executing the line it works because pdb isn’t under the __future__ import, so '\n\n' is a byte string:

> c:\test.py(2)<module>()
-> from __future__ import unicode_literals
(Pdb) n
> c:\test.py(3)<module>()
-> body = b'abc\n\ndef\xd7ghi'
(Pdb) n
> c:\test.py(4)<module>()
-> _,_,body = body.partition('\n\n')
(Pdb) _,_,body = body.partition('\n\n')   <<<<<<<<<<<<< manual
(Pdb) body                                <<<<<<<<<<<<< worked!
'def\xd7ghi'
(Pdb) n
UnicodeDecodeError: UnicodeD...ge(128)')  <<<<<<<<<<<<< failed!
> c:\test.py(4)<module>()
-> _,_,body = body.partition('\n\n')

Comments