Sunday, 25 October 2009

Of Python3's map

Hello, it's been a long time since the last post, but here we go.

So what is wrong with map? Its semantics have secretly changed between Python 2.x and Python 3.x, without even 2to3 knowing. While the old, non-lazy, map of 2.x stopped when the last sequence was depleted, while the lazy one of 3.x (like 2.x itertools.imap) stops when the first sequence is depleted. The reasonability of this change can be quarreled about, but that is not really the topic here.

The real topic is that not even 2to3 is aware of this subtle change. An easy fix for this would be to translate map(fun, a, b, ...) to list(map(fun, *zip(*itertools.zip_longest(a, b, ...)))) (itertools.izip_longest in 2.6). Another problem with 2to3 and map is that None cannot be given as the function argument with 3.x, but 2to3 is unaware of this. Thus it should translate map(None, ..) to map(lambda *a: a, ..), which it doesn't.

So here is what 2to3 does. It minds neither of the semantic changes.
--- map_test.py (original)
+++ map_test.py (refactored)
@@ -1,1 +1,1 @@
-map(None, [1, 2, 3], [1, 2, 3, 4])
+list(map(None, [1, 2, 3], [1, 2, 3, 4]))
Python 2.x:
>>> map(None, [1, 2, 3], [1, 2, 3, 4])
[(1, 1), (2, 2), (3, 3), (None, 4)]
Python 3.x:
>>> list(map(None, [1, 2, 3], [1, 2, 3, 4]))
Traceback (most recent call last):
File "", line 1, in
TypeError: 'NoneType' object is not callable
>>> list(map(lambda *a: a, [1, 2, 3], [1, 2, 3, 4]))
[(1, 1), (2, 2), (3, 3)]
>>> list(map(lambda *a: a,
... *zip(*itertools.zip_longest([1, 2, 3], [1, 2, 3, 4]))))
[(1, 1), (2, 2), (3, 3), (None, 4)]



This shows that the proposed fixes work.

Issue created by birkenfeld after discussing the issue with me

No comments: