[Python] How can I elide a function wrapper from the traceback in Python-3?

The issue

The Phantom Menace


Say i wrote a function decorator which takes the function, and wraps it in another function like so:

# File example-1.py
from functools import wraps

def decorator(func):
# Do something
@wraps(func)
def wrapper(*args, **kwargs):
# Do something
return func(*args, **kwargs)
# Do something
# Do something
return wrapper


Now lets suppose the function I'm decorating raises an exception:

@decorator
def foo():
raise Exception('test')


The result of running foo() will print out the following traceback (In any Python version):

Traceback (most recent call last):
File "./example-1.py", line 20, in <module>
foo()
File "./example-1.py", line 11, in wrapper
return func(*args, **kwargs)
File "./example-1.py", line 18, in foo
raise Exception('test')
Exception: test

Attack of the Clones


OK, now i look at my traceback and i see it goes through the wrapper function. What if I wrapped the function multiple times(presumably with a slightly more sophisticated decorator object which receives arguments in its constructor)? What if I use this decorator often in my code(I use it for logging, or profiling, or whatever)?

Traceback (most recent call last):
File "./example-1.py", line 20, in <module>
foo()
File "./example-1.py", line 11, in wrapper
return func(*args, **kwargs)
File "./example-1.py", line 11, in wrapper
return func(*args, **kwargs)
File "./example-1.py", line 11, in wrapper
return func(*args, **kwargs)
File "./example-1.py", line 11, in wrapper
return func(*args, **kwargs)
File "./example-1.py", line 18, in foo
raise Exception('test')
Exception: test


I don't want it "polluting" my traceback when i know from the function definition that the wrapper is there, and i don't want it showing up multiple times when the code snippet it displays is the unhelpful return func(*args, **kwargs)

Python 2

Revenge of the Sith


In Python-2, as this answer to a different question points out, the following trick does the job:

# In file example-2.py

def decorator(func):
# Do something
@wraps(func)
def wrapper(*args, **kwargs):
# Do something
info = None
try:
return func(*args, **kwargs)
except:
info = sys.exc_info()
raise info[0], info[1], info[2].tb_next
finally:
# Break the cyclical reference created by the traceback object
del info
# Do something
# Do something
return wrapper


By directly wrapping the call to the wrapped function with this idiom in the same block as the function I want to elide from the traceback, I effectively remove the current layer from the traceback and let the exception keep propagating. Every time the stack unwinding goes through this function, it removes itself from the traceback so this solution works perfectly:

Traceback (most recent call last):
File "./example-2.py", line 28, in <module>
foo()
File "./example-2.py", line 26, in foo
raise Exception('test')
Exception: test


(Note however that you can not encapsulate this idiom in another function, since as soon the stack will unwind from that function back into wrapper, it will still be added to the traceback)

Python 3

A New Hope


Now that we have this covered, lets move along to Python-3. Python-3 introduced this new syntax:

raise_stmt ::= "raise" [expression ["from" expression]]


which allows chaining exceptions using the __cause__ attribute of the new exception. This feature is uninteresting to us, since it modifies the exception, not the traceback. Our goal is to be a completely transparent wrapper, as far as visibility goes, so this won't do.

Alternatively, we can try the following syntax, which promises to do what we want (code example taken from the python documentation):

raise Exception("foo occurred").with_traceback(tracebackobj)


Using this syntax we may try something like this:

# In file example-3
def decorator(func):
# Do something
@wraps(func)
def wrapper(*args, **kwargs):
# Do something
info = None
try:
return func(*args, **kwargs)
except:
info = sys.exc_info()
raise info[1].with_traceback(info[2].tb_next)
finally:
# Break the cyclical reference created by the traceback object
del info
# Do something
# Do something
return wrapper

The Empire Strikes Back


But, unfortunately, this does not do what we want:

Traceback (most recent call last):
File "./example-3.py", line 29, in <module>
foo()
File "./example-3.py", line 17, in wrapper
raise info[1].with_traceback(info[2].tb_next)
File "./example-3.py", line 27, in foo
raise Exception('test')
Exception: test


As you can see, the line executing the raise statement shows up in the traceback. This seems to come from the fact that while the Python-2 syntax sets the traceback from the third argument to raise as the function is being unwound, and thus it is not added to the traceback chain(as explained in the docs under Data Model), the Python-3 syntax on the other hand changes the traceback on the Exception object as an expression inside the functions context, and then passes it to the raise statement which adds the new location in code to the traceback chain (the explanation of this is very similar in Python-3).

A workaround that comes to mind is avoiding the "raise" [ expression ] form of the statement, and instead use the clean raise statement to let the exception propagate as usual but modify the exception objects __traceback__ attribute manually:

# File example-4
def decorator(func):
# Do something
@wraps(func)
def wrapper(*args, **kwargs):
# Do something
info = None
try:
return func(*args, **kwargs)
except:
info = sys.exc_info()
info[1].__traceback__ = info[2].tb_next
raise
finally:
# Break the cyclical reference created by the traceback object
del info
# Do something
# Do something
return wrapper


But this doesn't work at all!

Traceback (most recent call last):
File "./example-4.py", line 30, in <module>
foo()
File "./example-4.py", line 14, in wrapper
return func(*args, **kwargs)
File "./example-4.py", line 28, in foo
raise Exception('test')
Exception: test

Return of the Jedi(?)


So, what else can i do? It seems like using the "traditional" way of doing this just won't work because of the change in syntax, and I wouldn't want to start messing with the traceback printing mechanism (using the traceback module) at the project level. This is because it'll be hard if not impossible to implement in an extensible which won't be disruptive to any other package that tries to change the traceback, print the traceback in a custom format at the top level, or otherwise do anything else related to the issue.

Also, can someone explain why in fact the last technique fails completely?

(I tried these examples on python 2.6, 2.7, 3.4, 3.6)

EDIT: After thinking about it for a while, in my opinion the python 3 behavior makes more sense, to the point that the python 2 behavior almost looks like a design bug, but I still think that there should be a way to do this kinda stuff.

Continue reading...