The `__transient__` attribute. What is it? Could it be thrown away?

[Peque]

It seems it is not a standard Python dunder attribute. For what I have seen it is used to exclude some attributes from __dict__ before serialization.

Why is this used instead of __getstate__?

It seems in save_inst() you are actually trying to use __getstate__ first if it exists, and only if it does not, then you look for __transient__. However, in save_reduce() you are always directly trying to look for this attribute (if protocol version is >= 2). Is this necessary? Couldn't __getstate__ tried to be used as well first?

Cross reference: irmen/Pyro4#179

[Peque]

Also, it seems that you assume state is a dictionary (i.e.: you execute state.copy()), but actually, if you define the __setstate__, then there is no requirement for the state object to be a dictionary (it could be a tuple for example). See the Python documentation on __setstate__. Wouldn't this be a cloudpickle bug?

I am curious also why adding this special case when it seems there is a standard approach (implementing get/set state). Code gets more complicated and can break things. Is there any particular project/module that you want to support with this? If so, wouldn't it be better to ask them to use the standard approach instead?

[Peque]

By the way, it seems this __transient__ support is here since the beginning, when @ygravrand commited 7aebb7e.

By that time it seems Python 2.6 and 2.7 were supported. Is there any chance that the __transient__ attribute was required to support 2.6 and maybe now it could be thrown away?

[irmen]

Simple example code that triggers the copy error in cloudpickle. The object being pickled has to have the transient dunder attribute (or in Pyro's case, getattr of it returns True). That triggers a piece of code in cloudpickle that tries to copy the pickle state, and thus fails:

import pickle
import cloudpickle


class Test(object):
    def __init__(self, a, b):
        self.a = a
        self.b = b
        
    def __getstate__(self):
        return self.a, self.b  # tuple
    
    def __setstate__(self, state):
        self.a, self.b = state  # tuple unpack
        
    def __transient__(self):
        return []



obj = Test(42,99)


# pickle
ser = pickle.dumps(obj)
x = pickle.loads(ser)
print("pickle:", x)

# cloudpickle
ser = cloudpickle.dumps(obj)    # crashes.
x = cloudpickle.loads(ser)
print("cloudpickle:", x)

maybe this should be in a separate issue 🤔

[Peque]

@rgbkrk / @pitrou Friendly ping. 😇

[pitrou]

I don't know anything about this, sorry.

[rgbkrk]

Yeah I don't know either. If it came from the original commits, then it's in the annals of PiCloud the company (which no longer exists).

[rgbkrk]

Which means that reverse engineering why it could have been done is the only option now. The code is flexible, so if you need to make changes then feel free.

Making a quick search on github leads to a lot of different code using __transient__ on their classes: https://github.com/search?l=Python&q=__transient__&type=Code&utf8=%E2%9C%93

[Peque]

Proposed pull-request: #110.

[irmen]

This pull-request will fix both the unnecessary lookup of an unexisting attribute on a pyro proxy and the crash when trying to cloudpickle a Pyro proxy object because it no longer tries to copy() the state. I wonder if there's other code out there that similarly triggers this error, but I think the chance is fairly small