Gevent "deadlock" [JIRA: CLIENTS-56]

[pigmej]

Hey,

from gevent import monkey
monkey.patch_all()
import gevent

import riak
import time


def save(x):
    client = riak.RiakClient(host='192.168.1.103', port=10017, transport_class=riak.RiakPbcTransport)

    bucket = client.bucket('test2')

    data = bucket.new('test%d' % x, data={'x': x})
    data.store()


def get(x):
    client = riak.RiakClient(host='192.168.1.103', port=10017, transport_class=riak.RiakPbcTransport)

    bucket = client.bucket('test2')

    return bucket.get('test%d' % x)


l = gevent.spawn(save, 1)

gevent.joinall([l])

print 'done'

l = gevent.spawn(get, 1)

gevent.joinall([l])

By using this simple script gevent + riak will deadlock on data.store().

(yes I know that this code itself doesn't make sense at all)

[evanmcc]

Hi, Jędrzej.

Could you see if this still happens with master? Otherwise it might take a
little while for one of us to try it.

[pigmej]

Hi Evan,

from gevent import monkey
monkey.patch_all()
import gevent

import riak
import time


def save(x):
    client = riak.RiakClient(host='192.168.1.103', pb_port=10017)
    bucket = client.bucket('test2')
    data = bucket.new('test%d' % x, data={'x': x})
    data.store()


def get(x):
    client = riak.RiakClient(host='192.168.1.103', pb_port=10017)
    bucket = client.bucket('test2')
    return bucket.get('test%d' % x)


l = gevent.spawn(save, 1)
gevent.joinall([l])

print 'done'

l = gevent.spawn(get, 1)
gevent.joinall([l])

This works fine on gevent master + riak-python-client master. How stable is current riak-python master ?

[pigmej]

Ok, works "fine" just with pb_port, http_port causes exactly the same deadlock as before for me.

//it seems that it sometimes works with HTTP.

[shuhaowu]

Side discussion: are we planning to add concurrent requests to the Python client?

[seancribbs]

@pigmej Without knowledge of what gevent's monkeypatches really do, I can't guarantee that the connection pool is still safe. I suspect that the source of the problem is that gevent reaches deep in and mucks with httplib, can you confirm?

@shuhaowu I don't know what you mean, #225?

[shuhaowu]

@seancribbs Yes that one.

[seancribbs]

@shuhaowu Yes.

[seancribbs]

I'm loathe to address this immediately, as it would involve rewriting the majority of our networking code. There is no way to write blocking/imperative code and have it seamlessly work with continuation-passing style.

Let's revisit this issue in a later release cycle.

[pigmej]

This issue might be related to #381 to which I made pull request today ( #382 ) (only if previous client was using pbc)