Skip to content

Replace pickle with json#1502

Merged
miguelgrinberg merged 2 commits intomainfrom
remove-pickle
Sep 30, 2025
Merged

Replace pickle with json#1502
miguelgrinberg merged 2 commits intomainfrom
remove-pickle

Conversation

@miguelgrinberg
Copy link
Copy Markdown
Owner

@miguelgrinberg miguelgrinberg commented Sep 30, 2025

It has been reported (credit: Ali Raza, BlueRock) that the use of the pickle package from the Python standard library to encode payloads that are passed between Socket.IO processes over a message queue (in a multi-server and/or external process configuration) can be exploited to create a remote code execution.

For this exploit to be viable, the attacker needs to connect to the message queue directly, and push a malicious payload to it. Given that the message queue is an internal component of the server deployment that is not accessible from the public internet, the only reason for concern would be if the queue is configured to listen publicly by mistake.

To confirm that you have a secure deployment, make sure that:

  1. Your message queue is only listening on an internal network interface (i.e localhost) or VPC. Never deploy the message queue for Socket.IO to the public internet.
  2. For added protection, configure your production message queue with authentication and encryption. Please consult the documentation of your message queue for instructions. The message queue credentials can then be added to the connection URL passed to python-socketio in the configuration. And of course, avoid committing a URL with credentials to source control.

I have added a section to the documentation on secure deployment of the message queue with the above recommendations.

In addition to the above, and considering that pickle does not provide any benefits over other serialization formats, I have decided to remove the use of pickle completely, so that any possibility of an attack, though unlikely, is eliminated. Version 5.14.0 uses JSON as the only communication format for message queue payloads (in previous versions both JSON and Pickle were supported, but Pickle was used by default).

The only side effect of this change is that a mix deployment that includes servers from before and after this change may fail to communicate, because the older versions will attempt to push Pickle packets to the queue and the newer servers do not understand that format anymore. If all servers are upgraded to 5.14 or newer there shouldn't be any problems.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Sep 30, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (f61e0be) to head (333349c).

Additional details and impacted files
@@            Coverage Diff            @@
##              main     #1502   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           31        31           
  Lines         2535      2531    -4     
  Branches       432       432           
=========================================
- Hits          2535      2531    -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@miguelgrinberg miguelgrinberg merged commit 53f6be0 into main Sep 30, 2025
44 checks passed
@miguelgrinberg miguelgrinberg deleted the remove-pickle branch September 30, 2025 19:22
@locus-x64
Copy link
Copy Markdown

Detailed blog post on the vulnerability and its excitability
https://www.bluerock.io/post/cve-2025-61765-bluerock-discovers-critical-rce-in-socket-io-ecosystem

@judilsteve
Copy link
Copy Markdown

judilsteve commented Nov 10, 2025

I think this has broken the ability to .emit() bytestring payloads

EDIT: Was able to fix the "object not JSON-serialisable" error by replacing data=(bytestring, ...) with data=[bytesring, ...] (Packet.data_is_binary probably needs to be updated to check tuples as well as lists), but now get stuck on the following:

  File "/redacted/.venv/lib/python3.13/site-packages/socketio/async_manager.py", line 37, in emit
    pkt = self.server.packet_class(
          ^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'packet_class'

For reference I am trying to use AsyncRedisManager standalone:
https://python-socketio.readthedocs.io/en/stable/server.html#emitting-from-external-processes

So nothing ever calls manager.set_server()

Was able to manually work around this by manually doing it with a stub server matching that of my main server instance in the other process:

manager.set_server(AsyncServer(async_mode="asgi"))

@miguelgrinberg
Copy link
Copy Markdown
Owner Author

@judilsteve Thanks. These are problems that started after I dropped the use of Pickle, which handled all these serialization issues transparently. I'll make sure bytesstrings are accepted as binary, and will also look at the server issue.

@miguelgrinberg
Copy link
Copy Markdown
Owner Author

@judilsteve I do not see the crash you reported above. Can you share the code that you are using that causes this crash on line 37 of async_manager.py? Thanks.

@shmcgough
Copy link
Copy Markdown

shmcgough commented Dec 7, 2025

I've also run into an issue since the switch to json was made. I'm using AsyncRedisManager and setting a tuple type as the data value no longer turns them into individual arguments for the (javascript) client, instead the first argument is an array of the arguments and the other arguments are undefined.

I can manually change the package code to go from pickle to json and see that it works one way and not the other.

Client handler

socket.on('order_update', function (tag: string, order: Order)

Prior to 5.14, both parameters are defined. After, tag is instead an array with two entries [tag, order]

Server side

# where tag is a string and order is a dict
await sio.emit("order_update", (tag, order), to=f"theplace")

@miguelgrinberg
Copy link
Copy Markdown
Owner Author

@shmcgough okay, yes, I need to figure out how to pass tuples over json. This change has really been a nightmare. There are lots of things that people were doing that are much harder to do without pickle!

@miguelgrinberg
Copy link
Copy Markdown
Owner Author

@shmcgough Would you be able to install the main branch of this repo and test your application with it? I think the support for multiple arguments via tuples is now restored. I'll cut a release in a few days, so let me know if you experience any remaining issues. Thanks!

@shmcgough
Copy link
Copy Markdown

@shmcgough Would you be able to install the main branch of this repo and test your application with it? I think the support for multiple arguments via tuples is now restored. I'll cut a release in a few days, so let me know if you experience any remaining issues. Thanks!

I should be able to test it tonight, I'll update you then. Thanks for your work on this.

@shmcgough
Copy link
Copy Markdown

@miguelgrinberg I was able to test the main branch and can confirm it works! Thanks again.

@miguelgrinberg
Copy link
Copy Markdown
Owner Author

@shmcgough v5.15.1 is now available with this fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants