Skip to content

Work around issue in dill for wordcount example save_main_session#7710

Merged
charlesccychen merged 1 commit into
apache:masterfrom
charlesccychen:fix-wordcount-py3
Feb 4, 2019
Merged

Work around issue in dill for wordcount example save_main_session#7710
charlesccychen merged 1 commit into
apache:masterfrom
charlesccychen:fix-wordcount-py3

Conversation

@charlesccychen

Copy link
Copy Markdown
Contributor

This change works around a bug in the dill library (uqfoundation/dill#300), where classes fail to unpickle in the main session if its super() is referenced. This change modifies the wordcount example to not use such a reference.


Follow this checklist to help us incorporate your contribution quickly and easily:

  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

It will help us expedite review of your Pull Request if you tag someone (e.g. @username) to look at it.

Post-Commit Tests Status (on master branch)

Lang SDK Apex Dataflow Flink Gearpump Samza Spark
Go Build Status --- --- --- --- --- ---
Java Build Status Build Status Build Status Build Status
Build Status
Build Status
Build Status Build Status Build Status
Python Build Status --- Build Status
Build Status
Build Status --- --- ---

@charlesccychen

Copy link
Copy Markdown
Contributor Author

R: @aaltay @tvalentyn @markflyhigh
CC: @robertwb

@charlesccychen

Copy link
Copy Markdown
Contributor Author

Run Python PostCommit

@charlesccychen

Copy link
Copy Markdown
Contributor Author

Run Python PostCommit

@robertwb

robertwb commented Feb 4, 2019

Copy link
Copy Markdown
Contributor

Nice. Is it worth noting that DoFn doesn't actually have an init method, so this is not a material change?

@charlesccychen

Copy link
Copy Markdown
Contributor Author

Run Python PostCommit

@charlesccychen

Copy link
Copy Markdown
Contributor Author

@robertwb Thanks, I incorporated your suggestion.

@robertwb robertwb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, I might just remove this whole thing, so it doesn't distract from the example. (Often the very first example people see.) You're call.

@charlesccychen

Copy link
Copy Markdown
Contributor Author

Thanks, I removed it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants