What would you like to happen?
AFAIK the Java SDK doesn't have an easy utility to print/log the outputs of a PCollection. I've always had to hack something together like:
rows.apply(MapElements.into(strings()).via(row -> {
System.out.println("row: " + row);
return "";
}));Would be nice to have an easy utility that can be used like:
The Python SDK has something like this:
|
class LogElements(PTransform): |
|
""" |
|
PTransform for printing the elements of a PCollection. |
|
|
|
Args: |
|
label (str): (optional) A custom label for the transform. |
|
prefix (str): (optional) A prefix string to prepend to each logged element. |
|
with_timestamp (bool): (optional) Whether to include element's timestamp. |
|
with_window (bool): (optional) Whether to include element's window. |
|
level: (optional) The logging level for the output (e.g. `logging.DEBUG`, |
|
`logging.INFO`, `logging.WARNING`, `logging.ERROR`). If not specified, |
|
the log is printed to stdout. |
|
with_pane_info (bool): (optional) Whether to include element's pane info. |
|
use_epoch_time (bool): (optional) Whether to display epoch timestamps. |
|
""" |
Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
Issue Components
What would you like to happen?
AFAIK the Java SDK doesn't have an easy utility to print/log the outputs of a PCollection. I've always had to hack something together like:
Would be nice to have an easy utility that can be used like:
The Python SDK has something like this:
beam/sdks/python/apache_beam/transforms/util.py
Lines 2183 to 2197 in 26c6ec7
Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
Issue Components