First sketch of XGBoost integration with Merlin Dataset by oliverholworthy · Pull Request #433 · NVIDIA-Merlin/models

oliverholworthy · 2022-05-12T14:19:12Z

First draft of XGBoost integration with a Merlin Dataset.

Relates to #132 (issue)

Goals ⚽

Make it easy to use a Merlin Dataset with XGBoost

Implementation Details 🚧

Some details to figure out:

Should the target objective Tag (e.g. Tags.REGRESSION) be specified in constructor or automatically figured out from schema.
- changed so that the XGBoost objective is passed in and we figure out the target tag from that.
What should the default params be for XGBoost and how should customisation work with this wrapper?
- Allowing all params supported by train function to be passed in to the constructor
find ranking objectives examples and see if this can be extended to work with learning to rank style models.
- added support for rank:* metrics. Defaults to grouping by the User ID column. Otherwise, the group column(s) can be passed into the fit method. The data will be sorted when the merlin dataset is converted to the DMatrix.
what do we do with list-like columns in dataset.
- ignoring list columns and logging a warning when found
check that categorical features don't need any special treatment with XGBoost here.
- As of XGBoost 1.6, the feature is experimental and has limited features. For the category dtype.

Testing Details 🔍

Example Usage

from merlin.datasets.entertainment import get_movielens
train, valid = get_movielens(variant="ml-1m")

from merlin.models.xgb import XGBoost
model = XGBoost(
    objective="binary:logistic",
    eval_metric=["logloss", "auc"],
    tree_method="gpu_hist",
)
model.fit(train)
model.evaluate(valid)
# => {'logloss': 0.56554, 'auc': 0.767453}

github-actions · 2022-05-12T14:35:28Z

Documentation preview

https://nvidia-merlin.github.io/models/review/pr-433

nvidia-merlin-bot · 2022-05-12T14:56:57Z

Click to view CI Results

GitHub pull request #433 of commit ddac89bf6b7dcda71ae078c3f83f6f7a9ecbd3de, no merge conflicts.
Running as SYSTEM
Setting status of ddac89bf6b7dcda71ae078c3f83f6f7a9ecbd3de to PENDING with url https://10.20.13.93:8080/job/merlin_models/272/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/433/*:refs/remotes/origin/pr/433/* # timeout=10
 > git rev-parse ddac89bf6b7dcda71ae078c3f83f6f7a9ecbd3de^{commit} # timeout=10
Checking out Revision ddac89bf6b7dcda71ae078c3f83f6f7a9ecbd3de (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f ddac89bf6b7dcda71ae078c3f83f6f7a9ecbd3de # timeout=10
Commit message: "First sketch of XGBoost integration with NVTabular schema"
 > git rev-list --no-walk 7f4e0dc6d361c1172f8dabcfa7672c16eb3f0475 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins4487940980277396325.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.1)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: traitlets>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.2.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.5.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (22.3.0)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
WARNING: You are using pip version 22.0.4; however, version 22.1 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 442 items / 2 skipped
tests/config/test_schema.py ....                                         [  0%]

tests/datasets/test_advertising.py .s                                    [  1%]

tests/datasets/test_ecommerce.py ..sss                                   [  2%]

tests/datasets/test_entertainment.py ....sss                             [  4%]

tests/datasets/test_social.py .                                          [  4%]

tests/datasets/test_synthetic.py .....                                   [  5%]

tests/tf/test_core.py ................                                   [  9%]

tests/tf/test_dataset.py ...............                                 [ 12%]

tests/tf/test_public_api.py .                                            [ 12%]

tests/tf/blocks/test_cross.py ............                               [ 15%]

tests/tf/blocks/test_dlrm.py ........                                    [ 17%]

tests/tf/blocks/test_interactions.py .                                   [ 17%]

tests/tf/blocks/test_mlp.py .............................                [ 23%]

tests/tf/blocks/core/test_aggregation.py .........                       [ 26%]

tests/tf/blocks/core/test_base.py .                                      [ 26%]

tests/tf/blocks/core/test_index.py ...                                   [ 26%]

tests/tf/blocks/core/test_masking.py .......                             [ 28%]

tests/tf/blocks/core/test_transformations.py ...........                 [ 30%]

tests/tf/blocks/retrieval/test_matrix_factorization.py ..                [ 31%]

tests/tf/blocks/retrieval/test_two_tower.py ...........                  [ 33%]

tests/tf/examples/test_01_getting_started.py .                           [ 34%]

tests/tf/examples/test_02_dataschema.py .                                [ 34%]

tests/tf/examples/test_03_exploring_different_models.py .                [ 34%]

tests/tf/examples/test_04_export_ranking_models.py .                     [ 34%]

tests/tf/examples/test_05_export_retrieval_model.py .                    [ 35%]

tests/tf/examples/test_06_advanced_own_architecture.py .                 [ 35%]

tests/tf/features/test_continuous.py .....                               [ 36%]

tests/tf/features/test_embedding.py ..........s.                         [ 39%]

tests/tf/features/test_tabular.py .......                                [ 40%]

tests/tf/integration/test_ci_01_getting_started.py .                     [ 40%]

tests/tf/integration/test_ci_02_dataschema.py .                          [ 41%]

tests/tf/integration/test_ci_03_exploring_different_models.py F          [ 41%]

tests/tf/integration/test_ci_04_export_ranking_models.py F               [ 41%]

tests/tf/integration/test_ci_05_export_retrieval_model.py F              [ 41%]

tests/tf/integration/test_ci_06_advanced_own_architecture.py .           [ 42%]

tests/tf/layers/test_queue.py ..............                             [ 45%]

tests/tf/losses/test_losses.py .......................                   [ 50%]

tests/tf/metrics/test_metrics_popularity.py .....                        [ 51%]

tests/tf/metrics/test_metrics_ranking.py .................               [ 55%]

tests/tf/models/test_benchmark.py .                                      [ 55%]

tests/tf/models/test_ranking.py ......................                   [ 60%]

tests/tf/models/test_retrieval.py ..................                     [ 64%]

tests/tf/prediction_tasks/test_classification.py ..                      [ 65%]

tests/tf/prediction_tasks/test_multi_task.py .......                     [ 66%]

tests/tf/prediction_tasks/test_next_item.py ......................       [ 71%]

tests/tf/prediction_tasks/test_regression.py ..                          [ 72%]

tests/tf/prediction_tasks/test_sampling.py ....................          [ 76%]

tests/tf/utils/test_batch.py ....                                        [ 77%]

tests/torch/test_dataset.py .........                                    [ 79%]

tests/torch/test_public_api.py .                                         [ 79%]

tests/torch/block/test_base.py ....                                      [ 80%]

tests/torch/block/test_mlp.py .                                          [ 80%]

tests/torch/features/test_continuous.py ..                               [ 81%]

tests/torch/features/test_embedding.py ..............                    [ 84%]

tests/torch/features/test_tabular.py ....                                [ 85%]

tests/torch/model/test_head.py ............                              [ 88%]

tests/torch/model/test_model.py ..                                       [ 88%]

tests/torch/tabular/test_aggregation.py ........                         [ 90%]

tests/torch/tabular/test_tabular.py ...                                  [ 91%]

tests/torch/tabular/test_transformations.py .......                      [ 92%]

tests/utils/test_schema_utils.py ................................        [100%]
=================================== FAILURES ===================================

__________________________________ test_func ___________________________________
self = <testbook.client.TestbookNotebookClient object at 0x7f6df419c580>

cell = {'cell_type': 'markdown', 'id': '3d964063', 'metadata': {}, 'source': 'NVTabular exported the schema file of our proce... other words, Merlin Models relies on the schema object to automatically build all necessary input and output layers.'}

kwargs = {}, cell_indexes = [0, 1, 2, 3, 4, 5, ...]

executed_cells = [{'cell_type': 'code', 'execution_count': 2, 'id': '5f49a48e', 'metadata': {'execution': {'iopub.status.busy': '2022-0...}, {'cell_type': 'markdown', 'id': '6f3b6bc9', 'metadata': {}, 'source': '## Feature Engineering with NVTabular'}, ...]

idx = 14
def execute_cell(self, cell, **kwargs) -> Union[Dict, List[Dict]]:
    """
    Executes a cell or list of cells
    """
    if isinstance(cell, slice):
        start, stop = self._cell_index(cell.start), self._cell_index(cell.stop)
        if cell.step is not None:
            raise TestbookError('testbook does not support step argument')

        cell = range(start, stop + 1)
    elif isinstance(cell, str) or isinstance(cell, int):
        cell = [cell]

    cell_indexes = cell

    if all(isinstance(x, str) for x in cell):
        cell_indexes = [self._cell_index(tag) for tag in cell]

    executed_cells = []
    for idx in cell_indexes:
        try:


          cell = super().execute_cell(self.nb['cells'][idx], idx, **kwargs)


/usr/local/lib/python3.8/dist-packages/testbook/client.py:133:

args = (<testbook.client.TestbookNotebookClient object at 0x7f6df419c580>, {'cell_type': 'code', 'execution_count': 7, 'id': ...join(output_path, 'valid', '*.parquet'), part_size="500MB")\n\n# define schema object\nschema = train.schema'}, 14)

kwargs = {}
def wrapped(*args, **kwargs):


  return just_run(coro(*args, **kwargs))


../../../.local/lib/python3.8/site-packages/nbclient/util.py:84:

coro = <coroutine object NotebookClient.async_execute_cell at 0x7f6dee94c540>
def just_run(coro: Awaitable) -> Any:
    """Make the coroutine run, even if there is an event loop running (using nest_asyncio)"""
    # original from vaex/asyncio.py
    loop = asyncio._get_running_loop()
    if loop is None:
        had_running_loop = False
        try:
            loop = asyncio.get_event_loop()
        except RuntimeError:
            # we can still get 'There is no current event loop in ...'
            loop = asyncio.new_event_loop()
            asyncio.set_event_loop(loop)
    else:
        had_running_loop = True
    if had_running_loop:
        # if there is a running loop, we patch using nest_asyncio
        # to have reentrant event loops
        check_ipython()
        import nest_asyncio

        nest_asyncio.apply()
        check_patch_tornado()


  return loop.run_until_complete(coro)


../../../.local/lib/python3.8/site-packages/nbclient/util.py:62:

self = <_UnixSelectorEventLoop running=False closed=False debug=False>

future = <Task finished name='Task-1072' coro=<NotebookClient.async_execute_cell() done, defined at /var/jenkins_home/.local/li...rn\x1b[39;00m dataset\n\n\x1b[0;31mIndexError\x1b[0m: list index out of range\nIndexError: list index out of range\n')>
def run_until_complete(self, future):
    """Run until the Future is done.

    If the argument is a coroutine, it is wrapped in a Task.

    WARNING: It would be disastrous to call run_until_complete()
    with the same coroutine twice -- it would wrap it in two
    different Tasks and that can't be good.

    Return the Future's result, or raise its exception.
    """
    self._check_closed()
    self._check_running()

    new_task = not futures.isfuture(future)
    future = tasks.ensure_future(future, loop=self)
    if new_task:
        # An exception is raised if the future didn't complete, so there
        # is no need to log the "destroy pending task" message
        future._log_destroy_pending = False

    future.add_done_callback(_run_until_complete_cb)
    try:
        self.run_forever()
    except:
        if new_task and future.done() and not future.cancelled():
            # The coroutine raised a BaseException. Consume the exception
            # to not log a warning, the caller doesn't have access to the
            # local task.
            future.exception()
        raise
    finally:
        future.remove_done_callback(_run_until_complete_cb)
    if not future.done():
        raise RuntimeError('Event loop stopped before Future completed.')


  return future.result()


/usr/lib/python3.8/asyncio/base_events.py:616:

self = <testbook.client.TestbookNotebookClient object at 0x7f6df419c580>

cell = {'cell_type': 'code', 'execution_count': 7, 'id': 'a0f0cc5f', 'metadata': {'execution': {'iopub.status.busy': '2022-05...path.join(output_path, 'valid', '*.parquet'), part_size="500MB")\n\n# define schema object\nschema = train.schema'}

cell_index = 14, execution_count = None, store_history = True
async def async_execute_cell(
    self,
    cell: NotebookNode,
    cell_index: int,
    execution_count: t.Optional[int] = None,
    store_history: bool = True,
) -> NotebookNode:
    """
    Executes a single code cell.

    To execute all cells see :meth:`execute`.

    Parameters
    ----------
    cell : nbformat.NotebookNode
        The cell which is currently being processed.
    cell_index : int
        The position of the cell within the notebook object.
    execution_count : int
        The execution count to be assigned to the cell (default: Use kernel response)
    store_history : bool
        Determines if history should be stored in the kernel (default: False).
        Specific to ipython kernels, which can store command histories.

    Returns
    -------
    output : dict
        The execution output payload (or None for no output).

    Raises
    ------
    CellExecutionError
        If execution failed and should raise an exception, this will be raised
        with defaults about the failure.

    Returns
    -------
    cell : NotebookNode
        The cell which was just processed.
    """
    assert self.kc is not None

    await run_hook(self.on_cell_start, cell=cell, cell_index=cell_index)

    if cell.cell_type != 'code' or not cell.source.strip():
        self.log.debug("Skipping non-executing cell %s", cell_index)
        return cell

    if self.skip_cells_with_tag in cell.metadata.get("tags", []):
        self.log.debug("Skipping tagged cell %s", cell_index)
        return cell

    if self.record_timing:  # clear execution metadata prior to execution
        cell['metadata']['execution'] = {}

    self.log.debug("Executing cell:\n%s", cell.source)

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors or "raises-exception" in cell.metadata.get("tags", [])
    )

    await run_hook(self.on_cell_execute, cell=cell, cell_index=cell_index)
    parent_msg_id = await ensure_async(
        self.kc.execute(
            cell.source, store_history=store_history, stop_on_error=not cell_allows_errors
        )
    )
    await run_hook(self.on_cell_complete, cell=cell, cell_index=cell_index)
    # We launched a code cell to execute
    self.code_cells_executed += 1
    exec_timeout = self._get_timeout(cell)

    cell.outputs = []
    self.clear_before_next_output = False

    task_poll_kernel_alive = asyncio.ensure_future(self._async_poll_kernel_alive())
    task_poll_output_msg = asyncio.ensure_future(
        self._async_poll_output_msg(parent_msg_id, cell, cell_index)
    )
    self.task_poll_for_reply = asyncio.ensure_future(
        self._async_poll_for_reply(
            parent_msg_id, cell, exec_timeout, task_poll_output_msg, task_poll_kernel_alive
        )
    )
    try:
        exec_reply = await self.task_poll_for_reply
    except asyncio.CancelledError:
        # can only be cancelled by task_poll_kernel_alive when the kernel is dead
        task_poll_output_msg.cancel()
        raise DeadKernelError("Kernel died")
    except Exception as e:
        # Best effort to cancel request if it hasn't been resolved
        try:
            # Check if the task_poll_output is doing the raising for us
            if not isinstance(e, CellControlSignal):
                task_poll_output_msg.cancel()
        finally:
            raise

    if execution_count:
        cell['execution_count'] = execution_count


  await self._check_raise_for_error(cell, cell_index, exec_reply)


../../../.local/lib/python3.8/site-packages/nbclient/client.py:965:

self = <testbook.client.TestbookNotebookClient object at 0x7f6df419c580>

cell = {'cell_type': 'code', 'execution_count': 7, 'id': 'a0f0cc5f', 'metadata': {'execution': {'iopub.status.busy': '2022-05...path.join(output_path, 'valid', '*.parquet'), part_size="500MB")\n\n# define schema object\nschema = train.schema'}

cell_index = 14

exec_reply = {'buffers': [], 'content': {'ename': 'IndexError', 'engine_info': {'engine_id': -1, 'engine_uuid': '09fa77a2-f645-4546...e, 'engine': '09fa77a2-f645-4546-a20d-7ed6333eda1e', 'started': '2022-05-12T14:48:11.251365Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
    self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:

    if exec_reply is None:
        return None

    exec_reply_content = exec_reply['content']
    if exec_reply_content['status'] != 'error':
        return None

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors
        or exec_reply_content.get('ename') in self.allow_error_names
        or "raises-exception" in cell.metadata.get("tags", [])
    )
    await run_hook(self.on_cell_error, cell=cell, cell_index=cell_index)
    if not cell_allows_errors:


      raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)


E E           ------------------
 E E E
 E           # define schema object
 E           schema = train.schema
 E           ------------------
 E
 E E E E E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E           �[1;32m E E
 E E           �[1;32m E           �[1;32m E           �[1;32m E E           �[1;32m E
 E E

nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
 train = Dataset(os.path.join(output_path, 'train', '.parquet'), part_size="500MB")
 valid = Dataset(os.path.join(output_path, 'valid', '.parquet'), part_size="500MB")
 �[0;31m---------------------------------------------------------------------------�[0m
 �[0;31mIndexError�[0m                                Traceback (most recent call last)
 Input �[0;32mIn [7]�[0m, in �[0;36m<cell line: 1>�[0;34m()�[0m
 �[0;32m----> 1�[0m train �[38;5;241m=�[39m �[43mDataset�[49m�[43m(�[49m�[43mos�[49m�[38;5;241;43m.�[39;49m�[43mpath�[49m�[38;5;241;43m.�[39;49m�[43mjoin�[49m�[43m(�[49m�[43moutput_path�[49m�[43m,�[49m�[43m �[49m�[38;5;124;43m'�[39;49m�[38;5;124;43mtrain�[39;49m�[38;5;124;43m'�[39;49m�[43m,�[49m�[43m �[49m�[38;5;124;43m'�[39;49m�[38;5;124;43m*.parquet�[39;49m�[38;5;124;43m'�[39;49m�[43m)�[49m�[43m,�[49m�[43m �[49m�[43mpart_size�[49m�[38;5;241;43m=�[39;49m�[38;5;124;43m"�[39;49m�[38;5;124;43m500MB�[39;49m�[38;5;124;43m"�[39;49m�[43m)�[49m
 2�[0m valid �[38;5;241m=�[39m Dataset(os�[38;5;241m.�[39mpath�[38;5;241m.�[39mjoin(output_path, �[38;5;124m'�[39m�[38;5;124mvalid�[39m�[38;5;124m'�[39m, �[38;5;124m'�[39m�[38;5;124m*.parquet�[39m�[38;5;124m'�[39m), part_size�[38;5;241m=�[39m�[38;5;124m"�[39m�[38;5;124m500MB�[39m�[38;5;124m"�[39m)
 4�[0m �[38;5;66;03m# define schema object�[39;00m
 File �[0;32m/core/merlin/io/dataset.py:303�[0m, in �[0;36mDataset.__init__�[0;34m(self, path_or_source, engine, npartitions, part_size, part_mem_fraction, storage_options, dtypes, client, cpu, base_dataset, schema, **kwargs)�[0m
 301�[0m �[38;5;28;01mif�[39;00m �[38;5;28misinstance�[39m(engine, �[38;5;28mstr�[39m):
 302�[0m     �[38;5;28;01mif�[39;00m engine �[38;5;241m==�[39m �[38;5;124m"�[39m�[38;5;124mparquet�[39m�[38;5;124m"�[39m:
 �[0;32m--> 303�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mengine �[38;5;241m=�[39m �[43mParquetDatasetEngine�[49m�[43m(�[49m
 304�[0m �[43m            �[49m�[43mpaths�[49m�[43m,�[49m�[43m �[49m�[43mpart_size�[49m�[43m,�[49m�[43m �[49m�[43mstorage_options�[49m�[38;5;241;43m=�[39;49m�[43mstorage_options�[49m�[43m,�[49m�[43m �[49m�[43mcpu�[49m�[38;5;241;43m=�[39;49m�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43mcpu�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m�[39;49m�[38;5;241;43m�[39;49m�[43mkwargs�[49m
 305�[0m �[43m        �[49m�[43m)�[49m
 306�[0m     �[38;5;28;01melif�[39;00m engine �[38;5;241m==�[39m �[38;5;124m"�[39m�[38;5;124mcsv�[39m�[38;5;124m"�[39m:
 307�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mengine �[38;5;241m=�[39m CSVDatasetEngine(
 308�[0m             paths, part_size, storage_options�[38;5;241m=�[39mstorage_options, cpu�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mcpu, �[38;5;241m�[39m�[38;5;241m�[39mkwargs
 309�[0m         )
 File �[0;32m/core/merlin/io/parquet.py:313�[0m, in �[0;36mParquetDatasetEngine.__init__�[0;34m(self, paths, part_size, storage_options, row_groups_per_part, legacy, batch_size, cpu, **kwargs)�[0m
 308�[0m �[38;5;28mself�[39m�[38;5;241m.�[39mdataset_kwargs �[38;5;241m=�[39m �[38;5;28mself�[39m�[38;5;241m.�[39mread_parquet_kwargs�[38;5;241m.�[39mpop(�[38;5;124m"�[39m�[38;5;124mdataset�[39m�[38;5;124m"�[39m, {})
 310�[0m �[38;5;28;01mif�[39;00m row_groups_per_part �[38;5;129;01mis�[39;00m �[38;5;28;01mNone�[39;00m:
 311�[0m     �[38;5;28mself�[39m�[38;5;241m.�[39m_real_meta, rg_byte_size_0 �[38;5;241m=�[39m run_on_worker(
 312�[0m         _sample_row_group,
 �[0;32m--> 313�[0m         �[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43m_path0�[49m,
 314�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mfs,
 315�[0m         cpu�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mcpu,
 316�[0m         memory_usage�[38;5;241m=�[39m�[38;5;28;01mTrue�[39;00m,
 317�[0m         �[38;5;241m�[39m�[38;5;241m�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mread_parquet_kwargs,
 318�[0m     )
 319�[0m     row_groups_per_part �[38;5;241m=�[39m �[38;5;28mself�[39m�[38;5;241m.�[39mpart_size �[38;5;241m/�[39m rg_byte_size_0
 320�[0m     �[38;5;28;01mif�[39;00m row_groups_per_part �[38;5;241m<�[39m �[38;5;241m1.0�[39m:
 File �[0;32m/core/merlin/io/parquet.py:338�[0m, in �[0;36mParquetDatasetEngine._path0�[0;34m(self)�[0m
 335�[0m �[38;5;129m@property�[39m  �[38;5;66;03m# type: ignore�[39;00m
 336�[0m �[38;5;129m@functools�[39m�[38;5;241m.�[39mlru_cache(�[38;5;241m1�[39m)
 337�[0m �[38;5;28;01mdef�[39;00m �[38;5;21m_path0�[39m(�[38;5;28mself�[39m):
 �[0;32m--> 338�[0m     �[38;5;28;01mreturn�[39;00m �[38;5;28mnext�[39m(�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43m_dataset�[49m�[38;5;241m.�[39mget_fragments())�[38;5;241m.�[39mpath
 File �[0;32m/core/merlin/io/parquet.py:368�[0m, in �[0;36mParquetDatasetEngine._dataset�[0;34m(self)�[0m
 365�[0m     dataset �[38;5;241m=�[39m pa_ds�[38;5;241m.�[39mdataset(paths, filesystem�[38;5;241m=�[39mfs)
 366�[0m �[38;5;28;01melse�[39;00m:
 367�[0m     �[38;5;66;03m# This is a directory or a single file�[39;00m
 �[0;32m--> 368�[0m     dataset �[38;5;241m=�[39m pa_ds�[38;5;241m.�[39mdataset(�[43mpaths�[49m�[43m[�[49m�[38;5;241;43m0�[39;49m�[43m]�[49m, filesystem�[38;5;241m=�[39mfs)
 369�[0m �[38;5;28;01mreturn�[39;00m dataset
 �[0;31mIndexError�[0m: list index out of range
 IndexError: list index out of range
../../../.local/lib/python3.8/site-packages/nbclient/client.py:862: CellExecutionError
During handling of the above exception, another exception occurred:
tb = <testbook.client.TestbookNotebookClient object at 0x7f6df419c580>
@testbook(REPO_ROOT / "examples/03-Exploring-different-models.ipynb", timeout=180, execute=False)
def test_func(tb):
    tb.inject(
        """
        import os
        os.environ["DATA_FOLDER"] = "/raid/data/aliccp/raw/"
        os.environ["NUM_ROWS"] = "999"
        os.environ["SYNTHETIC_DATA"] = "False"
        """
    )
    NUM_OF_CELLS = len(tb.cells)


  tb.execute_cell(list(range(0, NUM_OF_CELLS - 5)))


tests/tf/integration/test_ci_03_exploring_different_models.py:19:

self = <testbook.client.TestbookNotebookClient object at 0x7f6df419c580>

cell = {'cell_type': 'markdown', 'id': '3d964063', 'metadata': {}, 'source': 'NVTabular exported the schema file of our proce... other words, Merlin Models relies on the schema object to automatically build all necessary input and output layers.'}

kwargs = {}, cell_indexes = [0, 1, 2, 3, 4, 5, ...]

executed_cells = [{'cell_type': 'code', 'execution_count': 2, 'id': '5f49a48e', 'metadata': {'execution': {'iopub.status.busy': '2022-0...}, {'cell_type': 'markdown', 'id': '6f3b6bc9', 'metadata': {}, 'source': '## Feature Engineering with NVTabular'}, ...]

idx = 14
def execute_cell(self, cell, **kwargs) -> Union[Dict, List[Dict]]:
    """
    Executes a cell or list of cells
    """
    if isinstance(cell, slice):
        start, stop = self._cell_index(cell.start), self._cell_index(cell.stop)
        if cell.step is not None:
            raise TestbookError('testbook does not support step argument')

        cell = range(start, stop + 1)
    elif isinstance(cell, str) or isinstance(cell, int):
        cell = [cell]

    cell_indexes = cell

    if all(isinstance(x, str) for x in cell):
        cell_indexes = [self._cell_index(tag) for tag in cell]

    executed_cells = []
    for idx in cell_indexes:
        try:
            cell = super().execute_cell(self.nb['cells'][idx], idx, **kwargs)
        except CellExecutionError as ce:


          raise TestbookRuntimeError(ce.evalue, ce, self._get_error_class(ce.ename))


E               testbook.exceptions.TestbookRuntimeError: An error occurred while executing the following cell:

E               ------------------

E               train = Dataset(os.path.join(output_path, 'train', '.parquet'), part_size="500MB")

E               valid = Dataset(os.path.join(output_path, 'valid', '.parquet'), part_size="500MB")

E

E               # define schema object

E               schema = train.schema

E               ------------------

E

E               �[0;31m---------------------------------------------------------------------------�[0m

E               �[0;31mIndexError�[0m                                Traceback (most recent call last)

E               Input �[0;32mIn [7]�[0m, in �[0;36m<cell line: 1>�[0;34m()�[0m

E               �[0;32m----> 1�[0m train �[38;5;241m=�[39m �[43mDataset�[49m�[43m(�[49m�[43mos�[49m�[38;5;241;43m.�[39;49m�[43mpath�[49m�[38;5;241;43m.�[39;49m�[43mjoin�[49m�[43m(�[49m�[43moutput_path�[49m�[43m,�[49m�[43m �[49m�[38;5;124;43m'�[39;49m�[38;5;124;43mtrain�[39;49m�[38;5;124;43m'�[39;49m�[43m,�[49m�[43m �[49m�[38;5;124;43m'�[39;49m�[38;5;124;43m*.parquet�[39;49m�[38;5;124;43m'�[39;49m�[43m)�[49m�[43m,�[49m�[43m �[49m�[43mpart_size�[49m�[38;5;241;43m=�[39;49m�[38;5;124;43m"�[39;49m�[38;5;124;43m500MB�[39;49m�[38;5;124;43m"�[39;49m�[43m)�[49m

E               �[1;32m      2�[0m valid �[38;5;241m=�[39m Dataset(os�[38;5;241m.�[39mpath�[38;5;241m.�[39mjoin(output_path, �[38;5;124m'�[39m�[38;5;124mvalid�[39m�[38;5;124m'�[39m, �[38;5;124m'�[39m�[38;5;124m*.parquet�[39m�[38;5;124m'�[39m), part_size�[38;5;241m=�[39m�[38;5;124m"�[39m�[38;5;124m500MB�[39m�[38;5;124m"�[39m)

E               �[1;32m      4�[0m �[38;5;66;03m# define schema object�[39;00m

E

E               File �[0;32m/core/merlin/io/dataset.py:303�[0m, in �[0;36mDataset.__init__�[0;34m(self, path_or_source, engine, npartitions, part_size, part_mem_fraction, storage_options, dtypes, client, cpu, base_dataset, schema, **kwargs)�[0m

E               �[1;32m    301�[0m �[38;5;28;01mif�[39;00m �[38;5;28misinstance�[39m(engine, �[38;5;28mstr�[39m):

E               �[1;32m    302�[0m     �[38;5;28;01mif�[39;00m engine �[38;5;241m==�[39m �[38;5;124m"�[39m�[38;5;124mparquet�[39m�[38;5;124m"�[39m:

E               �[0;32m--> 303�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mengine �[38;5;241m=�[39m �[43mParquetDatasetEngine�[49m�[43m(�[49m

E               �[1;32m    304�[0m �[43m            �[49m�[43mpaths�[49m�[43m,�[49m�[43m �[49m�[43mpart_size�[49m�[43m,�[49m�[43m �[49m�[43mstorage_options�[49m�[38;5;241;43m=�[39;49m�[43mstorage_options�[49m�[43m,�[49m�[43m �[49m�[43mcpu�[49m�[38;5;241;43m=�[39;49m�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43mcpu�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m�[39;49m�[38;5;241;43m�[39;49m�[43mkwargs�[49m

E               �[1;32m    305�[0m �[43m        �[49m�[43m)�[49m

E               �[1;32m    306�[0m     �[38;5;28;01melif�[39;00m engine �[38;5;241m==�[39m �[38;5;124m"�[39m�[38;5;124mcsv�[39m�[38;5;124m"�[39m:

E               �[1;32m    307�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mengine �[38;5;241m=�[39m CSVDatasetEngine(

E               �[1;32m    308�[0m             paths, part_size, storage_options�[38;5;241m=�[39mstorage_options, cpu�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mcpu, �[38;5;241m�[39m�[38;5;241m�[39mkwargs

E               �[1;32m    309�[0m         )

E

E               File �[0;32m/core/merlin/io/parquet.py:313�[0m, in �[0;36mParquetDatasetEngine.__init__�[0;34m(self, paths, part_size, storage_options, row_groups_per_part, legacy, batch_size, cpu, **kwargs)�[0m

E               �[1;32m    308�[0m �[38;5;28mself�[39m�[38;5;241m.�[39mdataset_kwargs �[38;5;241m=�[39m �[38;5;28mself�[39m�[38;5;241m.�[39mread_parquet_kwargs�[38;5;241m.�[39mpop(�[38;5;124m"�[39m�[38;5;124mdataset�[39m�[38;5;124m"�[39m, {})

E               �[1;32m    310�[0m �[38;5;28;01mif�[39;00m row_groups_per_part �[38;5;129;01mis�[39;00m �[38;5;28;01mNone�[39;00m:

E               �[1;32m    311�[0m     �[38;5;28mself�[39m�[38;5;241m.�[39m_real_meta, rg_byte_size_0 �[38;5;241m=�[39m run_on_worker(

E               �[1;32m    312�[0m         _sample_row_group,

E               �[0;32m--> 313�[0m         �[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43m_path0�[49m,

E               �[1;32m    314�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mfs,

E               �[1;32m    315�[0m         cpu�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mcpu,

E               �[1;32m    316�[0m         memory_usage�[38;5;241m=�[39m�[38;5;28;01mTrue�[39;00m,

E               �[1;32m    317�[0m         �[38;5;241m�[39m�[38;5;241m�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mread_parquet_kwargs,

E               �[1;32m    318�[0m     )

E               �[1;32m    319�[0m     row_groups_per_part �[38;5;241m=�[39m �[38;5;28mself�[39m�[38;5;241m.�[39mpart_size �[38;5;241m/�[39m rg_byte_size_0

E               �[1;32m    320�[0m     �[38;5;28;01mif�[39;00m row_groups_per_part �[38;5;241m<�[39m �[38;5;241m1.0�[39m:

E

E               File �[0;32m/core/merlin/io/parquet.py:338�[0m, in �[0;36mParquetDatasetEngine._path0�[0;34m(self)�[0m

E               �[1;32m    335�[0m �[38;5;129m@property�[39m  �[38;5;66;03m# type: ignore�[39;00m

E               �[1;32m    336�[0m �[38;5;129m@functools�[39m�[38;5;241m.�[39mlru_cache(�[38;5;241m1�[39m)

E               �[1;32m    337�[0m �[38;5;28;01mdef�[39;00m �[38;5;21m_path0�[39m(�[38;5;28mself�[39m):

E               �[0;32m--> 338�[0m     �[38;5;28;01mreturn�[39;00m �[38;5;28mnext�[39m(�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43m_dataset�[49m�[38;5;241m.�[39mget_fragments())�[38;5;241m.�[39mpath

E

E               File �[0;32m/core/merlin/io/parquet.py:368�[0m, in �[0;36mParquetDatasetEngine._dataset�[0;34m(self)�[0m

E               �[1;32m    365�[0m     dataset �[38;5;241m=�[39m pa_ds�[38;5;241m.�[39mdataset(paths, filesystem�[38;5;241m=�[39mfs)

E               �[1;32m    366�[0m �[38;5;28;01melse�[39;00m:

E               �[1;32m    367�[0m     �[38;5;66;03m# This is a directory or a single file�[39;00m

E               �[0;32m--> 368�[0m     dataset �[38;5;241m=�[39m pa_ds�[38;5;241m.�[39mdataset(�[43mpaths�[49m�[43m[�[49m�[38;5;241;43m0�[39;49m�[43m]�[49m, filesystem�[38;5;241m=�[39mfs)

E               �[1;32m    369�[0m �[38;5;28;01mreturn�[39;00m dataset

E

E               �[0;31mIndexError�[0m: list index out of range

E               IndexError: list index out of range
/usr/local/lib/python3.8/dist-packages/testbook/client.py:135: TestbookRuntimeError

----------------------------- Captured stderr call -----------------------------

2022-05-12 14:48:08.708714: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA

To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

2022-05-12 14:48:10.678032: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory:  -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0

2022-05-12 14:48:10.678862: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14431 MB memory:  -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0

Error in atexit._run_exitfuncs:

Traceback (most recent call last):

File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown

h.close()

File "/var/jenkins_home/.local/lib/python3.8/site-packages/absl_py-0.12.0-py3.8.egg/absl/logging/init.py", line 945, in close

self.stream.close()

File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 445, in close

self.watch_fd_thread.join()

AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'

__________________________________ test_func ___________________________________
tb = <testbook.client.TestbookNotebookClient object at 0x7f6df41ab8b0>
@testbook(REPO_ROOT / "examples/04-Exporting-ranking-models.ipynb", timeout=180, execute=False)
def test_func(tb):
    tb.inject(
        """
        import os
        os.environ["DATA_FOLDER"] = "/raid/data/aliccp/raw/"
        os.environ["NUM_ROWS"] = "999"
        os.environ["SYNTHETIC_DATA"] = "False"
        os.environ["BATCH_SIZE"] = "16384"
        """
    )


  tb.execute()


tests/tf/integration/test_ci_04_export_ranking_models.py:19:

/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute

super().execute_cell(cell, index)

../../../.local/lib/python3.8/site-packages/nbclient/util.py:84: in wrapped

return just_run(coro(*args, **kwargs))

../../../.local/lib/python3.8/site-packages/nbclient/util.py:62: in just_run

return loop.run_until_complete(coro)

/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete

return future.result()

../../../.local/lib/python3.8/site-packages/nbclient/client.py:965: in async_execute_cell

await self._check_raise_for_error(cell, cell_index, exec_reply)

self = <testbook.client.TestbookNotebookClient object at 0x7f6df41ab8b0>

cell = {'cell_type': 'code', 'execution_count': 9, 'id': 'ba270c23', 'metadata': {'execution': {'iopub.status.busy': '2022-05...style='lines', \n    itemlimit=10, \n    depthlimit=3, \n    exclude_folders='.ipynb_checkpoints', \n    sort=True\n)"}

cell_index = 16

exec_reply = {'buffers': [], 'content': {'ename': 'FileNotFoundError', 'engine_info': {'engine_id': -1, 'engine_uuid': '1db975c1-4f...e, 'engine': '1db975c1-4fa7-4a32-a367-20baebf66c77', 'started': '2022-05-12T14:48:22.991476Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
    self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:

    if exec_reply is None:
        return None

    exec_reply_content = exec_reply['content']
    if exec_reply_content['status'] != 'error':
        return None

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors
        or exec_reply_content.get('ename') in self.allow_error_names
        or "raises-exception" in cell.metadata.get("tags", [])
    )
    await run_hook(self.on_cell_error, cell=cell, cell_index=cell_index)
    if not cell_allows_errors:


      raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)


E           nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:

E           ------------------

E           import seedir as sd

E           sd.seedir(

E               DATA_FOLDER,

E               style='lines',

E               itemlimit=10,

E               depthlimit=3,

E               exclude_folders='.ipynb_checkpoints',

E               sort=True

E           )

E           ------------------

E

E           �[0;31m---------------------------------------------------------------------------�[0m

E           �[0;31mFileNotFoundError�[0m                         Traceback (most recent call last)

E           Input �[0;32mIn [9]�[0m, in �[0;36m<cell line: 2>�[0;34m()�[0m

E           �[1;32m      1�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mseedir�[39;00m �[38;5;28;01mas�[39;00m �[38;5;21;01msd�[39;00m

E           �[0;32m----> 2�[0m �[43msd�[49m�[38;5;241;43m.�[39;49m�[43mseedir�[49m�[43m(�[49m

E           �[1;32m      3�[0m �[43m    �[49m�[43mDATA_FOLDER�[49m�[43m,�[49m�[43m �[49m

E           �[1;32m      4�[0m �[43m    �[49m�[43mstyle�[49m�[38;5;241;43m=�[39;49m�[38;5;124;43m'�[39;49m�[38;5;124;43mlines�[39;49m�[38;5;124;43m'�[39;49m�[43m,�[49m�[43m �[49m

E           �[1;32m      5�[0m �[43m    �[49m�[43mitemlimit�[49m�[38;5;241;43m=�[39;49m�[38;5;241;43m10�[39;49m�[43m,�[49m�[43m �[49m

E           �[1;32m      6�[0m �[43m    �[49m�[43mdepthlimit�[49m�[38;5;241;43m=�[39;49m�[38;5;241;43m3�[39;49m�[43m,�[49m�[43m �[49m

E           �[1;32m      7�[0m �[43m    �[49m�[43mexclude_folders�[49m�[38;5;241;43m=�[39;49m�[38;5;124;43m'�[39;49m�[38;5;124;43m.ipynb_checkpoints�[39;49m�[38;5;124;43m'�[39;49m�[43m,�[49m�[43m �[49m

E           �[1;32m      8�[0m �[43m    �[49m�[43msort�[49m�[38;5;241;43m=�[39;49m�[38;5;28;43;01mTrue�[39;49;00m

E           �[1;32m      9�[0m �[43m)�[49m

E

E           File �[0;32m/usr/local/lib/python3.8/dist-packages/seedir/realdir.py:275�[0m, in �[0;36mseedir�[0;34m(path, style, printout, indent, uniform, anystart, depthlimit, itemlimit, beyond, first, sort, sort_reverse, sort_key, include_folders, exclude_folders, include_files, exclude_files, regex, mask, formatter, slash, **kwargs)�[0m

E           �[1;32m    272�[0m �[38;5;28;01mif�[39;00m path �[38;5;129;01mis�[39;00m �[38;5;28;01mNone�[39;00m:

E           �[1;32m    273�[0m     path �[38;5;241m=�[39m os�[38;5;241m.�[39mgetcwd()

E           �[0;32m--> 275�[0m s �[38;5;241m=�[39m �[43mRealDirStructure�[49m�[43m(�[49m�[43mpath�[49m�[43m,�[49m

E           �[1;32m    276�[0m �[43m                     �[49m�[43mdepthlimit�[49m�[38;5;241;43m=�[39;49m�[43mdepthlimit�[49m�[43m,�[49m

E           �[1;32m    277�[0m �[43m                     �[49m�[43mitemlimit�[49m�[38;5;241;43m=�[39;49m�[43mitemlimit�[49m�[43m,�[49m

E           �[1;32m    278�[0m �[43m                     �[49m�[43mbeyond�[49m�[38;5;241;43m=�[39;49m�[43mbeyond�[49m�[43m,�[49m

E           �[1;32m    279�[0m �[43m                     �[49m�[43mfirst�[49m�[38;5;241;43m=�[39;49m�[43mfirst�[49m�[43m,�[49m

E           �[1;32m    280�[0m �[43m                     �[49m�[43msort�[49m�[38;5;241;43m=�[39;49m�[43msort�[49m�[43m,�[49m

E           �[1;32m    281�[0m �[43m                     �[49m�[43msort_reverse�[49m�[38;5;241;43m=�[39;49m�[43msort_reverse�[49m�[43m,�[49m

E           �[1;32m    282�[0m �[43m                     �[49m�[43msort_key�[49m�[38;5;241;43m=�[39;49m�[43msort_key�[49m�[43m,�[49m

E           �[1;32m    283�[0m �[43m                     �[49m�[43minclude_folders�[49m�[38;5;241;43m=�[39;49m�[43minclude_folders�[49m�[43m,�[49m

E           �[1;32m    284�[0m �[43m                     �[49m�[43mexclude_folders�[49m�[38;5;241;43m=�[39;49m�[43mexclude_folders�[49m�[43m,�[49m

E           �[1;32m    285�[0m �[43m                     �[49m�[43minclude_files�[49m�[38;5;241;43m=�[39;49m�[43minclude_files�[49m�[43m,�[49m

E           �[1;32m    286�[0m �[43m                     �[49m�[43mexclude_files�[49m�[38;5;241;43m=�[39;49m�[43mexclude_files�[49m�[43m,�[49m

E           �[1;32m    287�[0m �[43m                     �[49m�[43mregex�[49m�[38;5;241;43m=�[39;49m�[43mregex�[49m�[43m,�[49m

E           �[1;32m    288�[0m �[43m                     �[49m�[43mslash�[49m�[38;5;241;43m=�[39;49m�[43mslash�[49m�[43m,�[49m

E           �[1;32m    289�[0m �[43m                     �[49m�[43mmask�[49m�[38;5;241;43m=�[39;49m�[43mmask�[49m�[43m,�[49m

E           �[1;32m    290�[0m �[43m                     �[49m�[43mformatter�[49m�[38;5;241;43m=�[39;49m�[43mformatter�[49m�[43m,�[49m

E           �[1;32m    291�[0m �[43m                     �[49m�[38;5;241;43m�[39;49m�[38;5;241;43m�[39;49m�[43mstyleargs�[49m�[43m)�[49m

E           �[1;32m    293�[0m �[38;5;28;01mif�[39;00m printout:

E           �[1;32m    294�[0m     �[38;5;28mprint�[39m(s)

E

E           File �[0;32m/usr/local/lib/python3.8/dist-packages/seedir/folderstructure.py:212�[0m, in �[0;36mFolderStructure.__call__�[0;34m(self, folder, **kwargs)�[0m

E           �[1;32m    211�[0m �[38;5;28;01mdef�[39;00m �[38;5;21m__call__�[39m(�[38;5;28mself�[39m, folder, �[38;5;241m�[39m�[38;5;241m�[39mkwargs):

E           �[0;32m--> 212�[0m     �[38;5;28;01mreturn�[39;00m �[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43m_folderstructure�[49m�[43m(�[49m�[43mfolder�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m�[39;49m�[38;5;241;43m�[39;49m�[43mkwargs�[49m�[43m)�[49m�[38;5;241m.�[39mstrip()

E

E           File �[0;32m/usr/local/lib/python3.8/dist-packages/seedir/folderstructure.py:112�[0m, in �[0;36mFolderStructure._folderstructure�[0;34m(self, folder, depth, incomplete, extend, space, split, final, filestart, folderstart, depthlimit, itemlimit, beyond, first, sort, sort_reverse, sort_key, include_folders, exclude_folders, include_files, exclude_files, regex, mask, formatter, slash)�[0m

E           �[1;32m    109�[0m         current_itemlimit �[38;5;241m=�[39m �[38;5;241m0�[39m

E           �[1;32m    111�[0m �[38;5;66;03m# get all children, add depth to docket�[39;00m

E           �[0;32m--> 112�[0m listdir �[38;5;241m=�[39m �[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43mlistdir�[49m�[43m(�[49m�[43mfolder�[49m�[43m)�[49m

E           �[1;32m    113�[0m incomplete�[38;5;241m.�[39mappend(depth)

E           �[1;32m    115�[0m �[38;5;66;03m# sort and filter the contents of listdir�[39;00m

E

E           File �[0;32m/usr/local/lib/python3.8/dist-packages/seedir/folderstructurehelpers.py:87�[0m, in �[0;36mlistdir_fullpath�[0;34m(path)�[0m

E           �[1;32m     85�[0m �[38;5;28;01mdef�[39;00m �[38;5;21mlistdir_fullpath�[39m(path):

E           �[1;32m     86�[0m     �[38;5;124;03m'''Like os.listdir(), but returns absolute paths.'''�[39;00m

E           �[0;32m---> 87�[0m     �[38;5;28;01mreturn�[39;00m [os�[38;5;241m.�[39mpath�[38;5;241m.�[39mjoin(path, f) �[38;5;28;01mfor�[39;00m f �[38;5;129;01min�[39;00m �[43mos�[49m�[38;5;241;43m.�[39;49m�[43mlistdir�[49m�[43m(�[49m�[43mpath�[49m�[43m)�[49m]

E

E           �[0;31mFileNotFoundError�[0m: [Errno 2] No such file or directory: '/raid/data/aliccp/raw/'

E           FileNotFoundError: [Errno 2] No such file or directory: '/raid/data/aliccp/raw/'
../../../.local/lib/python3.8/site-packages/nbclient/client.py:862: CellExecutionError

----------------------------- Captured stderr call -----------------------------

2022-05-12 14:48:17.110476: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA

To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

2022-05-12 14:48:19.096091: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory:  -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0

2022-05-12 14:48:19.096938: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14431 MB memory:  -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0

Error in atexit._run_exitfuncs:

Traceback (most recent call last):

File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown

h.close()

File "/var/jenkins_home/.local/lib/python3.8/site-packages/absl_py-0.12.0-py3.8.egg/absl/logging/init.py", line 945, in close

self.stream.close()

File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 445, in close

self.watch_fd_thread.join()

AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'

__________________________________ test_func ___________________________________
tb = <testbook.client.TestbookNotebookClient object at 0x7f6df41b0af0>
@testbook(REPO_ROOT / "examples/05-Retrieval-Model.ipynb", timeout=180, execute=False)
def test_func(tb):
    tb.inject(
        """
        import os
        os.environ["DATA_FOLDER"] = "/raid/data/aliccp/raw/"
        os.environ["NUM_ROWS"] = "999"
        os.environ["SYNTHETIC_DATA"] = "False"
        """
    )


  tb.execute()


tests/tf/integration/test_ci_05_export_retrieval_model.py:18:

/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute

super().execute_cell(cell, index)

../../../.local/lib/python3.8/site-packages/nbclient/util.py:84: in wrapped

return just_run(coro(*args, **kwargs))

../../../.local/lib/python3.8/site-packages/nbclient/util.py:62: in just_run

return loop.run_until_complete(coro)

/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete

return future.result()

../../../.local/lib/python3.8/site-packages/nbclient/client.py:965: in async_execute_cell

await self._check_raise_for_error(cell, cell_index, exec_reply)

self = <testbook.client.TestbookNotebookClient object at 0x7f6df41b0af0>

cell = {'cell_type': 'code', 'execution_count': 5, 'id': '3e2ab7e0', 'metadata': {'execution': {'iopub.status.busy': '2022-05... train = nvt.Dataset(DATA_FOLDER + '/train/.parquet')\n    valid = nvt.Dataset(DATA_FOLDER + '/valid/.parquet')'}

cell_index = 8

exec_reply = {'buffers': [], 'content': {'ename': 'IndexError', 'engine_info': {'engine_id': -1, 'engine_uuid': '9d92dfe2-f5e6-441d...e, 'engine': '9d92dfe2-f5e6-441d-921c-5a94ff3f861a', 'started': '2022-05-12T14:48:31.321292Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
    self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:

    if exec_reply is None:
        return None

    exec_reply_content = exec_reply['content']
    if exec_reply_content['status'] != 'error':
        return None

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors
        or exec_reply_content.get('ename') in self.allow_error_names
        or "raises-exception" in cell.metadata.get("tags", [])
    )
    await run_hook(self.on_cell_error, cell=cell, cell_index=cell_index)
    if not cell_allows_errors:


      raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)


E           nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:

E           ------------------

E           from merlin.datasets.synthetic import generate_data

E

E           DATA_FOLDER = os.environ.get("DATA_FOLDER", "/workspace/data/")

E           NUM_ROWS = os.environ.get("NUM_ROWS", 1000000)

E           SYNTHETIC_DATA = eval(os.environ.get("SYNTHETIC_DATA", "True"))

E

E           if SYNTHETIC_DATA:

E               train, valid = generate_data("aliccp-raw", int(NUM_ROWS), set_sizes=(0.7, 0.3))

E           else:

E               train = nvt.Dataset(DATA_FOLDER + '/train/.parquet')

E               valid = nvt.Dataset(DATA_FOLDER + '/valid/.parquet')

E           ------------------

E

E           �[0;31m---------------------------------------------------------------------------�[0m

E           �[0;31mIndexError�[0m                                Traceback (most recent call last)

E           Input �[0;32mIn [5]�[0m, in �[0;36m<cell line: 7>�[0;34m()�[0m

E           �[1;32m      8�[0m     train, valid �[38;5;241m=�[39m generate_data(�[38;5;124m"�[39m�[38;5;124maliccp-raw�[39m�[38;5;124m"�[39m, �[38;5;28mint�[39m(NUM_ROWS), set_sizes�[38;5;241m=�[39m(�[38;5;241m0.7�[39m, �[38;5;241m0.3�[39m))

E           �[1;32m      9�[0m �[38;5;28;01melse�[39;00m:

E           �[0;32m---> 10�[0m     train �[38;5;241m=�[39m �[43mnvt�[49m�[38;5;241;43m.�[39;49m�[43mDataset�[49m�[43m(�[49m�[43mDATA_FOLDER�[49m�[43m �[49m�[38;5;241;43m+�[39;49m�[43m �[49m�[38;5;124;43m'�[39;49m�[38;5;124;43m/train/.parquet�[39;49m�[38;5;124;43m'�[39;49m�[43m)�[49m

E           �[1;32m     11�[0m     valid �[38;5;241m=�[39m nvt�[38;5;241m.�[39mDataset(DATA_FOLDER �[38;5;241m+�[39m �[38;5;124m'�[39m�[38;5;124m/valid/.parquet�[39m�[38;5;124m'�[39m)

E

E           File �[0;32m/core/merlin/io/dataset.py:303�[0m, in �[0;36mDataset.__init__�[0;34m(self, path_or_source, engine, npartitions, part_size, part_mem_fraction, storage_options, dtypes, client, cpu, base_dataset, schema, **kwargs)�[0m

E           �[1;32m    301�[0m �[38;5;28;01mif�[39;00m �[38;5;28misinstance�[39m(engine, �[38;5;28mstr�[39m):

E           �[1;32m    302�[0m     �[38;5;28;01mif�[39;00m engine �[38;5;241m==�[39m �[38;5;124m"�[39m�[38;5;124mparquet�[39m�[38;5;124m"�[39m:

E           �[0;32m--> 303�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mengine �[38;5;241m=�[39m �[43mParquetDatasetEngine�[49m�[43m(�[49m

E           �[1;32m    304�[0m �[43m            �[49m�[43mpaths�[49m�[43m,�[49m�[43m �[49m�[43mpart_size�[49m�[43m,�[49m�[43m �[49m�[43mstorage_options�[49m�[38;5;241;43m=�[39;49m�[43mstorage_options�[49m�[43m,�[49m�[43m �[49m�[43mcpu�[49m�[38;5;241;43m=�[39;49m�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43mcpu�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m�[39;49m�[38;5;241;43m�[39;49m�[43mkwargs�[49m

E           �[1;32m    305�[0m �[43m        �[49m�[43m)�[49m

E           �[1;32m    306�[0m     �[38;5;28;01melif�[39;00m engine �[38;5;241m==�[39m �[38;5;124m"�[39m�[38;5;124mcsv�[39m�[38;5;124m"�[39m:

E           �[1;32m    307�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mengine �[38;5;241m=�[39m CSVDatasetEngine(

E           �[1;32m    308�[0m             paths, part_size, storage_options�[38;5;241m=�[39mstorage_options, cpu�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mcpu, �[38;5;241m�[39m�[38;5;241m�[39mkwargs

E           �[1;32m    309�[0m         )

E

E           File �[0;32m/core/merlin/io/parquet.py:313�[0m, in �[0;36mParquetDatasetEngine.__init__�[0;34m(self, paths, part_size, storage_options, row_groups_per_part, legacy, batch_size, cpu, **kwargs)�[0m

E           �[1;32m    308�[0m �[38;5;28mself�[39m�[38;5;241m.�[39mdataset_kwargs �[38;5;241m=�[39m �[38;5;28mself�[39m�[38;5;241m.�[39mread_parquet_kwargs�[38;5;241m.�[39mpop(�[38;5;124m"�[39m�[38;5;124mdataset�[39m�[38;5;124m"�[39m, {})

E           �[1;32m    310�[0m �[38;5;28;01mif�[39;00m row_groups_per_part �[38;5;129;01mis�[39;00m �[38;5;28;01mNone�[39;00m:

E           �[1;32m    311�[0m     �[38;5;28mself�[39m�[38;5;241m.�[39m_real_meta, rg_byte_size_0 �[38;5;241m=�[39m run_on_worker(

E           �[1;32m    312�[0m         _sample_row_group,

E           �[0;32m--> 313�[0m         �[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43m_path0�[49m,

E           �[1;32m    314�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mfs,

E           �[1;32m    315�[0m         cpu�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mcpu,

E           �[1;32m    316�[0m         memory_usage�[38;5;241m=�[39m�[38;5;28;01mTrue�[39;00m,

E           �[1;32m    317�[0m         �[38;5;241m�[39m�[38;5;241m�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mread_parquet_kwargs,

E           �[1;32m    318�[0m     )

E           �[1;32m    319�[0m     row_groups_per_part �[38;5;241m=�[39m �[38;5;28mself�[39m�[38;5;241m.�[39mpart_size �[38;5;241m/�[39m rg_byte_size_0

E           �[1;32m    320�[0m     �[38;5;28;01mif�[39;00m row_groups_per_part �[38;5;241m<�[39m �[38;5;241m1.0�[39m:

E

E           File �[0;32m/core/merlin/io/parquet.py:338�[0m, in �[0;36mParquetDatasetEngine._path0�[0;34m(self)�[0m

E           �[1;32m    335�[0m �[38;5;129m@property�[39m  �[38;5;66;03m# type: ignore�[39;00m

E           �[1;32m    336�[0m �[38;5;129m@functools�[39m�[38;5;241m.�[39mlru_cache(�[38;5;241m1�[39m)

E           �[1;32m    337�[0m �[38;5;28;01mdef�[39;00m �[38;5;21m_path0�[39m(�[38;5;28mself�[39m):

E           �[0;32m--> 338�[0m     �[38;5;28;01mreturn�[39;00m �[38;5;28mnext�[39m(�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43m_dataset�[49m�[38;5;241m.�[39mget_fragments())�[38;5;241m.�[39mpath

E

E           File �[0;32m/core/merlin/io/parquet.py:368�[0m, in �[0;36mParquetDatasetEngine._dataset�[0;34m(self)�[0m

E           �[1;32m    365�[0m     dataset �[38;5;241m=�[39m pa_ds�[38;5;241m.�[39mdataset(paths, filesystem�[38;5;241m=�[39mfs)

E           �[1;32m    366�[0m �[38;5;28;01melse�[39;00m:

E           �[1;32m    367�[0m     �[38;5;66;03m# This is a directory or a single file�[39;00m

E           �[0;32m--> 368�[0m     dataset �[38;5;241m=�[39m pa_ds�[38;5;241m.�[39mdataset(�[43mpaths�[49m�[43m[�[49m�[38;5;241;43m0�[39;49m�[43m]�[49m, filesystem�[38;5;241m=�[39mfs)

E           �[1;32m    369�[0m �[38;5;28;01mreturn�[39;00m dataset

E

E           �[0;31mIndexError�[0m: list index out of range

E           IndexError: list index out of range
../../../.local/lib/python3.8/site-packages/nbclient/client.py:862: CellExecutionError

----------------------------- Captured stderr call -----------------------------

2022-05-12 14:48:29.270495: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA

To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

2022-05-12 14:48:31.243921: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory:  -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0

2022-05-12 14:48:31.244728: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14431 MB memory:  -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0

Error in atexit._run_exitfuncs:

Traceback (most recent call last):

File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown

h.close()

File "/var/jenkins_home/.local/lib/python3.8/site-packages/absl_py-0.12.0-py3.8.egg/absl/logging/init.py", line 945, in close

self.stream.close()

File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 445, in close

self.watch_fd_thread.join()

AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'

=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

if hasattr(pil_image, 'HAMMING'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

_PIL_INTERPOLATION_METHODS['hamming'] = pil_image.HAMMING
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

if hasattr(pil_image, 'BOX'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

_PIL_INTERPOLATION_METHODS['box'] = pil_image.BOX
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

if hasattr(pil_image, 'LANCZOS'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

_PIL_INTERPOLATION_METHODS['lanczos'] = pil_image.LANCZOS
../../../../../usr/lib/python3/dist-packages/requests/init.py:89

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

interpolation: int = Image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

interpolation: int = Image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11

/usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: libtorch_cuda_cu.so: cannot open shared object file: No such file or directory

warn(f"Failed to load image Python extension: {e}")
tests/tf/integration/test_ci_01_getting_started.py:7

/var/jenkins_home/workspace/merlin_models/models/tests/tf/integration/test_ci_01_getting_started.py:7: PytestUnknownMarkWarning: Unknown pytest.mark.ci - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html

@pytest.mark.ci
tests/tf/integration/test_ci_02_dataschema.py:7

/var/jenkins_home/workspace/merlin_models/models/tests/tf/integration/test_ci_02_dataschema.py:7: PytestUnknownMarkWarning: Unknown pytest.mark.ci - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html

@pytest.mark.ci
tests/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data

tests/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/tf/test_dataset.py::test_tf_catname_ordering

tests/tf/test_dataset.py::test_tf_map

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/tf/blocks/core/test_index.py: 3 warnings

tests/tf/models/test_retrieval.py: 28 warnings

tests/tf/prediction_tasks/test_next_item.py: 55 warnings

tests/tf/utils/test_batch.py: 2 warnings

/tmp/tmpxfj_ibfe.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag__.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/backend.py:6089: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

warnings.warn('tf.keras.backend.random_binomial is deprecated, '
tests/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)

return {key: torch.tensor(value) for key, value in data.items()}
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========================== short test summary info ============================

FAILED tests/tf/integration/test_ci_03_exploring_different_models.py::test_func

FAILED tests/tf/integration/test_ci_04_export_ranking_models.py::test_func - ...

FAILED tests/tf/integration/test_ci_05_export_retrieval_model.py::test_func

===== 3 failed, 431 passed, 10 skipped, 120 warnings in 826.64s (0:13:46) ======

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins8103301217258057943.sh

nvidia-merlin-bot · 2022-05-12T16:35:09Z

Click to view CI Results

GitHub pull request #433 of commit fd1215e5611078c4fad9c391b69cc6ea8f479fce, no merge conflicts.
Running as SYSTEM
Setting status of fd1215e5611078c4fad9c391b69cc6ea8f479fce to PENDING with url https://10.20.13.93:8080/job/merlin_models/275/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/433/*:refs/remotes/origin/pr/433/* # timeout=10
 > git rev-parse fd1215e5611078c4fad9c391b69cc6ea8f479fce^{commit} # timeout=10
Checking out Revision fd1215e5611078c4fad9c391b69cc6ea8f479fce (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f fd1215e5611078c4fad9c391b69cc6ea8f479fce # timeout=10
Commit message: "Fix import of movielens dataset in XGBoost docstring"
 > git rev-list --no-walk 4c3b14a9c752b5b7d0fc41409a6caab67c78c8d6 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins7330655397210580453.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: traitlets>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.2.0)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.1)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.5.1)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (22.3.0)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
WARNING: You are using pip version 22.0.4; however, version 22.1 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 442 items / 2 skipped
tests/config/test_schema.py ....                                         [  0%]

tests/datasets/test_advertising.py .s                                    [  1%]

tests/datasets/test_ecommerce.py ..sss                                   [  2%]

tests/datasets/test_entertainment.py ....sss                             [  4%]

tests/datasets/test_social.py .                                          [  4%]

tests/datasets/test_synthetic.py .....                                   [  5%]

tests/tf/test_core.py ................                                   [  9%]

tests/tf/test_dataset.py ...............                                 [ 12%]

tests/tf/test_public_api.py .                                            [ 12%]

tests/tf/blocks/test_cross.py ............                               [ 15%]

tests/tf/blocks/test_dlrm.py ........                                    [ 17%]

tests/tf/blocks/test_interactions.py .                                   [ 17%]

tests/tf/blocks/test_mlp.py .............................                [ 23%]

tests/tf/blocks/core/test_aggregation.py .........                       [ 26%]

tests/tf/blocks/core/test_base.py .                                      [ 26%]

tests/tf/blocks/core/test_index.py ...                                   [ 26%]

tests/tf/blocks/core/test_masking.py .......                             [ 28%]

tests/tf/blocks/core/test_transformations.py ...........                 [ 30%]

tests/tf/blocks/retrieval/test_matrix_factorization.py ..                [ 31%]

tests/tf/blocks/retrieval/test_two_tower.py ...........                  [ 33%]

tests/tf/examples/test_01_getting_started.py .                           [ 34%]

tests/tf/examples/test_02_dataschema.py .                                [ 34%]

tests/tf/examples/test_03_exploring_different_models.py .                [ 34%]

tests/tf/examples/test_04_export_ranking_models.py .                     [ 34%]

tests/tf/examples/test_05_export_retrieval_model.py .                    [ 35%]

tests/tf/examples/test_06_advanced_own_architecture.py .                 [ 35%]

tests/tf/features/test_continuous.py .....                               [ 36%]

tests/tf/features/test_embedding.py ..........s.                         [ 39%]

tests/tf/features/test_tabular.py .......                                [ 40%]

tests/tf/integration/test_ci_01_getting_started.py .                     [ 40%]

tests/tf/integration/test_ci_02_dataschema.py .                          [ 41%]

tests/tf/integration/test_ci_03_exploring_different_models.py F          [ 41%]

tests/tf/integration/test_ci_04_export_ranking_models.py F               [ 41%]

tests/tf/integration/test_ci_05_export_retrieval_model.py F              [ 41%]

tests/tf/integration/test_ci_06_advanced_own_architecture.py .           [ 42%]

tests/tf/layers/test_queue.py ..............                             [ 45%]

tests/tf/losses/test_losses.py .......................                   [ 50%]

tests/tf/metrics/test_metrics_popularity.py .....                        [ 51%]

tests/tf/metrics/test_metrics_ranking.py .................               [ 55%]

tests/tf/models/test_benchmark.py .                                      [ 55%]

tests/tf/models/test_ranking.py ......................                   [ 60%]

tests/tf/models/test_retrieval.py ..................                     [ 64%]

tests/tf/prediction_tasks/test_classification.py ..                      [ 65%]

tests/tf/prediction_tasks/test_multi_task.py .......                     [ 66%]

tests/tf/prediction_tasks/test_next_item.py ......................       [ 71%]

tests/tf/prediction_tasks/test_regression.py ..                          [ 72%]

tests/tf/prediction_tasks/test_sampling.py ....................          [ 76%]

tests/tf/utils/test_batch.py ....                                        [ 77%]

tests/torch/test_dataset.py .........                                    [ 79%]

tests/torch/test_public_api.py .                                         [ 79%]

tests/torch/block/test_base.py ....                                      [ 80%]

tests/torch/block/test_mlp.py .                                          [ 80%]

tests/torch/features/test_continuous.py ..                               [ 81%]

tests/torch/features/test_embedding.py ..............                    [ 84%]

tests/torch/features/test_tabular.py ....                                [ 85%]

tests/torch/model/test_head.py ............                              [ 88%]

tests/torch/model/test_model.py ..                                       [ 88%]

tests/torch/tabular/test_aggregation.py ........                         [ 90%]

tests/torch/tabular/test_tabular.py ...                                  [ 91%]

tests/torch/tabular/test_transformations.py .......                      [ 92%]

tests/utils/test_schema_utils.py ................................        [100%]
=================================== FAILURES ===================================

__________________________________ test_func ___________________________________
self = <testbook.client.TestbookNotebookClient object at 0x7fc1b9d9b580>

cell = {'cell_type': 'markdown', 'id': '3d964063', 'metadata': {}, 'source': 'NVTabular exported the schema file of our proce... other words, Merlin Models relies on the schema object to automatically build all necessary input and output layers.'}

kwargs = {}, cell_indexes = [0, 1, 2, 3, 4, 5, ...]

executed_cells = [{'cell_type': 'code', 'execution_count': 2, 'id': '5f49a48e', 'metadata': {'execution': {'iopub.status.busy': '2022-0...}, {'cell_type': 'markdown', 'id': '6f3b6bc9', 'metadata': {}, 'source': '## Feature Engineering with NVTabular'}, ...]

idx = 14
def execute_cell(self, cell, **kwargs) -> Union[Dict, List[Dict]]:
    """
    Executes a cell or list of cells
    """
    if isinstance(cell, slice):
        start, stop = self._cell_index(cell.start), self._cell_index(cell.stop)
        if cell.step is not None:
            raise TestbookError('testbook does not support step argument')

        cell = range(start, stop + 1)
    elif isinstance(cell, str) or isinstance(cell, int):
        cell = [cell]

    cell_indexes = cell

    if all(isinstance(x, str) for x in cell):
        cell_indexes = [self._cell_index(tag) for tag in cell]

    executed_cells = []
    for idx in cell_indexes:
        try:


          cell = super().execute_cell(self.nb['cells'][idx], idx, **kwargs)


/usr/local/lib/python3.8/dist-packages/testbook/client.py:133:

args = (<testbook.client.TestbookNotebookClient object at 0x7fc1b9d9b580>, {'cell_type': 'code', 'execution_count': 7, 'id': ...join(output_path, 'valid', '*.parquet'), part_size="500MB")\n\n# define schema object\nschema = train.schema'}, 14)

kwargs = {}
def wrapped(*args, **kwargs):


  return just_run(coro(*args, **kwargs))


../../../.local/lib/python3.8/site-packages/nbclient/util.py:84:

coro = <coroutine object NotebookClient.async_execute_cell at 0x7fc1b9af24c0>
def just_run(coro: Awaitable) -> Any:
    """Make the coroutine run, even if there is an event loop running (using nest_asyncio)"""
    # original from vaex/asyncio.py
    loop = asyncio._get_running_loop()
    if loop is None:
        had_running_loop = False
        try:
            loop = asyncio.get_event_loop()
        except RuntimeError:
            # we can still get 'There is no current event loop in ...'
            loop = asyncio.new_event_loop()
            asyncio.set_event_loop(loop)
    else:
        had_running_loop = True
    if had_running_loop:
        # if there is a running loop, we patch using nest_asyncio
        # to have reentrant event loops
        check_ipython()
        import nest_asyncio

        nest_asyncio.apply()
        check_patch_tornado()


  return loop.run_until_complete(coro)


../../../.local/lib/python3.8/site-packages/nbclient/util.py:62:

self = <_UnixSelectorEventLoop running=False closed=False debug=False>

future = <Task finished name='Task-1072' coro=<NotebookClient.async_execute_cell() done, defined at /var/jenkins_home/.local/li...rn\x1b[39;00m dataset\n\n\x1b[0;31mIndexError\x1b[0m: list index out of range\nIndexError: list index out of range\n')>
def run_until_complete(self, future):
    """Run until the Future is done.

    If the argument is a coroutine, it is wrapped in a Task.

    WARNING: It would be disastrous to call run_until_complete()
    with the same coroutine twice -- it would wrap it in two
    different Tasks and that can't be good.

    Return the Future's result, or raise its exception.
    """
    self._check_closed()
    self._check_running()

    new_task = not futures.isfuture(future)
    future = tasks.ensure_future(future, loop=self)
    if new_task:
        # An exception is raised if the future didn't complete, so there
        # is no need to log the "destroy pending task" message
        future._log_destroy_pending = False

    future.add_done_callback(_run_until_complete_cb)
    try:
        self.run_forever()
    except:
        if new_task and future.done() and not future.cancelled():
            # The coroutine raised a BaseException. Consume the exception
            # to not log a warning, the caller doesn't have access to the
            # local task.
            future.exception()
        raise
    finally:
        future.remove_done_callback(_run_until_complete_cb)
    if not future.done():
        raise RuntimeError('Event loop stopped before Future completed.')


  return future.result()


/usr/lib/python3.8/asyncio/base_events.py:616:

self = <testbook.client.TestbookNotebookClient object at 0x7fc1b9d9b580>

cell = {'cell_type': 'code', 'execution_count': 7, 'id': 'a0f0cc5f', 'metadata': {'execution': {'iopub.status.busy': '2022-05...path.join(output_path, 'valid', '*.parquet'), part_size="500MB")\n\n# define schema object\nschema = train.schema'}

cell_index = 14, execution_count = None, store_history = True
async def async_execute_cell(
    self,
    cell: NotebookNode,
    cell_index: int,
    execution_count: t.Optional[int] = None,
    store_history: bool = True,
) -> NotebookNode:
    """
    Executes a single code cell.

    To execute all cells see :meth:`execute`.

    Parameters
    ----------
    cell : nbformat.NotebookNode
        The cell which is currently being processed.
    cell_index : int
        The position of the cell within the notebook object.
    execution_count : int
        The execution count to be assigned to the cell (default: Use kernel response)
    store_history : bool
        Determines if history should be stored in the kernel (default: False).
        Specific to ipython kernels, which can store command histories.

    Returns
    -------
    output : dict
        The execution output payload (or None for no output).

    Raises
    ------
    CellExecutionError
        If execution failed and should raise an exception, this will be raised
        with defaults about the failure.

    Returns
    -------
    cell : NotebookNode
        The cell which was just processed.
    """
    assert self.kc is not None

    await run_hook(self.on_cell_start, cell=cell, cell_index=cell_index)

    if cell.cell_type != 'code' or not cell.source.strip():
        self.log.debug("Skipping non-executing cell %s", cell_index)
        return cell

    if self.skip_cells_with_tag in cell.metadata.get("tags", []):
        self.log.debug("Skipping tagged cell %s", cell_index)
        return cell

    if self.record_timing:  # clear execution metadata prior to execution
        cell['metadata']['execution'] = {}

    self.log.debug("Executing cell:\n%s", cell.source)

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors or "raises-exception" in cell.metadata.get("tags", [])
    )

    await run_hook(self.on_cell_execute, cell=cell, cell_index=cell_index)
    parent_msg_id = await ensure_async(
        self.kc.execute(
            cell.source, store_history=store_history, stop_on_error=not cell_allows_errors
        )
    )
    await run_hook(self.on_cell_complete, cell=cell, cell_index=cell_index)
    # We launched a code cell to execute
    self.code_cells_executed += 1
    exec_timeout = self._get_timeout(cell)

    cell.outputs = []
    self.clear_before_next_output = False

    task_poll_kernel_alive = asyncio.ensure_future(self._async_poll_kernel_alive())
    task_poll_output_msg = asyncio.ensure_future(
        self._async_poll_output_msg(parent_msg_id, cell, cell_index)
    )
    self.task_poll_for_reply = asyncio.ensure_future(
        self._async_poll_for_reply(
            parent_msg_id, cell, exec_timeout, task_poll_output_msg, task_poll_kernel_alive
        )
    )
    try:
        exec_reply = await self.task_poll_for_reply
    except asyncio.CancelledError:
        # can only be cancelled by task_poll_kernel_alive when the kernel is dead
        task_poll_output_msg.cancel()
        raise DeadKernelError("Kernel died")
    except Exception as e:
        # Best effort to cancel request if it hasn't been resolved
        try:
            # Check if the task_poll_output is doing the raising for us
            if not isinstance(e, CellControlSignal):
                task_poll_output_msg.cancel()
        finally:
            raise

    if execution_count:
        cell['execution_count'] = execution_count


  await self._check_raise_for_error(cell, cell_index, exec_reply)


../../../.local/lib/python3.8/site-packages/nbclient/client.py:965:

self = <testbook.client.TestbookNotebookClient object at 0x7fc1b9d9b580>

cell = {'cell_type': 'code', 'execution_count': 7, 'id': 'a0f0cc5f', 'metadata': {'execution': {'iopub.status.busy': '2022-05...path.join(output_path, 'valid', '*.parquet'), part_size="500MB")\n\n# define schema object\nschema = train.schema'}

cell_index = 14

exec_reply = {'buffers': [], 'content': {'ename': 'IndexError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'f98de1da-ae26-4a39...e, 'engine': 'f98de1da-ae26-4a39-9de9-ed01c0022911', 'started': '2022-05-12T16:26:26.116341Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
    self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:

    if exec_reply is None:
        return None

    exec_reply_content = exec_reply['content']
    if exec_reply_content['status'] != 'error':
        return None

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors
        or exec_reply_content.get('ename') in self.allow_error_names
        or "raises-exception" in cell.metadata.get("tags", [])
    )
    await run_hook(self.on_cell_error, cell=cell, cell_index=cell_index)
    if not cell_allows_errors:


      raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)


E           nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:

E           ------------------

E           train = Dataset(os.path.join(output_path, 'train', '.parquet'), part_size="500MB")

E           valid = Dataset(os.path.join(output_path, 'valid', '.parquet'), part_size="500MB")

E

E           # define schema object

E           schema = train.schema

E           ------------------

E

E           �[0;31m---------------------------------------------------------------------------�[0m

E           �[0;31mIndexError�[0m                                Traceback (most recent call last)

E           Input �[0;32mIn [7]�[0m, in �[0;36m<cell line: 1>�[0;34m()�[0m

E           �[0;32m----> 1�[0m train �[38;5;241m=�[39m �[43mDataset�[49m�[43m(�[49m�[43mos�[49m�[38;5;241;43m.�[39;49m�[43mpath�[49m�[38;5;241;43m.�[39;49m�[43mjoin�[49m�[43m(�[49m�[43moutput_path�[49m�[43m,�[49m�[43m �[49m�[38;5;124;43m'�[39;49m�[38;5;124;43mtrain�[39;49m�[38;5;124;43m'�[39;49m�[43m,�[49m�[43m �[49m�[38;5;124;43m'�[39;49m�[38;5;124;43m*.parquet�[39;49m�[38;5;124;43m'�[39;49m�[43m)�[49m�[43m,�[49m�[43m �[49m�[43mpart_size�[49m�[38;5;241;43m=�[39;49m�[38;5;124;43m"�[39;49m�[38;5;124;43m500MB�[39;49m�[38;5;124;43m"�[39;49m�[43m)�[49m

E           �[1;32m      2�[0m valid �[38;5;241m=�[39m Dataset(os�[38;5;241m.�[39mpath�[38;5;241m.�[39mjoin(output_path, �[38;5;124m'�[39m�[38;5;124mvalid�[39m�[38;5;124m'�[39m, �[38;5;124m'�[39m�[38;5;124m*.parquet�[39m�[38;5;124m'�[39m), part_size�[38;5;241m=�[39m�[38;5;124m"�[39m�[38;5;124m500MB�[39m�[38;5;124m"�[39m)

E           �[1;32m      4�[0m �[38;5;66;03m# define schema object�[39;00m

E

E           File �[0;32m/core/merlin/io/dataset.py:303�[0m, in �[0;36mDataset.__init__�[0;34m(self, path_or_source, engine, npartitions, part_size, part_mem_fraction, storage_options, dtypes, client, cpu, base_dataset, schema, **kwargs)�[0m

E           �[1;32m    301�[0m �[38;5;28;01mif�[39;00m �[38;5;28misinstance�[39m(engine, �[38;5;28mstr�[39m):

E           �[1;32m    302�[0m     �[38;5;28;01mif�[39;00m engine �[38;5;241m==�[39m �[38;5;124m"�[39m�[38;5;124mparquet�[39m�[38;5;124m"�[39m:

E           �[0;32m--> 303�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mengine �[38;5;241m=�[39m �[43mParquetDatasetEngine�[49m�[43m(�[49m

E           �[1;32m    304�[0m �[43m            �[49m�[43mpaths�[49m�[43m,�[49m�[43m �[49m�[43mpart_size�[49m�[43m,�[49m�[43m �[49m�[43mstorage_options�[49m�[38;5;241;43m=�[39;49m�[43mstorage_options�[49m�[43m,�[49m�[43m �[49m�[43mcpu�[49m�[38;5;241;43m=�[39;49m�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43mcpu�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m�[39;49m�[38;5;241;43m�[39;49m�[43mkwargs�[49m

E           �[1;32m    305�[0m �[43m        �[49m�[43m)�[49m

E           �[1;32m    306�[0m     �[38;5;28;01melif�[39;00m engine �[38;5;241m==�[39m �[38;5;124m"�[39m�[38;5;124mcsv�[39m�[38;5;124m"�[39m:

E           �[1;32m    307�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mengine �[38;5;241m=�[39m CSVDatasetEngine(

E           �[1;32m    308�[0m             paths, part_size, storage_options�[38;5;241m=�[39mstorage_options, cpu�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mcpu, �[38;5;241m�[39m�[38;5;241m�[39mkwargs

E           �[1;32m    309�[0m         )

E

E           File �[0;32m/core/merlin/io/parquet.py:313�[0m, in �[0;36mParquetDatasetEngine.__init__�[0;34m(self, paths, part_size, storage_options, row_groups_per_part, legacy, batch_size, cpu, **kwargs)�[0m

E           �[1;32m    308�[0m �[38;5;28mself�[39m�[38;5;241m.�[39mdataset_kwargs �[38;5;241m=�[39m �[38;5;28mself�[39m�[38;5;241m.�[39mread_parquet_kwargs�[38;5;241m.�[39mpop(�[38;5;124m"�[39m�[38;5;124mdataset�[39m�[38;5;124m"�[39m, {})

E           �[1;32m    310�[0m �[38;5;28;01mif�[39;00m row_groups_per_part �[38;5;129;01mis�[39;00m �[38;5;28;01mNone�[39;00m:

E           �[1;32m    311�[0m     �[38;5;28mself�[39m�[38;5;241m.�[39m_real_meta, rg_byte_size_0 �[38;5;241m=�[39m run_on_worker(

E           �[1;32m    312�[0m         _sample_row_group,

E           �[0;32m--> 313�[0m         �[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43m_path0�[49m,

E           �[1;32m    314�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mfs,

E           �[1;32m    315�[0m         cpu�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mcpu,

E           �[1;32m    316�[0m         memory_usage�[38;5;241m=�[39m�[38;5;28;01mTrue�[39;00m,

E           �[1;32m    317�[0m         �[38;5;241m�[39m�[38;5;241m�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mread_parquet_kwargs,

E           �[1;32m    318�[0m     )

E           �[1;32m    319�[0m     row_groups_per_part �[38;5;241m=�[39m �[38;5;28mself�[39m�[38;5;241m.�[39mpart_size �[38;5;241m/�[39m rg_byte_size_0

E           �[1;32m    320�[0m     �[38;5;28;01mif�[39;00m row_groups_per_part �[38;5;241m<�[39m �[38;5;241m1.0�[39m:

E

E           File �[0;32m/core/merlin/io/parquet.py:338�[0m, in �[0;36mParquetDatasetEngine._path0�[0;34m(self)�[0m

E           �[1;32m    335�[0m �[38;5;129m@property�[39m  �[38;5;66;03m# type: ignore�[39;00m

E           �[1;32m    336�[0m �[38;5;129m@functools�[39m�[38;5;241m.�[39mlru_cache(�[38;5;241m1�[39m)

E           �[1;32m    337�[0m �[38;5;28;01mdef�[39;00m �[38;5;21m_path0�[39m(�[38;5;28mself�[39m):

E           �[0;32m--> 338�[0m     �[38;5;28;01mreturn�[39;00m �[38;5;28mnext�[39m(�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43m_dataset�[49m�[38;5;241m.�[39mget_fragments())�[38;5;241m.�[39mpath

E

E           File �[0;32m/core/merlin/io/parquet.py:368�[0m, in �[0;36mParquetDatasetEngine._dataset�[0;34m(self)�[0m

E           �[1;32m    365�[0m     dataset �[38;5;241m=�[39m pa_ds�[38;5;241m.�[39mdataset(paths, filesystem�[38;5;241m=�[39mfs)

E           �[1;32m    366�[0m �[38;5;28;01melse�[39;00m:

E           �[1;32m    367�[0m     �[38;5;66;03m# This is a directory or a single file�[39;00m

E           �[0;32m--> 368�[0m     dataset �[38;5;241m=�[39m pa_ds�[38;5;241m.�[39mdataset(�[43mpaths�[49m�[43m[�[49m�[38;5;241;43m0�[39;49m�[43m]�[49m, filesystem�[38;5;241m=�[39mfs)

E           �[1;32m    369�[0m �[38;5;28;01mreturn�[39;00m dataset

E

E           �[0;31mIndexError�[0m: list index out of range

E           IndexError: list index out of range
../../../.local/lib/python3.8/site-packages/nbclient/client.py:862: CellExecutionError
During handling of the above exception, another exception occurred:
tb = <testbook.client.TestbookNotebookClient object at 0x7fc1b9d9b580>
@testbook(REPO_ROOT / "examples/03-Exploring-different-models.ipynb", timeout=180, execute=False)
def test_func(tb):
    tb.inject(
        """
        import os
        os.environ["DATA_FOLDER"] = "/raid/data/aliccp/raw/"
        os.environ["NUM_ROWS"] = "999"
        os.environ["SYNTHETIC_DATA"] = "False"
        """
    )
    NUM_OF_CELLS = len(tb.cells)


  tb.execute_cell(list(range(0, NUM_OF_CELLS - 5)))


tests/tf/integration/test_ci_03_exploring_different_models.py:19:

self = <testbook.client.TestbookNotebookClient object at 0x7fc1b9d9b580>

cell = {'cell_type': 'markdown', 'id': '3d964063', 'metadata': {}, 'source': 'NVTabular exported the schema file of our proce... other words, Merlin Models relies on the schema object to automatically build all necessary input and output layers.'}

kwargs = {}, cell_indexes = [0, 1, 2, 3, 4, 5, ...]

executed_cells = [{'cell_type': 'code', 'execution_count': 2, 'id': '5f49a48e', 'metadata': {'execution': {'iopub.status.busy': '2022-0...}, {'cell_type': 'markdown', 'id': '6f3b6bc9', 'metadata': {}, 'source': '## Feature Engineering with NVTabular'}, ...]

idx = 14
def execute_cell(self, cell, **kwargs) -> Union[Dict, List[Dict]]:
    """
    Executes a cell or list of cells
    """
    if isinstance(cell, slice):
        start, stop = self._cell_index(cell.start), self._cell_index(cell.stop)
        if cell.step is not None:
            raise TestbookError('testbook does not support step argument')

        cell = range(start, stop + 1)
    elif isinstance(cell, str) or isinstance(cell, int):
        cell = [cell]

    cell_indexes = cell

    if all(isinstance(x, str) for x in cell):
        cell_indexes = [self._cell_index(tag) for tag in cell]

    executed_cells = []
    for idx in cell_indexes:
        try:
            cell = super().execute_cell(self.nb['cells'][idx], idx, **kwargs)
        except CellExecutionError as ce:


          raise TestbookRuntimeError(ce.evalue, ce, self._get_error_class(ce.ename))


E               testbook.exceptions.TestbookRuntimeError: An error occurred while executing the following cell:

E               ------------------

E               train = Dataset(os.path.join(output_path, 'train', '.parquet'), part_size="500MB")

E               valid = Dataset(os.path.join(output_path, 'valid', '.parquet'), part_size="500MB")

E

E               # define schema object

E               schema = train.schema

E               ------------------

E

E               �[0;31m---------------------------------------------------------------------------�[0m

E               �[0;31mIndexError�[0m                                Traceback (most recent call last)

E               Input �[0;32mIn [7]�[0m, in �[0;36m<cell line: 1>�[0;34m()�[0m

E               �[0;32m----> 1�[0m train �[38;5;241m=�[39m �[43mDataset�[49m�[43m(�[49m�[43mos�[49m�[38;5;241;43m.�[39;49m�[43mpath�[49m�[38;5;241;43m.�[39;49m�[43mjoin�[49m�[43m(�[49m�[43moutput_path�[49m�[43m,�[49m�[43m �[49m�[38;5;124;43m'�[39;49m�[38;5;124;43mtrain�[39;49m�[38;5;124;43m'�[39;49m�[43m,�[49m�[43m �[49m�[38;5;124;43m'�[39;49m�[38;5;124;43m*.parquet�[39;49m�[38;5;124;43m'�[39;49m�[43m)�[49m�[43m,�[49m�[43m �[49m�[43mpart_size�[49m�[38;5;241;43m=�[39;49m�[38;5;124;43m"�[39;49m�[38;5;124;43m500MB�[39;49m�[38;5;124;43m"�[39;49m�[43m)�[49m

E               �[1;32m      2�[0m valid �[38;5;241m=�[39m Dataset(os�[38;5;241m.�[39mpath�[38;5;241m.�[39mjoin(output_path, �[38;5;124m'�[39m�[38;5;124mvalid�[39m�[38;5;124m'�[39m, �[38;5;124m'�[39m�[38;5;124m*.parquet�[39m�[38;5;124m'�[39m), part_size�[38;5;241m=�[39m�[38;5;124m"�[39m�[38;5;124m500MB�[39m�[38;5;124m"�[39m)

E               �[1;32m      4�[0m �[38;5;66;03m# define schema object�[39;00m

E

E               File �[0;32m/core/merlin/io/dataset.py:303�[0m, in �[0;36mDataset.__init__�[0;34m(self, path_or_source, engine, npartitions, part_size, part_mem_fraction, storage_options, dtypes, client, cpu, base_dataset, schema, **kwargs)�[0m

E               �[1;32m    301�[0m �[38;5;28;01mif�[39;00m �[38;5;28misinstance�[39m(engine, �[38;5;28mstr�[39m):

E               �[1;32m    302�[0m     �[38;5;28;01mif�[39;00m engine �[38;5;241m==�[39m �[38;5;124m"�[39m�[38;5;124mparquet�[39m�[38;5;124m"�[39m:

E               �[0;32m--> 303�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mengine �[38;5;241m=�[39m �[43mParquetDatasetEngine�[49m�[43m(�[49m

E               �[1;32m    304�[0m �[43m            �[49m�[43mpaths�[49m�[43m,�[49m�[43m �[49m�[43mpart_size�[49m�[43m,�[49m�[43m �[49m�[43mstorage_options�[49m�[38;5;241;43m=�[39;49m�[43mstorage_options�[49m�[43m,�[49m�[43m �[49m�[43mcpu�[49m�[38;5;241;43m=�[39;49m�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43mcpu�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m�[39;49m�[38;5;241;43m�[39;49m�[43mkwargs�[49m

E               �[1;32m    305�[0m �[43m        �[49m�[43m)�[49m

E               �[1;32m    306�[0m     �[38;5;28;01melif�[39;00m engine �[38;5;241m==�[39m �[38;5;124m"�[39m�[38;5;124mcsv�[39m�[38;5;124m"�[39m:

E               �[1;32m    307�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mengine �[38;5;241m=�[39m CSVDatasetEngine(

E               �[1;32m    308�[0m             paths, part_size, storage_options�[38;5;241m=�[39mstorage_options, cpu�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mcpu, �[38;5;241m�[39m�[38;5;241m�[39mkwargs

E               �[1;32m    309�[0m         )

E

E               File �[0;32m/core/merlin/io/parquet.py:313�[0m, in �[0;36mParquetDatasetEngine.__init__�[0;34m(self, paths, part_size, storage_options, row_groups_per_part, legacy, batch_size, cpu, **kwargs)�[0m

E               �[1;32m    308�[0m �[38;5;28mself�[39m�[38;5;241m.�[39mdataset_kwargs �[38;5;241m=�[39m �[38;5;28mself�[39m�[38;5;241m.�[39mread_parquet_kwargs�[38;5;241m.�[39mpop(�[38;5;124m"�[39m�[38;5;124mdataset�[39m�[38;5;124m"�[39m, {})

E               �[1;32m    310�[0m �[38;5;28;01mif�[39;00m row_groups_per_part �[38;5;129;01mis�[39;00m �[38;5;28;01mNone�[39;00m:

E               �[1;32m    311�[0m     �[38;5;28mself�[39m�[38;5;241m.�[39m_real_meta, rg_byte_size_0 �[38;5;241m=�[39m run_on_worker(

E               �[1;32m    312�[0m         _sample_row_group,

E               �[0;32m--> 313�[0m         �[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43m_path0�[49m,

E               �[1;32m    314�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mfs,

E               �[1;32m    315�[0m         cpu�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mcpu,

E               �[1;32m    316�[0m         memory_usage�[38;5;241m=�[39m�[38;5;28;01mTrue�[39;00m,

E               �[1;32m    317�[0m         �[38;5;241m�[39m�[38;5;241m�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mread_parquet_kwargs,

E               �[1;32m    318�[0m     )

E               �[1;32m    319�[0m     row_groups_per_part �[38;5;241m=�[39m �[38;5;28mself�[39m�[38;5;241m.�[39mpart_size �[38;5;241m/�[39m rg_byte_size_0

E               �[1;32m    320�[0m     �[38;5;28;01mif�[39;00m row_groups_per_part �[38;5;241m<�[39m �[38;5;241m1.0�[39m:

E

E               File �[0;32m/core/merlin/io/parquet.py:338�[0m, in �[0;36mParquetDatasetEngine._path0�[0;34m(self)�[0m

E               �[1;32m    335�[0m �[38;5;129m@property�[39m  �[38;5;66;03m# type: ignore�[39;00m

E               �[1;32m    336�[0m �[38;5;129m@functools�[39m�[38;5;241m.�[39mlru_cache(�[38;5;241m1�[39m)

E               �[1;32m    337�[0m �[38;5;28;01mdef�[39;00m �[38;5;21m_path0�[39m(�[38;5;28mself�[39m):

E               �[0;32m--> 338�[0m     �[38;5;28;01mreturn�[39;00m �[38;5;28mnext�[39m(�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43m_dataset�[49m�[38;5;241m.�[39mget_fragments())�[38;5;241m.�[39mpath

E

E               File �[0;32m/core/merlin/io/parquet.py:368�[0m, in �[0;36mParquetDatasetEngine._dataset�[0;34m(self)�[0m

E               �[1;32m    365�[0m     dataset �[38;5;241m=�[39m pa_ds�[38;5;241m.�[39mdataset(paths, filesystem�[38;5;241m=�[39mfs)

E               �[1;32m    366�[0m �[38;5;28;01melse�[39;00m:

E               �[1;32m    367�[0m     �[38;5;66;03m# This is a directory or a single file�[39;00m

E               �[0;32m--> 368�[0m     dataset �[38;5;241m=�[39m pa_ds�[38;5;241m.�[39mdataset(�[43mpaths�[49m�[43m[�[49m�[38;5;241;43m0�[39;49m�[43m]�[49m, filesystem�[38;5;241m=�[39mfs)

E               �[1;32m    369�[0m �[38;5;28;01mreturn�[39;00m dataset

E

E               �[0;31mIndexError�[0m: list index out of range

E               IndexError: list index out of range
/usr/local/lib/python3.8/dist-packages/testbook/client.py:135: TestbookRuntimeError

----------------------------- Captured stderr call -----------------------------

2022-05-12 16:26:23.591800: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA

To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

2022-05-12 16:26:25.571381: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory:  -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0

2022-05-12 16:26:25.572220: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14431 MB memory:  -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0

Error in atexit._run_exitfuncs:

Traceback (most recent call last):

File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown

h.close()

File "/var/jenkins_home/.local/lib/python3.8/site-packages/absl_py-0.12.0-py3.8.egg/absl/logging/init.py", line 945, in close

self.stream.close()

File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 445, in close

self.watch_fd_thread.join()

AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'

__________________________________ test_func ___________________________________
tb = <testbook.client.TestbookNotebookClient object at 0x7fc1b9daa910>
@testbook(REPO_ROOT / "examples/04-Exporting-ranking-models.ipynb", timeout=180, execute=False)
def test_func(tb):
    tb.inject(
        """
        import os
        os.environ["DATA_FOLDER"] = "/raid/data/aliccp/raw/"
        os.environ["NUM_ROWS"] = "999"
        os.environ["SYNTHETIC_DATA"] = "False"
        os.environ["BATCH_SIZE"] = "16384"
        """
    )


  tb.execute()


tests/tf/integration/test_ci_04_export_ranking_models.py:19:

/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute

super().execute_cell(cell, index)

../../../.local/lib/python3.8/site-packages/nbclient/util.py:84: in wrapped

return just_run(coro(*args, **kwargs))

../../../.local/lib/python3.8/site-packages/nbclient/util.py:62: in just_run

return loop.run_until_complete(coro)

/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete

return future.result()

../../../.local/lib/python3.8/site-packages/nbclient/client.py:965: in async_execute_cell

await self._check_raise_for_error(cell, cell_index, exec_reply)

self = <testbook.client.TestbookNotebookClient object at 0x7fc1b9daa910>

cell = {'cell_type': 'code', 'execution_count': 9, 'id': 'ba270c23', 'metadata': {'execution': {'iopub.status.busy': '2022-05...style='lines', \n    itemlimit=10, \n    depthlimit=3, \n    exclude_folders='.ipynb_checkpoints', \n    sort=True\n)"}

cell_index = 16

exec_reply = {'buffers': [], 'content': {'ename': 'FileNotFoundError', 'engine_info': {'engine_id': -1, 'engine_uuid': '75dc1c6b-af...e, 'engine': '75dc1c6b-af2b-4a33-a1e8-31c0278bcf59', 'started': '2022-05-12T16:26:37.857230Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
    self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:

    if exec_reply is None:
        return None

    exec_reply_content = exec_reply['content']
    if exec_reply_content['status'] != 'error':
        return None

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors
        or exec_reply_content.get('ename') in self.allow_error_names
        or "raises-exception" in cell.metadata.get("tags", [])
    )
    await run_hook(self.on_cell_error, cell=cell, cell_index=cell_index)
    if not cell_allows_errors:


      raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)


E           nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:

E           ------------------

E           import seedir as sd

E           sd.seedir(

E               DATA_FOLDER,

E               style='lines',

E               itemlimit=10,

E               depthlimit=3,

E               exclude_folders='.ipynb_checkpoints',

E               sort=True

E           )

E           ------------------

E

E           �[0;31m---------------------------------------------------------------------------�[0m

E           �[0;31mFileNotFoundError�[0m                         Traceback (most recent call last)

E           Input �[0;32mIn [9]�[0m, in �[0;36m<cell line: 2>�[0;34m()�[0m

E           �[1;32m      1�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mseedir�[39;00m �[38;5;28;01mas�[39;00m �[38;5;21;01msd�[39;00m

E           �[0;32m----> 2�[0m �[43msd�[49m�[38;5;241;43m.�[39;49m�[43mseedir�[49m�[43m(�[49m

E           �[1;32m      3�[0m �[43m    �[49m�[43mDATA_FOLDER�[49m�[43m,�[49m�[43m �[49m

E           �[1;32m      4�[0m �[43m    �[49m�[43mstyle�[49m�[38;5;241;43m=�[39;49m�[38;5;124;43m'�[39;49m�[38;5;124;43mlines�[39;49m�[38;5;124;43m'�[39;49m�[43m,�[49m�[43m �[49m

E           �[1;32m      5�[0m �[43m    �[49m�[43mitemlimit�[49m�[38;5;241;43m=�[39;49m�[38;5;241;43m10�[39;49m�[43m,�[49m�[43m �[49m

E           �[1;32m      6�[0m �[43m    �[49m�[43mdepthlimit�[49m�[38;5;241;43m=�[39;49m�[38;5;241;43m3�[39;49m�[43m,�[49m�[43m �[49m

E           �[1;32m      7�[0m �[43m    �[49m�[43mexclude_folders�[49m�[38;5;241;43m=�[39;49m�[38;5;124;43m'�[39;49m�[38;5;124;43m.ipynb_checkpoints�[39;49m�[38;5;124;43m'�[39;49m�[43m,�[49m�[43m �[49m

E           �[1;32m      8�[0m �[43m    �[49m�[43msort�[49m�[38;5;241;43m=�[39;49m�[38;5;28;43;01mTrue�[39;49;00m

E           �[1;32m      9�[0m �[43m)�[49m

E

E           File �[0;32m/usr/local/lib/python3.8/dist-packages/seedir/realdir.py:275�[0m, in �[0;36mseedir�[0;34m(path, style, printout, indent, uniform, anystart, depthlimit, itemlimit, beyond, first, sort, sort_reverse, sort_key, include_folders, exclude_folders, include_files, exclude_files, regex, mask, formatter, slash, **kwargs)�[0m

E           �[1;32m    272�[0m �[38;5;28;01mif�[39;00m path �[38;5;129;01mis�[39;00m �[38;5;28;01mNone�[39;00m:

E           �[1;32m    273�[0m     path �[38;5;241m=�[39m os�[38;5;241m.�[39mgetcwd()

E           �[0;32m--> 275�[0m s �[38;5;241m=�[39m �[43mRealDirStructure�[49m�[43m(�[49m�[43mpath�[49m�[43m,�[49m

E           �[1;32m    276�[0m �[43m                     �[49m�[43mdepthlimit�[49m�[38;5;241;43m=�[39;49m�[43mdepthlimit�[49m�[43m,�[49m

E           �[1;32m    277�[0m �[43m                     �[49m�[43mitemlimit�[49m�[38;5;241;43m=�[39;49m�[43mitemlimit�[49m�[43m,�[49m

E           �[1;32m    278�[0m �[43m                     �[49m�[43mbeyond�[49m�[38;5;241;43m=�[39;49m�[43mbeyond�[49m�[43m,�[49m

E           �[1;32m    279�[0m �[43m                     �[49m�[43mfirst�[49m�[38;5;241;43m=�[39;49m�[43mfirst�[49m�[43m,�[49m

E           �[1;32m    280�[0m �[43m                     �[49m�[43msort�[49m�[38;5;241;43m=�[39;49m�[43msort�[49m�[43m,�[49m

E           �[1;32m    281�[0m �[43m                     �[49m�[43msort_reverse�[49m�[38;5;241;43m=�[39;49m�[43msort_reverse�[49m�[43m,�[49m

E           �[1;32m    282�[0m �[43m                     �[49m�[43msort_key�[49m�[38;5;241;43m=�[39;49m�[43msort_key�[49m�[43m,�[49m

E           �[1;32m    283�[0m �[43m                     �[49m�[43minclude_folders�[49m�[38;5;241;43m=�[39;49m�[43minclude_folders�[49m�[43m,�[49m

E           �[1;32m    284�[0m �[43m                     �[49m�[43mexclude_folders�[49m�[38;5;241;43m=�[39;49m�[43mexclude_folders�[49m�[43m,�[49m

E           �[1;32m    285�[0m �[43m                     �[49m�[43minclude_files�[49m�[38;5;241;43m=�[39;49m�[43minclude_files�[49m�[43m,�[49m

E           �[1;32m    286�[0m �[43m                     �[49m�[43mexclude_files�[49m�[38;5;241;43m=�[39;49m�[43mexclude_files�[49m�[43m,�[49m

E           �[1;32m    287�[0m �[43m                     �[49m�[43mregex�[49m�[38;5;241;43m=�[39;49m�[43mregex�[49m�[43m,�[49m

E           �[1;32m    288�[0m �[43m                     �[49m�[43mslash�[49m�[38;5;241;43m=�[39;49m�[43mslash�[49m�[43m,�[49m

E           �[1;32m    289�[0m �[43m                     �[49m�[43mmask�[49m�[38;5;241;43m=�[39;49m�[43mmask�[49m�[43m,�[49m

E           �[1;32m    290�[0m �[43m                     �[49m�[43mformatter�[49m�[38;5;241;43m=�[39;49m�[43mformatter�[49m�[43m,�[49m

E           �[1;32m    291�[0m �[43m                     �[49m�[38;5;241;43m�[39;49m�[38;5;241;43m�[39;49m�[43mstyleargs�[49m�[43m)�[49m

E           �[1;32m    293�[0m �[38;5;28;01mif�[39;00m printout:

E           �[1;32m    294�[0m     �[38;5;28mprint�[39m(s)

E

E           File �[0;32m/usr/local/lib/python3.8/dist-packages/seedir/folderstructure.py:212�[0m, in �[0;36mFolderStructure.__call__�[0;34m(self, folder, **kwargs)�[0m

E           �[1;32m    211�[0m �[38;5;28;01mdef�[39;00m �[38;5;21m__call__�[39m(�[38;5;28mself�[39m, folder, �[38;5;241m�[39m�[38;5;241m�[39mkwargs):

E           �[0;32m--> 212�[0m     �[38;5;28;01mreturn�[39;00m �[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43m_folderstructure�[49m�[43m(�[49m�[43mfolder�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m�[39;49m�[38;5;241;43m�[39;49m�[43mkwargs�[49m�[43m)�[49m�[38;5;241m.�[39mstrip()

E

E           File �[0;32m/usr/local/lib/python3.8/dist-packages/seedir/folderstructure.py:112�[0m, in �[0;36mFolderStructure._folderstructure�[0;34m(self, folder, depth, incomplete, extend, space, split, final, filestart, folderstart, depthlimit, itemlimit, beyond, first, sort, sort_reverse, sort_key, include_folders, exclude_folders, include_files, exclude_files, regex, mask, formatter, slash)�[0m

E           �[1;32m    109�[0m         current_itemlimit �[38;5;241m=�[39m �[38;5;241m0�[39m

E           �[1;32m    111�[0m �[38;5;66;03m# get all children, add depth to docket�[39;00m

E           �[0;32m--> 112�[0m listdir �[38;5;241m=�[39m �[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43mlistdir�[49m�[43m(�[49m�[43mfolder�[49m�[43m)�[49m

E           �[1;32m    113�[0m incomplete�[38;5;241m.�[39mappend(depth)

E           �[1;32m    115�[0m �[38;5;66;03m# sort and filter the contents of listdir�[39;00m

E

E           File �[0;32m/usr/local/lib/python3.8/dist-packages/seedir/folderstructurehelpers.py:87�[0m, in �[0;36mlistdir_fullpath�[0;34m(path)�[0m

E           �[1;32m     85�[0m �[38;5;28;01mdef�[39;00m �[38;5;21mlistdir_fullpath�[39m(path):

E           �[1;32m     86�[0m     �[38;5;124;03m'''Like os.listdir(), but returns absolute paths.'''�[39;00m

E           �[0;32m---> 87�[0m     �[38;5;28;01mreturn�[39;00m [os�[38;5;241m.�[39mpath�[38;5;241m.�[39mjoin(path, f) �[38;5;28;01mfor�[39;00m f �[38;5;129;01min�[39;00m �[43mos�[49m�[38;5;241;43m.�[39;49m�[43mlistdir�[49m�[43m(�[49m�[43mpath�[49m�[43m)�[49m]

E

E           �[0;31mFileNotFoundError�[0m: [Errno 2] No such file or directory: '/raid/data/aliccp/raw/'

E           FileNotFoundError: [Errno 2] No such file or directory: '/raid/data/aliccp/raw/'
../../../.local/lib/python3.8/site-packages/nbclient/client.py:862: CellExecutionError

----------------------------- Captured stderr call -----------------------------

2022-05-12 16:26:32.039549: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA

To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

2022-05-12 16:26:34.039011: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory:  -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0

2022-05-12 16:26:34.039733: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14431 MB memory:  -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0

Error in atexit._run_exitfuncs:

Traceback (most recent call last):

File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown

h.close()

File "/var/jenkins_home/.local/lib/python3.8/site-packages/absl_py-0.12.0-py3.8.egg/absl/logging/init.py", line 945, in close

self.stream.close()

File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 445, in close

self.watch_fd_thread.join()

AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'

__________________________________ test_func ___________________________________
tb = <testbook.client.TestbookNotebookClient object at 0x7fc1b9db0b50>
@testbook(REPO_ROOT / "examples/05-Retrieval-Model.ipynb", timeout=180, execute=False)
def test_func(tb):
    tb.inject(
        """
        import os
        os.environ["DATA_FOLDER"] = "/raid/data/aliccp/raw/"
        os.environ["NUM_ROWS"] = "999"
        os.environ["SYNTHETIC_DATA"] = "False"
        """
    )


  tb.execute()


tests/tf/integration/test_ci_05_export_retrieval_model.py:18:

/usr/local/lib/python3.8/dist-packages/testbook/client.py:147: in execute

super().execute_cell(cell, index)

../../../.local/lib/python3.8/site-packages/nbclient/util.py:84: in wrapped

return just_run(coro(*args, **kwargs))

../../../.local/lib/python3.8/site-packages/nbclient/util.py:62: in just_run

return loop.run_until_complete(coro)

/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete

return future.result()

../../../.local/lib/python3.8/site-packages/nbclient/client.py:965: in async_execute_cell

await self._check_raise_for_error(cell, cell_index, exec_reply)

self = <testbook.client.TestbookNotebookClient object at 0x7fc1b9db0b50>

cell = {'cell_type': 'code', 'execution_count': 5, 'id': '3e2ab7e0', 'metadata': {'execution': {'iopub.status.busy': '2022-05... train = nvt.Dataset(DATA_FOLDER + '/train/.parquet')\n    valid = nvt.Dataset(DATA_FOLDER + '/valid/.parquet')'}

cell_index = 8

exec_reply = {'buffers': [], 'content': {'ename': 'IndexError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'ca6ddc6e-a955-4044...e, 'engine': 'ca6ddc6e-a955-4044-8373-0f6ba51e3851', 'started': '2022-05-12T16:26:45.964905Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
    self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:

    if exec_reply is None:
        return None

    exec_reply_content = exec_reply['content']
    if exec_reply_content['status'] != 'error':
        return None

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors
        or exec_reply_content.get('ename') in self.allow_error_names
        or "raises-exception" in cell.metadata.get("tags", [])
    )
    await run_hook(self.on_cell_error, cell=cell, cell_index=cell_index)
    if not cell_allows_errors:


      raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)


E           nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:

E           ------------------

E           from merlin.datasets.synthetic import generate_data

E

E           DATA_FOLDER = os.environ.get("DATA_FOLDER", "/workspace/data/")

E           NUM_ROWS = os.environ.get("NUM_ROWS", 1000000)

E           SYNTHETIC_DATA = eval(os.environ.get("SYNTHETIC_DATA", "True"))

E

E           if SYNTHETIC_DATA:

E               train, valid = generate_data("aliccp-raw", int(NUM_ROWS), set_sizes=(0.7, 0.3))

E           else:

E               train = nvt.Dataset(DATA_FOLDER + '/train/.parquet')

E               valid = nvt.Dataset(DATA_FOLDER + '/valid/.parquet')

E           ------------------

E

E           �[0;31m---------------------------------------------------------------------------�[0m

E           �[0;31mIndexError�[0m                                Traceback (most recent call last)

E           Input �[0;32mIn [5]�[0m, in �[0;36m<cell line: 7>�[0;34m()�[0m

E           �[1;32m      8�[0m     train, valid �[38;5;241m=�[39m generate_data(�[38;5;124m"�[39m�[38;5;124maliccp-raw�[39m�[38;5;124m"�[39m, �[38;5;28mint�[39m(NUM_ROWS), set_sizes�[38;5;241m=�[39m(�[38;5;241m0.7�[39m, �[38;5;241m0.3�[39m))

E           �[1;32m      9�[0m �[38;5;28;01melse�[39;00m:

E           �[0;32m---> 10�[0m     train �[38;5;241m=�[39m �[43mnvt�[49m�[38;5;241;43m.�[39;49m�[43mDataset�[49m�[43m(�[49m�[43mDATA_FOLDER�[49m�[43m �[49m�[38;5;241;43m+�[39;49m�[43m �[49m�[38;5;124;43m'�[39;49m�[38;5;124;43m/train/.parquet�[39;49m�[38;5;124;43m'�[39;49m�[43m)�[49m

E           �[1;32m     11�[0m     valid �[38;5;241m=�[39m nvt�[38;5;241m.�[39mDataset(DATA_FOLDER �[38;5;241m+�[39m �[38;5;124m'�[39m�[38;5;124m/valid/.parquet�[39m�[38;5;124m'�[39m)

E

E           File �[0;32m/core/merlin/io/dataset.py:303�[0m, in �[0;36mDataset.__init__�[0;34m(self, path_or_source, engine, npartitions, part_size, part_mem_fraction, storage_options, dtypes, client, cpu, base_dataset, schema, **kwargs)�[0m

E           �[1;32m    301�[0m �[38;5;28;01mif�[39;00m �[38;5;28misinstance�[39m(engine, �[38;5;28mstr�[39m):

E           �[1;32m    302�[0m     �[38;5;28;01mif�[39;00m engine �[38;5;241m==�[39m �[38;5;124m"�[39m�[38;5;124mparquet�[39m�[38;5;124m"�[39m:

E           �[0;32m--> 303�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mengine �[38;5;241m=�[39m �[43mParquetDatasetEngine�[49m�[43m(�[49m

E           �[1;32m    304�[0m �[43m            �[49m�[43mpaths�[49m�[43m,�[49m�[43m �[49m�[43mpart_size�[49m�[43m,�[49m�[43m �[49m�[43mstorage_options�[49m�[38;5;241;43m=�[39;49m�[43mstorage_options�[49m�[43m,�[49m�[43m �[49m�[43mcpu�[49m�[38;5;241;43m=�[39;49m�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43mcpu�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m�[39;49m�[38;5;241;43m�[39;49m�[43mkwargs�[49m

E           �[1;32m    305�[0m �[43m        �[49m�[43m)�[49m

E           �[1;32m    306�[0m     �[38;5;28;01melif�[39;00m engine �[38;5;241m==�[39m �[38;5;124m"�[39m�[38;5;124mcsv�[39m�[38;5;124m"�[39m:

E           �[1;32m    307�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mengine �[38;5;241m=�[39m CSVDatasetEngine(

E           �[1;32m    308�[0m             paths, part_size, storage_options�[38;5;241m=�[39mstorage_options, cpu�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mcpu, �[38;5;241m�[39m�[38;5;241m�[39mkwargs

E           �[1;32m    309�[0m         )

E

E           File �[0;32m/core/merlin/io/parquet.py:313�[0m, in �[0;36mParquetDatasetEngine.__init__�[0;34m(self, paths, part_size, storage_options, row_groups_per_part, legacy, batch_size, cpu, **kwargs)�[0m

E           �[1;32m    308�[0m �[38;5;28mself�[39m�[38;5;241m.�[39mdataset_kwargs �[38;5;241m=�[39m �[38;5;28mself�[39m�[38;5;241m.�[39mread_parquet_kwargs�[38;5;241m.�[39mpop(�[38;5;124m"�[39m�[38;5;124mdataset�[39m�[38;5;124m"�[39m, {})

E           �[1;32m    310�[0m �[38;5;28;01mif�[39;00m row_groups_per_part �[38;5;129;01mis�[39;00m �[38;5;28;01mNone�[39;00m:

E           �[1;32m    311�[0m     �[38;5;28mself�[39m�[38;5;241m.�[39m_real_meta, rg_byte_size_0 �[38;5;241m=�[39m run_on_worker(

E           �[1;32m    312�[0m         _sample_row_group,

E           �[0;32m--> 313�[0m         �[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43m_path0�[49m,

E           �[1;32m    314�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mfs,

E           �[1;32m    315�[0m         cpu�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mcpu,

E           �[1;32m    316�[0m         memory_usage�[38;5;241m=�[39m�[38;5;28;01mTrue�[39;00m,

E           �[1;32m    317�[0m         �[38;5;241m�[39m�[38;5;241m�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mread_parquet_kwargs,

E           �[1;32m    318�[0m     )

E           �[1;32m    319�[0m     row_groups_per_part �[38;5;241m=�[39m �[38;5;28mself�[39m�[38;5;241m.�[39mpart_size �[38;5;241m/�[39m rg_byte_size_0

E           �[1;32m    320�[0m     �[38;5;28;01mif�[39;00m row_groups_per_part �[38;5;241m<�[39m �[38;5;241m1.0�[39m:

E

E           File �[0;32m/core/merlin/io/parquet.py:338�[0m, in �[0;36mParquetDatasetEngine._path0�[0;34m(self)�[0m

E           �[1;32m    335�[0m �[38;5;129m@property�[39m  �[38;5;66;03m# type: ignore�[39;00m

E           �[1;32m    336�[0m �[38;5;129m@functools�[39m�[38;5;241m.�[39mlru_cache(�[38;5;241m1�[39m)

E           �[1;32m    337�[0m �[38;5;28;01mdef�[39;00m �[38;5;21m_path0�[39m(�[38;5;28mself�[39m):

E           �[0;32m--> 338�[0m     �[38;5;28;01mreturn�[39;00m �[38;5;28mnext�[39m(�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43m_dataset�[49m�[38;5;241m.�[39mget_fragments())�[38;5;241m.�[39mpath

E

E           File �[0;32m/core/merlin/io/parquet.py:368�[0m, in �[0;36mParquetDatasetEngine._dataset�[0;34m(self)�[0m

E           �[1;32m    365�[0m     dataset �[38;5;241m=�[39m pa_ds�[38;5;241m.�[39mdataset(paths, filesystem�[38;5;241m=�[39mfs)

E           �[1;32m    366�[0m �[38;5;28;01melse�[39;00m:

E           �[1;32m    367�[0m     �[38;5;66;03m# This is a directory or a single file�[39;00m

E           �[0;32m--> 368�[0m     dataset �[38;5;241m=�[39m pa_ds�[38;5;241m.�[39mdataset(�[43mpaths�[49m�[43m[�[49m�[38;5;241;43m0�[39;49m�[43m]�[49m, filesystem�[38;5;241m=�[39mfs)

E           �[1;32m    369�[0m �[38;5;28;01mreturn�[39;00m dataset

E

E           �[0;31mIndexError�[0m: list index out of range

E           IndexError: list index out of range
../../../.local/lib/python3.8/site-packages/nbclient/client.py:862: CellExecutionError

----------------------------- Captured stderr call -----------------------------

2022-05-12 16:26:43.906524: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA

To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

2022-05-12 16:26:45.900471: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1627 MB memory:  -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0

2022-05-12 16:26:45.901323: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 14431 MB memory:  -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0

Error in atexit._run_exitfuncs:

Traceback (most recent call last):

File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown

h.close()

File "/var/jenkins_home/.local/lib/python3.8/site-packages/absl_py-0.12.0-py3.8.egg/absl/logging/init.py", line 945, in close

self.stream.close()

File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 445, in close

self.watch_fd_thread.join()

AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'

=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

if hasattr(pil_image, 'HAMMING'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

_PIL_INTERPOLATION_METHODS['hamming'] = pil_image.HAMMING
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

if hasattr(pil_image, 'BOX'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

_PIL_INTERPOLATION_METHODS['box'] = pil_image.BOX
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

if hasattr(pil_image, 'LANCZOS'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

_PIL_INTERPOLATION_METHODS['lanczos'] = pil_image.LANCZOS
../../../../../usr/lib/python3/dist-packages/requests/init.py:89

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

interpolation: int = Image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

interpolation: int = Image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11

/usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: libtorch_cuda_cu.so: cannot open shared object file: No such file or directory

warn(f"Failed to load image Python extension: {e}")
tests/tf/integration/test_ci_01_getting_started.py:7

/var/jenkins_home/workspace/merlin_models/models/tests/tf/integration/test_ci_01_getting_started.py:7: PytestUnknownMarkWarning: Unknown pytest.mark.ci - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html

@pytest.mark.ci
tests/tf/integration/test_ci_02_dataschema.py:7

/var/jenkins_home/workspace/merlin_models/models/tests/tf/integration/test_ci_02_dataschema.py:7: PytestUnknownMarkWarning: Unknown pytest.mark.ci - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html

@pytest.mark.ci
tests/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data

tests/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/tf/test_dataset.py::test_tf_catname_ordering

tests/tf/test_dataset.py::test_tf_map

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/tf/blocks/core/test_index.py: 3 warnings

tests/tf/models/test_retrieval.py: 28 warnings

tests/tf/prediction_tasks/test_next_item.py: 55 warnings

tests/tf/utils/test_batch.py: 2 warnings

/tmp/tmpwo9h5kx4.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag__.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/backend.py:6089: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

warnings.warn('tf.keras.backend.random_binomial is deprecated, '
tests/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)

return {key: torch.tensor(value) for key, value in data.items()}
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========================== short test summary info ============================

FAILED tests/tf/integration/test_ci_03_exploring_different_models.py::test_func

FAILED tests/tf/integration/test_ci_04_export_ranking_models.py::test_func - ...

FAILED tests/tf/integration/test_ci_05_export_retrieval_model.py::test_func

===== 3 failed, 431 passed, 10 skipped, 120 warnings in 823.34s (0:13:43) ======

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins4640542881720983935.sh

merlin/models/xgb/__init__.py

nvidia-merlin-bot · 2022-05-18T13:44:14Z

Click to view CI Results

GitHub pull request #433 of commit cfc5a9ccfd8d3177e33056a5a88311167d7daaa5, no merge conflicts.
Running as SYSTEM
Setting status of cfc5a9ccfd8d3177e33056a5a88311167d7daaa5 to PENDING with url https://10.20.13.93:8080/job/merlin_models/302/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/433/*:refs/remotes/origin/pr/433/* # timeout=10
 > git rev-parse cfc5a9ccfd8d3177e33056a5a88311167d7daaa5^{commit} # timeout=10
Checking out Revision cfc5a9ccfd8d3177e33056a5a88311167d7daaa5 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f cfc5a9ccfd8d3177e33056a5a88311167d7daaa5 # timeout=10
Commit message: "Implement evaluate method using test dataset"
 > git rev-list --no-walk 469e77bd3620145e305a569e5569dec0a9e8de2d # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins7647065199722823503.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting testbook
  Downloading testbook-0.4.2-py3-none-any.whl (11 kB)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: traitlets>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.2.1.post0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.1)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.5.1)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (22.3.0)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
Installing collected packages: testbook
Successfully installed testbook-0.4.2
error: cannot open .git/FETCH_HEAD: Permission denied
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script  : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log" 
[merlin_models] $ /bin/bash /tmp/jenkins4008724805401585477.sh

nvidia-merlin-bot · 2022-05-18T17:14:12Z

Click to view CI Results

GitHub pull request #433 of commit e422564531903f6ad1acc1e406ef0a456daa03bb, has merge conflicts.
Running as SYSTEM
Setting status of e422564531903f6ad1acc1e406ef0a456daa03bb to PENDING with url https://10.20.13.93:8080/job/merlin_models/308/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/433/*:refs/remotes/origin/pr/433/* # timeout=10
 > git rev-parse e422564531903f6ad1acc1e406ef0a456daa03bb^{commit} # timeout=10
Checking out Revision e422564531903f6ad1acc1e406ef0a456daa03bb (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f e422564531903f6ad1acc1e406ef0a456daa03bb # timeout=10
Commit message: "Add xgboost requirements"
 > git rev-list --no-walk 970cd49d5a32d82b56289c2b6b2cdfbdbb611caf # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins2169273440432894659.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: traitlets>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.2.1.post0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.1)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.5.1)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (22.3.0)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
error: cannot open .git/FETCH_HEAD: Permission denied
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script  : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log" 
[merlin_models] $ /bin/bash /tmp/jenkins7030565799872082649.sh

merlin/models/xgb/__init__.py

nvidia-merlin-bot · 2022-05-19T14:18:03Z

Click to view CI Results

GitHub pull request #433 of commit e8458a915cb18f7476589542a5200bd030bc7d47, has merge conflicts.
Running as SYSTEM
Setting status of e8458a915cb18f7476589542a5200bd030bc7d47 to PENDING with url https://10.20.13.93:8080/job/merlin_models/321/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/433/*:refs/remotes/origin/pr/433/* # timeout=10
 > git rev-parse e8458a915cb18f7476589542a5200bd030bc7d47^{commit} # timeout=10
Checking out Revision e8458a915cb18f7476589542a5200bd030bc7d47 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f e8458a915cb18f7476589542a5200bd030bc7d47 # timeout=10
Commit message: "Remove normalize_regression_targets"
 > git rev-list --no-walk 1b219e966b6fff264c7dd9c70c1f5417918f416a # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins8138816030827708821.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.1)
Requirement already satisfied: traitlets>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.2.1.post0)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.5.1)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (23.0.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
error: cannot open .git/FETCH_HEAD: Permission denied
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script  : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log" 
[merlin_models] $ /bin/bash /tmp/jenkins418753807904372530.sh

nvidia-merlin-bot · 2022-05-19T14:45:13Z

Click to view CI Results

GitHub pull request #433 of commit dc40488ed08b51ad48a616b6a2b6616d3905b2a1, has merge conflicts.
Running as SYSTEM
Setting status of dc40488ed08b51ad48a616b6a2b6616d3905b2a1 to PENDING with url https://10.20.13.93:8080/job/merlin_models/322/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/433/*:refs/remotes/origin/pr/433/* # timeout=10
 > git rev-parse dc40488ed08b51ad48a616b6a2b6616d3905b2a1^{commit} # timeout=10
Checking out Revision dc40488ed08b51ad48a616b6a2b6616d3905b2a1 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f dc40488ed08b51ad48a616b6a2b6616d3905b2a1 # timeout=10
Commit message: "Add reg:squarederror to supported objectives"
 > git rev-list --no-walk e8458a915cb18f7476589542a5200bd030bc7d47 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins1022333672186530410.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.1)
Requirement already satisfied: traitlets>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.2.1.post0)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.5.1)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (23.0.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
error: cannot open .git/FETCH_HEAD: Permission denied
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script  : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log" 
[merlin_models] $ /bin/bash /tmp/jenkins6907309490978807341.sh

nvidia-merlin-bot · 2022-05-19T15:06:46Z

Click to view CI Results

GitHub pull request #433 of commit 311c8daabf81970c5fbb809dbafea324ade42ef6, no merge conflicts.
Running as SYSTEM
Setting status of 311c8daabf81970c5fbb809dbafea324ade42ef6 to PENDING with url https://10.20.13.93:8080/job/merlin_models/323/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/433/*:refs/remotes/origin/pr/433/* # timeout=10
 > git rev-parse 311c8daabf81970c5fbb809dbafea324ade42ef6^{commit} # timeout=10
Checking out Revision 311c8daabf81970c5fbb809dbafea324ade42ef6 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 311c8daabf81970c5fbb809dbafea324ade42ef6 # timeout=10
Commit message: "Add reg:squarederror to supported objectives"
 > git rev-list --no-walk dc40488ed08b51ad48a616b6a2b6616d3905b2a1 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins3460487890979102979.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: traitlets>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.2.1.post0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.1)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.5.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (23.0.0)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 433 items / 2 skipped
tests/unit/config/test_schema.py ....                                    [  0%]

tests/unit/datasets/test_advertising.py .s                               [  1%]

tests/unit/datasets/test_ecommerce.py ..sss                              [  2%]

tests/unit/datasets/test_entertainment.py ....sss.                       [  4%]

tests/unit/datasets/test_social.py .                                     [  4%]

tests/unit/datasets/test_synthetic.py .....                              [  5%]

tests/unit/tf/test_core.py ...........                                   [  8%]

tests/unit/tf/test_dataset.py ...............                            [ 11%]

tests/unit/tf/test_public_api.py .                                       [ 12%]

tests/unit/tf/blocks/test_cross.py ...........                           [ 14%]

tests/unit/tf/blocks/test_dlrm.py ........                               [ 16%]

tests/unit/tf/blocks/test_interactions.py .                              [ 16%]

tests/unit/tf/blocks/test_mlp.py .............................           [ 23%]

tests/unit/tf/blocks/core/test_aggregation.py .........                  [ 25%]

tests/unit/tf/blocks/core/test_base.py ...                               [ 26%]

tests/unit/tf/blocks/core/test_index.py ...                              [ 26%]

tests/unit/tf/blocks/core/test_masking.py .......                        [ 28%]

tests/unit/tf/blocks/core/test_transformations.py ...........            [ 30%]

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py ..           [ 31%]

tests/unit/tf/blocks/retrieval/test_two_tower.py ...........             [ 33%]

tests/unit/tf/examples/test_01_getting_started.py .                      [ 34%]

tests/unit/tf/examples/test_02_dataschema.py .                           [ 34%]

tests/unit/tf/examples/test_03_exploring_different_models.py .           [ 34%]

tests/unit/tf/examples/test_04_export_ranking_models.py .                [ 34%]

tests/unit/tf/examples/test_05_export_retrieval_model.py .               [ 35%]

tests/unit/tf/examples/test_06_advanced_own_architecture.py .            [ 35%]

tests/unit/tf/features/test_continuous.py .....                          [ 36%]

tests/unit/tf/features/test_embedding.py ............s.                  [ 39%]

tests/unit/tf/features/test_tabular.py .......                           [ 41%]

tests/unit/tf/layers/test_queue.py ..............                        [ 44%]

tests/unit/tf/losses/test_losses.py .......................              [ 49%]

tests/unit/tf/metrics/test_metrics_popularity.py .....                   [ 51%]

tests/unit/tf/metrics/test_metrics_ranking.py .................          [ 54%]

tests/unit/tf/models/test_base.py .....                                  [ 56%]

tests/unit/tf/models/test_benchmark.py ..                                [ 56%]

tests/unit/tf/models/test_ranking.py ..............                      [ 59%]

tests/unit/tf/models/test_retrieval.py ..................                [ 63%]

tests/unit/tf/prediction_tasks/test_classification.py ..                 [ 64%]

tests/unit/tf/prediction_tasks/test_multi_task.py .......                [ 66%]

tests/unit/tf/prediction_tasks/test_next_item.py ......................  [ 71%]

tests/unit/tf/prediction_tasks/test_regression.py ..                     [ 71%]

tests/unit/tf/prediction_tasks/test_sampling.py ....................     [ 76%]

tests/unit/tf/utils/test_batch.py ....                                   [ 77%]

tests/unit/torch/test_dataset.py .........                               [ 79%]

tests/unit/torch/test_public_api.py .                                    [ 79%]

tests/unit/torch/block/test_base.py ....                                 [ 80%]

tests/unit/torch/block/test_mlp.py .                                     [ 80%]

tests/unit/torch/features/test_continuous.py ..                          [ 81%]

tests/unit/torch/features/test_embedding.py ..............               [ 84%]

tests/unit/torch/features/test_tabular.py ....                           [ 85%]

tests/unit/torch/model/test_head.py ............                         [ 87%]

tests/unit/torch/model/test_model.py ..                                  [ 88%]

tests/unit/torch/tabular/test_aggregation.py ........                    [ 90%]

tests/unit/torch/tabular/test_tabular.py ...                             [ 90%]

tests/unit/torch/tabular/test_transformations.py .......                 [ 92%]

tests/unit/utils/test_schema_utils.py ................................   [100%]
=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

if hasattr(pil_image, 'HAMMING'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

_PIL_INTERPOLATION_METHODS['hamming'] = pil_image.HAMMING
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

if hasattr(pil_image, 'BOX'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

_PIL_INTERPOLATION_METHODS['box'] = pil_image.BOX
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

if hasattr(pil_image, 'LANCZOS'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

_PIL_INTERPOLATION_METHODS['lanczos'] = pil_image.LANCZOS
../../../../../usr/lib/python3/dist-packages/requests/init.py:89

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

interpolation: int = Image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

interpolation: int = Image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11

/usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: libtorch_cuda_cu.so: cannot open shared object file: No such file or directory

warn(f"Failed to load image Python extension: {e}")
tests/unit/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/unit/tf/test_dataset.py::test_tf_catname_ordering

tests/unit/tf/test_dataset.py::test_tf_map

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/tf/blocks/core/test_index.py: 6 warnings

tests/unit/tf/models/test_retrieval.py: 57 warnings

tests/unit/tf/prediction_tasks/test_next_item.py: 79 warnings

tests/unit/tf/utils/test_batch.py: 4 warnings

/tmp/tmps10imn36.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag__.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/backend.py:6089: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

warnings.warn('tf.keras.backend.random_binomial is deprecated, '
tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/numpy/core/numeric.py:2453: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.

return bool(asarray(a1 == a2).all())
tests/unit/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)

return {key: torch.tensor(value) for key, value in data.items()}
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========================== short test summary info ============================

SKIPPED [1] tests/unit/implicit/init.py:18: could not import 'implicit': No module named 'implicit'

SKIPPED [1] tests/unit/lightfm/init.py:18: could not import 'lightfm': No module named 'lightfm'

SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/tf/features/test_embedding.py:279: This test fails because cudf.from_dlpack() we use internally in EmbeddingFeatures().embedding_table_dataset() does not work properly. This test should be enabled when the following cudf issue we reported is resolved: rapidsai/cudf#10754

========== 425 passed, 10 skipped, 180 warnings in 1122.39s (0:18:42) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins6305934780516539140.sh

nvidia-merlin-bot · 2022-05-19T15:42:34Z

Click to view CI Results

GitHub pull request #433 of commit 1696d4a0a3ef9ecdce33a0bd2a0dd4a9e08b244f, no merge conflicts.
Running as SYSTEM
Setting status of 1696d4a0a3ef9ecdce33a0bd2a0dd4a9e08b244f to PENDING with url https://10.20.13.93:8080/job/merlin_models/325/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/433/*:refs/remotes/origin/pr/433/* # timeout=10
 > git rev-parse 1696d4a0a3ef9ecdce33a0bd2a0dd4a9e08b244f^{commit} # timeout=10
Checking out Revision 1696d4a0a3ef9ecdce33a0bd2a0dd4a9e08b244f (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 1696d4a0a3ef9ecdce33a0bd2a0dd4a9e08b244f # timeout=10
Commit message: "Add xgboost workflow"
 > git rev-list --no-walk 6c9df6420b789ef3d2be1b0b18e494d9a175d852 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins293250871120014381.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: traitlets>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.2.1.post0)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.5.1)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (23.0.0)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 433 items / 3 skipped
tests/unit/config/test_schema.py ....                                    [  0%]

tests/unit/datasets/test_advertising.py .s                               [  1%]

tests/unit/datasets/test_ecommerce.py ..sss                              [  2%]

tests/unit/datasets/test_entertainment.py ....sss.                       [  4%]

tests/unit/datasets/test_social.py .                                     [  4%]

tests/unit/datasets/test_synthetic.py .....                              [  5%]

tests/unit/tf/test_core.py ...........                                   [  8%]

tests/unit/tf/test_dataset.py ...............                            [ 11%]

tests/unit/tf/test_public_api.py .                                       [ 12%]

tests/unit/tf/blocks/test_cross.py ...........                           [ 14%]

tests/unit/tf/blocks/test_dlrm.py ........                               [ 16%]

tests/unit/tf/blocks/test_interactions.py .                              [ 16%]

tests/unit/tf/blocks/test_mlp.py .............................           [ 23%]

tests/unit/tf/blocks/core/test_aggregation.py .........                  [ 25%]

tests/unit/tf/blocks/core/test_base.py ...                               [ 26%]

tests/unit/tf/blocks/core/test_index.py ...                              [ 26%]

tests/unit/tf/blocks/core/test_masking.py .......                        [ 28%]

tests/unit/tf/blocks/core/test_transformations.py ...........            [ 30%]

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py ..           [ 31%]

tests/unit/tf/blocks/retrieval/test_two_tower.py ...........             [ 33%]

tests/unit/tf/examples/test_01_getting_started.py .                      [ 34%]

tests/unit/tf/examples/test_02_dataschema.py .                           [ 34%]

tests/unit/tf/examples/test_03_exploring_different_models.py .           [ 34%]

tests/unit/tf/examples/test_04_export_ranking_models.py .                [ 34%]

tests/unit/tf/examples/test_05_export_retrieval_model.py .               [ 35%]

tests/unit/tf/examples/test_06_advanced_own_architecture.py .            [ 35%]

tests/unit/tf/features/test_continuous.py .....                          [ 36%]

tests/unit/tf/features/test_embedding.py ............s.                  [ 39%]

tests/unit/tf/features/test_tabular.py .......                           [ 41%]

tests/unit/tf/layers/test_queue.py ..............                        [ 44%]

tests/unit/tf/losses/test_losses.py .......................              [ 49%]

tests/unit/tf/metrics/test_metrics_popularity.py .....                   [ 51%]

tests/unit/tf/metrics/test_metrics_ranking.py .................          [ 54%]

tests/unit/tf/models/test_base.py .....                                  [ 56%]

tests/unit/tf/models/test_benchmark.py ..                                [ 56%]

tests/unit/tf/models/test_ranking.py ..............                      [ 59%]

tests/unit/tf/models/test_retrieval.py ..................                [ 63%]

tests/unit/tf/prediction_tasks/test_classification.py ..                 [ 64%]

tests/unit/tf/prediction_tasks/test_multi_task.py .......                [ 66%]

tests/unit/tf/prediction_tasks/test_next_item.py ......................  [ 71%]

tests/unit/tf/prediction_tasks/test_regression.py ..                     [ 71%]

tests/unit/tf/prediction_tasks/test_sampling.py ....................     [ 76%]

tests/unit/tf/utils/test_batch.py ....                                   [ 77%]

tests/unit/torch/test_dataset.py .........                               [ 79%]

tests/unit/torch/test_public_api.py .                                    [ 79%]

tests/unit/torch/block/test_base.py ....                                 [ 80%]

tests/unit/torch/block/test_mlp.py .                                     [ 80%]

tests/unit/torch/features/test_continuous.py ..                          [ 81%]

tests/unit/torch/features/test_embedding.py ..............               [ 84%]

tests/unit/torch/features/test_tabular.py ....                           [ 85%]

tests/unit/torch/model/test_head.py ............                         [ 87%]

tests/unit/torch/model/test_model.py ..                                  [ 88%]

tests/unit/torch/tabular/test_aggregation.py ........                    [ 90%]

tests/unit/torch/tabular/test_tabular.py ...                             [ 90%]

tests/unit/torch/tabular/test_transformations.py .......                 [ 92%]

tests/unit/utils/test_schema_utils.py ................................   [100%]
=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

if hasattr(pil_image, 'HAMMING'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

_PIL_INTERPOLATION_METHODS['hamming'] = pil_image.HAMMING
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

if hasattr(pil_image, 'BOX'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

_PIL_INTERPOLATION_METHODS['box'] = pil_image.BOX
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

if hasattr(pil_image, 'LANCZOS'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

_PIL_INTERPOLATION_METHODS['lanczos'] = pil_image.LANCZOS
../../../../../usr/lib/python3/dist-packages/requests/init.py:89

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

interpolation: int = Image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

interpolation: int = Image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11

/usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: libtorch_cuda_cu.so: cannot open shared object file: No such file or directory

warn(f"Failed to load image Python extension: {e}")
tests/unit/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/unit/tf/test_dataset.py::test_tf_catname_ordering

tests/unit/tf/test_dataset.py::test_tf_map

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/tf/blocks/core/test_index.py: 6 warnings

tests/unit/tf/models/test_retrieval.py: 57 warnings

tests/unit/tf/prediction_tasks/test_next_item.py: 79 warnings

tests/unit/tf/utils/test_batch.py: 4 warnings

/tmp/tmpc0j8d2tk.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag__.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/backend.py:6089: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

warnings.warn('tf.keras.backend.random_binomial is deprecated, '
tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/numpy/core/numeric.py:2453: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.

return bool(asarray(a1 == a2).all())
tests/unit/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)

return {key: torch.tensor(value) for key, value in data.items()}
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========================== short test summary info ============================

SKIPPED [1] tests/unit/implicit/init.py:18: could not import 'implicit': No module named 'implicit'

SKIPPED [1] tests/unit/lightfm/init.py:18: could not import 'lightfm': No module named 'lightfm'

SKIPPED [1] tests/unit/xgb/init.py:19: could not import 'xgboost': No module named 'xgboost'

SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/tf/features/test_embedding.py:279: This test fails because cudf.from_dlpack() we use internally in EmbeddingFeatures().embedding_table_dataset() does not work properly. This test should be enabled when the following cudf issue we reported is resolved: rapidsai/cudf#10754

========== 425 passed, 11 skipped, 180 warnings in 1117.88s (0:18:37) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins4350387753414158838.sh

nvidia-merlin-bot · 2022-05-19T19:40:30Z

Click to view CI Results

GitHub pull request #433 of commit a2ad78e00a6241acfd50ceff37fc1a1fa471d5e9, no merge conflicts.
Running as SYSTEM
Setting status of a2ad78e00a6241acfd50ceff37fc1a1fa471d5e9 to PENDING with url https://10.20.13.93:8080/job/merlin_models/331/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/433/*:refs/remotes/origin/pr/433/* # timeout=10
 > git rev-parse a2ad78e00a6241acfd50ceff37fc1a1fa471d5e9^{commit} # timeout=10
Checking out Revision a2ad78e00a6241acfd50ceff37fc1a1fa471d5e9 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f a2ad78e00a6241acfd50ceff37fc1a1fa471d5e9 # timeout=10
Commit message: "Merge branch 'main' into xgb"
 > git rev-list --no-walk 139dfc0c3aa8a8bde12b8f25f905a1070ba998af # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins6593688322829998157.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: traitlets>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.2.1.post0)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.1)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.5.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (23.0.0)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 433 items / 3 skipped
tests/unit/config/test_schema.py ....                                    [  0%]

tests/unit/datasets/test_advertising.py .s                               [  1%]

tests/unit/datasets/test_ecommerce.py ..sss                              [  2%]

tests/unit/datasets/test_entertainment.py ....sss.                       [  4%]

tests/unit/datasets/test_social.py .                                     [  4%]

tests/unit/datasets/test_synthetic.py .....                              [  5%]

tests/unit/tf/test_core.py ...........                                   [  8%]

tests/unit/tf/test_dataset.py ...............                            [ 11%]

tests/unit/tf/test_public_api.py .                                       [ 12%]

tests/unit/tf/blocks/test_cross.py ...........                           [ 14%]

tests/unit/tf/blocks/test_dlrm.py ........                               [ 16%]

tests/unit/tf/blocks/test_interactions.py .                              [ 16%]

tests/unit/tf/blocks/test_mlp.py .............................           [ 23%]

tests/unit/tf/blocks/core/test_aggregation.py .........                  [ 25%]

tests/unit/tf/blocks/core/test_base.py ...                               [ 26%]

tests/unit/tf/blocks/core/test_index.py ...                              [ 26%]

tests/unit/tf/blocks/core/test_masking.py .......                        [ 28%]

tests/unit/tf/blocks/core/test_transformations.py ...........            [ 30%]

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py ..           [ 31%]

tests/unit/tf/blocks/retrieval/test_two_tower.py ...........             [ 33%]

tests/unit/tf/examples/test_01_getting_started.py .                      [ 34%]

tests/unit/tf/examples/test_02_dataschema.py .                           [ 34%]

tests/unit/tf/examples/test_03_exploring_different_models.py .           [ 34%]

tests/unit/tf/examples/test_04_export_ranking_models.py .                [ 34%]

tests/unit/tf/examples/test_05_export_retrieval_model.py .               [ 35%]

tests/unit/tf/examples/test_06_advanced_own_architecture.py .            [ 35%]

tests/unit/tf/features/test_continuous.py .....                          [ 36%]

tests/unit/tf/features/test_embedding.py ............s.                  [ 39%]

tests/unit/tf/features/test_tabular.py .......                           [ 41%]

tests/unit/tf/layers/test_queue.py ..............                        [ 44%]

tests/unit/tf/losses/test_losses.py .......................              [ 49%]

tests/unit/tf/metrics/test_metrics_popularity.py .....                   [ 51%]

tests/unit/tf/metrics/test_metrics_ranking.py .................          [ 54%]

tests/unit/tf/models/test_base.py .....                                  [ 56%]

tests/unit/tf/models/test_benchmark.py ..                                [ 56%]

tests/unit/tf/models/test_ranking.py ..............                      [ 59%]

tests/unit/tf/models/test_retrieval.py ..................                [ 63%]

tests/unit/tf/prediction_tasks/test_classification.py ..                 [ 64%]

tests/unit/tf/prediction_tasks/test_multi_task.py .......                [ 66%]

tests/unit/tf/prediction_tasks/test_next_item.py ......................  [ 71%]

tests/unit/tf/prediction_tasks/test_regression.py ..                     [ 71%]

tests/unit/tf/prediction_tasks/test_sampling.py ....................     [ 76%]

tests/unit/tf/utils/test_batch.py ....                                   [ 77%]

tests/unit/torch/test_dataset.py .........                               [ 79%]

tests/unit/torch/test_public_api.py .                                    [ 79%]

tests/unit/torch/block/test_base.py ....                                 [ 80%]

tests/unit/torch/block/test_mlp.py .                                     [ 80%]

tests/unit/torch/features/test_continuous.py ..                          [ 81%]

tests/unit/torch/features/test_embedding.py ..............               [ 84%]

tests/unit/torch/features/test_tabular.py ....                           [ 85%]

tests/unit/torch/model/test_head.py ............                         [ 87%]

tests/unit/torch/model/test_model.py ..                                  [ 88%]

tests/unit/torch/tabular/test_aggregation.py ........                    [ 90%]

tests/unit/torch/tabular/test_tabular.py ...                             [ 90%]

tests/unit/torch/tabular/test_transformations.py ...F...                 [ 92%]

tests/unit/utils/test_schema_utils.py ................................   [100%]
=================================== FAILURES ===================================

_______________________ test_stochastic_swap_noise[0.7] ________________________
replacement_prob = 0.7
@pytest.mark.parametrize("replacement_prob", [0.1, 0.3, 0.5, 0.7])
def test_stochastic_swap_noise(replacement_prob):
    NUM_SEQS = 100
    SEQ_LENGTH = 80
    PAD_TOKEN = 0

    # Creating some input sequences with padding in the end
    # (to emulate sessions with different lengths)
    seq_inputs = {
        "categ_seq_feat": torch.tril(
            torch.randint(low=1, high=100, size=(NUM_SEQS, SEQ_LENGTH)), 1
        ),
        "cont_seq_feat": torch.tril(torch.rand((NUM_SEQS, SEQ_LENGTH)), 1),
        "categ_non_seq_feat": torch.randint(low=1, high=100, size=(NUM_SEQS,)),
    }

    ssn = ml.StochasticSwapNoise(pad_token=PAD_TOKEN, replacement_prob=replacement_prob)
    out_features_ssn = ssn(seq_inputs, input_mask=seq_inputs["categ_seq_feat"] != PAD_TOKEN)

    for fname in seq_inputs:
        replaced_mask = out_features_ssn[fname] != seq_inputs[fname]
        replaced_mask_non_padded = torch.masked_select(
            replaced_mask, seq_inputs[fname] != PAD_TOKEN
        )
        replacement_rate = replaced_mask_non_padded.float().mean()


      assert replacement_rate == pytest.approx(replacement_prob, abs=0.15)


E           assert tensor(0.5400) == 0.7 ± 1.5e-01

E             comparison failed

E             Obtained: 0.5400000214576721

E             Expected: 0.7 ± 1.5e-01
tests/unit/torch/tabular/test_transformations.py:50: AssertionError

=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

if hasattr(pil_image, 'HAMMING'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

_PIL_INTERPOLATION_METHODS['hamming'] = pil_image.HAMMING
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

if hasattr(pil_image, 'BOX'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

_PIL_INTERPOLATION_METHODS['box'] = pil_image.BOX
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

if hasattr(pil_image, 'LANCZOS'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

_PIL_INTERPOLATION_METHODS['lanczos'] = pil_image.LANCZOS
../../../../../usr/lib/python3/dist-packages/requests/init.py:89

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

interpolation: int = Image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

interpolation: int = Image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11

/usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: libtorch_cuda_cu.so: cannot open shared object file: No such file or directory

warn(f"Failed to load image Python extension: {e}")
tests/unit/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/unit/tf/test_dataset.py::test_tf_catname_ordering

tests/unit/tf/test_dataset.py::test_tf_map

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/tf/blocks/core/test_index.py: 6 warnings

tests/unit/tf/models/test_retrieval.py: 57 warnings

tests/unit/tf/prediction_tasks/test_next_item.py: 79 warnings

tests/unit/tf/utils/test_batch.py: 4 warnings

/tmp/tmpcgjpuw6d.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag__.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/backend.py:6089: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

warnings.warn('tf.keras.backend.random_binomial is deprecated, '
tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/numpy/core/numeric.py:2453: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.

return bool(asarray(a1 == a2).all())
tests/unit/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)

return {key: torch.tensor(value) for key, value in data.items()}
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========================== short test summary info ============================

SKIPPED [1] tests/unit/implicit/init.py:18: could not import 'implicit': No module named 'implicit'

SKIPPED [1] tests/unit/lightfm/init.py:18: could not import 'lightfm': No module named 'lightfm'

SKIPPED [1] tests/unit/xgb/init.py:19: could not import 'xgboost': No module named 'xgboost'

SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/tf/features/test_embedding.py:279: This test fails because cudf.from_dlpack() we use internally in EmbeddingFeatures().embedding_table_dataset() does not work properly. This test should be enabled when the following cudf issue we reported is resolved: rapidsai/cudf#10754

===== 1 failed, 424 passed, 11 skipped, 180 warnings in 1115.85s (0:18:35) =====

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins5869926415841801861.sh

nvidia-merlin-bot · 2022-05-20T14:24:16Z

Click to view CI Results

GitHub pull request #433 of commit 4562785bb282e7ab3e8be8416f44b7790f6551c8, no merge conflicts.
Running as SYSTEM
Setting status of 4562785bb282e7ab3e8be8416f44b7790f6551c8 to PENDING with url https://10.20.13.93:8080/job/merlin_models/340/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/433/*:refs/remotes/origin/pr/433/* # timeout=10
 > git rev-parse 4562785bb282e7ab3e8be8416f44b7790f6551c8^{commit} # timeout=10
Checking out Revision 4562785bb282e7ab3e8be8416f44b7790f6551c8 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 4562785bb282e7ab3e8be8416f44b7790f6551c8 # timeout=10
Commit message: "Merge branch 'main' into xgb"
 > git rev-list --no-walk ebce3a880525bd2f29ad8e41e88d79f5d2c53c42 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins1902381881922309156.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: traitlets>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.2.1.post0)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.1)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.5.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (23.0.0)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 431 items / 3 skipped
tests/unit/config/test_schema.py ....                                    [  0%]

tests/unit/datasets/test_advertising.py .s                               [  1%]

tests/unit/datasets/test_ecommerce.py ..sss                              [  2%]

tests/unit/datasets/test_entertainment.py ....sss.                       [  4%]

tests/unit/datasets/test_social.py .                                     [  4%]

tests/unit/datasets/test_synthetic.py .....                              [  5%]

tests/unit/tf/test_core.py .........                                     [  7%]

tests/unit/tf/test_dataset.py ...............                            [ 11%]

tests/unit/tf/test_public_api.py .                                       [ 11%]

tests/unit/tf/blocks/test_cross.py ...........                           [ 14%]

tests/unit/tf/blocks/test_dlrm.py ........                               [ 16%]

tests/unit/tf/blocks/test_interactions.py .                              [ 16%]

tests/unit/tf/blocks/test_mlp.py .............................           [ 22%]

tests/unit/tf/blocks/core/test_aggregation.py .........                  [ 25%]

tests/unit/tf/blocks/core/test_base.py ..                                [ 25%]

tests/unit/tf/blocks/core/test_index.py ...                              [ 26%]

tests/unit/tf/blocks/core/test_masking.py .......                        [ 27%]

tests/unit/tf/blocks/core/test_transformations.py ...........            [ 30%]

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py ..           [ 30%]

tests/unit/tf/blocks/retrieval/test_two_tower.py ...........             [ 33%]

tests/unit/tf/examples/test_01_getting_started.py .                      [ 33%]

tests/unit/tf/examples/test_02_dataschema.py .                           [ 33%]

tests/unit/tf/examples/test_03_exploring_different_models.py .           [ 34%]

tests/unit/tf/examples/test_04_export_ranking_models.py .                [ 34%]

tests/unit/tf/examples/test_05_export_retrieval_model.py .               [ 34%]

tests/unit/tf/examples/test_06_advanced_own_architecture.py .            [ 34%]

tests/unit/tf/features/test_continuous.py .....                          [ 35%]

tests/unit/tf/features/test_embedding.py ..............                  [ 39%]

tests/unit/tf/features/test_tabular.py .......                           [ 40%]

tests/unit/tf/layers/test_queue.py ..............                        [ 44%]

tests/unit/tf/losses/test_losses.py .......................              [ 49%]

tests/unit/tf/metrics/test_metrics_popularity.py .....                   [ 50%]

tests/unit/tf/metrics/test_metrics_ranking.py .................          [ 54%]

tests/unit/tf/models/test_base.py ........                               [ 56%]

tests/unit/tf/models/test_benchmark.py ..                                [ 56%]

tests/unit/tf/models/test_ranking.py ..............                      [ 60%]

tests/unit/tf/models/test_retrieval.py ..................                [ 64%]

tests/unit/tf/prediction_tasks/test_classification.py ..                 [ 64%]

tests/unit/tf/prediction_tasks/test_multi_task.py .......                [ 66%]

tests/unit/tf/prediction_tasks/test_next_item.py ....................    [ 70%]

tests/unit/tf/prediction_tasks/test_regression.py ..                     [ 71%]

tests/unit/tf/prediction_tasks/test_sampling.py ....................     [ 76%]

tests/unit/tf/utils/test_batch.py ....                                   [ 77%]

tests/unit/torch/test_dataset.py .........                               [ 79%]

tests/unit/torch/test_public_api.py .                                    [ 79%]

tests/unit/torch/block/test_base.py ....                                 [ 80%]

tests/unit/torch/block/test_mlp.py .                                     [ 80%]

tests/unit/torch/features/test_continuous.py ..                          [ 80%]

tests/unit/torch/features/test_embedding.py ..............               [ 84%]

tests/unit/torch/features/test_tabular.py ....                           [ 85%]

tests/unit/torch/model/test_head.py ............                         [ 87%]

tests/unit/torch/model/test_model.py ..                                  [ 88%]

tests/unit/torch/tabular/test_aggregation.py ........                    [ 90%]

tests/unit/torch/tabular/test_tabular.py ...                             [ 90%]

tests/unit/torch/tabular/test_transformations.py .......                 [ 92%]

tests/unit/utils/test_schema_utils.py ................................   [100%]
=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

if hasattr(pil_image, 'HAMMING'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

_PIL_INTERPOLATION_METHODS['hamming'] = pil_image.HAMMING
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

if hasattr(pil_image, 'BOX'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

_PIL_INTERPOLATION_METHODS['box'] = pil_image.BOX
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

if hasattr(pil_image, 'LANCZOS'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

_PIL_INTERPOLATION_METHODS['lanczos'] = pil_image.LANCZOS
../../../../../usr/lib/python3/dist-packages/requests/init.py:89

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

interpolation: int = Image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

interpolation: int = Image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11

/usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: libtorch_cuda_cu.so: cannot open shared object file: No such file or directory

warn(f"Failed to load image Python extension: {e}")
tests/unit/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/unit/tf/test_dataset.py::test_tf_catname_ordering

tests/unit/tf/test_dataset.py::test_tf_map

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/tf/blocks/core/test_index.py: 6 warnings

tests/unit/tf/models/test_retrieval.py: 57 warnings

tests/unit/tf/prediction_tasks/test_next_item.py: 88 warnings

tests/unit/tf/utils/test_batch.py: 4 warnings

/tmp/tmp9tr27idh.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag__.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/backend.py:6089: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

warnings.warn('tf.keras.backend.random_binomial is deprecated, '
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/features/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/features/embedding.py:321: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack

embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/numpy/core/numeric.py:2453: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.

return bool(asarray(a1 == a2).all())
tests/unit/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)

return {key: torch.tensor(value) for key, value in data.items()}
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========================== short test summary info ============================

SKIPPED [1] tests/unit/implicit/init.py:18: could not import 'implicit': No module named 'implicit'

SKIPPED [1] tests/unit/lightfm/init.py:18: could not import 'lightfm': No module named 'lightfm'

SKIPPED [1] tests/unit/xgb/init.py:19: could not import 'xgboost': No module named 'xgboost'

SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR

========== 424 passed, 10 skipped, 194 warnings in 1063.37s (0:17:43) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins1796311984013042892.sh

marcromeyn · 2022-05-23T07:13:55Z

For the xgboost pytest marker, you should register it here.

nvidia-merlin-bot · 2022-05-23T07:32:11Z

Click to view CI Results

GitHub pull request #433 of commit d6609462893472fbac0e0add969c7e8efa33d224, no merge conflicts.
Running as SYSTEM
Setting status of d6609462893472fbac0e0add969c7e8efa33d224 to PENDING with url https://10.20.13.93:8080/job/merlin_models/341/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/433/*:refs/remotes/origin/pr/433/* # timeout=10
 > git rev-parse d6609462893472fbac0e0add969c7e8efa33d224^{commit} # timeout=10
Checking out Revision d6609462893472fbac0e0add969c7e8efa33d224 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f d6609462893472fbac0e0add969c7e8efa33d224 # timeout=10
Commit message: "Add test for rank objective"
 > git rev-list --no-walk 4562785bb282e7ab3e8be8416f44b7790f6551c8 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins2893444168385125490.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting testbook
  Downloading testbook-0.4.2-py3-none-any.whl (11 kB)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.1)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: traitlets>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.2.1.post0)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.5.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (23.0.0)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
Installing collected packages: testbook
Successfully installed testbook-0.4.2
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 431 items / 3 skipped
tests/unit/config/test_schema.py ....                                    [  0%]

tests/unit/datasets/test_advertising.py .s                               [  1%]

tests/unit/datasets/test_ecommerce.py ..sss                              [  2%]

tests/unit/datasets/test_entertainment.py ....sss.                       [  4%]

tests/unit/datasets/test_social.py .                                     [  4%]

tests/unit/datasets/test_synthetic.py .....                              [  5%]

tests/unit/tf/test_core.py .........                                     [  7%]

tests/unit/tf/test_dataset.py ...............                            [ 11%]

tests/unit/tf/test_public_api.py .                                       [ 11%]

tests/unit/tf/blocks/test_cross.py ...........                           [ 14%]

tests/unit/tf/blocks/test_dlrm.py ........                               [ 16%]

tests/unit/tf/blocks/test_interactions.py .                              [ 16%]

tests/unit/tf/blocks/test_mlp.py .............................           [ 22%]

tests/unit/tf/blocks/core/test_aggregation.py .........                  [ 25%]

tests/unit/tf/blocks/core/test_base.py ..                                [ 25%]

tests/unit/tf/blocks/core/test_index.py ...                              [ 26%]

tests/unit/tf/blocks/core/test_masking.py .......                        [ 27%]

tests/unit/tf/blocks/core/test_transformations.py ...........            [ 30%]

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py ..           [ 30%]

tests/unit/tf/blocks/retrieval/test_two_tower.py ...........             [ 33%]

tests/unit/tf/examples/test_01_getting_started.py .                      [ 33%]

tests/unit/tf/examples/test_02_dataschema.py .                           [ 33%]

tests/unit/tf/examples/test_03_exploring_different_models.py .           [ 34%]

tests/unit/tf/examples/test_04_export_ranking_models.py .                [ 34%]

tests/unit/tf/examples/test_05_export_retrieval_model.py .               [ 34%]

tests/unit/tf/examples/test_06_advanced_own_architecture.py .            [ 34%]

tests/unit/tf/features/test_continuous.py .....                          [ 35%]

tests/unit/tf/features/test_embedding.py ..............                  [ 39%]

tests/unit/tf/features/test_tabular.py .......                           [ 40%]

tests/unit/tf/layers/test_queue.py ..............                        [ 44%]

tests/unit/tf/losses/test_losses.py .......................              [ 49%]

tests/unit/tf/metrics/test_metrics_popularity.py .....                   [ 50%]

tests/unit/tf/metrics/test_metrics_ranking.py .................          [ 54%]

tests/unit/tf/models/test_base.py ........                               [ 56%]

tests/unit/tf/models/test_benchmark.py ..                                [ 56%]

tests/unit/tf/models/test_ranking.py ..............                      [ 60%]

tests/unit/tf/models/test_retrieval.py ..................                [ 64%]

tests/unit/tf/prediction_tasks/test_classification.py ..                 [ 64%]

tests/unit/tf/prediction_tasks/test_multi_task.py .......                [ 66%]

tests/unit/tf/prediction_tasks/test_next_item.py ....................    [ 70%]

tests/unit/tf/prediction_tasks/test_regression.py ..                     [ 71%]

tests/unit/tf/prediction_tasks/test_sampling.py ....................     [ 76%]

tests/unit/tf/utils/test_batch.py ....                                   [ 77%]

tests/unit/torch/test_dataset.py .........                               [ 79%]

tests/unit/torch/test_public_api.py .                                    [ 79%]

tests/unit/torch/block/test_base.py ....                                 [ 80%]

tests/unit/torch/block/test_mlp.py .                                     [ 80%]

tests/unit/torch/features/test_continuous.py ..                          [ 80%]

tests/unit/torch/features/test_embedding.py ..............               [ 84%]

tests/unit/torch/features/test_tabular.py ....                           [ 85%]

tests/unit/torch/model/test_head.py ............                         [ 87%]

tests/unit/torch/model/test_model.py ..                                  [ 88%]

tests/unit/torch/tabular/test_aggregation.py ........                    [ 90%]

tests/unit/torch/tabular/test_tabular.py ...                             [ 90%]

tests/unit/torch/tabular/test_transformations.py .......                 [ 92%]

tests/unit/utils/test_schema_utils.py ................................   [100%]
=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

if hasattr(pil_image, 'HAMMING'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

_PIL_INTERPOLATION_METHODS['hamming'] = pil_image.HAMMING
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

if hasattr(pil_image, 'BOX'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

_PIL_INTERPOLATION_METHODS['box'] = pil_image.BOX
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

if hasattr(pil_image, 'LANCZOS'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

_PIL_INTERPOLATION_METHODS['lanczos'] = pil_image.LANCZOS
../../../../../usr/lib/python3/dist-packages/requests/init.py:89

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

interpolation: int = Image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

interpolation: int = Image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11

/usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: libtorch_cuda_cu.so: cannot open shared object file: No such file or directory

warn(f"Failed to load image Python extension: {e}")
tests/unit/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/unit/tf/test_dataset.py::test_tf_catname_ordering

tests/unit/tf/test_dataset.py::test_tf_map

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/tf/blocks/core/test_index.py: 6 warnings

tests/unit/tf/models/test_retrieval.py: 57 warnings

tests/unit/tf/prediction_tasks/test_next_item.py: 88 warnings

tests/unit/tf/utils/test_batch.py: 4 warnings

/tmp/tmpgbtji84i.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag__.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/backend.py:6089: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

warnings.warn('tf.keras.backend.random_binomial is deprecated, '
tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/features/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/features/embedding.py:321: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack

embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))
tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/numpy/core/numeric.py:2453: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.

return bool(asarray(a1 == a2).all())
tests/unit/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)

return {key: torch.tensor(value) for key, value in data.items()}
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========================== short test summary info ============================

SKIPPED [1] tests/unit/implicit/init.py:18: could not import 'implicit': No module named 'implicit'

SKIPPED [1] tests/unit/lightfm/init.py:18: could not import 'lightfm': No module named 'lightfm'

SKIPPED [1] tests/unit/xgb/init.py:19: could not import 'xgboost': No module named 'xgboost'

SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR

========== 424 passed, 10 skipped, 194 warnings in 1071.71s (0:17:51) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins7553123863535610091.sh

oliverholworthy added the status/work-in-progress label May 12, 2022

marcromeyn reviewed May 13, 2022

View reviewed changes

merlin/models/xgb/__init__.py Show resolved Hide resolved

marcromeyn reviewed May 13, 2022

View reviewed changes

merlin/models/xgb/__init__.py Outdated Show resolved Hide resolved

marcromeyn reviewed May 13, 2022

View reviewed changes

merlin/models/xgb/__init__.py Outdated Show resolved Hide resolved

marcromeyn reviewed May 13, 2022

View reviewed changes

merlin/models/xgb/__init__.py Outdated Show resolved Hide resolved

marcromeyn reviewed May 19, 2022

View reviewed changes

merlin/models/xgb/__init__.py Outdated Show resolved Hide resolved

oliverholworthy added 12 commits May 19, 2022 15:45

First sketch of XGBoost integration with NVTabular schema

083ec28

Exclude all targets from input features in XGBoost class

37d3199

Fix import of movielens dataset in XGBoost docstring

0ca91a9

Use objective in constructor and use this to select the target tag

6f61f42

Implement evaluate method using test dataset

35d980e

remove default params

00abb2f

Extract metrics from string

659ddf5

Update docstring for XGBoost

868747d

remove the _to_numpy function

7521456

Add predict method

8d26e6d

Remove scheduler specified in ddf compute

fb4067c

Add tests module for xgboost

a617eef

karlhigley approved these changes May 19, 2022

View reviewed changes

oliverholworthy added 4 commits May 19, 2022 15:47

Add xgboost requirements

8d8db80

Add docstring to XGBoost predict method

fafb865

Add docstring to XGBoost evaluate method

396b83a

Update docsting of XGBoost fit method

4dfc5b4

oliverholworthy added 5 commits May 19, 2022 15:47

Use objective as argument to constructor

1ae3e5a

Add criteo data fixture

b5c288c

Remove normalize_regression_targets

e37a3c9

Add warning about list columns

b4a959a

Add reg:squarederror to supported objectives

311c8da

karlhigley marked this pull request as ready for review May 19, 2022 14:47

oliverholworthy force-pushed the xgb branch from dc40488 to 311c8da Compare May 19, 2022 14:47

karlhigley changed the title ~~[WIP] First sketch of XGBoost integration with Merlin Dataset~~ First sketch of XGBoost integration with Merlin Dataset May 19, 2022

oliverholworthy added 3 commits May 19, 2022 15:51

Update formatting (lint)

268368c

Move xgboost tests to unit subdirectory

bebc4dd

Add xgboost marker and make target

8966098

Add xgboost workflow

1696d4a

Merge branch 'main' into xgb

a2ad78e

Merge branch 'main' into xgb

4562785

oliverholworthy added 3 commits May 20, 2022 17:27

Support rank objectives and allow group columns to be specified

9774759

Update docstring and formatting

fb5836d

Add test for rank objective

d660946

karlhigley merged commit 797958b into NVIDIA-Merlin:main May 24, 2022

karlhigley removed the status/work-in-progress label May 24, 2022

benfred added the enhancement New feature or request label May 26, 2022

oliverholworthy linked an issue Jun 15, 2022 that may be closed by this pull request

XGBoost Integration #132

Closed

oliverholworthy mentioned this pull request Jun 22, 2022

[RMP] Support Tree Ranking Models (like XGBoost) in Merlin Models and Systems NVIDIA-Merlin/Merlin#105

Closed

4 tasks

Conversation

oliverholworthy commented May 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Goals ⚽

Implementation Details 🚧

Testing Details 🔍

Example Usage

Uh oh!

github-actions bot commented May 12, 2022

Documentation preview

Uh oh!

nvidia-merlin-bot commented May 12, 2022

Uh oh!

nvidia-merlin-bot commented May 12, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nvidia-merlin-bot commented May 18, 2022

Uh oh!

nvidia-merlin-bot commented May 18, 2022

Uh oh!

Uh oh!

nvidia-merlin-bot commented May 19, 2022

Uh oh!

nvidia-merlin-bot commented May 19, 2022

Uh oh!

nvidia-merlin-bot commented May 19, 2022

Uh oh!

nvidia-merlin-bot commented May 19, 2022

Uh oh!

nvidia-merlin-bot commented May 19, 2022

Uh oh!

nvidia-merlin-bot commented May 20, 2022

Uh oh!

marcromeyn commented May 23, 2022

Uh oh!

nvidia-merlin-bot commented May 23, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

oliverholworthy commented May 12, 2022 •

edited

Loading