Skip to content

re-write of test flags #232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jph00 merged 5 commits into from
Sep 4, 2020
Merged

re-write of test flags #232

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
846456e
re-write of test flags
pete88b Sep 3, 2020
6084eb2
make import nbdev_callbacks work when building docs
pete88b Sep 3, 2020
c7ec9b9
cleaned up import nbdev_callbacks
pete88b Sep 3, 2020
5ea9712
moved call_cb from imports.py back to test cells in nbs
pete88b Sep 4, 2020
121e3f6
minor change to docs
pete88b Sep 4, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion nbdev/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@
if IN_IPYTHON:
from .flags import *
from .showdoc import show_doc
#from .export import notebook2script
from .export import notebook2script
14 changes: 6 additions & 8 deletions nbdev/flags.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,23 +95,21 @@ def nbdev_add2all(line, local_ns):
try: [eval(s, local_ns) for s in parse_line(line)]
except Exception as ex: print(f'UsageError: {ex}')

"""
def _new_test_flag_fn(flag):
"Create a new test flag function and magic"
# don't create "empty" test flags if tst_flags is not set, set to whitespace, has trailing | etc
if not flag.strip(): return
exec(f'''def nbdev_{flag}_test(line):
"Put an `%nbdev_{flag}_test` magic on each "{flag}" test cell that you do not want to be run by default."
_validate_param(line, 'nbdev_{flag}_test', fixed_value='all')''')
exec(f'sys.modules[__name__].nbdev_{flag}_test = nbdev_{flag}_test')
exec(f'register_line_magic(nbdev_{flag}_test)')
"""
def _(line): _validate_param(line, f'nbdev_{flag}_test', fixed_value='all')
_.__doc__ = f"""Put an `%nbdev_{flag}_test` magic on each "{flag}" test cell that you do not want to be run by default.
To apply this flag to all tests in a notebook, one cell should contain: `%nbdev_{flag}_test all`
These tests can be executed with the command line tool: `nbdev_test_nbs --flags {flag}`'."""
register_line_magic(f'nbdev_{flag}_test')(_)

if IN_IPYTHON:
from IPython.core.magic import register_line_magic, needs_local_scope
fns = [nbdev_default_export, nbdev_export, nbdev_export_and_show, nbdev_export_internal,
nbdev_hide, nbdev_hide_input, nbdev_hide_output, nbdev_default_class_level,
nbdev_collapse_input, nbdev_collapse_output, needs_local_scope(nbdev_add2all)]
for fn in fns: register_line_magic(fn)
#for flag in Config().get('tst_flags', '').split('|'): _new_test_flag_fn(flag)
for flag in Config().get('tst_flags', '').split('|'): _new_test_flag_fn(flag)

2 changes: 1 addition & 1 deletion nbdev/imports.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,4 +84,4 @@ def call_cb(cb_name, *args):
except: nbdev_callbacks={}
finally: sys.path=_sys_path
if not hasattr(nbdev_callbacks, cb_name): return args[0] if args else None
return getattr(nbdev_callbacks, cb_name)(*args)
return getattr(nbdev_callbacks, cb_name)(*args)
30 changes: 12 additions & 18 deletions nbs/04_test.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,37 +251,30 @@
" finally: os.environ.pop(\"IN_TEST\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"%nbdev_hide\n",
"nbdev users should not need to `import nbdev_callbacks` but we need to patch `begin_test_nb` and `after_test_nb` on to the `nbdev_callbacks` module to test that `test_nb` is calling the callbacks properly."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%nbdev_hide\n",
"class TestCallbacks:\n",
" def begin_test_nb(self, nb, file_name, flags):\n",
" self.begin_test_nb_data=dict(nb=nb,file_name=file_name,flags=flags)\n",
" return nb\n",
" def after_test_nb(self, file_name):\n",
" self.after_test_nb_data=dict(file_name=file_name)\n",
"test_callbacks=TestCallbacks()\n",
"call_cb('this makes sure nbdev_callbacks is loaded from the right place')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import nbdev_callbacks"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"call_cb('This makes sure nbdev_callbacks is loaded from the right place')\n",
"import nbdev_callbacks\n",
"original_callbacks=nbdev_callbacks.begin_test_nb,nbdev_callbacks.after_test_nb\n",
"try:\n",
" nbdev_callbacks.begin_test_nb=test_callbacks.begin_test_nb\n",
Expand Down Expand Up @@ -319,6 +312,7 @@
"Converted 03_export2html.ipynb.\n",
"Converted 04_test.ipynb.\n",
"Converted 05_merge.ipynb.\n",
"Converted 05a_conda.ipynb.\n",
"Converted 06_cli.ipynb.\n",
"Converted 07_clean.ipynb.\n",
"Converted 08_flag_tests.ipynb.\n",
Expand Down
63 changes: 49 additions & 14 deletions nbs/08_flag_tests.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Note: Magic flags are being introduced progressively\n",
"## Note: Special comment flags are still supported\n",
"\n",
"In order to explore autocomplete and in-notebook help with magic flags we plan to;\n",
"- start by making `%nbdev_export` available as an alternative to `#export`\n",
"- support both comments and magic flags\n"
"Both comment and magic flags are currently supported and you can use both kinds in the same notebook.\n",
"\n",
"Ideally, comment and magic flags would work the same but ... there is a difference when we `split_flags_and_code`."
]
},
{
Expand Down Expand Up @@ -67,9 +67,7 @@
"source": [
"Once imported, you can explore the available flags and their documentation without leaving your notebook\n",
"\n",
"<img alt=\"Tab completion of nbdev magics\" width=\"900\" src=\"images/inspect_magics.png\" />\n",
"\n",
"Note: In the image above, we show some flags that have not yet been introduced to nbdev."
"<img alt=\"Tab completion of nbdev magics\" width=\"900\" src=\"images/inspect_magics.png\" />"
]
},
{
Expand All @@ -85,6 +83,31 @@
"<pre>UsageError: module_name \"bad module name\" must not contain whitespace</pre>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## How do comment flags correspond to magic flags?\n",
"\n",
"| Comment flag | Magic flag | |\n",
"|----------------------------------------|-----------------------------|--------------------------------------------------------------------------------|\n",
"| `default_exp` | `nbdev_default_export` | Define the name of the module everything should be exported in |\n",
"| `exports` | `nbdev_export_and_show` | Export and show code in the docs |\n",
"| `exporti` | `nbdev_export_internal` | Export but don’t show in docs and don’t add to `__all__` |\n",
"| `export` | `nbdev_export` | Export but don’t show in docs |\n",
"| `hide_input` | `nbdev_hide_input` | Do not show input of a test cell in docs |\n",
"| `hide_output` | `nbdev_hide_output` | Do not show output of a test cell in docs |\n",
"| `hide` | `nbdev_hide` | Do not show a test cell or markdown in docs |\n",
"| `default_cls_lvl` | `nbdev_default_class_level` | Define the default toc level of classes |\n",
"| `collapse_output` or `collapse-output` | `nbdev_collapse_output` | Inlcude output in the docs under a collapsable element |\n",
"| `collapse_show` or `collapse-show` | `nbdev_collapse_input open` | Inlcude intput in the docs under a collapsable element that is open by default |\n",
"| `collapse`, `collapse_hide` or `collapse-hide` | `nbdev_collapse_input` | Inlcude intput in the docs under a collapsable element |\n",
"\n",
"To add something to `__all__`, that is not picked automatically, you can assign a list of names to `_all_` or use the `nbdev_add2all` magic flag.\n",
"\n",
"You can use `show_doc` to show documentation for an element, or `nbdev_show_doc` to show documentation for any number of elements."
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down Expand Up @@ -128,6 +151,24 @@
" test_flag_output(s, 'UsageError: module_name \"bad module name\" must not contain whitespace')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"if IN_IPYTHON:\n",
" def test_flag_output(s, exp): test_eq(s.getvalue().strip(), exp.strip())\n",
" s = io.StringIO()\n",
" with redirect_stdout(s):\n",
" %nbdev_fastai_test\n",
" test_flag_output(s, '')\n",
" %nbdev_fastai_test all\n",
" test_flag_output(s, '')\n",
" %nbdev_fastai_test not_all\n",
" test_flag_output(s, 'UsageError: Invalid option \"not_all\". Usage `%nbdev_fastai_test [all]`')"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down Expand Up @@ -234,13 +275,7 @@
" test_flag_output(nbdev_default_class_level, str(i), '')\n",
"test_flag_output(nbdev_default_class_level, '0', 'UsageError: Invalid class level \"0\". Usage `%nbdev_default_class_level [int between 1 and 6]`')\n",
"test_flag_output(nbdev_default_class_level, '7', 'UsageError: Invalid class level \"7\". Usage `%nbdev_default_class_level [int between 1 and 6]`')\n",
"test_flag_output(nbdev_default_class_level, 'not_a_num', 'UsageError: Invalid class level \"not_a_num\". Usage `%nbdev_default_class_level [int between 1 and 6]`')\n",
"\n",
"# from nbdev.flags import nbdev_fastai_test\n",
"# test_flag_output(nbdev_fastai_test, '', '')\n",
"# test_flag_output(nbdev_fastai_test, ' ', '')\n",
"# test_flag_output(nbdev_fastai_test, 'all', '')\n",
"# test_flag_output(nbdev_fastai_test, 'All', 'UsageError: Invalid option \"All\". Usage `%nbdev_fastai_test [all]`')"
"test_flag_output(nbdev_default_class_level, 'not_a_num', 'UsageError: Invalid class level \"not_a_num\". Usage `%nbdev_default_class_level [int between 1 and 6]`')"
]
},
{
Expand Down
41 changes: 16 additions & 25 deletions nbs/09_nbdev_callback_test.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@
"metadata": {},
"outputs": [],
"source": [
"#hide_input\n",
"# hide input rather than just hide as we want this cell to be hidden from the docs, \n",
"# but we also need it to be run when tests are run and docs are built\n",
"from nbdev import *\n",
"from nbdev.imports import *"
]
Expand Down Expand Up @@ -73,6 +76,8 @@
"metadata": {},
"outputs": [],
"source": [
"%nbdev_hide\n",
"# first we'll test with valid callback names\n",
"test_eq(call_cb('begin_test_nb', 'arg1', 2, 3), 'arg1')\n",
"test_eq(call_cb('after_test_nb', 'file.name'), None)\n",
"# If we pass an invalid callback name, we get the 1st arg back ...\n",
Expand All @@ -88,33 +93,12 @@
"except TypeError: pass"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import nbdev_callbacks"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"nbdev_callbacks.raise_err = lambda: 1/0\n",
"try:\n",
" call_cb('raise_err')\n",
" assert False, 'An error should be raised because the cb handler raised an error'\n",
"except ZeroDivisionError: pass"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you ever need to import the `nbdev_callbacks` in your code, please make a call to `call_cb` first."
"%nbdev_hide\n",
"nbdev users should not need to `import nbdev_callbacks` &darr; but we need it for the show_doc calls and to patch a mock function on to the `nbdev_callbacks` module to test `call_cb` when a callback function raises an error."
]
},
{
Expand All @@ -123,7 +107,9 @@
"metadata": {},
"outputs": [],
"source": [
"call_cb('This makes sure nbdev_callbacks is loaded from the right place')"
"%nbdev_hide_input\n",
"call_cb('This makes sure nbdev_callbacks is loaded from the right place')\n",
"import nbdev_callbacks"
]
},
{
Expand All @@ -132,7 +118,12 @@
"metadata": {},
"outputs": [],
"source": [
"import nbdev_callbacks"
"%nbdev_hide\n",
"nbdev_callbacks.raise_err = lambda: 1/0\n",
"try:\n",
" call_cb('raise_err')\n",
" assert False, 'An error should be raised because the cb handler raised an error'\n",
"except ZeroDivisionError: pass"
]
},
{
Expand Down