diff --git a/README.md b/README.md index 366b8307..83f5c688 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,11 @@ wrong, let us know so we can add you to the GOTCHA hall of fame! > We also have more detailed documentation for your perusal at https://looselab.github.io/readfish > [!NOTE] -> Now also see our cool [FAQ](docs/FAQ.md). +> Now also see our cool [FAQ](docs/FAQ.md). + +> [!WARNING] +Breaking for any version of `MinKNOW <= 6.0.0` +As of `readfish >=2024.3.0` we no longer support guppy. readfish is a Python package that integrates with the [Read Until API](https://github.com/nanoporetech/read_until_api). @@ -23,13 +27,13 @@ way most fit for purpose, and a return call can be made to the server to unblock the read in progress and so direct sequencing capacity towards reads of interest. -**This implementation of readfish requires Guppy version >= 6.0.0 and MinKNOW version core >= 5.0.0 . It will not work on earlier versions.** +**This implementation of readfish requires Dorado server version >= 7.3.9 and MinKNOW version core >= 6.0.0 . It will not work on earlier versions.** -**Since MinKNOW version core >=5.9.0 and Dorado server version >=7.3.9, Dorado requires an alternate library, `ont-pybasecall-client-lib`. We have introduced a new`dorado` module to handle this.** +**To run with earlier versions of MinKNOW please use an earlier version of readfish.** -The code here has been tested with Guppy in GPU mode using GridION Mk1 and -NVIDIA RTX2080 on live sequencing runs and an NVIDIA GTX1080 using playback +The code here has been tested with Dorado in GPU mode using GridION Mk1 and +NVIDIA RTX4090s on live sequencing runs and on MacOSX M2Max using playback on a simulated run (see below for how to test this). This code is run at your own risk as it DOES affect sequencing output. You are **strongly** advised to test your setup prior to running (see below for @@ -41,6 +45,7 @@ The following platforms are supported: - **PromethION** Big Boy - **P2Solo** Smol Big Boy +- **P2i** Not so Smol Big Boy - **GridION** Box - **MinION** Smol Boy @@ -50,7 +55,7 @@ The following platforms are supported: The following OSs are supported: - **Linux** yay - - **MacOS** boo (Apple Silicon, Only with Dorado) + - **MacOS** boo (Apple Silicon Only) > [!NOTE] @@ -128,7 +133,7 @@ conda activate readfish_dev | MinKNOW is transitioning from Guppy to Dorado. Until MinKNOW version 5.9 both Guppy and Dorado used ont-pyguppy-client-lib.
As of MinKNOW version 5.9 and Dorado server version 7.3.9 and greater Dorado requires an alternate library, `ont-pybasecall-client-lib`.
The listed `ont-pyguppy-client-lib` or `ont-pybasecaller-client-lib` version may not match the version installed on your system. To fix this, Please see this [issue](https://github.com/LooseLab/readfish/issues/221#issuecomment-1381529409), using the appropriate library. | -[ONT's Guppy GPU](https://community.nanoporetech.com/downloads) should be installed and running as a server. +[ONT's Dorado Basecall Server GPU](https://community.nanoporetech.com/downloads) should be installed and running as a server.
Alternatively, install readfish into a python virtual-environment @@ -142,8 +147,8 @@ pip install --upgrade pip # Install our readfish Software pip install readfish[all] -# Install ont_pyguppy_client_lib that matches your guppy server version. E.G. -pip install ont_pyguppy_client_lib==6.3.8 +# Install ont_pybasecall_client that matches your dorado basecall server version. E.G. +pip install ont_pybasecall_client_lib==7.1.2 ```
@@ -298,8 +303,9 @@ Steps: ``` Change the text between the quotes to point to your downloaded bulk FAST5 file. -1. Optional, If running GUPPY in GPU mode, set the parameter `break_reads_after_seconds = 1.0` +1. Optional, If running Dorado in GPU mode, you can set the parameter `break_reads_after_seconds = 1.0` to `break_reads_after_seconds = 0.4`. This results in a smaller read chunk. For R10.4 this is not required but can be tried. For adaptive sampling on PromethION, this should be left at 1 second. +1. In MinKNOW >= 6.0.0 this value defaults to 0.8 which is a reasonable balance. 1. In the MinKNOW GUI, right click on a sequencing position and select `Reload Scripts`. Your version of MinKNOW will now playback the bulkfile rather than live sequencing. 1. Start a sequencing run as you would normally, selecting the corresponding flow @@ -335,13 +341,10 @@ Note: The plots here are generated from running readfish unblock-all on an Apple

Testing base-calling and mapping

 -To test selective sequencing you must have access to either a -[guppy basecall server](https://community.nanoporetech.com/downloads/guppy/release_notes) (>=6.0.0) or a [dorado basecall server](https://community.nanoporetech.com/downloads/dorado/release_notes). +To test selective sequencing you must have access to a [dorado basecall server](https://community.nanoporetech.com/downloads/dorado/release_notes). and a readfish TOML configuration file. -NOTE: guppy and dorado are used here interchangeably as the basecall server. Dorado is gradually replacing guppy. All readfish code is compatible with Guppy >=6.0.0 and dorado >=0.4.0 - 1. First make a local copy of the example TOML file: ```console curl -O https://raw.githubusercontent.com/LooseLab/readfish/master/docs/_static/example_tomls/human_chr_selection.toml @@ -532,7 +535,7 @@ These may or may not (!) be mistakes we have made already... - Double check your reference file is in the correct location. - Double check your targets exist in that reference file. - Double check your targets are correctly formatted with contig name matching the record names in your reference (Exclude description - i.e the contig name up to the first whitespace). -1. **Where has my reference gone?** If you are using a _live TOML file - e.g running iter_align or iter_cent, the previous reference MMI file is deleted when a new one is added. This obviously saves on disk space use(!) but can lead to unfortunate side effects - i.e you delete your MMI file. These can of course be recreated but user **beware**. + Happy readfish-ing! @@ -559,6 +562,14 @@ And for our Awesome Logo please checkout out [@tim_bassford](https://twitter.com # Changelog +## 2024.3.0 +### This release is breaking for ALL versions of `MinKNOW <= 6` and no longer supports Guppy. + +1. Introducing support for MinKNOW >=6.0.0 and deprecating support for earlier versions. +1. Removing support for legacy guppy base caller and only supporting Dorado in future. +1. Optimising batch sending to the base caller through the use of `pass_reads` rather than `pass_read` +1. Adding the new strand classifications as used by MinKNOW, including strand2 and short. + ## 2024.2.0 1. Add a dorado base-caller which addressed issue [#347](https://github.com/LooseLab/readfish/issues/347) - chiefly in Dorado 7.3.9 ONT have moved to `ont-pybasecall-client-lib`, and connections from `ont_pyguppy_client_lib` raise `Connection error. ... LOAD_CONFIG. Reply: INVALID_PROTOCOL` [(#344)](https://github.com/LooseLab/readfish/pull/344) diff --git a/docs/_static/readfish_light.svg b/docs/_static/readfish_light.svg new file mode 100644 index 00000000..43fb32d6 --- /dev/null +++ b/docs/_static/readfish_light.svg @@ -0,0 +1,203 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ReadFish + + diff --git a/docs/basecalling-params.md b/docs/basecalling-params.md new file mode 100644 index 00000000..bb277bd4 --- /dev/null +++ b/docs/basecalling-params.md @@ -0,0 +1,24 @@ +# Base-calling server parameters + +The default settings for `Dorado` GPU should be sufficient for most use cases. + +The one trick we sometimes use, on a _really_ high performance flow cell is to split the GPUs between `readfish` and `dorado`. + +This of course assumes that you have a multiple GPU machine, such as PromethION tower. +In the new P24 towers, there are 4x NVIDIA A100 GPUs. + +1. Stop dorado `sudo systemctl stop doradod` +1. View the available CUDA devices. `nvidia-smi` +1. Restart two dorado instances, splitting the devices between each. + +Dorado: + +```console +/opt/ont/dorado/bin/dorado_basecall_server --log_path /var/log/dorado --config dna_r10.4.1_e8.2_400bps_fast.cfg --ipc_threads 3 --port /tmp/.guppy/5555 --dorado_download_path /opt/ont/dorado-models --device cuda:0 +``` + +```console +/opt/ont/dorado/bin/dorado_basecall_server --log_path /var/log/dorado --config dna_r10.4.1_e8.2_400bps_fast.cfg --ipc_threads 3 --port /tmp/.guppy/5556 --dorado_download_path /opt/ont/dorado-models --device cuda:1 +``` + +1. Start readfish, base calling on port 5556 diff --git a/docs/developers-guide.md b/docs/developers-guide.md index a32c64ad..972b21f3 100644 --- a/docs/developers-guide.md +++ b/docs/developers-guide.md @@ -32,8 +32,8 @@ dev = ["readfish[all,docs,tests]"] # Running dependencies, this is a little bit clunky but works for now mappy = ["mappy"] mappy-rs = ["mappy-rs"] -guppy = ["ont_pyguppy_client_lib"] -all = ["readfish[mappy,mappy-rs,guppy]"] +dorado = ["ont-pybasecall-client-lib"] +all = ["readfish[mappy,mappy-rs,dorado]"] ``` An example install command would be `pip install readfish[all]`. @@ -159,11 +159,11 @@ Plugins are loaded as instances of `_PluginModules` - see source of `_PluginModu This would try to load the `foo` module from the `bar` package. ```toml -[caller_settings.readfish.plugins.guppy] +[caller_settings.readfish.plugins.dorado] ``` -This would load the readfish guppy `Caller` plugin explicitly. -There are instances of "builtin" plugins, which are the included `mappy`, `mappy_rs` and `guppy` plugins. +This would load the readfish dorado `Caller` plugin explicitly. +There are instances of "builtin" plugins, which are the included `mappy`, `mappy_rs` and `dorado` plugins. See the source of `readfish._config._PluginModule.load_module` for more details. ```{eval-rst} .. automethod:: readfish._config._PluginModule.load_module @@ -209,7 +209,7 @@ This next method is important. `validate()` The validate function is intended to be called in the __init__ method, before the actual `Aligner` or `Caller` is initialised. The contents of this method are left up to the author, however we suggest that people check for the things listed above. -The purpose of `validate` is to check that the given parameters will create a valid `Aligner` or `Caller`. For example, in the `guppy.py` `Caller` plugin, we check the permissions of the provided `Guppy` socket. If these are insufficient, Guppy only errors out after a 10 minute timeout. However of this is caught in `validate`, everyone ends up being left a lot happier. +The purpose of `validate` is to check that the given parameters will create a valid `Aligner` or `Caller`. For example, in the `dorado.py` `Caller` plugin, we check the permissions of the provided `dorado` socket. If these are insufficient, dorado only errors out after a 10 minute timeout. However if this is caught in `validate`, everyone ends up being left a lot happier. ```{literalinclude} ../src/readfish/plugins/_mappy.py :language: python diff --git a/docs/guppy-params.md b/docs/guppy-params.md deleted file mode 100644 index e25b803a..00000000 --- a/docs/guppy-params.md +++ /dev/null @@ -1,38 +0,0 @@ -# Base-calling server parameters - -The default settings for `Guppy` GPU should be sufficient for most use cases. -If you are really into optimisation, we recommend reading [Miles Benton's Guppy tuning guide] - until Dorado comes along this all should work. - -The one trick we sometimes use, on a _really_ high performance flow cell is to split the GPUs between `readfish` and `guppy`. - -This of course assumes that you have a multiple GPU machine, such as PromethION tower. -In the new P24 towers, there are 4x NVIDIA A100 GPUs. - -1. Stop guppy `sudo systemctl stop guppyd` or stop dorado `sudo systemctl stop doradod` -1. View the available CUDA devices. `nvidia-smi` -1. Restart two guppy/dorado instances, splitting the devices between each. - -Guppy: - -```console -guppy_basecall_server --port ipc:///tmp/.guppy/5555 --ipc_threads 2 --trim_adapters -c /opt/ont/guppy/data/dna_r10.4.1_e8.2_400bps_fast.cfg -x cuda:0 --log_path /tmp/ -``` - -```console -guppy_basecall_server --port ipc:///tmp/.guppy/5556 --ipc_threads 2 --trim_adapters -c /opt/ont/guppy/data/dna_r10.4.1_e8.2_400bps_fast.cfg -x cuda:1 --log_path /tmp/ -``` - -Dorado: - -```console -/opt/ont/dorado/bin/dorado_basecall_server --log_path /var/log/dorado --config dna_r10.4.1_e8.2_400bps_fast.cfg --ipc_threads 3 --port /tmp/.guppy/5555 --dorado_download_path /opt/ont/dorado-models --device cuda:0 -``` - -```console -/opt/ont/dorado/bin/dorado_basecall_server --log_path /var/log/dorado --config dna_r10.4.1_e8.2_400bps_fast.cfg --ipc_threads 3 --port /tmp/.guppy/5556 --dorado_download_path /opt/ont/dorado-models --device cuda:1 -``` - -1. Start readfish, base calling on port 5556 - - -[Miles Benton's Guppy tuning guide]: https://hackmd.io/@Miles/S12SKP115 diff --git a/docs/index.md b/docs/index.md index 682b4c4f..59aa2362 100644 --- a/docs/index.md +++ b/docs/index.md @@ -6,7 +6,7 @@ hidden: true getting-started toml readfish.console -guppy-params +basecalling-params FAQ post-analysis developers-guide diff --git a/docs/questions/creating_readfish-environment.question.md b/docs/questions/creating_readfish-environment.question.md index 11354c8c..2c10af4f 100644 --- a/docs/questions/creating_readfish-environment.question.md +++ b/docs/questions/creating_readfish-environment.question.md @@ -8,17 +8,21 @@ We recommend using conda to manage your readfish environments, especially on ONT To install conda instructions can be found [here](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html#regular-installation). An explanation is found in an answer to this [issue](https://github.com/LooseLab/readfish/issues/124#issuecomment-759599319) as well. -Once Conda is installed, copy the following code snippet into a file on your system, and replace the X.X.X of the `ont-pyguppy-client-lib` with the version output of +Once Conda is installed, copy the following code snippet into a file on your system, and replace the X.X.X of the `ont-pybasecall-client-lib` with the version output of ```console - guppy_basecall_server --version + dorado_basecall_server --version ``` You should get something like this back - Guppy Basecall Service Software, (C) Oxford Nanopore Technologies plc. Version 6.1.5+446c35524, client-server API version 11.0.0 + : Dorado Basecall Service Software, (C)Oxford Nanopore Technologies plc. Version 7.4.12+0e5e75c49, client-server API version 20.0.0 -The important part is the first three numbers of the version, in this case `6.1.5`. + Use of this software is permitted solely under the terms of the end user license agreement (EULA). + By running, copying or accessing this software, you are demonstrating your acceptance of the EULA. + The EULA may be found in /opt/ont/dorado/bin + +The important part is the first three numbers of the version, in this case `7.4.12`. ```yaml name: readfish @@ -27,10 +31,10 @@ channels: - conda-forge - defaults dependencies: - - python=3.9 + - python=3.11 - pip - pip: - git+https://github.com/nanoporetech/read_until_api@3.4.1 - - ont-pyguppy-client-lib==X.X.X + - ont-pybasecall-client-lib==X.X.X - git+https://github.com/LooseLab/readfish@main ``` diff --git a/docs/questions/py-guppy-client-error.question.md b/docs/questions/py-guppy-client-error.question.md index 49419a99..573abcc7 100644 --- a/docs/questions/py-guppy-client-error.question.md +++ b/docs/questions/py-guppy-client-error.question.md @@ -2,6 +2,7 @@ title: "Connection error. \\[timed_out\\] Timeout waiting for reply to request: LOAD_CONFIG" alt_titles: - "Error connecting to Guppy" + - "Error connecting to Dorado" - "LOAD_CONFIG error when connecting to py-guppy-client" - "LOAD_CONFIG error when connecting to Guppy" --- @@ -9,7 +10,7 @@ alt_titles: N.B. The answer below was taken from this [issue](https://github.com/LooseLab/readfish/issues/221#issuecomment-1375673490) Theoretically this should be caught in the latest readfish release, but if you are still having issues, please try the below. -This error often stems from a couple of sources. To test your Guppy Connection you can run +This error often stems from a couple of sources. To test your Guppy/Dorado Connection you can run ```console python -c 'from pyguppy_client_lib.pyclient import PyGuppyClient as PGC; \ @@ -17,9 +18,17 @@ python -c 'from pyguppy_client_lib.pyclient import PyGuppyClient as PGC; \ c.connect(); print(c)' ``` -Replacing the `5555` (The default guppy port) with whichever port Guppy may be running on. +And for Dorado: -The following tends to be the problem only on Computers which ar not provided by ONT, and have been set up manually. Guppy creates the socket file on which is listens as the `MinKNOW ` User, which doens't allow your User account to read/write to the socket. +```console +python -c 'from pybasecall_client_lib.pyclient import PyBasecallClient as PBC; \ + c = PBC("ipc:///tmp/.guppy/5555", "dna_r9.4.1_450bps_fast.cfg"); \ + c.connect(); print(c)' +``` + +Replacing the `5555` (The default Guppy/Dorado port) with whichever port Guppy/Dorado may be running on. + +The following tends to be the problem only on Computers which ar not provided by ONT, and have been set up manually. Guppy/Dorado creates the socket file on which is listens as the `MinKNOW ` User, which doens't allow your User account to read/write to the socket. To fix this, you can either add yourself to the `minknow` group and give the group write permission , or you can give everyone write permissions to the socket which is a bit less secure (Quick, but maybe try the group thing first). @@ -38,7 +47,7 @@ sudo chmod 775 /tmp/.guppy/5555 Which means any User has all permissions on this file. -Once you have done that if you run the above Python command changing the IPC port to whatever port Guppy is listening on. This is usually found in `/tmp/.guppy`, and can be seen by running `ls /tmp/.guppy` +Once you have done that if you run the above Python command changing the IPC port to whatever port Guppy/Dorado is listening on. This is usually found in `/tmp/.guppy`, and can be seen by running `ls /tmp/.guppy` ```console python -c 'from pyguppy_client_lib.pyclient import PyGuppyClient as PGC; \ diff --git a/docs/questions/py-guppy-client-version.question.md b/docs/questions/py-guppy-client-version.question.md index 2d6ae44b..16ca39a1 100644 --- a/docs/questions/py-guppy-client-version.question.md +++ b/docs/questions/py-guppy-client-version.question.md @@ -4,8 +4,12 @@ alt_titles: - "Error connecting to Guppy" - "LOAD_CONFIG error when connecting to py-guppy-client" - "INVALID_PROTOCOL error when connecting to Guppy" + - "INVALID PROTOCOL" --- +## UPDATE 20/08/2024 +The same all holds true for `Dorado`, although it is necessary to replace `ont-pyguppy-client-lib` with `ont-pybasecall-client-lib` + This is most likely a version mismatch between `ont-pyguppy-client-lib`, the python library that enables readfish to talk to Guppy and the installed version of Guppy. If you open a terminal and run diff --git a/docs/readfish.plugins.rst b/docs/readfish.plugins.rst index baab9ff5..27ec23ed 100644 --- a/docs/readfish.plugins.rst +++ b/docs/readfish.plugins.rst @@ -22,10 +22,10 @@ readfish plugins API ---- -``readfish.plugins.guppy`` module +``readfish.plugins.dorado`` module --------------------------------- -.. automodule:: readfish.plugins.guppy +.. automodule:: readfish.plugins.dorado :members: :undoc-members: :show-inheritance: diff --git a/docs/toml.md b/docs/toml.md index 06269b95..80846de9 100644 --- a/docs/toml.md +++ b/docs/toml.md @@ -39,45 +39,45 @@ These are expanded on in the {ref}`barcode configuration ` and ## Plugin configuration Readfish uses "plugins" to configure basecalling and alignment. -We provide some default plugins using Guppy (via [`ont-pyguppy-client-lib`]) and mappy (via [`mappy`]), for basecalling and alignment respectively. +We provide some default plugins using Dorado (via [`ont-pybasecall-client-lib`]) and mappy (via [`mappy`]), for basecalling and alignment respectively. The way that you tell readfish what plugin to load is by specifying it after the table name, for example: ```toml -[caller_settings.guppy] +[caller_settings.dorado] # ^^^^^^ - # Chooses the Guppy plugin + # Chooses the Dorado plugin ``` -This indicates that readfish should use Guppy for basecalling. +This indicates that readfish should use Dorado for basecalling. ### Basecaller -#### Guppy +#### Dorado -This section is specific to the Guppy plugin for basecalling. -The `caller_settings.guppy` [table] specifies the basecalling parameters used by the basecaller. +This section is specific to the Dorado plugin for basecalling. +The `caller_settings.dorado` [table] specifies the basecalling parameters used by the basecaller. An example table is given below: ```toml -[caller_settings.guppy] +[caller_settings.dorado] config = "dna_r10.4.1_e8.2_400bps_hac" address = "ipc:///tmp/.guppy/5555" debug_log = "basecalled_chunks.fq" #optional ``` -The only required keys are `address` and `config` these indicate to Guppy what server to connect to and what basecalling model to load. -The `config` parameter must a valid Guppy configuration excluding the file extension; these are found in the Guppy `data` folder (on linux `ls /opt/ont/guppy/data/*.cfg`). -The `address` is the path to the IPC socket Guppy is listening on, for Guppy servers run by MinKNOW the default is `ipc:///tmp/.guppy/5555`. +The only required keys are `address` and `config` these indicate to Dorado what server to connect to and what basecalling model to load. +The `config` parameter must be a valid Dorado configuration excluding the file extension; these are found in the Dorado `data` folder (on linux `ls /opt/ont/dorado/data/*.cfg`). +The `address` is the path to the IPC socket Dorado is listening on, for Dorado servers run by MinKNOW the default is `ipc:///tmp/.guppy/5555`. Why guppy? Blame history... The `debug_log` is an optional file that the basecalled FASTQ are written to. -| Key | Type | Description | Required | -| ------------: | :----: | :----------------------------------------------------------------------------: | :------- | -| `config` | string | Name of base calling config | True | -| ` address` | string | Address of Guppy socket - default ipc:///tmp/.guppy/5555 | True | +| Key | Type | Description | Required | +| ------------: | :----: |:------------------------------------------------------------------------------:| :------- | +| `config` | string | Name of base calling config | True | +| ` address` | string | Address of Dorado socket - default ipc:///tmp/.guppy/5555 | True | | `debug_log` | string | Optional - Filepath to write out base-calls to - should end in a FASTQ suffix. | False | -Any extra key value pairs are passed varbatim to the `PyGuppyClient` instance upon initialisation as keyword arguments. +Any extra key value pairs are passed verbatim to the `PyBasecallClient` instance upon initialisation as keyword arguments. For example: @@ -85,27 +85,27 @@ For example: server_file_load_timeout = 60 ``` -Will set the `server_file_load_timeout` parameter to 60 seconds for Guppy. +Will set the `server_file_load_timeout` parameter to 60 seconds for Dorado.
-Example Guppy server parameters +Example Dorado server parameters ```{eval-rst} -.. automethod:: pyguppy_client_lib.pyclient.PyGuppyClient.set_params +.. automethod:: pybasecall_client_lib.pyclient.PyBasecallClient.set_params :noindex: ```
-To see parameters that _your_ Guppy installation has run the following python snippet in your `readfish` environment +To see parameters that _your_ Dorado installation has run the following python snippet in your `readfish` environment ```console -PAGER=cat python -c "import pyguppy_client_lib.pyclient as pgc; help(pgc.PyGuppyClient.set_params)" +PAGER=cat python -c "import pybasecall_client_lib.pyclient as pgc; help(pgc.PyBasecallClient.set_params)" ``` #### Basecaller no operation (`no_op`) -In addition to the Guppy basecaller we provide a "no operation" basecaller. +In addition to the Dorado basecaller we provide a "no operation" basecaller. This plugin does nothing. It will only iterate the live chunks of data and pass through the minimal amount of data that is needed for the next steps. This is added using: @@ -306,10 +306,10 @@ Targets given in this format specify the entire contig as a target to select for (barcodes-config)= ## Barcode specific configuration -If you are using Guppy for basecalling then the additional `barcode_kits` parameter is required on the `caller_settings` table. +If you are using Dorado for basecalling then the additional `barcode_kits` parameter is required on the `caller_settings` table. ```toml -[caller_settings.guppy] +[caller_settings.dorado] config = "dna_r10.4.1_e8.2_400bps_fast" address = "ipc:///tmp/.guppy/5555" debug_log = "basecalled_chunks.fq" @@ -405,7 +405,7 @@ This can be disabled by passing the `--no-describe` flag. [tables]: https://toml.io/en/v1.0.0#table [arrays of tables]: https://toml.io/en/v1.0.0#array-of-tables [key-value pairs]: https://toml.io/en/v1.0.0#keyvalue-pair -[`ont-pyguppy-client-lib`]: https://pypi.org/p/ont-pyguppy-client-lib +[`ont-pybasecall-client-lib`]: https://pypi.org/p/ont-pybasecall-client-lib [`mappy`]: https://github.com/lh3/minimap2/tree/master/python [`mappy-rs`]: https://github.com/Adoni5/mappy-rs [`mappy` documentation]: https://github.com/lh3/minimap2/tree/master/python#class-mappyaligner diff --git a/pyproject.toml b/pyproject.toml index cfa247fa..7f641bb8 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -36,7 +36,7 @@ dependencies = [ "rtoml", "more_itertools", 'exceptiongroup ; python_version<"3.11"', - "readfish_summarise >= 0.2.5", + "readfish_summarise >= 0.2.7", "readfish[all]", ] dynamic = ["version"] @@ -48,14 +48,13 @@ readfish = "readfish._cli_base:main" # Development dependencies docs = ["sphinx-copybutton", "furo", "myst-parser", "faqtory"] tests = ["pytest", "coverage[toml]"] -tests-mappy = ["readfish[tests,mappy,guppy]"] +tests-mappy = ["readfish[tests,mappy,dorado]"] dev = ["readfish[all,docs,tests]", "pre-commit"] # Running dependencies, this is a little bit clunky but works for now mappy = ["mappy"] mappy-rs = ["mappy-rs >= 0.0.6"] -guppy = ["ont_pyguppy_client_lib"] -dorado = ["ont-pybasecall-client-lib"] -all = ["readfish[mappy,mappy-rs,guppy,dorado]"] +dorado = ["ont-pybasecall-client-lib>=7.4.12"] +all = ["readfish[mappy,mappy-rs,dorado]"] [project.urls] Documentation = "https://looselab.github.io/readfish" diff --git a/src/readfish/__about__.py b/src/readfish/__about__.py index a1763d2b..f4e6887c 100644 --- a/src/readfish/__about__.py +++ b/src/readfish/__about__.py @@ -1,4 +1,5 @@ """__about__.py Version of the read until software """ -__version__ = "2024.2.0" + +__version__ = "2024.3.0" diff --git a/src/readfish/_compatibility.py b/src/readfish/_compatibility.py index 8a44a47f..73fdba2d 100644 --- a/src/readfish/_compatibility.py +++ b/src/readfish/_compatibility.py @@ -19,12 +19,12 @@ from packaging.version import parse as parse_version from packaging.version import Version -LATEST_TESTED = "5.9.7" +LATEST_TESTED = "6.0.0" # The versions of MinKNOW which this version of readfish can connect to # Format - (lowest minknow version, highest version of minknow supported as an upper bound) MINKNOW_COMPATIBILITY_RANGE = ( - Version("5.0.0"), + Version("6.0.0"), Version(LATEST_TESTED), ) @@ -84,11 +84,10 @@ def check_compatibility( >>> check_compatibility(Version("4.9.0"), (Version("5.0.0"), Version("5.9.7"))) >>> if (action := check_compatibility(Version("6.0.0"), MINKNOW_COMPATIBILITY_RANGE)) in ( - ... DIRECTION.UPGRADE, - ... DIRECTION.DOWNGRADE, + ... DIRECTION.JUST_RIGHT, ... ): ... action - + """ ( lowest_supported_version, diff --git a/src/readfish/_config.py b/src/readfish/_config.py index 227daeec..8734d8dc 100644 --- a/src/readfish/_config.py +++ b/src/readfish/_config.py @@ -170,7 +170,6 @@ def load_module(self, override=False): override=True, the builtin module names are ignored. """ builtins = { - "guppy": "guppy", "dorado": "dorado", "mappy": "mappy", "mappy_rs": "mappy_rs", diff --git a/src/readfish/_read_until_client.py b/src/readfish/_read_until_client.py index d98fa139..f5a55216 100644 --- a/src/readfish/_read_until_client.py +++ b/src/readfish/_read_until_client.py @@ -1,6 +1,7 @@ """read_until_client.py Subclasses ONTs read_until_api ReadUntilClient added extra function that logs unblocks read_ids. """ + from __future__ import annotations import logging import time @@ -92,24 +93,21 @@ def __init__(self, *args, **kwargs): ) def unblock_read_batch( - self, reads: list[tuple[int, int, str]], duration: float = 0.1 + self, reads: list[tuple[int, str]], duration: float = 0.1 ) -> None: """Request for a bunch of reads be unblocked. - ``reads`` is expected to be a list of (channel, ReadData.number) + ``reads`` is expected to be a list of (channel, ReadData.id) - :param reads: List of (channel, read_number, read_id) + :param reads: List of (channel, read_id) :param duration: time in seconds to apply unblock voltage. """ - actions = list() - for channel, read_number, *read_id in reads: + actions = [] + for channel, read_id in reads: actions.append( - self._generate_action( - channel, read_number, "unblock", duration=duration - ) + self._generate_action(channel, read_id, "unblock", duration=duration) ) - if read_id: - self.unblock_logger.debug(read_id[0]) + self.unblock_logger.debug(read_id) self.action_queue.put(actions) def unblock_read( diff --git a/src/readfish/_statistics.py b/src/readfish/_statistics.py index 3a21b609..926c868e 100644 --- a/src/readfish/_statistics.py +++ b/src/readfish/_statistics.py @@ -17,7 +17,7 @@ class is able to update and query various statistics and counters regarding the >>> from readfish._statistics import ReadfishStatistics, DEBUG_STATS_LOG_FIELDS >>> stats = ReadfishStatistics(None) >>> stats.add_batch_performance(1,1) - >>> stats.log_read(**dict(zip(DEBUG_STATS_LOG_FIELDS, (1, 2, "test_read_id", 7, 1, 100, 3,\ + >>> stats.log_read(**dict(zip(DEBUG_STATS_LOG_FIELDS, (1, 2, "test_read_id", 7, 100, 3,\ "single_on", "stop_receiving", "exp_region", None, None, False, 0.0))), region_name="naff",\ overridden_action_name=None) >>> print(stats.get_batch_performance()) @@ -26,6 +26,7 @@ class is able to update and query various statistics and counters regarding the Counter({'single_on': 1}) """ + from __future__ import annotations from collections import Counter import logging @@ -38,7 +39,6 @@ class is able to update and query various statistics and counters regarding the "read_in_loop", "read_id", "channel", - "read_number", "seq_len", "counter", "mode", @@ -75,7 +75,7 @@ class ReadfishStatistics: >>> stats = ReadfishStatistics(None) >>> stats.add_batch_performance(1, 1) >>> stats.log_read(**dict(zip(DEBUG_STATS_LOG_FIELDS, (1, 2, "test_read_id",\ -7, 1, 100, 3, "single_on", "stop_receiving", "exp_region", None, None,\ +7, 100, 3, "single_on", "stop_receiving", "exp_region", None, None,\ False, 0.0))), region_name="naff", overridden_action_name=None) >>> print(stats.get_batch_performance()) 0001R/1.0000s; Avg: 0001R/1.0000s; Seq:1; Unb:0; Pro:0; Slow batches (>1.00s): 0/1 @@ -98,7 +98,7 @@ class ReadfishStatistics: ... stats = ReadfishStatistics(log_file=log_file_name) ... # Use the log_read method to log a sample read ... stats.log_read(**dict(zip(DEBUG_STATS_LOG_FIELDS,\ -(1, 2, "test_read_id", 7, 1, 100, 3, "single_on", "stop_receiving", "exp_region",\ +(1, 2, "test_read_id", 7, 100, 3, "single_on", "stop_receiving", "exp_region",\ None, None, False, 0.0))), region_name="naff", overridden_action_name=None) ... # in this test, we need a small amount of time to allow the logger to write the file ... time.sleep(0.1) @@ -107,7 +107,7 @@ class ReadfishStatistics: ... content = log_file.read() ... # Prepare the expected content ... header = " ".join(DEBUG_STATS_LOG_FIELDS) - ... expected_line = " ".join(map(str, (1, 2, "test_read_id", 7, 1, 100, 3, "single_on",\ + ... expected_line = " ".join(map(str, (1, 2, "test_read_id", 7, 100, 3, "single_on",\ "stop_receiving", "exp_region", None, None, False, 0.0))) ... expected = f"{header}\\n{expected_line}" ... # Check that the content matches, don't ask about the replaces, it was the only way diff --git a/src/readfish/_utils.py b/src/readfish/_utils.py index 458584e9..4380e661 100644 --- a/src/readfish/_utils.py +++ b/src/readfish/_utils.py @@ -1,6 +1,7 @@ """utils.py functions and utilities used internally. """ + from __future__ import annotations import sys import logging @@ -34,11 +35,11 @@ def __init__(self, channels): # self.tracker = defaultdict(Counter) self.tracker = [Counter() for _ in range(channels + 1)] - def seen(self, channel, read_number): - if read_number not in self.tracker[channel]: + def seen(self, channel, read_id: str): + if read_id not in self.tracker[channel]: self.tracker[channel].clear() - self.tracker[channel][read_number] += 1 - return self.tracker[channel][read_number] + self.tracker[channel][read_id] += 1 + return self.tracker[channel][read_id] class Severity(IntEnum): diff --git a/src/readfish/entry_points/targets.py b/src/readfish/entry_points/targets.py index a580aa8d..10345a5c 100644 --- a/src/readfish/entry_points/targets.py +++ b/src/readfish/entry_points/targets.py @@ -83,6 +83,7 @@ # Core imports from __future__ import annotations import argparse +from packaging.version import Version import logging import time from timeit import default_timer as timer @@ -403,17 +404,15 @@ def check_override_action( action = Action.stop_receiving if action is Action.stop_receiving: - stop_receiving_action_list.append((result.channel, result.read_number)) + stop_receiving_action_list.append((result.channel, result.read_id)) elif action is Action.unblock: if self.dry_run: # Log an 'unblock' action to previous action, but send a 'stop receiving' to prevent further read processing. action_overridden = True - stop_receiving_action_list.append((result.channel, result.read_number)) + stop_receiving_action_list.append((result.channel, result.read_id)) else: - unblock_batch_action_list.append( - (result.channel, result.read_number, result.read_id) - ) + unblock_batch_action_list.append((result.channel, result.read_id)) # If we have made a final decision for this read and we shouldn't see it again! if action is Action.unblock or action is Action.stop_receiving: @@ -495,7 +494,7 @@ def run(self): ) result.decision = make_decision(self.conf, result) action = condition.get_action(result.decision) - seen_count = self.chunk_tracker.seen(result.channel, result.read_number) + seen_count = self.chunk_tracker.seen(result.channel, result.read_id) # Check if there any conditions that override the action chose, exceed_max_chunks etc... ( previous_action, @@ -515,7 +514,6 @@ def run(self): read_in_loop=number_reads, read_id=result.read_id, channel=result.channel, - read_number=result.read_number, seq_len=len(result.seq), counter=seen_count, mode=result.decision.name, @@ -601,6 +599,13 @@ def run( https://github.com/LooseLab/readfish/issues""" ) + if minknow_version < Version("6.0.0"): + logger.critical( + f"This version of readfish ({__version__}) is not compatible with less than MinKNOW 6.X.X, you downgrade to at least readfish 2024.2.0" + f"This won't work, exiting..." + ) + raise SystemExit(1) + # Fetch sequencing device position = get_device(args.device, host=args.host, port=args.port) @@ -611,6 +616,13 @@ def run( filter_strands=True, cache_type=AccumulatingCache, timeout=args.wait_for_ready, + prefilter_classes={ + "strand", + "strand2", + "short_strand", + "adapter", + "unknown_positive", + }, ) # Load TOML configuration @@ -635,6 +647,7 @@ def run( first_channel=1, last_channel=read_until_client.channel_count, max_unblock_read_length_seconds=args.max_unblock_read_length_seconds, + accepted_first_chunk_classifications=["strand", "strand2", "short_strand", "adapter", "unknown_positive"], ) worker = Analysis( diff --git a/src/readfish/entry_points/validate.py b/src/readfish/entry_points/validate.py index f98cc0ea..306ab3c1 100644 --- a/src/readfish/entry_points/validate.py +++ b/src/readfish/entry_points/validate.py @@ -83,7 +83,7 @@ def run(parser, args, extras) -> int: except Exception as exc: logger.error("Caller could not be initialised.") logger.error( - "Possible reasons for this include a mismatch between the ont basecaller client and the versions of guppy or dorado you are connecting to." + "Possible reasons for this include a mismatch between the ont basecaller client and the version dorado you are connecting to." ) logger.error( "Additional information is available here: https://looselab.github.io/readfish/FAQ.html#connection-error-bad-reply-could-not-interpret-message-from-server-for-request-load-config-reply-invalid-protocol" diff --git a/src/readfish/plugins/_no_op.py b/src/readfish/plugins/_no_op.py index 61f52dc5..d283b8ef 100644 --- a/src/readfish/plugins/_no_op.py +++ b/src/readfish/plugins/_no_op.py @@ -21,6 +21,7 @@ In addition the ``_no_op.Aligner`` will pass through the iterable from the caller module without modifying/adding anything. This behaviour can be useful if a plugin can complete it's entire decision in a single step. """ + from __future__ import annotations from typing import Iterable, TYPE_CHECKING @@ -93,12 +94,11 @@ def basecall( :param chunks: Raw data wrapper from the MinKNOW RPC - :returns: Yields ``Result`` classes with the ``Result.channel``, ``Result.read_number``, ``Result.read_id``, and ``Result.seq`` fields set. + :returns: Yields ``Result`` classes with the ``Result.channel``, ``Result.read_id``, and ``Result.seq`` fields set. """ for channel, read in chunks: yield Result( channel=channel, - read_number=read.number, read_id=read.id, seq="", ) diff --git a/src/readfish/plugins/abc.py b/src/readfish/plugins/abc.py index f6596715..f3cde50f 100644 --- a/src/readfish/plugins/abc.py +++ b/src/readfish/plugins/abc.py @@ -12,6 +12,7 @@ - **writable outputs** - Check permissions on output files - **sufficient space/RAM/resource** - Check Disk space at least """ + from __future__ import annotations import abc from typing import Iterable, TYPE_CHECKING @@ -105,7 +106,7 @@ def basecall( :param signal_dtype: The NumPy :func:`numpy.dtype` for the raw signal bytes. :param daq_values: Mapping of channel number to it's ``CALIBRATION`` values. - :returns: Yields ``Result`` classes with the ``Result.channel``, ``Result.read_number``, ``Result.read_id``, and ``Result.seq`` fields set. + :returns: Yields ``Result`` classes with the ``Result.channel``, ``Result.read_id``, and ``Result.seq`` fields set. """ @abc.abstractmethod diff --git a/src/readfish/plugins/dorado.py b/src/readfish/plugins/dorado.py index 10fe43ff..fdd6d7aa 100644 --- a/src/readfish/plugins/dorado.py +++ b/src/readfish/plugins/dorado.py @@ -67,16 +67,19 @@ def __init__( self.supported_barcode_kits = None self.supported_basecall_models = None self.run_information = run_information + if self.run_information: - self.guppy_version = ( - self.run_information.software_versions.guppy_connected_version + self.basecaller_version = ( + self.run_information.software_versions.basecaller_connected_version ) - if parse_version(self.guppy_version) >= parse_version("7.3.9"): - logging.info(f"Connected to caller version {self.guppy_version}.") + if parse_version(self.basecaller_version) >= parse_version("7.4.12"): + logging.info(f"Connected to caller version {self.basecaller_version}.") else: logging.info( - f"Trying to use minKNOW with a caller version {self.guppy_version}. If this is causing readfish to crash, try using a version of Dorado >= 7.3.9. You should also check for any updates available to readfish." + f"Trying to use minKNOW with a caller version {self.basecaller_version}." + " If this is causing readfish to crash, try using a version of Dorado >= 7.3.9." + " You should also check for any updates available to readfish." ) # Set our own priority @@ -226,7 +229,8 @@ def basecall( # Attach the "RF-" prefix read_id = f"RF-{read.id}" t0 = time.time() - cache[read_id] = (channel, read.number, t0) + # cache the read id without the RF tag + cache[read_id] = (channel, read.id, t0) reads_to_send.append( package_read( read_id=read_id, @@ -285,9 +289,9 @@ def basecall( continue try: - channel, read_number, time_sent = cache.pop(read_id) + channel, read_id, _time_sent = cache.pop(read_id) except KeyError: - channel, read_number, time_sent = skipped.pop(read_id) + channel, read_id, _time_sent = skipped.pop(read_id) reads_sent += 1 res["metadata"]["read_id"] = read_id[3:] self.logger.debug( @@ -301,7 +305,6 @@ def basecall( # TODO: Add Filter here yield Result( channel=channel, - read_number=read_number, read_id=res["metadata"]["read_id"], seq=res["datasets"]["sequence"], barcode=barcode if barcode else None, diff --git a/src/readfish/plugins/guppy.py b/src/readfish/plugins/guppy.py deleted file mode 100644 index 4432cc0d..00000000 --- a/src/readfish/plugins/guppy.py +++ /dev/null @@ -1,292 +0,0 @@ -"""Guppy plugin module - -Extension of pyguppy Caller that maintains a connection to the basecaller -""" - -from __future__ import annotations -import logging -import os -import time -from collections import namedtuple -from pathlib import Path -from typing import Iterable, TYPE_CHECKING -from packaging.version import parse as parse_version - -import numpy as np -import numpy.typing as npt -from minknow_api.protocol_pb2 import ProtocolRunInfo - -try: - from pyguppy_client_lib.helper_functions import package_read - from pyguppy_client_lib.pyclient import PyGuppyClient -except ImportError: - pass - -from readfish._loggers import setup_logger -from readfish.plugins.abc import CallerABC -from readfish.plugins.utils import Result -from readfish._utils import nice_join - - -if TYPE_CHECKING: - import minknow_api - -__all__ = ["Caller"] - -logger = logging.getLogger("RU_basecaller") -CALIBRATION = namedtuple("calibration", "scaling offset") - - -class DefaultDAQValues: - """Provides default calibration values - - Mimics the read_until_api calibration dict value from - https://github.com/nanoporetech/read_until_api/blob/2319bbe/read_until/base.py#L34 - all keys return scaling=1.0 and offset=0.0 - """ - - calibration = CALIBRATION(1.0, 0.0) - - def __getitem__(self, _): - return self.calibration - - -_DefaultDAQValues = DefaultDAQValues() - - -class Caller(CallerABC): - def __init__( - self, run_information: ProtocolRunInfo = None, debug_log=None, **kwargs - ): - self.logger = setup_logger("readfish_guppy_logger", log_file=debug_log) - self.supported_barcode_kits = None - self.supported_basecall_models = None - self.run_information = run_information - logging.warn( - "Deprecation warning - As ONT has moved fully to dorado, this plugin is no longer maintained, and will be deprecated in a future release of readfish." - ) - if self.run_information: - self.guppy_version = ( - self.run_information.software_versions.guppy_connected_version - ) - - if parse_version(self.guppy_version) < parse_version("7.3.9"): - logging.info(f"Connected to caller version {self.guppy_version}.") - else: - logging.info( - f"Trying to connect to minKNOW with caller version {self.guppy_version}. This plugin requires a version of Dorado or Guppy < 7.3.9. If this is stopping readfish from running try changing [caller_settings.guppy] to [caller_settings.dorado]. You should also check for any updates available to readfish." - ) - - # Set our own priority - self.guppy_params = kwargs - - # Remove the sample rate from the guppy params as it isn't required for the PyGuppyClient - self.guppy_params.pop("sample_rate", None) - self.guppy_params["priority"] = PyGuppyClient.high_priority - # Set our own client name to appear in the guppy server logs - self.guppy_params["client_name"] = "Readfish_connection" - self.validate() - self.caller = PyGuppyClient(**self.guppy_params) - self.caller.connect() - - def validate(self) -> None: - """Validate the parameters passed to Guppy to ensure they will initialise PyGuppy Client correctly - - Currently checks: - 1. That the socket file exists - 2. That the Socket file has the correct permissions - 3. That the version of py guppy client lib installed matches the system version - - :return: None, if the parameters pass all the checks - - """ - for key in ("address", "config"): - if key not in self.guppy_params: - raise KeyError( - f"Required `caller_settings.guppy` {key} was not found in provided TOML. Please add." - ) - if self.guppy_params["address"].startswith("ipc://"): - # User is attempting to connect to an IPC socket - socket_path = Path(self.guppy_params["address"][6:]) - if not socket_path.exists(): - raise FileNotFoundError( - f"The provided guppy base-caller socket address doesn't appear to exist. Please check your Guppy Settings. {self.guppy_params['address']}" - ) - - # check user permissions: - if not os.access(socket_path, os.R_OK): - raise RuntimeError( - f"The user account running readfish doesn't appear to have permissions to read the guppy base-caller socket. Please check permissions on {self.guppy_params['address']}. See https://github.com/LooseLab/readfish/issues/221#issuecomment-1375673490 for more information." - ) - if not os.access(socket_path, os.W_OK): - raise RuntimeError( - f"The user account running readfish doesn't appear to have permissions to write to the guppy base-caller socket. Please check permissions on {self.guppy_params['address']}. See https://github.com/LooseLab/readfish/issues/221#issuecomment-1375673490 for more information." - ) - # If we are connected to a live run, test if the base-caller model is acceptable. - # Connected to a live run via the minknow_api - get supported basecall and barcoding kits from the run info. - # Check them against provided values - if self.run_information is not None: - tags = self.run_information.meta_info.tags - - self.supported_basecall_models = tags[ - "available basecall models" - ].array_value - # Make a CSV str a list of strings, removing quotes and square brackets - if self.supported_basecall_models and isinstance( - self.supported_basecall_models, str - ): - self.supported_basecall_models = ( - tags["available basecall models"] - .array_value[1:-1] - .replace('"', "") - .split(",") - ) - # Faff on with sorting out available barcoding kits - # See https://github.com/nanoporetech/minknow_api/blob/829dbe8ac8e49efdf268d385b50440c52473188b/python/minknow_api/tools/protocols.py#L97C1-L97C7 - self.supported_barcode_kits = tags["barcoding kits"].array_value - # workaround for the set of barcoding kits being returned as a string rather - # that array of strings - if self.supported_barcode_kits and isinstance( - self.supported_barcode_kits, str - ): - self.supported_barcode_kits = ( - tags["barcoding kits"].array_value[1:-1].replace('"', "").split(",") - ) - - if tags["barcoding"].bool_value: - self.supported_barcode_kits.append(tags["kit"].string_value) - # If we are connected to a live run, and have suitable base calling models check the base-caller model is suitable for the flowcell and kit - if ( - self.supported_basecall_models - and f"{self.guppy_params['config'].replace('.cfg', '')}.cfg" - not in self.supported_basecall_models - ): - raise RuntimeError( - """The {} base-calling config listed in the readfish config TOML is not suitable for this flowcell and kit combination. - Please check the guppy_config value in the caller_settings.guppy section of your TOML file. - The following models are are given by ONT as suitable for this flow cell/kit combo:\n\t{}""".format( - self.guppy_params["config"], - nice_join( - self.supported_basecall_models, - sep="\n\t", - conjunction="and", - ), - ) - ) - - # If we are barcoding and have connected to a live run - try checking the listed barcode kit works with the flowcell and kit - if ( - barcoding_kits := self.guppy_params.get("barcode_kits", None) - ) is not None: - barcoding_kits = barcoding_kits.split() - if barcoding_kits and not set(barcoding_kits).issubset( - self.supported_barcode_kits - ): - raise RuntimeError( - "Barcoding kits specified in TOML {} not amongst those supported by the selected kit and protocol.\nSupported kits are:\n\t{}".format( - nice_join(barcoding_kits, conjunction="and"), - nice_join( - self.supported_barcode_kits, sep="\n\t", conjunction="and" - ), - ), - ) - return None - - def disconnect(self) -> None: - """Call the disconnect method on the PyGuppyClient""" - return self.caller.disconnect() - - def basecall( - self, - reads: Iterable[tuple[int, minknow_api.data_pb2.GetLiveReadsResponse.ReadData]], - signal_dtype: npt.DTypeLike, - daq_values: dict[int, namedtuple] = None, - ): - """Basecall live data from minknow RPC - - :param reads: List or generator of tuples containing (channel, MinKNOW.rpc.Read) - :param signal_dtype: Numpy dtype of the raw data - :param daq_values: Dictionary mapping channel to offset and scaling values. - If not provided default values of 1.0 and 0.0 are used. - :yield: - :rtype: readfish.plugins.utils.Result - """ - # FIXME: Occasionally guppy can report a read as not sent when it is - # successfully sent. Therefore we capture not sent reads - cache, skipped = {}, {} - reads_received, reads_sent = 0, 0 - daq_values = _DefaultDAQValues if daq_values is None else daq_values - - for channel, read in reads: - # Attach the "RF-" prefix - read_id = f"RF-{read.id}" - t0 = time.time() - cache[read_id] = (channel, read.number, t0) - success = self.caller.pass_read( - package_read( - read_id=read_id, - raw_data=np.frombuffer(read.raw_data, signal_dtype), - daq_offset=daq_values[channel].offset, - daq_scaling=daq_values[channel].scaling, - ) - ) - if not success: - logging.warning(f"Could not send read {read_id!r} to Guppy") - # FIXME: This is resolved in later versions of guppy. - skipped[read_id] = cache.pop(read_id) - continue - else: - reads_sent += 1 - - sleep_time = self.caller.throttle - t0 - if sleep_time > 0: - time.sleep(sleep_time) - - while reads_received < reads_sent: - results = self.caller.get_completed_reads() - # TODO: incorporate time_received into logging? - # time_received = time.time() - - if not results: - time.sleep(self.caller.throttle) - continue - - for res_batch in results: - for res in res_batch: - read_id = res["metadata"]["read_id"] - try: - channel, read_number, time_sent = cache.pop(read_id) - except KeyError: - # FIXME: This is resolved in later versions of guppy. - channel, read_number, time_sent = skipped.pop(read_id) - reads_sent += 1 - res["metadata"]["read_id"] = read_id[3:] - self.logger.debug( - "@%s ch=%s\n%s\n+\n%s", - res["metadata"]["read_id"], - channel, - res["datasets"]["sequence"], - res["datasets"]["qstring"], - ) - barcode = res["metadata"].get("barcode_arrangement", None) - # TODO: Add Filter here - yield Result( - channel=channel, - read_number=read_number, - read_id=res["metadata"]["read_id"], - seq=res["datasets"]["sequence"], - barcode=barcode if barcode else None, - basecall_data=res, - ) - reads_received += 1 - - def describe(self) -> str: - """ - Describe the guppy Caller - - :return: Description of parameters passed to this guppy Caller plugin - """ - description = ["Utilising the Guppy base-caller plugin:"] - for param in self.guppy_params.keys(): - description.append(f"\t- {param}: {self.guppy_params[param]}") - return "\n".join(description) diff --git a/src/readfish/plugins/utils.py b/src/readfish/plugins/utils.py index b07684c7..cca60db6 100644 --- a/src/readfish/plugins/utils.py +++ b/src/readfish/plugins/utils.py @@ -481,7 +481,6 @@ class Result: barcoder, and then the aligner. :param channel: The channel that this read is being sequenced on - :param read_number: The read number value from the Read Until API :param read_id: The read ID assigned to this read by MinKNOW :param seq: The basecalled sequence for this read :param decision: The ``Decision`` that has been made, this will by used to determine the ``Action`` @@ -491,7 +490,6 @@ class Result: """ channel: int - read_number: int read_id: str seq: str decision: Decision = attrs.field(default=Decision.no_seq) diff --git a/src/readfish/read_until/LICENSE.md b/src/readfish/read_until/LICENSE.md new file mode 100644 index 00000000..455e5174 --- /dev/null +++ b/src/readfish/read_until/LICENSE.md @@ -0,0 +1,362 @@ +This Source Code Form is subject to the terms of the Mozilla Public +License, v. 2.0. If a copy of the MPL was not distributed with this +file, You can obtain one at http://mozilla.org/MPL/2.0/. + +Copyright (c) 2016-2024 Oxford Nanopore Technologies plc. + + +Mozilla Public License Version 2.0 +================================== + +### 1. Definitions + +**1.1. “Contributor”** + means each individual or legal entity that creates, contributes to + the creation of, or owns Covered Software. + +**1.2. “Contributor Version”** + means the combination of the Contributions of others (if any) used + by a Contributor and that particular Contributor's Contribution. + +**1.3. “Contribution”** + means Covered Software of a particular Contributor. + +**1.4. “Covered Software”** + means Source Code Form to which the initial Contributor has attached + the notice in Exhibit A, the Executable Form of such Source Code + Form, and Modifications of such Source Code Form, in each case + including portions thereof. + +**1.5. “Incompatible With Secondary Licenses”** + means + +* **(a)** that the initial Contributor has attached the notice described + in Exhibit B to the Covered Software; or +* **(b)** that the Covered Software was made available under the terms of + version 1.1 or earlier of the License, but not also under the + terms of a Secondary License. + +**1.6. “Executable Form”** + means any form of the work other than Source Code Form. + +**1.7. “Larger Work”** + means a work that combines Covered Software with other material, in + a separate file or files, that is not Covered Software. + +**1.8. “License”** + means this document. + +**1.9. “Licensable”** + means having the right to grant, to the maximum extent possible, + whether at the time of the initial grant or subsequently, any and + all of the rights conveyed by this License. + +**1.10. “Modifications”** + means any of the following: + +* **(a)** any file in Source Code Form that results from an addition to, + deletion from, or modification of the contents of Covered + Software; or +* **(b)** any new file in Source Code Form that contains any Covered + Software. + +**1.11. “Patent Claims” of a Contributor** + means any patent claim(s), including without limitation, method, + process, and apparatus claims, in any patent Licensable by such + Contributor that would be infringed, but for the grant of the + License, by the making, using, selling, offering for sale, having + made, import, or transfer of either its Contributions or its + Contributor Version. + +**1.12. “Secondary License”** + means either the GNU General Public License, Version 2.0, the GNU + Lesser General Public License, Version 2.1, the GNU Affero General + Public License, Version 3.0, or any later versions of those + licenses. + +**1.13. “Source Code Form”** + means the form of the work preferred for making modifications. + +**1.14. “You” (or “Your”)** + means an individual or a legal entity exercising rights under this + License. For legal entities, “You” includes any entity that + controls, is controlled by, or is under common control with You. For + purposes of this definition, “control” means **(a)** the power, direct + or indirect, to cause the direction or management of such entity, + whether by contract or otherwise, or **(b)** ownership of more than + fifty percent (50%) of the outstanding shares or beneficial + ownership of such entity. + + +### 2. License Grants and Conditions + +#### 2.1. Grants + +Each Contributor hereby grants You a world-wide, royalty-free, +non-exclusive license: + +* **(a)** under intellectual property rights (other than patent or trademark) + Licensable by such Contributor to use, reproduce, make available, + modify, display, perform, distribute, and otherwise exploit its + Contributions, either on an unmodified basis, with Modifications, or + as part of a Larger Work; and +* **(b)** under Patent Claims of such Contributor to make, use, sell, offer + for sale, have made, import, and otherwise transfer either its + Contributions or its Contributor Version. + +#### 2.2. Effective Date + +The licenses granted in Section 2.1 with respect to any Contribution +become effective for each Contribution on the date the Contributor first +distributes such Contribution. + +#### 2.3. Limitations on Grant Scope + +The licenses granted in this Section 2 are the only rights granted under +this License. No additional rights or licenses will be implied from the +distribution or licensing of Covered Software under this License. +Notwithstanding Section 2.1(b) above, no patent license is granted by a +Contributor: + +* **(a)** for any code that a Contributor has removed from Covered Software; + or +* **(b)** for infringements caused by: **(i)** Your and any other third party's + modifications of Covered Software, or **(ii)** the combination of its + Contributions with other software (except as part of its Contributor + Version); or +* **(c)** under Patent Claims infringed by Covered Software in the absence of + its Contributions. + +This License does not grant any rights in the trademarks, service marks, +or logos of any Contributor (except as may be necessary to comply with +the notice requirements in Section 3.4). + +#### 2.4. Subsequent Licenses + +No Contributor makes additional grants as a result of Your choice to +distribute the Covered Software under a subsequent version of this +License (see Section 10.2) or under the terms of a Secondary License (if +permitted under the terms of Section 3.3). + +#### 2.5. Representation + +Each Contributor represents that the Contributor believes its +Contributions are its original creation(s) or it has sufficient rights +to grant the rights to its Contributions conveyed by this License. + +#### 2.6. Fair Use + +This License is not intended to limit any rights You have under +applicable copyright doctrines of fair use, fair dealing, or other +equivalents. + +#### 2.7. Conditions + +Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted +in Section 2.1. + + +### 3. Responsibilities + +#### 3.1. Distribution of Source Form + +All distribution of Covered Software in Source Code Form, including any +Modifications that You create or to which You contribute, must be under +the terms of this License. You must inform recipients that the Source +Code Form of the Covered Software is governed by the terms of this +License, and how they can obtain a copy of this License. You may not +attempt to alter or restrict the recipients' rights in the Source Code +Form. + +#### 3.2. Distribution of Executable Form + +If You distribute Covered Software in Executable Form then: + +* **(a)** such Covered Software must also be made available in Source Code + Form, as described in Section 3.1, and You must inform recipients of + the Executable Form how they can obtain a copy of such Source Code + Form by reasonable means in a timely manner, at a charge no more + than the cost of distribution to the recipient; and + +* **(b)** You may distribute such Executable Form under the terms of this + License, or sublicense it under different terms, provided that the + license for the Executable Form does not attempt to limit or alter + the recipients' rights in the Source Code Form under this License. + +#### 3.3. Distribution of a Larger Work + +You may create and distribute a Larger Work under terms of Your choice, +provided that You also comply with the requirements of this License for +the Covered Software. If the Larger Work is a combination of Covered +Software with a work governed by one or more Secondary Licenses, and the +Covered Software is not Incompatible With Secondary Licenses, this +License permits You to additionally distribute such Covered Software +under the terms of such Secondary License(s), so that the recipient of +the Larger Work may, at their option, further distribute the Covered +Software under the terms of either this License or such Secondary +License(s). + +#### 3.4. Notices + +You may not remove or alter the substance of any license notices +(including copyright notices, patent notices, disclaimers of warranty, +or limitations of liability) contained within the Source Code Form of +the Covered Software, except that You may alter any license notices to +the extent required to remedy known factual inaccuracies. + +#### 3.5. Application of Additional Terms + +You may choose to offer, and to charge a fee for, warranty, support, +indemnity or liability obligations to one or more recipients of Covered +Software. However, You may do so only on Your own behalf, and not on +behalf of any Contributor. You must make it absolutely clear that any +such warranty, support, indemnity, or liability obligation is offered by +You alone, and You hereby agree to indemnify every Contributor for any +liability incurred by such Contributor as a result of warranty, support, +indemnity or liability terms You offer. You may include additional +disclaimers of warranty and limitations of liability specific to any +jurisdiction. + + +### 4. Inability to Comply Due to Statute or Regulation + +If it is impossible for You to comply with any of the terms of this +License with respect to some or all of the Covered Software due to +statute, judicial order, or regulation then You must: **(a)** comply with +the terms of this License to the maximum extent possible; and **(b)** +describe the limitations and the code they affect. Such description must +be placed in a text file included with all distributions of the Covered +Software under this License. Except to the extent prohibited by statute +or regulation, such description must be sufficiently detailed for a +recipient of ordinary skill to be able to understand it. + + +### 5. Termination + +**5.1.** The rights granted under this License will terminate automatically +if You fail to comply with any of its terms. However, if You become +compliant, then the rights granted under this License from a particular +Contributor are reinstated **(a)** provisionally, unless and until such +Contributor explicitly and finally terminates Your grants, and **(b)** on an +ongoing basis, if such Contributor fails to notify You of the +non-compliance by some reasonable means prior to 60 days after You have +come back into compliance. Moreover, Your grants from a particular +Contributor are reinstated on an ongoing basis if such Contributor +notifies You of the non-compliance by some reasonable means, this is the +first time You have received notice of non-compliance with this License +from such Contributor, and You become compliant prior to 30 days after +Your receipt of the notice. + +**5.2.** If You initiate litigation against any entity by asserting a patent +infringement claim (excluding declaratory judgment actions, +counter-claims, and cross-claims) alleging that a Contributor Version +directly or indirectly infringes any patent, then the rights granted to +You by any and all Contributors for the Covered Software under Section +2.1 of this License shall terminate. + +**5.3.** In the event of termination under Sections 5.1 or 5.2 above, all +end user license agreements (excluding distributors and resellers) which +have been validly granted by You or Your distributors under this License +prior to termination shall survive termination. + + +### 6. Disclaimer of Warranty + +> Covered Software is provided under this License on an “as is” +> basis, without warranty of any kind, either expressed, implied, or +> statutory, including, without limitation, warranties that the +> Covered Software is free of defects, merchantable, fit for a +> particular purpose or non-infringing. The entire risk as to the +> quality and performance of the Covered Software is with You. +> Should any Covered Software prove defective in any respect, You +> (not any Contributor) assume the cost of any necessary servicing, +> repair, or correction. This disclaimer of warranty constitutes an +> essential part of this License. No use of any Covered Software is +> authorized under this License except under this disclaimer. + +### 7. Limitation of Liability + +> Under no circumstances and under no legal theory, whether tort +> (including negligence), contract, or otherwise, shall any +> Contributor, or anyone who distributes Covered Software as +> permitted above, be liable to You for any direct, indirect, +> special, incidental, or consequential damages of any character +> including, without limitation, damages for lost profits, loss of +> goodwill, work stoppage, computer failure or malfunction, or any +> and all other commercial damages or losses, even if such party +> shall have been informed of the possibility of such damages. This +> limitation of liability shall not apply to liability for death or +> personal injury resulting from such party's negligence to the +> extent applicable law prohibits such limitation. Some +> jurisdictions do not allow the exclusion or limitation of +> incidental or consequential damages, so this exclusion and +> limitation may not apply to You. + + +### 8. Litigation + +Any litigation relating to this License may be brought only in the +courts of a jurisdiction where the defendant maintains its principal +place of business and such litigation shall be governed by laws of that +jurisdiction, without reference to its conflict-of-law provisions. +Nothing in this Section shall prevent a party's ability to bring +cross-claims or counter-claims. + + +### 9. Miscellaneous + +This License represents the complete agreement concerning the subject +matter hereof. If any provision of this License is held to be +unenforceable, such provision shall be reformed only to the extent +necessary to make it enforceable. Any law or regulation which provides +that the language of a contract shall be construed against the drafter +shall not be used to construe this License against a Contributor. + + +### 10. Versions of the License + +#### 10.1. New Versions + +Mozilla Foundation is the license steward. Except as provided in Section +10.3, no one other than the license steward has the right to modify or +publish new versions of this License. Each version will be given a +distinguishing version number. + +#### 10.2. Effect of New Versions + +You may distribute the Covered Software under the terms of the version +of the License under which You originally received the Covered Software, +or under the terms of any subsequent version published by the license +steward. + +#### 10.3. Modified Versions + +If you create software not governed by this License, and you want to +create a new license for such software, you may create and use a +modified version of this License if you rename the license and remove +any references to the name of the license steward (except to note that +such modified license differs from this License). + +#### 10.4. Distributing Source Code Form that is Incompatible With Secondary Licenses + +If You choose to distribute Source Code Form that is Incompatible With +Secondary Licenses under the terms of this version of the License, the +notice described in Exhibit B of this License must be attached. + +## Exhibit A - Source Code Form License Notice + + This Source Code Form is subject to the terms of the Mozilla Public + License, v. 2.0. If a copy of the MPL was not distributed with this + file, You can obtain one at http://mozilla.org/MPL/2.0/. + +If it is not possible or desirable to put the notice in a particular +file, then You may include the notice in a location (such as a LICENSE +file in a relevant directory) where a recipient would be likely to look +for such a notice. + +You may add additional accurate notices of copyright ownership. + +## Exhibit B - “Incompatible With Secondary Licenses” Notice + + This Source Code Form is "Incompatible With Secondary Licenses", as + defined by the Mozilla Public License, v. 2.0. diff --git a/src/readfish/read_until/_version.py b/src/readfish/read_until/_version.py index a5cfdf59..dae42b1b 100644 --- a/src/readfish/read_until/_version.py +++ b/src/readfish/read_until/_version.py @@ -1 +1 @@ -__version__ = "3.4.1" +__version__ = "3.5.2" diff --git a/src/readfish/read_until/base.py b/src/readfish/read_until/base.py index d04f9720..04bdcec9 100644 --- a/src/readfish/read_until/base.py +++ b/src/readfish/read_until/base.py @@ -138,7 +138,7 @@ def __init__( self.prefilter_classes = prefilter_classes # Stores the most recent read number that a decision has been made on (stop_receiving/unblock) - self.channel_read_latest_decision = defaultdict(int) + self.channel_read_latest_decision = defaultdict(str) try: self.connection = Connection( @@ -274,7 +274,7 @@ def reset(self): # requests before they are put on the gRPC stream. self.action_queue = queue.Queue() - self.channel_read_latest_decision = defaultdict(int) + self.channel_read_latest_decision = defaultdict(str) # the data_queue is used to store the latest chunk per channel self.data_queue = self.CacheType(size=self.cache_size) @@ -349,7 +349,7 @@ def get_read_chunks(self, batch_size=1, last=True): data = [ (channel, read) for (channel, read) in data - if read.number > self.channel_read_latest_decision[channel] + if read.id not in self.channel_read_latest_decision[channel] ] return data @@ -571,10 +571,10 @@ def _process_reads(self, reads): self.logger.debug( "Rereceived %s:%s after stop request.", read_channel, - read.number, + read.id, ) continue - self.stop_receiving_read(read_channel, read.number) + self.stop_receiving_read(read_channel, read.id) unique_reads.add(read.id) read_samples_behind = progress.acquired - read.chunk_start_sample samples_behind += read_samples_behind @@ -622,7 +622,7 @@ def _generate_action(self, read_channel, read_number, action, **params): action_kwargs = { "action_id": action_id, "channel": read_channel, - "number": read_number, + "id": read_number, } self.sent_actions[action_id] = action if action == "stop_further_data": diff --git a/src/readfish/read_until/read_cache.py b/src/readfish/read_until/read_cache.py index 20fc0c24..8b0db20b 100644 --- a/src/readfish/read_until/read_cache.py +++ b/src/readfish/read_until/read_cache.py @@ -78,7 +78,7 @@ def __setitem__(self, key, value): with self.lock: # Check if same read if key in self._dict: - if self._dict[key].number == value.number: + if self._dict[key].id == value.id: # Same read self.replaced += 1 else: @@ -219,7 +219,7 @@ def __setitem__(self, key, value): self._dict[key] = value else: # Key exists - if self[key].number == value.number: + if self[key].id == value.id: # Same read, update raw_data self[key].raw_data += value.raw_data self.replaced += 1 diff --git a/tests/conftest.py b/tests/conftest.py index 9cf4cddf..474b1326 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -9,7 +9,7 @@ def reset_file_perms(): """ TEST_DIR = Path(__file__).parent.resolve() SEARCH_PATH = Path(TEST_DIR / "static") - nr_socket_path = SEARCH_PATH / "guppy_validation_test" / "fail" / "5555_fail_nr" + nr_socket_path = SEARCH_PATH / "dorado_validation_test" / "fail" / "5555_fail_nr" os.chmod(nr_socket_path, 0o660) diff --git a/tests/static/guppy_validation_test/README.md b/tests/static/dorado_validation_test/README.md similarity index 100% rename from tests/static/guppy_validation_test/README.md rename to tests/static/dorado_validation_test/README.md diff --git a/tests/static/guppy_validation_test/fail/001_no_write_socket.txt b/tests/static/dorado_validation_test/fail/001_no_write_socket.txt similarity index 100% rename from tests/static/guppy_validation_test/fail/001_no_write_socket.txt rename to tests/static/dorado_validation_test/fail/001_no_write_socket.txt diff --git a/tests/static/guppy_validation_test/fail/002_missing_socket.txt b/tests/static/dorado_validation_test/fail/002_missing_socket.txt similarity index 100% rename from tests/static/guppy_validation_test/fail/002_missing_socket.txt rename to tests/static/dorado_validation_test/fail/002_missing_socket.txt diff --git a/tests/static/guppy_validation_test/fail/003_no_read_socket.txt b/tests/static/dorado_validation_test/fail/003_no_read_socket.txt similarity index 100% rename from tests/static/guppy_validation_test/fail/003_no_read_socket.txt rename to tests/static/dorado_validation_test/fail/003_no_read_socket.txt diff --git a/tests/static/guppy_validation_test/fail/5555_fail_nr b/tests/static/dorado_validation_test/fail/5555_fail_nr similarity index 100% rename from tests/static/guppy_validation_test/fail/5555_fail_nr rename to tests/static/dorado_validation_test/fail/5555_fail_nr diff --git a/tests/static/guppy_validation_test/fail/5555_fail_nw b/tests/static/dorado_validation_test/fail/5555_fail_nw similarity index 100% rename from tests/static/guppy_validation_test/fail/5555_fail_nw rename to tests/static/dorado_validation_test/fail/5555_fail_nw diff --git a/tests/static/guppy_validation_test/pass/001_correct_debug.txt b/tests/static/dorado_validation_test/pass/001_correct_debug.txt similarity index 100% rename from tests/static/guppy_validation_test/pass/001_correct_debug.txt rename to tests/static/dorado_validation_test/pass/001_correct_debug.txt diff --git a/tests/static/guppy_validation_test/pass/002_correct_no_debug.txt b/tests/static/dorado_validation_test/pass/002_correct_no_debug.txt similarity index 100% rename from tests/static/guppy_validation_test/pass/002_correct_no_debug.txt rename to tests/static/dorado_validation_test/pass/002_correct_no_debug.txt diff --git a/tests/static/guppy_validation_test/pass/5555_pass b/tests/static/dorado_validation_test/pass/5555_pass similarity index 100% rename from tests/static/guppy_validation_test/pass/5555_pass rename to tests/static/dorado_validation_test/pass/5555_pass diff --git a/tests/static/guppy_validation_test/fail/001_no_write_socket.toml b/tests/static/guppy_validation_test/fail/001_no_write_socket.toml deleted file mode 100644 index 4cf888dd..00000000 --- a/tests/static/guppy_validation_test/fail/001_no_write_socket.toml +++ /dev/null @@ -1,14 +0,0 @@ -[caller_settings.dorado] -address = "ipc://tests/static/guppy_validation_test/fail/5555_fail_nw" -config = "dna_r10.4.1_e8.2_400bps_hac" - -[mapper_settings.no_op] - -[[regions]] -name = "abc" -no_seq = "proceed" -no_map = "proceed" -single_on = "proceed" -single_off = "proceed" -multi_on = "proceed" -multi_off = "proceed" diff --git a/tests/static/guppy_validation_test/fail/002_missing_socket.toml b/tests/static/guppy_validation_test/fail/002_missing_socket.toml deleted file mode 100644 index e21c05f1..00000000 --- a/tests/static/guppy_validation_test/fail/002_missing_socket.toml +++ /dev/null @@ -1,14 +0,0 @@ -[caller_settings.dorado] -address = "ipc://tests/static/guppy_validation_test/fail/5555_not_there" -config = "dna_r10.4.1_e8.2_400bps_hac" - -[mapper_settings.no_op] - -[[regions]] -name = "abc" -no_seq = "proceed" -no_map = "proceed" -single_on = "proceed" -single_off = "proceed" -multi_on = "proceed" -multi_off = "proceed" diff --git a/tests/static/guppy_validation_test/fail/003_no_read_socket.toml b/tests/static/guppy_validation_test/fail/003_no_read_socket.toml deleted file mode 100644 index c7eaf018..00000000 --- a/tests/static/guppy_validation_test/fail/003_no_read_socket.toml +++ /dev/null @@ -1,14 +0,0 @@ -[caller_settings.dorado] -address = "ipc://tests/static/guppy_validation_test/fail/5555_fail_nr" -config = "dna_r10.4.1_e8.2_400bps_hac" - -[mapper_settings.no_op] - -[[regions]] -name = "abc" -no_seq = "proceed" -no_map = "proceed" -single_on = "proceed" -single_off = "proceed" -multi_on = "proceed" -multi_off = "proceed" diff --git a/tests/static/guppy_validation_test/pass/001_correct_debug.toml b/tests/static/guppy_validation_test/pass/001_correct_debug.toml deleted file mode 100644 index dbc44ac2..00000000 --- a/tests/static/guppy_validation_test/pass/001_correct_debug.toml +++ /dev/null @@ -1,14 +0,0 @@ -[caller_settings.no_op] -address = "ipc://tests/static/guppy_validation_test/pass/5555_pass" -config = "dna_r10.4.1_e8.2_400bps_hac" - -[mapper_settings.no_op] - -[[regions]] -name = "abc" -no_seq = "proceed" -no_map = "proceed" -single_on = "proceed" -single_off = "proceed" -multi_on = "proceed" -multi_off = "proceed" diff --git a/tests/static/guppy_validation_test/pass/002_correct_no_debug.toml b/tests/static/guppy_validation_test/pass/002_correct_no_debug.toml deleted file mode 100644 index dbc44ac2..00000000 --- a/tests/static/guppy_validation_test/pass/002_correct_no_debug.toml +++ /dev/null @@ -1,14 +0,0 @@ -[caller_settings.no_op] -address = "ipc://tests/static/guppy_validation_test/pass/5555_pass" -config = "dna_r10.4.1_e8.2_400bps_hac" - -[mapper_settings.no_op] - -[[regions]] -name = "abc" -no_seq = "proceed" -no_map = "proceed" -single_on = "proceed" -single_off = "proceed" -multi_on = "proceed" -multi_off = "proceed" diff --git a/tests/validation_test.py b/tests/validation_test.py index 1445e61f..2601df3b 100644 --- a/tests/validation_test.py +++ b/tests/validation_test.py @@ -11,14 +11,14 @@ def ch_file_perms(): """ - Change the file permissions on the fake Guppy sockets, if they aren't correct. + Change the file permissions on the fake dorado sockets, if they aren't correct. Most likely this is the case after a fresh clone. Sets the no read socket (5555_fail_nr) to allow no one to read, and the no write socket (5555_fail_nw) to allow no one to write. """ - nr_socket_path = SEARCH_PATH / "guppy_validation_test" / "fail" / "5555_fail_nr" - nw_socket_path = SEARCH_PATH / "guppy_validation_test" / "fail" / "5555_fail_nw" + nr_socket_path = SEARCH_PATH / "dorado_validation_test" / "fail" / "5555_fail_nr" + nw_socket_path = SEARCH_PATH / "dorado_validation_test" / "fail" / "5555_fail_nw" if os.access(nr_socket_path, os.R_OK): os.chmod(nr_socket_path, 0o260)