Importing Qiime2 biom file

[sbudree]

Hi,

I have created a feature table using Qiime2 and have exported this as a biom file. However, this biom file cannot be imported into phyloseq [Error in colnames<-(*tmp*, value = c("ta1", "ta0")) :
length of 'dimnames' [2] not equal to array extent
In addition: There were 50 or more warnings (use warnings() to see the first 50)]

Can you advise me on how to import the Qiime2 biom file into Phloseq?

[joey711]

Can you share the file?

[sbudree]

Hi, File attached.

…

-Shrish From: "Paul J. McMurdie" <notifications@github.com> Reply-To: joey711/phyloseq <reply@reply.github.com> Date: Saturday, September 16, 2017 at 7:19 PM To: joey711/phyloseq <phyloseq@noreply.github.com> Cc: sbudree <dr.s.budree@gmail.com>, Author <author@noreply.github.com> Subject: Re: [joey711/phyloseq] Importing Qiime2 biom file (#821) Can you share the file? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[joey711]

Actually, can you post a link? Although you've responded via email, this is actually still on the phyloseq issues tracker:

#821

so email attachments don't work. Any file-hosting site will do. Some (like dropbox) make it easy to stop sharing the file after a certain amount of time, or when you say so.

[sbudree]

Hi,

Link to biom file: https://www.dropbox.com/s/16qsa311jn2bjkb/feature-table.biom?dl=0

[ViridianaAvila]

Hi,
Today I had the same error. I realized for some reason biom files generated by QIIME2 does not include the taxonomy. I had to extract the taxonomy and the tree from the qza files and merge them with tsv biom then back to a new biom file. That solved my issue, I hope this information is useful. Here is what I did:

1. Uncompress the qza files (table, tree and taxonomy). unzip will do.

2. Enter to the folder of uncompressed table, you will find a feature-table.biom file.

3. For the easy manipulation of this feature-table.biom convert to .txt using:
   biom convert -i feature-table.biom -o otu_table.txt --to-tsv

4. This table can be open in excel or R anything where you can merge by OTU ID the taxonomy information using the taxonomy.tsv file will do. (This taxonomy.tsv file comes from the decompression of taxonomy.qza)

5. Once the taxonomy info is merged as a column in the otu_table.txt, convert again this file using the following command:
   biom convert -i otu_table.txt -o new_otu_table.biom --to-hdf5 --table-type="OTU table" --process-obs-metadata taxonomy

6. Loads in R
   biom_otu <- import_biom(BIOMfilename = "new_biom.biom", treefilename = "tree.nwk")
   (The tree file is coming from decompressing unrooted-tree.qza or rooted-tree.qza)

[ksdiaz]

Hello,

I've been having this similar issue as well. I can't use the biom convert circumvention because biom throws an error when trying to do Step 5 ViridianaAvila suggested ("TypeError: can only join an iterable", I suspect this has to do with the qiime format itself, as trying to validate the original biom file from the qiime qza returns it as an invalid file). So, I tried adding the taxonomy as a new column into the converted text file. Trying to load this into phyloseq like this:

otufile <- "feature-table-taxonomy.txt"
mapfile <- "phylGastrotricha_mapping.tsv"
treefile <- "phylGastro_tree.nwk"
qiimetable <- import_qiime(otufile, mapfile, treefile, parseFunction = parse_taxonomy_qiime)

gives me the following error:

Processing map file...
Processing otu/tax file...
Reading file into memory prior to parsing...
Detecting first header line...
Header is on line 2  
Converting input file to a table...
Defining OTU table... 
Adding new column 'Consensus Lineage' then assigning NULL (deleting it).Adding new column
 '#OTU ID' then assigning NULL (deleting it).Parsing taxonomy table...
Error in taxlist[[i]] : subscript out of bounds

Traceback in Rstudio shows me this:

Error in taxlist[[i]] : subscript out of bounds 
3. build_tax_table(taxlist) 
2. import_qiime_otu_tax(otufilename, parseFunction, verbose = verbose) 
1. import_qiime(otufile, mapfile, treefile, parseFunction = parse_taxonomy_qiime)

I've attached the table I was trying to import.

feature-table-taxonomy.txt

[jme6f4]

Hi, Just came across this and had the same problem. I'm able to run the "moving pictures" tutorial fine (otu_table_mc2_w_tax_no_pynast_failures.biom), but not my own data. The only difference that I can spot is that my .biom file was produced by QIIME2.

[jackmen]

Hi guys,

the approach of @ViridianaAvila works and is the one I have been using quite often in the past. Be aware that you have to give a header to the taxonomy column in your otu_table.txt and this header should be the same name as used behind the command "obs-metadata". @ksdiaz : Your taxonomy has no column header. Name the column header for the taxonomy column in the otu_table.txt "taxonomy" when using this command:

biom convert -i otu_table.txt -o new_otu_table.biom --to-hdf5 --table-type="OTU table" --process-obs-metadata taxonomy

@jme6f4 : You might want to check if you did the same mistake.

Then in phyloseq simply do:

biom <-import_biom ("new_otu_table.biom", parseFunction = parse_taxonomy_greengenes)
map <-import_qiime_sample_data ('mapping_file.txt')
tree <- read_tree_greengenes ("tree.nwk")
class <- merge_phyloseq (biom, map, tree)

Also: The qiime2 taxonomy column header in taxonomy.tsv is "Taxon" and if only pasted to otu_table.txt is not recognized. I guess changing to "obs-metadata Taxon" should also work fine.

Good luck!

[HRRTPH]

Hi everyone,
I am trying to import Qiime2 output files into Phyloseq. My commands are here:
otufile = system.file("extdata", "feature-table.biom", package="phyloseq")
mapfile = system.file("extdata", "G3_metadata.txt", package="phyloseq")
trefile = system.file("extdata", "GP_tree_rand_short.newick.gz", package="phyloseq")
rs_file = system.file("extdata", "dna-sequences.fasta", package="phyloseq")
qiimedata = import_qiime(otufile, mapfile, trefile, rs_file)

I used qiime tools export to get those files except the mapfile which extension was changed from tsv to txt. After running qiimedata, i got an error:

Processing map file...
Error in read.table(file = mapfilename, header = TRUE, sep = "\t", comment.char = "") :
no lines available in input
In addition: Warning message:
In file(file, "rt") : file("") only supports open = "w+" and open = "w+b": using the former

Does anyone know this error?

Please help!!!

Thank you very much,

Toan

[Biancabrown]

Hello,
I'm currently having the same issue. When I try to convert the txt file that I generated with the taxonomy header to a biom file I get the following error message "ValueError: could not convert string to float: NA" Any ideas?

[Nourhanelsahly]

hello,
I have the same problem (importing biom to phyloseq), I followed up the steps in @ViridianaAvila and @jackmen comments, it went fine until converting the merged file (otu and taxonomy) to biom format again. The new biom file didn't contain the taxonomy column (I converted it to txt to see). So, its not imported to phyloseq.

Do you have any clue please?
I am attaching the merged txt file

output.txt

Hi not sure if you have sorted this out, however, I was having similar problems. This may not be your issue, but it was one of mine. In your output.txt file you have quotation marks for the boundaries of each cell, and the convert function can't recognise these. Instead of writing a .txt file in R I wrote it as .csv and used sep="\t" to get rid of the quotation marks. I then opened in excel and checked the headers were aligned and saved it as a .txt file. I then opened it in notepad and pasted #Constructed from BIOM file# at the start of the txt file as this was inserted by the software after the first convert and figured it might be important. At the moment you have "#OTU ID" and when you remove the brackets the # will probably stop it reading anything after it. Mine starts with this: #Constructed from BIOM file# OTU ID Sample1 Sample2 and so on. Once I had this format sorted everything else worked fine, good luck.

[brookeweigel]

Hello! I am also having problems getting my QIIME2 data into phyloseq. So far, I tried all of the above steps, including making a new .biom file with OTU abundances + taxonomy. I had the same problem as @Nourhanelsahly, even after following the above changes from @Ajsnevets by changing the header to match the OTUID for each file. Please help! I would love to use phyloseq, and since QIIME2 is now widely used, I wish that it was a lot easier to transition from QIIME2 output to phyloseq. I really want to figure this out!

See .txt file below:
table.from_biom.txt

See code:
biom add-metadata -i core-metrics-results/rarified-feature-table/feature-table.biom -o table-with-taxonomy.biom --observation-metadata-fp core-metrics-results/rarified-feature-table/taxonomy.tsv --sc-separated taxonomy

Apart from the issue of getting a .biom file with both taxonomy and OTU abundances, I am having trouble importing my data into phyloseq. See below.
otufile = system.file("extdata", "table-with-taxonomy.biom", package="phyloseq")
mapfile = system.file("extdata", "Sea_Cucumber_metadata.txt", package="phyloseq")
trefile = system.file("extdata", "tree.nwk", package="phyloseq")
qiimedata = import_qiime(otufile, mapfile, trefile)

Here is my mapping file:
Sea_Cucumber_metadata.txt

I get this error:

Processing map file...
Error in read.table(file = mapfilename, header = TRUE, sep = "\t", comment.char = "") :
no lines available in input
In addition: Warning message:
In file(file, "rt") :
file("") only supports open = "w+" and open = "w+b": using the former

There is no taxonomy data in this table, so I guess you didn't manage to merge them?
The first line of the table.from_biom.txt reads #[space]constructed from..[space]#[no space]OTU ID. This is one of several problems but the current placement of your [spaces] is going to stop it reading correctly, it has to be: #[no space]Constructed from BIOM file[no space]#[space]OTU ID Sample1 Sample2

Additionally though, the problem is that with your "table.from_biom.txt" file, the columns names are not lined up with the columns. Not sure how this got messed up.

(side note) Your tables seem to have been rarefied, I am no expert but my understanding is that DESEq2 allows you to look at the data without rarefying it like with qiime2.

I would go back to the original table.qza file, unzip your table.qza file and go through the folders until you found the .biom and then qiime2 convert it with:
biom convert -i feature-table.biom -o otu_table.txt --to-tsv
open the table in R where you can merge by OTU ID the taxonomy information using the taxonomy.tsv file. (This taxonomy.tsv file comes from the decompression of taxonomy.qza). If you can not open it in R (read.table(file = 'otu.tsv', sep = '\t', header = TRUE) then it is probably because of the #constructed from...# problem so open this in notepad, fix it and then open it in R

[brookeweigel]

Hi @Ajsnevets and everyone else trying to get QIIME2 data into phyloseq... after some exporting and merging in R (to get around the fact that after filtering out chloroplasts and mitochondria, my taxonomy files and OTU matrix have a different # of taxa), I was finally able to wrangle my QIIME2 data into Phyloseq without using any .biom files. Here is a pipeline that I wrote (see PDF below). It is pretty clunky and uses QIIME2 + R + excel, but at least it works!

It is also partially based on the answer from https://forum.qiime2.org/t/converting-biom-files-with-taxonomic-info-for-import-in-r-with-phyloseq/2542/5 from that doesn't use any .biom files.

QIIME2_to_Phyloseq.pdf