Word count

[m-Py]

Hi,

is it currently possible to count the number of words in the manuscript (excluding abstract and references) automated? The example manuscript states 'too lazy to count', which I take as a hint that this is not possible? Any other suggestions about how to do it, other than manual counting?

Best regards, Martin

[crsh]

Hi Martin,

the answer to this question depends a little on the target document type you are working with. For MS Word documents I'd recommend to simply generate the document and then enter the word count by hand using Word's word count feature. If you are going through LaTeX to produce a PDF I suggest you try TeXcount, which was probably installed along with your TeX distrubution. It's a command line tool (but there is also a web interface, see the link) and has served me well. Let me know if this works for you.

Best regards,
Frederik

[crsh]

Thinking a little more about this, I forgot to mention that the way pandoc manages references TeXcount will, by default, count your references against the word limit. So maybe you need to remove the reference section manually from the TeX source file before calling TeXcount.

[m-Py]

Hi Frederik,

thanks a lot for your response! I will look into TeXCount. I am converting into pdf, but I'll probably need to use MS Word as well as soon as I need to collaborate with others. Removing the reference section for word count is not a problem.

Best regards,
Martin

[crsh]

There's a new kid on the block: Try wordcountaddin. This is an R solution that works on the RMarkdown file directly. I haven't played with it and I'm not sure how reliable it is; e.g., I don't know if it counts text in tables etc. If you give it a try, I'd be interested in your experience.

[m-Py]

Ah, that seems nice. It says it is an RStudio addin, but you can also use an internal function to process any R strings without using RStudio. In a first test this worked, for example r chunks were not counted. It should be able to read whole Rmd text files, too. Thanks for that catch! I will report as soon as I have done more testing with Rmd files.

[m-Py]

I played around a little and it seems that I was able to write a function that reads an Rmd file and then uses the word count function of wordcountaddin to count all words in this file. Indeed, the r chunks, inline r and yaml headers are ignored. Inline LaTex commands are not ignored, but maybe this will be implemented at some point.

The two word count methods that are offered (koRpus & stringi) differ rather strongly in my case, but koRpus yields reasonable results. The koRpus estimate is close to what I get when I import the text into Libre Office and just remove all r chunks and the yaml header (stringi is actually very far off).

I will use this function from now on :-)

[crsh]

That's good to know. Thanks for providing the feedback. I had noticed rather large discrepancies between the word count methods, too, but I hadn't yet cross-validated them. Thanks for that. I'd be interested in the function you wrote because I've been wanting to automate word counting in papaja. Would you be willing to share your function, e.g., in a gist?

[m-Py]

Sure, gladly, here is the gist:

https://gist.github.com/m-Py/faf679a0a0be3dbafa2b43b390519923

I crossvalidated the function with the RStudio addin - the results are the same (at least they were for me, you might double check that ;) ).

[benmarwick]

@m-Py that's good to know about the accuracy of the two methods (I'm the author of the wordcountaddin), thanks for sharing your test results. I might drop the stringi method from the addin.

The addin will count text that are present in markdown tables in the Rmd file before the file is knit, but excluding those is on my list of things to do. It won't count tables generated by R code that only appear in the rendered document.

[m-Py]

@benmarwick glad I could help. The difference between the two estimates was really rather large, koRpus yielded ~ 10,000 words and stringi only estimated ~ 6,000 words. Thank you for creating this nice package, it is really of great use to me.

If you intend to include a function in your package that counts the words without necessarily using RStudio, feel free to just use the code in my gist above. I tried some code to process the text of an Rmd file so that it is apparently formatted as a text selection in RStudio, so this should work.

[ebergelson]

would be supercool if this could be integrated into the wordcount header in a .rmd manuscript!