Articles: Academic Workflow

Zotero and Roam Research

What makes academic writing different from most other writing? It's that you provide sources for facts and the claims you make.

Most of the writing you encounter does not do that. Think about how many newspaper articles you have read that mention a "new study by scientists at X university found that" without any link to the actual paper they are talking about.

Or the thousands of bloggers, Youtubers or Reddit commentators that do the same.

In academia, no-one takes you seriously if you do not provide documentation on where you got a finding, piece of information or dataset from. And academia is not alone in this – researchers gathering information for the finance and insurance industries, for example, also take great care to ensure that information is accurate and traceable.

It's no secret that I'm a believer in Roam Research to do my personal knowledge management, storing and connecting quotes I extract from my reading and writing papers and articles.

But while Roam is an incredible tool for thought, it's far from a useful reference management system on its own.

Which is fine, because there's another tool that plays very nicely with Roam if you set it up right. That tool is Zotero.

Zotero is a reference manager – it's the digital equivalent of a filing cabinet full of index cards with bibliographic information, and there's many alternatives. Endnote, Mendeley, Papers, Bibdesk…there are many reference managers out there.

What makes Zotero stand out is that it's open source and very flexible in how you get information into and out of it. That makes it a perfect fit for a workflow where you collect your sources, both files and bibliographic info, in Zotero, copy that info to Roam and link back to Zotero to have easy access to the files.

If you want to learn more about why Zotero is an awesome reference manager, I've written about that here.

In this guide I won't go into how to install Zotero or set it up – what I'll show you is how to get data out of Zotero and into Roam in a way that makes it useful in Roam and gives you easy access to attached files.

If you prefer a video version of this guide, you can also watch this video on Youtube:

Why use Roam together with Zotero?

Roam is a fantastic tool for thought, and provides you with many ways in which you can explore your reading. By getting metadata out of Zotero into Roam, you can use it to see new connections and relationships between the papers and books you read that you couldn't get out of using Zotero alone. And by leveraging Zotero's import and export capabilities, you have a much easier time getting that information into Roam than without Zotero.

To get information into Zotero, all you need is its browser connector. With that, getting author, publication date and more is trivial. But how do you get that information from Zotero into Roam?

Getting Metadata out of Zotero

The trick for this is to use a custom Zotero translator as they call them. If you know how to code it's relatively easy to write one yourself, but if you can't I'll share mine with you that should cover all the basics quite nicely.

Zotero translators are Javascript files that look into your Zotero database, take the information stored there and format it in a way that's useful for you.

The translator I've written produces output like this:

[[marquardt2019what]]
  author:: [[Kyle L. Marquardt]] [[Daniel Pemstein]] [[Brigitte Seim]] [[Yi-ting Wang]] 
  title:: [[What makes experts reliable? Expert reliability and the estimation of latent traits]]
  publication:: [[Research & Politics]]
  year:: [[2019]]
  url:: [Marquardt et al. (2019). What makes experts reliable? Expert reliability and the estimation of latent traits. Research & Politics](https://doi.org/10.1177/2053168019879561)
  Zotero link:: [Zotero Link](zotero://select/items/1_B6KWR3W2)
  topics:: 
  reading status:: [[to read]]
  citation graph:: {{[[query]]: {and: [[>]] [[citing]] [[marquardt2019what]] {not: [[query]]}}}}

Which in Roam turns into this:

Paper Metadata
Figure 1: Paper Metadata in Roam after Export from Zotero

What can you do with this?

First, I can look at the pages for individual authors and check out all the papers they have written or contributed to. I can also look at all the papers that appeared in "Research & Politics", and easily share a URL to the paper by simply copying from Roam without the need to open Zotero.

I can also directly go to the paper within Zotero to read the PDF, and by filling out the topics attribute I can have a look at all the papers that deal with expert surveys, for example.

Another cool capability is that the citation graph attribute and its query lets me collect all the places in other papers that refer to this one. All I need to do is indent a citing: attribute below a blockquote (created through [[>]]) that links to the title of the papers page (marquardt2019what) to see who referenced it in what context and who they reference. When you're exploring your literature, that's quite neat to have and impossible to get otherwise.

If you want the file that lets you do that, head over here and download it. To install it, you copy the file over to your Zotero > translators folder. Once you've done that, open up the Zotero preferences, head to the "Export" tab and select "Roam Metadata Export" in the "Default Format" dropdown. Now, all you need to do is hit Cmd-Shift-C when selecting a source in Zotero and it will copy the information formatted like above. Paste into Roam, and you're done! If anything is unclear, do watch the video – having a visual reference can be helpful.

A quick note: the translator requires you to have an author for every source, so make sure that information is there for each reference. It also works best if you use the BetterBibTex plugin for Zotero, which I highly recommend you install as well.

What makes academic writing different from most other writing? It's that you provide sources for facts and the claims you make.

Most of the writing you encounter does not do that. Think about how many newspaper articles you have read that mention a "new study by scientists at X university found that" without any link to the actual paper they are talking about.

Or the thousands of bloggers, Youtubers or Reddit commentators that do the same.

In academia, no-one takes you seriously if you do not provide documentation on where you got a finding, piece of information or dataset from. And academia is not alone in this – researchers gathering information for the finance and insurance industries, for example, also take great care to ensure that information is accurate and traceable.

It's no secret that I'm a believer in Roam Research to do my personal knowledge management, storing and connecting quotes I extract from my reading and writing papers and articles.

But while Roam is an incredible tool for thought, it's far from a useful reference management system on its own.

Which is fine, because there's another tool that plays very nicely with Roam if you set it up right. That tool is Zotero.

Zotero is a reference manager – it's the digital equivalent of a filing cabinet full of index cards with bibliographic information, and there's many alternatives. Endnote, Mendeley, Papers, Bibdesk…there are many reference managers out there.

What makes Zotero stand out is that it's open source and very flexible in how you get information into and out of it. That makes it a perfect fit for a workflow where you collect your sources, both files and bibliographic info, in Zotero, copy that info to Roam and link back to Zotero to have easy access to the files.

If you want to learn more about why Zotero is an awesome reference manager, I've written about that here.

In this guide I won't go into how to install Zotero or set it up – what I'll show you is how to get data out of Zotero and into Roam in a way that makes it useful in Roam and gives you easy access to attached files.

If you prefer a video version of this guide, you can also watch this video on Youtube:

Why use Roam together with Zotero?

Roam is a fantastic tool for thought, and provides you with many ways in which you can explore your reading. By getting metadata out of Zotero into Roam, you can use it to see new connections and relationships between the papers and books you read that you couldn't get out of using Zotero alone. And by leveraging Zotero's import and export capabilities, you have a much easier time getting that information into Roam than without Zotero.

To get information into Zotero, all you need is its browser connector. With that, getting author, publication date and more is trivial. But how do you get that information from Zotero into Roam?

Getting Metadata out of Zotero

The trick for this is to use a custom Zotero translator as they call them. If you know how to code it's relatively easy to write one yourself, but if you can't I'll share mine with you that should cover all the basics quite nicely.

Zotero translators are Javascript files that look into your Zotero database, take the information stored there and format it in a way that's useful for you.

The translator I've written produces output like this:

[[marquardt2019what]]
  author:: [[Kyle L. Marquardt]] [[Daniel Pemstein]] [[Brigitte Seim]] [[Yi-ting Wang]] 
  title:: [[What makes experts reliable? Expert reliability and the estimation of latent traits]]
  publication:: [[Research & Politics]]
  year:: [[2019]]
  url:: [Marquardt et al. (2019). What makes experts reliable? Expert reliability and the estimation of latent traits. Research & Politics](https://doi.org/10.1177/2053168019879561)
  Zotero link:: [Zotero Link](zotero://select/items/1_B6KWR3W2)
  topics:: 
  reading status:: [[to read]]
  citation graph:: {{[[query]]: {and: [[>]] [[citing]] [[marquardt2019what]] {not: [[query]]}}}}

Which in Roam turns into this:

Paper Metadata
Figure 1: Paper Metadata in Roam after Export from Zotero

What can you do with this?

First, I can look at the pages for individual authors and check out all the papers they have written or contributed to. I can also look at all the papers that appeared in "Research & Politics", and easily share a URL to the paper by simply copying from Roam without the need to open Zotero.

I can also directly go to the paper within Zotero to read the PDF, and by filling out the topics attribute I can have a look at all the papers that deal with expert surveys, for example.

Another cool capability is that the citation graph attribute and its query lets me collect all the places in other papers that refer to this one. All I need to do is indent a citing: attribute below a blockquote (created through [[>]]) that links to the title of the papers page (marquardt2019what) to see who referenced it in what context and who they reference. When you're exploring your literature, that's quite neat to have and impossible to get otherwise.

If you want the file that lets you do that, head over here and download it. To install it, you copy the file over to your Zotero > translators folder. Once you've done that, open up the Zotero preferences, head to the "Export" tab and select "Roam Metadata Export" in the "Default Format" dropdown. Now, all you need to do is hit Cmd-Shift-C when selecting a source in Zotero and it will copy the information formatted like above. Paste into Roam, and you're done! If anything is unclear, do watch the video – having a visual reference can be helpful.

A quick note: the translator requires you to have an author for every source, so make sure that information is there for each reference. It also works best if you use the BetterBibTex plugin for Zotero, which I highly recommend you install as well.