How to create a catalogue automatically using ONIX and InDesign

Did you go into publishing so that you could spend your days copying and pasting ever-changing metadata from spreadsheets, emails and databases into InDesign? You did? No need to read on.

If you didn’t, perhaps this article will interest you. In it I’m going to show you how to create a catalogue in a single click.

This isn’t a sales pitch. You don’t need to scan ahead for the paragraph where it says “and all you need to do is buy our MagicSoftware™ for twenty grand a year”. All the tools you’ll need are very likely to be there on your work computer. I’m assuming, you being a publisher, that you have a copy of InDesign, and that you can get an ONIX file of your data. (If you don’t have an ONIX file then, actually, this is a sales pitch. Producing an ONIX file of your book data is a basic publishing necessity: call me.)

Here’s what we’re going to cover in this article. First, I’ll prepare an InDesign template. Then I’ll get the data arranged into a suitable format for the InDesign template to understand. Finally, I’ll import the data into the template and revel in the catalogue I’ve created in a single click.

The most exciting bit about this is that you only have to do things once. You set up the InDesign template once, with one sample book entry. You write the XML transformation code once. Then, when you import the data, the single-book template replicates itself to create as many catalogue entries as there are book records in your data — potentially thousands and thousands of catalogue entries with a single click. And of course you can reuse your template and code for the next season’s catalogue, and the next.

Before we start it’s probably a good idea to read this article about XML and ONIX.

ONIX is a wonderful way of sending rich book data to the companies who need it – Nielsen, Bowker and the like. But ONIX’s comprehensive nature means that it can be quite verbose. In a catalogue, we don’t need every last bit of information about a book. So I’m going to convert the ONIX XML message into another XML file that’s a bit less wordy, using a language called XSLT. Then I’ll be able to import it into InDesign. (In fact, InDesign can handle the conversion process itself, when I import the XML.)

Download the files here. You’ll see I’ve prepared an InDesign template. Here’s the structure pane of the InDesign template, showing how each entry will be structured: I’m going to need to whittle my ONIX down until I’ve got an XML file that matches this structure.

So I’ll prepare my data for it, using an XSL transformation. Open the transform.xsl file in a text editor such as Atom: this is what the transform.xsl file contains.

Keep calm! Whilst this is code, and it might look a bit overwhelming at first glance, it’s not all zeroes and ones. It’s readable, because it uses English words.

So, one line at a time, I’ll read this code. (This, by the way, is the very first code I ever wrote, in 2006.)

Line 1 tells the computer that this is an XML file.

Line 2 adds the first tag, called

1
  <BK>

and gives a bit more information about this part of the file – which is that this code is XSL.

Line 3 is where it starts to get interesting.

1
  <xsl:for-each select="ONIXMessage/Product">

means “find each mention of ONIXMessage/Product in the ONIX file.”

The next bit adds a new XML tag, called

1
  <Book>

If you peek ahead to line 24, you’ll see its sister, the closing tag

1
  </Book>

(We know it’s a closing tag because of the backslash.)

Lines 5 to 7 set up some variables. A variable is like having a shortcut. For example, I can define a variable called “address”:

1
  address = “Chiltern House, Thame Road, Haddenham, Aylesbury, Buckinghamshire HP17 8BY, UK”

Now, next time I want to give my office location, I don’t have to spell out the whole thing. The variable “address” is a shorthand way of saying the whole thing.

Back to my code. I’m going to define three variables on the next lines: one called form, one called date and the other called isbn, so I can use them later. I’m getting the values of the variables from the ONIX file. I select the data tagged up with ProductFormDetail and assign it to the variable called form.

Similarly, I find the value in the ONIX tagged up as PublicationDate, and save that as the variable date.

Lastly I find the bit in the ONIX tagged up as ProductIdentifier[ProductIDType='03']/IDValue and set that as the variable called isbn. That last one is a bit complicated but if I show you the ONIX, it’ll make more sense.

1
2
3
4
5
6
7
8
    <ProductIdentifier>
      <ProductIDType>03</ProductIDType>
      <IDValue>9780954575984</IDValue>
    </ProductIdentifier>
    <ProductIdentifier>
      <ProductIDType>02</ProductIDType>
      <IDValue>0954575984</IDValue>
    </ProductIdentifier>

There’s two ProductIdentifier tags here. Which IDValue do I want to select? The one where the ProductIDType is 03. (A lot of learning programming is about getting familiar with the syntax – the particular way that the programming language is laid out. Here, I’m using square brackets to nest ProductIDType after ProductIdentifier. Don’t fret about learning syntax. Understanding the actual logic is the thing to really get a handle on. It’s why kids’ learn-to-code tools such as Scratch are so great – they teach the principles and the logic, not one language’s particular way of writing them.)

Line 8 defines another XML tag, called <image>. Because it’s an image tag, it takes an href attribute. And can you see how I say where the computer should look for the right file? That’s right – I use my freshly-minted “isbn” variable. So this URL:

1
  <image href="file:///{$isbn}.jpg" />

will become:

1
  <image href="file:///9780954575984.jpg" />

The next line is quite fruity, but I’ve included it so I can show how malleable data can be in the hands of a computer. The line defines a new XML tag, <year>. I’m using a new XLS command:

1
  <xsl:value-of>

Line 9 says “Get the value of the ‘date’ variable”. It also fiddles around with the date quite a lot. If the line read

1
  <xsl:value-of select="$date"/>

then the new <year> XML tag would read <year>20120228</year>. But who wants to use ISO-format dates in a catalogue? Instead I want it to read <year>28/02/2012</year>. So whilst concat(substring($dt,7,2), ‘/’,substring($dt,5,2), ‘/’,substring($dt,1,4)) looks fairly unholy at a glance, what it’s doing is saying

“Concatenate the following:
Get the value of the date variable which is 20120228.
Count to the 7th digit, which is 2.
Then get the 2 following digits, which would be 28.
Then put a backslash in.
Next, use that same date variable and get the fifth number, which is 2, and then the next two digits, which gives you 02.
Put another backslash in.
Finally, get that same date variable, and get the first number, which is 2, and then the next four digits, which gives you 2012.
All of that gives me 28/02/2012.”

See, it looks awful to start with, but you have to sit and read the code until it makes sense.

The next line, line 10, is also an <xsl:value-of>, and I reuse my handy isbn variable again (which is one of the points of using a variable). This line produces another XML tag called <productid>.

All these tags will be used in the InDesign document I’ll show you in a bit.

I’m going to use one more XSL command, which is

1
  <xsl:choose>

On line 11, I add another XML tag, called <format>. But what to put in this tag? The xsl:choose syntax means “Choose from the following: when the ONIX form variable is B104, put in A Format Paperback. When the ONIX form variable is B105, put “B Format Paperback” in the new <format> tag.” And so on.

The next bit is best shown on video. Watch how I’m going to use XSLT code to transform ONIX into a more concise XML file and automatically flow it into an InDesign template. (You’d best watch it on HD so you can see the detail.)

Use the files you’ve downloaded to practice, and then use your own company’s ONIX file and have a go. Then, tweak the InDesign template so that the colours and fonts are to your taste, and produce your own catalogue.

This demo is a deliberately pared-down example. You’ll likely want to include more information such as the title, the blurb and the price. I can recommend A Designer’s Guide to InDesign and XML even though it’s out of print which contains even more templates and excellent descriptions of how to start using InDesign properly and productively.

You know, I could have automated this catalogue’s production in a number of ways. If you use Consonance, you can export pared-down XML without having to convert ONIX, or you can export data directly to PDF. If you still store your metadata in Excel (shudder) you can nevertheless get it to export an XML file. The point is that if you explore the tools you have available to you right now on your machine, you’ll find that your computer could be working a lot harder for you. Automation is the key to a happy, cost-effective modern working life – and when your happiness and financial stability is at stake, that’s worth a bit of an investment of time in learning about automation processes like this, wouldn’t you say.

Originally published on Bookmachine, February 2015. This version includes additional edits and changing from the second to first person, which reads a bit letter.

Are your current systems sabotaging your growth ambitions? Are you hungry to implement new business models, but concerned you lack the strong administrative foundations needed for innovation?

We're always amazed at how resigned publishers have had to become to the low bar in publishing management systems. Demand more.