Project Looking Glass

Regular readers will know that Newspaper Club is split between two offices: London and Glasgow. Glasgow is now the HQ, where we do all the operations, logistics, customer service, and much more. And London is where we develop the products and services, writing code, drawing pictures, and so on.

We have a Campfire chat, which we natter in all day long, but for a while I’ve been trying to find other ways of joining the offices together that are less direct and a bit more, well, silly.

Our first go at this is the awfully titled ‘Project Looking Glass’. It’s a two way video screen that sits in the corner of each office, letting us wave at the other office, and them to wave back. It looks a bit like this:

Looking Glass #2

Looking Glass #3

It’s always on, it doesn’t need to dial up or sign in, and if the network drops, it should recover as soon as it can.

It’s built on a pair of Raspberry Pi’s, and you should be able to put one together yourself for under £150, if you can find a couple of spare monitors. I’ve written up much more about it, including how to make one, on my blog.

We’ve only had it running for a couple of days, so it’s a bit of an experiment for us. So far it seems quite fun, with some good drawings appearing on whiteboards. But that might fade, and it might end up being a bit weird, in which case we’ll turn it off and try something else.

But we also know that we’re going to have to get better at remote working, as we grow and as our business get more complicated. It feels like we can learn a lot from trying things like this out while we’re small enough, and those things will be useful as more people join us, in our offices and out.

We’ll keep you posted on how it’s working out for us, and if you end up making something similar for your offices, we’d love to hear how it’s going.

Morning Message from Glasgow

Posted by Tom | Comments Off on Project Looking Glass

Filed under: engineering, running a business

ARTHR updates: No need to sign up, import pictures from Flickr and Instagram, and more!

ARTHR machine

The Engineering Dept have been hard at work on ARTHR (pictured above), and we’ve made a few improvements we’d like to tell you about.

First of all, you no longer need to make an account to get going with ARTHR. Just head over to the site, start making a newspaper, and when you’re ready, sign up and we’ll save your paper into your new account.

We encourage you to make an account as soon as possible after you’ve decided whether ARTHR is for you: not because we’re greedy for your details, but in case you move computers or change browser, and we can’t match you up with your newspaper any more – we don’t want you to lose your paper!

There’s also a lovely new homepage for ARTHR, so tell all your friends.

ARTHR Wizard Screenshot

But that’s not all! We’ve also:

  • Revamped the controls at the top of the screen, based on your feedback, making it clearer how to add and remove pages from your newspaper.
  • Added a feature to be able to add an image to your newspaper directly from a URL. You can even give us a Flickr or Instagram photo page and we’ll do the hard work to fetch the image out of it.
  • Made adding a new piece of content smarter, so it doesn’t overlap anything already on the page, picking the next suitable space in your newspaper.

And finally, a little reminder that the best paper in the Newsagent each month wins a £100 voucher. The Engineering Dept will even throw in a Twix if it’s made in ARTHR.

Posted by Tom | Comments Off on ARTHR updates: No need to sign up, import pictures from Flickr and Instagram, and more!

Filed under: engineering

This is my kettle

My Kettle

This is my kettle. There are many like it, but this one is mine. Kettles form a core part of Newspaper Club operations, and it’s important to pick one that has a number of key features to ensure a quality cup of tea. Below I will detail the factors that contribute to this kettle’s daily success, from a systems engineering point of view.


Uptime is important with critical pieces of infrastructure. Things that move are things that break. This kettle has few moving parts – no electric heating element, or on/off switch. It’s heated by applying energy (ignited gas) to the base and waiting. This gives me the confidence that I can trust it in daily operations for many decades.

Graceful Degradation

However, if it was to fail, (say the handle fell off), it’ll still be partially usable (with an oven glove) until a repair can be made at a convenient time.

Control Surfaces

It’s not the fastest kettle, but by varying the gas throughput I can vary the time it takes to boil. If I dial it back a bit, it takes just a little less time to boil than my porridge takes to cook, so the tea is brewed just as the porridge is done.


Given a typical UK energy mix, heating things with gas is more efficient than heating things with electricity. However, as more of the UK energy mix (hopefully) shifts to renewable sources over the next few decades, gas will have a relatively larger carbon impact.


It’s a nice red.

Posted by Tom | Comments (3)

Filed under: engineering

ARTHR is dead, long live ARTHR!

Today we’re very pleased to announce the full release of the new and improved version of our online newspaper making tool, ARTHR. It’s been in testing for a while, but it’s now the default tool if you go into your dashboard and press “Make a Newspaper“. Hurrah!

As part of this, we’re shutting down the old version of ARTHR, now known as ARTHR Classic. This post details how the shutdown will work, and if you’re still using it, important dates to finish your work by.

As of today you can still make a paper in ARTHR Classic, but it’s not recommended unless you have a good reason. If you do, we’d like to hear about it, so we can see how the new version of ARTHR might meet your needs.

Barring any problems, on 18th March, we’ll stop people being able to make new newspapers in ARTHR Classic. If you’ve already started one before then, it’ll still be editable.

Then, on 1st April, we’ll stop people being able to edit newspapers in ARTHR Classic, and the tool will disappear from the site. On that date your newspaper will be converted to a PDF on our site, and you’ll be able to download the contents as a PDF, InDesign document or a ZIP file containing the stories and pictures.

If you’re working on a paper in ARTHR Classic you’ll need to finish and order it before then. Unfortunately it’s not possible to automatically move your paper from ARTHR Classic to the new version of ARTHR as the structure of the document is significantly different.

If any of that causes you a problem, please let us know and we’ll do our best to help out.

Posted by Tom | Comments Off on ARTHR is dead, long live ARTHR!

Filed under: developments, engineering, news and press

How ARTHR & ERNIE work: Backbone.js, Rails, Cocoa, and more.

We thought it might be interesting to take you, the dear reader, on a little tour through ARTHR II and how it all works. This post is unashamedly technical, so if all gets a little bit too much, don’t worry: like the Underground at rush hour, there will be another post along shortly.

I will start at the beginning, for the beginning is a good place to start. ARTHR II (henceforth known as ARTHR – the Automated Real-time Text Handling Resource), is our online newspaper layout tool. It provides a drag-and-drop WYSIWYG style interface to make a great looking newspaper quickly and easily. And it’s entirely browser based, with no plugins required, and nothing to download.

It looks a bit like this:

(I know I’ve used this video twice before, but dammit, I’m going to use it again. If you don’t have the time to poke around with ARTHR, it should give you an idea of what you can do with it. If you do have time to have a go, head over here and try it out!)


There are three main components, all of which integrate tightly together.


  • The frontend Javascript application: the entire user interface, some layout intelligence, and more.
  • The backend Ruby on Rails application: handles storage, persistence, integration with the main Newspaper Club site, background jobs, etc.
  • ERNIE, a Mac OS X/Cocoa rendering server: takes a document layout and converts it into JPG previews or print-ready PDFs

The Frontend

Much of the logic and intelligence in ARTHR runs in the browser, putting it as close to the user as possible to make it as fast as possible.

The frontend is mostly a Backbone.js application, written in classical, Crockford-compliant, Javascript. We’ve been very happy with Backbone – it provides just enough structure to support building things quickly, but without forcing you to go ‘all in’ to their way of doing things. There’s plenty of room to pick your own templating language, or drop down to your own AJAX requests, without feeling like you’re going against the grain. It cleanly separates the presentation layer (HTML/CSS) from the logic, leading to good quality, accessible HTML. And last, but most certainly not least, there’s no magic going on: the entire source code is imminently readable, and we’ve regularly walked through it, by eye or in the debugger, to check we understand the behaviour of something.

We’ve tried to write it as you would a well-structured Object Orientated GUI application, and took much from Soundcloud’s blog post about building Next Soundcloud.

With so much of the app’s complexity running in the user’s browser, it becomes much harder to track down tricky bugs. To help with this, we wrote our own logging engine, with a server-side component. Browser exceptions are automatically sent over, along with a stracktrace if available, and contextual information like the git revision in use, and the IP address. For persistently tricky browser/OS combinations, we can flip a user’s account into remote debug mode, sending us live log statements, effectively letting us tail a user’s actions and get good diagnostic information.

The Backend

The backend of ARTHR is quite a slim component, mostly handling persistence, storage of uploaded images, validation checks, and providing a conduit for the frontend to communicate with ERNIE, the rendering engine.

It’s a Rails 3.2 application, running a Postgres database, with Sidekiq for background jobs. We’re making good use of the hstore type to be able to store arbitrary key/value pairs for different content types.

All that sits in front of nginx, running the brilliant nginx-push-stream-module. This lets us handle long-running EventSource/Server-Sent Event connections while still using our standard Unicorn single-threaded Rails server. We just make an HTTP POST request containing the message payload to a specific channel, and it arrives at any clients listening on that channel shortly after. It’s simple, it just works, and keeps the number of server components to a minimum.


When we developed the first version of ARTHR, a few years ago, we used InDesign Server to provide layout, high quality typography with PDF and JPG output. It served us fairly well, but it’s difficult to integrate tightly with, and it’s often very slow. We needed to speed it up by a couple of orders of magnitude, and have much finer information about layout and text flows. It simply wasn’t possible to do that with IDS.

We needed something to let us layout a document, and quickly convert to both JPG (for previews) and PDF. To make things a little more complex, it needs to support CMYK, and integration with a high-quality typography library for things like ligatures, hyphenation and justification.

After working through most of the options, we ended up at Quartz, part of Cocoa’s CoreGraphics stack in OS X. It’s fast – really fast – because it’s designed to run at 60fps on the desktop. Because Quartz maps very closely to the PDF format rendering to a PDF context is fast too. And it natively supports CMYK throughout, something that Skia and Cairo, two of the main open-source rasterisation engines don’t. For typography there’s the Core Text Layout System, a collection of classes for handling typography and typesetting.

Using all this we built ERNIE, the Expertly Rendered Newspaper Internet Engine. It’s an OS X 10.8 application which exposes an HTTP RPC interface, receiving requests for content size calculations, or document rendering, and returning arrays of JPGs or PDF files.

ERNIE logs

We’re really pleased with the performance. It typically takes ~50ms to render a simple, text heavy document, up to ~700ms for a longer, more graphics heavy document. It’s often 100-200x faster than comparable documents in InDesign Server.

The PDFs Quartz produces have a few oddities, so we post-process them in Ghostscript to improve compatibility with our presses and ensure they’re PDF/X-3:2002 compatible.

Putting it all together

Let’s walk through adding a new bit of text to your document, as an illustration of how this all pieces together.

The user types into the text box and hits save. The frontend immediately sends the new content to the backend for storage. During that request, the backend takes the new content and performs a test rendering against ERNIE, calculating how the text will flow, where the paragraph breaks are, the number of lines per paragraph, etc.

When the request returns, the frontend uses the calculated flow to perform a prediction of where the text will display in the document, and renders the new piece of text in the blueprint. It’s then ready to be moved around the document by the user.

The preview images are now out of date, as they don’t include the piece of text we just added, so the frontend fades them out, and requests a rebuild of them. The backend returns a unique ID for the rebuild request, which the frontend uses to listen for changes in the state of the rebuild, using an EventSource request connected to the nginx-push-stream-module mentioned earlier.

The rebuild request is picked up by one of the background workers which takes the new document structure, including the new text, and passes it through to ERNIE. ERNIE processes the document layout and returns an array of JPG files. These are written to disk by the background worker, and the build is marked as completed. The frontend, listening for changes to the build status, is immediately notified, and requests those JPG previews from the server. As they load, it fades them in and they become visible to the user.

For most users this all happens in less than half a second, end to end. And if the user moves the piece of text around, it all happens again. Of course, the user doesn’t have to wait for the preview images before continuing to edit their document – it’ll keep them up to date as they go.

The End

Hopefully that’s given you an idea of how an application like ARTHR works, all the way through the stack. We’re very happy with the technologies we’ve chosen, and we’re very grateful to everyone who has contributed to all the open source projects we’re building upon.

We’re keen to chat to folks working on similar online publishing tools, and always open to sharing what we’ve learnt. If that’s you: get in touch!

Posted by Tom | Comments (3)

Filed under: engineering

ARTHR II: Print Harder

Things are proceeding apace in the twin Newspaper Club offices of London and Glasgow. It’s shaping up to be a busy year already, with a sortie of new products and services being planned and built as I type, right this minute.

We released some improvements to our Dashboard pages yesterday, including a sneaky little link to try out the new version of ARTHR. Go on, click it.

ARTHR screenshot

ARTHR II (internal codename: Print Harder) will soon be our main tool, and we’ll be turning off ARTHR I shortly after. Now’s your opportunity to get ahead of the game and let us know what you think.

New features include:

  • Drag things around with your mouse!
  • Style your text with all sorts of headers and formatting and stuff like that!
  • Choose from a number of typefaces and sizes!
  • Superfast performance using our proprietary Expertly Rendered Newspaper Internet Engine (ERNIE)!
  • Automatically pull a webpage into your paper!
  • And much, much, more!

If you’re making a paper in the old version of ARTHR, don’t worry – we’ll give you plenty of notice before turning it off, and all your work will be available to download in a number of handy formats.

Everyone else: go go go!

Posted by Tom | Comments (1)

Filed under: engineering

Engineering Notes

Engineering Dept.

Phew, it’s been a busy few months in the Engineering Dept., but the last fortnight has been especially full on.

Last week we released the new version of ARTHR to a few customers who had used the old one, to see what they thought of it.

We received lots of very useful feedback and discovered lots of bugs too, both little and large. We’re ploughing through all of that at the moment, releasing tweaks and fixes a few times a day. Thank you everyone who helped out.

The first 90% of the project is always great: you’re breaking new ground, inventing something new, and everything is exciting and good.

The second 90% of the project gets a bit trickier: now you have to make difficult compromises about how to prioritise your time, and the realities of putting your creation into the hands of other people hit home.

We’re well into the third and final 90%. Tracking down frustrating browser bugs, trying to fix the problem with bad connections from corporate networks, wrangling the produced PDFs to be prepress ready.

The good news is that every couple of days I surface from wrestling an especially gnarly PDF problem, or tweaking the server configuration, and I go and have a proper play with it. And I think it’s still good.

For a little flavour here’s a spin through that I just recorded. (Play spot the bug!)

Posted by Tom | Comments (2)

Filed under: engineering

Week 160

The date of the first blog post (well, not the very first) tells me this is Week 160 of Newspaper Club, or day 1,120.

We’ve not been as talkative as we used to be about running the business. That’s mostly because there’s a lot more business than there used to be – both customers, and all the stuff to support that: profit and loss reports, management accounts, Skype calls, meetings, planning.

But that’s no excuse. We’re into the really difficult stage of running a business. We’re still small, but there’s real money, real jobs and real pressures. But, this is also where the meaty stuff is. We’ve learnt a lot, and we’re learning even more. Those thing seem worth sharing, and if not for you, then for our own sake. We’re going to try harder.

So, what’s going on?

For a start, we’ve had two brilliant new people join us. Rosemary is up in Glasgow working full-time with Anne, Emily and Silje to manage all operations, customer service, and well… everything. This is brilliant, because a) Rosemary is brilliant, and b) Anne can go on holiday now.

And Mike is down in London, working with me, to develop the site and future products and services. This is brilliant, because a) Mike is brilliant, and b) I can go on holiday now. He’s sat to the right of me, looking very serious, staring at the Backbone.js documentation.

We’ve been joined part-time over the summer by Jase, who is helping us with some design work. He’s sat opposite me, doing some typing wearing a great pair of red headphones.


Behind me is a whiteboard. It says: “ARTHR II: PRINT HARDER”, with lots of post-it notes underneath.


I’m really worried about talking about stuff before it’s real. Talk is easy, and shipping is hard. But dammit, I’m really excited about this. So, if you promise not to hold us to it, we’re completely overhauling ARTHR, using everything we’ve learnt about how people have used it over the last couple of years. When there’s something to show, you’ll see it here first.

Anyway, I need to stop typing this, and go and read some blog posts about how to test asynchronous Javascript with QUnit. Onwards.

Posted by Tom | Comments (5)

Filed under: engineering, running a business

For Immediate Release: The Next Generation of PDF Uploading Solutions

MD and Customer Service have gone for lunch, so it’s time for Engineering to get on the blog and crank out a badly worded and ill-informed press release full of crimes against verbs. Hold onto your browser.

Newspaper Club (London, United Kingdom & Glasgow, Glorious Independent Republic of Scotland), are pleased to announce the release of the next generation of PDF uploading.

This significant advancement will make it even easier to leverage the Newspaper Club printing platform, and seamlessly synergize your credit card with our state of the art Web-Site.

From today, all PDF files uploaded to Newspaper Club will be scanned by the unique PDF management solution before ordering. Valued customers will immediately receive a report detailing common issues, with helpful hints on how to resolve them going forward.

Common issues, such as fonts not embedded correctly, or the use of spot colours, will be highlighted in a simple report, indicating the page or the area in which they occur, allowing the customer to take action to resolve them, or to proceed regardless.

Anne Ward, Managing Director at Newspaper Club, noted, “Newspaper Club is a world leader in the uploading and ordering of PDFs for newspaper printing purposes, and this advance will cement our position at the top of the table.”

The 1700 page PDF file specification has almost killed me, so this better be worth it”, mumbled Tom Taylor, Head of Engineering. “But I ask customers with any feedback about this valuable service to send it in, allowing us to action their comments.”

Newspaper Club’s commitment to and sponsorship of the innovative, open source, Ruby based, pdf-preflight gem will enable others to leverage their effort and contribute onward improvements.

About Newspaper Club:

Newspaper Club helps anyone make and print their own newspapers. Since 2009, Newspaper Club has printed more than a million newspapers, and in 2010, was the recipient of a Design Museum Designs of the Year award. Learn more at

Posted by Tom | Comments Off on For Immediate Release: The Next Generation of PDF Uploading Solutions

Filed under: engineering

New! Embed Your Newspaper on Your Site

We’ve just added a little feature to the Newsagent. You can now embed a newspaper on your site, just by copying and pasting the HTML code provided on each paper’s page.

For example, here’s the paper that started it all:

And seventhirtyeight, which Anne wrote about earlier today:

There’s a nice little scroller, and you can click through to the paper on the Newspaper Club site. The medium size is designed to fit neatly into a typical blog post, and there’s a slightly larger option too.

By default anyone can embed a paper, but if you want to turn it off, just untick the option in the sharing settings.

It turns out writing the code to do this is a bit tricker than we thought, so if you notice the design looking a bit wonky on your site, please let us know.

Posted by Tom | Comments Off on New! Embed Your Newspaper on Your Site

Filed under: developments, engineering

← Newer Posts | Older Posts →

These are the posts from the Newspaper Club Blog filed under engineering.

We’re here to help everyone make and print their own newspapers.

Thanks for stopping by the Newspaper Club blog. Here you can read the stories behind some of the best papers we’ve printed—and meet the happy customers who made them. We’ll also post occasional updates about what’s going on behind the scenes and inside the presses.

Ready to print your own newspaper? Head to How It Works to get started or get in touch. We’d love to hear from you.


Subscribe via RSS