Pinboard: transforming communication across the newsroom (part 1 of 3)

Pinboard is a discussion and asset sharing tool (or rather tool within other tools) which is gradually transforming how the news room communicate as news stories move through the various phases of the production process

Tom Richards, Ana Pradas

Published on Thursday, 8 February 2024

Pinboard  


This is part one of three (part two and part three).

Origin Story

For the past few years, a lot of digital communication in the newsroom had been happening over email and chat – crucially outside our editorial tools (tools being the place where the content people are communicating about is actually being produced) which creates the potential for confusion and mistakes given the pace of the news agenda. Meanwhile, there was a general feeling that pieces would benefit from preparing assets (images and video) earlier in the production process.

David Blishen (the group product manager of the Editorial Tools team at the time) set the team a very broad yet succinct brief which was titled ‘content association’. Myself and Jenny (another developer in the team at the time) set about exploring the problem and prototyping some ideas, with the loose goal of being able to communicate better within our own tooling, with a focus on sharing assets (ideally earlier in the production process). We soon had our wonderful product designer, Ana Pradas, join our exploration.

We approached the challenge by starting with a discovery phase. We wanted to understand:

So that we could learn what their main obstacles were and what the core problem we needed to solve was, we shadowed and talked to people across London, US and Sydney offices – working in different roles in editorial, from commissioning editor, to imaging, to picture editor, liveblogger, subeditor and designer.

From discovery all the way to beta testing we found that the biggest problems we needed to solve for users were:

Some of our main UX and design challenges were:

The Technology

Let’s start with language, it’s all TypeScript, for client, server and infrastructure. We have shared constants and logic between all three and as such the Developer Experience has been great. TypeScript’s type system is pretty awesome these days, you can do lots of functional stuff and it’s fairly light.

We define infrastructure with CDK, a good chunk of which is GuCDK, but some of the earlier parts of Pinboard we have not yet migrated, since GuCDK was just getting started when Pinboard was taking shape.

We build with esbuild, which is super fast. This is great locally, but combined with some parallelisation in our CI we have kept our build times down below a couple of minutes despite it being quite a big project these days.

We use AWS Lambda for everything server-side, eight lambdas at the last count. So aside from the database it’s all serverless, which made scaling straight forward and splitting things up has provided easy separation of concerns compared with a big monolith. Crucially, too, lambdas are very quick to deploy, and so if we have no infrastructure changes in a release, it typically takes about two seconds – great for rolling back, fixing quickly and combined with fairly quick builds makes for a fairly snappy feedback loop.

We use AWS RDS (Postgres flavour) for our own data (eg. what messages have been sent). Prior to that, we used DynamoDB for a good while until we needed a more relational structure.

We use AppSync (AWS’ managed GraphQL) for abstracting over the data we get from various sources; our own from our database, but also pulling in from other systems, such as workflow, the grid etc. and crucially for the real-time aspects of pinboard (but I’ll dig into that a bit later). We use Apollo on the client for connecting to AppSync.

Then on the client we have a fairly typical Guardian setup; React (actually, Preact in compatibility mode to keep the bundle size down) with Emotion for styling and bundled using Webpack (although we used Parcel for a good while).

Take a look at the architecture diagram to see how all that comes together:

Pinboard Architecture Diagram as of 5 Oct 2023. It shows how all the key parts of pinboard fit together, notably how the host application authenticates and loads from the bootstrapping-lambda, how the client then requests things from AWS AppSync, and how these are routed to various other lambdas, including the database-bridge-lambda.
Pinboard Architecture Diagram as of 5 Oct 2023 (see https://github.com/guardian/pinboard#architecture-diagram for latest) Illustration: Tom Richards

This is part one of three (part two and part three).

Credits

Pinboard was built by Tom Richards, Jenny Graham-Jones, Thalia Silver, Andrew Nowak, Phillip Barron & Ara Cho with additional development contributions from Fred O’Brien & Samantha Gottlieb. Product Design from Ana Pradas and Product direction from Calvin Dickson. All the while supported by the rest of the content production team in the Product & Engineering department. Not forgetting the input/time/effort from countless Guardian journalists, who have helped shape Pinboard.

Continue reading

The Digital Fellowship is your foot in the door to the future of news Pinboard: transforming communication across the newsroom (part 2 of 3)