I wrote a feature for this week’s issue of the journal Nature on “petascale” data-centers — giant data-centers used in scholarship and science, from Google to the Large Hadron Collider to the Human Genome and Thousand Genome projects to the Internet Archive. The issue is on stands now and also available free online. Yesterday, I popped into Nature’s offices in London and recorded a special podcast on the subject, too. This was one of the coolest writing assignments I’ve ever been on, pure sysadmin porn. It was worth doing just to see the the giant, Vader-cube tape-robots at CERN.

At this scale, memory has costs. It costs money — 168 million Swiss francs (US$150 million) for data management at the new Large Hadron Collider (LHC) at CERN, the European particle-physics lab near Geneva. And it also has costs that are more physical. Every watt that you put into retrieving data and calculating with them comes out in heat, whether it be on a desktop or in a data centre; in the United States, the energy used by computers has more than doubled since 2000. Once you’re conducting petacalculations on petabytes, you’re into petaheat territory. Two floors of the Sanger data centre are devoted to cooling. The top one houses the current cooling system. The one below sits waiting for the day that the centre needs to double its cooling capacity. Both are sheathed in dramatic blue glass; the scientists call the building the Ice Cube.
Blank slate

The fallow cooling floor is matched in the compute centre below (these people all use ‘compute’ as an adjective). When Butcher was tasked with building the Sanger’s data farm he decided to implement a sort of crop rotation. A quarter of the data centre — 250 square metres — is empty, waiting for the day when the centre needs to upgrade to an entirely new generation of machines. When that day comes, Butcher and his team will set up in that empty space the yet-to-be-specified systems for power, cooling and the rest of it. Once the new centre is up, they’ll be able to shift operations from the obsolete old centre in sections, dismantling and rebuilding without a service interruption, leaving a new patch of the floor fallow — in anticipation of doing it all again in a distressingly short space of time.

The first rotation may come soon. Sequencing at the Sanger, and elsewhere, is getting faster at a dizzying pace — a pace made possible by the data storage facilities that are inflating to ever greater sizes. Take the human genome: the fact that there is now a reference genome sitting in digital storage brings a new generation of sequencing hardware into its own. The crib that the reference genome provides makes the task of adding together the tens of millions of short samples those machines produce a tractable one. It is what makes the 1000 Genomes Project, which the Sanger is undertaking in concert with the Beijing Genomics Institute in China and the US National Human Genome Research Institute, possible — and with it the project’s extraordinary aim of identifying every gene-variant present in at least 1% of Earth’s population.

Big data: Welcome to the petacentre,

Podcast about Petacentres,

My Flickr photos of petacenters

Roy Trumbull (who previously recorded a free podcast of my story The Super Man and the Bugout) has just recorded another podcast, this time of my story Craphound, my first-ever professional publication. It’s a nostalgic story about aliens who come to earth for our yardsales and the humans whom they befriend. It’s the third audio adaptation so far, and it sounds great. Roy’s a terrific reader.

Craphound had wicked yard-sale karma, for a rotten, filthy alien bastard. He was
too good at panning out the single grain of gold in a raging river of
uselessness for me not to like him — respect him, anyway. But then he found the
cowboy trunk. It was two months’ rent to me and nothing but some squirrelly
alien kitsch-fetish to Craphound.

So I did the unthinkable. I violated the Code. I got into a bidding war with a
buddy. Never let them tell you that women poison friendships: in my experience,
wounds from women-fights heal quickly; fights over garbage leave nothing behind
but scorched earth.

Craphound spotted the sign — his karma, plus the goggles in his exoskeleton,
gave him the advantage when we were doing 80 kmh on some stretch of back-highway
in cottage country. He was riding shotgun while I drove, and we had the radio on
to the CBC’s summer-Saturday programming: eight weekends with eight hours of old
radio dramas: “The Shadow,” “Quiet Please,” “Tom Mix,” “The Crypt-Keeper” with
Bela Lugosi. It was hour three, and Bogey was phoning in his performance on a
radio adaptation of _The African Queen_. I had the windows of the old truck
rolled down so that I could smoke without fouling Craphound’s breather. My arm
was hanging out the window, the radio was booming, and Craphound said “Turn
around! Turn around, now, Jerry, now, turn around!”

Craphound on Internet Archive, MP3 download

See also: Super Man and the Bugout reading: what if Superman had been a nice Jewish boy from Toronto

Tor.com has just published a new story of mine, “The Things that Make Me Weak and Strange Get Engineered Away” (the title is from “The Future Soon,” a Jonathan Coulton song), which is about geek monasteries that house smart people who can’t get along in the world and put them to work as coders. The story is the first Tor.com piece to be Creative Commons licensed and you’re encouraged to remix it, translate it, whatever. There’s already a podcast of me reading the story (also CC licensed) and PDF, Mobipocket and Sony reader files are already available.

Lawrence’s cubicle was just the right place to chew on a thorny logfile problem: decorated with the votive fetishes of his monastic order, a thousand calming, clarifying mandalas and saints devoted to helping him think clearly.

From the nearby cubicles, Lawrence heard the ritualized muttering of a thousand brothers and sisters in the Order of Reflective Analytics, a susurration of harmonized, concentrated thought. On his display, he watched an instrument widget track the decibel level over time, the graph overlaid on a 3D curve of normal activity over time and space. He noted that the level was a little high, the room a little more anxious than usual.

He clicked and tapped and thought some more, massaging the logfile to see if he could make it snap into focus and make sense, but it stubbornly refused to be sensible. The data tracked the custody chain of the bitstream the Order munged for the Securitat, and somewhere in there, a file had grown by 68 bytes, blowing its checksum and becoming An Anomaly.

Order lore was filled with Anomalies, loose threads in the fabric of reality—bugs to be squashed in the data-set that was the Order’s universe. Starting with the pre-Order sysadmin who’d tracked a $0.75 billing anomaly back to foreign spy-ring that was using his systems to hack his military, these morality tales were object lessons to the Order’s monks: pick at the seams and the world will unravel in useful and interesting ways.

The Things that Make Me Weak and Strange Get Engineered Away,

MP3 link

Roy Trumbull has performed an excellent reading of my short story The Super Man and the Bugout — a story about Superman as a Jewish boy raised in Toronto’s suburbs, put out of work by the arrival of benevolent aliens who welcome Earth to the Galactic Federation.

MP3 Link

Craphound, the first short story I ever published in a professional market, has been turned into a fine little audio reading by Literal Systems (using the Creative Commons license), read by Rosalia Triana.

Craphound had wicked yard-sale karma, for a rotten, filthy alien bastard. He was
too good at panning out the single grain of gold in a raging river of
uselessness for me not to like him — respect him, anyway. But then he found the
cowboy trunk. It was two months’ rent to me and nothing but some squirrelly
alien kitsch-fetish to Craphound.

So I did the unthinkable. I violated the Code. I got into a bidding war with a
buddy. Never let them tell you that women poison friendships: in my experience,
wounds from women-fights heal quickly; fights over garbage leave nothing behind
but scorched earth.

Craphound spotted the sign — his karma, plus the goggles in his exoskeleton,
gave him the advantage when we were doing 80 kmh on some stretch of back-highway
in cottage country. He was riding shotgun while I drove, and we had the radio on
to the CBC’s summer-Saturday programming: eight weekends with eight hours of old
radio dramas: “The Shadow,” “Quiet Please,” “Tom Mix,” “The Crypt-Keeper” with
Bela Lugosi. It was hour three, and Bogey was phoning in his performance on a
radio adaptation of _The African Queen_. I had the windows of the old truck
rolled down so that I could smoke without fouling Craphound’s breather. My arm
was hanging out the window, the radio was booming, and Craphound said “Turn
around! Turn around, now, Jerry, now, turn around!”

Link, MP3 Link

(Thanks, Bri!)