Kindle preparation part 5 – formatting and media queries

March 21, 2017 RichardAbbott Leave a comment

Last time in this series we looked at some of the difficulties of presenting a book in an electronic medium, where the layout on every page can change according to user settings and from one device to another. In particular, there is the problem of handling the two extremes – small or large font size compared with screen width. At one end of the spectrum you get a lot of words per line, and the tendency for the eye to lose its place in a screen full of text. At the other, with few words per line, word spacing becomes irregular and “rivers” of white space tend to open up.

On grounds of aesthetic appearance and readability studies, there is broad agreement that left-aligned text is preferable in the limit of few words per line. For many words per line, text which is flush both sides is more familiar and traditional, giving an impression of neatness. The question then arises, is it possible to have the best of both worlds?

Now with traditional print layout, all of this is decided by author or publisher, and the reader has no choice. Once the design is chosen, that’s it. The phenomenal rise of epublishing, with free and easy to use tools enabling indie publishing, has given huge empowerment to authors. But I sometimes think that authors have not caught on to the fact that it has also given huge empowerment to readers, who want to exercise their freedom of choice to change the layout of the book to suit their own preferences.

So what does that mean for Kindle formatting? A Kindle or ebook is basically a long thin web site, conveniently represented in pages by software or hardware. The basic building blocks are HTML files and CSS style sheets, together with some added contextual information to tie the whole lot together. This is hidden from many authors, who may simply upload a Word document to an aggregator site which itself does the difficult work.

New model Kindle with automatic left alignment

But being essentially a web page gives access to another set of options. Ebook devices like Kindle do not usually support everything that a real web site would – for example you cannot use script commands to query the settings in a dynamic way. But there is support for something called a media query, embedded inside the style settings. Media queries are often used to render a page suitable for printing, or for voice readers, or to accommodate a wide range of screen sizes from mobile to wall-mounted TV with the same basic design. So Kindle books can be responsive, but not dynamic in the strict technical sense.

Regular media queries are of limited use here – in terms of pixels or centimetres, the screen is what it is. Happily, there is a fairly straightforward solution. A web device – including an ereader – allows widths to be specified in a unit called em. An em is directly related to the font size, unlike physically derived units like pixels or centimetres. So while the size of the screen stays the same in terms of pixels, it changes in terms of ems as different fonts are selected. Better still, the em width also varies as the user chooses different margin widths. For a standard device font at default size, by convention 1em = 16px. But this can be changed in various ways, including user selection of font size and page margin.

So here is the way to have an ebook layout which is responsive to user choices. It’s not as flexible as what you can do in a real web site, but then the device doesn’t allow you to make so many adjustments. Anyway, it’s a whole lot better than having a one-size-fits-all compromise. Basically you can have a different set of style rules for large fonts than for small ones, and so maintain appearance across a very wide range of settings. Left alignment is easier for the eye to follow with only a few words per line – and avoids the tendency of justified text to leave large rivers of white space in the middle of pages. Justified text is preferred where there are lots of words per line. Newer versions of the Android and iPhone apps make this choice for themselves unless told otherwise, but most actual Kindles do not. With media queries you don’t have to pick one or the other and make do – you can have both, as part of a responsive Kindle design. You can have as many multiple separate queries as you wish, though personally for simplicity I would tend to stick with two – one for large fonts (compared to screen width) and one for small. Similar comments apply to paragraph settings – regular spacing with indent, or a small vertical gap with no indent.

What are the rules to follow here? Well, firstly you should always have a default set of styles which apply in the absence of more specific choices. And the default ones should always go first, and those governed by a media query afterwards. That’s because of how the style sheet is parsed – the whole file from start to finish, and any later directives which happen to apply are chosen in preference to earlier ones.

Finally, do keep it simple. Media query support, along with style sheet support in general, is patchy on most ereaders. The number of legacy and old-model systems is high, for several reasons. People hang on to their Kindles for a long time, so long as they continue to function. Many software companies producing phone and tablet reader apps don’t bother to code for recent enhancements, reckoning that the extra investment in time is not worth it. So any media queries used in a Kindle or generic epub book must be simple. It would be great to have different styles for whether your reader has chosen normal screen (dark text on pale background) or inverted (pale on dark), or indeed the several colour options available on some devices. But support for the media queries “inverted-colors” and “color-depth” is very erratic and cannot be relied upon. So you should not specify colours in your style which might end up unreadable for the colour scheme chosen by your readers. Better to avoid colours altogether and just let the device choose.

As I have said several times, not all Kindle devices, or Kindle software apps on computers, phones and tablets, treat the content the same way. My phone Kindle app (both Android and iPhone) handles changes of font size differently from my various actual Kindles, including the way it decides to justify text. This is something built into the app itself, not a thing I have direct control over. I like some reading preferences that other people don’t, so anything that you as author do by way of styles and media queries should not intrude on personal preference.

I started with some screenshots of how things would be if you had to make do with just one style, and moreover used measurements in fixed units. Here by contrast is the same content, using media queries and a responsive design. Personally I like the flexibility, and the way the presentation adapts to changes in user choices. Not everybody will, and maybe not everybody will want to dig in to the details of how their Kindle or epub book is being constructed. Those who simply hand over a Word document to Smashwords or a similar site may be perplexed by all this.

Authors spend a great deal of time and effort researching the background to their books. They look out for what they consider a good cover. They may pay for the services of an editor. Yet many authors dislike the thought of engaging with the technology that finally delivers their book into the reader’s hands. By way of doing something, many just try to copy the methods they used for print. But ebooks are a different medium to print, and need their own treatment. Happily, it is relatively easy to offer a better reading experience for those who want it. Complete consistency across devices is not possible, but through media queries and the use of ems to measure dimensions we can get a good way towards that.

This is almost the end of this little series, and the last item will be a quick summary of key points.

Extract, Fantasy, Half Sick of Shadows

Half Sick of Shadows – some extracts and the cover

March 16, 2017 RichardAbbott Leave a comment

Draft Cover - Half Sick of Shadows — Draft Cover – Half Sick of Shadows

I thought for today that I would put together several snippets from Half Sick of Shadows, together with a good draft of the planned cover. Both picture and writing may change a little over the next couple of months, but they’re pretty close now to final version. All being well, I am hoping to release the book around or just after Easter this year.

Half Sick of Shadows is a fantasy, of novella length rather than full-length novel, and owes a great deal to Tennyson’s poem, The Lady of Shalott, which itself is based on 13th century material concerning Elaine of Astolat. However, I have taken the plot in what I suspect is an entirely different direction than the earlier authors had in mind.

======================

First extract – from quite near the beginning

From that point on the lady sang each day, whether the people were in view or not. It made her self-conscious at first, and she felt riddled with doubt about the quality of her voice. But then she reasoned that since neither could hear the other, it could hardly matter. The main thing was to join herself in spirit with them. One day in the future, when she finally met them and learned how they spoke, she would concern herself with matching their tone.
…
But all unknown to her, her voice slipped out from behind her walls, and spilled like a faint echo of the river’s song across the eyot and over to the further shore. Passers-by listened to the leaping sounds, and whispered to each other of places where another world came close.
“It is a goddess of the running waters,” said some, “a queen in exile,” said others, and a few just sighed, aching in their heart for the loss of a place they had never known.
She did not hear that, but she saw how people came out from the village to look at her walls, or kneel with arms outstretched and faces turned up to the sky. A cairn of pebbles started to grow where the bank came closest, and when it was waist-high they left gifts there, little offerings out of their meagre possessions.

Second extract – further on

Unquenchable hunger seized her again. She tried not to eat, but it was stronger than gravity, irresistible as wind, and she could not deny it. Great helpless tears rolled down her face even as she tore at great strips of leaf and swallowed brimming bowls of sap.

Heavy, and feeling full to bursting, she wallowed on her couch, desperate for nightfall to come. Would she have even one more day before the unstoppable urge to sleep overwhelmed her?

They came that evening, and held up the infant again so she could see it. She sang again for them, and her song was full of both the beauty and the sorrow of the passing world. She watched the glow of wonder on their faces as they heard her. She knew what they could not, that this would be the last time she would see them, and she sang to bless them as the shortening day eased into night.

Long after they had gone, she lay looking at the riverbank where they had stood. The world was made up of shadows now. When her brother and sister next came, when they held up the infant for her to see, she would no longer be there. She would be lost in her own world of slumber and transformation, and the quick years of the world would roll unseen around her.

How long would they continue to come, she wondered, once the sound of her singing was gone? Would they think that she was lost to them, lost somewhere in the shadows? She watched herself stuffing food into her body, slithering awkwardly, heavily, into her chamber, and she felt that her heart was breaking.

Third extract – towards the end

The lady saw, and passed softly among the raucous din to stand near him.
“You know it too, don’t you? You know that you should be with her. Not this king, for all the food that fills his larder.”
He shivered and looked around. The man beside him asked a question, but he shook his head, puzzled, took a pinch of salt and tossed it over his shoulder. The lady withdrew, and his anxiety retreated again

.…

The king gestured to the minstrel and sat again. The room hushed in anticipation.
His singing was beautiful, she realised. The assistant kept the rhythm steady and flowing on the longer strings, as the master sang out the tale, plucking out higher riffs and ornaments here and there. She watched with admiration as his lay unfolded, not knowing the words but appreciating the patterns. And her own voice lifted up and joined him, even though her body lay on the couch within her chamber.
The lady moved among the guests, less than a shadow among them, step by step up to the musicians. She stood in front of them, basking in the melody. The singer’s words never faltered, but his gaze followed her as she came up to him. She had no idea what he saw of her – perhaps some extra brightness against the firelight, or a flicker of movement like a hidden bird within a thicket – but something in him knew that she was there.
The people heard his song, though not hers, and they were wild with delight as he finished, stilling the strings with the flat of his hand. The king took a ring from his own hands to give to the minstrel, but he shook his head. Instead, he stood and bowed very low before the lady. The room was silent now, waiting to see what happened. She wanted to lift him up: this adulation was altogether too much. But she knew that the desire was fruitless, and that she could not touch him.
The king spoke, a note of puzzlement in his voice, and the minstrel stood upright again. His answer was quiet, respectful, and he gestured to where the lady stood. The king, eyes narrowed, glanced here and there, but could not see her. She looked beyond him to the queen, and her face was alive with interest. She was aware, and so was the king’s right-hand man, who had moved across behind the queen to protect her.
There was a growing noise in the room, a buzz of speculation, and suddenly the focused attention became too much. The lady fled the room in haste, pulled herself from the couch and its loom, and pattered to and fro in the courtyard, slowly being soothed by the sights and scents of her garden.
Finally, she curled up on a bench in the pale sunshine. She could only face a few people at a time, she realised.

======================

Not too long to wait now…

History, Politics

Identity, belonging, and taxes

March 14, 2017 RichardAbbott Leave a comment

I’ve been meaning to write about this for some time now, but last week’s Budget here in the UK crystallised my thoughts. For non-UK readers, the Budget is a financial appraisal and forward plan presented by the Chancellor of the Exchequer. It’s an event where various changes to taxes are made, funding for different major projects is announced, and so on. It’s a chance for everyone to see what the Chancellor and his advisers think is going to happen, and how the country’s finances are going to be handled.

I’m not going to talk about the Budget itself – if you want, the whole document is available online. But there was one tiny snippet in the associated news which caught my eye:

Making tax digital – The start date for unincorporated businesses and landlords with turnover below the VAT registration threshold is deferred by one year to April 2019. Unincorporated business and landlord with turnover above the VAT registration threshold will be required to keep records in a digital format, and make quarterly reports of their results, from April 2018, using appropriate software.

In brief, it means that all firms, however small, will soon need to install suitable software, and be periodically linked up to the government tax computers to transfer earnings details. Those of us who are employed already get tax routinely withheld from our wages every payday, and the picture for the self-employed is rapidly converging.

Seeing as how it is only a short time since the tax authorities dealt exclusively in annual paper documents, this shift to online quarterly assessment is a vast change. And it is probably only a staging post on the way towards daily accountability. It’s a huge step from days of yore, when if the local land owner – or his lord, or the king, or the conquering overlord – needed some revenue, he just sent his heavies around and took it.

We live in a world where increasing numbers of people expect to move internationally during their working life – perhaps as much as 1/3 or 1/2 of people entering employment now will make such a move before they retire. So daily accountability is a necessary step to make sure that the right amount of taxes are paid in the country where they are due… a highly charged matter which several multinationals have run foul of.

So all that collided in my thoughts with another modern trend. This second one has been called “filiation versus affiliation“. It describes a major shift which has happened from the ancient world until now. Back then, your birth family – filiation – counted for everything, even in adult life. Social mobility was extraordinarily hard. We are moving towards a situation where as an adult you choose your own affiliation – your friends, your employment, your peer group, your gang, or whatever. Different countries are at different places along that trend, but we are all slowly moving that way. I find it exciting that I can chat with, work with, and be friends with people across the world who I am never likely to meet… but there are problems and difficulties as well.

So all that made me think, what if you could choose your country – your national affiliation – as easily as joining a Facebook group? Maybe I like the education system in Sweden, the stance on nuclear weapons in New Zealand, and a whole collection of other policies scattered around the world’s nations. But geographically I like living here in the UK, and in particular have a number of tip-top favourite places.

So… what if I could shop around with my tax liability? I go off to work, earn a few shekels, spend some of them on the necessities of life… and owe some agreed fraction in taxes. What if I could then decide which nations I wanted to support with those taxes, and effectively buy fractional citizenship there? In much the same way as I dispose of the rest of my income – so much to the supermarket, so much to a local independent store, so much to Amazon, so much put aside for a rainy day, etc.

I don’t think this would be an easy transition to make – for example, how would I pay London Underground for the trips I make on the Northern Line? I’m quite sure my ticket price doesn’t cover the cost of capital investment and replacement. How would I pay for the running of the National Parks that I love? There’d be all kinds of difficulties to work through.

But on the other hand, this could be a logical direction for democracy. Right now, the world’s major democracies are struggling with how to manage situations of narrow majorities. Big policy changes are being made on both sides of the Atlantic which go against the wishes of almost half of a population. So why not consider not only voting in a ballot box, but voting with the results of our labour? A parliamentary or presidential democracy results – seemingly inevitably – in governments becoming increasingly hardened in their stance on issues, and the voices of minority groups become increasingly hard to hear. Maybe the ability to move our citizenship, or fractions of it, away from one country and into another would undo that.

Kickstarter Logo (www.kickstarter.com)

At very least it might make budgets more like a kind of kickstarter pitch. Last week, the overall majority enjoyed by the current British government meant that the Chancellor could pretty much do as he pleased. He didn’t have to win anyone over to his position – although in the days since his speech we have seen some backpedalling after interventions from the Prime Minister, so perhaps a bit more discussion beforehand would have been prudent. But what if I could listen to his ideas, decide if I liked them, and then decide if I wanted to support them by means of citizenship and taxes? I might take into account the present government’s track record on things I care about, or the present Chancellor’s level of experience and expertise. Maybe I’d be persuaded, or maybe I’d take my tax elsewhere to someone who convinced me better.

I think people have a capacity to take a long view, so this wouldn’t rule out big projects taking years to come to fruition. After all, we already do that with pensions, or funds set up in childhood for major events later in life. The biggest risk is, perhaps, that money, and those with it, would potentially overwhelm everything. But in many ways that’s already true – public funding has been successively cut back on a whole raft full of artistic, educational, and environmental causes over the last few years. Maybe a kickstarter style approach to national budgets would bring funding back to some of these.

The present system has winners and losers, and I dare say so will any new replacement one. But I wonder if overall such a system would be more equitable, or less?

Cumbria, Science, Software, Timing

Cumbrian voice skills and Martian course corrections

March 7, 2017 RichardAbbott Leave a comment

My first piece of news today is by way of celebration that I have been getting some Alexa voice skills active on the Amazon store. These can now be enabled on any of Amazon’s Alexa-enabled devices, such as the Dot or Echo. One of these skills has to do with The Review blog, in that it will list out and read the opening lines of the last few posts there (along with a couple of other blogs I’m involved with). So if you’re interested in a new way to access blogs, and you’ve got a suitable piece of equipment, browse along to the Alexa skills page and check out “Blog Reader“. I’ll be adding other blogs as time goes by.

Cumbria Events Logo

The second publicly available skill so far relates to my geographical love for England’s Lake District. Called “Cumbria Events“, this skill identifies upcoming events from the Visit Cumbria web site, and will read them out for the interested user. You can expect other skills to do with both writing and Cumbria to appear in time as I put them together. It’s a pity that Alexa can’t be persuaded to use a Cumbrian accent, but to date that is just not possible. Also, the skills are not yet available on the Amazon US site, so far as I know, but that should change before too long.

Amazon Dot - Active — Amazon Dot – Active

In the process I’ve discovered that writing skills for Alexa is a lot of fun! Like any other programming, you have to think about how people are going to use your piece of work, but unlike much of what I’ve done over the years, you can’t force the user to interact in a particular way. They can say unexpected things, phrase the same request in any of several ways, and so on. Alexa’s current limitation of about 8 seconds of comprehension favours a conversational approach in which the dialogue is kept open for additional requests. The female-gendered persona of my own science fiction writing, Slate, is totally conversational when she wants to be.

It all makes for a fascinating study of the current state of the art of AI. I feel that if we can crack unstructured, open-ended conversation from a device – with all of the subtleties and nuances that go along with speech – then it will be hard to say that a machine cannot be intelligent. Alexa is a very long way from that just now – you reach the constraints and limitations far too early. But even accepting all that, it’s exciting that an easily available consumer device has so much capability, and is so easy to add capabilities.

Artists's impression, MAVEN and Mars (NASA/JPL) — Artists’s impression, MAVEN and Mars (NASA/JPL)

But while all that was going on, a couple of hundred million kilometres away NASA ordered a course correction for the Mars Maven Orbiter. This spacecraft, which has been in orbit for the last couple of years, was never designed to return splendid pictures. Instead, its focus is the Martian atmosphere, and the way this is affected by solar radiation of various kinds. As such, it has provided a great deal of insight into Marian history. So MAVEN was instructed to carry out a small engine burn to keep it well clear of the moon Phobos. Normally they are well separated, but in about a week’s time they would have been within a few seconds of one another. This was considered too risky, so the boost ensures that they won’t now be too close.

Now this attracted my attention since Phobos plays a major part in Timing – it’s right there on the cover, in fact. In the time-frame of Timing, there’s a small settlement on Phobos, which is visited by the main characters Mitnash and Slate as they unravel a financial mystery. This moon is a pretty small object, shaped like a rugby ball about 22 km long and about 17 or 18 km across its girth, so my first reaction was to think what bad luck it was that Maven should be anywhere near Phobos. But in fact MAVEN is in a very elongated orbit to give a range of science measurements, so every now and again its orbit crosses that of Phobos – hence the precautions. This manoeuvre is expected to be the last one necessary for a very long time, given the orbital movements of both objects. So we shall continue getting atmospheric observations for a long while to come.

Formatting, Kindle preparation

Kindle preparation part 4 – formatting

March 1, 2017 RichardAbbott Leave a comment

Kindle formatting has sometimes been the subject of intense debates. At one end of the spectrum there is a belief that it should mirror print conventions as far as possible. Alternatively, there is a contrasting belief that we have a new display medium which needs to be free to develop its own ideas. I’m inclined towards the second view, though I do think that some print conventions still have value. To extract parts of a long debate on The Passive Voice blog about text alignment, which attracted quite opposing views:

“Do not force the reader to read the way you want her to… Just get out of the way… Good readable text that flows nicely… keep it left justified… Is there anything that shouts ‘amateur and non-pro” more than a justified book?… I’ll take amateur and readable for people who read with large font sizes… There’s a lot to be said for left justification on a smaller screen with larger font”.

Obviously a lively topic. But first, there is the question of what is feasible. Some expectations may simply not be realistic, and we’ll come back to this later. Even if you restrict yourself to Kindle files, there is considerable variation in how the same file is displayed by different devices. Step away from Kindles into the diverse world of epub readers and reading apps, and the variation only increases. As usual, most of this post will talk about Kindles specifically, but an exactly parallel process applies to epub books.

Now, as well as differences between devices, you are faced with individual reader choices. A Kindle device, or a software app emulating a Kindle on a phone, tablet, or computer, places enormous flexibility of choice in the hands of the reader. The reader can change font, font size, margin width, line spacing, and background colour, as well as swap between portrait and landscape orientation. Many epub readers offer even more choices.

Now with traditional print layout, all of these are decided by author or publisher, and the reader has no choice. Once the design is chosen, that’s it. This has given rise over the years to a set of conventions for printed books. Not so with ebooks. The phenomenal rise of epublishing, with free and easy to use tools enabling indie publishing, has given huge empowerment to authors. But I sometimes think that authors have not caught on to the fact that it has also given huge empowerment to readers.

As reader, I don’t have to put up with someone else’s choices. If I want a serif font instead of san serif – or a dyslexic-friendly font for that matter – or I like big text size, or wide margins, or two columns side by side, or different background colour, or whatever, I can choose that. If the author or publisher tries to stop me, I’ll get frustrated. It’s not appropriate any longer for an author to try to decide how a reader ought to read his or her book. Sometimes I hear people say that what matters is the layout of the book when originally downloaded, but this shows a misunderstanding of how the devices work. My reader settings are mine, and yours are yours, and they get applied equally to newly downloaded or existing books. There is nothing magic about the settings chosen by the author. If I want to read your ebook with right-aligned text, I can do so, and I can feel frustrated if you try to stop me.

As I mentioned before, not all Kindle devices, or Kindle software apps on computers, phones and tablets, treat the content the same way. My phone Kindle app (both Android and iPhone) handles changes of font size differently from my various actual Kindles, including the way it decides to justify text. This is something built into the app itself, not a thing I have direct control over.

So what does that mean for Kindle formatting? A layout that looks good when the font is small compared to the page width may well be confusing when the font is large. A layout that is easily readable when the font is large compared to the page size may well seem non-standard when the font is small.

Now, being a new technology, and moreover one which has grown up linked closely to the world of web page design, there has been a great deal of systematic study of the readability of different styles. This hasn’t happened to the same degree with the world of print, except in the very specific area of font design. Some choices which are routinely made in a printed book, and which have become part of the lore of book production, were originally made for all kinds of diverse reasons including wartime economy. In particular, the normal print practice of justifying text both left and right is not based on considerations of readability, but rather on maximising the number of characters on what was a scarce resource – paper. Change the ratio of words per line significantly away from what is common for a novel, and flush-both-sides becomes rather unreadable, as reported by numerous systematic studies.

If the paper is wide compared to the typical word size, then your eye loses its ability to scan lines comfortably. This is why newspapers and magazines split text into columns – and why the landscape mode in recent ereaders gives the user the option to do just this.

If the page is narrow compared to the words, there is a tendency for the layout to become erratic and irregular. Areas of open space appear, called “rivers”, creating a ragged and untidy appearance. Text which is left-aligned only, with ragged right margin, is better spaced, aesthetically more pleasing, and also more readable. In times past, the Kindle layout engine was heavily criticised for its poor showing in this area. It has improved, but is still way behind the result that can be achieved with a fixed page width. If you think about it, this is inevitable. Every time you change the font size, or the margin, or the aspect – even if you highlight a piece of text and then later jump back to it – every time something like this happens, the Kindle software has to recalculate the position of each word. You get huge advantages over a physical book with rigid layout, but you also have to recognise, and cope with, the limitations. Newer versions of the Kindle layout software deliberately switch to flush left only (ragged right) as the font size increases, to address this very issue.

Readability considerations, then, suggest that flush-both-sides works best in the middle range of Kindle fonts, degrading in different ways as you go towards the extremes of size options. And although modern Kindles have recognised this, and provided a built-in solution, older ones do not. So next time we’ll look at another option that can be used by authors which they can control. For today, it’s enough to recognise that there is something to think about here, and that trying to simply copy what is done in print does not necessarily work well.

I mentioned earlier about things which are simply infeasible in ebooks, even though normal and appropriate in print. One of these concerns hyphenation. The print version, with fixed word positioning, can be carefully laid out to get hyphens in the perfect places. Kindle books can’t – any choice which is correct for one person’s device and settings will be wrong for the next. Recent software layout engines do a reasonable job of inserting hyphens, but they struggle with some words, especially proper nouns. You can easily see this if you find a book line with several long words, then expand the font size. At some point the layout engine just gives up.

Another area is that of widows and orphans – single lines appearing at the bottom or top of a page, which are considered distracting to the reader. A publisher of a printed book tries hard to eliminate these by judicious choice of word selection and spacing. It can’t be done on a Kindle. If as author you did carefully sort all that out on your own device, it will all go wrong with a change of settings or on a new gadget.

So there are stylistic choices which are sound and reasonable in print, but which cannot be carried over to ebooks. It’s well worth thinking about this when you’re preparing a book, and only spending time on the issues that can be fixed. And do try out how the book appears when viewed with completely different user settings than the ones you personally like!

Far from the Spaceports, Science, Science fiction, Timing

Some space news

February 22, 2017 RichardAbbott Leave a comment

I ran out of time this week to do much by way of blogging, so here are three bits of space news which may well make their way into a story sometime.

Stop Press: just today NASA announced that a relatively close star (39 light years away) has no less than 7 planets approximately Earth size orbiting it… see and the schematic picture at the end of the blog.

False colour image of the area of interest (NASA/JPL)

Firstly, the Dawn probe, still faithfully orbiting the asteroid Ceres, has detected complex organic molecules in two separate areas in the middle latitudes of the dwarf planet. The onboard instruments are not accurate enough to pin the molecules down precisely, but it seems likely that they are forms of targets. The analysis also suggests that they formed on Ceres itself, rather than being deposited there by a meteor. The most likely cause is thought to be the action of warm water circulating through chemicals under the surface. Some of the headlines suggest that this could signal the presence of life, but it’s more cautious to say that it shows that the conditions under which life could develop are present there.

Recent cratering on Mars (HiRise camera, U Arizona)

The second snippet spells difficulty for my hypothetical Martian settlements. This picture was captured by the Mars Orbiter and shows two larger impact craters surrounded by a whole array of smaller ones. The likely scenario is that one object split into a cluster of fragments as it passed through the Martian atmosphere. This of itself wouldn’t be too surprising, but inspection of older photos of the same area shoes that this impact happened between 2008 and 2014. No time at all in cosmic terms, and not so much fun if you’d carefully built yourself a habitable dome there.

The problem is the thinness of the Martian atmosphere. It is considerably deeper than our one here on Earth, but hugely less dense. So when meteors arrive at the top of the layer of air, they don’t burn up so comprehensively as Earth-bound ones. More of them reach the surface. Even a comparatively small rock has enough kinetic energy to really spoil your day. Something that will need some planning…

Artist's impression of Kuiper Belt object (NASA) — Artist’s impression of Kuiper Belt object (NASA)

Finally we zoom right out to the cold, dark reaches of the outer solar system. A long way beyond the orbit of Pluto there is a region called the Kuiper Belt, and out in the Kuiper Belt a new dwarf planet has recently been found. It goes by the catchy name of 2014 UZ224 and it took nearly two years to confirm its existence. Best estimates are that it is a little over 300 miles across – about half the size of Ceres. I’ve never sent Mitnash and Slate out anywhere like that – it’s about twice as far from Earth as Pluto, and the journey alone would take about four months one-way. I do have vague plans for a story set out in the Kuiper Belt, but appropriately enough it’s some way off yet. But even at that distance, you’re still less than half a percent of the distance to the nearest star… space is really big!

Schematic picture of Trappist-1's planets — Schematic picture of Trappist-1’s planets

Far from the Spaceports, Science, Science fiction, Software, Timing

Who is Alexa, where is she?

February 14, 2017 RichardAbbott 1 Comment

Since as far back as written records go – and probably well before that – we humans have imagined artificial life. Sometimes this has been mechanical, technological, like the Greek tales of Hephaestus’ automata, who assisted him at his metalwork. Sometimes it has been magical or spiritual, like the Hebrew golem, or the simulacra of Renaissance philosophy. But either way, we have both dreamed of and feared the presence of living things which have been made, rather than evolved or created.

Modern science fiction and fantasy has continued this habit. Fantasy has often seen these made things as intrusive and wicked. In Tolkein’s world, the manufactured orcs and trolls (made in mockery of elves and ents) hate their original counterparts, and try to spoil the natural order. Science fiction has positioned artificial life at both ends of the moral spectrum. Terminator and Alien saw robots as amoral and destructive, with their own agenda frequently hostile to humanity. Asimov’s writing presented them as a largely positive influence, governed by a moral framework that compelled them to pursue the best interests of people.

But either way, artificial life has been usually conceived as self-contained. In all of the above examples, the intelligence of the robots or manufactured beings went about with them. They might well call on outside information stores – just like a person might ask a friend or visit a library – but they were autonomous.

Yet the latest crop of virtual assistants that are emerging here and now – Alexa, Siri, Cortana and the rest – are quite the opposite. For sure, you interact with a gadget, whether a computer, phone, or dedicated device, but that is only an access point, not the real thing. Alexa does not live inside the Amazon Dot. The pattern of communication is more like when we use a phone to talk to another person – we use the device at hand, but we don’t think that our friend is inside it. At least, I hope we don’t…

So where is Alexa and her friends? When you ask for some information, buy something, book a taxi, or whatever, your request goes off across cyberspace to Amazon’s servers to interpret the request. Maybe that can be handled immediately, but more likely there will be some additional web calls necessary to track down what you want. All of that is collated and sent back down to your local device and you get to hear the answer. So the short interval between request and response has been filled with multiple web messages to find out what you wanted to know – plus a whole wrapper of security details to make sure you were entitled to find that out in the first place. The internet is a busy place…

So part of what I call Alexa is shared between every single other Alexa instance on the planet, in a sort of common pool of knowledge. This means that as language capabilities are added or upgraded, they can be rolled out to every Alexa at the same time. Right now Alexa speaks UK and US English, and German. Quite possibly when I wake up tomorrow other languages will have been added to her repertoire – Chinese, maybe, or Hindi. That would be fun.

But other parts of Alexa are specific to my particular Alexa, like the skills I have enabled, the books and music I can access, and a few features like improved phrase recognition that I have carried out. Annoyingly, there are national differences as well – an American Alexa can access the user’s Kindle library, but British Alexas can’t. And finally, the voice skills that I am currently coding are only available on my Alexa, until the time comes to release them publicly.

Amazon Dot - Inactive — Amazon Dot – Inactive

So Alexa is partly individual, and partly a community being. Which, when you think about it, is very like us humans. We are also partly individual and partly communal, though the individual part is a considerably higher proportion of our whole self than it is for Alexa. But the principle of blending personal and social identities into a single being is true both for humans and the current crop of virtual assistants.

So what are the drawbacks of this? The main one is simply that of connectivity. If I have no internet connection, Alexa can’t do very much at all. The speech recognition bit, the selection of skills and entitlements, the gathering of information from different places into a single answer – all of these things will only work if those remote links can be made. So if my connection is out of action, so is Alexa. Or if I’m on a train journey in one of those many places where UK mobile coverage is poor.

There’s also a longer term problem, which will need to be solved as and when we start moving away from planet Earth on a regular basis. While I’m on Earth, or on the International Space Station for that matter, I’m never more than a tiny fraction of a second away from my internet destination. Even with all the other lags in the system, that’s not a problem. But, as readers of Far from the Spaceports or Timing will know, distance away from Earth means signal lag. If I’m on Mars, Earth is anywhere from about 4 to nearly 13 minutes away. If I go out to Jupiter, that lag becomes at least half an hour. A gap in Alexa’s response time of that long is just not realistic for Slate and the other virtual personas of my fiction, whose human companions expect chit-chat on the same kind of timescale as human conversation. The code to understand language and all the rest has to be closer at hand.

So at some point down the generations between Alexa and Slate, we have to get the balance between individual and collective shifted more back towards the individual. What that means in terms of hardware and software is an open problem at the moment, but it’s one that needs to be solved sometime.

Formatting, Kindle preparation, Writing

Kindle preparation part 3 – navigation

February 7, 2017 RichardAbbott Leave a comment

A shorter blog today focusing specifically on navigation. I mentioned before that there were two different ways of navigating through the sections of an ebook, and this little post will focus on how they work. The two methods appear differently in the book – the HTML contents page is part of the regular page flow of the book, and the NCX navigation is outside of the pages, as we’ll see later.

It’s an area where ebooks behave quite differently to print versions. If you have a table of contents (TOC) in a print book it’s essentially another piece of static text, listing page numbers to turn to. Nonfiction books frequently have other similar lists such as tables or pictures, but I’ll be focusing only on section navigation – usually chapters but potentially other significant divisions. In an ebook this changes from a simple static list into a dynamic means of navigation.

Let’s take the HTML contents first. It looks essentially the same as the old print TOC, except that the page numbers are omitted (since they have no real meaning) and are replaced by dynamic links. Tap the link and you go to the corresponding location. They look just like links in a web page, for the very good reason that this is exactly what they are!

So the first step is to construct your HTML contents list, for which you need to know both the visible text – “Chapter 1”, perhaps – and the target location. Authors who use Word or a similar tool can usually generate this quite quickly, while those of us who work directly with source files have the easy task of inserting the anchor targets by hand. It’s entirely up to you how you style and structure your contents page – maybe it makes sense to have main parts and subsections, with the latter visually indented. It’s your choice.

The NCX navigation is a bit different. It’s a separate file, and the pairing of visible text and target link is done by means of an XML files of a specific structure. Again. some commercial software will be able to generate this for you, using the HTML TOC as a starting point, but it’s as well to know what it is doing for you. Conventionally the two lists of contents mirror each other, but this doesn’t have to be the case. For example, it might suit you better to have the HTML version with an exhaustive list, and the NCX version with just a subset. It’s up to you. However, the presence of NCX navigation in some form is a requirement of Amazon’s, sufficiently so that they reserve the right to fail validation if it’s not present. And it’s a mandatory part of the epub specifications, and a package will fail epubcheck if NCX is missing. You’ll get an error message like this:

Validating using EPUB version 2.0.1 rules.
ERROR(RSC-005): C:/Users/Richard/Dropbox/Poems/test/Test.epub/Test_epub.opf(39,8): Error while parsing file ‘element “spine” missing required attribute “toc”‘.
ERROR(NCX-002): C:/Users/Richard/Dropbox/Poems/test/Test.epub/Test_epub.opf(-1,-1): toc attribute was not found on the spine element.

Of course, if you don’t check for validation, or if you just use Kindlegen without being cautious, you will end up with an epub or Kindle mobi file that you can ship… it will just be lacking an important feature.

Interestingly, you don’t get an error if you omit the HTML TOC – so long as everything else is in order, your epub file will pass validation just fine. This is the opposite of what folk who are used to print books might guess, but it reflects the relative importance of NCX and HTML contents tables in an ebook.

So what exactly do they each do? The main purpose of the HTML version is clear – it sits at the front of the book so that people can jump directly to whatever chapter they want. It would do this even if you just included the file in the spine listing. But if you are careful to specify its role in the OPF file, it also enables a link in the overall Kindle (or epub) navigation. This way the user can jump straight to the TOC from anywhere.

The NCX navigation enables the rest of this “Go To” menu. If it’s missing, or incorrectly hooked up in the OPF file, the navigation will be missing, and you are leaving your readers struggling to work out how to flick easily to and fro. On older Kindles, there were little hardware buttons (either on the side of the casing or marked with little arrows on the front) which would go stepwise forwards and backwards through the NCX entries.

So that’s it for the two kinds of navigation. They’re easy to include, they add considerably to the user experience, and in one way or another are considered essential.

Far from the Spaceports, Science, Science fiction, Timing

The Power of Speech

January 31, 2017 RichardAbbott Leave a comment

I recently invested in an Amazon Dot, and therefore in the AI software that makes the Dot interesting – Alexa, Amazon’s virtual assistant. But I’m not going to write about the cool stuff that this little gizmo can do, so much as what it led me to think about AI and conversation.

The ability to interact with a computer by voice consistently, effectively, and on a wide range of topics is seen by the major industry players as the next big milestone. Let’s briefly look back at the history of this.

Punched card with Fortran programming - I started with that language, long ago... (Wiki) — Punched card with Fortran programming – I started with that language, long ago… (Wiki)

Once upon a time all you could use was a highly artificial, structured set of commands passed in on punched cards, or (some time later) via a keyboard. If the command was wrong, the machine would not do what you expected. There was no latitude for variation, and among other things this meant that to use a computer needed special training.

The first breakthrough was to separate out the command language from the user’s options. User interfaces were born: you could instruct the machine what you wanted to do without needing to know how it did it. You could write documents or play games without knowing a word of computer language, simply by typing some letters or clicking with a mouse pointer. Somewhere around this time it became possible to communicate easily with machines in different locations, and the Internet came into being.

The next change appeared on phones first – the touch screen. At first sight there’s not a lot of change from using a mouse to click, or your finger to tap. But actually they are worlds apart. You are using your body directly to work with the content, rather than indirectly through a tool. Also, the same interface – the screen – is used to communicate both ways, rather than the machine sending output through the screen and receiving input via movements of a gadget on an entirely different surface. Touch screens have vastly extended the extent to which we can access technology and information: advanced computers are quite literally in anyone’s pocket. But touch interfaces have their problems. It’s not especially easy to create passages of text. It’s not always obvious how to use visual cues to achieve what you want. It doesn’t work well if you’re making a cake and need to look up the next stage with wet and floury hands!

Which brings us to the next breakthrough – speech. Human beings are wired for speech, just as we are wired for touch. The human brain can recognise and interpret speech sounds much faster than other noises. We learn the ability in the womb. We respond differently to different speakers and different languages before birth, and master the act of communicating needs and desires at a very early age. We infer, and broadcast, all kinds of social information through speech – gender, age, educational level, occupation, emotional state, prejudice and so on. Speech allows us to explain what we really wanted when we are misunderstood, and has propelled us along our historical trajectory. Long before systematic writing was invented, and through all the places and times where writing has been an unknown skill to many, talking has still enabled us to make society.

Enter Alexa, and Alexa’s companions such as Siri, Cortana, or “OK Google”. The aim of all of them is to allow people to find things out, or cause things to happen, simply by talking. They’re all at an early stage still, but their ability to comprehend is seriously impressive compared to a few short years ago. None of them are anywhere near the level I assume for Slate and the other “personas” in my science fiction books, with whom one can have an open-ended dialogue complete with emotional content, plus a long-term relationship.

What’s good about Alexa? First, the speech recognition is excellent. There are times when the interpreted version of my words is wrong, sometimes laughably so, but that often happens with another person. The system is designed to be open-ended, so additional features and bug fixes are regularly applied. It also allows capabilities (“skills”) to be developed by other people and added for others to make use of – watch this space over the next few months! So the technology has definitely reached a level where it is ready for public appraisal.

Hidden Markov model - an algorithm often used in speech recognition (Wiki) — Hidden Markov model – an algorithm often used in speech recognition (Wiki)

What’s not so good? Well, the conversation is highly structured. Depending on the particular skill in use, you are relying either on Amazon or on a third-party developer, to anticipate and code for a good range of requests. But even the best of these skills is necessarily quite constrained, and it doesn’t take long to reach the boundaries of what can be managed. There’s also very little sense of context or memory. Talking to a person, you often say “what we were talking about yesterday...” or “I chatted to Stuart today…” and the context is clear from shared experience. Right now, Alexa has no memory of past verbal transactions, and very little sense of the context of a particular request.

But also, Alexa has no sense of importance. A human conversation has all kinds of ways to communicate “this is really important to me” or “this is just fun”. Lots of conversations go something like “you know what we were talking about yesterday…“, at which the listener pauses and then says, “oh… that“. Alexa, however, cannot distinguish at present between the relative importance of “give me a random fact about puppies“, “tell me if there are delays on the Northern Line today“, or “where is the nearest doctor’s surgery?”

These are, I believe, problems that can be solved over time. The pool of data that Alexa and other similar virtual assistants work with grows daily, and the algorithms that churn through that pool in order to extract meaning are becoming more sensitive and subtle. I suspect it’s only a matter of time until one of these software constructs is equipped with an understanding of context and transactional history, and along with that, a sense of relative importance.

Alexa is a long way removed from Slate and her associates, but the ability to use unstructured, free-form sentences to communicate is a big step forward. I like to think that subsequent generations of virtual assistants will make other strides, and that we’ll be tackling issues of AI rights and working partnerships before too long.

Meanwhile, back to writing my own Alexa skill…

Historical fiction

Kindle preparation part 2 – the files

January 24, 2017 RichardAbbott Leave a comment

Last time I looked at the basic principles of a Kindle mobi or general epub file. This time I’ll be focusing a bit more on what the different ingredients do. We’ll also start to uncover a few more places where Kindle and epub handle things differently. For reference, here is a sample set of files you need for an epub book – Kindle is essentially the same but some “administrative” bits are inserted automatically by KindleGen so you don’t need to worry.

Somebody who uses Microsoft Word or some similar software to construct their book may find the following paragraphs confusing, since they will probably never have needed to address this directly. But under the bonnet this is what is happening with your book preparation, and many years of technical software and QA work has convinced me it’s better to know rather than not know. At very least this may help diagnose when something goes wrong!

So, the key ingredient is the opf file which ties everything together. It has four main sections. The first is Metadata – a general information section containing things like author name, book title, publisher, ISBN (if any), price, brief description, and so on. The kind of detail you might expect to see on a library card or catalogue entry. I’ll be giving specific examples of the different files later in this series but for now want to concentrate on principles rather than details. There are also some important places where you have to ensure that a reference in one place matches one somewhere else – again, I’ll return to this.

The second section is a list of resources – the Manifest. For epub this must be complete, and although KindleGen is clever enough to fill in some gaps, it is good practice to be thorough here as well. So this identifies all content files, any separate style sheets, all images including cover, the ncx navigation file, and anything else you intend to include. But it’s a simple list, like the ingredients for a recipe before you get to the directions, and this section doesn’t tell KindleGen or an epub reader how to assemble the items into a book.

The third section – the Spine – does this work of assembly. It lists the items that a reader will encounter in their correct order. This section turns your simple list of included items into a proper sequence, so that chapter two comes after chapter one. Here you also link in the ncx file so it can do its job.

The final section – the Guide – defines key global features of the finished book. For example, this is where you define the cover, the HTML contents page, and the start point – the place where the book opens for the very first time, and the target for the navigation command “Go to Beginning” (or equivalent). It’s worth remembering that the start point doesn’t have to be the first page – many books set this after the front matter, so that you skip over title pages and such like and begin at the beginning of the actual story. But be warned that following a scam to do with counts of pages read, Amazon does not take kindly to people putting the start point too far through the book.

epub treatment of png and gif transparency

Images can cause unexpected problems. The opf file expects you to supply not just a file name, but also the file type, such as jpeg, png or gif. And here we encounter one of those annoying differences between devices. Kindle accepts png files along with jpeg and gif, and many people are used to the convenient feature of the png format that it allows transparency. A png with transparency will – usually – take on the background from whatever happens to be behind it, like the page background for example. An epub file will do exactly this if you use coloured background.

Kindle treatment of png and gif transparency

But KindleGen does not. You can supply a png file successfully, but internally it will be converted to jpeg format… and jpegs do not allow transparency. The background will be converted to white, and the final effect will not be what you hoped for. The way round this is to use gif images if you want transparency, but since this is an old format many people do not suspect that this is necessary.

Now, sometimes this won’t matter – for example if you want to insert a map, and have it look as though it is on white paper. But other times it looks decidedly odd, when it is intended to be just a logo or divider symbol. It’s a thing which particularly catches out those who are used to printed books, or older Kindles which only supported black-and-white. It’s not very long since I discovered this hidden conversion png -> jpeg that KindleGen does, and as a result expanded my pre-publication testing considerably.

A couple of closing comments about the files themselves. The contents files ought to be valid HTML – this sounds obvious, but most browsers, and KindleGen, are very forgiving about syntax errors, so people often forget to be careful. But although the output file may be generated, such errors can lead to surprising changes of appearance between paragraphs. It is good practice to use the w3.org online validator to check this – it’s completely free, and will either confirm that the file is valid or else tell you what’s wrong and how to fix it. Alternatively, once you have built an epub file, the free epubcheck utility will do a similar job as one of its several checks. (I’ll come back to epubcheck when I talk about building an epub file). Other than that, you are at liberty to split up content however you like – all in a single file, or one per chapter, or whatever. It’s up to whatever you find convenient (though a few epub apps load just one file at a time so you might notice a slight delay every now and again while reading).

Writing Slate