Lontar

Developing Keyboards for iOS

Norbert Lindenberg
December 5, 2014

With iOS 8, Apple provides an extension API that lets third-party developers create custom keyboards for iOS. This article provides some tips for developers from my experience developing the English IPA keyboard, from looking at a number of other keyboards, and from discussions on Apple’s developer forum.

Get started

To get started, read the following Apple materials:

There are several tutorials that go slightly beyond Apple’s guide, for example by Andrei Puni and Tope, as well as a report on all the troubles encountered by Alexei Baboulevitch.

Consider the constraints

The keyboard extension API is very much a minimum viable product: It provides the core mechanisms that enable a custom keyboard to load and interact with an app, but there are numerous limitations. The section System Keyboard Features Unavailable to Custom Keyboards of the Guide lists many of them. Most importantly, you cannot draw outside the keyboard’s rectangle – there’s no way to indicate spelling mistakes or suggestions in the text being edited, and no enlarged keys popping up from the top row of the keyboard (unless you expand the keyboard to gain some extra space).

keys popping up above the top row of the keyboard
Not possible in custom keyboards: Keys popping up above the top row.

Overall these constraints mean that you’ll have to create a keyboard that’s significantly different and at the same time full-featured if you want to convince users to replace Apple’s keyboards with yours. Just letting the user choose the color of the keys may not be sufficient if you don’t offer spelling correction. On the other hand, if you enable input for languages or Unicode blocks that Apple doesn’t support with its keyboards, the bar is lower.

Require iOS 8.1

Do yourself and any remaining iOS 8.0 users a favor and require iOS 8.1 as the deployment target. iOS 8.0 shipped before its support for third-party keyboards was fully baked, leading to keyboards frequently not appearing or not being functional. iOS 8.1 is a lot more stable.

Provide maximum functionality without Full Access

By default, keyboards can only communicate with the app for which they’re currently producing input, the host app. They cannot communicate with the app with which they’re bundled (the containing app) or anything else. If they do need the ability to share data with the containing app or access the network, they can request permission by setting the RequestsOpenAccess flag in the Info.plist file to YES, and the user can then grant or deny that permission using the Full Access setting. Many users do not want to grant full access because they don’t want their text input sent off to unknown companies. It’s therefore a good idea to provide as much functionality as possible without full access. Areas to consider:

Yes, some of these suggestions sound a little desperate, and for many keyboards you will at some point have to request full access. Things would be a lot easier if Apple allowed containing apps to write into their keyboards’s containers, similar to how the Mail app delivers attachments into an app’s Inbox directory, without the need for full access. This would let containing apps send dictionaries and configuration settings to the keyboard while protecting user input from being leaked out of the keyboard.

Provide digits

Section 25.6 of the App Store Review Guidelines requires that “Keyboard extensions must provide Number and Decimal keyboard types as described in the App Extension Programming Guide or they will be rejected”. However, the Guide says nothing about Number and Decimal keyboard types. The closest it gets to the topic is by listing “appropriate layout and features based on keyboard type trait” as a feature for which “you can decide whether or not to implement” it. However, before you decide not to implement: A number of keyboards have been rejected based on section 25.6, so it seems something is required. For the first version of the English IPA keyboard, I argued that the keyboard was designed as a supplementary keyboard, not as a user’s primary keyboard, and therefore shouldn’t have to support all the keyboard types, similar to Apple’s emoji keyboard. My keyboard was approved, but at least one other developer using similar arguments saw his app rejected. (In later versions of mine, I added more layers and had some spare space that I filled with digits.)

Based on what I see in shipping keyboards, it seems that the minimum requirement is to have a set of digits somewhere in the keyboard. Many shipping keyboards don’t respond to the keyboardType property of the text document proxy, so apparently that’s not required. Some keyboards use non-Western digits, for example, the Khmer digits ១, ២, ៣, ៤, ៥, ៦, ៧, ៨, ៩, ០ on a Khmer keyboard – that seems fine. What Apple really wants to accomplish with this requirement remains a mystery.

Update January 25, 2015: Version 1.3 of the English IPA keyboard has reached the App Store this week, again without any digits. Customers wanted more IPA characters, to make space for those I had to remove the digits, and together with the argument that this is a supplementary keyboard that was apparently sufficient to convince the reviewer.

Design for both iPhone and iPad

You can’t create a keyboard for only iPhone or only iPad. You can label the containing app as designed for only iPhone or only iPad, but that doesn’t restrict where users can use the keyboard. Apps designed for iPhone only can all be run on iPad, either in original iPhone size (small) or pixel-doubled (ugly). Keyboards that come with such apps can be used within any app on the iPad, and are given normal iPad-sized keyboard areas when used with iPad apps and iPhone-sized keyboard areas when used with iPhone apps. Containing apps designed for iPad only can only be run on iPad, but keyboards that come with such apps can also be used within iPhone apps on the iPad, and are then given iPhone-sized keyboard areas. This means that keyboards that aren’t designed for both iPhone and iPad will probably look out of place in one environment, not taking advantage of large keyboard areas on iPad or squeezing too many keys into small keyboard areas on iPhone. Note also that keyboards usually provide a key to dismiss the keyboard on iPad, but not on iPhone or within an iPhone app on iPad. To make things more interesting, UIDevice.userInterfaceIdiom will always report an iPad on iPad, whether the keyboard is used within an iPad app or an iPhone app, so you’ll need to look at the actual keyboard view frame to decide which design to use.

Reconsider Auto Layout

The keyboard extension framework provides you with an empty gray rectangle. How you fill it with keys or other user interface elements is entirely up to you. It may be tempting to open up a storyboard, drop in a few dozen buttons, and tie them up with constraints until Auto Layout produces the desired results. However, for my keyboard I found it far easier to handle the layout in code – like most keyboards, it’s just a simple grid of keys, with a few variations such as larger function keys or indented rows. Alexei Baboulevitch, who tried Auto Layout first and then implemented his own layout code, reports that this also resulted in a significant performance improvement.

Use images for function keys

The one and only key that every keyboard must provide is a “next keyboard” key, for which the built-in keyboards use a globe. As it turns out, the emoji font in iOS also contains a globe: 🌐. It is tempting to use that emoji for the “next keyboard” key, and then start looking for other symbol characters in this or other fonts. However, you’ll likely find that not all the symbols you need are in Unicode, not all that are in Unicode have a glyph in some iOS font, and the collection of glyphs you find doesn’t harmonize (🌐⇪⌫⌨?). It’s much better to design custom images for the function keys that your keyboard needs. There doesn’t seem to be any way for third-party keyboards to access the images that Apple’s built-in keyboards use.

Handle “return” key types just as labels

The text document proxy’s returnKeyType property indicates how the host app will interpret taps on your keyboard’s “return” key. Note that it’s the app that will interpret – all the keyboard has to do is to label the key according to the return key type, and call proxy.insertText("\n") when the key is tapped. Apple’s keyboards usually use labels in the language that the keyboard itself supports, so there’s no need to translate the labels into other languages. The labels for Google and Yahoo are now just “Search”.

Don’t expect textDidChange or selectionDidChange to be called when text or selections change

Reading the documentation might lead you to expect that your input view controller’s textDidChange method will be called when “text has changed in the document”, or selectionDidChange when “the selection has changed in the document”. Reality as of iOS 8.1 is that textDidChange and its buddy textWillChange are called when the keyboard’s client view becomes or resigns as first responder and when the selection changes, and selectionDidChange and its peer selectionWillChange are never called. Of course, this might get fixed eventually, so don’t rely on this either. In my keyboard, I just take them to indicate that it’s time to check on the current state of the document proxy’s text length and text input traits and update the keyboard UI as needed.

Understand that deleteBackward deletes ... something

The documentation for UIKeyInput.deleteBackward states that it deletes “a character”. Those familiar with Unicode, NSString, and Swift’s String type know that the word “character” is quite overloaded – it could mean a Unicode code point, a UTF-16 code unit, or an extended grapheme cluster. But that’s only where the trouble starts.

What deleteBackward really does, depends on its implementation in the view that your keyboard is interacting with, which could be a UITextView, or a text field within a web application running in Safari, or a custom view in a third party app. Given the lack of a real specification, implementations may differ from each other and change over time.

UITextView and Safari in iOS 8.1 don’t use any of the units mentioned above: They delete a base character with all following non-spacing marks (such as ä̟́), a complete Korean syllable, or a complete flag emoji (such as 🇪🇸) in a single call to deleteBackward. But then there are special situations: If the current selection is not empty, then deleteBackward deletes the selected text. If the insertion point happens to be right after an image embedded in content-editable text in Safari, then deleteBackward deletes the image and no characters at all. Neither the currently selected text nor images are visible through documentContextBeforeInput or documentContextAfterInput.

This all makes it impossible to calculate in advance how often you have to call deleteBackward in order to delete a specific piece of text, such as a word – you just have to keep calling the method until the text is gone, and hope that the user did want to delete the intervening content that’s not visible through the API.

Implementing forward deletion as described in the API Quick Start for Custom Keyboards section of the Guide also does not work. It depends on the assumption that the increment used by adjustTextPositionByCharacterOffset matches what deleteBackward deletes, but in reality that’s not always the case. In Safari, the Hindi word नमस्ते requires an offset of 4 to adjust the position from its start to its end, but 6 calls to deleteBackward to delete. In UITextView, offset and calls happen to match for this word (6 each), but then the flag 🇰🇷 requires an offset of 4, giving callers an opportunity to step into the middle of a code point and to delete a partial character.

Play input clicks only when you have full access

It seems unlikely that you could invade the user’s privacy by playing click sounds when she taps a key, but the method in charge of these sounds, UIDevice.playInputClick, hangs for several seconds rather than doing its job when called without full access. Call it only when you have full access, and be quiet otherwise.

When you do have full access, the following code enables input clicks:

extension UIInputView: UIInputViewAudioFeedback {

 

public var enableInputClicksWhenVisible: Bool { get { return true } }

 

func playInputClick​() {

UIDevice​.currentDevice​().playInputClick​()

}

}

Use the right language tag

A keyboard should specify the language it supports using a BCP 47 language tag, either in the PrimaryLanguage field in the Info.plist file or, if it can change at runtime, through the primaryLanguage property of UIInputViewController. Apps may depend on this information for language-specific processing. The W3C has a nice guide to language tags and how to construct them, and the IANA language subtag registry provides all the subtags you can use. An IPA keyboard for English should use en-fonipa, a N’Ko keyboard nqo, and a keyboard for non-linguistic items (such as emoji) zxx.

Make the containing app (somewhat) useful

Apple requires that the containing app with which a keyboard is submitted to the App Store provides functionality to the user. Don’t let that scare you; the bar is actually pretty low. Currently shipping apps typically provide some (rarely all) of the following:

And with that, I wish you success with your custom keyboard!

Acknowledgments

Special thanks to Muthu Nedumaran, creator of the Sangam keyboards, for reviewing a draft of this article.