Infotronics is a term used to describe in-vehicle multimedia, telematics, and infotainment technologies, which can be divided into a number of functional areas, such as vehicle integration, remote vehicle services, and near-vehicle interaction.

Integration involves better control and integration of a myriad of in-vehicle devices, including factory installed, dealer installed, after market, and third party. The advantage of Infotronics is the late binding of vehicle technology to take advantage of the most recent consumer/vehicle devices in a safe and secure manner. Remote services include Internet-based services commonly available to desktop computer users, such as e-mail, calendaring, e-commerce, and streaming media via cell-based network protocols such as CDMA, GSM, GPRS, and WCDMA. Near-vehicle interaction includes smart highways, tolls, gas pump-based services, and vehicle-to-vehicle safety systems such as collision detection and avoidance.

The typical vehicle design cycle takes a few years, and it's becoming increasingly difficult, if not impossible, to design an HMI (user interface) module that's flexible enough to address all needs, not only after the vehicle is delivered, but also at the time of design/production. In addition, the vehicle may have a few HMI devices operating simultaneously, further complicating the task of writing HMI applications. Taking into account the integration of consumer devices into the vehicle (as part of the convergence strategy being pursued by car manufacturers) makes everything even more complicated. Many consumer devices, such as PDAs, are HMI devices from the vehicle point of view.

With such a panoply of architectural diversity, creation of these systems is fraught with difficulty. Most vendors have chosen to simplify their solution offerings. Devices that perform these functions are sometimes called Telematics Control Units (TCU), and these solutions tend to fit in two groups.

The first group is a sort of auto personal computer - a "one box fits all" that has a graphical user interface and some reasonable voice recognition capability. This solution has radio/CD/tuner integration, climate control, e-mail, and navigation capabilities.

The second is a vehicle service provider model with a TCU that's securely connected to a remote service provider. The driver signs up for a service account that allows services (in the form of software packages) to be deployed to the TCU. These services include off-board navigation, driver assistance, e-mail, and media capabilities, to name a few.

The problem with both of these solutions is that they assume that all cars are the same. They tend to be expensive, so only the top models and brands integrate them. They become just another expensive toy. In order to be useful, the TCU has to be flexible for a wide variety of functionality and scalable in terms of price and performance for all brands and models.

Beginnings
Ford Research Laboratory built an innovative vehicle architecture that operates with all three classes of functionality. Building diverse types of vehicle applications across all the various Ford models and brands for different types of vehicle components and interactions is a daunting task - some would say impossible. This article focuses on one component of this architecture: the Human-Machine Interface subsystem, which is based on a novel XML markup language.

After looking at the user interaction part of this architecture, we decided that the "one box does all" solution was the source of too many problems. It was too expensive, not scalable across vehicle lines, too hard to upgrade, and so on. The driving costs of the one-box solution were the more expensive processor and memory needed to drive speech recognition and graphics, both of which are notorious memory and performance hogs.

A New Approach
With this in mind, we redrew the relationship with a minimum of two boxes, one for user interaction and the other to act as a hosting platform for applications. The hosting platform, since its requirements are usually less expensive, can be used in a much wider range of vehicle lines. The user interaction unit can then be differentiated by model and brand. In cheaper vehicles, a full user interface and a speech recognition system aren't required, or have fewer requirements, so this model can be made more inexpensively. In upper-end vehicles, this unit has the complete range of options available for refined comforts and tastes.

In order to take advantage of this user interaction unit, we posed the question, "How do you make this device agnostic to its intended use?" Building hundreds of customized user interfaces only displaces the cost and makes the software development life cycle very expensive. The answer: the user interface software on each unit needed to be independent of the functionally of its user interface. This sounds hard to achieve.

One advantage of the hosting platform was that it was designed to handle complex provisioning (or deployment and installation of software). We were able to provision almost anything and decided to provision user interfaces. The hosting platform would get a new user interface and push it to the user interaction unit. The user interaction unit would then install it, and the new functionality would be present for usage.

Another design point to the system was that we wanted the hosting platform and the user interaction unit to be extremely independent of each other in terms of OS and language. While we used Java on the hosting platform, we wanted to be able to take advantage of all the user interface platform models out there. We felt that with a greater competitive landscape, Ford would not be locked into a single proprietary solution for the user interface component.

With all these design factors in play, the choice to use XML seemed self-evident. Figure 1 depicts a process we use to create in-vehicle HMI applications using XML. A user interface (graphical, speech, haptic [touch]) is created by a designer knowing little of programming. This design will be translated into XML manually or using special automated tools. It is then deployed to the vehicle and passed to the user interaction unit. The interaction unit parses the XML document and renders a representation of this user interface into a target language model (runtime only, no compilation). The user interacts with this user interface. The renderer has a shallow functionality since the majority of the service functionality (that is updateable through provisioning service) is on the hosting platform. All interaction rules are encoded.

We named this new XML markup language VUML (vehicle user interface markup language). It's currently being defined, and plans for standardization have not yet been made. The next section describes VUML life cycles and models.

Vehicle User Interface Markup Language
VUML consists roughly of two parts:
1.   Interaction mechanism (between HMI and the hosting platform) described in very generic terms
2.   Data presentation description

The same VUML message will be used to describe the data presentation for touchscreen display or a voice-based system. The purpose of the HMI in this solution doesn't go beyond pure HMI functionality: to present the data to the user and to get input from him or her. The HMI module is unaware of the meaning of the data it's dealing with (which will be delivered using VUML). The only requirement for the HMI module in this case is that it be able to understand VUML and to implement a set of predefined user interface constructs (more on this in the "Rendering and Interaction" section).

When hosting platforms send VUML messages to the HMI module, each predefined constraint is accompanied by the name of this particular construct (naturally, there could be more than one constraint of the same type in the same HMI application). When the HMI module sends the message back to the hosting platform (as a result of the user interaction), the construct that is part of this message is again referenced by its name. This name reference is useful for the hosting platform (and HMI application running on this platform) only.

VUML has enough flexibility to describe HMI module behavior that doesn't require sending messages to the hosting platform, such as navigation between different screens.

Writing a new markup language for this purpose isn't enough; tools, deployment rules, life cycles, and rendering mechanisms are all required to satisfy the needs of the system. There are three areas associated with the architectural model. These are VUML creation, deployment, and rendering and interaction. Each area is relatively independent from the others, so independent software vendors can create commoditization in each of the areas, from VUML creation to repositories to rendering engines.

VUML creation
The choice of tools is up to the vendors. Since we're creating the first set ourselves primarily for testing, we can sketch out what a particular tool might look like.

The VUML builder is a WYSIWYG drag-and-drop editor that hides the complexity of the underlying markup detail. The design of a specific user interface starts with the selection of a specific vehicle signature, which is defined as a set of branding elements, skins, and component choices together with a set of ergonomic rules for using them. These signatures are predefined and stored in an online repository that is held by the vehicle manufacturer who's responsible for defining the signatures. The signature is downloaded in the builder on demand and cached for efficiency.

The builder allows the designer to describe the user interface along with actions that each component will perform. The designer can simulate the finished user interface to determine suitability and look and feel.

Once finished, the user interface can be deployed to the UI repository, where it's registered for use in a target vehicle.

Deployment
To support an efficient mechanism for user interface deployment across potentially millions of automobiles, it's necessary to define an Internet-based architecture founded on current best practices. Since this article does not focus explicitly on this part of the system, it is described in overview only.

VUML builders and deployments are accessed via a portal infrastructure (see Figure 2). Deployed user interfaces are registered for a specific automobile or group subject to security and applicability of the designer authority and vehicle type. The act of registration entails further checks on the user interface in order to check suitability and security. Once registered, the user interfaces are stored in a user interface repository. This serves two purposes: tracking and redistribution (in case the UI requires reloading).

Once registered and stored, the user interface is scheduled for provisioning to the target automobile(s). The provisioning server handles the downloading of the UI to the vehicle. Since the automobile is a mobile platform, it has all the problems associated with one. The automobile may or may not be in contact, and the type of network connectivity it has may or may not be suitable for immediate downloading. The provisioning server uses heuristics based on many factors, including connectivity parameters and the type of activity that the automobile is doing at the time. For example, deploying a new user interface while the automobile is in motion is generally a bad idea.

In the vehicle itself, a provisioning service handles the provisioning connection to the back-end server. This provisioning server handles several different types of provisioning entities, one of which is the VUML for a new user interface. When the provisioning service receives a new user interface in VUML, it passes this to the user interaction unit (HMI). The renderer handles the parsing and translation into a new user interface.

Rendering and interaction
The rendering engine reads the VUML and creates a specific set of user interfaces using a specific language. This is done by mapping the VUML objects directly onto the language-dependent user interface constructs. For a graphical user interface, this may include panels, forms, dialog boxes, buttons, drop-down lists, etc. For a nonvisual interface, it's not so straightforward. We made the decision to define a generic hierarchy of command and response categories and their respective actions for each type of user interface so that any particular HMI interaction or rendering could be described using the same constructs for different kinds of HMI modules. In other words, it will always be possible to match a particular construct to the relevant HMI module.

In a speech-based user interface, there are three components: a speech recognition engine; a speech synthesis engine; and a dictation engine, which captures speech and renders it to text form without interpreting for commands.

A haptic (touch and feedback) interface is more basic, with a set of feedback (similar in many ways to those found in games) directed to specific car areas such as shift stick, steering wheel, seat, and so on. These feedback sensory mechanisms accentuate the interaction with the vehicle, for example, collision avoidance feedback built into the steering wheel so that turning into another vehicle on your blind side results in a vibration, causing you to stop. In many ways this is more useful than an audio warning causing you to look around for the cause and increasing driver distraction. For this type of interface, VUML is targeted to those vehicle components and the actions they cause.

The specification for the HMI module defines the set of components that will be preinstalled in the HMI, although it isn't a static set since we generally provide mechanisms for loading new components into the HMI modules that support this feature. These components will be initialized (but not necessarily rendered) at startup time. The only performance hit is an initialization cost when the new VUML is loaded. Since this is done only once, it's possible to store multiple user interfaces and load them when appropriate circumstances permit, such as a diagnostic interface that automotive technicians can use for servicing.

The rendering engine is composed of multiple sections, a rendering kernel, and a UI component repository (see Figure 3). The kernel provides VUML parsing, component linking, and user interaction. The UI component repository is intended to be updateable, and the VUML allows alternative components to be used. For example, if a particular type of button, panel, or skin element were specified, the renderer would store the new component in the repository for use. The support of this dynamic mode feature depends on the underlying language target. In the case of Java, this is just another JAR file.

Summary
Ford Research Laboratory's innovative vehicle architecture, specifically the Human-Machine Interface subsystem, uses VUML, a novel XML markup language. VUML consists of an interaction mechanism and data presentation description. Associated with the architecture are creation, deployment, and rendering and interaction, which are relatively independent of one another, allowing ISVs to create commoditization in each.