Sensor data fusion enhances gaming, navigation, and VR experiences

Article By : Kaustubh Gandhi & Amithash Kankanallu Jagadish

Effective design techniques and software solutions are required for obtaining reliable sensor data in today's electronic consumer devices.

Whether it be smartphones, wearables, virtual reality headsets, or even robotic vacuum cleaners, today’s users expect and demand such devices to consistently behave as commanded, to smoothly and accurately adapt to their changing surrounding environments. This requires precise sensing of pitch, roll, and heading, which is formulated inside devices by fusing data collected from accelerometers, gyroscopes, and magnetometers.

As is usually the case, in the real world things are never as simple as they seem. For example, accurately determining the heading (observation) direction is a major challenge as magnetometer measurements are negatively affected by multiple objects in their vicinity. These undesirable magnetic influences, commonly known as hard and soft iron distortions, can be caused by various elements located both within the device itself and external magnetic objects in the user’s immediate environment.

This article aims to provide deeper insight and understanding into effective design techniques and software solutions required for obtaining reliable sensor data in today’s electronic consumer devices, and to improve user satisfaction with the final product. It will provide examples of powerful sensor data fusion techniques such as the utilization of estimated magnetometer offsets based on gyroscope signals obtained during standard use and the impact of this on user-relevant features such as pedestrian and head tracking.

The magnetic challenge

Have you ever taken the wrong exit from a roundabout because of wrong directions given by a smartphone navigation app? Have you ever experienced a sudden nauseating bout of motion sickness when using a virtual reality headset? Or has your ‘smart’ robotic vacuum cleaner been repeatedly getting stuck in corners? Most of these issues are, at least partially, the result of incorrect heading information derived from imprecise inertial sensor data fusion. So, why do highly precise state-of-the-art sensors still register inaccurate information, and with such wide deviations?

Outside the laboratory, the rigidly straight lines of the Earth’s supposedly constant magnetic field are constantly being modified by various objects such as door frames, tables, chairs, and other metallic items. Based on their specific magnetic characteristics, these objects alter their surrounding magnetic field through phenomena known as hard iron and soft iron distortions.

Figure 1 Sources of compass errors: external magnetic fields

Hard magnetic materials (“hard iron”) such as NdFeB, AlNiCo induce a high remnant B-field or “magnetic memory,” whereas soft magnetic materials (“soft iron”) are typically materials such as Iron (Fe), Nickel (Ni), and their respective alloys.

When magnetometers are used in devices, hard iron distortions are created by objects that generate a magnetic field, e.g. magnets inside a speaker, resulting in a bias known as a ‘constant offset’ in the sensor output, which then needs to be compensated for. On the other hand, soft iron distortions are created by objects that ‘passively’ affect or distort their surrounding magnetic field but do not necessarily generate a magnetic field themselves, e.g. memory card slots, batteries, wireless antennae, door and window frames and various other standard objects in the ambient environment. This type of distortion alters the actual shape of the magnetic sphere and is largely dependent upon the orientation of the material relative to the sensor and the magnetic field.

As shown in Figure 2, in a typical indoor area, due to magnetic field distortions caused by the presence of common objects, the compass heading varies significantly, i.e. the red ‘north’ needle of the compass points wildly in all directions.

Figure 2 Variations in sensor readings (magnetometer) in a typical indoor area

Thus, compensating for both hard- and soft-iron distortions is critically important for achieving meaningful magnetometer readings. This compensation requires a sophisticated procedure during the design of the device and the incorporation of the outcome into the sensor’s software during actual use, as further described in this article.

Embracing distortions

The following systematic approaches are used to compensate for distortions affecting magnetometer readings:

  • In-design compensation using a soft iron matrix
  • In-use calibration software with standard ‘figure-of-eight movements’
  • Smart calibration software with ‘natural-use movements’

In-design compensation using a soft iron matrix

Soft iron distortions coming from components located inside an end-device, such as a smartphone, are constant, and hence can be compensated for by utilizing a one-off solution. Such compensation requires a so-called “Soft Iron Compensation Matrix” (SIC Matrix), where the designer has a wider range of placement options within the device. These compensated sensor readings have a substantially higher accuracy, i.e. ±2° compared to uncompensated readings where the error range can easily reach ±10°. Calibration is performed using a 3D coil system (Helmholtz coils), which consists of two solenoid electromagnets aligned on the same axis, which cancel out these undesirable external magnetic fields to provide a “clean” magnetic environment. The device with inertial sensors is placed into this clean environment and measurements are taken to create a raw data log for the magnetometer, which is then subsequently fed into a data driven tool, to generate a SIC Matrix. This SIC matrix is then incorporated into the software driver and permanently compensates for the in-device soft iron distortions affecting magnetometer data.

Since this method estimates soft iron effects under laboratory conditions, naturally, after market changes and effects of add-ons cannot be compensated for. Nevertheless, this is a very effective technique for in-device component calibration and is highly recommended during the design phase in conjunction with experts from sensor manufacturers who can accurately generate SIC matrices and apply them.

Figure 3 3D (Helmholtz) coils for in-device magnetometer calibration

Unfortunately, it is often the case that laboratory calibration results simply do not work accurately when applied to an actual PCB, where areas known as ‘forbidden zones’ are created, rendering such devices so inaccurate as to make them practically unusable.

Bosch Sensortec’s 3D soft iron compensation technique reduces this ‘forbidden zone’ substantially. For example, when distortion of sensor data was measured just 9 mm from an NFC antenna, before compensation, the maximum heading error was 8°; whereas, after compensation, the maximum error at all altitudes was just 1.5°.

Figure 4 Magnetic sphere without soft iron compensation

Figure 5 Magnetic sphere with soft iron compensation

[Continue reading on EDN US: In-use calibration using figure-of-eight movements]

Kaustubh Gandhi is Senior Product Manager of Software at Bosch Sensortec.

Amithash Kankanallu Jagadish develops motion sensor algorithms at Bosch Sensortec GmbH.

Leave a comment