Low cost JPEG Image Displayer on STM32

As digital cameras and cell phones with high-pixel cameras are getting more popular day by day, people tend to store their digital photos in storage media and avoid making any prints. A Digital Photo Frame is a perfect solution to have a glimpse of your digital images in a lively way. Bundling an image displaying solution with the STM32 shows the capability of the MCU in the world of multimedia in an easily accessible way.

Digital photo frames based on STM32F103RBT6.

It is evident from statistics that in spite of the large consumption of digital cameras and similar devices, only a few percentages of the pictures are actually printed. Digital Photo Frames are eco-friendly; there is no need to make prints of images to put in frames. Digital Photo Frames looks like normal photo frames, which can be hung on the walls or set on the tables, shelves etc. They display digital images one by one in a slideshow manner.

The digital images captured using digital cameras and cell phones are mainly in JPEG format. JPEG images are compressed format of image data that acquire minimal space in storage media. To make these images available for display on a TFT screen, they need to be decompressed into raw RGB data. The core of a digital photo frame is a microcontroller that reads the memory card for the JPEG images, decompresses and resizes them, and displays them on the screen. Added features require additional firmware or hardware implementations. These are generally based on a performance rich microcontroller with enough computing power to do all the jobs.

In a low cost digital photo frame application, an ARM Cortex-M3 core based STMicroelectronics’ microcontroller, namely the STM32 family microcontroller, can be used. The STM32 is the first MCU family combining 32-bit performance and features with the integration and end-user cost of today’s 16-bit MCU.

Building Blocks and Explanation

Block diagram.

The above block diagram is a proven prototype based on the STM32 with the following building blocks:

Hardware:
1. Microcontroller (STM32)
2. TFT (AM-240320L8TNQW00H)
3. Storage media (Micro SD card)
4. MEMS (LIS331DLH)
5. Temperature Sensor (STLM75)
6. STouch (STMPE1208SQTR)
7. ZigBee Module (SPZB260)
8. Battery Charger (STW4102 & L6920D)
9. USB Massstorage

Firmware:
1. JPEG decoder
2. File system (FAT like) library
3. Others

Digital photo frame based on STM32F103VET6.

STM32 Microcontroller
The STM32F103VET6 is a 32bit MCU based on the popular ARM 32-bit Cortex-M3 CPU, running at 72MHz with a performance of 90 DMIPS with 1.25 DMIPS/MHz. The memories embedded in it are up to 64kB of SRAM and 512kB of flash memory. The microcontroller has single-cycle multiplication and hardware divisions. It has up to 112 fast general purpose IOs to enhance the overall performance. The IOs are 5v tolerant.

The microcontroller has up to 13 communication interfaces, which include two I2C (400kHz), five USARTs (4.5Mbps), three SPIs (18MHz), CAN 2.0B Active interface, USB 2.0 (12Mbps) full speed interface, SDIO interface, and FSMC interface. Among these, this application is using the two SPIs, two I2C, SDIO, FSMC, and USB interfaces for this application.

The microcontroller works on single voltage from 2V to 3.6V, unlike several microcontrollers requiring dual voltage.

Many features of the microcontroller help in reducing the overall cost of the system. For example, inbuilt RTC can be used for implementation of a calendar.

The clock scheme of STM32 is very flexible. The factory trimmed internal RC oscillator is 8MHz ±1 percent over. So, if communication using critical peripherals is not used, then the whole system can actually work on internal frequency of 64MHz generated by PLL.

To implement DPF, the firmware uses a file system library (FAT16/32) on the STM32. The FAT library manages the file system in the SD/MMC card where the images are stored. The file system reads the files from the memory card one by one.

The TFT driver firmware library will write the picture file from the MMC/SD card in the TFT RAM to display it on the TFT screen.

TFT
The TFT is interfaced with the microcontroller through FSMC. FSMC is a parallel mode of interface, using which, image data dumping in the TFT RAM is faster.

SD/MMC Card
SD/MMC cards are very common these days in the field of mass storage media. They are commonly found in maximum digital cameras and cell phones. SD cards are found in different form factors. As per their dimensions, they are called SD, mini-SD, or micro-SD cards.

In this application, we are using SD (SD/Micro-SD) cards for storing JPEG images. The file system library handles the FAT table of the card and will read image files from the cards.

An SD card can store images depending on its memory capacity. For instance, a 64MB card can store up to 200 JPEG images of 2 megapixel (1600x1200) resolution.

The SD card is connected using The SDIO interface available with the STM32F103VET6. The SDIO is a dedicated IP for interfacing SD cards. EMIF06-MSD02N16 has been mounted in-between the SD card and MCU as an ESD protection device.

MEMS (LIS331DLH)
The LIS331DLH is a motion sensor device. A MEMS sensor is used in this system for add-on feature like auto-orientation of the image as per the frame alignments such as landscape or portrait. The MEMS uses SPI to interface with STM32.

Temperature Sensor (STLM75)
The STLM75, a temperature sensor, has been included in the application to show the room temperature on the TFT screen. An I2C interface has been used to connect STLM75 with the microcontroller.

STouch (STMPE1208SQTR)
The STMPE1208SQTR is a touch sensing device, which is used in this demo as a user interface for navigation through the menu, setting the date, and time, etc. Twelve touch keys have been used for this purpose. An I2C interface connects the STMPE1208SQTR with the STM32.

ZigBee Module (SPZB260)
The SPZB260 is a ZigBee module. It is a low-speed wireless media. Its functionality in this demo is application based. For example, the DPF can receive images wirelessly from a surveillance camera in a continuous manner and display it on the TFT screen. In this demo, the SPZB260 is interfaced through the SPI.

Battery Charger (STW4102 & L6920D)
To play the DPF demo as a stand alone application, it is necessary to run the demo with rechargeable batteries. For this, an I2C interfaced battery charger device, STW4102, and charge pump, L6920D, have been used.

USB Mass-storage
Using USB mass-storage, a user can access the on-board SD card through a PC for copying images into the card, as well as reading back the same. To connect the board with a USB cable, a USB connector is available on the board. An ESD protection device, the USBLC6-2P6, has been placed on the board to protect the device.

JPEG Decoder
JPEG images are in compressed format of image data. The advantage of this compressed format of image data is that it takes less space in a mass-storage device. Hence it helps the user to store a larger number of images in the same memory space.

The JPEG decoder library plays a vital role in DPF application. To display an image on TFT, the image needs to be in raw RGB format. So here the JPEG decoder comes into the picture. It decompresses the compressed image data into raw RGB.

Firmware architecture layer.

File System Library
The file system library handles the files in the SD/MMC card managed with FAT16/32 firmware library. It reads the image files stored in the card. There is some freeware for this file system on the web and also some licensed versions available from other companies.

The images are stored in the SD card using the file system mechanism. To retrieve these images, the MCU has to read this file system stored in the SD card using the file system library embedded in it.

Other firmware parts include drivers for TFT, MEMS etc. Also included is the firmware for interfacing the STouch device, a RTC for date and time, temperature sensor, menu navigation etc.

How the System Works
A JPEG image displayer is a gadget that will display images one by one in a slideshow manner. Images will change automatically with a regular interval, fixed by the user.

Decompression

The microcontroller retrieves the JPEG image data from the SD card using the file system library. The JPEG decoder library processes the image data. It decompresses the JPEG image data (Y, Cr, and Cb) into raw RGB data. One byte each from R, G, and B make a pixel. One pixel is normally of three bytes. But for certain TFTs, it can be of two bytes. In that case R, G , and B contain 5, 6, and 5 bits respectively.

Example of a JPEG image containing MCUs.

Different types of MCUs.

JPEG images are comprised of MCUs (Minimum Coded Units). MCUs are different types depending on the aspect ratio of the image resolution. For example, MCUs can be 8x8, 8x16, 16x8, or 16x16 pixels.

The MCUs of an image are decoded one by one into RGB. After one line of MCUs is decoded, it is then time to resize them to fit the TFT resolution. For resizing the image, a pixel dropping method is adopted. Pixels are skipped as per the down sampling factor.

Figure 8 and 9 cite an example of resizing of a VGA (640x480) image to fit on to a QVGA (320x240) TFT screen. Figure 10 is a 16 x 16 pixel MCU, where the cross marked pixels are to be skipped. Selected pixels are to be stored in the temporary buffer before displaying on the TFT.

Image and TFT resolution.

Pixel dropping in an MCU for resizing of the image.

After decompression and resizing of one MCU line of image, the image data are written in the RAM of the TFT. It makes one MCU line of the image to be displayed on TFT. The same process is repeated for each line of the image. After one image is displayed, using the file system library, the microcontroller retrieves the image data for next JPEG file, and the same procedure is repeated for each image one by one.

Image decompression and displaying principle.

Flow Chart
Basic Flow

JPEG decompression, Resizing and Display on TFT Flow

Conclusion
In this fast changing electronic world, a Digital photo frame is a gadget that everyone will want to replace their existing photo frames that display only one photo for years.


For more information, visit www.st.com/stm32