Blog Posts

Autonomous vision chip for IoT applications

    By Samer Ismail

    Organizations around the world are increasingly adopting advanced technologies, driving the internet of things (IoT) market. According to a Fortune Business Insight report, the global IoT market was valued at $190 Billion USD in 2018 and is projected to reach 1,111 Billion USD by 2026.

    The Internet of Things (IoT) facilitates the interchange of information between machines and devices and can include components like sensors and meters, network connectivity devices and software.

    As part of our focus on the IoT market, we have introduced Heimdal 2, a second-generation vision chip. Characterized according to EMVA1288, the standard for Machine Vision Sensors and Cameras, Heimdal 2 enables on-chip image processing with configurable algorithms to interpret images and is suitable for multiple applications such as the detection of objects, intelligent interpretation of movement and many more.

    Heimdal 2 is a mature chip that incorporates performance improvements the 1st-generation chip and offers low power consumption and low production cost.

    Low power consumption in miniature size

    Compared to typical image sensors, which are equipped with many megapixels and capture detailed images, Heimdal 2 uses low image resolution of 64×64 pixels. The goal is to achieve a high-level understanding of visual content by interpreting low-resolution images quickly, while consuming minimum energy.

    The low number of pixels reduces both processing time and power consumption when the algorithm processes captured images. In addition, less physical space is consumed since both RAM and the image sensor take up less space.

    Heimdal 2 can be combined with DELTA’s energy-harvesting technology, creating a single autonomous ASIC that can view, analyse and act on the outside world, while obtaining its own energy from the surroundings, without the need for a battery. Together, the low power consumption, short processing times and a silicon area of just 9mm2 make Heimdal 2 attractive for use in IoT products.

    PICTURE: Figure 1 – Heimdal 2 encased in a 40 pin QFN package measuring 5mmx5mm. Optics are mounted on the package.

    Vision chip in standard CMOS process

    The Heimdal 2 vision chip is manufactured using a standard CMOS process, offering both price and functionality advantages.

    Because a standard CMOS process does not offer access to photodiode IPs with enough performance, we have developed our own photodiodes in a standard CMOS process, which we continuously optimized.

    The imaging function implements algorithms tailored to different applications. The algorithms take an image or sequence of images as input and analyse these to then provide meaningful high-level output. The algorithms are implemented in software and run on a 16-bit processor. This makes it flexible to implement new algorithms and update existing ones. The image sensor, in combination with the algorithms, forms an autonomous system, which is useful for indoor / outdoor navigation, robotic technology and intelligent monitoring. For demonstration purposes, we offer two vision algorithms: one detects motion in images and provides the direction vector for the effective motion in the image; the second type of algorithm detects the number of objects in the image and follows the objects from image to image.

    Use cases

    Finding a vacant parking space can be difficult and time consuming for many drivers especially in densely populated areas. To aid drivers in finding a vacant parking space a system based on a vision chip and a mobile app can be used. A solar cell powered vision module based on Heimdal 2 could be distributed around the city to cover all parking spaces. Each vision system would then continuously monitor the target parking space. When the parking space has changed status either from occupied or not occupied the vision system would then wake up a wireless module to transmit the changed status to a centralized server. The drivers would then be able to navigate to nearest vacant parking space utilizing a mobile app which is connected to the server.

    Another application could be gathering information on shopper behaviour in the retails industry. Vision chips based on Heimdal 2 would be placed in the ceiling of a mart. The vision chip would then detect the motions vector of the shoppers. Motion vectors from each vision chip would then be transmitted to a server for further data processing. The server would then be able to analyse shopper behaviour patterns, i.e., which product they looked at the most, how long they spent looking at each product type and the path shoppers follow when they enter the mart. The data could then be used to analyse behaviour and optimize the layout and organization of products in the store.

    Related Projects

    This is a unique website which will require a more modern browser to work! Please upgrade today!