FPGA system for image acquisition and pixel classification

Problem Statement:

Resonon, Inc. sells machine vision systems that utilize imaging spectrometers (aka, hyperspectral imagers). In brief, the imaging spectrometer collects spectral images that provide far more color (spectral) data per pixel than conventional color cameras. This data is transferred to a computer, which also obtains input from a belt encoder. The data are analyzed to classify each pixel in the image (e.g., “good product” or “defect”) and a classification map is generated. Input from the encoder is used so objects in the image can be located by the actuator, such as a robot.

Currently, Resonon utilizes a conventional computer for data acquisition and data analysis. We would be interested in exploring the possibility of using an FPGA. The first step would likely be to capture the data from a high-speed camera. (At least 100 frames/sec.) A second, and probably more complicated, phase would be to classify the pixels. This would require incorporating an appropriate machine vision algorithm and also developing an interface to enable training.

See Solution

  • Cyclone V

  • Advisor: Dr. Ross Snider

  • Advisor: Dr. Ross Snider

  • FPGA System.

  • Altera Architecture with ARM processor.

Hyperspectral Imaging

Hyperspectral imaging, like other spectral imaging, collects and processes information from across the electromagnetic spectrum. Such as the human eye sees visible light in three bands (red, green, and blue), spectral imaging divides the spectrum into many more bands. This technique of dividing images into bands can be extended beyond the visible. [wiki]

Hyperspectral Imaging Solutions

1. Hyper-spectral Camera
2. GigE – Camera can output only with gigabit Ethernet. Gigabit Ethernet can handle up to 125MB/s of data. Our camera can only output up to ~50MB/s.
3. FPGA – Data mover (in our system) and manipulator (in the future system)
4. Display – HDMI input display monitor

SoC toolkit integration for REAL Time Operation

Current system uses a Computer – Very hard to make Real Time. New system will use an FPGA, which will be able to operate Real Time. Moreover, Computers lag and things get delayed. Here, FPGAs have fixed latencies so processing time is guaranteed. The new system will have dedicated logic such that the data flow rate throughout the system will be constant. We can guarantee that our robot will get commands within a set period of time so that it will eject the right almonds.