Farming is difficult and time intensive. Between preparing a field for seeding and harvesting an entire crop, there is a large amount of work that must occur. With a decreasing number of workers available to do this work, many of the technological innovations within the agriculture industry today focus on increasing the efficiency of farming both to reduce reliance on workers and to increase profits and production.

One such technology is swath control systems, which give real-time data and insights to drivers of large farming machinery as they follow swaths – the strips of cut crop made by either people with scythes or mowing-machines. These sophisticated systems let farmers improve the distribution of plant-growth products and better find the locations of harvested or leftover crops.

Swath Detection Technologies Compared

The key capability that enables automated swath detection and following is the 3D sensing system, which provides realtime 3D “maps” of the environment in front of the farm equipment. Generating an accurate, realtime 3D model of the vehicle’s surroundings is complicated and there are several technologies that can be used for this purpose. Most prevalent is lidar – a sensor that bounces a laser off the environment and measures the time photons take to bounce off structures and return to the sensor. The second is monocular camera systems, which use AI to approximate the distance to known structures. And last is stereo vision sensors, which use camera pairs to accurately measure distance. All three modalities have strengths and weaknesses, but one stands out as superior across all metrics.

First, let’s look at lidar. Relatively high cost and low resolution, lidar laser scanners produce 3D maps of their surroundings by scanning the scene with a laser. These sensors are precise and reliable but suffer from high price and poor performance in the dusty conditions typical to a farm. Most lidars also have spinning mechanical components which have short lifetimes and need to be replaced after roughly 7,000 hours (~365 active days). At $2,500-$7,000/lidar, replacement costs can add up.

Another alternative 3D sensor is camera-based “monocular” imaging. This technique uses a single camera backed by a software-based neural network running on a processor that analyzes each frame produced by the camera. The AI (neural network) has been trained to recognize known structures, such as a car, human, tractor or cow. These types of sensors are effective for simple scenes with known objects and/or scenes with significant relative movement, such as with cars on a highway. This method does not work well with uneven or featureless terrain and unknown objects, so is not well suited for autonomous farming.

Other common sensing methods – GPS and radar – do not have the fine detail, granularity, or precision required in swath management.

As compared with lidar and monocular camera sensors, stereo vision sensors do have the right attributes to enable automated swath guidance.Stereo vision is a camera-based technology that compares the images of two cameras to accurately measure distance, similar to the way human eyes work. Stereo vision sensors have the resolution, accuracy, reliability, and cost that makes them an ideal source of 3D data for automated swath following systems.

The Rise of Stereo Vision Technology

The use of two cameras to measure the distance to points in a scene provides accurate depth perception and high-definition, 3D maps that can be used for automated navigation of farming equipment. These systems have been shown to measure objects as far away as 1 kilometer and offer robust performance at night and in bad weather.

Of course, stereo vision has existed since the late 1800s. But these sensors have always been limited in use due to their sensitivity to issues of calibration. Vibration and shock can disrupt the measurements taken by a stereo vision sensor, and therefore the cameras have historically been rigidly bound together by a stiff metallic beam, limiting their range, accuracy, and mounting options. As such, the use of stereo vision in environments with a lot of vibration, like on heavy machinery or bumpy roads, has been limited, until recently.

To operate in rough environments such as farming, stereo vision cameras must be calibrated many times per second to ensure their relative alignment and that all measurements are accurate.

With the development of high speed GPUs (graphics processing units) and improvements in camera technology, “online” calibration has been made possible. Online calibration enables the realtime calibration of two cameras in software, such that the cameras can now be mounted independently and yet remain well-calibrated, even in the face of significant vibrations and shocks. This new technology, in turn, has enabled a next generation of stereo vision sensors – sensors that can handle the vibration typical on a farm and that can produce ultra high-resolution 3D imagery that far surpases that which is available from lidars and monocular camera systems.

These new stereo vision sensors can detect an animal running across the field, a human lying down, or accurately detect and follow a 10cm swath from up to 50 meters away, enabling a new generation of automated farming equipment.

The tech world likes to say that “software is eating the world.” The agriculture market is poised to experience the benefits of this worldwide trend towards software. With farmers needing to do more with fewer resources – especially as the climate changes – technologies like advanced stereo vision are enabling increasing levels of automation on the farm, increasing both efficiency and productivity.

by Brad Rosen | Co-Founder and COO of Nodar