A Closer Look At Multi-code Barcode Reading With Mobile Devices

There is no denying the productivity gains in both speed and accuracy barcode scanning can bring to many workflows. Virtually every retail outlet from grocers to sporting goods benefit from scannable product barcodes for not just point of sale, but also for inventory management and replenishment. Distribution centers, warehouses, event tickets, airlines... the list goes on and on for the industries and applications for barcodes.

Seeking even greater productivity gains, organizations now look to read multiple barcodes in a single scan. For applications like a pallet label, which may have half a dozen or so related GS1 barcodes, this is an obvious extension. Scanning a multi-barcode product label for inventory receiving is another common example. But as mobile device solutions have become more powerful, with better cameras and faster processors, users have been looking for even more advanced multi-code scanning uses.

In one such case, we recently had a customer approach us with what they perceived as a compelling use case for multi-code barcode reading. In their case, as items were used from a supply cabinet, product labels were placed on a reorder form; each label had two barcodes identifying the product; each page could have anywhere from one to 15 labels, placed in one or two columns. They wanted to scan each page of barcodes using a mobile device, just hold the mobile phone above the page and scan the entire page.

They showed us a slick demonstration from a competitive product which would highlight the barcodes relatively quickly, then return a list of results. Their first attempt with this demo involved trying to hold the mobile phone above the piece of paper to capture the entire page at once; but this didn’t work very well as the barcodes were relatively small, some of which were of questionable quality and the software couldn’t always read all of them at once from a distance.

Their second attempt then was to use a “batch mode” where you held the mobile phone closer to the codes, then moved over the page in more of a sweeping fashion and it collected the barcodes as you went. Since the mobile phone was being held closer, this method performed better. Just two problems: the barcodes were not returned in the same order they appeared on the piece of paper (which was critical to their needs) and sometimes some of the codes were mysteriously missing from the results. All we had to do was provide a solution that did the same but returned all the codes and in the correct order. Simple, right? Well, no.

How multi-code barcode reading works

Cognex has been providing barcode scanning systems for decades with fixed-mount systems, handheld scanners, and more recently, mobile barcode terminals, mobile phones, and tablets. Multi-code barcode reading is common in fixed-mount applications, but less common with handheld and mobile devices. The main reason is image stability: in fixed-mount applications (where a barcode scanning system is mounted in a fixed position) the camera is not moving, is in a known orientation, and is tuned for the optimal scan distance and field of view for the items being scanned. Further, illumination and motion are typically well controlled or understood; allowing for even better tuning of the scanning performance. While these characteristics of a fixed-mount scanning implementation make for a more controlled environment for multi-code scanning, it is still by no means an easy task.

In an ideal world, an image scanner would be able to take a single snapshot of the field of barcodes and be able to locate and decode every one; but this is rarely the case—not every barcode is going to be locatable nor decodable due to variations in focus, lighting, skew, etc. Even in fixed-mount applications this virtually never happens. Instead, multi-code decoding is achieved by examining many individual images (frames) from a live video stream coming from the camera, then combining the results from this series of scans. In most applications, this means merging the decoded results from 10, 20, 30 or even more frames, as no single frame is likely to have every single barcode in it.

As multiple frames are analyzed and the results combined, the decoding algorithms must be sophisticated enough to eliminate duplicates, but without filtering out valid data. Consider the following simple example of four barcodes:

When our user scans these codes with a multi-code scanner, they are expecting four results: { A, B, C, A}. Let’s assume that when we do scan these codes, due to variations in the focus and lighting, we are required to scan two images (frames) to find all of the codes. On the first frame, we decode the top two barcodes, { A, B }:

While on the second frame, we decode the bottom three { B, C, A }:

Each frame represents a set of barcodes, but we can’t just do a simple union of these two sets: otherwise our results would be { A, B, C }; we’ve missed the second occurrence of A! Thus, merging results from multiple frames needs to be much more sophisticated than just eliminating duplicate barcode values.

Adding in a little (or a lot) of motion

It may seem that an obvious solution to merging the results from multiple frames would be to include position information as well as the barcode value in filtering duplicates; after all, it’s plain to see in our example that the second occurrence of BARCODE A is positioned distinctly from the first occurrence found in the first frame. Indeed, there are some multi-code implementations that do just that to determine if a code is a duplicate or not. However, this relies on the fact that the relative positions of the barcodes don’t change, or at least change very little, from one image frame to the next.

But again, consider our use case of a mobile device, where the typical scanning experience involves providing a live, on-screen preview from the camera so the user can position (aim) the mobile device over the barcodes to be scanned. As the user positions the mobile phone, a lot of frames are being captured and decoded, with the phone moving the entire time even in the steadiest of hands.

The example above is a simple one, where the user can likely position the camera to capture all four barcodes at once. What if there are too many codes though (like our customer’s example; an entire page) and now the user is “sweeping” the camera across the barcodes to scan them all? In other words, moving the mobile device is required to capture all the codes.

Handling motion with a mobile scanner can be very challenging since the speed and direction of motion are generally not known to the scanner—but worse, they will almost certainly vary as the user continues to hold and move the mobile device.

If only multi-code scanning were that simple

Yet another complexity to multi-code scanning is not just finding all the barcodes, but determining when to stop looking for more barcodes. In our simple example above, the number of barcodes to scan is known. The scanner can be configured to explicitly look for four barcodes before returning results.

But what about our customer’s use case where the page may have anywhere from two to 30 barcodes? If we have a page with some unknown number of barcodes and some are missed or damaged, how would a multi-code system inform the user? How would the multi-code scanner even know that all the codes were not read? By far, this is the greatest pitfall to a poorly thought out multi-code scanning solution: missed codes. If not properly handled, missed codes can easily eliminate the productivity gains using a multi-code scanning approach was intended to realize.

Don’t treat the demo as the end-all and be-all

In our customer’s instance, the demo they were attempting to implement was returning unreliable results, likely because:

• The barcodes they were attempting to scan had duplicate codes that were valid data—but they were being filtered out as duplicate reads of the same barcode by the scanning software.
• The codes could not be returned in the precise order they appeared on the page because the scanner software had no accurate way to gauge the direction or orientation of the scan as the user moved the phone over the codes and they were acquired. Since the scanning software doesn’t know where the “top” of the page is, the barcodes were being returned in scan order, not the physical order on the page.

But even if these two issues were addressed, this would still be a poor workflow when an unreadable barcode is encountered—there would just be no way for the user of such a system to know which code was missed and had to be handled manually.

In the end, we still recommended a multi-code solution: but one where the user scanned each 2 code label individually, in progression: that way the application could verify each label/scan as the user moved along, providing positive feedback (and negative feedback if there was an issue). The resulting application wasn’t as desirable as the original demo seemed to be, but it provided a solid and usable workflow for their users.

With their high-resolution cameras and powerful processors, mobile phones and tablets are becoming increasingly capable of performing complex barcode scanning tasks, including multi-code reading. But don’t assume that attempting to scan an entire sheet of barcodes or a whole shelf of products all at one time is the best workflow. These scenarios can make for some sizzling demos because they are typically performed under ideal conditions, but you’ll want to be certain when you design a multi-code workflow that you consider all the variables that can impact its effectiveness. To learn about the mobile terminals and other mobile solutions available from Cognex, download the Mobile Solutions Product Guide.