A lesson learned in measuring microenterprise

Measuring microenterprise profits is hard. Most owners of these small firms keep no records and have very volatile incomes.  Asking firm owners what their profits are can lead to high refusal rates, noisy data, trouble with recall, and then there is the added concern that reporting may change with interventions  (either because an intervention like business training changes recording, or because people who get given finance may want you to think it has benefited them). Yet profits are the most important outcome for judging the success of many firm-level interventions, so we need to figure out better ways to measure it.

The promise of RFID

Radio frequency identification (RFID) tags in principle offer a technological solution to this problem. An RFID tag has an embedded microchip which allows it to store data, and an embedded antenna to transmit this information. When a passive RFID tag gets within range of a reader, the radio-frequency energy transmitted by the reader runs the circuitry on the tag and reflects the signal back to the reader.

RFID tags are increasingly being used in large U.S. retailers like Kohls, Walmart, LLBean, and Best Buy for inventory management. In principle one can apply the tags to new stock as it comes in, and then use a reader to measure stock levels at any point in time without having to physically scan items one by one as would be the case with bar codes. Measuring the flow of stock coupled with price data then can provide data on sales, which could then in turn be coupled with unit cost or mark-up data to provide a measure of profits.

The tags cost 22 cents each, and the fixed costs of hardware (printer and reader) and software was $8600. So certainly way beyond the cost of being cost-effective for a microenterprise to use by itself to track inventory. But not so expensive relative to the cost of surveys to prevent it potentially being used by researchers to help understand the stock flow and profitability of such small firms.

A proof of concept trial in Sri Lanka

A new working paper (joint with Suresh de Mel, Dammika Herath and Yuvraj Pathak) details our attempts to test out this technology. We implement this process in 22 microenterprises in Kandy, Sri Lanka. We accompany the tagging of inventories with physical stock-takes, and with survey elicitation of inventory levels from the firm owners. This enables comparison of the accuracy of RFID reads compared to survey responses. In addition, we tested the accuracy of RFID tagging on a larger range of products in our field office in order to provide evidence on which types of products this technology works for best.

Here’s a short video showing the process.

We have three main results:

1. Available off-the-shelf technology is more difficult to use and more cumbersome than we had envisaged, and than is suggested by media accounts of the spreading use of this technology

Our (naïve) prior based on online descriptions of printers as easy to use, and of standard desktop printers, was that this should be simple plug-and-play technology that could be easily set up within a day or two. In addition, we were under the impression that since the RFID reader did not have to physically scan a barcode item by item, it would be able to quickly scan the entire tagged inventory.

In practice the process turned out to be much more difficult to set up and more time consuming to employ than expected. The printer is big and heavy (shipping weight 49 pounds) which caused it to be held up for more than 2 weeks in customs. The set-up process required considerable trial and error to correctly calibrate the printer to correctly print the tags, to figure out which memory bank on the RFID tag to store the product information on, and to configure the software correctly for both printing and reading purposes. In addition to being more time-consuming to set-up than anticipated, the time taken to read the scanned product information was much longer than expected. When the reader scans the tags, it searches through a picklist to find each one, looping through each time. It took one to two hours to generate a new picklist each evening to use the next day, and then averaged 10-15 minutes to scan the selected 280 or so items in a firm, and another 15 minutes to process the tags by finding matches against the database on the reader

2. It is possible to get firms to use the technology, and it works reasonably well for paper products

Microenterprise owners were told that the purpose of the study was to test the feasibility of a new technology for helping monitor stock, and were offered 5000 LKR ($38) to compensate them for their time and cooperation in the study. Firms were willing to participate under this incentive, although 2 out of 24 dropped out after an initial test, and 2 more dropped out for health reasons partway through.

Just printing tags and reading them in the office, without attaching them to anything, we did two trials: in the first we printed 70 tags and could read 69 of them; in the second we printed 200 tags and read 198 of them.

When we attached the tags to stationary items, we could read approximately 90 percent of the tagged items, with items being harder to read when stacked on top of one another.

3. Mostly though the read efficiency is terrible

In total out of 4,773 tagged items physically counted in the stores by our field teams, the RFID reader was only able to read 1,210 items, or 25.4 percent. Moreover, there is considerable variation in the read efficiency for the same product over different days, and for the same type of product over different brand/product size/store combinations. For example, one type of soap bars(sunlight soap small) had 52.5 percent average read efficiency, although with a standard deviation of 18 percent across days. Moreover, five other types of soap bars had read efficiencies below 10 percent. This means you can’t simply scale up by some multiple to get the true level.

Why didn’t it work?

The three main issues are liquids, metals, and tag shadowing. Materials containing a large amount of water absorb radio-frequency energy, so that the tag fails to receive enough energy to reflect back a strong signal. Metal can reflect energy away from a tag, or reflect the tag’s signal away from the reader. Finally, if items are stacked so that tags are lined up behind one another, the first tag can capture the reader’s energy, shadowing the tags on items behind it. The result can be that the first item is read but those behind it are not. We had avoided tagging items like coke cans and bottled water for the first two reasons, but anyone who has stepped foot in a microenterprise knows that stacked items are the reality, and individually pulling out each item to scan one at a time would amount to a stock-take and defeat the purpose.

A silver lining?

So this technology is currently not ready for use. But a silver lining is that in addition to the RFID reads and physical stock counts, we also asked survey recall questions. Simple survey questions asking microenterprise owners how much they have of each item seem to do very well on average at matching the amount measured by physical stock counts. So while the quest continues for a better way of capturing firm profits, at least our survey measures seem not so bad after all.

Published in collaboration with The World Bank.

Author: David McKenzie is a Lead Economist in the World Bank’s Development Research Group, Finance and Private Sector Development Unit.

Image: RFID tags on products. World Bank.