AHRA

According to the American Cancer Society, early research suggests that CAD systems help radiologists diagnose more earlystage cancers than mammograms alone.¹ But some doctors disagree about the accuracy of identifying cancers with CAD software. Some feel that CAD devices are not as effective as simply having a second radiologist review films or digital images, due to the occurrence of false-positive identification of benign breast changes, deemed suspicious as cancer.

 

False-positive readings result in increased demands on medical systems, undue patient anxiety, and distress. Additional medical costs are incurred due to follow-up examinations, such as repeated biopsies and recalls for screening examinations. The extra costs (monetary and emotional) of false-positive mammograms, especially in women under 50, are an often neglected, but substantial problem. The emotional stress of false-positives on women is gaining prominence with radiologists and the CAD community.

 

One of the biggest problems facing developers of CAD systems, as well as radiologists, has been the high number of false-positive marks on CAD interpretations. The occurrence of a large number of false-positive marks by a CAD system (with current rates 5–10 times higher than those of radiologists) can significantly hinder its usefulness by distracting the interpreting radiologist. A significant reduction in false-positive rates is required in order for a CAD system to become comparable in performance to the opinion of a second radiologist.

 

What Can Be Done?

CAD systems have been, until now, only focused on the early detection of cancer rather than the elimination of false-positive rates. New advancements in image analysis and recognition technology can potentially enable CAD for mammography to be used as a second opinion, with false-positive rates approaching those of a second radiologist.

 

The integration of several complementary algorithms within a single CAD system or the utilization of 2 CAD systems and sophisticated voting methods can enable the next generation CAD systems to achieve high sensitivity and low falsepositive rates for greater levels of certainty.

 

Application of the Voting Methodology

The entire process of applying voting methodology during image processing and analysis can be described as the work of a group of highly-skilled experts. Each member of the group has unique skills, expertise, and favored approaches, which are especially efficient in some cases and produce good enough results in others. When they work together as a team, the areas of expertise complement each other, resulting in improved overall performance. Similarly, all engines apply various methods to analyze dozens of features for a given image. Each of them has a special area of expertise and exploits a unique method, which may be viewed as distinctive to that particular engine. Sophisticated logic analyzes and combines the interim results obtained from the individual engines or algorithms to determine the final result and the corresponding confidence value.

 

The Voting Mechanism Approach

It is possible to reduce the false-positive rate and increase sensitivity by combining the individual results of multiple engines and employing a voting mechanism that considers each. However, developing a voting scheme and algorithms to provide improvement for a particular combination of engines is a challenging task, requiring special aptitude and experience. The number of engines included in voting schema, their types, key expertise, and accuracy of each engine are just a few factors that have to be considered in choosing the right voting algorithm. In particular, it is very important to analyze whether or not the engines use similar approaches and whether they produce different or similar types of errors. Engines using similar approaches to recognition and producing similar types of errors are called non-orthogonal. Engines based on varying methodologies and, therefore, making different errors are called orthogonal. Voting scheme and algorithms are very different from non-orthogonal to orthogonal engines. In any case, voting algorithms are uniquely created to homogenize results of particular engines, focus on individual engines’ strengths, avoid their weaknesses, and suppress unimportant results. In some cases, voting eliminates the results of an engine or algorithm altogether; in others it combines them with the results of others.

 

Successful medical imaging technologies need to consider the strengths and weaknesses of particular engines, their peculiarities and individual characteristics, and other factors. Employing non-optimal voting algorithms may not bring about an improvement of results or even deliver poor performance.

 

The First Applications of Voting Methodology

The implementation of a powerful combination of engines using a number of fundamentally different algorithms and techniques was initially applied in the postal automation industry. A combination of a human-like holistic analysis, multiple neural networks and sophisticated statistical voting algorithms enabled a significant  improvement in recognition rates and a decrease in error rates in mail processing.

 

These advancements in mail automation were first achieved in 1998, when the Remote Computer Reader (RCR) applied by the United States Postal Service was recognizing about 35% of machine printed and 2% of handwritten letter mail pieces.² Modern systems (thanks to the use of voting methodology) can today recognize 93% of machine printed and about 88% of handwritten letter mail—or more than 90% cumulatively.² Similarly, the application ofmultiple engines in a voting scheme for the banking and financial services industry raised the read rates in payment automation from 40% in 1997 to 80% at a 1% error rate today.

 

The universality of these algorithms and methods makes them fully applicable to the medical imaging market. The distinctive approaches currently used in document imaging industries are waterfall voting and parallel processing voting.

 

The predecessor of voting is a so-called “waterfall” scheme, where multiple recognition engines are configured sequentially, one after another. In this arrangement, the first recognition engine (or algorithm) reviews all pieces of the image stream and finalizes what it can. The second recognition system gets a shot at the rejects. If this auxiliary system can recognize, with high enough confidence, any of the images that the first system could not, more images are finalized and the final recognition rate is improved. Generally, the second recognition engine makes moderate improvements in the overall read rate because it works only on a part of the stream. One potential downfall is that there is a tendency to increase the final error rate if the second engine makes errors on the type of images the first engine successfully rejects.

 

The simple voting algorithm, called “parallel processing,” combines the power of 2 or more recognition engines or algorithms. Using this method, each engine reviews 100% of the document stream. The result is determined based on the majority ranking alone and not on confidence factors. Parallel processing is a simple and effective way to reduce errors. It is also possible to further improve performance by taking advantage of confidence levels reported by different engines. Even 2 engines are sufficient if confidence values are involved because the system has a lot more information with which to work.

 

The latest achievement in voting utilizes a few engines with known internal processes and intermediate results. This allows a voting scheme to choose the best outcome, leveraging an understanding of the choices available at each stage of the engine’s decision-making tree. This method is efficient and delivers better recognition rates and accuracy.

 

Today, voting algorithms are commonly used to improve recognition results in postal and payment automation, such as reading and sorting mail and checks. Sophisticated and creative voting methods allowed the industries to achieve unprecedented recognition improvement and surpass industries’ expectations for accuracy.

 

Voting mechanisms applied to medical imaging promise similar breakthroughs in recognition performance.

 

 

Multiple, parallel recognition processes can offer many technological advances in medical imaging. Each image recognition process may identify areas of interest on the mammogram image independently, without sharing information with other image recognition processes. Image recognition processes might also work together to identify different areas of interest. After image recognition processes individually identify areas of interest or objects on the mammogram image, the different areas can be compared to determine a confidence value related to the accuracy of the identifications. The comparison can be done using a voting process. Comparing the results of multiple image recognition processes allows for the mitigation of the inherent faults of the image recognition process, thus leading to reduced false-positive and false-negative rates. Additionally,methods utilizing multiple image recognition processes, rather than a single one, amicably lend themselves to multiple processor systems or networks. Thereby, they allow image recognition processes to be determined in parallel, increasing computational efficiency and spreading the workload across multiple processors. The result is fast and produces accurate actionable information.

 

The voting mechanism experience in other markets suggests that utilizing multiple image recognition processes is the most efficient means by which superior performance can be delivered to the market. In fact, this same experience shows that not using such an approach rapidly becomes a competitive disadvantage to the supplier that does not offer it to the market once it becomes available and accepted.

 

The reduction of false-positive readings achieved through the utilization of voting methodology in medical imaging will ultimately result in decreased medical costs, emotional stress, follow-up examinations, and recalls. It is only when CAD systems achieve sensitivity and false-positive rates approaching those of a radiologist that they will be embraced as a second opinion tool with no hesitation.

 

---

References