Demystifying Performance

9 March 2016
US Air Force

As we stride ahead as Abaco Systems, software is playing a key role in differentiating us from our competitors because it’s a significant strength for us.

One of the things we see is how the embedded software world has now opened out to encompass the enormous open community of Linux, bringing a huge choice of open APIs, open source tools and utilities. That choice, though, can be overwhelming. How do you choose which is right for your application? 

The problem with many of these is, while they may be open source or provide open standard APIs, they may not actually be fit for purpose when developing real-time embedded applications. You also could be well into your application development before you find this out. The way forward, it seems to me, is to identify tools that are fit for purpose—and work back from there.

The Abaco Systems AXIS software tools suite is designed specifically to support the development of the most demanding, most sophisticated embedded applications. Until now, we have focused our AXIS software tools suite on the High Performance Embedded Computing (HPEC) end of the embedded spectrum. It has become apparent to the software developers using AXIS internally in Abaco Systems, however, that AXIS has great value across our range of offerings—from applications running on a single Tegra-K1 based COM Express module or a single 3U VPX single board computer, right though to a radar processor system consisting of ten DSP282 multiprocessors communicating over high speed fabric.

That’s a pretty broad spread of application and platform types—but what they all have in common is that they reflect today’s reality in which the majority of CPUs are now effectively complex multiprocessor systems on a chip, capable of running many concurrent threads. 

And: multithreaded applications can be notoriously complex to design, develop, debug and tune. 

The AXIS suite helps with all of that—we optimized it explicitly for multiprocessing applications—and this has led to us refocusing AXIS to encompass all our processing offerings. Part of that has involved enhancing and spinning out tools previously embedded in the AXISPro tool suite, so that they’re now available standalone. AXIS EventView is one of those tools.

So: how effective is EventView in practice?

Demystifying your application performance

When we took on the “Next Generation Radar COTS Study” for MIT Lincoln Laboratory on behalf of the USAF, we were presented with two complex radar benchmarks—one for Synthetic Aperture Radar (SAR) and one for Ground Moving Target Indication (GMTI). We needed to meet some very challenging performance targets on our hardware platforms.

We baselined both benchmarks, and—unsurprisingly—we were way outside of the performance targets. Clearly, we needed to make substantial optimizations—but where to start? Each of these benchmarks had many functions and algorithms in its processing pipeline—and they were also distributed across many threads (30+ in some cases) and included communication between these threads.

We needed an in depth, behind the scenes look at the performance of each benchmark to identify the "long poles" in the tent that is the benchmark processing pipeline. This would allow us to spend the time optimizing the areas that would have the biggest impact on the overall benchmark performance.

Fortunately, we had just the tool—AXIS EventView. One day of instrumentation of each benchmark, and bang! The event traces gave us an instant visual breakdown of the performance of each benchmark, and there were some real surprises. 

EventView made obvious some problem areas which we were able to address in a few hours—and we were able to double the performance of the GMTI within a day. Once we’d picked the low hanging fruit, we were able to put a plan in place to make high impact optimizations within the challenging timeframe.

We then used the tool as part of our development cycle to monitor the impact of our improvements, and it also provided a great vehicle to present these improvements to the customer—and the customer loved the tool too.

The focused effort enabled by EventView, and the insights it provided, enabled us to beat the performance targets for both benchmarks by a comfortable margin by the end of the study. It was paramount to our success—it really did demystify performance, enabling us to achieve much more in a shorter timeframe than would otherwise have been possible.

Could we have achieved the same results with an open source tool or traditional profiler in the same timeframe? Probably not…

If you’re facing similar challenges—the need to wring the maximum possible performance out of a system in the minimum possible time—it could be that EventView could be as invaluable to you as it was to us. Why don’t you put it to the test?

David Tetley

David is engineering manager of Abaco Systems' HPEC Center of Excellence in Boston, responsible for both the AXIS software suite and HPEC system development. He has a background in military signal and image processing, starting his career as a scientific officer for the UK Ministry of Defence working with lasers and missile seeker technology, and then moving into software development in the days of the Texas instruments C40 and Analog Devices Sharc processors. As engineering manager in Boston, he has led the development of the AXIS software suite and helped shape Abaco’s HPEC strategy.