SW/FW Automated Test Framework and Debug Toolkit for System Testing In Design/IP Track Presentation of the 53rd Design Automation Conference (DAC 2016, Austin TX, Jun 4 – Jun 9, 2016)
This presentation outlines a novel SW/FW automated test framework and debug toolkit for system testing that supports automated regression and effective interactive debug. The framework is based on Google Test with expansions incorporating C++14 features and various open source libraries. Major features include: test flow utilities, argument parser, JSON files for device configuration, control test equipment via Tcl_Eval(), interactive debug prompt, control of FW running in the embedded CPU via remote gdb, C++ reflective API, etc. Engineers are more productive developing testcase in this test framework develop testcases compare to writing testcase with plain old C in ad-hoc fashion.
A Methodology to Port a Complex Multi-Language Design and Testbench for Simulation Acceleration In Proceeding of the DVCON 2015(San Jose, US, Mar 1 – Mar 4, 2015).
PMC’s verification teams started exploring simulation acceleration (SA) with hardware-assisted verification in 2011, as one of the early adopters of UVM Acceleration. They undertook this effort because of the complexity and size of their mixed-language designs, which were coded in SystemVerilog, Verilog, and VHDL, and stimulated using state-of-the-art testbenches coded in UVM-e.
A few years later, the task of porting a design and testbench from simulation to acceleration evolved into a methodology and is now re-used across multiple verification teams. Finally, PMC has achieved the holy grail of SA, conquering the most complex challenges of SA verification including: 1) Speed – achieving 67x speed up, 2) Time to First Test – taking only a month to port a verification environment to run in acceleration mode, 3) Consistent – Running the same tests with RTL and an accelerated DUT, producing the same results.
This methodology exploits essential capabilities of the tools in use, and production proven procedures. This paper outlines a step-by-step guide to port an existing UVM-e testbench to SA. The verification user community can use this paper as a template to plan their migration from simulation to hardware acceleration.
Maximize Vertical Reuse, Building Module to System Verification Environments with UVM e In Proceeding of the DVCON 2013(San Jose, US, Feb 24 – Feb 28, 2013).
Given the size and complexity of modern ASICs/SoC, coupled with their tight project schedule, it is impractical to build a complete system or chip level verification environment from scratch. Instead, in order to increase productivity, maximizing reuse of existing verification components seamlessly with the project has become one of the biggest opportunities to increase verification efficiency. In this paper, we present a testbench framework to maximize vertical reuse within a project. The framework presented here has been proven on the ground-up development of a 200M gates ASIC. In our framework, the system testbench is built in a hierarchical manner by recursively importing lower level block or module testbenches. From the lowest level to the highest level, all the testbenches are designed to support plug-and-play integration. Verification engineers can hook up several lower level testbenches and turn them into a higher level testbench. The system testbench inherits the device configuration sequences, traffic generation sequences, checkers and monitors from the imported module testbenches without duplication of effort. As a result, vertical reuse shortens the development time of the system testbench, improves the quality of testbench code and allows fast bring up during system integration.
Can You Even Debug a 200M+ Gate Design? In Proceeding of the DVCON 2013(San Jose, US, Feb 24 – Feb 28, 2013).
Verification debug consumes a large portion of the overall project schedule, and performing efficient debug is a key concern in ensuring projects tape out on time and with high quality. In this paper, we outline a number of key verification debug challenges we were faced with and how we addressed them using a combination of tools, technology and methodology. The root cause of failures can be traced, most often, to either an issue in the RTL, an issue in the testbench or, in our case, the software interacting with the hardware. Speeding the debug turnaround time (the time to re-run a failing sim to replicate the issue for debug) is critical for efficient debug. Periodic saving of the simulation state was utilized extensively to narrow the debug turnaround time to a very small window. Once a re-run was launched, waveform verbosity levels could be set by users to dump the appropriate amounts of information for debug of the re-run scenario. For additional performance on the testbench side, coding methodology was introduced that allowed for maximum performance of stable sections of code. To speed SW debug, a software driver was implemented into the testbench to allow for debug of SW related issues very early on in the project.
Hardware/Software co-verification using Specman and SystemC with TLM ports In Proceeding of the DVCON 2012(San Jose, US, Feb 28 – Mar 1, 2012).
In modern ASIC/SoC design, the hardware and software have to work seamlessly together to deliver the functions, requirements and performance of the embedded system. To accelerate time-to-market and to reduce overall development cost, it is crucial to co-verify the software code with the hardware design prior to tape-out. The software team can start developing and debugging their code with the actual hardware RTL code to shorten their overall development cycle. The hardware team can use the software code to identify performance bottlenecks and incorrect functional behaviors early in the development cycle which helps to reduce the risk of increasingly expensive device revisions.
The current approach to co-verification is primarily running the software on the embedded processor inside the hardware design, either within the simulator or with ICE (in-circuit emulation). The disadvantage of this approach is slow debug turnaround time and the higher cost is procuring and supporting a dedicated emulation box or FPGA platform. In addition, the software is running in isolation relative to the testbench, hence it is often challenging and inconvenient to integrate the software with other verification IP in the testbench.
In this paper, we will present an alternate approach on how to integrate the software driver into the simulator using Specman and SystemC with TLM ports. The software is running in the same memory space as the testbench, both of which run through the simulator on the Linux host. The advantage of this approach is fast execution speed of the software and the interoperability of the software with other verification components in the testbench. The software code runs in zero simulation time and the testbench has full control of the software using TLM ports and direct memory access via pointers. In addition, the software code can invoke gdb or any other C debugger to make debugging easier.
Functional Verifi cation of Next-Generation ICs with Next-Generation Tools: Applying Palladium XP Simulation Acceleration to an Existing Specman Testbench Framework In CDNLive! 2012 (San Jose, US, Mar 13 – Mar 14, 2012).
Next-generation ICs from PMC are ever-larger, to the scale of more than 100M gates. Using conventional simulators, typical datapath simulations for telecom applications can take hours to send a complete frame, while full regression suites can take a week. With more features causing longer simulation times, it’s challenging to complete a comprehensive verifi cation plan while meeting time-to-market demands. To solve this, PMC implemented a transaction-level testbench infrastructure using the Specman Elite® tool, based on the Cadence hardware-acceleration–friendly Universal Verifi cation Methodology (UVM). High-level protocol UVM verifi cation components (UVCs) generate transactions driven to low-level interface UVCs, which generate signals that enter the device under test. To support the Palladium XP hardware acceleration platform, the aspect-oriented programming nature of Specman was exploited. Interface UVCs were extended by splitting BFMs and collectors across Specman and SystemVerilog RTL, leaving protocol UVCs unchanged. Thus, PMC’s verifi cation capabilities expanded with virtually no disruption to its ongoing verifi – cation plan. The addition of Palladium XP provides accelerated simulations that complete in 40x the speed of normal simulations. Thus, regressions can be completed in days instead of weeks and interactive debugging of top-level simulations are now possible, allowing PMC to complete a full verifi cation plan on complex ICs while reducing time to market.
I should write a article on: What is the difference between reusing and salvaging…
by David Murray, 12/15/2010, Design and Reuse
As a hardware design engineer, I was never comfortable when someone talked about IP integration as ‘stitching a chip together’. First of all, it sounded like a painful process involving sharp needles, usually preceded by a painful accident. I happened to be the recipient of said stitches when, at 8 years of age, I contested a stairs post with my forehead, and sorely lost. I have to say, luckily, I have been quite adept at avoiding the needle and thread ever since. That was of course until once when, an hour before that important customer presentation, my top-shirt button, due to an over enthusiastic yawn, pinged across my hotel room floor like a nano-UFO. A panicked retrieval of the renegade button was followed quickly with a successful hunt for an elusive emergency sewing-kit. The crisis quickly dissipated as I stitched back the button in a random-but-directed type of methodology. Needle-less to say stitching, whilst sometimes necessary, makes me uncomfortable.
Stitching, according to Wikipedia, is “.. the fastening of cloth, leather, furs, bark, or other flexible materials, using needle and thread. Its use is nearly universal among human populations and dates back to Paleolithic times (30,000 BCE).” It also states that stitching predates the weaving of cloth. So, 32,000 years later, in these hi-tech times we are still stitching things together. It’s not fur this time, but ‘ports’. Stitching a chip together involves connecting ports together with wires. (Note the terminology also where, if you don’t use certain ports you ’tie’ them off).
Weaving is a different game altogether. One definition simplifies weaving as ‘creating fabric’. Thus a key differentiator between stitching and weaving is that stitching may refer to fixing/mending things whilst weaving is used to create. Stitching is an emergency, an ah-hoc approach (please refer to my stitched button above) whilst weaving is more structured, more planned. Stitching invokes the image of being bent over, eyes squinted, immersed in the tiniest of detail. Weaving is more graceful and productive. In IC design flow terms, I equate stitching with scripting. It is task that is useful to join pieces of the flow together. Weaving creates something. It transforms thread to cloth, and therefore equates more to synthesis. Weaving is a process.
So when it came to developing and naming a tool used to effectively integrate IP and create a chip hierarchy, in a structured manner, we didn’t consider consider ‘STITCHER’ – It had to be ‘WEAVER’.
Stitching is important to fix things, and is necessary in emergency situations, however it has its limitations and as if used as a core creation process, it may come undone. So as I ranted on during that vital presentation, as I got to the cusp of the value-add, I curbed my enthusiasm, keep it slightly in check just in case those button stitches came undone and resulted in a serious eye injury of an altogether innocent customer. What then, of those poor stitched chips? What if those threads start to unravel and your chip integration is running very late. You may have to resort to different type of Weaving, when dealing with your management, customers or partners.
Looks like Cadence is finally on offensive and pushing Specman strongly over SystemVerilog. As a long time user of Specman, I would like to see Specman wins the HVL war at the end which will make my skill and experience more valuable.
Here is some words of wisdom on verification from thinkverification.com. I could not agree with this article any more, especially on the last point. We verifiers should education the importance of verification to the company, especially to the executives who make the budget decision.
I had to come to work early today to meet up with the representative from Cadence. Somehow I am chose to baby sit them, give them the requirement from the team. I have a feeling they come to help us just because they want to have Cadence’s product more erode into PMC’s tool chain. So I end up not thing any work at all today. Make it worse, I have to come in early tomorrow morning to let them into the building as well. One good thing about it is one of the representative is base off in Toronto. I am looking forward to keep him as a contact in what company that I may potentially switch when moving back home. Companies using Specman would make a perfect match for my skill set.
The story of my friend and her new crush keeps evolving and it’s getting more and more interesting. It seems she already know more about that guy than I do, even though it is me who introduce her the guy. Let’s see how this romantic adventure will turns out, Pat and I will pay close attention to the latest development.