Reuse existing verification assets with the Portable Test and Stimulus Standard
Article By : Matthew Ballance
How do we shorten our path to the rich rewards that PSS promises?
The productivity promise of portable stimulus has rapidly become well known in our industry. The high level of interest exhibited in the Accellera Portable Test and Stimulus Standard (PSS) makes sense: who wouldn’t welcome a way to ensure that all the thought, effort, and time we put into building new verification environments and creating meaningful tests can be reused and scaled across verification engines, technologies, and projects? So how do we shorten our path to the rich rewards that PSS promises?
Being human, it’s only natural that we want it. And we want it now. And maybe we’re being a little selfish, but can we leverage at least some of the verification components, tests, and environments we’ve created in the past? Well, the good news is that we can have our cake and eat it too by using assets we already have on hand. Here is a methodology and a planning process to minimize duplicated effort and maximize the reuse benefits of adopting PSS.
Take stock of what you have
Performing an inventory of existing assets helps to ensure maximum benefit from previous investment. You may be surprised by the wealth of information in your design and verification environments that can be used to implement a portable stimulus environment. SystemVerilog constraints and test realization code are two good sources for jump-starting PSS descriptions.
PSS descriptions capture test intent in a declarative manner. Declarative descriptions, as we’ve seen from the use of the declarative constraint description in SystemVerilog, lend themselves very nicely to reuse and automation. The format of SystemVerilog constraints is sufficiently similar to PSS constraints that their reuse can be as simple as copying and pasting the SystemVerilog constraints into the PSS model. Typical targets for reuse are configuration classes that specify the rules for configuring IP BLOCK and subsystem operation modes.
Constraints are often in a form that isn’t immediately recognizable, though. Think about a spreadsheet that specifies the memory map for an SoC. With a little bit of work, however, this information that has been captured in a machine-readable format can easily be converted into PSS constraints to specialize the accesses that are targeted to different memory regions.
Existing environments have a wealth of test realization descriptions, though often in a form that needs to be modified a bit to work with a PSS description. In Universal Verification Methodology (UVM) environments, look for utility sequences that perform simple operations on an IP block, such as setting its configuration, performing an operation, and the like. Sometimes these sequences are created with random constraints and variables. In other cases, tasks are provided with arguments to control the different operation modes. In both cases, PSS descriptions can easily leverage these test realizations.
Test realization code for implementing portable stimulus test intent is nearly always written in existing imperative languages, such as SystemVerilog, C++, or C. Figure 1 shows a code snippet from an existing SystemVerilog task within a virtual sequence that is setting up the DMA engine to perform a transfer on a given channel. A PSS model could leverage this code to interface with the DMA engine. Note that, because this is UVM, this task uses a UVM register model to access registers within the DMA engine.
Figure 1 PSS can reuse this SystemVerilog test realization to access DMA engine registers.
Figure 2 shows a similar C function for programming the DMA engine to perform a single transfer. We can also leverage this code to provide test realization for PSS test intent for the DMA engine. Note that the two sets of existing code are similar but not the same. We’ll need to determine how best to interface to these from our PSS.
Figure 2 PSS can also leverage embedded C test realizations for DMA control.
Reusable results checking and strategies
One aspect of testing that varies significantly across the IP-block to SoC verification continuum is results checking. At the block level, it’s common to use detailed scoreboard-based checking that looks at details of how an operation was carried out, as well as its overall result. At the SoC level, that level of visibility into the design isn’t feasible, and result checking tends to be based only on the operation’s overall result.
Defining a checking strategy that will be usable all the way from an IP block to the full SoC will be important if vertical-reuse portability is a high priority for your organization. In this case, it is highly recommended to focus on building into the PSS description the types of checks that retain validity at the SoC level. Typically, such checks will be based on in-memory data and will focus on the overall success (or failure) of an operation.
It is always possible to augment functional checks with implementation checks. For example, at the block level, the DMA engine operation can be checked from a portable stimulus perspective by purely functional checks (i.e., verifying that the data at the destination is the same as the data at the source). The block-level scoreboard can still be active in checking the details of how the DMA transfer was carried out, however. This strategy can be extended to the subsystem and SoC levels as well, for example by bringing performance-checking scoreboards in at the SoC level.
Reuse friendly test realization
Having multiple implementations of test realization is effectively mandatory. It’s important for verifiers working in UVM to be able to take advantage of the services that UVM provides, such as using a register model. At the same time, it’s important for verifiers working with embedded software to be able to take advantage of the register-access mechanisms that they are familiar with, such as packed data structures, bit fields, and the like.
Defining common APIs for test realization code and ensuring that the test realizations for different IP blocks can interoperate, ensures that portable stimulus reuse is facilitated, and not limited, by test realization code. It is important to maximize the commonality between the different test realization implementations. Designing a common API that can be used by all implementations is a first step in this direction. Figure 3 shows an API for use by a DMA action that is built on top of the SV tasks reused from the block-level verification environment.
Figure 3 A common DMA API can be built on top of reused block-level SV tasks.
Figure 4 shows an implementation of the same functionality in C for use in an embedded-software environment. Note that the implementation is slightly different with respect to the devid parameter because SystemVerilog is an object-oriented language, while C is not. In the SystemVerilog environment, the mem2mem task is a member of a class that holds needed data, such as the register model. In a SystemVerilog environment, the devid parameter the PSS model specifies will be mapped to the appropriate class object. Since C is not object-oriented, the code must deal with mapping the devid parameter from the PSS model to the data object holding the data that the utility code needs. This approach of keeping a functionally equivalent API, even if the underlying details differ a bit, dramatically simplifies the task of mapping from PSS to the various implementations of test realization.
Figure 4 A common DMA API can also be built using C.
If vertical reuse – the reuse of test intent from IP-block to SoC level – is of high importance, it’s important to consider whether it’s worth investing in an environment-compatibility layer, such as the UEX hardware-access layer shown in Figure 5. The UEX hardware-access layer provides a C API for accessing platform memory and threading capabilities in several ways. Using a compatibility layer like this enables test realization code for an embedded-software environment to be developed and debugged much earlier in the verification process and reused across more of the verification process.
Figure 5 Using an environment-compatibility layer like this UEX hardware-access layer allows test realization code to be developed and debugged early in the verification process.
Whether it’s a compatibility layer that spans several platforms or a series of environment-specific APIs, it is important to consider how the test realization for different IP blocks will cooperate. The test realization code for all IP blocks will likely need to access memory. The test realization for many IP blocks will require notification when an interrupt occurs. In production code, an operating system provides the glue that connects the driver code for various IP blocks. In a verification environment, however (whether UVM or embedded software), something much more lightweight is required.
Figure 6 shows an interrupt-service routine for a DMA IP block that uses the UEX API to read the DMA registers and notify waiting routines when a DMA transfer is complete. The UEX library enables this same code to run in a UVM environment, an embedded bare-metal software environment, or an OS-based environment. This reuse of test realization code enables early debug of code for accessing IP blocks and helps minimize rework.
Figure 6 Using the UEX API for this DMA IRQ routine allows the code to run in several different environments.
Actions and test realization code for a specific type of IP block are expected to interact with multiple instances of that IP. Specifying a common way to select, from the PSS layer, which IP block instance is being accessed is important to ensuring uniformity across different test realization implementations. Figure 7 shows a base component action that specifies a built-in field named devid that controls which instance of an IP block a given action will access. Defining a reusable base component and action type in this way ensures that all PSS descriptions developed within your organization specify in the same way which IP block instance is in use.
Figure 7 Defining a reusable base component and action type helps ensure commonality in PSS descriptions.
Figure 8 shows how the devid field is referenced from a PSS exec block for one of the DMA actions.
Figure 8 The PSS block can reference the component devid field for DMA actions.
One best practice when developing PSS test realization code is to minimize the volume of data exchanged between the PSS model and the test realization code. Other languages that invoke a foreign language, such as SystemVerilog and Java, share this best practice. Generally speaking, the PSS description is an executive that specifies the high-level view of operations for which the test realization will carry out the details.
Take, for example, the actions involved in a DMA transfer. Before transferring data from a memory location, that memory location should be initialized. Instead of writing a PSS description to fill in memory byte-by-byte, though, the PSS description shown in Figure 9 simply specifies a memory region to initialize and delegates the details of how memory is initialized to the test realization function.
Figure 9 By delegating implementation details to the test realization function, this PSS description of a memory initialization operation minimizes the volume of data that must pass between them.
An implementation of the gendata functionality written in C is shown in Figure 10. This delegation of responsibilities enables the PSS description to stay at a high level where declarative programming is most efficient, while delegating the detail work to an imperative language, which is most efficient at carrying out these tasks.
Figure 10 This C-based test realization for the memory initialization operation also allows PSS to stay at a high level where declarative programming is most efficient.
These various design strategies all help to ensure that you and your organization can maximize the productivity benefits that portable stimulus offers — right now. Many can be implemented by leveraging the test code you already have. So, why wait until tomorrow when you can start enjoying the obvious advantages of reuse and scalability today?
Matthew Ballance is a Product Engineer and Portable Stimulus Technologist at Mentor Graphics, working with the Questa inFact Portable Stimulus product.
Related articles:
- The verification revolution
- Design patterns in SystemVerilog OOP for UVM verification
- Getting in sync with UVM sequences
- Inheritance and polymorphism of SystemVerilog OOP for UVM verification
