Simplifying Custom Template-Generated Content

 

As a verification engineer, it's quite common to work with data and code that follow a regular pattern. Having an efficient way to create this repetitive code is a significant productivity boost. While there certainly are places in the code where 'your critical generation or checking algorithm' goes, much of the structure of an agent, a test environment, etc remain the same. The same goes for other parts of the flow, such as project meta-data, test lists, etc. There are two things that keep us from just making copies of a set of 'golden' files to create the basis for a new UVM agent, project, etc: some or all of the files need to have some data substituted or changed. For example, we want to substitute the name of the new UVM agent we're creating into most of the new SystemVerilog source code.

Custom code generators have been developed for some of these tasks. These often focus on providing a domain-specific way to capture input data, such as the structure of a UVM testbench or the layout of registers in a design. But there are many more opportunities to generate template-driven code that cannot justify the investment to create a focused solution.

A few years ago, I created the Verification Template Engine (VTE) to serve my needs for generating template-driven content. I developed VTE with three user-experience requirements in mind:

• Creating a new template should be very easy, but have access to powerful generation features
• Managing the available templates should be simple for a user. 
• The core tools should be generic, and make few or no assumptions about what is being generated

VTE focuses on organizing and discovering template content, but leverages the Jinja2 template engine to do the heavy lifting of template expansion. In some sense, you can think of VTE as providing a user interface to the Jinaj2 library.

I've been using VTE since developing it, but am just getting back to create proper documentation, which you can find here: https://fvutils.github.io/vte/. As part of that work, I created a quickstart guide which is both in the documentation, and forms the remainder of this post. 

Installing VTE

The easiest way to install VTE is from PyPi.

% python3 -m pip install --user vte

Test that you can run VTE by running the command (vte) and/or invoking the module:

% vte --help

% python3 -m vte --help

Creating a Template

VTE discovers templates by searching directories on the VTE_TEMPLATE_PATH environment variable. VTE uses a marker file named .vte to identify the root of a template. All files and directories in and below a template directory are considered to be part of the template. The template identifier is composed from the directory names between the directory listed in VTE_TEMPLATE_PATH and the directory containing the .vte marker file.

Let’s look at an example to illustrate the rules.

templates

  uvm

    agent

      .vte

    component

      .vte

  doc

    blog_post

      .vte

    readme

      .vte

Let’s assume we add the templates directory to VTE_TEMPLATE_PATH. VTE will find four templates:

uvm.agent

uvm.component

doc.blog_post

doc.readme

All files in and below the directory containing the .vte marker will be rendered when the template is used.

Creating the Template Structure

Let’s create a very simple template structure. Create the following directory structure:

templates

  doc

    readme

Change directory to templates/doc/readme and run the quickstart command:

% vte quickstart

Verification Template Engine Quickstart

Template directory: templates/doc/readme

Template Description []? Create a simple README

This command will prompt for a description to use for the template. Enter a description and press ENTER. This will create the .vte marker file.

View the .vte file. You’ll see that the initial version is quite simple. For now, this is all we need.

template:

  description: Create a simple README

  parameters: []

#   - name: param_name

#     description: param_desc

#     default: param_default

Creating the Template File

Now, let’s create the template file that will be processed when we render the template. Our readme template only has one file: README.md.

Create a file named README.md containing the following content in the templates/doc/readme directory:

# README for {{name}}

TODO: put in some content of interest

VTE supports defining and using multiple parameters, but defines one built-in parameter that must be supplied for all templates: name. Our template file references name using Jinja2 syntax for variable references.

We have now created a simple template for creating README.md files.

Rendering a Template

In order to render templates, VTE must first be able to discover them. Add the templates directory to the VTE_TEMPLATE_PATH environment variable.

% export VTE_TEMPLATE_PATH=<path>/templates # Bourne shell

% setenv VTE_TEMPLATE_PATH <path>/templates # csh/tsh

Let’s test this out by running the vte list command:

% vte list

doc.readme - Create a simple README

If you see the doc.readme line above, VTE has successfully discovered the template.

Now, let’s actually generate something. Let’s create a new directory parallel to the templates directory in which to try this out

% mkdir scratch

% cd scratch

Finally, let’s run the generate command:

% vte generate doc.readme my_project

Note: processing template README.md

VTE prints a line for each template file is processes. The output above confirms that is processed the template README.md file.

Let’s have a look at the result. View the README.md file in the scratch directory.

# README for my_project

TODO: put in some content of interest

Node that the {{name}} reference was replaced by the name (my_project) that we specified.

You have now created your first VTE template!

Conclusion

As the tutorial above illustrates, creating a new template for use with VTE is no more effort than making a few name substitutions. If you use the template more than once, you will already have received a positive return on the effort invested. While templates can be simple, you have the full power of the Jinja2 template engine when you need to do something more complex. I encourage you to check out the VTE documentation and look for opportunities where using template-driven content generation can make your life easier and make you more productive.

Copyright 2022 Matthew Ballance

The views and opinions expressed above are solely those of the author and do not represent those of my employer or any other party.

PyVSC: Working with Coverage Data

 

I’ve been investing some time in documentation updates this weekend, after a couple of PyVSC users pointed out some under-described aspects of the PyVSC coverage flow. Given that these areas were under-documented in the past, it seemed a good opportunity to highlight what can be done with functional coverage data once it is sampled by a PyVSC covergroup.

So, we’ve described some functional coverage goals using a PyVSC covergroup and coverpoints, created a covergroup instance, and sampled some coverage data – perhaps it was randomly-generated stimulus or data sampled from a monitor. What now?

Runtime Coverage API

One simple thing we can do is to query coverage achieved using the coverage APIs implemented by PyVSC covergroup classes. The `get_coverage` method returns the coverage achieved by all instances of a covergroup type. The `get_inst_coverage` method returns the coverage achieved by the specified covergroup instance.

Let’s look at an example:

In the example above, we define a covergroup with a coverpoint that contains four bins (1, 2, 4, 8). We create two instances of this covergroup and sample them with two different values. After each call to sample, we display the coverage achieved by all instances of the covergroup (type coverage) and the coverage achieved by each instance.

The output from this example is shown above. After sampling the first covergroup, the coverage achieved for that, and all, instances is 25% since one of four bins was hit. After sampling the second covergroup, the coverage achieved for that covergroup instance is also 25%. Because two different bins are hit between the two covergroup instances, two of four bins are hit (50%) for type coverage.

Runtime Coverage Reports

Another way to look at collected coverage is via a coverage report. PyVSC provides two methods that are nearly identical for obtaining a textual coverage report:

• get_coverage_report – Returns the report as a string
• report_coverage – Writes the report to a string (stdout by default)

Both of these methods accept a keyword parameter named ‘details’ which controls whether bin hits are reported or just the top-level coverage achieved. Let’s look at a derivative of the first example to better understand the textual coverage report options.

This example is nearly identical to the first one, but with calls to ‘report_coverage’ instead of calls to the covergroup get_coverage methods.

The output from running this example is shown above. When reporting ‘details’ is enabled, the content of each coverage bin is reported. When reporting ‘details’ is disabled, only the top-level coverage achieved is reported. Displaying a coverage report with details is often helpful for confirming the correctness of a coverage model during development.

Saving Coverage Data

The PyUCIS library implements a Python interface to coverage data via the Accellera UCIS data model. It implements an object-oriented interface to coverage data, in addition to the Python equivalent of the UCIS C API. PyVSC uses the PyUCIS library to save coverage data, and can do so in a couple of interesting ways. Coverage data is written via the vsc.write_coverage_db method.

PyVSC can save coverage data to the XML interchange format defined by the UCIS standard. This is the default operation model for write_coverage_db. The example below shows saving it to a file named  ‘cov.xml’. 

PyVSC can also save coverage data to a custom database format, provided the tool that implements that database implements the UCIS C API. The example below saves coverage data to a custom database using the UCIS C API implemented in the shared library named ‘libucis.so’.

Both of these paths to saving coverage may provide ways to bring coverage data collected by PyVSC into coverage-analysis flows implemented by commercial EDA tools. Check your tool’s documentation and/or check with your application engineer to understand which options may be available. Feel free to report what works for you on the PyVSC discussion forum so that others can benefit as well.

Viewing Coverage Data

Obviously, you can use commercial EDA tools to view coverage data from PyVSC if your tool provides a path to bring UCIS XML in, or if it implements the UCIS C API. PyUCIS Viewer provides a very simple open-source graphical application for viewing coverage in UCIS XML format. 

To use PyUCIS Viewer, save coverage data in UCIS XML interchange format, then run PyUICIS Viewer on that XML file:

% pyucis-viewer cov.xml

A simple tree-based graphical viewer will open to show type and instance coverage. 

Conclusion

There are several options for viewing and manipulating coverage once it has been collected via a covergroup modeled with PyVSC. In a future post, we’ll look at some additional manipulation and reporting options being implemented within PyUCIS. 

Until then, check out the latest additions to the PyVSC documentation and raise questions and issues on the PyVSC GitHub page.

Copyright 2022 Matthew Ballance

The views and opinions expressed above are solely those of the author and do not represent those of my employer or any other party.

Arrays, Dynamic Arrays, Queues: One List to Rule them All

Randomizable lists are, of course, very important in modeling more-complex stimulus, and I've been working to support these within PyVSC recently. Thus far, PyVSC has attempted to stay as close as possible to both the feature set and, to the extent possible, the look and feel of SystemVerilog features for modeling constraints and coverage.  With randomizable lists, unlike other features, I've decided to diverge from the SystemVerilog. Keep reading to learn a bit more about the capabilities of randomizable lists in PyVSC and the reason from diverging from the SystemVerilog approach.

SystemVerilog: Three Lists with Different Capabilities

SystemVerilog is, of course, three or so languages in one. There's the synthesizable design subset used for capturing an RTL model of the design. There's the testbench subset that is an object-oriented language with classes, constraints, etc. There's also the assertion subset. These different subsets of the language have different requirements when it comes to data structures. These different requirements have led SystemVerilog to have three array- or list-like data structures:

Fixed-size arrays, as their name indicates, have a size specified as part of their declaration. A fixed-size array never changes size. Because  the array size is captured as part of the declaration, methods that operate on fixed-size arrays can only operate on a single-size array.

The size of dynamic-size arrays can change across a simulation. The size of a dynamic-size array is specified when it is created using the new operator. Once a dynamic-size array instance has been created, the only way to change its size is to re-create it with another new call. Well, actually, there is one other way. Randomizing a dynamic-size array also changes the size.

The size of a queue is changed by calling methods. Elements can be appended to the list, removed, etc. A queue is also re-sized when it is randomized.

PyVSC: One List with Three Options

If you've done a bit of Python programming, you're well aware that Python has a single list. Python's list is closest to SystemVerilog's queue data structure. My initial thought on supporting randomizable lists with PyVSC was just to create an equivalent to the list and be done. But then I thought a bit more about use models for arrays in verification. Each SystemVerilog array type represents a useful use model, but there's also another use model that I've never properly figured out how to easily represent in SystemVerilog. Fundamentally, there are two use cases for randomizable lists:

• List with non-random elements
• List with random elements, whose size is not random
• List with random elements, whose size is random

When the size of a list whose size is not randomizable is modified by appending or removing elements, its size is preserved when the list is subsequently randomized.

Here are a few examples.

@vsc.randobj
class my_item_c(object):
    def __init__(self):
      self.my_l = vsc.rand_list_t(vsc.uint8_t(), 4)

The example above declares a list that initially contains four random elements.

@vsc.randobj
class my_item_c(object):
    def __init__(self):
      self.my_l = vsc.randsz_list_t(vsc.uint8_t())

    @vsc.constraint
    def my_l_c(self):
        self.my_l.size in vsc.rangelist((1,10))

The example above declares a list whose size will be randomized when the list is randomized. A list with randomized size must have a top-level constraint that specifies the maximum size of the list. Note that in this case the size of the list will be between 1 and 10.

If you wish to use a list of non-random values in constraints, you must store those values in an attribute of type list_t. This allows PyVSC to properly capture the constraints.

@vsc.randobj
class my_item_c(object):
    def __init__(self):
      self.a = vsc.rand_uint8_t()
      self.my_l = vsc.list_t(vsc.uint8_t(), 4)

      for i in range(10):
          self.my_l.append(i)

    @vsc.constraint
    def a_c(self):
      self.a in self.my_l

it = my_item_c()
it.my_l.append(20)

with it.randomize_with(): 

      it.a == 20 

In the example above, the class contains a non-random list with values 0..9. After an instance of the class is created, the list is modified to also contain 20. Then we randomize the class with an additional constraint that a must be 20. This randomization will succeed because the my_l list does contain the value 20.

Using Lists in Foreach Constraints 

PyVSC now also supports the foreach constraint. By default, a foreach constraint provides a reference to each element of the array. 

@vsc.randobj
class my_s(object):
    def __init__(self);
        self.my_l = vsc.rand_list_t(vsc.uint8_t(), 4)

    @vsc.constraint
    def my_l_c(self):
        with vsc.foreach(self.my_l) as it:
            it < 10

In the example above, we constrain each element of the list to have a value less then 10. However, it can also be useful to have an index to use in computing values. The foreach construct allows the user to request that an index variable be provided instead.

@vsc.randobj
class my_s(object):
    def __init__(self);
        self.my_l = vsc.rand_list_t(vsc.uint8_t(), 4)

    @vsc.constraint
    def my_l_c(self):
        with vsc.foreach(self.my_l, idx=True) as i:
            self.my_l[i] < 10

The example above is identical semantically to the previous one. However, in this case we refer to elements of the list by their index. But, what if we want both index and value iterator?

@vsc.randobj
class my_s(object):
    def __init__(self);
        self.my_l = vsc.rand_list_t(vsc.uint8_t(), 4)

    @vsc.constraint
    def my_l_c(self):
        with vsc.foreach(self.my_l, it=True, idx=True) as (i,it):
            it == (i+1)

Just specify both 'it=True' and 'idx=True' and both index and value-reference iterator will be provided.

One List to Rule them All

As of the 0.0.4 release (available now!) PyVSC supports lists of randomizable elements whose size is either fixed or variable with respect to randomization. Check it out and see how it helps in modeling more-complex verification scenarios in Python!

Disclaimer
The views and opinions expressed above are solely those of the author and do not represent those of my employer or any other party.

Python Verification Stimulus and Coverage: Constraints

Over the past few blog posts, we've looked at:

• The fundamentals of modeling stimulus and functional coverage in Python
• Modeling verification data types in Python
• Modeling and capturing functional coverage in Python
• Making use of captured coverage data

In this post, we will look at how to model constraints in Python using the PyVSC library. Several libraries that I'm familiar with (mostly C++) provide a way to embed constraints using an embedded domains-specific language. I've felt some discontent with the way that this is done because it's fairly different from what we're familiar with in languages that treat constraints as first-class language constructs.  Specifically, in C++, constraints are often modeled as lists of comma-separated expressions, and not as blocks of statements the way a first-class language would model them. This means that constraints that would normally be grouped in a block need to be split up into multiple blocks. And, when constraints are combined, they become difficult to read. One of my motivations in examining Python's support for embedded domain-specific languages was to see if Python offered a better approach. My feeling is that it most definitely does, and I hope you agree.

The examples in this post come from the PyVSC documentation. The latest version is always directly available on readthedocs.io.

Constraint Blocks

Constraint blocks in PyVSC are special class methods that are decorated with a constraint Python decorator. The presence of the constraint decorator causes PyVSC to see treat these methods, not as a regular Python method, but as an element to construct a constraint expression model. Here's a simple example:

@vsc.randobj
 class my_base_s(object):

     def __init__(self):
         self.a = vsc.rand_bit_t(8)
         self.b = vsc.rand_bit_t(8)

     @vsc.constraint
     def ab_c(self):
        self.a < self.b

PyVSC calls the methods marked as constraints once during the class-elaboration process and creates an internal model of the constraint statements captured in these constraint blocks. It also replaces the method with a class field that has special properties. More on that later.

By overloading operators (such as "<"), PyVSC is able to get an expression tree that can be given to the Boolector SMT solver that does all the heavy-lifting of constraint solving for PyVSC. 

As in SystemVerilog, constraint blocks are considered "virtual". In other words, having a constraint in a sub-class with the same name as a constraint in a base class causes the base-class constraint to be replaced with the sub-class constraint. 

@vsc.randobj
class my_base_s(object):

    def __init__(self):
        self.a = vsc.rand_bit_t(8)
        self.b = vsc.rand_bit_t(8)
        self.c = vsc.rand_bit_t(8)
        self.d = vsc.rand_bit_t(8)

    @vsc.constraint
    def ab_c(self):
       self.a < self.b

@vsc.randobj
class my_ext_s(my_base_s):

    def __init__(self):
        super().__init__()
        self.a = vsc.rand_bit_t(8)
        self.b = vsc.rand_bit_t(8)
        self.c = vsc.rand_bit_t(8)
        self.d = vsc.rand_bit_t(8)

    @vsc.constraint
    def ab_c(self):
       self.a > self.b

In the example above, the ab_c constraint in the sub-class my_ext_s overrides the ab_c constraint in the base class my_base_s. The effect is that instances of my_ext_s will enforce the relationship (a > b) instead of the base-class relationship (b < a).

Constraint Expressions

PyVSC supports specifying constraints using the familiar set of expression constructs we're all used to using: <, >, ==, !=, etc. The one set of operators that do not behave as you would expect in Python are the logical ones: and, or, etc. Python doesn't allow overriding these logical operators, unlike other operators. Consequently, we use bit-wise operators instead. 

with my_i.randomize_with() as it:
    it.a_small() | it.a_large()

In the example above, a_small and a_large are boolean constraints. Normally, in Python, we would combine these using or. In a PyVSC constraint, we use | instead.

There are two special expressions to be aware of that can be used in constraints: the 'in' expression and part select. Both of these are frequently used in Python, and both can be used in constraints with PyVSC.

@vsc.randobj
class my_s(object):

    def __init__(self):
        self.a = vsc.rand_bit_t(8)
        self.b = vsc.rand_bit_t(8)
        self.c = vsc.rand_bit_t(8)
        self.d = vsc.rand_bit_t(8)

    @vsc.constraint
    def ab_c(self):

       self.a in vsc.rangelist(1, 2, [4,8])
       self.c != 0
       self.d != 0

       self.c < self.d
       self.b in vsc.rangelist([self.c,self.d])

The 'in' expression operates on a PyVSC-type field on the left-hand side and a PyVSC rangelist on the right-hand side. A rangelist can contain individual values and ranges of values. A rangelist can contain both constant expressions, such as the literals shown in the first rangelist, and expressions involving other PyVSC variables.

Python provides array-slicing expressions, and PyVSC reuses these to implement bit-slicing of scalar fields.

@vsc.randobj
class my_s(object):

    def __init__(self):
        self.a = vsc.rand_bit_t(32)
        self.b = vsc.rand_bit_t(32)

    @vsc.constraint
    def ab_c(self):

        self.a[7:3] != 0
        self.a[4] != 0

In this example, the part-select operator is used to ensure that certain bits and bit-ranges within the 'a' field are non-zero.

Constraint Statements

Statements pose two additional challenges when developing an embedded domain-specific language: often keywords are already in use by the host language (and cannot be overloaded), and languages often don't provide easy way to overload statement blocks. Here, Python definitely has some benefits. Now, not surprisingly, Python doesn't allow overloading of built-in statements.But, it does provide an easy-to-use way of creating scopes that can be used to capture block statements. 

PyVSC provides three constraint statements. If/else and Implies are block statements, while the 'unique' constraint is not.

Python's mechanism for introducing a block of statements is the 'with' statement. When combined with a user-defined object, the 'with' statement allows a library writer to comprehend statement blocks. Here is an example of an if/else constraint:

@vsc.randobj
class my_s(object):

    def __init__(self):
        self.a = vsc.rand_bit_t(8)
        self.b = vsc.rand_bit_t(8)
        self.c = vsc.rand_bit_t(8)
        self.d = vsc.rand_bit_t(8)

    @vsc.constraint
    def ab_c(self):

        self.a == 5

        with vsc.if_then(self.a == 1):
            self.b == 1
        with vsc.else_if(self.a == 2):
            self.b == 2

The if_then class is used to form the 'if' branch of an if/else statement. Additional 'else-if' branches can be added on with the 'else_if' class. The final 'else' case is described with the 'else_then' class. 

The implies constraint is similar, since 'implies' applies to a constraint block:

class my_s(object):

    def __init__(self):
        super().__init__()
        self.a = vsc.rand_bit_t(8)
        self.b = vsc.rand_bit_t(8)
        self.c = vsc.rand_bit_t(8)

    @vsc.constraint
    def ab_c(self):

        with vsc.implies(self.a == 1):
            self.b == 1

Finally, the 'unique' constraint ensures that the specified terms have a unique value:

@vsc.rand_obj
class my_s(object):

    def __init__(self):
        self.a = vsc.rand_bit_t(32)
        self.b = vsc.rand_bit_t(32)
        self.c = vsc.rand_bit_t(32)
        self.d = vsc.rand_bit_t(32)

    @vsc.constraint
    def ab_c(self):
        vsc.unique(self.a, self.b, self.c, self.d)

Customizing Declared Constraints

In general, we want to specify most of our constraints as part of the class hierarchy. However, there are quite a few cases where we need to customize things a bit when we randomize the class. PyVSC provides two capabilities to enable this: in-line constraints, and constraint mode.

In-line constraints allow us to add extra constraints that are specific to a randomization. To use this capability, use the 'randomize_with' call instead the the 'randomize' call as shown below:

@vsc.randobj
class my_base_s(object):

    def __init__(self):
        self.a = vsc.rand_bit_t(8)
        self.b = vsc.rand_bit_t(8)

    @vsc.constraint
    def ab_c(self):
       self.a < self.b

item = my_base_s()
for i in range(10):
   with item.randomize_with() as it:
     it.a == i

The example above ensures that 'a' increments 0..9 across 10 randomization calls. Note that the fields of the class can either be referenced via the handle that is being randomized (item) or via a special variable created by the 'with' block.

Constraint-mode allows entire constraint blocks to be turned on and off. Error-injection is one use-case for this feature. 

 @vsc.randobj
 class my_item(object):

     def __init__(self):
         self.a = vsc.rand_bit_t(8)
         self.b = vsc.rand_bit_t(8)

     @vsc.constraint
     def valid_ab_c(self):
        self.a < self.b

item = my_item()
# Always generate valid values
for i in range(10):
   item.randomize()

item.valid_ab_c.constraint_mode(False)

# Allow invalid values
for i in range(10):
   item.randomize()

As shown in the example above, the first 10 randomizations have all constraints enabled. The next 10 randomizations have the 'valid_ab_c' constraint block disabled, allowing invalid values to be produced.

Next Steps

Over the past few posts, we've looked at the key features and capabilities of PyVSC, a Python library for modeling constraints and functional coverage for verification. I would encourage you to use PyVSC, report bugs, and suggest enhancements. PyVSC is, of course, an open-source project, so contributions in the form of pull requests for bug fixes, documentation updates, and enhancements are always welcome as well.

For the last few months (pretty close to a year, actually) I've been really focused on hardware-centric functional verification infrastructure, with a focus on Python. I'm thinking it's time to get back to focusing a bit more on firmware development and embedded-software verification. There have been some interesting developments in this space, recently, which have the potential to enable significant changes in the way we develop and verify firmware and other low-level software. Look for more on that soon. Until then, stay safe and keep learning!

 

Disclaimer
The views and opinions expressed above are solely those of the author and do not represent those of my employer or any other party.