VHDL Test Bench Package: A Partner Scripting Language

Using a partner language.

This blog entry will show how a “partner” language can be of great assistance to the verification effort. I am specifically referring to the many free scripting languages that are available, such as tcl/tk, perl, python, java ... I currently use tcl/tk as my partner language. I chose tcl because it is used by many tool vendors that provide simulators for HDL simulation. Many poohoo the choice of TCL, but what ever gets the job done is good. I found the GUI part of tcl/tk most interesting. The ability to whip together a small GUI that can do huge amounts of work, yet be intuitive to use, has increased my productivity. Specially because the GUI “script” is so easy and quick to put together.

This blog entry will also tie together topics mentioned in the Variables and Randomization posts.

In the following text, the word “script” refers to partner language code. The words “test case” and “stimulus file”, refer to test bench scripts that the VHDL Test Bench Package would parse.

One might ask, “What can a partner language do for me? After all, it is more code and yet another language to learn and more programs to maintain.”

To answer the first question, I have personally created scripts to do the following (to mention a few)

• Provide GUI interface representing DUT programmable features for test case writers
• Provide base stimulus file generation based on programmable features selected
• Provide auto generation of Register INCLUDE files for test cases
• Provide auto generation of Register indclude “.h” files for software
• RS-232 interface for dynamic configuration regressions on FPGA devices
• push button simulation regression
• simulation regression log viewing tool
• Random configuration and test case generation
• Register set and RAM test case generation
• Complicated Diff tool for specialized file diffing
• SUDOKO game
• TTBgen GUI
• assembler (for custom assembly code)

The GUI in the first point included the four proceeding points as part of it's functionality. This script grew to a size of 30000 lines in a matter of 5 years. In other words it evolved with the project from a simple test case generation tool to a hardware regression tool and even used in manufacturing. So the effort was seen as a good one, even though it took three to six months to become skillful in tcl/tk coding.

Test Case Generation:

The tool/script mentioned above was initially created to simplify the test case writing effort because the DUT was very complicated to program. There were many calculations to perform based on configuration settings. The output from the calculations produced values that had to be written to DUT registers and RAM. This effort proved to be too much for any team member to overcome and prompted the creation of the tool. The tool provided the user the ability to select and set functions and then view the resulting calculated values and register settings for the selected configuration. For each special functional block a “tab page” presented the registers and default values for user editing. Once the user was happy with all the values presented, a button could be hit to generate a stimulus file that could be used in simulations with the VHDL Test Bench Package. To re-assure the user that his configuration was what was wanted, a graphical representation of the configuration was also provided.

As we all know, register set definitions change as a design progresses. As suggested in the Variables blog post, a master file for holding register specifications are a good thing. As an aid to myself, I added some special function buttons on an Options “tab page” that enabled me to generate a variable DEFINE include file, for simulations. Also generated from the same master file were Register set test cases, RAM test cases and .h include files for software.

Of course during play time some of us program. I created a Sudoko program for fun. I learned how to use randomization, in tcl, through that effort. The Sudoko code, though ugly, does the job. I then proceeded to make it so the tool, at work, could randomize itself. The user could provide a “seed” and hit a button and a random configuration would be created. With a little more effort a RS-232 interface was created so that connection to the FPGA processor block could be done through a debug link. The randomly generated configurations were then be applied to the FPGA and each test was performed in real time. To help put this in perspective, 1024 random configurations could be tested on the FPGA in four hours. The same random configurations run in simulation, on one machine, would take 256 days. (assuming six hours per simulation) That is a huge time savings.

The above real life story demonstrates what can be achieved through the use of a scripting language. Not only were stimulus files generated but real time regressions were realized in the end. Each iteration of FPGA design had the 1024 random regression applied as a qualification for delivery to software.

When you are looking at the job of writing test cases, it may seem that the effort could never end. I have found the fastest way to reduce the effort is to create a test case generation program. The use of a GUI is recommended as it enables users to interact with a familiar medium. Presented by the GUI are all the functional buttons and knobs that can be set and adjusted in the DUT. Many, if not all, of the DUT functions can be represented by simple “check buttons” and fields for user interaction. To help get started, if you are interested in tcl/tk, feel free to use the TTBgen GUI tool as a template. Once familiar with language constructs, it is surprising how quickly an application can take shape.

Randomization:

Most modern scripting languages have a random function that can be used to produce randomization for your testing, through random test case generation. This can greatly reduce the time it takes to create test cases. As well, the quality of testing should be better when randomization is used.

For instance, if you are developing a CPU, a large set of random test cases would be nice to run in regressions. To create a large set of random test cases would take a person a long time to write. They would have to roll dice for each instruction to help them in getting some randomization included. If we consider a large set of test cases to be 200 each with 10,000 instructions, then there is many die rolls to be done. But if instead, the person took the time to learn and use a partner scripting language to generate those test cases for them, they gain twice. First the test cases can be generated much faster than can be written by hand and second, the person learned better how to use their partner language to assist them in their verification efforts. So now the question is not how to write all the test cases, but how to run them all in a decent time frame.

If your testing requirements lead you to have to create a complicated GUI, this in itself can be a blessing. The quickest way to get randomization is to provide a “Randomize” button and a “Seed” field. Make the tool randomize its own fields and then generate test cases from the random configuration. You will be surprised how quickly this can be done and how useful this will be.

If an FPGA is part of the design process, take advantage of it to assist in your over all testing. One thing to keep in mind is that, if you randomly generate configurations for FPGA regressions, be sure you can replicate those configurations in simulation. In a high paced development environment, you will most likely uncover some bugs while regressing on the FPGA. Being able to replicate the failing configurations in simulation will greatly enhance the debug effort.

I hope that it is now obvious how valuable a scripting language can be to your verification efforts.

I personally use TCL from ActiveState.

Sckoarn