VHDL Test Bench: Simulation Log File Messages

This blog post will be about the VHDL Test Bench and messaging. This refers to the messages that are output to the simulation log file. For modern verification methods, messaging is an important feature and there is a messaging control structure for controlling what messages will be output. Messaging control is usually termed “Verbosity” and can have many levels of control. The current implementation of the VHDL Test Bench provides two levels of verbosity, always on and debug messages.

When it comes to messages provided by an environment, I see two distinct levels of messaging. One level is the simulation log messages that need to be generated to determine what test ran and if it passed or failed. The other level of messaging is debug. The debug messages are used to help debug the test case during test bench development and RTL bug tracing.

The pass / fail message in a log file is required. I have yet to see a regression that does not need to scan log files to find out which tests passed and which failed. Of course the log file scanning is done by an external script after all the simulations have been run.  The pass/fail message is an "always on" message.

Debug messaging can take several forms. One form of debug messaging is generated by the stimulus file commands. For instance a READ command could report the address read and the value returned during the read operation. Or a VERIFY command could report success (note) as well as failure (failure). For the above two examples every time a READ or VERIFY command is encountered in a stimulus file, messages will be output with details of the operation. Another form of debug messaging are those generated by BFMs or models. These kinds of messages are not directly controllable from stimulus file commands unless a specific implementation is done in the BFM. As an example, if a RAM is generated to be 29x32, (29 locations X 32 bits wide) it may also contain assertions that check that the address applied to the RAM is not greater than 29. This makes sense because the address bus will be capable of addressing 32 locations. Though your RTL will most likely not address the full range, it is likely that your BIST tester will. This has the potential to generate a HUGE amount of messages to the log file during BIST testing. In this case a way of disabling the checking in the RAM model should be provided / implemented.

So, what is all this talk about? Why is there a concern over messages? They go to the log file and are there if we need to look at them, right? My main beef with messages is that, while a message is being output, your simulation is not progressing. To me it is an efficiency issue. I would rather be simulating than be told that some value was read from some address, and never look have at it. I have worked with an environment that generated 2-3 G byte log files. This is ok if you like to spend several minutes waiting for your editor to load the file. (I'm not a VI guy) But this becomes a real problem when your 2000 tests cases cause your disk to become full. Bottom line is that messages waste time and use space. Messaging should be optimized to enable messages that are appropriate for the development phase they are running in.

The VHDL Test Bench provides the two levels of messaging as stated at the beginning.

Those messages that are required to be output, “always on”, can use the '” <text>' facility of the test bench package. This is the “Dynamic Text Strings” as detailed in the VHDL Test Bench users guide. This text string will be output when ever the command line is executed. I would personally use this for statements that a requirement is being tested. They should not be used in loops as this will cause many messages to be output, which may not have any real value. Also, failure messages should also be output with no means to turn them off. I am referring to an assert statement with a failure severity. The VERIFY commands I create use “severity failure” to terminate the test case and output a clear message that the test case failed. (When, Where, Why, What) The FINISH command is the default “passing test” termination command, but if you look closely it also uses an assert with severity failure.

For those messages that are considered debug messages, there should be a mechanism to turn them off. Once the test cases are finished, and you are regressing, debug messages should not be using up simulation time because the test cases should be working by then. The VHDL Test Bench package commands, MESSAGE_ON and MESSAGE_OFF, are in place to enable the control of debug messages generated from the stimulus file. In the bhv file, there is a boolean variable “messages” defined. The MESSAGE_ON & _OFF commands set this variable to false and true respectively. To use this, simply put the “message” variable in an assert statement that you want to control the output of. As an example look at the default “LOOP” command. The message variable is used to control messaging for each loop iteration. By default, the messages are turned off. In the end the test case should have no message control commands. When the test case is finished there should be no debugging needed so the commands to turn on and off messages can be removed. Also, the message commands can be used anywhere in the test case. This enables control over any section of the test case that is being debugged to generate messages by turning messaging on and off where needed.

If the VHDL Test Bench Package provided debug on/off verbosity level is not enough for your needs, the implementation of messages can be augmented. A verbosity variable can be added, and assigned a verbosity level based on an integer. This can then be used in the assert statement along with the messages variable to control different levels of messages. All modern verification methodologies, currently used today, have many levels of verbosity provided in their messaging system implementation.

As mentioned above, BFMs can be a source of messages. As a practice it is good to have a BFM able to generate messages. These can be in any form needed for debug or required messages. As an example of a required BFM message output, consider a protocol monitoring BFM. Some new feature has been added to the protocol and you want to make the BFM output a message stating when the new feature has been seen. This may be done this way because it would be more difficult to determine this from the stimulus file. The same BFM can output debug messages stating when packets are transferred across the interface it is monitoring. These debug type messages should have the ability to be turned off. This can simply be implemented in a register bit of the BFMs register set, as described in BFMs #1 Posting .

I have always found the messaging facilities of the VHDL Test Bench Package to meet my needs. I have also seen how other users use messages not knowing the full effect. One test case I got from a co-worker took 30 minutes to run, and looking at the output, there was a message for every DUT access. The message stated the address and data that was being written or read. There were about 10,000 accesses DUT register and RAM test, hence 20,000+ lines in the log file of messages. I do not like waiting for simulations, so I disabled the messages and the simulation run in under ten minutes. It was then that I realized the effect of messages on the simulation. Though I had always been one to minimize messages, I did not know they could have such a drastic effect on simulation time.

I hope this posting about messages, helps users create effective VHDL Test Bench environments.

Sckoarn

VHDL Test Bench: DO-254 FPGA Verification

The VHDL Test bench is well suited for DO-254 FPGA/ASIC verification efforts. For those that do not know, DO-254 is a certification that electronics have to pass to be put on an air plane. The simplicity and straight forwardness of the test bench implementation is exactly what the certification authorities like to see. The test bench is really a very small part of the over all certification effort. Usually the simulation effort is more to prove the design in another way besides physical demonstration. On one side, simulation verification alone is not enough to get DO-254 certification. On the other side, simulation enables much more exposure to design details than demonstration.

The scripting system, that the VHDL Test bench implements, enables scripting commands that are simple to read and understand for those not involved in details. This can make test cases easier to review. Test cases can be made to generate messages, in log files, when requirements are being tested. It is important to provide traceability. The message output facility of the test bench enables the ability to provide verbose log file output. The VHDL test bench can help simplify the review of the simulation effort. This can save time and instill confidence in the over all quality of the verification.

As usually the simulation effort is not the sole output used for certification, simulation tools are waived from having to be themselves certified. If this is the case, then the code that runs on the simulation platform and facilities it provides, are not reviewed. (simulation tools) This includes the VHDL Test Bench Package. Of course all the details will be stated in your PHAC (Plan for Hardware Aspects of Certification) document.

In the DO-245 project I was involved in, the certification was based mostly on the demonstration of a test plan on the target hardware. There was two separate test plans and requirements were traced in both hardware and simulation test plans. There were some requirements that were not verifiable in each of the test platforms. The simulation effort, though uncovering bugs along the development path, provided the code coverage that enabled dead code to be identified. This is probably the single most important output from the simulation effort, dead code identification. Since most FPGA/ASIC designs are initially a program, the identification and removal of dead code is seen as a major step to increasing reliability and hence safety.

Requirements traceability is another major part of the DO-254 verification effort. The VHDL test bench scripts can have comments added to state the requirements they cover. The test cases can be made to generate messages that state when requirements are being tested. As well as the results of the testing. There are several tools available that will enable the tracing of requirements to be automated. These tools search documents (test cases, log files, design & requirement specs) and provide a matrix that enables the tracing of a requirement through those documents. This also enables holes to be identified quickly. How requirements are traced is detailed in your HPAC. Try to define and use an automated process for requirements tracing. This will greatly reduce the effort of tracing for verification and reviews.

If your design is in VHDL, then the VHDL Test Bench is a perfect fit. The whole effort can be handled by any standard VHDL simulator. This also enables design and verification to share a test environment and reduce the over all effort expended on creating simulation verification environments. A single language and simple test environment enables the team to focus on the design.

The process around a DO-254 development effort, is significant. This process is in place to ensure all efforts are made to ensure safety. Anything that can make the process go smoother, is something that should be considered. I think that, with the right planning, the VHDL Test bench can make a positive contribution to that goal.

Sckoarn

VHDL Testbench : Call for stories

Hello everyone.

I have been busy with work and life over the past two years. This makes it difficult to find the time or the will to publish new articles. After a full day of coding and verification, I just do not have the drive to write and code in my spare time.

I am currently working with a consulting group called XtremeEDA. We are specialists in the field of verification and design of integrated circuits. Clients of XtremeEDA are those companies that produce silicon designs of all sizes and applications. Some of the clients of XtremeEDA are the biggest players in the semi-conductor industry. This includes EDA vendors.

The contracts I have worked on, this past couple of years, have been a VHDL DO-254 project and some smaller projects proving IP for large SOC designs. I plan to post more information about how the VHDL Testbench can be used for DO-254 verification efforts.

This post is mainly a call for readers to post up some information about how they used the VHDL Testbench for their verification efforts. With the number of pages having been read, above 10,000 there has to be a few that actually used the VHDL Testbench package for something. For those that did use the VHDL Testbench, I like to hear how it was used, problems encountered, solutions to the problems and any general description of usage. I am sure that future readers will find the stories interesting and possibly useful. Hearing from users may also inspire me to write more about VHDL Testbench usage.

NOTE: If you do post something, please do not post proprietary information. If you work for a company then you most likely know what this means. For those that do not, please do not post details about your “design” if you do not own it. For test bench users, you can describe everything except details about code produced in BFMs and instructions, unless you are the owner.

To post something, simplely add a comment to this post.  Once I reveiw it, as long as it is realated to this blog, it will be published.

Sckoarn

VHDL Testing: VHDL 2008 FIFO of type ....

This post is about how VHDL 2008 can be used to implement a FIFO or list of items of any type by implementing a “protected type”.

Many applications store or delay data transfers as part of the implementation. For instance, packet switching, video processing and Digital Signal Processing all usually have some delay element during the processing of the information. A packet switch may store hundreds of packets in order to meet traffic constraints. A video processing unit may store several frames of video data as it processes the video data stream. A DSP application may have delays as it processed blocks of data.

What ever the case is, you have supplied stimulus to the DUT and there is a delay before you get results out. Where do you store the input or the expected results? Wouldn't it be nice to include a time stamp in your input data so you could determine the time it took to process the data?

This can all be realized using VHDL 2008 protected types. A protected type can be used as a shared variable as well, so, knowing how to do this can save time and make life easier.

First step is to decide what kind of record to define that will be the FIFO or list item. This can include anything that you can define in a VHDL record type. For the example, the arr128x8 type will be associated with a FIFO type of fifo_data_type. The type that is going to be the FIFO or list item must include a pointer to the type. In the case of this example the FIFO item type is called fifo_item, defined as a record. The pointer is fifo_item_ptr is a type of pointer to a fifo_item type. The pointer to fifo_item type enables a linked list of fifo_items to be traversed. This implementation is called a single linked list. In order to implement a double link list, a prev_rec definition could be added to the fifo_item definition. The fifo_item type must be defined in a separate package. The pgen package used in the example packet generation BFM example, has been extended and put in a separate file. Included below:

------------------------------------------------------------------------------

-- First we start of with the definition of the packet array type

-- and the pack_out record for pins on the entity and comp.

-- The size of the whole system can be changed by changing the

-- array and record types.

library IEEE;

use IEEE.STD_LOGIC_1164.all;

package pgen is

type arr128x8 is array(0 to 127) of std_logic_vector(7 downto 0);

type fifo_data_array is array(0 to 127) of std_logic_vector(7 downto 0);

-- define the fifo_item type and access pointer

type fifo_item; -- the item

type fifo_item_ptr is access fifo_item; -- pointer to item

type fifo_item is record -- full definition of item

   data : fifo_data_array;

   next_rec : fifo_item_ptr;

end record;

type pack_out is record

   dout : arr128x8;

   drdy : std_logic;

   end record;

end package pgen;

Once the type has been defined in a package the protected type is defined in another package. For this example the package is simply called fifo_pkg. The example is presented as a pure implementation of a protected type in the fifo_pkg, but of course you could have many other items defined, including other protected types.

The first thing to define is the header portion of the package. It include the definition of the protected type and its impure functions and procedures.

use std.textio.all ;

library ieee ;

use ieee.std_logic_1164.all ;

use ieee.numeric_std.all ;

use ieee.math_real.all ;

use work.pgen.all; -- include the types, as needed by shared variables

package fifo_pkg is

   type tb_fifo is protected

     procedure push(data : fifo_data_array);

     impure function push(data : fifo_data_array) return integer;

     impure function pop return fifo_data_array;

     impure function fifo_depth return integer;

   end protected tb_fifo;

end fifo_pkg;

Some simple utilities are defined, push in both a procedure and function form, a pop function and a fifo_depth function. Basic FIFO functions. Note the “is protected”, “end protected” and “impure function” syntax.

The body section is there the implementation of the procedures and impure functions are done. The example code for the body is presented below:

package body fifo_pkg is

type tb_fifo is protected body

   variable fifo_ptr : fifo_item_ptr := null;

   variable fifo_cnt : integer := 0;

   -- push function

   impure function push(data : fifo_data_array) return integer is

     variable new_pkt : fifo_item_ptr;

     variable tmp_ptr : fifo_item_ptr;

   begin

     if(fifo_cnt = 0) then

       new_pkt := new fifo_item;

       fifo_ptr := new_pkt;

       -- copy the packet to the new space

       new_pkt.data := data;

       new_pkt.next_rec := null;

       fifo_cnt := 1;

     else

       new_pkt := new fifo_item;

       tmp_ptr := fifo_ptr;

       -- get to the end of the fifo

       while(tmp_ptr.next_rec /= null) loop

         tmp_ptr := tmp_ptr.next_rec;

       end loop;

       -- copy the packet to the new space

       new_pkt.data := data;

       new_pkt.next_rec := null;

       tmp_ptr.next_rec := new_pkt;

       fifo_cnt := fifo_cnt + 1;

     end if;

     return 1;

   end function push;

   -- push procedure

   procedure push(data : fifo_data_array) is

     variable new_pkt : fifo_item_ptr;

     variable tmp_ptr : fifo_item_ptr;

   begin

     if(fifo_cnt = 0) then

       new_pkt := new fifo_item;

       fifo_ptr := new_pkt;

       -- copy the packet to the new space

       new_pkt.data := data;

       new_pkt.next_rec := null;

       fifo_cnt := 1;

     else

       new_pkt := new fifo_item;

       tmp_ptr := fifo_ptr;

       -- get to the end of the fifo

       while(tmp_ptr.next_rec /= null) loop

         tmp_ptr := tmp_ptr.next_rec;

       end loop;

       -- copy the packet to the new space

       new_pkt.data := data;

       new_pkt.next_rec := null;

       tmp_ptr.next_rec := new_pkt;

       fifo_cnt := fifo_cnt + 1;

     end if;

   end procedure push;

   -- pop function

   impure function pop return fifo_data_array is

     variable data : fifo_data_array := (others => (others => 'U'));

     variable tmp_ptr : fifo_item_ptr;

     variable prev_ptr : fifo_item_ptr;

   begin

       case fifo_cnt is

         when 0 =>

           return data;

         when 1 =>

           fifo_cnt := 0;

           data := fifo_ptr.data;

           fifo_ptr := null;

           return data;

         when others =>

           tmp_ptr := fifo_ptr;

           fifo_ptr := tmp_ptr.next_rec;

           tmp_ptr.next_rec := null;

           fifo_cnt := fifo_cnt - 1;

           return tmp_ptr.data;

       end case;

   end function pop;

   -- fifo_depth function

   impure function fifo_depth return integer is

   begin

     return fifo_cnt;

   end function fifo_depth;

 end protected body tb_fifo;

end fifo_pkg;

At the top of the package is the definition of fifo_ptr and fifo_cnt. The fifo_ptr item is the pointer to the top of the linked list of items. The fifo_cnt integer is a count of how many items there are on the fifo.

The package is very simple and does not have many of the nice things like overloaded write and print functions, searching, indexing, deleting and replacing that could be implemented.

Of course to test this the VHDL Test Bench was used and the following stimulus commands were created.

Define just after the architecture statement  shared variable test_fifo : tb_fifo;

or it can be defined as a regular variable as part of the read_file process

-------------------------------------------------------------------------------------------

   elsif (instruction(1 to len) = "PUSH") then

     for i in 0 to test_data'high loop

       --v_dat_array(i) := std_logic_vector(conv_unsigned(v_randv.RandInt(0, 255),8));

       --test_data(i) := std_logic_vector(conv_unsigned(v_randv.RandInt(0, 255),8));

       test_data(i) := std_logic_vector(conv_unsigned(i, 8));

     end loop;

   test_fifo.push(test_data);

   print("fifo depth is: " & integer'image(test_fifo.fifo_depth));

   temp_int := test_fifo.push(test_data);

   print("fifo depth is: " & integer'image(test_fifo.fifo_depth));

------------------------------------------------------------------------------------------

   elsif (instruction(1 to len) = "POP") then

     test_data := (others => (others => 'X'));

     print("POPing fifo depth is: " & integer'image(test_fifo.fifo_depth));

     test_data := test_fifo.pop;

As well the packages have to be included in the tb_ent file. If randomization is used to generate the data, the synthworks Random package must be included.

And the stimulus file used to test the example FIFO implementation:

PUSH
PUSH

POP
POP
POP
POP
POP

PUSH
PUSH

POP
POP
POP
POP
POP

FINISH

The operation was observed by setting break points in the code and looking at the content and status as the simulation progressed.

The example is simple, and could easily be adapted to implement list functions and others. The actual “item” in the FIFO can be changed to include such things as message strings, time stamps, and any other VHDL type. The only draw back is that a FIFO of an unknown type can not be created. If the FIFO item definition is expected to change, some planning should go into the definition so that compatibility can be maintained in the future.

I have used linked lists in the past for storing packet data, but it was not easy. The VHDL Test Bench Package uses linked list techniques extensively. So the only new part to this presentation is the protected types and how you can associate functions and procedures to them. Makes the code that uses protected types more readable I think. As well, access the shared variables of protected types is controlled. I plan to use this FIFO in a checker BFM and in the model of the DUT in the expanded example currently being worked on. (slowly)

The FIFO example could be converted to a list implementation with some additions. For each instance of the example tb_fifo there will be an instance of fifo_cnt. For a list, an index may be needed so that a particular item can be found quickly or ordering can be achieved. Several other functions or procedures will be need to be added to search, re-order and manipulate the list of items. An index field would be added to the record, as it is record specific. Where as a FIFO ID string would be added in the protected body section of the protected type definition, one ID string per fifo. A function or procedure will have to be created that enables the ID string to be set. This is like the fifo_cnt field in the above example code.

Hope that helps some get more acquainted with VHDL 2008 and how it can help your verification efforts.

Sckoarn

VHDL Test Bench Package: ttb_gen_plus_2B Bug Fix

ttb_gen_plus_2B has been updated with a bug fix.
The Beta version of ttb_gen_plus had a re-write when I first released it, a couple months back. The parsing was updated to include multi-pin definitions on the same line, like the real syntax allows. I personally do not use that syntax, I only define one pin per line in an entity definition. So there was quite a bit of code change, and I did not test one case. The bug was that if you have an inline comment in the entity definition such as PIN_NAME : std_logic; -- inline comment, the comment would be considered pins and create incorrect output.

The bug was fixed by stripping off trailing comments from the code lines as they are read from the file. One little case I just did not consider nor test for. The new version of ttb_gen_plus_2B is v2.02. It can be downloaded from here ttb_gen_plus_B2.

If you are using the Beta version of the VHDL test bench package, and old version of ttb_gen_plus was included with that package. To take advantage of the bug fix you will have to download the newest version as linked above.

If you have any problems with or suggestions for the ttb_gen_plus tool, please add a comment to this post or email me.

Sckoarn

VHDL Test Bench Package: A Partner Scripting Language

Using a partner language.

This blog entry will show how a “partner” language can be of great assistance to the verification effort. I am specifically referring to the many free scripting languages that are available, such as tcl/tk, perl, python, java ... I currently use tcl/tk as my partner language. I chose tcl because it is used by many tool vendors that provide simulators for HDL simulation. Many poohoo the choice of TCL, but what ever gets the job done is good. I found the GUI part of tcl/tk most interesting. The ability to whip together a small GUI that can do huge amounts of work, yet be intuitive to use, has increased my productivity. Specially because the GUI “script” is so easy and quick to put together.

This blog entry will also tie together topics mentioned in the Variables and Randomization posts.

In the following text, the word “script” refers to partner language code. The words “test case” and “stimulus file”, refer to test bench scripts that the VHDL Test Bench Package would parse.

One might ask, “What can a partner language do for me? After all, it is more code and yet another language to learn and more programs to maintain.”

To answer the first question, I have personally created scripts to do the following (to mention a few)

• Provide GUI interface representing DUT programmable features for test case writers
• Provide base stimulus file generation based on programmable features selected
• Provide auto generation of Register INCLUDE files for test cases
• Provide auto generation of Register indclude “.h” files for software
• RS-232 interface for dynamic configuration regressions on FPGA devices
• push button simulation regression
• simulation regression log viewing tool
• Random configuration and test case generation
• Register set and RAM test case generation
• Complicated Diff tool for specialized file diffing
• SUDOKO game
• TTBgen GUI
• assembler (for custom assembly code)

The GUI in the first point included the four proceeding points as part of it's functionality. This script grew to a size of 30000 lines in a matter of 5 years. In other words it evolved with the project from a simple test case generation tool to a hardware regression tool and even used in manufacturing. So the effort was seen as a good one, even though it took three to six months to become skillful in tcl/tk coding.

Test Case Generation:

The tool/script mentioned above was initially created to simplify the test case writing effort because the DUT was very complicated to program. There were many calculations to perform based on configuration settings. The output from the calculations produced values that had to be written to DUT registers and RAM. This effort proved to be too much for any team member to overcome and prompted the creation of the tool. The tool provided the user the ability to select and set functions and then view the resulting calculated values and register settings for the selected configuration. For each special functional block a “tab page” presented the registers and default values for user editing. Once the user was happy with all the values presented, a button could be hit to generate a stimulus file that could be used in simulations with the VHDL Test Bench Package. To re-assure the user that his configuration was what was wanted, a graphical representation of the configuration was also provided.

As we all know, register set definitions change as a design progresses. As suggested in the Variables blog post, a master file for holding register specifications are a good thing. As an aid to myself, I added some special function buttons on an Options “tab page” that enabled me to generate a variable DEFINE include file, for simulations. Also generated from the same master file were Register set test cases, RAM test cases and .h include files for software.

Of course during play time some of us program. I created a Sudoko program for fun. I learned how to use randomization, in tcl, through that effort. The Sudoko code, though ugly, does the job. I then proceeded to make it so the tool, at work, could randomize itself. The user could provide a “seed” and hit a button and a random configuration would be created. With a little more effort a RS-232 interface was created so that connection to the FPGA processor block could be done through a debug link. The randomly generated configurations were then be applied to the FPGA and each test was performed in real time. To help put this in perspective, 1024 random configurations could be tested on the FPGA in four hours. The same random configurations run in simulation, on one machine, would take 256 days. (assuming six hours per simulation) That is a huge time savings.

The above real life story demonstrates what can be achieved through the use of a scripting language. Not only were stimulus files generated but real time regressions were realized in the end. Each iteration of FPGA design had the 1024 random regression applied as a qualification for delivery to software.

When you are looking at the job of writing test cases, it may seem that the effort could never end. I have found the fastest way to reduce the effort is to create a test case generation program. The use of a GUI is recommended as it enables users to interact with a familiar medium. Presented by the GUI are all the functional buttons and knobs that can be set and adjusted in the DUT. Many, if not all, of the DUT functions can be represented by simple “check buttons” and fields for user interaction. To help get started, if you are interested in tcl/tk, feel free to use the TTBgen GUI tool as a template. Once familiar with language constructs, it is surprising how quickly an application can take shape.

Randomization:

Most modern scripting languages have a random function that can be used to produce randomization for your testing, through random test case generation. This can greatly reduce the time it takes to create test cases. As well, the quality of testing should be better when randomization is used.

For instance, if you are developing a CPU, a large set of random test cases would be nice to run in regressions. To create a large set of random test cases would take a person a long time to write. They would have to roll dice for each instruction to help them in getting some randomization included. If we consider a large set of test cases to be 200 each with 10,000 instructions, then there is many die rolls to be done. But if instead, the person took the time to learn and use a partner scripting language to generate those test cases for them, they gain twice. First the test cases can be generated much faster than can be written by hand and second, the person learned better how to use their partner language to assist them in their verification efforts. So now the question is not how to write all the test cases, but how to run them all in a decent time frame.

If your testing requirements lead you to have to create a complicated GUI, this in itself can be a blessing. The quickest way to get randomization is to provide a “Randomize” button and a “Seed” field. Make the tool randomize its own fields and then generate test cases from the random configuration. You will be surprised how quickly this can be done and how useful this will be.

If an FPGA is part of the design process, take advantage of it to assist in your over all testing. One thing to keep in mind is that, if you randomly generate configurations for FPGA regressions, be sure you can replicate those configurations in simulation. In a high paced development environment, you will most likely uncover some bugs while regressing on the FPGA. Being able to replicate the failing configurations in simulation will greatly enhance the debug effort.

I hope that it is now obvious how valuable a scripting language can be to your verification efforts.

I personally use TCL from ActiveState.

Sckoarn

The VHDL Test Bench Package: Randomization

Randomization ... a hot topic in the ASIC/FPGA verification world.

In VHDL, the IEEE math_real package, currently provides a method to generate random numbers through the uniform procedure. Getting a random number from uniform is a multi-step process and is not convenient to use. In the past I have used a LFSR construct coded in a procedure to generate random numbers. In stead of home cooking your own random number generation system or directly using uniform, I strongly recommend you look here:  SynthWorks Downloads   Specifically the random package RandomPkg_2_0. This free download, of source code and documentation, is a very capable package for generating random numbers. Besides the package content, the package also demonstrates some advanced VHDL 2008 coding techniques. The rest of this post will assume that the reader has reviewed or plans to review the SynthWorks' RandomPkg documentation and is familiar with the package and its definitions of randomization.

The VHDL Test Bench Package does not hinder the implementation of randomization. In fact, randomization may remove the need for scripts altogether. Taking randomization to the fullest, you may find that the tb_bhv file just becomes a container for your randomization objects and BFMs and no script is needed to make your simulations work. Randomization can be viewed from four different “levels”. With level one being simple randomization scripting instructions to enable a general randomization facility in scripts. Level two is the BFMs randomizing their output under the control of the scripting system, i.e. the stimulus bus. Level three is the randomization through randomly generated scripts. Finally at level 4, the system is completely controlled by an object that is able to generate random transaction based interaction with the BFMs. This post will cover the level 1 and 2 implementations and leave the level 3 and 4 for future posts.

The most obvious randomization within the scripting system would be to create a RAND_VAR instruction. Like the EQU_VAR (equate variable) instruction, provided by the test bench template, a RAND_VAR instruction could be created to randomize the contents of a variable. This variable could then be used as the address or data field in a write instruction, for instance. I am sure an implementer can come up with several other randomization type instructions for use in scripting. I have found that randomization be better done within a BFM.

There is an issue with random number generation that has to be addressed by the implementation. If for instance, we want to randomly write, random data, to a RAM as well as read back and verify that data, you will have to implement a more complex random number generation system. To re-state the problem, we want to randomize the address and data for both write and read operations. Lets also assume that we want to start reading and testing after four writes have been done. (This is done to offset the reads from writes, possibly uncovering some functional bug.) This can be done in one of two ways.

First, the write address and data could be stored in a FIFO construct, and reused for the read operations. This implementation causes the implementer to create two random numbers that have different random sequences. This is because the restrictions on address generation may not be the same as the data generation. If the address is sixteen bits wide and the data is only eight bits wide, the constraints on the two numbers being generated will be different. The address will be constrained to a number that fits in sixteen bits where the data generation will be constrained to fit within eight bits. The two numbers could be generated from a single random number stream, but that makes them dependent on each other. For instance, if I change the constraint on the address generation, that will effect the data values generated. The solution is to have more than one random number stream and in this case two. The reason for the two different random number streams, is because you may want to randomize both independent of each other. So, for one randomly generated address stream, you can generate many different random data streams.

Second, four different random number streams could be implemented and used to create a test to randomly test the RAM. The write and read data random variables could be initialize with the same seed and a different seed used for the write and read data.

Confused yet? A review of the SynthWorks' Random_Pkg documentation may help. I will attempt to make it clear here as well. For each random number stream that you want to have concurrently generating random numbers, you need to have a variable of type RandomPType defined. The RAND_VAR instruction will have to be expanded to include an index to random number stream to be used for the random operation. The RAND_VAR instruction should also have, as parameters, the max and min values to constrain generation within. This makes the simple RAND_VAR instruction much more complicated to implement in a script as it requires the script writer to understand and manage random number generation streams. If possible, the implementer should try to avoid complication in the scripts and move it into a BFM.

In a level 2 implementation, as defined above, the randomization is done within a BFM. This can hide complication from the script system. For items to be generated randomly a variable for each can be defined giving complete sequence independence for all fields.

Lets take an ATM signalling BFM as an example. The packet fields include GFC, VPI, VCI, PTI, CLP, HEC and data. One of the modes of operation for this BFM is random generation for any combination of fields. Through the stimulus access port, registers can be written to set the BFM to operate in different modes as well as provide random generation limits. The test case writer now only has to set up the register set for the configuration and not worry about the random number sequences. The sequences of random numbers will be controlled by the implementation of the BFM randomization system.

A simple example of random generation in a BFM is presented in the packet_gen example. This is part of the current Beta VHDL Test Bench Package download:  Beta Test Bench Package

In the coming weeks, I will be posting a full set of BFM's for an environment to test a simple self threading switch. This will enable example code of a complete randomization system to be presented, with enough complexity to take the example into a master BFM implementation. I am currently working on coding this example DUT and test bench. Until the example is released, there is enough information in this post to keep some busy playing with randomization.

Sckoarn