Testing
[What's pAVR?]

Modules
group	Bugs and debugging
group	FPGA prototyping

---

Detailed Description

Testing strategy

When testing a certain entity, the following testing strategy was adopted:

• embed that entity into a larger one that also includes all other ingredients needed for a real-life simulation of the tested entity. Typical such `other ingredients' are RAMs and multiplexers.
• run custom VHDL tests that test as much as possible of the functionality of the device under test. Extreme cases are the first situations to be tested.

Two kinds of tests were conducted on pAVR:

• every module of pAVR was separately tested as described in the testing strategy above.
• pAVR as whole was tested as described in the testing strategy above.

Testing pAVR modules

Each pAVR module was separately tested.
The particular tests carried out are presented below, grouped by the entities under test:

• utilities defined in `std_util.vhd'
  The associated test file is `test_std_util.vhd'.
  The utilities defined in `std_util.vhd' here are:
  • type conversion routines often used throughout the other source files in this project
  • basic arithmetic functions
  • sign and zero-extend functions
    Both are tested in `test_std_util.vhd'.
    Extreme cases and typical cases are considered.
    
  • vector comparision function
    Tested in `test_std_util.vhd'.
    Extreme cases and typical cases are considered.
    
• ALU
  The associated tests are defined in `test_pavr_alu.vhd'. They consist of checking the ALU output and flags output for all ALU opcodes, one by one, for all of these situations:
  • carry in = 0
  • carry in = 1
  • additions generate overflow
  • substractions generate overflow
  There are 26 ALU opcodes to be checked for each situation.
• Register File
  The associated tests are defined in `test_pavr_register_file.vhd'.
  The following tests are done:
  • read all ports, one at a time
    • read port 1 (RFRD1)
    • read port 2 (RFRD2)
    • write port (RFWR)
    • write pointer register X (RFXWR)
    • write pointer register Y (RFYWR)
    • write pointer register Z (RFZWR)
  • combined RFRD1, RFRD2, RFWR
    They should work simultaneousely.
  • combined RFXWR, RFYWR, RFZWR
    They should work simultaneousely.
  • combined RFRD1, RFRD2, RFWR, RFXWR, RFYWR, RFZWR
    That is, all RF ports are accessed simultaneousely. They should do their job.
    However, note that the pointer registers are accessible for writting by their own ports but also by the RF write port. Writing them via pointer register write ports overwrites writing via general write port. Even though concurrent writing could happen in a perfectly legal AVR implementation, AVR's behavior is unpredictible (what write port has priority). We have chosen for pAVR the priority as mentioned above.
• IO File
  The associated tests are defined in `test_pavr_io_file.vhd'.
  The following tests are performed on the IOF:
  • test the IOF general write/read/bit processing port.
    Test all opcodes that this port is capable of:
    • wrbyte
    • rdbyte
    • clrbit
    • setbit
    • stbit
    • ldbit
  • test the IOF port A.
    Port A is intended to offer to pAVR pin-level IO connectivity with the outside world.
    Test reading from and writing to Port A.
    Test that Port A pins correctly take the appropriate logic values (high, low, high Z or weak high).
  • test Timer 0.
    • test Timer 0 prescaler.
    • test Timer 0 overflow.
    • test Timer 0 interrupt.
  • test External Interrupt 0.
    Test if each possible configuration (activation on low level, rising edge or falling edge) correctly triggers External Interrupt 0 flag.
• Data Memory
  The tests defined in `test_pavr_dm.vhd' are simple read-write confirmations that the Data Memory does its job.

Testing the pAVR entity

pAVR as a whole was tested by building an upper entity that embedds a pAVR, its Program Memory and some multiplexers. Those multiplexers are meant to give Program Memory control to the test entity (for properly setting up Program Memory contents) or to pAVR (while pAVR is actually being monitored as it executes intructions from the Program Memory).

The binary file that will be executed by pAVR during the test is automatically loaded into the Program Memory using an ANSI C utility, TagScan. The test entity has a number of tags spread over the source code, as comments. The TagScan utility reads the binary file to be loaded, scans the test file, and inserts VHDL statements into the properly tagged places. These statements load the Program Memory using its own write port. This way of initializing the Program Memory seems more general (and surely more interesting) than using file IO VHDL functions.
The TagScan utility is also used for other purposes. For example, for inserting a certain header in all source files. It is heavily used as a general preprocessor.

Testing pAVR as a whole actually means designing and running binaries that put pAVR on extreme situations.
The following tests are done:

• Interrupts
  This exercises pAVR interrupt handling.
  All interrupts are tested.
  The associated peripherals (Port A, Timer 0 and External Interrupt 0) are put in a variety of conditions.
  Results:
  tbd
• General test
  This is a hand-written assembler source that is meant to be assembled and run on pAVR.
  It exercises each of pAVR instructions, one by one.
  It tries to put pAVR in most difficult situations, for each instruction. For example, it exercises:
  • concurrent stalls
  • stalls combined with 32 bit instructions
  • stalls combined with intructions that change the instruction flow
  • control hazard candidates (stress the Program Memory Manager and the Stall and Flush Unit)
  • data hazard candidates (stress the Bypass Unit)
  Results:
  Passed OK. The verification consisted of checking each instruction, each intermediate result and each relevant intermediate internal state.

  Assembler	Clocks	Instructions	CPI
  avrasm32, by Atmel	667	361	1.85

• Sieve
  Sieve of Eratosthenes; finds the the first 100 prime numbers.
  Written in ANSI C.
  Results:

  Compiler	Clocks	Instructions	CPI
  avr-gcc, O0	12170	8851	1.37
  avr-gcc, O3	11946	8824	1.35

• TagScan
  Exercises string manipulating routines.
  Written in ANSI C.
  Results:

  Compiler	Clocks	Instructions	CPI
  tbd	tbd	tbd	tbd

• C compiler
  Written in ANSI C.
  Results:

  Compiler	Clocks	Instructions	CPI
  tbd	tbd	tbd	tbd

• Waves
  Simulates waves on the surface of a liquid.
  Written in ANSI C.
  Uses floating point numbers (observation: the avr-gcc compiler seems to take about 200 pAVR clocks per floating point operation).
  A mesh of only 5x5 points is considered, and only 5 iterations are done. Bigger values make the simulation unacceptably long on the available computer.
  Checking the result is done by converting the array of 25 floats into a scaled array of 25 chars, copying these chars from Data Memory (by hand), constructing a 3D image of the result, and comparing it to a reference 3D image.
  Results:
  Passed OK. As expected, the chars array to be tested exactly matches the reference array.

  Compiler	Clocks	Instructions	CPI
  avr-gcc	209,175	122,236	1.71

---