Using MACE to automate function/module testing

Noral Technologies Group - The Debugging Company

Using MACE to automate function/module testing

If part of your system has data processing functionality i.e. takes an input data set and transforms it into an output data set, it is usually possible to
provide automated module/function testing using MACE. The basic method is: 
  • Get the target CPU to a state where it is ready to process the data 
  • MACE reads the input data from a test data set (usually stored in a disk file) and install it into the target system 
  • Run the target CPU until it has processed the data 
  • MACE reads the results from the target system and writes them to a disk file 
  • Compare the newly generated file with a reference file of known correct output 
It is also possible to arrange the reading out of intermediate algorithm values during the processing of the data set. 

To achieve the above MACE needs to be able to control the target system and read/write memory or program variables within the target program.
MACE has many built in commands (function calls) for communicating with flex - and thus interacting with the target system. 

To see how this works in practice we have produced an example function test MACE macro ffttest.m which works in conjunction with the free Noral
68K simulator which is available for download from sim68k.zip. The MACE macro tests out the operation of the FFT function within the demo
program shipped with the simulator. The function void FFT (Complex *Data, Complex *Spectrum, Complex *W) in the ffti.c module
transforms an input data set contained in the Data[] array into an output data set in the Spectrum[] array. The complete function is listed below: 

0084 LOCAL void FFT (Complex *Data, Complex *Spectrum, Complex *W) 
0085 { 
0086     int K; 
0087     Complex m, fp, fq, Wnum; 
0088     unsigned p, q, L, J, Pof2, KStep, JLimit; 
0089     LongComplex modifier; 
0090 
0091     for (Looper = 0; Looper < NoOfElements; ++Looper) 
0092         Spectrum[BitReverse(Looper)] = Data[Looper]; 
0093 
0094     Pof2 = NoOfBits; 
0095     KStep = 1; 
0096     for (L = 1; L < = NoOfBits; ++L) { 
0097         JLimit = KStep-1; 
0098         KStep <<= 1; 
0099         --Pof2; 
0100         for (J = 0; J <= JLimit; ++J) { 
0101             Wnum = W[(J << Pof2) & (NoOfElements-1)]; 
0102             for (K = 0; K <= NoOfElements-KStep; K+=KStep) { 
0103                 p = J+K; 
0104                 q = p + JLimit + 1; 
0105                 fq = Spectrum[q]; 
0106                 fp = Spectrum[p]; 
0107                 modifier.R = (long int) fq.R * (long int) Wnum.R - 
0108                              (long int) fq.I * (long int) Wnum.I; 
0109                 modifier.I = (long int) fq.R * (long int) Wnum.I + 
0110                              (long int) fq.I * (long int)Wnum.R; 
0111                 m.R = (int) (modifier.R >> 14); 
0112                 m.I = (int) (modifier.I >> 14); 
0113                 Spectrum[q].R = fp.R - m.R; 
0114                 Spectrum[q].I = fp.I - m.I; 
0115                 Spectrum[p].R = fp.R + m.R; 
0116                 Spectrum[p].I = fp.I + m.I; 
0117             } 
0118         } 
0119     } 
0120 } 

Our test requirements are to: 

     run a known Data[ ] array through the FFT routine 
     log the values of the variables p and q at line 105 
     log the values in the Spectrum[ ] array at the end of the function - line 120 

To achieve this the MACE macro must: 

  1. Initialise the target system 
  2. Download the simulator demo program 
  3. Set a break at line 91 which invokes a MACE function which writes the Data[ ] array with the input data set 
  4. Set a break at line 105 which logs the values of the p and q variables 
  5. Set a break at line 120 which logs the values in the Spectrum[] array 
  6. Start execution 
Lets investigate the MACE macro which acheives the above: 

Initialize the target system

This is quite simple for the simulator environment but may end up quit complicated for an actual target system. It should prepare the target system so
that it can accept the program to be downloaded at the next step. This typically involves initialising chip selects or a DRAM controller and maybe the
disabling of a watchdog device which could disrupt the running of the test. Note that the routines which start with an '_' are functions built into MACE
which talk to flex. You can get detailed help on these functions from the MACE on-line help system. 

void SysInit() 
{ 
    unsigned char BVal; 
    /* 
    ** Connect to Flex debugger 
    */ 
    Connect(); 
    /* 
    ** Perform reset soft reset 
    */ 
    _restart(); 
    _reset(0); 
    /* 
    ** For our demo we dont want the simulated UART to cause the demo 
    ** program to wait for TxRDY so we force TxRDY always ready. 
    */ 
    BVal = 0xFF; 
    _setbytes("0xFF00",1,&BVal); 
    /* Nuke all existing execution breakpoints */ 
    _delallexecbreaks(); 
} 

Download the simulator demo program

This function downloads the simulator demo program and executes it until it reaches main(). It then brings up the window set saved as workspace #1. 

void LoadExample() 
{ 
    unsigned long MainBrk; 
    char strFile[128]; 

    /* 
    ** Load a file to debug 
    */ 
    strcpy(strFile,strFlexPath); 
    strcat(strFile,"\\examples\\68K\\sim68k\\test.abs"); 
    _loadfile(strFile,"IEEE695A,StripLeadingUS=true"); 
    /* 
    ** Set breakpoint on main. 
    */ 
    MainBrk = _setexecbreak("main"); 
    /* 
    ** Validate breakpoint has been set. 
    */ 
    if (!MainBrk) { 
        _flexstatusmessage("Failed to set breakpoint on main!"); 
        printf("Failed to set breakpoint on main!\n"); 
        return; 
    }
    /* 
    ** Start execution. 
    */ 
    _run(); 
    /* 
    ** Wait until breakpoint hit. 
    */ 
    while (_isexecuting()); 
    /* 
    ** Remove breakpoint. 
    */ 
    _delexecbreak(MainBrk); 
    /* 
    ** Load predefined window arrangement. 
    */ 
    _loadworkspace(1); 
} 

Set a break at line 91 which invokes a MACE function which writes the Data[ ] array with the input data set
Set a break at line 105 which logs the values of the p and q variables
Set a break at line 120 which logs the values in the Spectrum[ ] array
Start execution
These are all done by the RunTest() routine. The most interesting point here is the form of the breakpoint specification. Each breakpoint specification
string identifies a MACE macro (function) which is to be run when the breakpoint is reached. It is these functions which implement the core of the
data setup and logging procedure. 

void RunTest() 
{ 
    /* Make sure we are not executing */ 
    if (_isexecuting()) 
        _break(); 
    /* Nuke all existing execution breakpoints */ 
    _delallexecbreaks(); 
    /* Set the break which fill the Data[] array */ 
    hBrk[0] = _setexecbreak( "ffti.c#91, 1, Yes, @MACE(FFT_DataWrite())" ); 
    /* Set the break which logs ffti loop variables */ 
    hBrk[1] = _setexecbreak( "ffti.c#105, 1, Yes, @MACE(FFT_Log())" ); 
    /* Set the break which logs the ffti results */ 
    hBrk[2] = _setexecbreak( "ffti.c#120, 1, Yes, @MACE(FFT_SpectrumRead())" );
    /* Now start executing */ 
    _run(); 
} 

The FFT_DataWrite function is 'called' by flex when we hit the 1st line of code in the FFT routine (so that the Data parameter is in scope). This
routine reads the values for the Data[] array from a disk file data.vpi. To do this we use the 'viewport' feature of MACE. The viewport is in effect a
data transfer window for MACE. Using special routines (which all start with _vp) a MACE function can read/write data from/to the viewport
window. The added feature we use here is the ability to redirect the viewport input data to be read from a disk file. The content of the disk file is
exactly what would be typed into the viewport window if it was done manually from the keyboard. This routine expects 16 lines of input with each line
containing 2 comma separated integral values. Here is a link to the data.vpi file we used for the test data.vpi. Some points of note in this routine are: 
 

  • the ffttest.m file must be placed into the correct directory which will usually be c:\flex\sim68k\mace\examples\sim68k 
  • the data.vpi file must be placed into the correct directory which will usually be c:\flex\sim68k\examples\68k\sim68k 
  • _vpinputreset() must be called to reset the file pointer to the start of the input file 
  • the routine uses the flex expression evaluator _dbgexpr() to assign the Data[ ] array (which is an array of structures) entries. 
  • the routine prepares the viewport output redirection to the file viewport.vpo 
  • the routine returns 0 which tells flex to continue execution 


 int FFT_DataWrite() 
{ 
    int Idx; 
    int RVal, IVal; 
    char strFile[128]; 
    char strExpr[80]; 

    /* 
    ** Activate view port (data transfer area) 
    */ 
    _vpopen(); 
    /* 
    ** Form path to Data input file and re-direct viewport input 
    ** from the file. 
    */ 
    strcpy(strFile,strFlexPath); 
    strcat(strFile,"\\examples\\68K\\sim68k\\data.vpi"); 
    _vpinput(strFile, 1, 0); 
    _vpinputreset(); 
    /* 
    ** Read the 16 complex values into the target 
    */ 
    for (Idx=0; Idx < 16; ++Idx) { 
        _vpscanf("%d,%d", &RVal, &IVal); 
        sprintf(strExpr,"Data[%d].R=%d", Idx, RVal); 
        if (_dbgexpr(strExpr, (char*)0, (char*)0) == 0) { 
            printf("Failed to eval expression %s\n", strExpr); 
            return 1; /* Tell flex to break! */ 
        } 
        sprintf(strExpr,"Data[%d].I=%d", Idx, IVal); 
        if (_dbgexpr(strExpr, (char*)0, (char*)0) == 0) { 
            printf("Failed to eval expression %s\n", strExpr); 
            return 1; /* Tell flex to break! */ 
        } 
    } 
    /* 
    ** Prepare for subsequent data variable logging by forming the path 
    ** to Data output file and re-direct viewport output to the file. 
    */ 
    strcpy(strFile,strFlexPath); 
    strcat(strFile,"\\examples\\68K\\sim68k\\spectrum.vpo"); 
    _vpoutput(strFile, 1); 
    _vpoutputclear(); 
    /* Tell flex to continue execution */ 
    return 0; 
} 

The FFT_Log function is 'called' by flex each time the target program reaches line 105 of ffti.c. This routine writes the values of p and q to the
viewport which has been redirected to the spectrum.vpo file. Some points of note in this routine are: 

  • the routine uses flex's expression evaluator to get the values of p and q. The value is returned as a string into NVal and is written directly to the viewport. 
  • the routine returns 0 to tell flex to continue execution 
int FFT_Log() 
{ 
    char NVal[50]; 

    /* Get q */ 
    if (_dbgexpr("q", (char*)0, NVal) == 0) { 
        printf("Failed to eval q\n"); 
        return 1; /* Tell flex to break! */ 
    } 
    _vpprintf("q=%s ", NVal); 
    /* Get p */ 
    if (_dbgexpr("p", (char*)0, NVal) == 0) { 
        printf("Failed to eval p\n"); 
        return 1; /* Tell flex to break! */ 
    } 
    _vpprintf("p=%s\n", NVal); 
    /* Tell flex to continue execution */ 
    return 0; 
} 

The FFT_SpectrumRead function is 'called' by flex when we reach line 120 of ffti.c i.e. at the end of the FFT function. This routine writes the content
of the Spectrum[ ] array to the viewport which has been redirected to the spectrum.vpo file. Some points of note in this routine are: 

  • the routine uses flex's expression evaluator to get the values held in the Spectrum[ ] array (which is an array of structures). 
  • the routine returns 1 which tells flex it should remain broke and not re-execute. 
int FFT_SpectrumRead() 
{ 
    int Idx; 
    char strExpr[80]; 
    char strNVal[50]; 
    /* 
    ** Write the 16 complex values from the target 
    */ 
    for (Idx=0; Idx < 16; ++Idx) { 
        /* Get Spectrum[Idx].R */ 
        sprintf(strExpr,"Spectrum[%d].R",Idx); 
        if (_dbgexpr(strExpr, (char*)0, strNVal) == 0) { 
            printf("Failed to read %s\n", strExpr); 
            return 1; /* Tell flex to break! */ 
        } 
        _vpprintf("Spectrum[%d]={%s", Idx, strNVal); 
        /* Get Spectrum[Idx].I */ 
        sprintf(strExpr,"Spectrum[%d].I",Idx); 
        if (_dbgexpr(strExpr, (char*)0, strNVal) == 0) { 
            printf("Failed to read %s\n", strExpr); 
            return 1; /* Tell flex to break! */ 
        } 
        _vpprintf(",%s}\n", strNVal); 
    } 
    /* 
    ** Close viewport 
    */ 
    _vpclose(); 
    /* Tell flex to break - we are all done */ 
    return 1; 
}  
After the test is complete the file c:\flex\sim68k\examples\68k\sim68k\spectrum.vpo looks like: 
q=1 p=0 
q=3 p=2 
q=5 p=4 
q=7 p=6 
q=9 p=8 
q=11 p=10 
q=13 p=12 
q=15 p=14 
q=2 p=0 
q=6 p=4 
q=10 p=8 
q=14 p=12 
q=3 p=1 
q=7 p=5 
q=11 p=9 
q=15 p=13 
q=4 p=0 
q=12 p=8 
q=5 p=1 
q=13 p=9 
q=6 p=2 
q=14 p=10 
q=7 p=3 
q=15 p=11 
q=8 p=0 
q=9 p=1 
q=10 p=2 
q=11 p=3 
q=12 p=4 
q=13 p=5 
q=14 p=6 
q=15 p=7 
Spectrum[0]={277,0} 
Spectrum[1]={664,-7} 
Spectrum[2]={-247,-5} 
Spectrum[3]={-86,-2} 
Spectrum[4]={-220,0} 
Spectrum[5]={-45,-1} 
Spectrum[6]={-68,-1} 
Spectrum[7]={99,-1} 
Spectrum[8]={31,0} 
Spectrum[9]={100,1} 
Spectrum[10]={-67,1} 
Spectrum[11]={-44,0} 
Spectrum[12]={-220,0} 
Spectrum[13]={-83,3} 
Spectrum[14]={-246,5} 
Spectrum[15]={667,7} 

To run the test procedure

  1. Download and install the 68K Simulator (sim68k.zip). We will assume you install to c:\flex\sim68k. 
  2. Download ffttest.m and place it into the c:\flex\sim68k\mace\examples\sim68k directory 
  3. Download data.vpi and place it into the c:\flex\sim68k\examples\68k\sim68k directory 
  4. Start Flex Sim68K and press the MACE button on the toolbar 
  5. In MACE load the ffttest.m file 
  6. Press the Run button on MACE's toolbar 
You should see the demo program loaded, MACE opening the viewport window and displaying the logged data which is also written to
spectrum.vpo in the c:\flex\sim68k\examples\68k\sim68k directory. This file should look the same as the one presented above. 

Conclusion

By inspecting the demo program operation in conjunction with the ffttest.m MACE macro file you should gain enough insight to write your own
module/function test procedures. One point to notice which may cause you concern is the use of source line numbers when setting the execution
breakpoints. If the source was altered (which is the main reason you may wish to re-test the code!) the line numbers may change. Some possible
ways round this are: 
  • Try declaring a C label at the point you wish to log the data and use the label reference in the breakpoint. This may or may not be possible for you depending on how the compiler passes the label information to flex. 
  • Call an empty function at the point you wish to log the data. You can then use the function name in the breakpoint. Be aware than some compilers may optimise out empty functions! 

 
 

 
Return to Application Notes Menu Page top of page
Noral -The Company | More Information | Jobs, Vacancies | Distributors | Flex Tools | FREE 68000 Simulator | File Download | Press Releases | Application Notes | Home