CS210 Lab: Computational Complexity and Performance Measurement

Prelab Questions:

For a review of relevant topics click here.

Lab Exercise:

Click the little computer above for a detailed description.
In the exercise, the students can add the library, which has been created in this lab, to existing code to test the runtime of three different search algorithms and three different sorting algorithms.

1. Computational Complexity and Performance Measurement

A routine's performance can be judged in many ways. In CS210, you have been/will be learning Big O. The idea is that you can estimate the routine's projected execution times as a function of the number of data items (N) that it manipulates as it performs its task. The results are estimates of the form O(N), O(LogN) and so on.

Big O allows us to group routines based on their projected performance under different conditions (best case, worst case, etc). Remember that Big O is only an estimate. It does not take into account factors specific to a particular environment, such the specific implementation, the type of computer system we are using, and the data that we are processing.

To determine how well or poorly a routine will perform in a particular environment, we need to evaluate the routine in that environment.

In "Part 1" of the laboratory exercise, you will measure the performance of three search routines:

• binarySearch()
• linearSearch()
• unknownSearch()
The first two search routines were discussed in CS115. For a review, click here

In summary, for the binary search, you are given a sorted array. The idea is to divide the array into two equal parts and narrow down the range of the array to be searched either to the low part of the array or high part of the array. We will continue narrowing down the range until the value is located, or until we realize that the value searched for is not in the array.

The linear search is also known as the sequential search. The idea is start with the first element, and sequentially search through the array until we find a match.

Before we can measure the performance of these routines, we must first develop a set of tools that allow us to measure the execution time.

The general approach will be the following:

• Get the current system time (startTime)
• Execute the routine
• Get the current system time (stopTime)
• Calculate the routine's execution time=stopTime-startTime
If the routine executes very rapidly, then the difference between startTime and stopTime may be too small for your computer system to measure. If that is the case, we can execute the routine several times (say n) and then divide the resulting time by the number of repetitions (n) as follows:
• Get the current system time (startTime)
• Execute the routine n times
• Get the current system time (stopTime)
• Calculate the routine's execution time = (stopTime-startTime)/n
To capture the startTime and stopTime, we will use a class called Timer which has the following member functions:
• start()
• stop()
• getElapsedTime()
To add some complexity to this lab, we will turn this Timer class into a library and use this library for the lab exercise at the end.

2. Creating a Library and Using it

Our first step will be to download the Timer class and a sample program just to try it out

Transfer three files to your local PC. Get the files from ftp://ftp.cs.uregina.ca/pub/class/210/ftp/Library

The three files are :

 complexity.cpp: This is the main program. timer.cpp: This is the implementation of the Timer module.  It can be compiled into a library once it is bug-free. timer.h: This is the header file of the Timer module

Store the three files where you can easily find them.

2.2 Compile the C++ programs

Using the procedures discussed in Lab 1 do the following:
• create a new project called labtst
• add the three files to this project
• build and execute this project
Let's have a look at this code:
• timer.h and timer.cpp are useful for calculating running time measurements. Because we may want to use the class Timer over and over again, we may want to put it in a library.

Our second step will be to create a library

2.3 What is a library?

A library is a collection of object files - .obj files similar to the .o files you saw in Unix. Object files can contain individual functions, collections of functions or an entire class implementation. Each object file is stored individually. When your program references a function or class contained in a library, it links the object it belongs to and adds its code to your program. This way you can have one big file to store related things but only the pieces that you actually use in your program are added to the executable file.

2.4 What are header files?

Each function or class defined in the C or C++ standard library has a header file associated with it. The header files that relate to the functions that you use in your programs should be included (using #include) in your program.
There are two reasons for this:

1. to get the data types that work with many functions or classes in the standard library. Your program must have access to these data types that are defined in the header file related to each function or class
2. to obtain the prototypes for the standard library functions

Interestingly one library may have more than one header file.

2.5 Why create your own Library

Libraries can be regarded as pre-compiled functions or classes that do not need to be re-compiled if other parts of your project change. If you have a "bug-free" module (a piece of self-contained code) that could be reused with other projects, you can compile this module to a library. After you link this library to your new project, you will then be able to use any of the functions or classes in this library as if they are defined locally in your project. One major advantage is that the functions or classes in the library will not be recompiled, which saves time compiling your entire project.

In our example, "timer" is a perfect candidate: it does not contain any bugs and we can use it again in other projects.

To use "timer" in a new project, we will have to first compile the C++ programs into a library, and then link this library to the new project.

2.6 Compile the C++ programs using a library.

In this program, timer.h and timer.cpp define and implement a time measuring class Timer. This class is independent of the kind of program it is measuring. In other words, we can reuse this Timer class to measure the running time of other programs. It naturally follows that once we make this class bug-free, we can compile it into a library.  To use this time-measuring class, we simply include the header file (i.e.timer.h) in the program and link the library to this program.

The following shows you how we do this:

Step 1. Create the library

• Start Microsoft Visual Studio
• Choose File→New→Project...
• A "New Project" Dialogue box will appear
• Expand the "Visual C++ Projects" folder and select the "Win32" sub folder
• From the "Templates" pane, select "Win32 Console Application"
• Name: mylibrary
• Take note of the location. You will need it later.
• Click OK
• The "Win32 Application Wizard" Dialogue box will appear (see the below picture)
• First, under Application Settings, choose Static library
• Then uncheck Precompiled header

Step 2. Add the necessary files to mylibrary and build the library

• Copy the files timer.cpp and timer.h into the mylibrary directory
• Choose Project→Add Existing Item...
• Highlight timer.cpp and timer.h

Step 3. Build mylibrary

• Select Build→Build mylibrary

The following message should be displayed in the output area:

```1>------ Build started: Project: mylibrary, Configuration: Debug Win32 ------
1>  timer.cpp
1>  mylibrary.vcxproj → C:\Users\????\Documents\Visual Studio 2012\Projects\mylibrary\Debug\mylibrary.lib
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
```

The library will be called mylibrary.lib and should be stored in the solution's Debug folder. You can see the path to it in the compiler's output. Go to the folder and look for it.

Step 4. Link the library to your project

• Choose File→New→Project...
• A "New Project" Dialogue box will appear
• Expand the "Visual C++ Projects" folder and select the "Win32" sub folder
• From the "Templates" pane select "Win32 Console Project"
• Name: complexity
• Click OK
• In the "Win32 Application Wizard" Dialogue box, select Application Settings, and check Empty project
• Copy complexity.cpp into the complexity directory (you do not need to add timer.cpp, since you have already compiled it into a library)
• Choose Project→Add Existing Item...
• Highlight complexity.cpp , then click Add
You now have to do three things to add the library that you created to the "link line".
1. Add mylibrary.lib to the Project Properties
• From the main menu, select: Project→Properties
• A "complexity Property Pages" Dialogue Box should be displayed (as shown below)
• Expand the Linker folder and select the Input subcategory
• Click on the "Additional Dependencies" on the right-hand side
• Add mylibrary.lib to the list of libraries.

2. Add the corresponding path (or location of this library) to the project. The path is used to locate the library files when the VC++ project is being built.
• In the "complexity Property Pages", under the expanded Linker folder select the General subcategory.
• Click on the "Additional Library Directories" on the right-hand side
• Add the path to the folder where mylibrary.lib is found.

3. Add the path where the timer.h file is to be found.
• Again, in the "complexity Property Pages" dialog, expand the C/C++ folder.
• Select the subcategory General and click on "Additional Include Directories".
• Add the path to the folder where timer.h is found.
4. Your project should build and run like this if you have followed the above steps correctly.

3. Lab Exercise

You will plot and analyse the execution times of three searching routines.

Step 1

• Get the files:
```Click here
To get the zipped files for this exercise
```
• Extract all of the files. There are three files used in this program:
• search.h contains the implementation of a set of searching routines.
• sort.h contains the implementation of a set of sorting routines (used to sort the array so that binary search will work).
• timesrch.cpp — the main program. Outputs the runtime of the three searching routines.

Your primary tasks for this exercise are:

1. Create a normal C++ Win32 Console Application using the three files
2. Include the timer library into this program.
3. Compare the execution times of the three searching routines.

Steps include:

• Adjust the project properties to link to "mylibrary".
Don't forget the three steps:
1. Add mylibrary.lib to the "Additional Dependencies"
2. Tell it where to find the mylibrary.lib
(The setting is in the Property Pages, under Linker→General)
3. Tell it where to find the timer.h file
(The setting is in the Property Pages, under C/C++→General)

• Compile and run the project. You will notice that the values are printed in the following format:
```
Enter the number of keys (must be at least 1000) : 1000

Search    | Number of|  Total Time  |   Time  per
Type     | Searches |     (s)      |   Search(s)
--------------|----------|--------------|--------------
linearSearch  |   1400000|    3.898e+000|    2.784e-006
binarySearch  |  12600000|    1.813e+000|    1.439e-007
unknownSearch |    700000|    3.866e+000|    5.523e-006
Press any key to continue . . .
```
The 2.087e-004 is equivalent to 2.087*10-4 or 0.0002087.

• Fill in the handout provided by your instructor (Step 2 - Step 5).
This involves plotting the time that it takes to run the three algorithms with different numbers of keys. In the end, you will have three lines for the three searches: linear, binary, and unknown.

Suggestion: The increments for the "Search time" axis have been left up to you. First, write everthing as 10-6 seconds. Then, create six tick marks along the y-axis: 3, 6, 9, 12, 15, and 18.