CS315 Lab 7: OpenGL ES


Highlights of this lab:

This lab is an introduction to OpenGL ES programming on the iPod Touch and iPhone. The assignment is based on program that uses texture mapping on a thunderbird airplane.

Assignment:

Online OpenGL ES 2 Manual


Seminar Notes

"OpenGL® ES is a royalty-free, cross-platform API for full-function 2D and 3D graphics on embedded systems - including consoles, phones, appliances and vehicles. It consists of well-defined subsets of desktop OpenGL, creating a flexible and powerful low-level interface between software and graphics acceleration. OpenGL ES includes profiles for floating-point and fixed-point systems and the EGL™ specification for portably binding to native windowing systems. OpenGL ES 1.X is for fixed function hardware and offers acceleration, image quality and performance. OpenGL ES 2.X enables full programmable 3D graphics. OpenGL SC is tuned for the safety critical market.

"For programmable hardware: OpenGL ES 2.0 is defined relative to the OpenGL 2.0 specification and emphasizes a programmable 3D graphics pipeline with the ability to create shader and program objects and the ability to write vertex and fragment shaders in the OpenGL ES Shading Language. OpenGL ES 2.0 does not support the fixed function transformation and fragment pipeline of OpenGL ES 1.x."

Khronos.org OpenGL ES Overview

What all that means, is that programming OpenGL ES 2 is a lot like programming in Core Profile. You need to write shaders, load your vertex data into buffers and provide your own transformations. The Core Profile labs should port naturally to an OpenGL ES 2 environment. Sometimes OpenGL ES is made available as a Java object, as on Android platforms. In this case you would need to provide your own transformation library. Apple's iOS Objective-C platform is mostly compatible with C++, so you can port your code with relative ease.

A. Starting an OpenGL ES Project

You need XCode, the iPhone SDK and a Mac to write applications for the iPhone and Touch. If you have a Mac, you can download the iPhone SDK and XCode from http://developer.apple.com/iphone/. You need to register with Apple as a developer if you want XCode and access to their programmind documentation. You also need a bit of time - the download is large. The install is easy, but you should read through their instructions to be safe.

The iPhone SDK for XCode provides several templates for building iPhone applications and libraries. We will build our OpenGL ES project with one of these.

B. Tour of the Major Components

Objective-C and C++

As you explore the code you will notice that there are .h files and .m files. These are the extensions for the Objective-C language.

Although OpenGL ES is an established standard for embedded 3D that is available on many platforms, the way to access it is different on every platform. The popular Apple iPhone line of products presents a special problem for programmers new to Apple development. The default programming language for Apple UI products is Objective-C. Objective-C is a C based OOP language that is very different from C++. Its class definition and member calling syntax is very alien to those familiar with Java, C# or C++. Fortunately, if you are programming OpenGL ES, you can mostly ignore Objective-C because OpenGL ES is a C based library that is very similar to OpenGL and Objective-C is extremely compatible with C and C++.

To write and include C++ code in your iPhone programs you can signal to the compiler that you want to use the Objective-C++ flavour of the language. To do this simply change all .m files to .mm files.

The new constructs in Objective-C deal primarily with object oriented programming. The following is a quick primer so that you know what you are reading in the code today.

If you want to learn more about Objective-C programming for iOS, you can:

Source files

main.m - in the Supporting Files folder.
Its sole purpose is to house main() which starts memory management and hands execution over to the UI toolkit.
 
AppDelegate.h/m
In Cocoa, the UI toolkit used on the iPhone and Mac, some classes specialize their behaviour by handing jobs over to other classes called delegates. Delegates are supposed to implement functions that match certain prototypes. They are often used to handle UI events.

The AppDelegate class is designed in intercept application start, and stop events and send signals to our OpenGL ES view. In a typical application the delegate sets appropriate framerates for different application states.
 
GameViewController.h/m
This is where you will do your OpenGL magic. This controller responds to draw and interaction events. It has functions that correspond to draw, reshape and init. It also defines an update event. It extends the functionality of a specialized OpenGL ES view controller defined by GLKit called GLKViewController which controls an Embedded Apple GL view or EAGLview class. You are able to call EAGLview functions from here if necessary.

Selected GameViewController Class Members

setupGL
Performs a function similar to the init function we have used in our freeglut programs.
 
tearDownGL
Cleans up before the OpenGL context shuts down. Since freeglut does not always shut down cleanly, we have no equivalent in our code.
 
loadShaders, compileShader, linkProgram, validateProgram
These have the same role as Dr. Angel's InitShader function.
 
dealloc
The view's destructor. Calls tearDownGL, deallocates the rendering context and calls destructors for the view.
 
viewDidLoad
Indicates that the view is ready to use. This is the place to configure and initialize a rendering context.
context
Pointer to the Open GL ES Rendering Context for this class.
 
program
OpenGL pointer for a shader program.
 
drawInRect
This is where we do our OpenGL ES drawing.
 
update
This is an animation update function. If you wish to animate your code, you could send updates from here. Note the code for accessint the time since last update. It is also serves the role of reshape function. Note how it learns the current dimensions of the view.
 

C. OpenGL ES 2

Once you know where to put your OpenGL ES calls you need to know the calls to use. Fortunately, they are mostly the same as the OpenGL ones you already know. There are some differences, though. Here are a few you might run into:

You can study manuals or learn about these by trial and error. As you work on this week's lab I highlight the important differences needed to port our sample project to iOS. You can refer to the OpenGL ES 2 man pages for details on some of these changes.

Apple documents some details of OpenGL ES and its OpenGL ES management class, EAGL, in About OpenGL ES. If you are interested in OpenGL ES development on Android devices, take a look at their developer site. It has several GLES lessons complete with projects written in Dalvik/Java and even some sample code in C for the Android Native Development Kit (NDK). Since Java is not C++ and the NDK only supplies a limited set of headers AND makes it hard to load project resources, porting this week's sample code to the Android platform is, unfortunately, a Herculean task, but you can try porting your 2D art from lab 2 to one of their code samples instead of doing the exercise described below.


Lab Exercise

In this exercise you will port Dr. Angel's libraries, the shaders and the rendering code from the Thunderbird program to an iOS project.

Goals:

Instructions:

  1. Start with a new OpenGL ES iPhone Application.

  2. By default iPhone apps are written in Objective-C which is C based. To be able to mix C++ into your application you need to signal the compiler that you want to use Objective-C++. The way to do this is to use the .mm extension on your Objective-C files.
  3. Copy the following files from Thunderbird to your iPhone project folder, add them and rename all .cpp files to .mm files. (That's everything except the FreeImage folder, glut_shell.cpp, IMGLoader.cpp, IMGLoader.h and the GLEW framework.)

    Be sure to add the textures and shaders to the app's Copy Bundle Resources rule in Build Phases.

    Try building. You will get errors. You will fix them in the next several steps. Build again after each major step to update the errors that are reported.


  4. One of the errors happens because we didn't add IMGLoader. This was on purpose - rather than use FreeImage, I chose to rewrite IMGLoader around iOS's native image classes UIImage and CGImageRef.
  5. Many of the errors come because OpenGL ES requires different headers than OpenGL did. You are using Apple's iOS windowing system. You also do not need GLEW.
    Replace the GL, GLUT and GLEW includes and all associated preprocessor logic in the copied files with these:
    #include <OpenGLES/ES2/gl.h>
    #include <OpenGLES/ES2/glext.h>
    
    Make sure to remove and replace all other references to the old OpenGL and related headers. In particular you should check Angel.h, which is where they are collected. Any other files should be clear from compile error reports.


  6. Now that you have working OpenGL headers, recompile and remove error types that do not apply to OpenGL ES 2 from CheckError.h. You will know what they are because they will not be recognized by the compiler at compile time.


  7. Clean up functions in thunderbird.mm to remove code that is incompatible with OpenGL ES. The freeglut code that is superceded by Cocoa UI code was all in glut_shell.cpp, so it is already gone. All you need to do is change some GL calls.
  8. Vertex array objects don't exist in in plain OpenGL ES 2, but Apple provides an extension that replaces them.

    In OpenGL ES 2 glUniformMatrix does not do matrix transposing.

    OpenGL ES does not support BGR→RGB swizzling with glTexImage2D() so GL_BGR is not understood. Also, our texture loader should always provide GL_RGBA textures.

    At this point you should be getting a clean build, but the App's behaviour hasn't changed. We'll fix that by adding an instance of the thunderbirdScene class to the GameViewController and calling its member functions from the appropriate places.


  9. In your GameViewController.mm file:
  10. Now your code should build and run, but it will crash. The console, if you have it open, will say that the shaders can't be found or read.

    Resource files for iOS apps have complex paths that change from device to device. You will need special functions to learn this path for each file you want to load. The Objective-C function for this is a member of the application's main bundle called pathForResource:. Details for loading an image with Objective-C are here. This function is available from where we load shaders because we switched to Objective-C++. To use it:
  11. Now your shaders won't compile... there are minor things to fix in them.

    OpenGL ES's version of GLSL is similar to GLSL 1.2, but our shader is written for GLSL 1.5. You will need to change the following to make it work:
    Fix that stuff and you should be done!! Yay!!

Deliverables

Bring your completed work (ie. showing the thunderbird plane displayed in the iPhone Retina (3.5-inch) simulator) to the lab in two weeks time.

References