Introduction to Programming 3D Applications

CE0056-1

Lecture 15

Input and interaction in 3D Environments

Topics

Interaction Input devices Event-driven input Display lists Menus Picking Buffering Virtual Trackball

1963 Sketchpad The first system to explore pen-based

user-interfaces by Ivan Southerland (of Evans and …) broached field of HCI not graphics but important

OpenGL doesn’t support interaction directly limits portability not specifically graphics we’ll use GLUT for device interactions

Input devices

Physical vs. logical Physical

mouse, keyboard, etc. perspective of how they interact with system

Logical function perspective of what they send application higher-level

Input devices Logical examples

Output: when we use printf in C, the actual string may be output to a monitor or printer

Input: we get the position of a user’s cursor whether we use a mouse or a data tablet

Our application doesn’t care at high level Two categories, pointer and keyboard

pointers may be mice, pens, trackballs, etc. pointers do not return character codes can return relative or absolute positions

Input devices

Absolute-positioning (w.r.t. some fixed position) pen returns position w.r.t. tablet (settable) data glove always returns 3D position coords

Relative-positioning (w.r.t. current position) mouse pos. always begins where cursor is

Trigger & measure trigger: physical input on device (button, key) measure: what device returns to application (button,

state, x position, y position)

Input modes

Request & sample modes request_locator (device_id, &measure);

after trigger, wait until input received ex. get x,y pos. after right mouse button depressed

sample_locator (device_id, &measure); assumes trigger, get input from a queue ex. get current x,y position of mouse

limits user control of program which device to sample from? modal, user must specify context (not natural)

Input modes

Event mode - the most flexible input mode triggers generate events (events store data) measures are sent to

1) an event queue

program checks queue

events are examined and acted upon

2) a special-purpose function

program associates function with trigger up front

function is called with the event trigger generates

the function is called a callback

Event-driven input

Callback for display glutDisplayFunc(display)

Callback for mouse glutMouseFunc(mouse) void mouse (int btn, int state, int x, int y) {

if (btn == GLUT_LEFT_BUTTON && state == GLUT_DOWN) drawSquare(x,y);

if (btn == GLUT_MIDDLE_BUTTON && state == GLUT_DOWN) exit ();

}

Event-driven input

drawSquare() function use new mouse position to draw new square

drawSquare(int x, int y) {y=w*h-y;

glColor3ub(r, g, b);

glBegin(GL_POLYGON)

glVertex(x+size, y+size);

glVertex(x-size, y+size);

glVertex(x-size, y-size);

glVertex(x+size, y-size);

glEnd(); glFlush(); }

What are we doing here?What are we doing here?

500

(90, 100) (90, 400)

500

Account for different origins

Event-driven input

Callback for keyboard glutKeyboardFunc(keyboard);

Callback for display glutDisplayFunc(display); call in indirectly with glutPostRedisplay();

after changing parameters, attributes, etc. if nothing has changed, system will avoid drawing

Callback for idle glutIdleFunc(idle)

perform tasks when nothing else is happening

Event-driven input

Callback for window resize event glutReshapeFunc(reshape) What do we do in reshape(width, height)?

What if new window is bigger or smaller?Do we grow, shrink, or resize the image?

What if aspect ratio has changed?Do we similarly resize the image, distort it, or clip it out?

Use the new width and height to adjust clip rectangle and viewport sizes

gluOrtho2D (0.0, width, 0.0, height);glViewport (0, 0, width, height);

Clients/server Model

In general Servers perform tasks for clients

In OpenGL assumes the OpenGL program is the client assumes workstations are graphics servers that

provide display and input services to the OpenGL program

for example, the client can display its output on any networked server

Display lists

History refreshing the screen required special processing

via a display processor that output a display list that was stored display memory

the display list was rendered fast enough to avoid flicker, leaving the host computer free

there is no longer any distinction between host and display processor (unless we count special graphics processing hardware)

Display lists In immediate-mode,

we compute, rasterize, and display (cube.c) we re-compute to redisplay

In retained-mode, we store the results of our computation in a

display-list we send an instruction to redisplay disadvantages

require memory on server require overhead to create

Display list example #define Box 1 glNewList(Box, GL_COMPILE) glBegin(GL_POLYGON) glColor3f(1.0, 0.0, 0.0); glVertex2f(-1.0, -1.0); glVertex2f(1.0, -1.0); glVertex2f(1.0, 1.0); glVertex2f(-1.0, 1.0); glEnd(GL_POLYGON); glEndList();

Display list example glMatrixMode(GL_PROJECTION); for (i=1; i<5; i++) { glLoadIdentity(); gluOrtho2D(-2.0*i, 2.0*i, -2.0*i, 2.0*i); glCallList(Box); } Clip rectangle results (left, right, bottom, top) i=1: (-2.0, 2.0, -2.0, 2.0) i=2: (-4.0, 4.0, -4.0, 4.0) i=3: (-6.0, 6.0, -6.0, 6.0)

What if we want to change the color each time through?What if we want to change the color each time through?

What happens as clip rectangle grows?What if it was huge?

What happens as clip rectangle grows?What if it was huge?

Display lists

Create multiple lists glGenLists(number) generates multiple ids glCallLists displays multiple lists

Menus

Glut provides pop-up windows define entries link menu to mouse button define callback function for each menu entry

Menu example Menu creation is order-dependentMenu creation is order-dependent /*Create a menu named sub_menu with callback named size_menu, with two /*Create a menu named sub_menu with callback named size_menu, with two

entries*/entries*/

sub_menu = glutCreatMenu(size_menu);sub_menu = glutCreatMenu(size_menu);

glutAddMenuEntry (“incr. square size”, 2);glutAddMenuEntry (“incr. square size”, 2);

glutAddMenuEntry(“decr. square size”, 3);glutAddMenuEntry(“decr. square size”, 3);

/*Create a menu with callback named top_menu, with two entries*//*Create a menu with callback named top_menu, with two entries*/

glutCreateMenu(top_menu);glutCreateMenu(top_menu);

glutAddMenuEntry(“quit”, 1);glutAddMenuEntry(“quit”, 1);

glutAddSubMenu(“resize”, sub_menu);glutAddSubMenu(“resize”, sub_menu);

/*Make right mouse button trigger menu event*//*Make right mouse button trigger menu event*/

glutAttachMenu(GLUT_RIGHT_BUTTON);glutAttachMenu(GLUT_RIGHT_BUTTON);

Menu creation is order-dependentMenu creation is order-dependent /*Create a menu named sub_menu with callback named size_menu, with two /*Create a menu named sub_menu with callback named size_menu, with two

entries*/entries*/

sub_menu = glutCreatMenu(size_menu);sub_menu = glutCreatMenu(size_menu);

glutAddMenuEntry (“incr. square size”, 2);glutAddMenuEntry (“incr. square size”, 2);

glutAddMenuEntry(“decr. square size”, 3);glutAddMenuEntry(“decr. square size”, 3);

/*Create a menu with callback named top_menu, with two entries*//*Create a menu with callback named top_menu, with two entries*/

glutCreateMenu(top_menu);glutCreateMenu(top_menu);

glutAddMenuEntry(“quit”, 1);glutAddMenuEntry(“quit”, 1);

glutAddSubMenu(“resize”, sub_menu);glutAddSubMenu(“resize”, sub_menu);

/*Make right mouse button trigger menu event*//*Make right mouse button trigger menu event*/

glutAttachMenu(GLUT_RIGHT_BUTTON);glutAttachMenu(GLUT_RIGHT_BUTTON);

quitresize incr. square size

decr. square size

incr. square sizedecr. square size

Menu example You write the callback functionsYou write the callback functions /*top_menu callback checks for quit only*//*top_menu callback checks for quit only*/

void top_menu (int id) {void top_menu (int id) {

if (id == 1) exit();if (id == 1) exit();

}} /*sub_menu processes resize options *//*sub_menu processes resize options */

void size_menu (int id) {void size_menu (int id) {

if (id == 2) size = 2.0*size;if (id == 2) size = 2.0*size;

else if (id == 3 && size > 1) size = size /2.0;else if (id == 3 && size > 1) size = size /2.0;

else if (id == 3) printf(“At minimum square size”);else if (id == 3) printf(“At minimum square size”);

glutPostRedisplay();glutPostRedisplay();

}}

You write the callback functionsYou write the callback functions /*top_menu callback checks for quit only*//*top_menu callback checks for quit only*/

void top_menu (int id) {void top_menu (int id) {

if (id == 1) exit();if (id == 1) exit();

}} /*sub_menu processes resize options *//*sub_menu processes resize options */

void size_menu (int id) {void size_menu (int id) {

if (id == 2) size = 2.0*size;if (id == 2) size = 2.0*size;

else if (id == 3 && size > 1) size = size /2.0;else if (id == 3 && size > 1) size = size /2.0;

else if (id == 3) printf(“At minimum square size”);else if (id == 3) printf(“At minimum square size”);

glutPostRedisplay();glutPostRedisplay();

}}

Picking

User selection of object on screen In immediate mode, we compute, rasterize and

display. How does this present a problem for identifying objects on screen? we pick a pixel how to work backward from pixel to object?

Several solutions render each object to its own clip rectangle and viewport for every position in frame buffer we add an obj id to a lookup

table that is examined at each pick for every object we compute a bounding rectangle or box (or

extents)*

Picking

What if objects overlap? return list of objects, not just one

What if cursor is near but not on object? bounding box will accommodate some also, you can leave slack in accuracy when you check

position against object extents What if pick is in a virtual 3D environment?

we will cast a ray into the scene and check each object for an intersection, returning list

Buffering Single buffering

objects are always rendered into the same framebuffer, which is always being displayed delay of clearing of and re-drawing into frame-buffer

will cause flicker if re-draw takes longer than refresh or refresh and animation not synced

Double buffering keep two buffers (front and back) always display front, always render into back* swap front and back when rendering complete

Virtual Trackball

Following slides developed by Ed Angel

Professor of Computer Science, Electrical and Computer Engineering,

and Media ArtsUniversity of New Mexico

Physical Trackball

The trackball is an “upside down” mouse

If there is little friction between the ball and the rollers, we can give the ball a push and it will keep rolling yielding continuous changes

Two possible modes of operation Continuous pushing or tracking hand motion Spinning

A Trackball from a Mouse

Problem: we want to get the two behavior modes from a mouse

We would also like the mouse to emulate a frictionless (ideal) trackball

Solve in two steps Map trackball position to mouse position Use GLUT to obtain the proper modes

Trackball Frame

origin at center of ball

Projection of Trackball Position

We can relate position on trackball to position on a normalized mouse pad by projecting orthogonally onto pad

Reversing Projection

Because both the pad and the upper hemisphere of the ball are two-dimensional surfaces, we can reverse the projection

A point (x,z) on the mouse pad corresponds to the point (x,y,z) on the upper hemisphere where

y =222 zxr if r |x| 0, r |z| 0

Computing Rotations Suppose that we have two points that were

obtained from the mouse. We can project them up to the hemisphere to

points p1 and p2

These points determine a great circle on the sphere

We can rotate from p1 to p2 by finding the proper axis of rotation and the angle between the points

Using the cross product The axis of rotation is given by the normal to

the plane determined by the origin, p1 , and p2

n = p1 p2

Obtaining the angle The angle between p1 and p2 is given by

If we move the mouse slowly or sample its position frequently, then will be small and we can use the approximation

| sin | = ||||

||

21 pp

n

sin

Implementing with GLUT

We will use the idle, motion, and mouse callbacks to implement the virtual trackball

Define actions in terms of three booleans trackingMouse: if true update trackball

position redrawContinue: if true, idle function posts

a redisplay trackballMove: if true, update rotation

matrix

Example

In this example, we use the virtual trackball to rotate the color cube we modeled earlier

The code for the colorcube function is omitted because it is unchanged from the earlier examples

Initialization

#define bool int /* if system does not support bool type */#define false 0#define true 1#define M_PI 3.14159 /* if not in math.h */

int winWidth, winHeight;

float angle = 0.0, axis[3], trans[3];

bool trackingMouse = false;bool redrawContinue = false;bool trackballMove = false;

float lastPos[3] = {0.0, 0.0, 0.0};int curx, cury;int startX, startY;

The Projection Stepvoid trackball_ptov(int x, int y, int width, int height, float v[3]){

float d, a; /* project x,y onto a hemisphere centered within width, height , note z is up here*/

v[0] = (2.0*x - width) / width; v[1] = (height - 2.0F*y) / height; d = sqrt(v[0]*v[0] + v[1]*v[1]); v[2] = cos((M_PI/2.0) * ((d < 1.0) ? d

: 1.0)); a = 1.0 / sqrt(v[0]*v[0] + v[1]*v[1] +

v[2]*v[2]); v[0] *= a; v[1] *= a; v[2] *= a;}

glutMotionFunc (1)void mouseMotion(int x, int y){ float curPos[3], dx, dy, dz; /* compute position on hemisphere */ trackball_ptov(x, y, winWidth, winHeight,

curPos); if(trackingMouse) { /* compute the change in position on the hemisphere */ dx = curPos[0] - lastPos[0]; dy = curPos[1] - lastPos[1]; dz = curPos[2] - lastPos[2];

glutMotionFunc (2)if (dx || dy || dz) { /* compute theta and cross product */ angle = 90.0 * sqrt(dx*dx + dy*dy + dz*dz); axis[0] = lastPos[1]*curPos[2] – lastPos[2]*curPos[1]; axis[1] = lastPos[2]*curPos[0] – lastPos[0]*curPos[2]; axis[2] = lastPos[0]*curPos[1] – lastPos[1]*curPos[0]; /* update position */ lastPos[0] = curPos[0]; lastPos[1] = curPos[1]; lastPos[2] = curPos[2]; } } glutPostRedisplay();}

Idle and Display Callbacksvoid spinCube(){ if (redrawContinue) glutPostRedisplay();}

void display(){ glClear(GL_COLOR_BUFFER_BIT|

GL_DEPTH_BUFFER_BIT); if (trackballMove) {

glRotatef(angle, axis[0], axis[1], axis[2]);

}colorcube();

glutSwapBuffers();}

Mouse Callbackvoid mouseButton(int button, int state, int x, int y){

if(button==GLUT_RIGHT_BUTTON) exit(0);

/* holding down left button allows user to rotate cube */

if(button==GLUT_LEFT_BUTTON) switch(state) { case GLUT_DOWN:

y=winHeight-y; startMotion( x,y); break;

case GLUT_UP: stopMotion( x,y); break;

} }

Start Functionvoid startMotion(int x, int y){ trackingMouse = true; redrawContinue = false; startX = x; startY = y; curx = x; cury = y; trackball_ptov(x, y, winWidth, winHeight,

lastPos); trackballMove=true;}

Stop Function

void stopMotion(int x, int y){ trackingMouse = false; /* check if position has changed */ if (startX != x || startY != y)

redrawContinue = true; else {

angle = 0.0; redrawContinue = false; trackballMove = false;

}}

Quaternions

Because the rotations are on the surface of a sphere, quaternions provide an interesting and more efficient way to implement the trackball

Quaternions will be covered later