1

Oracle Database Applications

Database Connectivity

2

Outline• Motivation• Architecture• 7 steps• Sample program• Metadata

3

Motivation• Most popular form of database

system is the relational database system.

• Structured Query Language (SQL) is used among relational databases to construct queries.

• Applications need to query databases

4

Simple Database Application

APPLICATION

DBMS DB

Oracle

5

Multi-DatabasesAPPLICATION

DBMS 1

DBMS 2

DBMS 3

DB

DB

DB

6

Standard Access to DBAPPLICATION

DRIVER

MGR

DBMSDriver 1

DBMSDriver 2

DBMSDriver 3

DBMS 1

DBMS 2

DBMS 3

DB

DB

DB

7

ODBC ArchitectureApplication

Class1 Class2

Driver Manager

DriverType1 DriverType2 DriverType3

DataSource2DataSource1 DataSource3

ODBC

8

Open Database Connectivity (ODBC) Standard

• ODBC standard is an interface by which application programs can access and process SQL databases in a DBMS-independent manner. It contains:– A Data Source that is the database, its associated DBMS,

operating system and network platform– A DBMS Driver that is supplied by the DBMS vendor (Oracle)

or independent software companies– A Driver Manager that is supplied by the vendor of the O/S

platform (e.g., Windows/UNIX/Mac) where the application is running

– E.g Control Panel->Administrative Tools->Data Sources (ODBC) in Windows. You can use ODBC to access MS Access or even

Excel documents using the corresponding drivers.

9

ODBC Interface• System-independent interface to database

environment – requires an ODBC driver to be provided for

each database system from which you want to manipulate data.

• The ODBC driver manager bridges the differences between – The ODBC DBMS-independent interface and the

DBMS-dependent interface of the DBMS driver

10

An Example

Application DriverManager

Oracle Driver DB2 driver SQLServer driver

odbc standard

API

11

Java Support for Database Connectivity

• When applications written in Java want to access data sources, they use classes and associated methods provided by Java DBC (JDBC) API.

• JDBC is specified an “interface.”• An interface in Java can have many

“implementations.”– it provides a convenient way to realize many “drivers”

• JDBC can be an interface to an ODBC driver manager.

• Also it provides a JDBC API as a standard way to directly connect to common relational databases.

12

Application in Java

Application inJava

DriverManagerodbc standard

API

jdbc API

Oracle Driver DB2 driver SQLServer driver

13

Application in Java

Application inJava

Oracle DriverJdbc API

14

Java Support for SQL• Java supports embedded SQL.

– Embedded SQL in Java (SQLJ is one version of it) provides a standardized way for Java programs to interact with multiple databases, but at a higher level than the existing JDBCAPI.

– Embedded SQL allows connecting to a database by including SQL code right in the program.

– An Embedded SQL preprocessor converts the SQL statements to Java code at pre-compile time.   The preprocessor generates code that includes the driver functionality for direct connection to the  DBMS via JDBC.

• java.sql package and an extensive exception hierarchy.

15

Data Source and Driver• Data source: database created using

any of the common database applications.

• Your system should have the driver for the database you will be using.– E.g., MS SQL Server on a Windows system.

• There are a number of JDBC drivers available. http://industry.java.sun.com/products/jdbc/drivers

For oracle: http://jdbc.oracle.org/

16

JDBC Components• Driver Manager: Loads database drivers, and

manages the connection between application & driver.• Driver: Translates API calls to operations for a specific

data source.• Connection: A session between an application and a

driver.• Statement: A SQL statement to perform a query or

an update operation.• Metadata: Information about the returned data,

driver and the database.• Result Set : Logical set of columns and rows returned

by executing a statement.

17

JDBC Classes• Java supports DB facilities by providing

classes and interfaces for its components • DriverManager class• Connection interface (abstract class)• Statement interface (to be instantiated

with values from the actual SQL statement)

• ResultSet interface

18

java.sql• JDBC is implemented via classes in the

java.sql package– Supports SQL-2 entry level

• Define objects for:– Remote connection to DB– Executing query

• 8 interfaces to define objects:– Statement, CallableStatement,

PreparedStatement, DatabaseMetaData, ResultSetMetaData, ResultSet, Connection, Driver

19

DriverManager

Driver

Connection

Statement

ResultSet

• Load the driver• Define the Connection URL• Establish the Connection• Create a Statement object• Execute a query• Process the result• Close the connection

Seven Steps

20

Loading the Driver• Registering the driver directly

automatically:Class.forName(“org.oracle.Driver");

• Calling Class.forName, which automatically – creates an instance of the driver– registers the driver with the

DriverManager

21

Identifying Data Sources• Gives the required information for

making the connection to the database

• Specified using the URL format.<scheme>: <sub_scheme>:<scheme-specific-part>

• Example:jdbc:oracle://foo.itu.edu/mydatabasejdbc:oracle://localhost/testdb

22

Another Option• Create an instance of the driver and

register it with the Driver Manager:

Driver driver = (Driver) Class.forName("org.oracle.Driver").newInstance();

DriverManager.registerDriver(driver);

23

Connection• A Connection represents a session

with a specific database

• Within the context of a Connection, SQL statements are executed and results are returned

24

Connections• There can be multiple connections

to a database• A connection provides “metadata”,

i.e., information about the database, tables, and fields

• Connection object has methods to deal with transactions

25

Creating a Connection• Use getConnection on the Driver

Connection getConnection(String url, String user, String password)

• Connects to given JDBC URL with given user name and password

• Throws java.sql.SQLException• returns a Connection object

26

Creating a ConnectionConnection connection =

DriverManager.getConnection("jdbc:oracle://localhost/moviedb", "testuser", "mypassword");

27

Statements• Statement createStatement()

– returns a new Statement object– Used for general queries

• PreparedStatement prepareStatement(String sql)– returns a new PreparedStatement object– For a statement called multiple times with different

values (precompiled to reduce parsing time)• CallableStatement prepareCall(String sql)

– returns a new CallableStatement object– For stored procedures

28

Statements• A Statement object is used for

executing a static SQL statement and obtaining the results produced by it

• executeQuery is used for statements that return an output result

• executeUpdate is used for statements that need not return an output

29

Executing Queries and Updates

• ResultSet executeQuery(String) – Execute a SQL statement that returns a

single ResultSet • int executeUpdate(String)

– Execute a SQL INSERT, UPDATE or DELETE statement

– Used for CREATE TABLE, DROP TABLE and ALTER TABLE

– Returns the number of rows changed

30

Timeout• Use setQueryTimeOut to set a

timeout for the driver to wait for a statement to be completed

• If the operation is not completed in the given time, an SQLException is thrown

• What is it good for?

31

Cursor• What is the result of a query?• How can a database send the result

of a query through communication lines?

• The answer: using a cursor

32

Result Set• A ResultSet provides access to a table of

data generated by executing a Statement• Only one ResultSet per Statement can be

open at once• The table rows are retrieved in sequence• A ResultSet maintains a cursor pointing to

its current row of data• The 'next' method moves the cursor to the

next row– you can’t rewind

33

Working with ResultSet• boolean next()

– activates the next row– the first call to next() activates the first row– returns false if there are no more rows

• void close() – disposes of the ResultSet– automatically called by most Statement

methods

34

• Type getType(int columnIndex)– returns the given field as the given type– E.g., int getInt(5); string getString(3);– fields indexed starting at 1 (not 0)

• Type getType(String columnName)– same, but uses name of field– less efficient

• int findColumn(String columnName)– looks up column index given column name

Getting Values from Rows

35

isNull• In SQL, NULL means the field is

empty• Not the same as 0 or “”• In JDBC, you must explicitly ask if a

field is null by calling ResultSet.isNull(column)

36

Mapping Java Types to SQL TypesSQL type Java TypeCHAR, VARCHAR, LONGVARCHAR StringNUMERIC, DECIMAL java.math.BigDecimalBIT booleanTINYINT byteSMALLINT shortINTEGER intBIGINT longREAL floatFLOAT, DOUBLE doubleBINARY, VARBINARY, LONGVARBINARY byte[]DATE java.sql.DateTIME java.sql.TimeTIMESTAMP java.sql.Timestamp

37

Database Time• Times in SQL are nonstandard• Java defines three classes to help• java.sql.Date

– year, month, day• java.sql.Time

– hours, minutes, seconds• java.sql.Timestamp

– year, month, day, hours, minutes, seconds, nanoseconds

– usually use this one

38

Optimized Statements• Prepared Statements

– SQL calls that you make again and again– allows driver to optimize (compile) queries– created with Connection.prepareStatement()

• Stored Procedures– written in DB-specific language– stored inside database– accessed with Connection.prepareCall()

39

Prepared Statement Example

PreparedStatement updateSales; String updateString = "update COFFEES " +

"set SALES = ? where COF_NAME like ?";updateSales = con.prepareStatement(updateString); int [] salesForWeek = {175, 150, 60, 155, 90}; String [] coffees = {"Colombian", "French_Roast",

"Espresso","Colombian_Decaf","French_Roast_Decaf"};

int len = coffees.length;for(int i = 0; i < len; i++) {

updateSales.setInt(1, salesForWeek[i]); updateSales.setString(2, coffees[i]); updateSales.executeUpdate();

}

40

JDBC Class Diagram

41

Metadata• Connection:

– DatabaseMetaData getMetaData()• ResultSet:

– ResultSetMetaData getMetaData()

42

ResultSetMetaData• What's the number of columns in the

ResultSet? • What's a column's name? • What's a column's SQL type? • What's the column's normal max width in

chars? • What's the suggested column title for use in

printouts and displays? • What's a column's number of decimal digits? • Does a column's case matter? • and so on...

43

DatabaseMetaData• What tables are available?• What's our user name as known to the

database? • Is the database in read-only mode? • If table correlation names are supported

(association of a column with the table it comes from, when multiple columns of the same name appear in the same query - multi-table queries) , are they restricted to be different from the names of the tables?

• and so on…

44

Useful Methods of Metadata• getColumnCount• getColumnDisplaySize• getColumnName• getColumnType• isNullabale

Imagine the case where you want to print the result

45

Transaction Management• A transaction: a sequence of SQL statements• Transactions are not explicitly opened and

closed• Instead, the connection has a state called

AutoCommit mode• if AutoCommit is true, then every statement

is automatically committed• default case: true

46

AutoCommitConnection.setAutoCommit(boolean)• if AutoCommit is false, then every

statement is added to an ongoing transaction

• you must explicitly commit or rollback the transaction using Connection.commit() and Connection.rollback()

47

Connection Manager• For a large threaded database server, create a

Connection Manager object• It is responsible for maintaining a certain number of

open connections to the database• When your applications need a connection, they ask

for one from the CM’s pool• Why? Because opening and closing connections

takes a long time• Warning: the CM should always

setAutoCommit(false) when a connection is returned

48

More info• Go through the tutorial at:

http://developer.java.sun.com/developer/onlineTraining/Database/JDBC20Intro/

• http://java.sun.com/products/jdbc/learning.html

49

Optimizing JDBC

50

Goal–Illustrate techniques for optimizing JDBC API-based calls from the Java platform –Design better JDBC implementations–Recognize potential performance bottlenecks

51

Outline1) Why optimize?2) Basic API techniques.3) Design Strategies.4) Advanced Driver Tuning methods.

52

Why Optimize?• On average, a web request

performs 4 database queries.• Experience has shown that

database calls are typical performance bottleneck.

• Bad JDBC can overwhelm the database.

53

JDBC API

• SQL: “SELECT * FROM TABLE”• java.sql.PreparedStatement• java.sql.CallableStatement• Cache data on client.

Most Versatile

Most Optimized

54

JDBC APISQL Statements

• Most flexible• Least reliable• Must be recompiled in database for

each use

55

JDBC APIPreparedStatement

•Represents a precompiled SQL statement

•Can be used to efficiently execute statement multiple times

•Somewhat flexible – can create new ones as needed

56

JDBC APIPreparedStatement pstmt =

con.prepareStatement("UPDATE EMPLOYEES SET SALARY = ? WHERE ID = ?");

pstmt.setBigDecimal(1, 153833.00);pstmt.setInt(2, 110592);pstmt.execute();

57

JDBC APIjava.sql.CallableStatement

•Used to execute SQL stored procedures.

•Same syntax as PreparedStatement.•Least flexible.•Most optimized DB call.

58

JDBC APICache

• Keep data within client to reduce the number of round-trips to the database.

• Lesson: The less JDBC the better.

59

Basic Design TechniquesUse Database Connection Pool

• Don’t use DriverManager.getConnection() often. JDBC connections can take 0.5 to 2 seconds to create.

• Create Pool of Connections and reuse them.

• Necessity for any production system.

60

Basic Design TechniquesUse multi-threading with Connection Pooling to address network latency:

•Threads can issue queries over separate database connections.

•This improves performance to a point.

61

Basic Design TechniquesSingle-batch Transactions

Collect set of operations and submit transaction in one statement:

BEGIN TRANSACTIONUPDATE TABLE1... INSERT INTO TABLE2…DELETE TABLE3

COMMIT

62

Basic Design TechniquesSingle-batch Transactions

• DB obtains necessary locks on rows and tables, uses and releases them in one step

• Depending on transaction type, separate statements and commits can result in more DB calls and hold DB locks longer

63

Basic Design TechniquesDon’t have transaction span user input

• Application sends BEGIN TRAN and SQL, locking rows or tables for update

• Application waits for user to press key before committing transaction

64

Basic Design TechniquesSmart Queries

• Make queries as specific as possible• Put more logic into SQL statements• DB are designed to use SQL efficiently• Proper use of SQL can avoid

performance problems

65

Basic Design TechniquesSmart Query Ex: get employees in ENG deptInstead of: SELECT * FROM employees;SELECT * FROM dept;(and joining on Java application side)

Use database join:SELECT employees.* FROM employees E, dept D WHERE E.DEPTNO = D.DEPTNO AND D.DEPTTYPE = ‘ENG’;

66

Basic Design TechniquesSmart Queries

• Minimize ResultSet before crossing network

• Many performance problems come from moving raw data around needlessly

67

Basic Design TechniquesSmart Query Guidelines

• Use DB for filtering• Use Java for business logic• DB does filtering very well• DB business logic is poor

– At least very inconsistent between database vendors.

68

Basic Design TechniquesKeep operational data set small as possible • Move non-current data to other

tables and do joins for rarer historical queries

• Otherwise, index and cluster so frequently used data is logically and physically localized

69

Advanced Driver Tuning•Special options for each JDBC driver•No common standard•Improve performance by reducing round trips to the database.

•Example: Oracle driver performance extensions

70

Advanced Driver TuningOracle Performance Extensions

1) Row Prefetch2) Batch Updates

71

Advanced Driver Tuning1. Row Prefetch

•Use client-side buffers•Replace round trips by local manipulation of rows returned by query

•Use OracleStatement.setRowPrefetch()

72

Advanced Driver Tuning2. Batch Updates

•Reverse Prefetch•Does for data going to DB what prefetch does for data coming from it

•OraclePreparedStatement.setExecuteBatch

73

Advanced Driver Tuning2. Batch Updates

• Standard JDBC makes a trip to DB for each PreparedStatement.executeUpdate

• Batching: When number of queued requests reaches batch size, sends them to DB

74

SummaryOptimization Stages

1) Leverage the strengths of the DB2) Use the full range of java.sql API3) Design for Performance – Connection

Pools, Multi-Threading, etc.4) Implement driver performance

extensions