jdbc (@postgresql). dbi 2008 huji-cs 2 java database connectivity jdbc (java database connectiveity)...

JDBC JDBC (@(@PostgreSQLPostgreSQL))

HUJI-CSHUJI-CSDBI 2008DBI 2008 2

Java Database ConnectivityJava Database Connectivity

• JDBC (Java Database Connectiveity) is an API (Application Programming Interface),– That is, a collection of classes and interfaces

• JDBC is used for accessing (mainly) databases from Java applications

• Information is transferred from relations to objects and vice-versa

– databases optimized for searching/indexing

– objects optimized for engineering/flexibility


JDBC ArchitectureJDBC Architecture

Java Application JDBC

Oracle

DB2

Postgres

OracleDriver

DB2Driver

Postgres Driver

These are

Java classes

Network

We will

use this one…


JDBC Architecture (cont.)JDBC Architecture (cont.)

• Java code calls JDBC library

• JDBC loads a driver

• The driver talks to a particular database

• An application can work with several databases by using all corresponding drivers

• Ideal: change database engines w/o changing any application code (not always in practice)

Application JDBC Driver


Seven StepsSeven Steps

• Load the driver

• Define the connection URL

• Establish the connection

• Create a Statement object

• Execute a query using the Statement

• Process the result

• Close the connection


Registering the DriverRegistering the Driver

• To use a specific driver, instantiate and register it within the driver manager:

Driver driver = new org.postgresql.Driver();DriverManager.registerDriver(driver);

Driver driver = new org.postgresql.Driver();DriverManager.registerDriver(driver);


A Modular AlternativeA Modular Alternative

• We can register the driver indirectly using

• Class.forName loads the given class dynamically

• When the driver is loaded, it automatically

– creates an instance of itself

– registers this instance within DriverManager

• Hence, the driver class can be given as an argument of the application

Class.forName("org.postgresql.Driver");Class.forName("org.postgresql.Driver");


An ExampleAn Example

// A driver for imaginary1Class.forName("ORG.img.imgSQL1.imaginary1Driver");// A driver for imaginary2Driver driver = new ORG.img.imgSQL2.imaginary2Driver();DriverManager.registerDriver(driver);//A driver for PostgreSQLClass.forName("org.postgresql.Driver");

imaginary1imaginary2

Registered Drivers

Postgres


Connecting to the DatabaseConnecting to the Database

• Every database is identified by a URL

• Given a URL, DriverManager looks for the driver that can talk to the corresponding database

• DriverManager tries all registered drivers, until a suitable one is found

• How is this done?


Connecting to the DatabaseConnecting to the Database

Connection con =

DriverManager.getConnection("jdbc:imaginaryDB1");

imaginary1imaginary2

Registered Drivers

Postgres

acceptsURL("jdbc:imaginaryDB1")?


The URLs in HUJI-CSThe URLs in HUJI-CS

In CS, a URL has the following structure:

The machine running PostgrSQL

The machine running PostgrSQL

jdbc:postgresql://dbserver/public?user=??

Your loginYour login

You can only access your own account!


Interaction with the DatabaseInteraction with the Database

• We use Statement objects in order to

– Query the db

– Update the db (insert, update, create, drop, …)

• Three different interfaces are used:

Statement, PreparedStatement, CallableStatement

• All are interfaces, hence cannot be instantiated

• They are created by the Connection


Querying with StatementQuerying with Statement

executeQuery returns a ResultSet object representing the query result (discussed later…)

String queryStr = "SELECT * FROM Member " +"WHERE Lower(Name) = 'harry potter'";

Statement stmt = con.createStatement();ResultSet rs = stmt.executeQuery(queryStr);


Changing DB with StatementChanging DB with Statement

String deleteStr = "DELETE FROM Member " +"WHERE Lower(Name) = 'harry potter'";

Statement stmt = con.createStatement();int delnum = stmt.executeUpdate(deleteStr);

• executeUpdate is for data manipulation: insert, delete, update, create table, etc.

– Anything other than querying!

• executeUpdate returns the number of rows modified (or 0 for DDL commands)


About Prepared StatementsAbout Prepared Statements

• Prepared statements are used for queries that are executed many times

• Parsed (compiled) by the DBMS only once

• Values of some columns are set after compilation

• Instead of values, use ‘?’ and setType methods

• Hence, prepared statements can be thought of as statements that contain placeholders to be substituted later with actual values


Querying with Querying with PreparedStatementPreparedStatement

String q = "SELECT * FROM Items " +"WHERE Name = ? and Cost < ?";

PreparedStatement pstmt=con.prepareStatement(q);

pstmt.setString(1, "t-shirt");pstmt.setInt(2, 1000);

ResultSet rs = pstmt.executeQuery();


Updating with Updating with PreparedStatementPreparedStatement

String dq = "DELETE FROM Items " +"WHERE Name = ? and Cost > ?";

PreparedStatement pstmt = con.prepareStatement(dq);

pstmt.setString(1, "t-shirt");pstmt.setInt(2, 1000);

int delnum = pstmt.executeUpdate();


StatementStatement vs. vs. PreparedStatementPreparedStatement: Be Careful!: Be Careful!

String val = "abc";PreparedStatement pstmt = con.prepareStatement("select * from R where A=?");pstmt.setString(1, val);ResultSet rs = pstmt.executeQuery();

String val = "abc";PreparedStatement pstmt = con.prepareStatement("select * from R where A=?");pstmt.setString(1, val);ResultSet rs = pstmt.executeQuery();

String val = "abc";Statement stmt = con.createStatement( );ResultSet rs = stmt.executeQuery("select * from R where A=" + val);

String val = "abc";Statement stmt = con.createStatement( );ResultSet rs = stmt.executeQuery("select * from R where A=" + val);

Are these the same? What do they do?


What can be Assigned to “?” What can be Assigned to “?”

• Will this work?

• No!!! “?” can only represent a column value (to enable pre-compilation)

PreparedStatement pstmt = con.prepareStatement("select * from ?");pstmt.setString(1, myFavoriteTableString);

PreparedStatement pstmt = con.prepareStatement("select * from ?");pstmt.setString(1, myFavoriteTableString);


PreparedStatement and SecurityPreparedStatement and Security• Suppose Google was implemented in JDBC

without a PreparedStatement. The main DB query might have been implemented like this:

Statement s;s.executeQuery("SELECT URL,Title from Internet " +"WHERE Content LIKE ‘%" + searchString + "%’");

• What would happen if a hacker searched for:aaaaa’ UNION SELECT Company AS URL, CreditCardNum AS Title FROM AdvertisingClients WHERE Company LIKE ‘

• This technique is known as SQL InjectionSQL Injection and is the main reason for using PreparedStatements


TimeoutTimeout

• Use setQueryTimeOut(int seconds) of Statement to set a timeout for the driver to wait for a query to be completed

• If the operation is not completed in the given time, an SQLException is thrown

• What is it good for?


ResultSetResultSet• ResultSet objects provide access to the

tables generated as results of executing Statement queries

• Only one ResultSet per Statement or PreparedStatement can be open at a given time!

• The table rows are retrieved in sequence– A ResultSet maintains a cursor pointing to its

current row – next() moves the cursor to the next row


ResultSet MethodsResultSet Methods• boolean next()

– Activates the next row

– First call to next() activates the first row

– Returns false if there are no more rows

– Not all of the next calls actually involve the DB

• void close() – Disposes of the ResultSet

– Allows to re-use the Statement that created it

– Automatically called by most Statement methods


ResultSet Methods (cont’d)ResultSet Methods (cont’d)• Type getType(int columnIndex)

– Returns the given field as the given type

– Indices start at 1 and not 0!

– Add the column name as a comment if it is known!

• Type getType(String columnName)– Same, but uses name of field

– Less efficient (but may not be your bottleneck anyway)

• Examples: getString(5), getInt(“salary”), getTime(…), getBoolean(…), ...

• int findColumn(String columnName)– Looks up column index given column name


ResultSet ExampleResultSet Example

Statement stmt = con.createStatement();ResultSet rs = stmt. executeQuery("select name,age from Employees"); // Print the resultwhile(rs.next()) { System.out.print(rs.getString(1) + ":"); System.out.println(rs.getShort("age"));}


Mapping Java Types to SQL TypesMapping Java Types to SQL TypesSQL TypeJava TypeCHAR, VARCHAR, LONGVARCHARStringNUMERIC, DECIMALjava.math.BigDecimalBITbooleanTINYINTbyteSMALLINTshortINTEGERintBIGINTlongREALfloatFLOAT, DOUBLEdoubleBINARY, VARBINARY, BYTEAbyte[]DATEjava.sql.DateTIMEjava.sql.TimeTIMESTAMPjava.sql.Timestamp


Null ValuesNull Values

• In SQL, NULL means the field is empty

• Not the same as 0 or “”!

• In JDBC, you must explicitly ask if the last-read field was null

– ResultSet.wasNull(column)

• For example, getInt(column) will return 0 if the value is either 0 or NULL!


Null ValuesNull Values

When inserting null values into placeholders of a PreparedStatement:

– Use setNull(index, Types.sqlType) for primitive types (e.g. INTEGER, REAL);

– For object types (e.g. STRING, DATE) you may also use setType(index, null)


Result-Set Meta-Data Result-Set Meta-Data

A ResultSetMetaData is an object that can be used to get information about the properties of the columns in a ResultSet object

An example: Write the columns of the result set

ResultSetMetaData rsmd = rs.getMetaData();int numcols = rsmd.getColumnCount();

for (int i = 1 ; i <= numcols; i++) System.out.print(rsmd.getColumnLabel(i)+" ");


Database TimeDatabase Time

• Times in SQL are notoriously non-standard

• Java defines three classes to help

• java.sql.Date– year, month, day

• java.sql.Time– hours, minutes, seconds

• java.sql.Timestamp– year, month, day, hours, minutes, seconds,

nanoseconds

– Usually use this one


Cleaning Up After YourselfCleaning Up After Yourself

Remember: close Connections, Statements,

Prepared Statements and Result Sets

con.close();stmt.close();pstmt.close();rs.close()


Dealing With ExceptionsDealing With Exceptions

An SQLException is actually a list of exceptions

catch (SQLException e) { while (e != null) {

System.out.println(e.getSQLState());System.out.println(e.getMessage());System.out.println(e.getErrorCode());e = e.getNextException();

}}


General SQL AdviseGeneral SQL Advise

• Take the time to carefully design your database before you start coding – this will save you time and frustration

• The same data can be organized in very different ways – When designing your DB and when deciding what your primary keys should be, always think about typical use cases and deduce from them which queries are most common and which queries must run fastest

• Add as many constraints as possible (NOT NULL, UNIQUE, etc…) – this will help you debug prevent data corruption even if you missed a bug

Transaction ManagementTransaction Management


Transactions and JDBCTransactions and JDBC

• Transaction:Transaction: more than one statement that must all succeed (or all fail) together– e.g., updating several tables due to customer purchase

• Failure− System must reverse all previous actions

• Also can’t leave DB in inconsistent state halfway through a transaction

• COMMIT = complete transaction

• ROLLBACK = cancel all actions


An ExampleAn ExampleSuppose that we want to transfer money from

bank account 13 to account 72:

PreparedStatement pstmt = con.prepareStatement("update BankAccount set amount = amount + ? where accountId = ?");

pstmt.setInt(1,-100); pstmt.setInt(2, 13);pstmt.executeUpdate();

pstmt.setInt(1, 100); pstmt.setInt(2, 72);pstmt.executeUpdate();

What happens if this update fails?


Transaction LifetimeTransaction Lifetime

• Transactions are not opened and closed explicitly

• A transaction starts on 1st (successful) command

– After a connection is established

– After the previous transaction ends

• A transaction ends when COMMIT or ROLLBACK are applied– Either explicitly or implicitly (see next 4 slides)


Committing a TransactionCommitting a Transaction

How do we commit?

• Explicitly invoking Connection.commit()

• Implicitly – After every query execution, if AutoCommit is true

– When the user normally disconnects (i.e., appropriately closes the connection)

– In some DBs: After invoking a DDL command (CREATE, DROP, RENAME, ALTER, …)


Automatic CommitmentAutomatic Commitment

• A Connection object has a boolean AutoCommit

• If AutoCommit is true (default), then every statement is automatically committed

• If AutoCommit is false, then each statement is added to an ongoing transaction

• Change using setAutoCommit(boolean)

• If AutoCommit is false, need to explicitly commit or rollback the transaction using Connection.commit() and Connection.rollback()


Rolling BackRolling Back

• Rolling Back: Undoing any change to data within the current transaction

• The ROLLBACK command explicitly rolls back (and ends) the current transaction

• ROLLBACK is implicitly applied when the user abnormally disconnects (i.e., without appropriately closing the connection)


Fixed ExampleFixed Example

con.setAutoCommit(false);try { PreparedStatement pstmt = con.prepareStatement("update BankAccount

set amount = amount + ? where accountId = ?");

pstmt.setInt(1,-100); pstmt.setInt(2, 13); pstmt.executeUpdate(); pstmt.setInt(1, 100); pstmt.setInt(2, 72); pstmt.executeUpdate(); con.commit();catch (SQLException e) { con.rollback(); …; }


Transaction IsolationTransaction Isolation

• How do different transactions interact?

• Does a running transaction see

uncommitted changes?

• Does it see committed changes?

• Don’t worry about this now – we will

discuss it in future lessons (see the

Appendix if you really can’t wait)

Managing Large ObjectsManaging Large Objects

In PosgreSQL-JDBC


LOBs: Large OBjectsLOBs: Large OBjects

• A database can store large pieces of data– e.g., images or other files

• Sometimes, two distinguished types:– CLOB: Character large object (a large

number of characters)

– BLOB: Binary large object (a lot of bytes)

• Actual data is not stored in the table, only a pointer (Object ID) to the data storage


LOBs in PostgreSQLLOBs in PostgreSQL

LOBs

911

53

24

113

MyImagesMyImages

NameDescriptioncontent

image1Human face53

image2Rabbit113

Create a new lob

Get an in-stream to Obj. 24 LOB LOB ManagerManager

LOB LOB ManagerManager

Get an out-stream Obj. 53


Storing LOBs (1)Storing LOBs (1)

• Given: A binary source (file, socket, etc.), readable through an InputStream object

• Goal: Store the content of the source in the table MyImages

CREATE TABLE MyImages (name VARCHAR(20) , content OID);

CREATE TABLE MyImages (name VARCHAR(20) , content OID);



1. Disable auto-commit (why???):

2. Obtain the LOB manager:

• Downcast con into org.postgresql.PGConnection

3. Create a new LOB

LargeObjectManager lobm = con.getLargeObjectAPI()

con.setAutoCommit(false)

int oid = lobm.create(LargeObjectManager.READ |

LargeObjectManager.WRITE)



4. Insert a new row, with the LOB’s oid

PreparedStatement pstmt = con.prepareStatement ("INSERT INTO MyImages VALUES (?, ?)");

pstmt.setString(1, imageName); pstmt.setInt(2, oid);pstmt.executeUpdate();

pstmt.close();



5. Fill the LOB with the input bytes

6. Finalize

LargeObject obj = lobm. open(oid, LargeObjectManager.WRITE);

int bytesRead = 0; byte[] data = new byte[4096];while ((bytesRead = iStream.read(data)) >= 0)

obj.write(data,0,bytesRead);

obj.close(); iStream.close(); con.commit();


Retrieving LOBs (1)Retrieving LOBs (1)

1. Obtain the LOB ID from the relevant row:

PreparedStatement pstmt = con.prepareStatement ("select bytes from MyImages where name = ?");pstmt.setString(1, filename);

ResultSet rs = pstmt.executeQuery(); rs.next(); int oid = rs.getInt(1);

rs.close(); pstmt.close();



2. Using oid, obtain a the corresponding LargeObject instance from the LOB Manager

• Downcast con into org.postgresql.PGConnection

LargeObjectManager lobm = con.getLargeObjectAPI(); LargeObject obj = lobm.open(oid,LargeObjectManager.READ);



3. Read the content of the large object (that acts as an input/output stream)

while ((bytesRead = obj.read(data,0,data.length)) > 0) oStream.write(data, 0, bytesRead);

obj.close(); oStream.close();

Appendix:Appendix: PostgreSQL PostgreSQL Command-Line InterfaceCommand-Line Interface


Entering the Command-Line UIEntering the Command-Line UI

psql -hdbserver public

Server running PostgreSQL

Database Name

Automatically enters your own part of the database (based on your login)

Recommendation:

>echo ”alias psql ’psql –hdbserver public’” >> ~/.aliases


A ScreenshotA Screenshot

You can now write SQL queries


Additional Useful CommandsAdditional Useful Commands

\dtList all tables

\d table-nameShow table definition

\dTList all types

\i sql-fileInclude SQL from file

\dlList all large objects

\lo_export oid fileExport a large object

\lo_import fileImport a large object

\qQuit


Transactions in Command LineTransactions in Command Line

• By default, the command-line session runs in AUTOCOMMIT mode – That is, commit is implicitly applied after every

command

• To use transactions, you need to explicitly specify the beginning and end of a transaction by the commands BEGIN and

• At the end of the transaction, you either apply COMMIT, ROLLBACK or END

Appendix: Appendix: Byte-ArraysByte-Arrays


Representing Byte ArraysRepresenting Byte Arrays

Use the SQL type BYTEA to hold byte arrays

CREATE TABLE MyBStrings (name VARCHAR(20) , content BYTEA);

CREATE TABLE MyBStrings (name VARCHAR(20) , content BYTEA);

(works with PostgreSQL, may not work with other DBMSs)


Storing Byte ArraysStoring Byte ArraysStore array b[] of bytes:

1. Prepare a row-inserting statement

2. Set the statement values:

3. Execute and close

PreparedStatement pstmt =con.prepareStatement ("INSERT INTO MyBStrings VALUES (?, ?)");

pstmt.setString(1, desiredName);pstmt.setBytes(2, b);

pstmt.executeUpdate();pstmt.close();


Retrieving Byte ArraysRetrieving Byte ArraysRetrieve bytes with the name “myBytes”:

1. Apply a selection query

2. Obtain the byte-array from the result

3. Close resources

PreparedStatement pstmt =con.prepareStatement (“SELECT content from MyBStrings where name=?"); pstmt.setString(1, "myBytes");ResultSet rs = ps.executeQuery();

rs.next();byte[] theBytes = rs.getBytes(1);

rs.close(); pstmt.close();

Appendix: Appendix: Transaction Transaction IsolationIsolation


Transaction IsolationTransaction Isolation

• How do different transactions interact?

• Does a running transaction see

uncommitted changes?

• Does it see committed changes?


Transaction Isolation LevelsTransaction Isolation Levels

• The isolation level determines the capabilities of a transaction to read/write data that is accessed by other transactions

• In PostgreSQL, two levels of isolation:1.READ COMMITTED (default)

2.SERIALIZABLE

• Each transaction determines its isolation level– Connection.setTransactionIsolation(int level)


Isolation Level in the Command-Line UIIsolation Level in the Command-Line UI

SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL READ COMMITTED;

SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL SERIALIZABLE;

Changing to read committed:

Changing to serializable:


READ COMMITED & SERIALIZABLEREAD COMMITED & SERIALIZABLE

• In principle, a query in PostgreSQL never reads uncommitted (i.e. dirty) data

• SERIALIZABLE: During the whole transaction, statements read only the changes that were committed by the time the transaction begun (and the changes made by the transaction itself)

• READ COMMITTED: A statement reads the data that was committed by the time the statement (not the transaction) begun


Update: Row-Level LockingUpdate: Row-Level Locking

• PostgreSQL disables one from updating a row that is updated by an uncommitted transaction– The second updating transaction is blocked until the

first one commits or rolls back

• If 2nd update is within READ COMMITTED, then 2nd update is done after 1st transaction commits or rolls back

• If 2nd update is within SERIALIZABLE, then 2nd update fails if 1st transaction commits; otherwise, it succeeds


Some DefinitionsSome Definitions

• Dirty reads:Dirty reads: A transaction reads data that is written by another, uncommitted transaction

• Non-repeatable reads: Non-repeatable reads: A transaction rereads data it previously read and finds that a committed transaction has modified or deleted that data

• Phantom reads:Phantom reads: A transaction re-executes a query returning a set of rows satisfying a search condition and finds that a committed transaction inserted additional rows that satisfy the condition


READ COMMITED vs. SERIALIZABLEREAD COMMITED vs. SERIALIZABLE

READ COMMITED READ COMMITED SERIALIZABLESERIALIZABLE

Dirty ReadsImpossibleImpossible

Non-repeatable Reads

PossibleImpossible

Phantom Reads

PossibleImpossible


What Happens Here (1)?What Happens Here (1)?

1.1. CREATE TABLE pairs (x INTEGER, y INTEGER);

2. 2. select * from pairs3.3. insert into pairs values(1,1)

6. 6. select * from pairs

8.8. COMMIT



8.8. COMMIT



9.9. COMMIT



9.9. COMMITT.1: R. COMMITTED

T.2: SERIALIZABLE




T.1: R. COMMITTED

T.2: SERIALIZABLE

2.2. insert into pairs values(1,1)

4.4. COMMIT5. 5. select * from pairs

9. 9. select * from pairs10. 10. COMMIT





6. 6. select * from pairs7.7. insert into pairs values(1,2)8.8. COMMIT
















T.1: SERIALIZABLE

T.2: SERIALIZABLE

Is it equivalent to any truly serial execution of the transactions?




2.2. insert into pairs values(1,1)3.3. COMMIT

5. 5. update pairs set y=3 where x=1






6. 6. select * from pairs7.7. COMMIT



T.1: SERIALIZABLE

T.2: SERIALIZABLE














T.1: R. COMMITTED

T.2: SERIALIZABLE

Appendix: Appendix: Changing Changing PasswordsPasswords


Changing Your PasswordChanging Your Password

Suppose that snoopy wants to change his password from “snoopy” to “snoopass”:

ALTER USER snoopy WITH PASSWORD 'snoopass';

jdbc (@postgresql). dbi 2008 huji-cs 2 java database connectivity jdbc (java database connectiveity)...

Documents

imaginary2 driver driver

driver class

driver manager

specific driver

postgresql slide

connection slide

registerdriverdriver

database connection