Heap-File Page

Minibase Project 1

Spring 2009

TA: Yeo, Yeon-miProfessor: Whang, Kyu-Young

This document is supplementary document that was created by referring Minibase Project of Univ. of Wisconsin

Table of Contents

Ø IntroductionSimple DBMS Architecture

MINIBASE

Ø Heap-File Page (HFPage)Heap-File Page Data Structure

Heap-File Page Interface

Error Handling

Ø How to Start ?Get codes, Configure Environments, Compile, Running

Submit

1

Ø IntroductionSimple DBMS Architecture

MINIBASE

Ø Heap-File Page (HFPage)Heap-File Page Data Structure

Heap-File Page Interface

Error Handling

Ø How to Start ?Get codes, Configure Environments, Compile, Running

Submit

Architecture of a DBMS

Query Optimization and ExecutionRelational Operators

(Sort, Join, etc.)Files and Access Methods

(File, Index, etc.)

Queries

2

Files and Access Methods

(File, Index, etc.)Buffer Management

Disk Space Management

DB

Minibase

Ø Relational Database Management System (RDBMS)made by Wisconsin University for educational purpose

Ø Independent implementation of each component is possible

No pain to implementing whole DBMS

Suitable to understand the internal architecture of DBMS

Ø ComponentsParser / Optimizer / Query Processor/ Heap file / B+-tree index / Buffer pool manager / Disk space management system

3

Ø Relational Database Management System (RDBMS)made by Wisconsin University for educational purpose

Ø Independent implementation of each component is possible

No pain to implementing whole DBMS

Suitable to understand the internal architecture of DBMS

Ø ComponentsParser / Optimizer / Query Processor/ Heap file / B+-tree index / Buffer pool manager / Disk space management system

Heap-File Page (HFPage)

Ø Simple Heap-File Page StructureHeap-File Page consists of page informations, data area and slot array.

Data area in a Heap-File Page is always compacted.

We assume that the supported record data type is only character.

Ø OperationsInsert / Delete a record in a heap-file page

Get a first / next record id

Get the data / pointer of a record

Get some informations

4

Ø Simple Heap-File Page StructureHeap-File Page consists of page informations, data area and slot array.

Data area in a Heap-File Page is always compacted.

We assume that the supported record data type is only character.

Ø OperationsInsert / Delete a record in a heap-file page

Get a first / next record id

Get the data / pointer of a record

Get some informations

Class HFPage (1/4)const int INVALID_SLOT = -1;const int EMPTY_SLOT = -1;

class HFPage {protected:

struct slot_t {short offset;short length; // equals EMPTY_SLOT if slot is not in use

};static const int DPFIXED = sizeof(slot_t) + 4 * sizeof(short) + 3 * sizeof(PageId);

short slotCnt; // number of slots in useshort usedPtr; // offset of first used byte in data[ ] short freeSpace; // number of bytes free in data[ ]short type; // an arbitrary value used by subclasses as neededPageId prevPage; // backward pointer to data pagePageId nextPage; // forward pointer to data pagePageId curPage; // page number of this pageslot_t slot[1]; // first element of slot array.char data[MAX_SPACE - DPFIXED];

5

const int INVALID_SLOT = -1;const int EMPTY_SLOT = -1;

class HFPage {protected:

struct slot_t {short offset;short length; // equals EMPTY_SLOT if slot is not in use

};static const int DPFIXED = sizeof(slot_t) + 4 * sizeof(short) + 3 * sizeof(PageId);

short slotCnt; // number of slots in useshort usedPtr; // offset of first used byte in data[ ] short freeSpace; // number of bytes free in data[ ]short type; // an arbitrary value used by subclasses as neededPageId prevPage; // backward pointer to data pagePageId nextPage; // forward pointer to data pagePageId curPage; // page number of this pageslot_t slot[1]; // first element of slot array.char data[MAX_SPACE - DPFIXED];

HFPage Structure

Page Informations

Slot Array

Data Area

short slotCnt;

short usedPtr;short freeSpace;

short type;

PageId prevPage;PageId nextPage;PageId curPage;

short length; short offset; slot[0] slot[1]

6

Data Area

offset from data[0]

data[offset] points tothe data of record

Class HFPage (2/4)public:

void init(PageId pageNo); // initialize a new page

void dumpPage(); // dump contents of a page

PageId getNextPage(); // returns value of nextPage

PageId getPrevPage(); // returns value of prevPage

void setNextPage(PageId pageNo); // sets value of nextPage to pageNo

void setPrevPage(PageId pageNo); // sets value of prevPage to pageNo

PageId page_no() { return curPage;} // returns the page number

// inserts a new record pointed to by recPtr with length recLen onto

// the page, returns RID of record

Status insertRecord(char *recPtr, int recLen, RID& rid);

// delete the record with the specified rid

Status deleteRecord(const RID& rid);

7

public:

void init(PageId pageNo); // initialize a new page

void dumpPage(); // dump contents of a page

PageId getNextPage(); // returns value of nextPage

PageId getPrevPage(); // returns value of prevPage

void setNextPage(PageId pageNo); // sets value of nextPage to pageNo

void setPrevPage(PageId pageNo); // sets value of prevPage to pageNo

PageId page_no() { return curPage;} // returns the page number

// inserts a new record pointed to by recPtr with length recLen onto

// the page, returns RID of record

Status insertRecord(char *recPtr, int recLen, RID& rid);

// delete the record with the specified rid

Status deleteRecord(const RID& rid);

State change in Page when insertRecord( ) was called (case 1)

short offset;short length = EMPTY_SLOT;

slot entry and record dataof inserted record

use empty slot entry

empty slot entry

used slot entryused slot entry

8

used slot entry

State change in Page when insertRecord( ) was called (case 2)

slot entry and record dataof inserted record

use new slot entryall slot entry is used

9

State change in Page when deleteRecord( ) was called

short offset;short length = EMPTY_SLOT;

slot entry and record dataof the record that will be deleted

the length of the record slot entry that will be deleted should be

EMPTY_SLOT

10

record data after the record data thatwill be deleted be compacted

offset of this slot entry should be modified

Class HFPage (3/4)

// returns RID of first record on page// returns DONE if page contains no records. Otherwise, returns OKStatus firstRecord(RID& firstRid);

// returns RID of next record on the page// returns DONE if no more records exist on the pageStatus nextRecord (RID curRid, RID& nextRid);

// copies out record with RID rid into recPtrStatus getRecord(RID rid, char *recPtr, int& recLen);

// returns a pointer to the record with RID ridStatus returnRecord(RID rid, char*& recPtr, int& recLen);

// returns the amount of available space on the pageint available_space(void);

// Returns true if the HFPage is has no records in it, false otherwise.bool empty(void);

11

// returns RID of first record on page// returns DONE if page contains no records. Otherwise, returns OKStatus firstRecord(RID& firstRid);

// returns RID of next record on the page// returns DONE if no more records exist on the pageStatus nextRecord (RID curRid, RID& nextRid);

// copies out record with RID rid into recPtrStatus getRecord(RID rid, char *recPtr, int& recLen);

// returns a pointer to the record with RID ridStatus returnRecord(RID rid, char*& recPtr, int& recLen);

// returns the amount of available space on the pageint available_space(void);

// Returns true if the HFPage is has no records in it, false otherwise.bool empty(void);

Class HFPage (4/4)protected:

// Compacts the slot directory on an HFPage.

// WARNING -- this will probably lead to a change in the RIDs of

// records on the page. You CAN'T DO THIS on most kinds of pages.

void compact_slot_dir();

}

12

State change in Page when compact_slot_dir( ) was called

slot entries with EMPTY_SLOT state

slot entries with EMPTY_SLOTstate is compacted

13

Caution (1/2)Ø The variables that should be initialized when init( ) of HFPage was

calledshort slotCnt = 0;

short usedPtr = sizeof(data);

short freeSpace = sizeof(data);

PageId prevPage = INVALID_PAGE;

PageId nextPage = INVALID_PAGE;

PageId curPage = pageNo;

Ø return value of available_space( ) isn’t the current empty space inPage. It is the maximum record length that can be inserted.

return value = freeSpace – sizeof(slot_t);

14

Ø The variables that should be initialized when init( ) of HFPage was called

short slotCnt = 0;

short usedPtr = sizeof(data);

short freeSpace = sizeof(data);

PageId prevPage = INVALID_PAGE;

PageId nextPage = INVALID_PAGE;

PageId curPage = pageNo;

Ø return value of available_space( ) isn’t the current empty space inPage. It is the maximum record length that can be inserted.

return value = freeSpace – sizeof(slot_t);

Caution (2/2)Ø the length of Slot Array entry that isn’t being used currently should be

set to EMPTY_SLOT

Ø slotCnt value in Page shouldn’t be changed when the deleteRecord( )was calledd

Ø slotCnt value in Page should be changed when the compact_slot_dir( ) was calledd

Ø When you copy or move data to the record data area in Page, use memcpy( ) and memmove( )

15

Ø the length of Slot Array entry that isn’t being used currently should be set to EMPTY_SLOT

Ø slotCnt value in Page shouldn’t be changed when the deleteRecord( )was calledd

Ø slotCnt value in Page should be changed when the compact_slot_dir( ) was calledd

Ø When you copy or move data to the record data area in Page, use memcpy( ) and memmove( )

Return Values of APIsØ Status insertRecord(char *recPtr, int recLen, RID& rid);

If there is not enough space for new record, return DONE

Ø Status firstRecord(RID& firstRid);If there is no record in Page, return DONE

Ø Status nextRecord (RID curRid, RID& nextRid);If there is no next record in Page, return DONE

Ø Othersif slotNo of record Id, input parameter, isn’t appropriate,

return INVALID_SLOTNO

16

Ø Status insertRecord(char *recPtr, int recLen, RID& rid);If there is not enough space for new record, return DONE

Ø Status firstRecord(RID& firstRid);If there is no record in Page, return DONE

Ø Status nextRecord (RID curRid, RID& nextRid);If there is no next record in Page, return DONE

Ø Othersif slotNo of record Id, input parameter, isn’t appropriate,

return INVALID_SLOTNO

Error Handling

Ø Errors in minibase component

There is no one in this project

Ø Errors in Heap-File Page

if (slotNo < 0 || slotNo >= slotCnt || slot[slotNo].length == EMPTY_SLOT) return MINIBASE_FIRST_ERROR(HEAPFILE, INVALID_SLOTNO);

else return OK;

Ø Heap-File Page Error CodesINVALID_SLOTNO "hash table error"

17

Ø Errors in minibase component

There is no one in this project

Ø Errors in Heap-File Page

if (slotNo < 0 || slotNo >= slotCnt || slot[slotNo].length == EMPTY_SLOT) return MINIBASE_FIRST_ERROR(HEAPFILE, INVALID_SLOTNO);

else return OK;

Ø Heap-File Page Error CodesINVALID_SLOTNO "hash table error"

Where to Get codes?

Ø Get codesYou should have your account in the adam server to do project

Log in your account in adam server and copy like the following

$ cp –R ~cs360w/project/HFPage .

18

Ø Get codesYou should have your account in the adam server to do project

Log in your account in adam server and copy like the following

$ cp –R ~cs360w/project/HFPage .

How to Compile?

Ø Configure EnvironmentsAdd the following to the .cshrc file

setenv MINIBASE_ROOT ~cs360w/minibase2.0 setenv PLATFORM $MINIBASE_ROOT/bin/sparc-sun-solaris2.7 set path=(/usr/local/gcc_2.95/bin $path $MINIBASE_ROOT/bin)setenv LD_LIBRARY_PATH /lib:/usr/local/gcc_2.95/lib:/usr/local/lib: /usr/local/X11/lib:$LD_LIBRARY_PATH

Make the modified environmental variable take effect

$ source .cshrc

19

Ø Configure EnvironmentsAdd the following to the .cshrc file

setenv MINIBASE_ROOT ~cs360w/minibase2.0 setenv PLATFORM $MINIBASE_ROOT/bin/sparc-sun-solaris2.7 set path=(/usr/local/gcc_2.95/bin $path $MINIBASE_ROOT/bin)setenv LD_LIBRARY_PATH /lib:/usr/local/gcc_2.95/lib:/usr/local/lib: /usr/local/X11/lib:$LD_LIBRARY_PATH

Make the modified environmental variable take effect

$ source .cshrc

Ø CompileModify Makefile in src directory like the followingMINIBASE= < The directory position that src, include, lib was copied>===>MINIBASE=/adam/u22/early/HFPage

Execute make

$ make

If you see the hfpage.c, there is only defined API. There is no real implementation so that if you compile it, you will get error. You should implement hfpage.C in this project.

20

Ø CompileModify Makefile in src directory like the followingMINIBASE= < The directory position that src, include, lib was copied>===>MINIBASE=/adam/u22/early/HFPage

Execute make

$ make

If you see the hfpage.c, there is only defined API. There is no real implementation so that if you compile it, you will get error. You should implement hfpage.C in this project.

How to Run?

Ø RunningIf you compile, you will get the execution file “hfpage”. Execute it and compare the result with the one that you will get after executing the result file.

21

What to Submit?

Ø hfpage.h & hfpage.CIt is OK to modify hfpage.h if you need

The first line of these files should contain the following line

ex) /* Project1 CS 20028088 Hong,Gil-Dong */

Ø design.txtExplain the design and implementation that you did

This file should contain the project number, department, student

number, name

Print this file and submit to Homework Box (until the next day of Due Date)

22

Ø hfpage.h & hfpage.CIt is OK to modify hfpage.h if you need

The first line of these files should contain the following line

ex) /* Project1 CS 20028088 Hong,Gil-Dong */

Ø design.txtExplain the design and implementation that you did

This file should contain the project number, department, student

number, name

Print this file and submit to Homework Box (until the next day of Due Date)

How to Submit?

Ø You should use the submit program to submit each project

Ø Usage: submit <operation> <proj#> <file(s)><proj#> : project number <file(s)> : file names <operation>

-s : submit homeworks -w : withdraw homeworks -l : list submitted homeworks

ex) ~cs360w/submit/submit -s 1 hfpage.h hfpage.C design.txt

23

Ø You should use the submit program to submit each project

Ø Usage: submit <operation> <proj#> <file(s)><proj#> : project number <file(s)> : file names <operation>

-s : submit homeworks -w : withdraw homeworks -l : list submitted homeworks

ex) ~cs360w/submit/submit -s 1 hfpage.h hfpage.C design.txt

Ø CautionBefore you submit the files, the files should be able to be accessed from others. (chmod 644 filename)If the access permission wasn’t properly set, there comes error such as “can't open ....”Due Date : Feb. 26th, 2009Delay Penalty: 20% (~ Mar. 5th, 2009)No submission after one week from due dateIt will be determined your submission to be delay by the Server time

Ø Project GradeExecution Result(50%), Design Report(50%)

24

Ø CautionBefore you submit the files, the files should be able to be accessed from others. (chmod 644 filename)If the access permission wasn’t properly set, there comes error such as “can't open ....”Due Date : Feb. 26th, 2009Delay Penalty: 20% (~ Mar. 5th, 2009)No submission after one week from due dateIt will be determined your submission to be delay by the Server time

Ø Project GradeExecution Result(50%), Design Report(50%)

Reference

Ø Homepagehttp://dblab.kaist.ac.kr/Course/minibase/

http://www.cs.wisc.edu/coral/minibase/minibase.html

Ø BooksR. Elmasri and S. B. Navathe, Fundamentals of Database Systems, Addison-Wesley, 2003.

Raghu Ramakrishnan, Database Management Systems, McGraw-Hill, 1997.

J. Gray and A. Reuter, Transaction Processing : Concepts and Techniques, Morgan Kaufmann, 1992.

25

Ø Homepagehttp://dblab.kaist.ac.kr/Course/minibase/

http://www.cs.wisc.edu/coral/minibase/minibase.html

Ø BooksR. Elmasri and S. B. Navathe, Fundamentals of Database Systems, Addison-Wesley, 2003.

Raghu Ramakrishnan, Database Management Systems, McGraw-Hill, 1997.

J. Gray and A. Reuter, Transaction Processing : Concepts and Techniques, Morgan Kaufmann, 1992.