chap4. fundamental file structure concepts

1

Chap4. Fundamental File Structure Concepts

2

Chapter Objectives

Introduce file structure concepts dealing with

Stream files

Reading and writing fields and records

Field and record boundaries

Fixed-length and variable-length fields and records

Packing and unpacking records and buffers

Present an object-oriented approach to file structures Methods of encapsulating object value and behavior in classes

Classes for buffer manipulation

Class hierarchy for buffer and file objects and operations

Inheritance and virtual functions

Template classes

3

Contents

4.1 Field and Record Organization

4.2 Using Classes to Manipulate Buffers

4.3 Using Inheritance for Record Buffer Classes

4.4 Managing Fixed-Length, Fixed-Field Buffers

4.5 An Object-Oriented Class for Record Files

A Stream file

File structure ==> Persistency ==> Programs outlive the data in files

Simple representation: a file organized as a stream of bytes

Simple, but Reverse Humpty-Dumpty problem

In case of putting all information as a byte of stream, there is no way to get

it apart

Solution : Use field structure

The Need of Field Concept

Consider the function “write Person as a stream of bytes”!

Ostream & operator << (ostream & outputFile, Person & p)

{ // insert (write) fields into stream

outputFile << p.LastName << p.FirstName << p.Address

<< p.City << p.State << p.ZipCode;

return outputFile;

}

(input)

Mary Ames 123 Maple Stillwater, Ok 74074

Alan Mason 90 Eastgate Ada, Ok 74820

(output)

AmesMary123 MapleStillwaterOK74075MasonAlan90 Eastgate…..

6

Field Organization

Field: The smallest logically meaningful unit of information in a file (not physical)

Field structures (4 methods)

Fix the length of fields

Begin each field with a length indicator

Separate the fields with delimiters

Use a “Keyword = value” expression

Continued

7

Four methods for organizing files

Ames John 123 Maple Stillwater OK74075377-1808Mason Alan 90 Eastgate Ada OK74820

(a) Field lengths fixed. Place blanks in the spaces where the phone number would go.

Ames|John|123 Maple|Stillwater|OK|74075|377-1808|Mason|Alan|90 Eastgate|Ada|OK|74820||

(b) Delimiters are used to indicate the end of a field. Place the delimiter for the "empty"field

immediately after the delimiter for the previous field.

Ames|...|Stillwater|OK|74075|377-1808|#Mason|... 90Eastgate|Ada|OK|74820|#...

(c) Place the field for business phone at the end of the record. If the end-of-record mark is encountered,assume that the field is missing.

SURNAME=Ames|FIRSTNAME=John|STREET=123 Maple|...|ZIP=74075|PHONE=377-1808|#...

(d) Use a keyword to identify each field each field. If the ketword is missing, the corresponding field isassumed to missing.

RW files with Field ConceptExtraction operator for delimited fields into a Person object

istream & operator >> (istream & stream, Person & p)

{ // read delimited fields from file

char delim;

stream.getline(p.LastName, 30, delim);

if (strlen(p.LastName) == 0) return stream;

stream.getline(p.FirstName,30,delim);

stream.getline(p.Address,30,delim);

…..

return stream;

}

** By 부록 D.5 과 D.7

Last Name ‘Ames’

First Name ‘Mary’

Address ‘123 Maple’

……….

Last Name ‘Mason’

First Name ‘Alan’

……...

9

Record Organization

Record: a set of fields that belong together

Record organization(5 methods)

Make records a predictable number of bytes (Fixed-length records) Fig4.5. (a)(b)

Make records a predictable number of fields Fig4.5. (c)

Begin each record with a length indicator Fig4.6. (a)

Use an index to keep track of addresses Fig4.6. (b)

Place a delimiter at the end of each record Fig4.6. (c)

Write a var-length delimited buffer to a file(from memory to disk)

Const int MaxBUfferSize = 200;

int WritePerson(ostream & stream, Person & p)

{ char buffer [MaxBufferSize];

strcpy(buffer, p.LastName); strcat(buffer, “|”);

strcat(buffer, p.FistName); strcat(buffer, “|”);

…..

strcat(buffer,p.Zipcode); strcat(buffer, “|”);

short length=strlen(buffer);

stream.write (&length, sizeof(length));

stream.write(&buffer, length)

}

Figure 4.7 (pp 129)

Reading Variable Records

Records preceded by lengths (variable length records)

40 Ames|Mary|123 Maple|Stillwater|OK|74075|

36 Mason|Alan|90 Eastgate|Ada|OK|74820

int ReadVariablePerson (istream & stream, Person & p)

{ // read a variable sized record from stream and store it in p

short length;

stream . read (&length, sizeof(lenth));

char * buffer = new char[length + 1]; // create a buffer space

stream . read (buffer, length);

buffer [ length] = 0; // treminate buffer with null

istrstream strbuff (buffer); // create a string stream

strbuff >> p; // use the istream extraction operator

return 1;

}

14

Read-file using File Dump

File-dump gives us the ability to look inside a file at the actual bytes that are stored

Octal Dump: od -xc filename

e.g. The number 40, stored as ASCII characters and as a short integer

(a) 40 stored as ASCII chars:

(b) 40 stored as a 2-byte integer:

Decimal value ofnumber

40

40

Hex value storedin bytes

ASCIIcharacter form

'4' '0'

'\0' "("

34 30

00 28

Using Classes to Manipulate Buffers

Examples of three C++ classes to encapsulate operation of buffer object

Function : Pack, Unpack, Read, Write

– Output: pack into a buffer & write a buffer to a file

– Input: read into a buffer from a file & unpack a buffer

– ‘pack and unpack’ deals with only one field

DelimTextBuffer class for delimited fields

LengthTextBuffer class for length-based fields

FixedTextBuffer class for fixed-length fields

Appendix E : Full implementation

Buffer Class for Deliminated Text Fields(1)

Variable-length buffer

Fields are represented as delimited text

Class DelimTextBuffer{ public:

DelimTextBuffer (char Delim = ‘|’, int maxBtytes = 1000);

int Read(istream & file);int Write (ostream & file) const;int Pack(const char * str, int size = -1);int Unpack(char * str);

private:char Delim; // delimiter characterchar * Buffer; // character array to hold field

valuesint BufferSize; // current size of packed fieldsint MaxBytes; // maximum # of characters in the

bufferint NextByte; // packing/unpacking position in

buffer};


int DelimTextBuffer :: Pack (const char * str, int size)// set the value of the next field of the buffer;// if size = -1 (default) use strlen(str) as Delim of field{

short len; // length of string to be packedif (size >= 0) len = size;else len = strlen (str);if (len > strlen(str)) // str is too short!

return FALSE;int start = NextByte; // first character to be packedNextByte += len + 1;if (NextByte > MaxBytes) return FALSE;memcpy (&Buffer[start], str, len);Buffer [start+len] = Delim; // add delimeterBufferSize = NextByte;return TRUE;

}

Pack() method copies the characters of its argument to the buffer and then adds the delimiter characters.


int DelimTextBuffer::Unpack(char *str)// extract the value of the next field of the buffer{

int len = -1; // length of packed stringint start = NextByte; // first character to be unpackedfor(int i = start; i < BufferSize; i++)

if(Buffer[i] == Delim) {len = i-start; break;}if(len == -1) return FALSE; // delimiter not foundNextByte += len + 1;if(NextByte > BufferSize) return FALSE;strncpy (str, &Buffer[start], len);str[len] = 0; // zero termination for stringreturn TRUE;

}

Unpack() is extracking one field from a record in a buffer.


Read method of DelimTextBuffer Clears the current buffer contents Extracts the record size Read the proper number of bytes into buffer Set the buffer size

int DelimTextBuffer::Read(istream & stream){

Clear();stream.read((char *)&BufferSize, sizeof(BufferSize));if (Stream.fail()) return FALSE;if (BufferSize > MaxBytes) return FALSE; // buffer overflowstream.read(Buffer, BufferSize);return stream.good();

}

Extending Class Person with Buffer Operations

class Person{public:char lastname[11]; char firstname[11]; …char zipcode[10];

// method…int Pack(DelimTextBuffer &buf) const; // buffer operation Pack...

}

int Person::Pack(DelimTextBuffer &buf) const{ // pack the fields into a DelimTextBuffer

int result;result = buf.Pack(lastname); result = result && buf.Pack(firstname);…return result = result && buf.Pack(zipcode);

}

* pack deals with only one field!

Buffer Classes for Length-Based Fields

Almost same as the deliminated field class (compare with the previous page)

Change in the implementations of the Pack and Unpack

class LengthTextBuffer{ public:

LengthTextBuffer(int maxBytes = 1000);int Read(istream & file);int Write(ostream & file) const;int Pack(const char * field, int size = -1);int Unpack(char * field);

private:char * Buffer; // character array to hold field valuesint BufferSize; // size of packed fieldsint MaxBytyes; // maximum # of characters in the bufferint NextByte; // packing/unpacking position in buffer

};

Buffer Classes for Fixed-length Fields

Class FixedTextBuffer{ public:

FixedTextBuffer (int maxBytes = 1000);int AddField (int fieldSize);int Read(isteram * file);int Write(ostream *file) const;int Pack(const char * field);int Unpack (char * field);

private: // character array to hold field values

char * Buffer; // size of packed fields

int BufferSize; // Max # of chars in the buffer

int MaxBytes; // packing/unpacking position in buffer

int NextByte; // array of field sizes

int * FieldSizes;}

int Person::InitBuffer (FixedTextBuffer &buffer){

buffer.Init(6, 61); // 6 필드 , 61 바이트buffer.AddField (10);buffer.AddField (10);buffer.AddField (15);buffer.AddField (15);buffer.AddField (2);buffer.AddField (9);

return 1;}

Inheritance in the C++ Stream Classes

class istream: virtual public ios { …class ostream: virtual public ios { …class iostream: virtual istream, public ostream { …class ifstream: public fstreambase, public istream { …class ostream: public fstreambase, public ostream {…class fstream: public fstreambase, public iostream { …

Operations that work on base class objects also work on derived class objects

Class Hierarchy for Record Buffer Objects(1)

D elim ited F ieldB uff erp ac k an d un p ac k o p eratio n s

fo r delim ited fi eld s

L en g thF ield B uff erp ac k an d un p ac k o p eratio n s

fo r len g th- b as ed fi eld s

Variab leL en g thB uff erread an d write o p eratio n sfo r variab le len g th rec o rd s

F ixed F ieldB uff erp ac k an d un p ac k o p eratio n s fo r

fi xed s ized fi eld s

F ixed L en g thB uff erread an d write o p eratio n s

fo r fi xed len g th rec o rds

IO B uff erc har array fo r b uff er value

Appendix F : full implementation

Inheritance allows multiple classes share members and methods


class IOBuffer{ public:

IOBuffer (int maxBytes = 1000); // a MAX of maxBytevirtual int Read (istream &) = 0; // read a buffervirtual int Write (ostream &) = 0; // write a buffervirtual int Pack (const void * field, int size = -1) = 0;virtual int Unpack (void * field, int maxbytes = -1) = 0;

protected:char * Bufffer; // character array to hold field valuesint BufferSize; // sum of the sizes of packed fieldsint MaxBytes; // MAX # of characters in the buffer

};


Class VariableLengthBuffer: public IOBuffer{ public:

VariableLengthBuffer (int MaxBytes = 1000);int Read (istream &);int Write (ostream &) const;int SizeOfBuffer () const; // return current size of buffer

};

class DelimFieldBuffer: public VariableLengthBuffer{ public:

DelimFieldBuffer (char Delim = -1, int maxBytes = 1000);int Pack (const void *, int size = -1);int Unpack (void *field, int maxBytes = -1);

protected:char Delim;

};

Managing Fixed-Length, Fixed-Field Buffers

class FixedFieldBuffer: public FixedLengthBuffer{ public:

FixedFieldBuffer (int maxFields, int RecordSzie = 1000);FixedFieldBuffer (int maxFields, int *fieldSize);int AddField (int fieldSize); // define the next fieldint Pack(const void * field, int size = -1);int Unpack(void * field, int maxBytes = -1);int NumberOfFields () const; // return # of defined fields

protected:int * FieldSzie; // array to hold field sizesint MaxFields; // MAX # of fieldsint NumFields; // actual # of defined fields

};

Object-Oriented Class for Record Files

So far, we defined buffer classes

Now, we encapsulate all of our file operations!

class BufferFile // file with buffers{ public:

BufferFile (IOBuffer &); // create with a buffer int Open(char * fname, int MODE); // open an existing file

int Create (char * fname, int MODE); // create a new fileint Close();int Rewind(); // reset to the first data record// Input and Output operationsint Read(int recaddr = -1);int Write(int recaddr = -1);int Append(); // write the current buffer at the end of file

protected:IOBuffer & Buffer; // reference to the file’s bufferfstream File; // the C++ stream of the file

};

Usage: DelimFieldBuffer buffer; BufferFile file(buffer);

file.open(myfile); file.Read(); buffer.Unpack(myobject);

chap4. fundamental file structure concepts

Documents