from ldap to patron load from ldap to patron load generating a patron update sif file from an ldap...

From LDAP to Patron From LDAP to Patron LoadLoad Generating a patron update SIF file from an LDAP directory dump

Michael Doran, Systems Librarian

Endeavor Users Group Meeting, Chicago, IL

Session 8, Thursday, April 26, 2007

Michael Doran, Systems Librarian doran@uta.edu

Cataloging

Acquisitions

Circulation

OPAC

Michael Doran123 Elm St.Arlington, TXPATRON BLOCK

Integrated library system

patron database

patron DB

Michael Doran, Systems Librarian doran@uta.edu

Getting patron data into Voyager Manual data entry via the

Circulation client

D’oh!

Michael Doran, Systems Librarian doran@uta.edu

Getting patron data into Voyager Manual data entry via the

Circulation client Batch loading via the

Pptrnupdt job Technical User’s Guide

6 : Patron Update 19 : Patron Record

Standard Interface File

Michael Doran, Systems Librarian doran@uta.edu

The Patron SIF specification Each record will consist of

1 base segment - fixed length 52 fixed length fields certain fields are required

1-9 address segments - fixed length 19 fixed length fields one address is required

1 optional notes segment - variable length 1 variable length field

1 end-of-record mark

Michael Doran, Systems Librarian doran@uta.edu

Patron SIF – one record

Michael Doran, Systems Librarian doran@uta.edu

Patron SIF – one record

Michael Doran, Systems Librarian doran@uta.edu

Patron SIF documentation

Michael Doran, Systems Librarian doran@uta.edu

Patron SIF documentation

Item (field) number – 72 total

Item (field) name

Offset – starting point

Format n - numeric s - string d - date

Length of field

Item (field) description

Required field?

Michael Doran, Systems Librarian doran@uta.edu

Patron SIF files the easy way…

Patron SIF

… and voila

Here’s your Patron SIFfile. Enjoy!

You send the Patron SIF documentationto your University IT department…

programmer

Michael Doran, Systems Librarian doran@uta.edu

Patron SIF files the hard way…

Voyager

Manuals

your boss

I want you to write a program that generates

Patron SIF files. Enjoy!

Michael Doran, Systems Librarian doran@uta.edu

So we have our marching orders

Michael Doran, Systems Librarian doran@uta.edu

Somewhere on your campus is… One or more directories of

information on people registered to attend classes

One or more directories of information on people who work for the university

students

facultyand staff

“patrons”

Michael Doran, Systems Librarian doran@uta.edu

Directories, databases, protocols, APIs directory

an alphabetical or classified list (as of names or addresses)

databasea collection of data organized for rapid retrieval (as by a computer)

protocola standard that controls the connection, communication, and data transfer between two computing endpoints

application programming interface (API)

an abstraction layer between the gory details of an application and the world outside that software

Michael Doran, Systems Librarian doran@uta.edu

Directories, databases, protocols, APIs directory

an alphabetical or classified list (as of names or addresses)

databasea collection of data organized for rapid retrieval (as by a computer)

protocola standard that controls the connection, communication, and data transfer between two computing endpoints

application programming interface (API)

an abstraction layer between the gory details of an application and the world outside that software

LDAP directory

Lightweight Directory Access Protocol

Michael Doran, Systems Librarian doran@uta.edu

Directories, databases, protocols, APIs directory

an alphabetical or classified list (as of names or addresses)

databasea collection of data organized for rapid retrieval (as by a computer)

protocola standard that controls the connection, communication, and data transfer between two computing endpoints

application programming interface (API)

an abstraction layer between the gory details of an application and the world outside that software

Voyagerpatron load

“Standard Interface File” (SIF)

Pptrnupdt

Michael Doran, Systems Librarian doran@uta.edu

Voyager patron database tables and relationships

Michael Doran, Systems Librarian doran@uta.edu

Voyager

LDAP client

patron info

The pieces and players

Pptrnupdt

patron SIF

Perl program

Net::LDAP

Directory

patron info

LDAP server

?

LDAP Protocol

Michael Doran, Systems Librarian doran@uta.edu

Voyager

LDAP client

patron info

The pieces and players

Pptrnupdt

patron SIF

Perl program

Net::LDAP

Directory

patron info

LDAP server

?

LDAP Protocol

patron info

Michael Doran, Systems Librarian doran@uta.edu

Three easy(?) steps…

Query the LDAP directory filter for valid patrons

Process the records minor data formatting decision algorithms output records in SIF format

Load SIF file into Voyager Pptrnupdt

Michael Doran, Systems Librarian doran@uta.edu

Three easy(?) steps…

Query the LDAP directory filter for valid patrons

Process the records minor data formatting decision algorithms output records in SIF format

Load SIF file into Voyager Pptrnupdt

Perl program

Michael Doran, Systems Librarian doran@uta.edu

Voyager

LDAP client

patron info

Before we start Perl programming

Pptrnupdt

patron SIF

Perl program

Net::LDAP

Directory

patron info

LDAP server

?

LDAP Protocol ldapsearch

Michael Doran, Systems Librarian doran@uta.edu

What you need to get started command line client

ldapsearch comes standard on Solaris LDAP server info

hostname port numbers

389 – default ldap port 636 – default ldaps port

LDAP Schema bind credentials

Distinguished Name (DN) anonymous your own (e.g. NetID username & password) “superuser” (provided by LDAP administrator)

Michael Doran, Systems Librarian doran@uta.edu

ldapsearch search

ldapsearch \ -h "ldap.uta.edu" \ -D "uid=doran,cn=accounts,dc=uta,dc=edu" \ -w "<password>" \ -b "cn=people, dc=uta, dc=edu" \ "utaAccountName=doran"

Michael Doran, Systems Librarian doran@uta.edu

ldapsearch search

ldapsearch \ -h "ldap.uta.edu" \ -D "uid=doran,cn=accounts,dc=uta,dc=edu" \ -w "<password>" \ -b "cn=people, dc=uta, dc=edu" \ "utaAccountName=doran"

LDAP server hostnameBind Distinguished Name

<uid>

Bind CredentialsSearch BaseSearch Filter

Michael Doran, Systems Librarian doran@uta.edu

ldapsearch outputcedarid=1085691573581181000,cn=People,dc=uta,dc=eduutaAccountName=doranutaPrevAccountName=doranutaPrimaryAccount=doranutaPrimaryEmail=doran@uta.eduutaHomeStreet1=123 Elm St.utaEmployeeTitle=LibrarianutaDeptCode=6800000utaEmployeeStatus=activeutaEmployeeCampusBox=19497eduPersonPrimaryAffiliation=staffutaHomeCity=ArlingtonutaEmployeeTitleCode=0100sn=Dorancedarid=1085691573581181000utaHomeState=TXhomePhone=+1 817 555 1234utaEmployeeJobCategory=prof_non_facultycn=Michael D DoranutaEmployeeType=monthlyutaHomeZip=76013utaEmployeeBldg=LIBgivenName=Michael

displayName=doran, michael dutaEmployeeExemption=exemptutaEmployeePhone=+1 817 272 5326homePostalAddress=123 Elm St.$Arlington$TX$76013utaID=1000068572utaEmployeePercentTime=100.00utaEmployeeRetirementCode=9utaEmplID=1000068572utaMiddleName=DobjectClass=topobjectClass=personobjectClass=inetOrgPersonobjectClass=utaPersonobjectClass=utaEmployeeeduPersonScopedAffiliation=employee@uta.edueduPersonScopedAffiliation=staff@uta.eduutaPersonAffiliation=employeeutaPersonAffiliation=staffutaPersonAffiliation=administrativeeduPersonAffiliation=employeeeduPersonAffiliation=staff

search filter match

“patron-worthy” search filter

Michael Doran, Systems Librarian doran@uta.edu

#!/usr/local/bin/perl -w

use Net::LDAP;

$ldap = Net::LDAP->new("ldap.uta.edu");

$mesg = $ldap->bind("uid=doran,cn=accounts,dc=uta,dc=edu", password => "<password>");

$mesg = $ldap->search( base => 'cn=people, dc=uta, dc=edu', filter => "(utaAccountName=doran)" );

foreach $entry ($mesg->all_entries) { $entry->dump; };

$ldap->unbind();

exit(0);

LDAP search with Perl Net::LDAP

Michael Doran, Systems Librarian doran@uta.edu

LDAP search with Perl Net::LDAP#!/usr/local/bin/perl -w

use Net::LDAP;

$ldap = Net::LDAP->new("ldap.uta.edu");

$mesg = $ldap->bind("uid=doran,cn=accounts,dc=uta,dc=edu", password => "<password>");

$mesg = $ldap->search( base => "cn=people, dc=uta, dc=edu", filter => "(utaAccountName=doran)" );

foreach $entry ($mesg->all_entries) { $entry->dump; };

$ldap->unbind();

exit(0);

LDAP server hostnameBind Distinguished NameBind CredentialsSearch BaseSearch Filter

this Perl module will need to be installed

Michael Doran, Systems Librarian doran@uta.edu

Perl Net::LDAP outputdn:cedarid=1085691573581181000,cn=People,dc=uta,dc=edu

utaAccountName: doran utaPrimaryAccount: doran utaPrimaryEmail: doran@uta.edu utaHomeStreet1: 123 Elm St. utaEmployeeTitle: Librarian utaDeptCode: 6800000 utaEmployeeStatus: active utaEmployeeCampusBox: 19497eduPersonPrimaryAffiliation: staff utaHomeCity: Arlington utaEmployeeTitleCode: 0100 sn: Doran cedarid: 1085691573581181000 utaHomeState: TX homePhone: +1 817 555 1234 utaEmployeeJobCategory: prof_non_faculty cn: Michael D Doran utaEmployeeType: monthly utaHomeZip: 76013 utaEmployeeBldg: LIB givenName: Michael displayName: doran, michael d ...: ...

Michael Doran, Systems Librarian doran@uta.edu

#!/usr/local/bin/perl -w

use Net::LDAP;

$ldap = Net::LDAP->new("ldap.uta.edu");

$mesg = $ldap->bind("cn=library,cn=app,dc=uta,dc=edu", password => "<password>");

$mesg = $ldap->search( base => 'cn=people, dc=uta, dc=edu', filter => "(|(utaStudentStatus=enrolled) (utaEmployeeStatus=active) (utaEmployeeStatus=retired))" );

foreach $entry ($mesg->all_entries) { $entry->dump; };

...

Superuser & valid patron filtering

a superuser Bind DN

patron-worthy LDAP records

Michael Doran, Systems Librarian doran@uta.edu

Three easy(?) steps…

Query the LDAP directory filter for valid patrons

Process the records minor data formatting decision algorithms output records in SIF format

Load SIF file into Voyager Pptrnupdt

Michael Doran, Systems Librarian doran@uta.edu

LDAP records simplifieddn:cedarid=1085691573581181000,cn=People,dc=uta,dc=edu

utaAccountName: doran utaPrimaryAccount: doran utaPrimaryEmail: doran@uta.edu utaHomeStreet1: 123 Elm St. utaEmployeeTitle: Librarian utaDeptCode: 6800000 utaEmployeeStatus: active utaEmployeeCampusBox: 19497eduPersonPrimaryAffiliation: staff utaHomeCity: Arlington utaEmployeeTitleCode: 0100 sn: Doran cedarid: 1085691573581181000 utaHomeState: TX homePhone: +1 817 555 1234 utaEmployeeJobCategory: prof_non_faculty cn: Michael D Doran utaEmployeeType: monthly utaHomeZip: 76013 utaEmployeeBldg: LIB givenName: Michael displayName: doran, michael d ...: ...

attributes values

attribute-value pair

Michael Doran, Systems Librarian doran@uta.edu

additionalattributes foremployees

LDAP records made more complex Not all records have all the same attributes

let’s consider “people” records

additionalattributes for

students

genericattributes for

all people

Michael Doran, Systems Librarian doran@uta.edu

Attributes continued

Some attributes are required, but many (most) are optional

Some attributes have a constrained set of possible values

Some attributes are “multi-valued” Attributes will not necessarily

be sorted in any order LDAP

Schema

for information…

Michael Doran, Systems Librarian doran@uta.edu

Sample schema page (UTA LDAP)

Michael Doran, Systems Librarian doran@uta.edu

Sample schema page (UTA LDAP)

Michael Doran, Systems Librarian doran@uta.edu

$mesg = $ldap->search( base => 'cn=people, dc=uta, dc=edu', filter => "(|(utaStudentStatus=enrolled) (utaEmployeeStatus=active) (utaEmployeeStatus=retired))", callback => \&ProcessRecord );

foreach $entry ($mesg->all_entries) { $entry->dump; };

Processing records

we want to process records

dumps records

The “callback” option allows you to processeach record as it is received

Michael Doran, Systems Librarian doran@uta.edu

sub ProcessRecord { my ($mesg,$entry) = @_;

if (!$entry) {&Finish(0);}

$surname = $entry->get_value('sn'); $first_name = $entry->get_value('givenName'); $middle_name = $entry->get_value('utaMiddleName'); $uta_id = $entry->get_value('utaID'); @uta_affil = $entry->get_value('utaPersonAffiliation'); $email_addr = $entry->get_value('utaPrimaryEmail'); $home_phone = $entry->get_value('homePhone'); if ($home_phone) { $home_phone =~ s/\+1 //; $home_phone =~ s/ /\-/g; }

# ...yada, yada, yada

ProcessRecord subroutine

Michael Doran, Systems Librarian doran@uta.edu

sub ProcessRecord { my ($mesg,$entry) = @_;

if (!$entry) {&Finish(0);}

$surname = $entry->get_value('sn'); $first_name = $entry->get_value('givenName'); $middle_name = $entry->get_value('utaMiddleName'); $uta_id = $entry->get_value('utaID'); @uta_affil = $entry->get_value('utaPersonAffiliation'); $email_addr = $entry->get_value('utaPrimaryEmail'); $home_phone = $entry->get_value('homePhone'); if ($home_phone) { $home_phone =~ s/\+1 //; $home_phone =~ s/ /\-/g; }

# ...yada, yada, yada

assigning LDAP attribute values to program variablessingle-valued attributes => assigned to a scalarmulti-valued attributes => assigned to an array

ProcessRecord subroutine

minor reformatting of data

Michael Doran, Systems Librarian doran@uta.edu

LDAP recorddn:cedarid=1085691573581181000,cn=People,dc=uta,dc=edu

utaAccountName: doran utaPrimaryAccount: doran utaPrimaryEmail: doran@uta.edu utaHomeStreet1: 123 Elm St. utaEmployeeTitle: Librarian utaDeptCode: 6800000 utaEmployeeStatus: active utaEmployeeCampusBox: 19497eduPersonPrimaryAffiliation: staff utaHomeCity: Arlington utaEmployeeTitleCode: 0100 sn: Doran cedarid: 1085691573581181000 utaHomeState: TX homePhone: +1 817 555 1234 utaEmployeeJobCategory: prof_non_faculty cn: Michael D Doran utaEmployeeType: monthly utaHomeZip: 76013 utaEmployeeBldg: LIB givenName: Michael displayName: doran, michael d ...: ...

Find the Voyager patron group

The LDAP record doesn’t have a patron group

Michael Doran, Systems Librarian doran@uta.edu

if ($stud_status eq "enrolled" && $stud_class) { if ($stud_class =~ /freshman/i || $stud_class =~ /sophomore/i || $stud_class =~ /junior/i || $stud_class =~ /senior/i ) { $patron_group = "und"; } elsif ($stud_class =~ /masters/i || $stud_class =~ /doctoral/i || $stud_class =~ /graduate/i) { $patron_group = "grad"; $expire_days = "180"; }} elsif ($stud_status eq "enrolled") { $patron_group = "und";}

Decision algorithmspatron group assignment exampleprogram looks at student attributes first

student classification not a required attributein LDAP record, so need a default

Michael Doran, Systems Librarian doran@uta.edu

if ($empl_status eq "active") { if (! $patron_group) { $patron_group = "staff"; } foreach my $affil (@uta_affil) { if (($affil =~ /faculty/i) || ($affil =~ /admin/i) || ($affil =~ /assistant/i)) { $patron_group = "fac"; $expire_days = "400"; } }}

Decision algorithmsprogram looks at employee attributes second

default only if no studentpatron group assigned

Michael Doran, Systems Librarian doran@uta.edu

Decision algorithms continued

workaddress& phone

studentaddress& phone

home address& phone

multiple sets of address attributes

employees students

all people

Michael Doran, Systems Librarian doran@uta.edu

Three easy(?) steps…

Query the LDAP directory filter for valid patrons

Process the records minor data formatting decision algorithms output records in SIF format

Load SIF file into Voyager Pptrnupdt

Michael Doran, Systems Librarian doran@uta.edu

A poetry break…

“Our time is fixed, and all our days are number'd; How long, how short, we know not.”

Robert Blair, The Grave

“The SIF fields are fixed, and all the fields are number'd; How long, how short, we better know.”

anonymous programmer

Michael Doran, Systems Librarian doran@uta.edu

Patron SIF formatting simplified For each patron SIF record

Format each field s string – left-justified / blank-filled n numeric – right-justified / zero-filled n* “special” numeric – if no value, then all

blanks d date – YYYY.MM.DD format

Concatenate all fields into one long string Slap an end-of-record marker on string Write string to file

Michael Doran, Systems Librarian doran@uta.edu

Remember this?

Michael Doran, Systems Librarian doran@uta.edu

Formatting with sprintf*

$first_name = sprintf("%-20s",Michael);

$patron_id = sprintf("%010d",123456);

*sprintf help

$concatenation = $first_name . $patron_id;print $concatenation;

Michael 0000123456

Michael Doran, Systems Librarian doran@uta.edu

Three easy steps…

Query the LDAP directory filter for valid patrons

Process the records minor data formatting decision algorithms output records in SIF format

Load SIF file into Voyager Pptrnupdt

Michael Doran, Systems Librarian doran@uta.edu

Things I didn’t cover

Encryption

Data validation

Log files

Session file

Cron job setup

yada, yada, yada…

Michael Doran, Systems Librarian doran@uta.edu

Perlprogramming

You* can do this!

Patron SIF LDAP

protocol

*…you, or yourresident geek

geek stigmata

Michael Doran, Systems Librarian doran@uta.edu

Resources Voyager Technical User’s Guide

Patron SIF specification Patron Update (Pptrnupdt)

Perl Net::LDAP module download from CPAN documentation

Perl Date::Calc module download from CPAN documentation

LDAP to patron SIF webpage (ldump.pl)

Michael Doran, Systems Librarian doran@uta.edu

Questions?

Please fill out the session evaluation!