more php functions for using regular expressions up to now, we have seen just one of the library of...

More PHP functions for using regular expressions

• Up to now, we have seen just one of the library of functions which PHP provides for using regular expressions

• The full library is described in Chapter CVIII of the PHP Manual

• We will consider just two more of them– preg_match and– preg_match_all

preg_match

• The format of a call to this function isint preg_match ( string pattern, string subject

[, array &matches [, int flags [, int offset]]] )

• As can be seen, only the first two arguments are required, so a minimum-argument call is of the form

int preg_match ( string regexp, string subject)

which returns0 if the regexp is not matched inside the subject string, or

1 if the regexp is matched inside the string

• PHP code<?php

$document = "<h1>France</h1>

Foods of France:

<ol><li>wine</li><li>bread</li></ol>";

$regexp = "%.+%";

if ( preg_match($regexp,$document) )

{ echo "Yes"; }

else { echo "No"; }

?>

• OutputNo

• Why no match?• Answer: on next slide

• We need to make the dot match newlines

• Revised PHP code<?php

$document = "<h1>France</h1>

Foods of France:

<ol><li>wine</li><li>bread</li></ol>";

$regexp = "%.+%s";

if ( preg_match($regexp,$document) )

{ echo "Yes"; }

else { echo "No"; }

?>

• OutputYes

preg_match (contd.)

• Frequently, it is useful to use the third, optional, argument

int preg_match ( string pattern, string subject, array &matches )

• As before, this returns o or 1 depending on whether a match was found in the subject string

• However, in addition, elements of the array in the third argument are set to match parts of the matching substring of the target string– matches[0] is set to contain the whole substring– matches[1] is set to contain the first () substring– matches[2] is set to contain the second () substring– … etc

• PHP code<?php$document = "<h1>France</h1> Foods of France: <ol><li>wine</li><li>bread</li></ol>";echo "document is ".str_replace("<","<",$document)." ";$regexp = " %(.+)%s ";if ( preg_match($regexp,$document,$matches) ) { echo "Yes "; echo "matches[0] is ".str_replace("<","<",$matches[0])." "; echo "matches[1] is ".str_replace("<","<",$matches[1])." "; } else { echo "No"; }?>

• Outputdocument is <h1>France</h1> Foods of France:

<ol><li>wine</li><li>bread</li></ol> Yes matches[0] is Foods of France: <ol><li>wine</li><li>bread</li></ol>matches[1] is Foods of France: <ol><li>wine</li><li>bread</li></ol>

preg_match_all

• So why would we need a function calledpreg_match_all

• See next slide

• PHP code<?php$document = " This is paragraph 1. And this is paragraph 2.";echo "document is ".str_replace("<","<",$document)." ";$regexp = "%(.+?)%s";if ( preg_match($regexp,$document,$matches) ) { echo "Yes "; echo "matches[0] is ".str_replace("<","<",$matches[0])." "; echo "matches[1] is ".str_replace("<","<",$matches[1])." "; } else { echo "No"; }?>

• Outputdocument is This is paragraph 1. And this is paragraph 2. Yes matches[0] is This is paragraph 1.matches[1] is This is paragraph 1.

• That is, preg_match only finds the first match

preg_match_all

• preg_match_all is like preg_match except – that it finds all matches and– thus, the value returned in $matches is actually an array of

arraysint preg_match_all ( string pattern, string subject, array &matches )

• $matches[0] is an array of all the substrings which match the overall regular expression

• $matches[1] is an array of all the substrings which match the first parenthesised sub-expression

• $matches[2] is an array of all the substrings which match the second parenthesised sub-expression

• and so on

• PHP code<?php$document = "This is paragraph 1. And this is paragraph 2.Paragraph 3.";echo str_replace("<","<",$document)." ";$regexp = "%(.+?)%s";if ( preg_match_all($regexp,$document,$matches) ) {$numMatches = count($matches[0]); for ($i=0;$i < $numMatches; $i++) {echo "matches[0][$i] is ".str_replace("<","<",$matches[0][$i])." "; } for ($i=0;$i < $numMatches; $i++) {echo "matches[1][$i] is ".str_replace("<","<",$matches[1][$i])." "; } } else { echo "No"; }?>

• OutputThis is paragraph 1. And this is paragraph 2. Paragraph 3.matches[0][0] is This is paragraph 1.matches[0][1] is And this is paragraph 2.matches[0][2] is Paragraph 3.matches[1][0] is This is paragraph 1.matches[1][1] is And this is paragraph 2.matches[1][2] is Paragraph 3.

PHP Filesystem Functions

• Chapter XXXVIII of the PHP manual

• We will consider just four• resource fopen ( string filename, string mode [, bool use_include_path [,

resource zcontext]] )

Usually used as

resource fopen ( string filename, string mode)

• bool fclose ( resource handle )

• string fread ( resource handle, int length )

• int fwrite ( resource handle, string someString [, int length] )

Usually used as

int fwrite (resource handle, string someString )

fopen• Typical call format:

resource fopen ( string filename, string mode)

• Example calls

$fileHandle1 = fopen("names.txt","w");

opens, for writing, a file called "names.txt" in the same directory

as the PHP program

$fileHandle1 = fopen("names.txt","r");

opens, for reading, a file called "names.txt" in the same directory

as the PHP program

$fileHandle1 = fopen(" "/usr/csr/names.txt ","w");

opens, for writing, a file called "/usr/csr/names.txt" on the same

computer as the PHP program

$fileHandle1 = fopen(" "/usr/csr/names.txt ","r");

opens, for reading, a file called "/usr/csr/names.txt" on the same

computer as the PHP program

$fileHandle1 = fopen("http://www.rte.ie/index.html","r");

opens, for reading, a file on an external web-site

fread and file_get_contents• Typical call format:

string fread ( resource handle, int length )

• Example calls

$contents = fread($fileHandle1,1000);

reads the next 1000 bytes from the file with handle $fileHandle1 or

up to the end of the file if there are less than 1000 bytes still unread

in the file

$contents = fread($fileHandle1,100000000);

reads the next 100 MB bytes from the file with handle $fileHandle1 or

up to the end of the file if there are less than 100 MB bytes still

unread in the file -- since very few files are as large 100 MB, this

probably just makes the computer read up to the end of the file;

• Typical call format:

string file_get_contents ( string filename [, bool use_include_path

[, resource context [, int offset [, int maxlen]]]] )Example call

$contents = file_get_contents($someURL);

fwrite• Typical call format:

int fwrite ( resource handle, string someString )

• Example call

$result = fwrite($fileHandle1,"<h1>Blah blah</h1>");

writes the string <h1>Blah blahM/h1> into the file with handle

$fileHandle1 and returns the number of bytes written into the file or

returns 0 (FALSE) if there was an error

fclose• Typical call format:

string fclose ( resource handle)

• Example call

fclose($fileHandle1);

closes the file with handle $fileHandle1

Example usage• PHP code:

<?php

$rte = fopen("http://www.rte.ie/","r");

$contents = fread($rte,100000000);

fclose($rte);

echo str_replace("<","<",$contents);

?>

• Output:

<html> <head> <META http-equiv="Content-Type" content="text/html; charset=iso-8859-15"> <title>RTE.ie - Irish Public Service TV and radio stations online</title> <META name="Description" content="RTE.ie - Irish public service television and radio broadcaster on the World Wide Web - bringing you Irish news, sports, business, entertainment, weather, television and radio, programmes, current affairs, health, motors, travel, video and audio."> <META name="Keywords" content="rte, rte.ie, irish, television, radio, ireland, Irish, news, business, sport, results, news, Ireland, video, audio, broadcaster, irish"> <STYLE TYPE="text/css"></STYLE> <STYLE TYPE="text/css"> </STYLE> <script language="JavaScript"> <!-- function AertelPage() { var

Compare output of program with page seen in browser

<html> <head> <META http-equiv="Content-Type" content="text/html; charset=iso-8859-15"> <title>RTE.ie - Irish Public Service TV and radio stations online</title> <META name="Description" content="RTE.ie - Irish public service television and radio broadcaster on the World Wide Web - bringing you Irish news, sports, business, entertainment

Example application• Extracting output from website of The Guardian:

– The Guardian maintains a page, updated almost daily, of recent stories on Israel and the Middle East at

http://www.guardian.co.uk/israel– Its appearance on 25 October 2005 is on the next

slide– The first part of the the source code for the page,

gotten from a browser, is on the slide after that– The complete source code for the 25 October 2005

version of the page is in the file http://www.cs.ucc.ie/j.bowen/cs4408/resources/GuardianIsrael25October2005.txt

(Note that this is an exact copy of the page from the Guardian site, so the src and href attribute values in the file assume that the page is stored at the Guardian URL above.)

• We want to extract the text stories, as shown in the third-next slide

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">



<html lang="en">

<head>

<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">

<meta name="artifact" content="377264">

<title>Guardian Unlimited | Special reports | Special report: Israel & the Middle East</title>

<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">

<link rel="stylesheet" href="/external/styles/basic/0,14491,,00.css" type="text/css">

<style type="text/css">

<!--

a.GUARDIANUNLIMITED {

text-decoration:none;

We want to produce a page like that on the right-hand side below from that on the left-hand side

We want to get all stories between Latest and Audio reports

Here is the source code around the headline Latest on the page

<div class="maintrail">

Latest<hr size="1">

<a HREF="/israel/Story/0,2763,1599840,00.html">Israel still in control of Gaza, says envoy</a> October 25: The international Middle East envoy, James Wolfensohn, has accused Israel of behaving as if it has not withdrawn from the Gaza Strip, by blocking its borders and failing to fulfil commitments to allow the movement of Palestinians and goods.



Qur'an test<hr size="1">

Here is the source code around the headline Audio reports

<a HREF="/israel/Story/0,2763,1596052,00.html">Israel's closed zone</a> October 20, letters: You graphically highlight the continuing expansionism of the Israeli government (Report, October 18).



Audio reports<hr size="1">

<a HREF="http://stream.guardian.co.uk:7080/ramgen/sys-audio/Guardian/audio/2005/09/12/120905McGreal.ra">Palestinians rush to Gaza</a> September 12: Many descended on the former Jewish Gaza settlements intent on causing chaos, but others came simply to see the beach for the first time, reports Chris McGreal from Khan Yunis. (2min 33s)

This PHP program will extract all the source code between the two headlines

<?php

$f1 = fopen("http://www.guardian.co.uk/israel/","r");

$document = fread($f1,100000);

fclose($f1);$regexp = "%Latest<hr size=\"1\"> (.+) Audio reports<hr size=\"1\">%s";

preg_match($regexp,$document,$matches);

$stories = $matches[1];

echo $stories;

?>

But we also want to remove all the intermediate headlines and rulings between the stories

Wolfensohn, has accused Israel of behaving as if it has not withdrawn from the Gaza Strip, by blocking its borders and failing to fulfil commitments to allow the movement of Palestinians and goods.



Qur'an test<hr size="1">

<a HREF="/israel/Story/0,2763,1599227,00.html">Qur'an competition tests participants' memories</a> October 24: With senior militant leaders looking on, Palestinian officials opened an international competition yesterday testing participants' knowledge of the Qur'an.



Comment and analysis<hr size="1">

<a HREF="/israel/Story/0,2763,1596291,00.html">Christian leanings at the Jerusalem Post</a> October 20, Chris McGreal: The strange and uneasy embrace between the Jewish state and America's evangelical right is being tightened.

 

<a HREF="/israel/comment/0,10551,1590082,00.html">12.10.05, Jonathan Freedland: One and three-quarter state solution</a> 

<a HREF="/israel/Story/0,2763,1584308,00.html">04.10.05, Chris McGreal: House that became a war zone</a>

West Bank<hr size="1">

<a HREF="/israel/Story/0,2763,1596168,00.html">Israel accused of 'road apartheid' in West Bank</a> October 20: Army seals off main route to Palestinian vehicles · Opponents say plan is to carve out new borders.

This program will extract stories and remove all intermediate headlines and rulings between stories<?php



fclose($f1);$regexp = "%Latest<hr size=\"1\">(.+)Audio reports<hr size=\"1\">%s";



$regexp = "%.+<hr size=\"1\">%";

/* Equivalent to $regexp = "%.+?<hr size=\"1\">%s"; */

$stories = preg_replace($regexp,"",$stories);

echo $stories;

?>

Correcting the URLs

• The URLs on the Guardian page assume that the page is being delivered from the Guardian server

• Anchor elements are of this form:<a HREF="/israel/Story/0,2763,1599840,00.html">Israel still in control of

Gaza, says envoy</a>

• Thus, if someone clicks a hotlink on our output, the browser will think the target page is on our server

• We must make the URLs point to the Guardian server by making them full (or "absolute") URLs

This program corrects the URLsNotice that we need only string manipulation

<?php






$regexp = "%.+<hr size=\"1\">%";

$stories = preg_replace($regexp,"",$stories);$stories = str_replace('HREF="/','HREF="http://www.guardian.co.uk/',$stories);

echo "<h1>Today's Guardian stories on Palestine</h1>";

echo $stories

?>

• You can run this program at http://www.cs.ucc.ie/j.bowen/cs4408/resources/GuardianExtractor1.php

Output from program on previous page

Putting the stories into a local file<?php






$regexp = "%.+?<hr size=\"1\">%";


$f2 = fopen("todaysGuardian.html","w");

fwrite($f2,"<h1>Today's Guardian Palestine stories</h1>");

fwrite($f2,$stories);

fclose($f2);

?>

<a href="todaysGuardian.html">See result</a>

There is a problem

• The trouble is that our program does not have write-access in the directory which contains the program

• We must get it to write to a different directory, where it will have write-access

Putting the stories into a file in a write-access directory<?php






$regexp = "%.+?<hr size=\"1\">%";


$f2 = fopen("writable/todaysGuardian.html","w");

fwrite($f2,"<h1>Today's Guardian Palestine stories</h1>");

fwrite($f2,$stories);

fclose($f2);

?>

<a href="writable/todaysGuardian.html">See result</a>

Output from program on previous page

Contents of page generated by previous program

A note on allowing PHP programs to write files

• PHP programs are run by your Apache server

• On Unix/Linux machines, the Apache server is treated as an ordinary user, with the name "nobody"

• Thus, your PHP programs can only write into a directory where user nobody has write -permission

A note on allowing PHP programs to write files (contd.)

• This could be a directory where you have given everybody write-access, as in

drwxrwxrwx 2 jabowen staff 35 Oct 25 09:58 writable

• But this is unsafe• It is better to create a group which contains only

yourself and nobody and give write access to that group, as in

drwxrwxr-x 2 jabowen jbApach 35 Oct 25 09:58 writable

where jbApach is a group that contains jabowen and nobody

Generating this page automatically every day

• To generate this page automatically every day, we need to run this program automatically

http://www.cs.ucc.ie/j.bowen/cs4408/resources/GuardianExtractor3.php

• To do this, we can use some Linux/Unix features– a utility called wget– another one called nohup– a third one called crontab

wget

• wget is for non-interactive download of files from the Web. • It supports HTTP, HTTPS, and FTP protocols, as well as retrieval

through HTTP proxies.• wget is non-interactive, meaning that it can work in the background,

while the user is not logged on • This allows you to start a retrieval and disconnect from the system,

letting Wget finish the work. • By contrast, most of web browsers require constant user's presence

Using wget

Using wget (contd.)Output is saved into a local file with same name as remote web-page

Using wget (contd.)See contents of local file, part 1

Using wget (contd.)Local file contains exactly the output from our program,

i.e. without any HTTP headers

• The local file generated by wget contains the output from our program

• Of course, we are not interested in that output• We are simply interested in the fact that that file

output/todaysGuardian.html was generated • We are not even interested in hanging around while wget

excutes our web program• Thus, we use another utility, called nohup, to call wget to

run our programnohup wget http://www.cs.ucc.ie/j.bowen/cs4408/resources/GuardianExtractor3.php

Automating all this

• Suppose we want to be able to look at the file output/todaysGuardian.html first thing every morning

• Suppose we want to ensure that it is updated each morning before we wake up

• We can use the crontab utility to make this happen

crontab

• crontab will execute programs automatically at times we specify

• To ask it to execute our program, 7 days a week at 3:30 AM, use the following:

30 3 * * 1,2,3,4,5,6,7 nohup wget http://cs.ucc.ie/j.bowen/cs4408/resources/GuardianExtractor3.php

more php functions for using regular expressions up to now, we have seen just one of the library of...

Documents