If you notice the little Cardinals VS Cubs 'Widget' or 'Portlet' as it's know as in the Yii framework you'll see that I'm displaying the team records and percentages for the year.

While absolutely unimpressive, I did have to install three software packages in order to get this where it is today. :)

My Problem

I definitely didn't want to update these values myself and I wasn't able to locate a free web service that would provide these stats for me though xml or something similar, however I was not able to find that at all. So I decided to use a technique called 'screen scraping' to get my data.

Screen Scraping

This is when you request an html page from another server, and parse the page to extract the information you were looking for.

So naturally I was looking for a site that would present the data that I wanted and in a reasonably easy to extract form.

Code I found that looked good enough.

 
 <tr class="cnnRow2">
    <td class="cnnCol0"><a href="/baseball/mlb/teams/cardinals/">
    <img title="Cardinals" alt="Cardinals" src="http://i.cdn.......gif">
    St. Louis&nbsp;Cardinals</a></td>
 
    <td class="cnnCol1">8</td>
    <td class="cnnCol2">3</td>
    <td class="cnnCol3">.727</td>
    <td class="cnnCol4">--</td>
    <td class="cnnCol5">5-2</td>
    <td class="cnnCol6">3-1</td>
 
    <td class="cnnCol7">8-2</td>
    <td class="cnnCol8">Won 2</td>
 </tr>
 <tr class="cnnRow2">
    <td class="cnnCol0"><a href="/baseball/mlb/teams/cubs/">
    <img title="Cubs" alt="Cubs" src="http://i...,.all/mlb/logos/cubs_30.gif">
    Chicago&nbsp;Cubs</a></td>
    <td class="cnnCol1">5</td>
 
    <td class="cnnCol2">4</td>
    <td class="cnnCol3">.556</td>
    <td class="cnnCol4">2</td>
    <td class="cnnCol5">1-2</td>
    <td class="cnnCol6">4-2</td>
    <td class="cnnCol7">5-4</td>
 
    <td class="cnnCol8">Lost 2</td>
 </tr>
 

To change gears really quick I'm going to show you the main file in creating a Portlet in Yii. This will act as our Portlet Controller.

The minimum portlet file would look like this.

 
<?php
// Filename is protected/components/Cardinals.php
class Cardinals extends Portlet
{
    public $title='Cardinals VS Cubs';
 
    protected function renderContent()
    {
        $this->setData();
        $this->render('cardinals',array());
    }
 
}
 

You can see it has one main method called 'renderContent()' which will render the 'cardinals' view.

Cardinals View is located in the view subdirectory of components. (protected/components/views/cardinals.php)

 
<div class="row">
 
<?php echo $this->title;?>
<br><br>
Pretty impressive eh?
 
</div>
 

The title comes from the Controller, and that is a way we can access all of those properties.

Now that you know the basics, we'll move on to fleshing it out.

I'm just going to show you the first version of what I wrote to deal with parsing my data and rendering my view. When you build portlets in the Yii Framwork you'll add a file into protected/components/ and name it whatever you want(with a PHP extension) It looks like the naming convention for this file is the first letter should be capitalized. In my case I'll name it Cardinals.php

 
<?php
class Cardinals extends Portlet
{
    public $title='Cardinals VS Cubs';
    public $CardsCubsObj;
 
    protected function renderContent()
    {
        $this->setData();
        $this->render('cardinals',array());
    }
 
    private function setData() {
        $CardsCubsObj = $this->createAndReturnCardsCubsObject();
    }
 
    private function createAndReturnCardsCubsObject() {
              //See The Blog Post Titled 'Replacement For file_get_contents (using cURL)'
        $data = CA_HTTP::get_contents("http://sportsillustrated.cnn.com/baseball/mlb/standings/");
        $CardsCubsObj = new stdClass();
        $CardsCubsObj->cards = $this->getArraysFromHTML($data, 'St. Louis&nbsp;Cardinals');
        $CardsCubsObj->cubs = $this->getArraysFromHTML($data, 'Chicago&nbsp;Cubs');
        return $CardsCubsObj;
    }
 
    private function getArraysFromHTML($data, $team) {
        $array = array();
        $str_pos = stripos($data, $team);
        $cards = substr($data, $str_pos, 400);
        $array['wins'] = $this->getRecord($cards, '<td class="cnnCol1">', '</td>');
        $array['losses'] = $this->getRecord($cards, '<td class="cnnCol2">', '</td>');
        $array['percentage'] = $array['wins'] / ($array['losses'] + $array['wins']);
        $array['percentage'] = number_format($array['percentage'], 3, ".", ",");
        return $array;
    }
 
    public function getRecord($data, $start_string, $end_string) {
        $count_start = strlen($start_string);
        $pos_start = strpos($data, $start_string);
        $pos_end = strpos($data, $end_string, $pos_start);
        $real_start = $pos_start+$count_start;
 
        return substr($data, $real_start, $pos_end - $real_start);
    }
 
}
 
 

These methods should be pretty self-explanatory and if you're not sure what one of these functions does, click on it and it will take you to php's documentation on it.

Now it comes to actually putting our new data on the page.

This is our protected/components/views/cardinals.php file

 
<div class="row">
<table align="center">
    <tr>
        <td>Cardinals:</td>
        <td><?php echo "{$this->CardsCubsObj->cards['wins']} -
 {$this->CardsCubsObj->cards['losses']}";?> 
(<?php echo $this->CardsCubsObj->cards['percentage'];?>)</td>
    </tr>
    <tr>
        <td>Cubs:</td>
        <td><?php echo "{$this->CardsCubsObj->cubs['wins']} - 
{$this->CardsCubsObj->cubs['losses']}";?> 
(<?php echo $this->CardsCubsObj->cubs['percentage'];?>)</td>
    </tr>
</table>
 
</div>
 

It's pretty straightforward here. (Sorry for having to split the code like that..)

Lastly we need to activate this Portlet in our main layout file.

Open 'protected/views/layouts/main.php

If you used the demo you will have widgets already installed. Since I did I just used the structure that was on the rest of them.

 
    <?php $this->widget('RecentComments'); ?>
 
    <?php $this->widget('UserLogin',array('visible'=>Yii::app()->user->isGuest)); ?>
 
    <?php $this->widget('Cardinals'); ?>
 

Now after adding the line above my Cardinals VS Cubs Widget popped up, and displayed the data.

The Downfall of the Current Design

The downfall here is the fact that every single time a page is hit from my site, my site will have to download the html from another site, parse it and then display it.

The data will only change for each team at max 4 times, and that would only happen if the Cardinals an the Cubs played double-headers. So I don't need to parse the data that often.

Possible Solutions

Run A Cronjob that parses that data, and save it to a database.
Cache The Data In Memory.

Meet Memcache

Memcached is a caching server that will store objects and data in memory instead of constantly going to the DB or an external source. Memcached is actually a server process that runs constantly, you must install it separately and then install the Memcache pecl installation to get this to work.

This tool is commonly used to cache expensive database query results, but I'm using it for the purpose of learning it, I've worked with databases 1000s of times, so might as well try memcache.

Memcache is VERY easy to use. Getting everything up and running is fairly simple, a google search for 'installing memcache for php' should point you in the right direction.

I'm just going to cover how to use it in this micro-tutorial.

Basically how it works is you instantiate the Memcache object, connect to your memcache server and then save or query the server for your data.

Simple as that.

Let's look at the setData method as that is where we Slapped in our Memcaching.

 
<?php
class Cardinals extends Portlet
 
    private function setData() {
        $memcache = new Memcache;
        $memcache->connect('localhost', 11211) or die ("Could not connect");
        $CardsCubsObj = $memcache->get('cardinals_cubs');
        if($CardsCubsObj === false) {
            $CardsCubsObj = $this->createAndReturnCardsCubsObject();
            $memcache->set('cardinals_cubs', $CardsCubsObj, false, 60*15)
or die ("Failed to save data at the server");
        }
        else {
            //echo "Cached";
        }
        $this->CardsCubsObj = $CardsCubsObj;
    }
 
 
 
}
 

So we call $memcache->get('cardinals_cubs'). The 'cardinals_cubs' part represents our key in memcache. So it's just saying get our cardinals and cubs value.

If it returns false, it will not be in the cache anymore so we will generate it again and then cache it. the set method takes 4 arguments: the key, the object or value you want to store, then something else, and then the amount of time the value should remain in the cache.