Category: PHP

Notes from Facebook's HipHop for PHP Debut

Posted by Kris Jordan | Comments ( 2 ) | Filed in: PHP | | | | |

This evening the developers behind HipHop dove into more depth on the technology over a Live UStream cast. In case you missed it, here are my notes from the presentation.

Performance compared to PHP with APC opcode cache turned on:

  • 30% less CPU time serving traffic in API usage (shorter requests pulling data from fewer sources)
  • 50% less CPU time serving traffic in WEB page rendering usage

PHP, being a scripted language, has a certain amount of 'non-magic' code that looks and feels much like static code you might write in C/C++. This code and logic can be compiled down to very fast static function calls, static variable look-ups, static types. This is where HipHop's performance gains come from. The 'magic' / dynamic features of PHP are much harder to optimize and run roughly as fast as they do on the Zend engine. Magic includes:

$$x = $$y;
eval($x . $y);
$$$$foo();
function foo($x) { include $x; }

There are some tricks that can be applied for this style of dynamic code, things like prehashing dynamic variable look-ups, efficient jump tables, etc.
 "If the code has a lot of dynamicness it will execute at roughly the same speed."

Transformation Process:

1. Static analysis
* Collect info on who declares what, dependencies
2. Type inference
* Pick most specific type for every variable possible
** C++ scalars string array, classes, object, and variant
* Type hints in code are analyzed
3. Code Generation

Facebook hand wrote the runtime libraries for string/array/object classes. Similar to Zend engine's. They rewrote Zend engine because it primarily deals with variant types. In Hip-Hop there are special cases for static types made possible by type inference. Code for "what if this variable is an integer", etc. Another reason to rewrtie the runtime is that once the whole runtime in C++ it should be easier to author extensions based on primitive C++/FaceBook types than their Zend engine, dynamic counter-parts.

"We want people to write PHP like they're used to writing PHP."

Supported magical PHP features:

  • dynamic function call including call_user_func()
  • dynamic object properties and methods
  • dynamic variables, including extract()
  • dynamic includes
  • redeclared functions
  • redeclared classes
  • redeclared constance
  • magic methods __toString(), __get(), __set(), __call()

Features not supported:
1. eval, create function, preg_replace /e
2. "if(function_exists('foo'){print 'foo';})
function foo() {}"

HipHop runs as one process with multiple threads. No downtimes during restarts (port takeover). Uses libevent internally.

In the next month the goal is to bring it up to speed with 5.2.12. After that the focus switches to PHP 5.3+. HipHop may potentially support some of the type suggestion and typing features that did not make the 5.3 cut.

Some Q&A
Do you support old PHP extensions?
We've converted extensions because they have their own datastructures. They've converted a lot of them because facebook's codebase uses a lot of extensions. Keep same logic switching out data structures.

There are some PHP extensions that are not thread-safe, how do we handle them?
We've fixed the problems they had and made them thread-safe. Once the source is opened you will get the full list of supported extensions.

How did you know it "worked".
- We counted the number of function calls and compare with PHP, it has to be identical.
- We compared network traffic between machines running HipHop and running PHP and made sure the in and out were practically identical.
- High enough volume that every machine gets roughly the same kind of traffic making this analysis possible.

Have you tried to support other compilers like Intel's?
We want to try intel's with great support for memcopy.

Why did you choose not to go the just-in-time / LLVM route?
So V8 has great optimization over Javascript but it was not available at the time we started the project. We absolutely considered LLVM. Subconciously I was thinking about having a tool that automatically converts code to C++. We may be able to borrow some of the optimization techniques being used by V8.

 

 

This is is really exciting technology. Once the source is released we'll be working out the details on how to compile Recess on HipHop as soon as possible.

Recess Community Updates

Posted by Kris | Comments ( 0 ) | Filed in: Community, PHP | | | | |

In the month since the new Recess Forums have launched the number of posts and interesting topics has been flourishing. Lots of great things are bubbling up:

PostgreSQL support is being developed on the database stack lead by Ryan Day. You can pull Ryan's postgres bits from Github here. Ryan has also put up two great, introductory tutorials on his blog. The first is on modifying the generated scaffolding code. The second is on beefing up date input fields. Awesome work.

Christiaan has been working on a Recess project that changes the Recess flow-of-control policy to achieve a more nested MVC approach. Check out his project alive.

Issue tracking has moved from lighthouse to Github to keep our source and bugs closer together.

Some other interesting threads from the forums:

If you haven't joined the forums or checked back in on them in a while, you should head that way.

Dynamic Invocation in PHP: is_callable, call_user_func, and call_user_func_array - Functional PHP 5.3 Part III

Posted by Kris Jordan | Comments ( 3 ) | Filed in: PHP | | | | |

In PHP you can dynamically invoke functions, lambdas, and methods in a number of ways. In this third part of the series we're going to explore callables in PHP 5.3. Check out Part I of the series if you're unfamiliar with anonymous functions, lambdas and closures in PHP 5.3. Part II followed up with a look at higher order PHP functions by implementing map and reduce.

What is Dynamic Invocation?

We've seen a flavor of dynamic invocation in the previous posts of this series using higher-order methods. The functions map and reduce receive a lambda as an argument and dynamically invoke it at runtime. PHP doesn't know (or care) in advance which lambda it is passed and it won't know until runtime:

$lambda = rand(0,1) ? function() { echo "Heads!"; } :
                      function() { echo "Tails!"; };
$lambda(); // <= Dynamic Invocation!

It's not until we run the script and flip the coin that PHP will know which anonymous function is being invoked. PHP has actually had the ability to do dynamic invocation for quite some time. Let's take a look at a similar example using code that runs pre- PHP 5.3:

function heads() { echo "Heads!"; }
function tails() { echo "Tails!"; }
$function = rand(0,1) ? 'heads' : 'tails';
$function(); // <= Dynamic Invocation!

We start by defining two functions named heads and tails, respectively. We then flip a coin and assign the string "heads" or "tails" to $function. Finally we invoke $function as if it were a lambda and the correct function gets called! Under its breath PHP is saying "looks like a function call, but this is actually a string whose value is 'heads'. Do I know of a function named 'heads'? Yes! Initiate dynamic invocation thrusters in 5...4...3...2..."

What if $function's value was a string with no corresponding method name, or an integer, or a boolean? You would murder our friend PHP. You should know that death by dysfunctional dynamic invocation is a most cruel and unusual death.

$fatality = 'Finish Him!';
$fatality();
// Fatal error: Call to undefined function Finish Him!()

is_callable to PHP's defense

How can you stop yourself from murdering PHP with dysfunctional dynamic invocations? Enter: is_callable. If safety is your concern, is_callable is your defense. Pass any variable you want to is_callable and it will tell you true or false whether or not it can be dynamically invoked.

function heads() { echo "Heads!"; }
$function = 'heads';
is_callable($function); // true

$lambda = function() { echo "Tails!"; }
is_callable($lambda); // true

$kimJungIl = "fireNukesAndSingAboutLoneliness";
is_callable($kimJungIl); // false

With the powers of is_callable we can learn whether making a dynamic call to the contents of a variable makes sense. Notice that is_callable works on both the function name string variable $function, and the anonymous function $lambda. We can call them just the same, too, so they can be used interchangeably:

function heads() { echo "Heads!"; }
$callable = rand(0,1) ? 'heads' : function() { echo "Tails!"; };
if(is_callable($callable)) {
    $callable();
}

This is really sweet because with higher-order functions like map and reduce we can pass either a lambda as demonstrated in part II, or string that is the name of a first-class function, and it will work just the same. Dynamic invocation enables flexibility through indirection and late-binding. Now that we've seen dynamic invocations of functions and anonymous functions, what about methods and static methods?

Dynamically Invoking Instance and Static Methods

While practicing the powers of our ability cast dynamically invocations of methods and lambdas, an elder PHP programmer presents a challenge: dynamically invoke an object's method without our script becoming a statistic, just another PHP fatality. With a mischeivious grin she hints, "Try not with a string, you will. Do with a two element array, you must."

class Object { 
	function method() { echo "Great success!"; }
}
$callableArray = array(new Object, 'method');
if(is_callable($callableArray)) {
    $callableArray();
}

Taking the hint we place an instance of the object as the first element of our array, and the name of the method as the second. We protect ourselves by only commiting to the dynamic invocation if the array is callable. We're safe, right? So, we run the script... and we murder PHP. Fatal error: Function name must be a string The elder chuckles and walks away...

Use call_user_func and call_user_func_array for Robust Dynamic Invocation in PHP

PHP's achilles heel is its inconsitencies. It may never live them down. C'est la vie. It turns out is_callable doesn't mean directly callable, but rather callable using the utility functions call_user_func or call_user_func_array. Let's try that example again:

class Object { 
	function method($greatWho, $greatWhat) { 
		echo "$greatWho are Great $greatWhat!";
	}
}
$callableArray = array(new Object, 'method');
if(is_callable($callableArray)) {
    call_user_func($callableArray, "You", "Success");
    call_user_func_array($callableArray, array("You", "Success"));
}

"You are Great Success!" Says the script. Notice the subtle variation in how each is called. call_user_func takes a list of arguments following the $callable just like any other function call, whereas call_user_func _array takes an array that will be used as arguments. In practice this means you can use call_user_func when you know exactly the arguments you want to send to the callable, and call_user_func_array when you don't know how many arguments or when it varies. Can you think of a time when you wouldn't know the exact arguments to use in a dynamic invocation?

Thought of an example yet? How about if we wanted a generic way to wrap new behavior around a function? A simple example...

function wrapWithEcho($callable) {
    return function() use ($callable) {
        echo "Calling!";
        call_user_func_array($callable, func_get_args()); // !!!
        echo "Done calling!"; 
	};
}

function aDeliciousFunction($subject) { echo "Mmm, $subject!"; }
$aDeliciousFunction = 'aDeliciousFunction';
$aDeliciousFunction('candy'); 
// Mmm, Candy!
$aDeliciousFunction = wrapWithEcho($aDeliciousFunction);
$aDeliciousFunction('candy');
// Calling! Mmm, Candy! Done Calling!

function anotherDeliciousFunction($subject, $verb) {
	echo "I love to $verb $subject!";
}
$anotherDeliciousFunction = 'anotherDeliciousFunction';
$anotherDeliciousFunction('candy','eat'); 
// I love to eat candy!
$anotherDeliciousFunction = wrapWithEcho($anotherDeliciousFunction);
$anotherDeliciousFunction('candy','unwrap'); 
// Calling! I love to unwrap candy! Done calling!

The focal point of this example is near the three bangs (!!!). Our wrapWithEcho function is returning an anonymous function that closes over the callable and does not know, or care, how many arguments that the callable should be passed. In fact, we wrap two different functions that take two different parameter counts and it all just works! Without call_user_func_array, in a situation like this, we'd have to resort to a huge switch statement that counted the number of arguments and then invoked call_user_func with exactly the right count.

There is another way to dynamically invoke callables, using reflection, which will be the subject of a short follow-up post.

Calling all Callables! (Directly)

Let's have a little fun and tie together concepts from each post in the series thus far to create a uniform means for doing direct dynamic invocation. To accomplish this we'll use PHP's closures and anonymous functions (part I) to create a higher-order PHP function (part II) that will return a directly invokable value for all callables  (part III). We're going to totally unify our dynamic invocations so we don't have to use call_user_func in our code, but can instead do this:

function aFunFunction() { echo "Radical."; }
$function = callable('aFunFunction');
$function();

$lambda = callable(function() { echo "Awesome!"; });
$lambda();

class Object { 
    function method() { echo "Great success!"; }
    static function staticMethod() { echo "More great success!"; }
}
$method = callable(array(new Object, 'method'));
$method();

$staticMethod = callable(array('Object','staticMethod'));
$staticMethod();

Now we just need to write this callable method. If you think back to our wrapper example, it's got the hints we need. Our callable function will return an anonymous function that closes over the callable. Ready?

function callable($aCallable) { 
     if(!is_callable($aCallable)) { 
		throw new Exception("Callable only works on is_callable's!");
	}
    // Return our anonymous function
    return function() use ($aCallable) {
        return call_user_func_array($aCallable, func_get_args());
    };
}

Pretty awesome, but I know what you're thinking. You're right, you can do better. Remember what we started out learning in this post? There are two kinds of callables that are directly callable: strings and anonymous functions. (Aside: there are actually three, the third is an object that implements the magical __invoke method, we'll get there later in this series.) Right, strings and anonymous functions can be called directly, it's the callables in the form of an array that need call_user_func. We can make our code a lot faster by simply returning callable non-arrays because they're already directly callable!

function callable($aCallable) {
	if(!is_callable($aCallable)) { 
		throw new Exception("Callable only works on is_callable's!");
	}
    if(!is_array($aCallalbe)) return $aCallable;
    return function() use ($aCallable) {
        return call_user_func_array($aCallable, func_get_args());
    };
}

Awesome. Just awesome. Using a closure we've brought unification to direct dynamic invocation in PHP 5.3 with our shiny new callable function.

Just a simple example of the power and flexibility functional PHP 5.3 brings to the table. Are you enjoying this series? Let me know in the comments. If so, you should subscribe to our RSS feed as parts IV and V are on their way...

You can also follow along on Twitter to get updates as this series continues...

Understanding and Implementing the famous map and reduce functions - (Functional PHP 5.3 Part II)

Posted by Kris | Comments ( 2 ) | Filed in: PHP | | | | |

To demonstrate some uses of PHP 5.3's fun new anonymous functions let's implement the famous functions: map and reduce.

In Part I we deconstructed the differences between anonymous functions, lambdas, and closures in PHP. You may want to get comfy with those terms before continuing here. As a refresher, lambdas and anonymous functions are essentially the same idea: functions that are values just like integers and strings. Closures are what enable lambdas to refer to or use variables defined outside of the lambda, even after those variables have fallen out of scope. Now, onto map reduce.

We should chat about the butterflies, the birds, and the bees...

By now you've heard the cool kids at school raving about the euphoria of recursion and lambdas with higher order functions like map and reduce. We'll get there, we will, but first we need to have a little talk about butterflies, birds, and bees.

There's a moment in every little caterpillar's life when it becomes a butterfly. Metamorphosis is a transformative process. It's a transformation function, caterpillar in, butterfly out. (Caterpillar) -> (Butterfly) What is important to understand is not how metamorphosis works but that it is a function, a special kind of function that transforms a caterpillar into a butterfly.

Can you think of other functions that transform an input into an output? Sure: function square($x) { return x * x; } or strtolower or function filter($dirty) { /* clean up */ return $clean; }. There are a lot of transformer functions! Some, like square may take an input and give you the same type of input back int -> int. Others, like metamorphosis, will take one type of input and give you another caterpillar -> butterfly. For our purposes let's classify all of these types of functions simply transformer functions.

Transformer!

Are there other types of functions too? You bet. This is the point where we talk about the birds and the bees. On second thought, I've never really understood how birds and bees combine to make anything, so lets stick to butterflies. When two butterflies are come together in a certain way they create a new butterfly (yes, I know this analogy also has holes, just play along). Let's refrain from thinking about the details of how they come together, and focus on the beauty of what is happening. Two butterflies combining to make one! (Butterfly, Butterfly) -> (Butterfly). It's another function, a combiner function, that takes two like things and results in another like thing.

Combiner!

So, what does this have to do with map / reduce and functional programming in PHP?

map and reduce are higher-order functions. This means each has a parameter that is a function, but not just any function. Each takes a special type of function. map uses a transformer function and reduce uses a combiner function. map transforms a list of things, like say a bunch of caterpillars, into a list of another things, like butterflies. reduce reduces a list of things like butterflies until there's only one left. Here's a visual of how map reduce works:

The Code! For crying out loud, show us The Code!

Ok, ok. You get it! map transforms a list of things into a list of other things, reduce combines a list of things into one thing. What does this look like in PHP?

PHP has two built-in functions: array_map and array_reduce. (Unfortunately the two functions take the array and the callback parameters in different orders. We won't make that mistake when we implement map/reduce will we?) A callback has a lot of different meanings in PHP which we'll cover in the next post in this series. For now take a callback to mean an "anonymous function".

The canonical map/reduce example is word counting. So let's take on the challenge in PHP: given an array of strings we must count the occurrences of each word in all strings. Here is our input and desired output:

<?php
$lines = array(
             'one two three four',
             'two three four',
             'three four',
             'four',
             );
// Desired Output, array of type word => count
// array ( 'one' => 1, 'two' => 2, 'three' => 3, 'four' => 4, ) 
?>

Let's break down the problem into what we know:

  • Inputs: (String of Words)
  • Output: Array(Word => Count)

We need to get from our input type to our output type. How can we use a transformer function to get us half-way there? And a reducer to bring us home? Think of the caterpillars and the butterflies.

Our caterpillar is a plain old, space delimited, lowercase string. Our transformer function must take a single string and metamorphosize it into our butterfly, the output type: a single array where keys are words and values are counts. (Line of Words) -> (Array: Word => Count) Let's code this anonymous transformer function up (we'll cheat and use PHP's built-in array_count_values to count the words):

<?php
// Transforms (Line of Words) -> (Array: Word => Count)
$lineToWordCounts = 
    function($line) { 
        return array_count_values(explode(' ', $line));
    };
    
// Test on a single line:
var_export($lineToWordCounts('one two three four'));
// Output: array ( 'one' => 1, 'two' => 1, 'three' => 1, 'four' => 1, )

// Test with array_map:
$counts = array_map($lineToWordCounts, $lines);
var_export($counts);
// Output: array ( 0 => array ( 'one' => 1, 'two' => 1, 'three' => 1, 'four' => 1, ), 
//                 1 => array ( 'two' => 1, 'three' => 1, 'four' => 1, ),
//                 2 => array ( 'three' => 1, 'four' => 1, ),
//                 3 => array ( 'four' => 1, ), )
?>

Sweet mother nature! The strings have transformed into beautiful butterflies! Now all we need to do is combine them down into a single array. If we can write a combiner function that takes two word count arrays and combines them into one we can use reduce to combine all of them into one for a total word count. Let's take a stab:

<?php 
// Combiner (Array:Word=>Count,Array:Word=>Count)->(Array:Word=>Count)
$sumWordCounts =
    function($countsL, $countsR) {
        // Get all the words
        $words = array_merge(array_keys($countsL), array_keys($countsR));
        $out = array();
        // Put them in a new (Array: Word => Count)
        foreach($words as $word) {
            // Sum their counts
            $out[$word] = isset($countsL[$word]) ? $countsL[$word] : 0;
            $out[$word] += isset($countsR[$word]) ? $countsR[$word] : 0;
        }
        return $out;
    };
$totals = array_reduce($counts, $sumWordCounts, array());
var_export($totals);
// Output: array ( 'one' => 1, 'two' => 2, 'three' => 3, 'four' => 4, ) 
?>

Just like that we've implemented a multi-line word count using PHP's built-in array_map and array_reduce functions and our very own transformer and combiner lambdas. The map step transforms each line from a string into an array of word counts for that line. The reduce step combines those arrays of word counts into a single total count for all lines.

Now that we understand map's use of a transform function, and reduce's use of combine function, let's implement map and reduce on our own.

So, you're ready to implement map/reduce in PHP?

Map's implementation is so simple, it barely needs an explanation. All we're doing is calling the transform function on every input element and storing each result in an array to be returned. Here it is:

<?php
/**
 * $transformer lambda(caterpillar) -> butterfly
 * $in array of caterpillars
 */
function map($transformer, $in) {
    $out = array();
    foreach($in as $item) {
        $out[] = $transformer($item);
    }
    return $out;
}
?>

Not too bad, eh? Somewhere John Mccarthy's just stooled himself, I know. Hang tight, John, we'll get to recursion! Reduce is a little trickier because of edge cases where arrays with no or only 1 element are provided. Don't get caught in those details, just focus on the snippet where we've got more than 1 element to reduce:

<?php
/**
 * $combiner lambda(butterfly, butterfly) -> butterfly
 * $in array of butterflies
 */
function reduce($combiner, $in, $identity) {
    if(count($in) <= 1) {
        $out = $identity;
    } else if(count($in) > 1) {
        $out = array_shift($in);
        do {
            $next = array_shift($in);
            $out = $combiner($out, $next);
        } while(!empty($in));
    }
    return $out;
}
$totals = reduce($sumWordCounts, map($lineToWordCounts, $lines), array());
var_export($totals);
// Output: array ( 'one' => 1, 'two' => 2, 'three' => 3, 'four' => 4, ) 
?>

With reduce we begin by shifting the first butterfly off the list. We combine it with the next butterfly to make a super butterfly. We then combine the super butterfly with the next butterfly on the list, and so on until we have the ultimate single butterfly. This is a little trippy at first so refer back to the diagram for a visual explanation.

Map/Reduce, Take 2: Recursive Implementations, Please

Remember those crazy trips to lambdaland full of recursion all the cool kids on Reddit and HN are taking? Well, your time has come, too. Our initial implementations of map and reduce were done imperatively with loops. The functional folks believe loops are, well, garbage. (Don't worry, they're not.) Why do you need loops when you have recursive functions? How would map and reduce look if we got rid of the loops?

Before we get there, let's implement a few helper functions to make our recursive implementations beautiful. Here they are:

<?php
// First element of an array
function first($in) { 
    return empty($in) ? null : $in[0];
}

// Everything after the first element of an array
function rest($in) {
    $out = $in;
    if(!empty($out)) { array_shift($out); }
    return $out;
}

// Take an element and an array
//  and fuse them together so that the element
//  is at the front of the array
function construct($first, $rest) {
    array_unshift($rest, $first);
    return $rest;
}
?>

Now that we've got those helpers out of the way, allowing us to work with arrays like they're lists, let's do recursive map and reduce functions in PHP.

<?php
/**
 * $transformer lambda(caterpillar) -> butterfly
 * $in array of caterpillars
 */
function map($transformer, $in) {
    return !empty($in) ?    construct(  $transformer(first($in)), 
                                        map($transformer,rest($in)))
                        :    array();
};

/**
 * $combiner lambda(butterfly, butterfly) -> butterfly
 * $in array of butterflies
 */
function reduce($combiner, $in, $identity) {
    return !empty($in) ?    $combiner(first($in),
                                      reduce($combiner, rest($in), $identity))
                         :  $identity;
};

$totals = reduce($sumWordCounts, map($lineToWordCounts, $lines), array());
var_export($totals);
// Output: array ( 'one' => 1, 'two' => 2, 'three' => 3, 'four' => 4, ) 
?>

If you're more comfortable with for loops than recursion this is going to look trippy. Chew on it a little longer.

Some things that may help you reason about the recursive version of map:

  1. If there is at least one caterpillar, it will be transformed. map then calls itself with any remaining caterpillars.
  2. Once there are no caterpillars left, this is our base case: an empty array is returned.
  3. Once the empty array is returned, the previous map will construct a new array with its butterfly and return the array.
  4. The construction of the butterflies array happens one-by-one until every butterfly is in it, and the top-level map returns.

How did that feel for you? Beautifully functional? No loops, yet, we're iterating through a list!

I'll leave the interpretation of reduce for you to mull over. We've now written higher-order functions by implementing map and reduce, imperatively and functionally, in PHP 5.3 with anonymous functions.

So, That Was Fun. What's Next?

We've deciphered lambdas, anonymous functions, and closures, now we've seen higher order functions with callbacks by implementing map/reduce, so what's next? Follow the RSS feed for the upcoming articles in this series on functional PHP 5.3.

You should follow me on Twitter to get updates as this series continues to evolve.

Functional PHP 5.3 Part I - What are Anonymous Functions and Closures?

Posted by Kris | Comments ( 3 ) | Filed in: PHP | | | | |

One of the most exciting features of PHP 5.3 is the first-class support for anonymous functions. You may have heard them referred to as closures or lambdas as well. There's a lot of meaning behind these terms so let's straighten it all out.

What is the difference between Anonymous Functions, Lambdas, and Closures?

You'll see the terms "anonymous functions", "lambdas", and "closures" thrown around in reference to the new features of PHP 5.3. Even the the URL php.net/closures redirects to php.net/manual/functions.anonymous.php. The difference between 'lambda' and 'anonymous function'? None, for all intents and purposes they are two words for the same concept which is a descendant of lambda calculus. Languages with anonymous functions consider functions to be first-class value types, just like integers or booleans. Anonymous functions can thus be passed as arguments to another function or even returned by a function. Let's make this concrete in PHP:

<?php
$lambda = function() { 
            echo "I am an anonymous function, 
                  aka a lambda!<br />";
            };
$anonymousFunction = $lambda;
$anonymousFunction(); 
// Output: I am an anonymous function, aka a lambda!

function nCallsTo($n, $function) {
    for($i = 0; $i < $n; $i++) {
        $function();
    }
    return function() { echo "I am also an anonymous function!<br />"; };
}

$anotherAnon = nCallsTo(3, $anonymousFunction);
// Output:
// I am an anonymous function, aka a lambda!
// I am an anonymous function, aka a lambda!
// I am an anonymous function, aka a lambda!

$anotherAnon();
// Output: I am also an anonymous function!
?>

Notice how we did not assign a name to the function, we assigned a function to be the value of a variable. Just like a string or any other primative. We then assign it to another variable. The function is just a value, it has no name, hence the term "anonymous function". We then create a regular function named nCallsTo that takes two arguments, $n being the number of times to make a call to $function an anonymous function.

The existance of higher-order functions opens the door for techniques like map/reducenext post.nCallsTo is a higher-order function on two accounts: 1) it takes a function as an argument, and 2) it returns a function as a value. Higher-order functions open the doors for techniques like map/reduce and deserves a post in itself. The point is lambdas and anonymous functions are the same things: functions that are values.

If anonymous functions are values, what does PHP consider their type to be? Let's find out:

<?php 
$lambda = function() { echo "anonymous function"; };
echo gettype($lambda) . '<br />';
// Output: object
echo get_class($lambda) . '<br />';
// Output: Closure
?>

On the Closure object in PHP 5.3

What is a closure? So far it's a misnomer. We haven't actually spotted a closure even though PHP assigns all anonymous functions the type Closure. Since we haven't actually seen a closure yet, let's take a look at one:

<?php
function letMeSeeAClosure() {
    $aLocalNum = 10;
    return function() use (&$aLocalNum) { return ++$aLocalNum; };
}
$aClosure = letMeSeeAClosure();
echo $aClosure();
// Output: 11
echo $aClosure();
// Output: 12
$anotherClosure = letMeSeeAClosure();
echo $anotherClosure();
// Output: 11
echo $aClosure();
// Output: 13
echo $aLocalNum;
// Notice: Undefined Variable: aLocalNum
?>

Chew on that for a minute. Do you spot the funny business? $aLocalNum is a local variable defined within the scope of the plain-old function letMeSeeAClosure. With the new use syntax the variable $aLocalNum is bound or closed over to create the closure. This allows the returned Closure to retain a reference to $aLocalNum, even after $aLocalNum falls out of lexical scope when the function returns. The notice error occurs when trying to reference $aLocalNum directly from outside of the function's scope.

To recap, the terms, 'lambda' or 'anonymous function' refer to the same concept: functions that are values. Closures refer to a related, but different concept: the lifetime of a variable that is 'closed over', or in PHP 5.3 use'd, by a closure, is bound to the lifetime, or extent, of the closure. Anonymous functions that are constructed with the use keyword are also closures. As mentioned, in PHP 5.3, anonymous functions are typed as Closure and the three words have, so far, been thrown about. The high-order bit to take away is an understanding that PHP 5.3 now includes language features for anonymous functions and closures.

More on Functional PHP 5.3

If you're interested in PHP software development you should subscribe to our feed. Upcoming parts to this series:

Follow this series on RSS.

Models and Relationships at a Glance

Posted by Kris Jordan | Comments ( 9 ) | Filed in: PHP | | | | |

The Recess Framework is equip with a lightweight Object-Relational Mapping system that simplifies common database tasks. If you've encountered Active Record style ORMs in other frameworks like Ruby on Rails', Cake's, or Django's the concepts should be familiar: Recess Models use a Djagno-style method chaining API with Rails-style relationships.

A Quick Look

Lets say we need a controller method that is given a user $id and $keyword. The method will find the tags of all posts whose title contains $keyword that was written by the user with id of $id.

/** !Route GET, user/$id/keyword/$keyword/tags */
function getTagsForUserByPostTitleKeyword($id, $keyword) {
	$this->user = new User($id);
	$this->tags = $user
					->posts()
					->like('title',"%$keyword%")
					->tags();
}

In a view we can iterate through the tags with a simple foreach:

foreach($tags as $tag) {
	echo $tag->name, '<br />';
}

Now, let's look at the code for the User, Post, and Tag models that make the above code snippet possible. Not much code at all...

/** !HasMany posts */
class User extends Model { }

/** 
 * !BelongsTo user
 * !HasMany tags, Through: PostsTags
 */
class Post extends Model { }

/**
 * !HasMany posts, Through: PostsTags
 */
class Tag extends Model { }

/**
 * This model represents the join table between the many-to-many
 * Posts <-> Tags relationship.
 *
 * !BelongsTo post
 * !BelongsTo tag
 */
class PostsTags extends Model { }

Creating Models with Recess Tools

The Recess Framework includes a web app to aid development called 'Recess Tools'. Generating new Models for an application and creating corresponding tables in the database is quick work. By browsing to your application and selecting 'new' Model you'll be taken to the new Model helper. After providing a name, table information, and properties the model and, if needed, table will be generated.

A quick peak at the code generated for Post:

<?php
/**
 * !Database Default
 * !Table posts
 */
class Post extends Model {
	/** !Column PrimaryKey, Integer, AutoIncrement */
	public $id;

	/** !Column String */
	public $title;

	/** !Column String */
	public $body;

	/** !Column Integer */
	public $authorId;

}
?>

Lots of annotations! Why so many? Being explicit is a good thing- especially when you don't have to do any of the extra writing. Starting from the top: the Database annotation is what allows Recess to have Models from multiple data sources in a single app. The Table annotation is straightforward: the name of the table in the database the Model maps to. Following is a HasMany relationship using a join table with the Through argument. For Rails folks this should look decently familiar. More to come on relationships.

Each property uses a Column annotation to provide additional semantic typing information. Specifying properties and column mappings is optional in Recess but it is encouraged for three reasons. One, you can look at a Model's code and know exactly which properties are available. This is different from Rails or Cake models. Two, Recess checks to ensure your annotations and the database types match. Three, Recess can regenerate tables from Models marked up with annotations.

Simple Queries

Queries are constructed using a simple API. They're also lazy so queries are not executed until you need the results. The following are some example queries we could perform with the Post model:

$post = new Post();

$allPosts = $post->all(); // Select all Posts

$postsWithPhp = $allPosts->like('title', '%PHP%');

$lastFivePhpPosts = $postsWithPhp->orderBy('id DESC')->limit(5);
// Logically equivalent to:
$lastFivePhpPosts = $allPosts
						->orderBy('id DESC')
						->limit(5)
						->like('title', '%PHP%');

$postWithAuthorId5 = $post->equal('authorId', 5);

The all() method is essentially a SELECT * on the Model's table. Notice how we can take the result of all() and continue refining our query with a LIKE clause on line 3. As mentioned above, Recess Models use lazy evaluation. The code above would not issue any SQL queries unless later in code the results were accessed in a foreach loop or with the array index syntax (i.e. $postsWithAuthorId[0]). Other simple query operators include: notEqual, between, greaterThan, greaterThanOrEqualTo, lessThan, lessThanOrEqualTo, notLike.

Write to the Database with Insert/Update/Delete

Persisting changes is as simple as calling save() for INSERTs or UPDATEs and delete() for DELETEs. Let's take a look:

$post = new Post();
$post->title = 'Hello World';
$post->body = 'Welcome to Models in Recess!';
$post->save(); // internally calls $post->insert()
$post->title = 'Hello World! With a Bang!';
$post->save(); // internally calls $post->update()
echo 'New Post ID: ', $post->id;

$oldPost = new Post(10); // Post with ID 10
if($oldPost->exists()) {
	$oldPost->delete();
}

$postsWithPhp = new Post();
$postsWithPhp->like('title', '%Ruby%')->delete();

In lines 1-7 a new post is created saved and updated with some filler values. Lines 9-12 delete a Post with an ID of 10. Finally lines 14-15 delete all posts containing 'Ruby' in the title. Recess has support for cascading deletes across relationships and this will be the topic of a future post.

Relationships

A Post belongs to a User and has many Tags. Lets take another look at how this is represented using annotations:

<?php
/**
 * !BelongsTo user, Key: authorId
 * !HasMany tags, Through: PostTag
 */
class Post extends Model { /** Stripped for Brevity */ }
?>

The BelongsTo annotation denotes the 'one' side of a one-to-many relationship. We specify some additional information using the Key modifier to say that the foreign key column name is actually authorId instead of userId which is what it would be by convention. With a belongs to relationship Post has an attached method of user() which will return the User model a post is associated with. It also adds attached methods for setting and unsetting the user: setUser($user) unsetUser(). Attached methods are a low-level feature of Recess written in RecessObject which allow methods to be added to classes dynamically at run time.

The HasMany annotation is a special variant of HasMany because it uses the Through modifier. This tells the HasMany relationship to use a join table thus making it a many-to-many relationship instead of a one-to-many. The HasMany annotation attaches the following methods to the Post class: tags(), addToTags($tag), and removeFromTags($tag).

Chaining it all Together

Queries can be chained across relationships. To revisit the example that started us off lets find all tags of posts with PHP in the title written by a single user:

$this->user = new User($id);
$this->tags = $user->posts()->like('title',"%$keyword%")->tags();

Notice how we're traversing relationships and adding criteria along the way. We could have even more fun with the following:

$this->tags = Make::a('User')
				->equal('id', 1)
				->posts()
				->like('title',"%$keyword%")
				->tags()
				->orderBy('name ASC')
				->limit(10);

The Make::a($className) method is a little short hand to instantiate a new User object and be able to chain methods directly off of it. *sigh* If only PHP allowed (new User)->... but I digress. The point to be made is that as we traverse relationships and add new criteria it is applied to the last Model referenced. So the 'like' after posts applies to Post.

Try Out Recess

So that's a quick, general look at Models in the Recess Framework. Want to try it out? Register for the Recess Preview Program and download away. Happy Modeling!

On Hiding our Dick Tracy Underwear behind Annotations

Posted by Kris Jordan | Comments ( 0 ) | Filed in: PHP | | | | |

Recess uses annotations as a means to a more declarative style of programming in PHP. Plus, it keeps our underwear nice and hidden.

I remember it like it was yesterday. An entire classroom was laughing and I was the center of attention. That’s a lie. My Dick Tracy underwear were. The girl I had an elementary school crush somehow snuck up behind me and dropped my trousers. Did my 2nd grade peers really need to know that I was wearing whity-tighty briefs screen printed with a cartoon character and his bad-ass watch phone from the future? No. Everyone would have been just without those details.

“Don’t expose your underpants” is an often said phrase amongst systems builders. Well designed systems hide implementation details from interfaces. This philosophy manifests itself in many ways and is beyond what a single blog post can capture so let’s focus on one particular strategy for not exposing our guts: declarative programming.

Declarative programming is emphasizing “what” you want to accomplish rather than “how” to. Take a look at the following pseudo-code:

Mom: which drawer is my newest pair of clean Dick Tracy underwear in?

If you were to ask MySql that question it would tell you: ‘Check your manual for the right syntax to use near “Hey Mom”’. Hmm. Let’s try again in SQL:

SELECT drawer_number FROM underwear WHERE print = “Dick Tracy” AND last_worn_on < last_washed_at ORDER BY date_purchased DESC

That’s not too far removed from our first request. We’re telling a SQL server what we want, not how to do it. To express this in Java would look something (very roughly) like:

public Underwear getNewestCleanPair(List underwears) {
	List results;
	for(Underwear u : underwears) {
		if(u.getPrint().equals(“Dick Tracy”) && u.getLastWornOn() < u.getLastWashedAt()) {
			results.add(u);
		}
	}
	if(results.size() > 0) {
		Collections.sort(results, new UnderwearAgeComparator());
		return results[0];
	} else {
		return null;
	}
}

Notice how in Java we’re achieving the same result by more explicitly specifying how to find the result. This is a more imperative style of programming as compared to SQL.

There are two resounding reasons for promoting a declarative style when possible. The first is evident from reading the code listings above: it’s more concise. Why? You’re working with a more appropriate level of abstraction perhaps even a domain-specific language like SQL. There are fewer details. A lack of details leads us to the second big reason: the “underwear”, the implementation, is someone else’s problem.

Your query doesn’t need to know MySQL uses B-Trees or R-Trees or quicksort. An underlying beauty of this is that it allows MySQL to change its implementation, improve its algorithms, and refine its data structures without changing the way you interact with it.

What does this have to do with Recess, anyway?

So far the most controversial feature of Recess is the use of annotations. The idea of putting metadata within a DocComment block sits nicely with some folks and is an outrage to others. Some of the initial feedback annotations have received:

“Hmm, so your [annotations] are ‘structural comments.’ Isn't this what XML is for?” ~Cosmo7

“Please don't make us use comments to annotate code in a way that is meant to be understood by computers. It's evil and defeats the whole purpose of commenting the code for your fellow humans.” ~Ivan

These are completely warranted concerns and I expected this reaction when making the design decision. So why make such a controversial design decision?

Annotations enable a more declarative style of programming/metaprogramming.

In Recess, where possible, there is an emphasis on telling the framework what you want rather than how to do it. Currently this is best seen in routing in controllers and object/relational mapping in models. The ability to add a relationship to a model, like a has_many relationship for the Rails-goers or a ForeignKey for Django-ites, is declarative short hand that sidesteps significant amounts of implementation detail (i.e. has_many introduces additional methods/properties on a model that did not exist before).

DocComments are the only PHP5 supported means for declaratively saying anything “meta” about a class, method, or property.

PHP5 does not have support for attributes or annotations. It does have support for DocComments, though, and DocComments are the only native, first-class way PHP gives us to say something about a class, method, or property. Any other mechanism used implies a user-defined association between some metadata and a PHP construct.

There are innumerable ways we could go outside of PHP and establish systems for providing metadata about code. In order to do this, though, we would be custom writing the machinery to associate with SomeController::foo() in PHP. DocComments are PHP’s way of giving us room for metadata within the code that is designed to be associated with classes, methods, and properties.

The other side-effect of designing a non-DocComment system is that there is duplication of information. In the example above your XML duplicates knowledge in your code (that SomeController has a method foo()). This duplication inhibits programmers from having fun. Now every time you want to refactor a class, perhaps something as simple as changing a method name, you have to go open another file and propagate those changes manually. There’s no fun in that.

We can hide our Underwear

Until PHP 5.3 there are some serious shortcomings to the PHP object-model. For Recess, the biggest problem is a lack of “late static binding”. What is that? Imagine you declare an abstract base class, say Model, that has a static property, say $table, and a static method, say findAll() that returns all rows of the table. Now we sub-class Model in a new class called Post. Post overrides the value for $table with ‘posts’. One would expect to call Post::findAll() and be returned all rows in the ‘posts’ table. With late static binding, in PHP 5.3, this will be true. Versions of PHP prior to 5.3 ignore the inheritance hierarchy with static method calls.

Cake gets around the late static binding issue by duplicating ‘class’ level information in every instance. Every single Model instance in Cake holds a copy of the table name, relationships, etc. This is a violation of OO that has degraded performance in both time and space. Wasted time spent building those data structures in every constructor and wasted space carrying around duplicate information. What’s worse is that for Cake to change its API once 5.3 becomes the norm will require Cake users to change their code and learn a new interface. Because we’re hiding our underwear in Recess with annotations we’ll be able to change our underlying implementation to take advantage of these PHP 5.3 features without forcing a change in the API.

Conclusion

So, even if Recess is wearing really awesome underwear featuring Dick Tracy and a bad-ass wristwatch from the future you don’t need to know that in order to use the framework. But it is open source, after all, so I suppose you can take a peek, if you’re into that kind of thing.