Introduction

This repository contains our ongoing work on PHP program analysis.

Running Our Software

The main prerequisites to running the PHP and Rascal code used to implement our analysis are:

• Java JDK version 1.7, available from the Java download
• Eclipse, available on the Eclipse download page
• PHP, available from the PHP download page in source format or as part of the Mac Xcode toolset

Eclipse versions 3.7 and 4.2 (Juno) should both work fine. The latest version of PHP should also work -- we use PHP to parse PHP files, but otherwise just pick the version you need to run the PHP software you plan to use (variants of 5.3 and 5.4 should both work).

Once these are installed, you can install the Eclipse Rascal plugin by following the directions at the Rascal download page.

Note that we are actively developing Rascal, which means the standard plugin is more stable, but potentially slower and missing some new features. You can use the Rascal unstable release channel to get the most recent version of Rascal which builds and passes all unit tests. This release is available at http://www.rascal-mpl.org/unstable-updates.

Also, the Rascal installation instructions specify that you should allocate 1GB of RAM for Rascal; in some cases, since we are working with the parsed representation of large software systems, this may need to be adjusted upwards even beyond this. For most functionality in the analysis, a maximum heap size of 4GB is fine. Some operations may require more, especially those that work over not just one system, but multiple systems at the same time, for instance in the work we are doing on comparing how features are used in various systems.

To parse PHP code, we are using a fork of an open-source PHP Parser. This is also available in our Github repository, and is named PHP-Parser.

Note that we assume, for this README, that all code is being placed in directory ~/PHPAnalysis. To check out the parser from Github, one would do:

cd ~/PHPAnalysis
git clone https://github.com/cwi-swat/PHP-Parser.git

Downloading the Corpus

The corpus is rather large, so here we are providing just those systems that we used in our experiments in our ISSTA 2013 submission (the same corpus was used in our ICSE 2013 submission) and in our ESEC/FSE 2013 submission. The first part of the corpus is used in both papers, while the second is used just in the ESEC/FSE 2013 submission.

Assuming that wget is installed:

cd ~/PHPAnalysis
wget http://homepages.cwi.nl/~hills/experiments/corpus-icse13.tgz
tar -xpzvf corpus-icse13.tgz

and, if needed:

wget http://homepages.cwi.nl/~hills/experiments/corpus-includes-extension.tgz
tar -xpzvf corpus-includes-extension.tgz

Assuming just the first, this will place the files into a subdirectory named corpus-icse13. If wget is not installed, click on the base corpus link, save this to the ~/PHPAnalysis directory, and extract it with the command given above.

Checking Out the PHP Analysis Project

To use the PHP Analysis code, you will first need to check it out from its Github repository. This can be done either with the git command-line tools or using the git support built in to Eclipse. For instance, you can import the PHP Analysis project using the Eclipse Import functionality. There is currently only one branch, master, which should be checked out.

If you use the Eclipse functionality for checking out the Github repository, make sure to NOT put the repository in the same location that Eclipse will create the project (i.e., under the workspace location you selected when you started Eclipse). If you put the repository in this location, you will get an error, since Eclipse will try to create a directory for the project with the same name as the directory you gave for the repository.

Making Configuration Changes

Once the PHP Analysis project is imported, the paths to several files need to be modified. This can be done in Rascal module lang::php::util::Config, found in the project under src/lang/php/util in file Config.rsc.

• phploc points to the location of the php binary
• parserLoc points to the location of the PHP-Parser project
• rgenLoc and rgenCwd then indicate the actual file and working directory used to perform parsing
• baseLoc provides the base location for a number of files created as part of the parsing and extraction process, including the directory where parsed files are stored and the root of the corpus

Given the working directory mentioned above, the configuration file would contain the following lines. Obviously, /Users/mhills should be substituted for the location of your home directory, or whichever other directory PHPAnalysis has been installed in:

module lang::php::util::Config

public loc phploc = |file:///usr/bin/php|;

public loc parserLoc = |file:///Users/mhills/PHPAnalysis|;
public loc rgenLoc = parserLoc + "PHP-Parser/lib/Rascal/AST2Rascal.php";
public loc rgenCwd = parserLoc + "PHP-Parser/lib/Rascal";

public loc baseLoc = |file:///Users/mhills/PHPAnalysis|;
public loc parsedDir = baseLoc + "parsed";
public loc statsDir = baseLoc + "stats";
public loc corpusRoot = baseLoc + "corpus-icse13";
public loc countsDir = baseLoc + "counts";

public bool useBinaries = false;

Make sure that useBinaries is false; this should only be true in cases where you have the binaries built and no longer have the source.

Building Binaries for Each Corpus Item

To build the binaries, first load lang::php::util::Utils in a Rascal console (right click in an open Rascal file, such as the Utils module mentioned above, and select Start Console, then import lang::php::util::Utils). Then, you can run the command to build all the binaries:

import lang::php::util::Utils;
buildBinaries();

Several of the files do not parse, so parse errors are reported in the console. These are expected errors: one of the files, part of the Zend Framework, is intended to trigger a parsing error, while the others are template files that are intended to be filled in and are not valid PHP. There are also several files that show the error Expected Script, but got node; this did not occur during our tests with Eclipse 3.7 running on Linux, but this occurred with 3 files while running with Eclipse 4.2 on Mac OS X. We are working to track down the source of this bug and repair it.

Building Binaries with Includes Information

At runtime, the actual program is the PHP page that is visited with all includes resolved. We have developed an analysis that tries to resolve these includes statically, and are currently working on improving this. To use this information in the analysis, we first build binaries (as above), but with includes resolved, where possible, to literal paths. To do this, run the following code:

import lang::php::stats::Unfriendly;
writeIncludeBinaries();

This will generate a new binary, with includes resolved, for each product in the corpus. Note that this is needed to recreate the feature usage information, since that accounts for features included transitively where possible.

Running the PHP Feature Usage Analysis Code

The code to extract information on PHP feature usage is in folder lang/php/stats, shown as package lang.php.stats in Eclipse. Most of this code is in the Rascal module lang::php::stats::Unfriendly, in file Unfriendly.rsc. This module contains code that measures the use of "analysis unfriendly" features such as variable variables and magic methods. This code also uses a number of functions defined in Stats.rsc for extracting information on the uses of individual features from PHP files.

To make our experiments easily reproducible, folder lang/php/experiments contains a subdirectory for each set of experiments, categorized by the paper in which they were collected, with a file containing calls that build the figures and tables. For instance, module lang::php::experiments::issta2013::ISSTA2013 contains one function for each table and figure in the submitted paper. Tracing through these functions shows the analysis steps taken to yield the results we reported.