PHP and Progressive Uploads : A small tutorial

Tutorials on PHP, databases and other aspects of web development. Before posting a question, check in here to see whether there's a tutorial that covers your problem.

Moderator: General Moderators

User avatar
infolock
DevNet Resident
Posts: 1708
Joined: Wed Sep 25, 2002 7:47 pm

PHP and Progressive Uploads : A small tutorial

Post by infolock »

So recently I was given the task to make a Progress Bar work with PHP. From my own research a few years ago, this task was impossible and so I thought I would need some sort of Flash + Javascript + PHP combination to make this happen.

Until now.

I found a nice recipe from IBM that actually makes this process much simpler. It's by using a simple PECL extension called Alternative PHP Cache (APC for short). This brilliantly simple extension allows us to FINALLy be able to monitor file upload progress! I read the tutorial and right away, I was throwing my fists into the air: it's finally possible to do progress bars with PHP WITHOUT THE NEED OF CGI !!!!

For anyone who has ever run into this issue, you are probably the only ones to truly respect this awesome new toy. What's even more bizarre is that this functionality has been out since PHP 5.2 and this is truly the first I've ever heard about it.

My problems though soon caught up with me. Apparently IBM's little "tutorial" is a little short-winded and doesn't exactly cover all the bases that are needed to make this new extension work. So, being the great guy I am, I decided to write a step-by-step tutorial on how to use this for the weak at heart.

================================================================================================
*** INSTALLATION ***

First, you need to get the APC module. There are 2 downloads available:

Linux and Windows


Next, you need to Install the module.

-- FreeBSD/Linux Users only --
First things first: you'll need root to make all this work, so su into the root user.

After downloading the APC package (I downloaded the 3.0.18 stable bundle), you need to unpackage it. Once you have untar'ed the file, you need to cd into the apc directory and run the following set of commands:

Code: Select all

 
phpize
./configure --enable-apc --enable-mmap
make install
cp modules/apc.so /usr/local/lib/php/
 
The phpize command is used to prepare the build environment for a PHP extension. If you don't have phpize installed, you will want to do so via RPM, ports, or whatever method your nix distro supports.
Here are some commands for a few linux distros:
Fedora Core x:

Code: Select all

yum -y install php-devel
Gentoo Click here for the gentoo quick guide
FreeBSD - Installed by default with PHP

I can't cover them all, but these 3 seem to be the most popular (along with Ubuntu but I'm not familiar with it)

next, you'll need to open the php.ini file to make a few quick changes.

Code: Select all

 
vi /usr/local/etc/php.ini
 
Locate where the Windows Extensions are loaded (yes I know you aren't using Winblows, but you'll see what I'm getting at next).

Below the ;extension=php_zip.dll line, add a new line and enter the following extension:

extension=apc.so

Return a few lines down and enter the following:

Code: Select all

 
;;;;;;;;;;;;;;;;;;;;;;;
; APC LAYER CONTROL   ;
;;;;;;;;;;;;;;;;;;;;;;;
apc.rfc1867=on
apc.max_file_size=200M
upload_max_filesize=200M
post_max_size=200M
 
Now, you do NOT have to use 200M. I simply did to test out uploading huge 191M video files to see if this progress bar truly works. Adjust this to suit your own personal needs.


Finally, save the file and exit by typing:

Code: Select all

 
:wq!
 
You'll need the bang(!) because php.ini is by default a read-only file.

The last step now is just to restart apache:

Code: Select all

 
apachectl restart
 
apachectl is the command that restarts Apache in FreeBSD. However, it may or may not be available in your distro/PATH (user's profile) so you may need to browse into that directory and run the appropriate server command used in your distro (example: /etc/rc.id/init.d/httpd -k restart).


Since I have never done this on windows, I'm just going to copy/paste IBM's installation method here. NOTICE: It's asking you to use the WAMP install package, but most developers here manually have their's installed. If you have issues, just post it here and we'll see if a solution can't be found.
--- WINDOWS USERS ONLY ---
APC is not enabled by default in PHP V5.2. Since the new hooks are a part of APC, we need to make sure to install the extension and make it available to the PHP interpreter. This is accomplished by downloading the php_apc extension files. In our case, we are using an installation of WAMP, a freely available packaged PHP for Windows®, which includes Apache and MySQL. It offers a nice user interface and is easy to manage with menus that support configuration options.

To set up APC on WAMP:

1. See Resources to download the libraries and WAMP.
2. Install WAMP.
3. Put the php_apc.dll file in the extensions folder for PHP. This is <wamproot>/php/ext by default.
4. Use the system tray WAMP menu to select PHP settings>PHP Extensions>Add Extension.
5. In the command-line interface that pops up, type php_apc.dll and press Enter.
6. Using a text editor, open <wamproot>/php/php.ini and add the line apc.rfc1867 = on (it doesn't matter where). If you're trying to test locally and plan to upload large files so you can actually see progress, you'll also want to add the following directives: apc.max_file_size = 200M, upload_max_filesize = 200M, and post_max_size = 200M. Don't do this on a live production server, though, or you're likely to use up bandwidth and disk space allotments, not to mention slowing everyone else down to a crawl.
7. Restart the webserver.

APC should now be set up and initialized. The RFC1867 features of APC — the features that enable you to track file uploads — should now be enabled as an option, and you should be ready to look into our file uploads to enable real-time status.


If you have trouble installing this on windows, just post and I'm sure we can figure it out. In the meantime, I think these steps are self-explanatory enough to get you started.



================================================================================================
Testing out the installation

Testing out the installation is pretty simple. Just go into the directory where you initially unpackaged the APC bundle and find the file apc.php. Copy this file into your docroot and open it up in your browser. If the installation worked, you should see a big long explanation about all of APC's features, along with a nice pie chart if the GD library is also installed.



================================================================================================
Making A Progress Bar Test

Finally, we're ready to test out this progress bar stuff! It's very freaking easy, and I'm first going to just give you all the code you'll need.


progress.php

Code: Select all

 
<?php
   $id = uniqid("");
?>
<html>
<head><title>Upload Example</title>
 
<script type="text/javascript">
 
function getProgress(){
  CDownloadUrl('get', "getprogress.php?progress_key=<?php echo($id)?>", 
    function(percent) {
      document.getElementById("progressinner").style.width = percent+"%";
      if (percent < 100){
        setTimeout("getProgress()", 100);
      }
    }
  );
}
 
function CDownloadUrl(method, url, func) {
   var httpObj;
   var browser = navigator.appName;
   if(browser.indexOf("Microsoft") > -1)
      httpObj = new ActiveXObject("Microsoft.XMLHTTP");
   else
      httpObj = new XMLHttpRequest();
 
   httpObj.open(method, url, true);
   httpObj.onreadystatechange = function() {
      if(httpObj.readyState == 4){
         if (httpObj.status == 200) {
            var contenttype = httpObj.getResponseHeader('Content-Type');
            if (contenttype.indexOf('xml')>-1) {
               func(httpObj.responseXML);
            } else {
               func(httpObj.responseText);
            }
         } else {
            func('Error: '+httpObj.status);
         }
      }
   };
   httpObj.send(null);
}
 
function startProgress(){
  document.getElementById("progressouter").style.display="block";
  setTimeout("getProgress()", 1000);
}
 
</script>
</head>
<body>
<iframe id="theframe" name="theframe" src="upload.php?id=<?php echo($id) ?>" style="border: none; height: 100px; width: 400px;" > </iframe><br/><br/>
<div id="progress_win"></div>
 
<div id="progressouter" style="width: 500px; height: 20px; border: 6px solid red; display:none;">
   <div id="progressinner" style="position: relative; height: 20px; background-color: purple; width: 0%; "> </div>
</div>
 
</body>
</html>
 
upload.php

Code: Select all

 
<?php
$id = $_GET['id'];
 
?>
 
<form enctype="multipart/form-data" id="upload_form" action="target.php" method="POST">
<input type="hidden" name="APC_UPLOAD_PROGRESS" id="progress_key" value="<?php echo $id?>"/>
<input type="file" id="test_file" name="test_file"/><br/>
<input onclick="window.parent.startProgress(); return true;" type="submit" value="Upload!"/>
</form>
 
target.php

Code: Select all

 
<?php
if($_SERVER['REQUEST_METHOD']=='POST') {
  move_uploaded_file($_FILES["test_file"]["tmp_name"], "/usr/local/www/apache22/data/progressbar/uploads/" . $_FILES["test_file"]["name"]);
  echo "<p>File uploaded.  Thank you!</p>";
}
?>
 
getprogress.php

Code: Select all

 
<?php
header('Expires: Mon, 26 Jul 1997 05:00:00 GMT');
header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT");
header('Cache-Control: no-store, no-cache, must-revalidate');
header('Cache-Control: post-check=0, pre-check=0', FALSE);
header('Pragma: no-cache');
 
if(isset($_GET['progress_key'])) {
  $status = apc_fetch('upload_'.$_GET['progress_key']);
  echo $status['current']/$status['total']*100;
}
?>
 

========================================================================================
Some Explaining to Do

First, let me say that the above code is direct code examples from IBM, but altered a little bit to make our lives much simpler.

In the first file, progress.php, IBM originally had a Google MAPS API work-around using the GDownloadUrl function. As anyone knows who has tried to use this behind a firewalled web server, you can't validate your KEY (i.e.: <script src="http://maps.google.com/maps?file=api&v= ... OURKEYHERE" type="text/javascript"></script>) without Google being able to access your server. Since this is true, the now awesome GDownloadUrl function is unusable. Fortunately I found the same idea they use for this function already exists in a separate implementation called CDownloadUrl that does not require Google Authentication. As I suspected, it's basically just an XHTTPRequest method that asks for the results of getprogress.php


Also in progress.php there is a bit of confusion as to what exactly the $id value does. Well, the $id value, even though it appears blank, is anything but. It holds a unique ID value, so that we can track our current session without worrying about compromising it with someone else's results. To see it's value, just put "echo $id;" somewhere after the <body> tag to view its result.



Next, is target.php. This file basically just tells our application WHERE to store the new Uploaded file.

Finally, getprogress.php. In this file is one more foreign entry that IBM does not give you: no-caching.

Code: Select all

 
header('Expires: Mon, 26 Jul 1997 05:00:00 GMT');
header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT");
header('Cache-Control: no-store, no-cache, must-revalidate');
header('Cache-Control: post-check=0, pre-check=0', FALSE);
header('Pragma: no-cache');
//....
 
We HAVE to put the no-cache statement in there for this to work in both Firefox and IE 6+. IE 7 (maybe IE 6, I haven't tested this in that browser yet) CACHES your value. So unless you put this no-cache statement in there, the result you are going to see is the progress bar jump 1 time only (usually around 4-7% on big files or 90% to 100% on smaller files). This is because once IE's Ajax call retrieves the value once, it freaking caches the value and just sits there. And this little annoyance <span style='color:blue' title='I'm naughty, are you naughty?'>smurf</span> ME OFF (sorry, but it did). I can't believe that not only I have to put hacks in place for IE in css, but now also because it caches VALUES when I don't want it to??????? Sigh........



========================================================================================
Conclusion

Run the progress bar example. It works. Works like a freaking dream!! I couldn't believe it but it has finally made me smile once again :)

Anyways, I hope you enjoyed this tutorial. Take it easy.

========================================================================================
Resources
What's new in PHP V5.2, Part 5: Tracking file upload progress
PHP Manaul for APC
CDownloadURL Javascript Function


Author: Jonathon Hibbard
Application Developer at HSR Business to Business
Last edited by infolock on Wed May 14, 2008 9:44 am, edited 1 time in total.
daleyjem
Forum Newbie
Posts: 4
Joined: Sun Mar 09, 2008 11:32 am

Re: PHP and Progressive Uploads : A small tutorial

Post by daleyjem »

this looks about 500x easier than the CGI one i used. i'm definitely gonna hang on to this one.
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: PHP and Progressive Uploads : A small tutorial

Post by Christopher »

Nice stuff Infolock. I plan to give your instructions a whirl to see how they work.
(#10850)
User avatar
infolock
DevNet Resident
Posts: 1708
Joined: Wed Sep 25, 2002 7:47 pm

Re: PHP and Progressive Uploads : A small tutorial

Post by infolock »

Thanks :) Hope it works for you.

FYI though, there is apparently a bug with the new APC release. When enabled, it blows all your $_POST data away to blank.

Bug is here: http://pecl.php.net/bugs/bug.php?id=10715&thanks=3

Not sure on an ETA, but the fix is to turn APC back off. Downgrading from the .18 to the .12 release fixes it, so we should be fixed in the .19 release but we'll see.
User avatar
Ambush Commander
DevNet Master
Posts: 3698
Joined: Mon Oct 25, 2004 9:29 pm
Location: New Jersey, US

Re: PHP and Progressive Uploads : A small tutorial

Post by Ambush Commander »

I should add that adding progressive upload support is not the primary function of APC; it's a bytecode cache, which means it speeds up PHP. Still definitely worth installing :-)
jmut
Forum Regular
Posts: 945
Joined: Tue Jul 05, 2005 3:54 am
Location: Sofia, Bulgaria
Contact:

Re: PHP and Progressive Uploads : A small tutorial

Post by jmut »

Nice article, good stuff.
Well I really like php solution, but on the other hand I totally don't like this post_max_size 200MB - for small stuff this is probably ok, but then again you don't need progress bar there. I need to upload stuff as big as DVDs and I think if I allow such MB for post max size etc it's a killer - don't tell me to use ftp :)
With perl cgi script I don't think there is such issue.
webaddict
Forum Commoner
Posts: 60
Joined: Wed Mar 14, 2007 6:55 am
Location: The Netherlands

Re: PHP and Progressive Uploads : A small tutorial

Post by webaddict »

Well, if you're into installing PECL components anyway, why not just use http://pecl.php.net/package/uploadprogress ?
User avatar
infolock
DevNet Resident
Posts: 1708
Joined: Wed Sep 25, 2002 7:47 pm

Re: PHP and Progressive Uploads : A small tutorial

Post by infolock »

jmut wrote:Nice article, good stuff.
Well I really like php solution, but on the other hand I totally don't like this post_max_size 200MB - for small stuff this is probably ok, but then again you don't need progress bar there. I need to upload stuff as big as DVDs and I think if I allow such MB for post max size etc it's a killer - don't tell me to use ftp :)
With perl cgi script I don't think there is such issue.

I only put 200MB for testing first of all. Secondly, there is no reason why you can't up the amount of post_max_size so long as you have enough memory on the server to handle it. But if you are going to be uploading DVD's, your server is going to suffer anyways unless you have the bandwidth to compensate for it.

Perl CGI scripts aren't going to be that much of a help either, as you are still going to have the same problem there.
User avatar
seodevhead
Forum Regular
Posts: 705
Joined: Sat Oct 08, 2005 8:18 pm
Location: Windermere, FL

Re: PHP and Progressive Uploads : A small tutorial

Post by seodevhead »

BRAVO infolock!!! BRAVO!!! Thank you so much this tutorial as this is something I have been looking for for a LONG LONG time. I can't wait to fool around with the code a bit.

Quick question... the instance of where I want to use a progress bar is on a form where people submit multiple photos... but the form is a "sticky form". In other words, when the user fills out the form and hit's submit button, it sends the form data to the same page and processes it on that same page.

At first glance I can't really tell if this is "retoolable" to work with a sticky form... so the progress bar simply loads on the same page upon submission.. but thought I'd ask to see if you knew off the top of your head or had any recommendations concerning that kinda setup.

Thanks so much again for your great work!
User avatar
lafever
Forum Commoner
Posts: 99
Joined: Sat Apr 05, 2008 2:03 pm
Location: Taylor, MI

Re: PHP and Progressive Uploads : A small tutorial

Post by lafever »

:bow:
User avatar
infolock
DevNet Resident
Posts: 1708
Joined: Wed Sep 25, 2002 7:47 pm

Re: PHP and Progressive Uploads : A small tutorial

Post by infolock »

seodevhead wrote:BRAVO infolock!!! BRAVO!!! Thank you so much this tutorial as this is something I have been looking for for a LONG LONG time. I can't wait to fool around with the code a bit.

Quick question... the instance of where I want to use a progress bar is on a form where people submit multiple photos... but the form is a "sticky form". In other words, when the user fills out the form and hit's submit button, it sends the form data to the same page and processes it on that same page.

At first glance I can't really tell if this is "retoolable" to work with a sticky form... so the progress bar simply loads on the same page upon submission.. but thought I'd ask to see if you knew off the top of your head or had any recommendations concerning that kinda setup.

Thanks so much again for your great work!
Thank you, glad you found it helpful. As for the sticky form, HTML by default does not allow file information to be sticky. The most common method of achieving this is not exactly putting the file information in the <input type="file" /> field, but by using a combination of a hidden input type field, and then a link to the file, or an <img src="#"> to preview/display the image (if that's what you uploaded).
qubesys
Forum Newbie
Posts: 2
Joined: Fri Dec 09, 2011 2:39 am

Re: PHP and Progressive Uploads : A small tutorial

Post by qubesys »

Nice tutorial

really helpful stuff

Hope to see more such posting from your side
brownjason
Forum Newbie
Posts: 2
Joined: Thu Jul 04, 2013 5:10 am

Re: PHP and Progressive Uploads : A small tutorial

Post by brownjason »

great tutorial,thanks for your kind sharing :D
ashi12
Forum Newbie
Posts: 1
Joined: Mon Jul 08, 2013 8:17 pm

Re: PHP and Progressive Uploads : A small tutorial

Post by ashi12 »

It's a very nice tutorial. It's gave really a very good info. And way of sharing info is very nice. Step by step told.
joo003464
Forum Newbie
Posts: 1
Joined: Thu Jun 27, 2013 10:29 am

Re: PHP and Progressive Uploads : A small tutorial

Post by joo003464 »

Its been nice to go through your post.
It has given me much knowledge & so many valuable information.

I'm feeling very nice to be here. so enjoyable... ;) ;) ;)
Post Reply