DOC776.org

My internet provider is not one of the best for one damn reason: It has bandwidth limit. So there I was wondering how am I going to transfer my 15gb website from old location to this shiny new one, knowing I have only 1GB on my monthly bandwidth left and it resets about 10 days from now. First attempt is to use one of those web ftp websites i found, well they all got their own bandwidth or they just freeze. I tried connecting to the old website using my own ftp script but it failed, and so did the remade scripts I tried using. The conclusion is that the old website does not allow direct ftp or special access, but my new one is liek a free world, do what you want sorta thing. Problem solved! I made mini script for teh old website that shows all files and folders in an orderly fashion in a big list. Then I made a simple file_get_content() and file_put_content() script on my new site. It was perfect, thought the rapid request rate shuts down the only connection between my two websites. Added a sleep(1) right after each download and flush() right after it is echo ‘file copied’ and it was done. Just sit back and relax as my new site copied all the files nice and fast.

For this script you would need to ahve PHP5. If you on PHP4 then you got to use fopen, fget, fput and fclose, classic style. You would also need to be able to modify your php ini variables with ini_set().

The other script that you will need to put at the source from where you copying is basically scandir() and echo all the files will full path with a separator like
or anything that is uncommon.

//depends on what si teh biggest fiel that would be copied
ini_set("memory_limit","256M");
 
//you can let it run for hours on end till its copied
ini_set("max_execution_time","60000");
 
//list of files to be copied from source website, get them here
$files = explode('<br>',file_get_contents('http:///source.website.com/list.php'));
 
/*on the listing i just printed each file with a new line tag at end, so the last line 
also got the tag, when exploded results in the last slot of the array to be empty*/
//remove that last empty value in the array
array_pop($files);
 
//total files to be copied
$total = count($maps);
 
//copied count
$cop = 0;
 
//you can stop this script and re run it again, so you get files that already exist on new location
//previously copied count
$ext = 0;
 
//start teh big loop
foreach($files as $k => $m) {
 
//display that we starting to copy this
 echo 'Copying '.($k+1).'/'.$total.' '.$m.' ... ';
 flush();
 
//if its not copied previously
	if(!is_file('path/to/new/place/here/'.$m)){
 
//get the file from source or show that we cant
	$f = @file_get_contents('http://source.website.com/'.$m) or er();
 
//if get works fire away
if($f) {
//put it on our server now
		if(file_put_contents('path/to/new/place/here/'.$m, $f)) {
 
//show that we done copying
			echo 'Done!<br>';
			$cop++;
			flush();
 
//wait haf a second or 1 second not to flame your source website
			usleep(500000);//500000 = 0.5 seconds / 1000000 = seconds
			}else echo 'Cannot put file.<br>';
		}else echo 'Cannot get file.<br>';
	}
	else
	{
		//show if already exists
echo 'Already exists.<br>';
		$ext++;
 
	}
 
}
//litle info
echo 'Copied '.$cop.' and '.$ext.' existed of '.$total.'. Failed: '.($total - $ext - $cop);

The flush() just shows current generated content to use.

Note:
I have noticed that if you pass URLs that have spaces in them, it will return as error also. I tried encoding teh URL and same thing happand. My solutioin: I used a curl version of file_get_content(). Here is the script, it will not return error on URLs with spaces:

function curl_get_file_contents($URL)
    {
        $c = curl_init();
        curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($c, CURLOPT_URL, $URL);
        $contents = curl_exec($c);
        curl_close($c);
        if ($contents) return $contents;
            else return FALSE;
    }

The mini script that generates the list of files to copy on the source end of this deal is easy if you only copying folder by folder.

$dir = @opendir("copy/this/");
while (($file = readdir($dir)) !== false)
{
	if($file != "." && $file != "..")
	{ echo $file."<br>"; }
}

There is no limit to how much you can automate the procedure. Remember that.

Output from when I was using it:

Copying 6868/8593 6a8f9cbcaa6e54ad6032250c359bd233.png ... Done!
Copying 6869/8593 2997daad502d72d34ddb1e68499ebe08.png ... Done!
Copying 6870/8593 382c4d6be6670d61d2cc3c074f1a15bb.png ... Done!
Copying 6871/8593 72790f18f902f42d35a648cc32f69571.png ... Done!
Copying 6872/8593 66f037fb5bfb7aed444617dfcd071e95.png ... Done!
Copying 6873/8593 96544d7d415904d2b5443738713f5ab1.png ... Done!

Yeah I was copying directly one folder, my resources (don’t ask why there is 8000 of them, that’s just half of it). It finished it in around 2 hours. I just left the page on and minimized it as i was doing some other work. If you accidentally stop it, just refresh it. It will skip thought all the existing files in seconds and continue copying the rest. Saves you time relay, unlike with big back up and upload scripts here the only error that can happen if you are requesting too fast and it will skip that file, to fix that just refresh the page to get all those skipped ones, fast and easy.