Ask Question Forum:
Model Library:2025-02-08:A.I. model is online auto reply
C
O
M
P
U
T
E
R
2
8
Show
#
ASK
RECENT
←
- Underline
- Bold
- Italic
- Indent
- Step
- Bullet
- Quote
- Cut
- Copy
- Paste
- Table
- Spelling
- Find & Replace
- Undo
- Redo
- Link
- Attach
- Clear
- Code
Below area will not be traslated by Google,you can input code or other languages
Hint:If find spelling error, You need to correct it,1 by 1 or ignore it (code area won't be checked).
X-position of the mouse cursor
Y-position of the mouse cursor
Y-position of the mouse cursor
Testcursor
caretPos
Attachment:===
Asked by duncanb7
at 2024-05-12 11:15:06
ModifyPoint:500 Replies:4 POST_ID:828503USER_ID:11059
Topic:
Hypertext Markup Language (HTML);Linux;PHP Scripting Language
I am using curl in php to grab one remote webpage site that is more than 36Meg byte
And I get the fatal error message which is "Out of memory(allocated 3407872( tried to
allocate 3147593 byte) in .../data/include/curl.php on line42" line42 is my $htm=curl_exec($url);
I have checked my php system 's max memory limit is 128M byte by php's phpinfo(). Why I will have such error ?
I go to google it about this at http://drupal.org/node/29268
and I changed my .htaccess file with php_value memory_limit 120M, I talked to my hosting domain support, they just replied it is right there is memory limit=128M byte.
So anyone have suggestion to solve the fatal error issue ?
I also use this in php at the top , ini_set("memory_limit", "120M");, but it still fail
I also use phpinfo() and host support confirm it is 128Meg in my php system.
Any variable use related to curl_exec() is just 2 so it should be less than 120Meg usage in my php ($htm, $file_url)
And attached my .htaccess file and my php sytem configuration.
Is it one variable such as $file_url can not store more than 32Meg byte ?
Please advise & Thanks
Duncan
my php script
=====================
<?php
$file_url=my_curl("wwww.example.com","" , 3000, TRUE);
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
// PREPARE THE ARGUMENT STRING IF NEEDED
// $get_string = '';
// foreach ($get_array as $key => $val)
// {
// $get_string
// = $get_string
// . urlencode($key)
// . '='
// . urlencode($val)
// . '&';
// }
// $get_string = rtrim($get_string, '&');
if (!empty($get_string)) $url .= '?' . $get_string;
$curl = curl_init();
// HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
$header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: "; // BROWSERS USUALLY LEAVE BLANK
// SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
curl_setopt( $curl, CURLOPT_URL, $url );
curl_setopt( $curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6' );
curl_setopt( $curl, CURLOPT_HTTPHEADER, $header );
curl_setopt( $curl, CURLOPT_REFERER, 'http://www.google.com' );
curl_setopt( $curl, CURLOPT_ENCODING, 'gzip,deflate' );
curl_setopt( $curl, CURLOPT_AUTOREFERER, TRUE );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE );
// curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE );
curl_setopt( $curl, CURLOPT_TIMEOUT, $timeout );
// RUN THE CURL REQUEST AND GET THE RESULTS
$htm = curl_exec($curl);
// ON FAILURE HANDLE ERROR MESSAGE
//global $track; // declaring global variable
if ($htm === FALSE)
{
if ($error_report)
{
$err = curl_errno($curl);
$inf = curl_getinfo($curl);
// echo "FAILC: $url TIMEOUT=$timeout, CURL_ERRNOC=$err";
var_dump($inf);
}
curl_close($curl);
return FALSE;
}
// ON SUCCESS RETURN XML / HTML STRING
curl_close($curl);
return $htm;
}
?>
And I get the fatal error message which is "Out of memory(allocated 3407872( tried to
allocate 3147593 byte) in .../data/include/curl.php on line42" line42 is my $htm=curl_exec($url);
I have checked my php system 's max memory limit is 128M byte by php's phpinfo(). Why I will have such error ?
I go to google it about this at http://drupal.org/node/29268
and I changed my .htaccess file with php_value memory_limit 120M, I talked to my hosting domain support, they just replied it is right there is memory limit=128M byte.
So anyone have suggestion to solve the fatal error issue ?
I also use this in php at the top , ini_set("memory_limit", "120M");, but it still fail
I also use phpinfo() and host support confirm it is 128Meg in my php system.
Any variable use related to curl_exec() is just 2 so it should be less than 120Meg usage in my php ($htm, $file_url)
And attached my .htaccess file and my php sytem configuration.
Is it one variable such as $file_url can not store more than 32Meg byte ?
Please advise & Thanks
Duncan
my php script
=====================
<?php
$file_url=my_curl("wwww.example.com","" , 3000, TRUE);
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
// PREPARE THE ARGUMENT STRING IF NEEDED
// $get_string = '';
// foreach ($get_array as $key => $val)
// {
// $get_string
// = $get_string
// . urlencode($key)
// . '='
// . urlencode($val)
// . '&';
// }
// $get_string = rtrim($get_string, '&');
if (!empty($get_string)) $url .= '?' . $get_string;
$curl = curl_init();
// HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
$header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: "; // BROWSERS USUALLY LEAVE BLANK
// SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
curl_setopt( $curl, CURLOPT_URL, $url );
curl_setopt( $curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6' );
curl_setopt( $curl, CURLOPT_HTTPHEADER, $header );
curl_setopt( $curl, CURLOPT_REFERER, 'http://www.google.com' );
curl_setopt( $curl, CURLOPT_ENCODING, 'gzip,deflate' );
curl_setopt( $curl, CURLOPT_AUTOREFERER, TRUE );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE );
// curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE );
curl_setopt( $curl, CURLOPT_TIMEOUT, $timeout );
// RUN THE CURL REQUEST AND GET THE RESULTS
$htm = curl_exec($curl);
// ON FAILURE HANDLE ERROR MESSAGE
//global $track; // declaring global variable
if ($htm === FALSE)
{
if ($error_report)
{
$err = curl_errno($curl);
$inf = curl_getinfo($curl);
// echo "FAILC: $url TIMEOUT=$timeout, CURL_ERRNOC=$err";
var_dump($inf);
}
curl_close($curl);
return FALSE;
}
// ON SUCCESS RETURN XML / HTML STRING
curl_close($curl);
return $htm;
}
?>
my .htaccess file====================# -FrontPage-<Limit GET POST>order deny,allowdeny from allallow from all</Limit><Limit PUT DELETE>order deny,allowdeny from all</Limit>php_value memory_limit 120MAuthUserFile /home/aab/public_html/_vti_pvt/service.pwdAuthGroupFile /home/aab/public_html/_vti_pvt/service.grpAuthName aaa.comIndexIgnore .htaccess */.??* *~ *# */HEADER* */README* */_vti* Accept and Award Points Accept as Solution 1:2:3:4:5:6:7:8:9:10:11:12:13:14:15:16:17:18:19:
my Php Configuration===================PHP CoreDirective =>" Local Value =>" Master Valueallow_call_time_pass_reference =>" On =>" Onallow_url_fopen =>" On =>" Onallow_url_include =>" Off =>" Offalways_populate_raw_post_data =>" Off =>" Offarg_separator.input =>" &" =>" &"arg_separator.output =>" &" =>" &"asp_tags =>" Off =>" Offauto_append_file =>" no value =>" no valueauto_globals_jit =>" On =>" Onauto_prepend_file =>" /usr/lib/php/head.php =>" /usr/lib/php/head.phpbrowscap =>" no value =>" no valuedefault_charset =>" no value =>" no valuedefault_mimetype =>" text/html =>" text/htmldefine_syslog_variables =>" Off =>" Offdetect_unicode =>" On =>" Ondisable_classes =>" no value =>" no valuedisable_functions =>" symlink,shell_exec,exec,proc_close,proc_open,popen,system,dl,passthru,escapeshellarg,escapeshellcmd,show_source =>" symlink,shell_exec,exec,proc_close,proc_open,popen,system,dl,passthru,escapeshellarg,escapeshellcmd,show_sourcedisplay_errors =>" STDOUT =>" STDOUTdisplay_startup_errors =>" Off =>" Offdoc_root =>" no value =>" no valuedocref_ext =>" no value =>" no valuedocref_root =>" no value =>" no valueenable_dl =>" Off =>" Offerror_append_string =>" no value =>" no valueerror_log =>" error_log =>" error_logerror_prepend_string =>" no value =>" no valueerror_reporting =>" 6135 =>" 6135expose_php =>" On =>" Onextension_dir =>" /usr/local/lib/php/extensions/no-debug-non-zts-20060613 =>" /usr/local/lib/php/extensions/no-debug-non-zts-20060613file_uploads =>" On =>" Onhighlight.bg =>" <"font style="color: #FFFFFF">"#FFFFFF<"/font>" =>" <"font style="color: #FFFFFF">"#FFFFFF<"/font>"highlight.comment =>" <"font style="color: #FF8000">"#FF8000<"/font>" =>" <"font style="color: #FF8000">"#FF8000<"/font>"highlight.default =>" <"font style="color: #0000BB">"#0000BB<"/font>" =>" <"font style="color: #0000BB">"#0000BB<"/font>"highlight.html =>" <"font style="color: #000000">"#000000<"/font>" =>" <"font style="color: #000000">"#000000<"/font>"highlight.keyword =>" <"font style="color: #007700">"#007700<"/font>" =>" <"font style="color: #007700">"#007700<"/font>"highlight.string =>" <"font style="color: #DD0000">"#DD0000<"/font>" =>" <"font style="color: #DD0000">"#DD0000<"/font>"html_errors =>" Off =>" Offignore_repeated_errors =>" On =>" Onignore_repeated_source =>" On =>" Onignore_user_abort =>" Off =>" Offimplicit_flush =>" On =>" Oninclude_path =>" .:/usr/lib/php:/usr/local/lib/php =>" .:/usr/lib/php:/usr/local/lib/phplog_errors =>" On =>" Onlog_errors_max_len =>" 1024 =>" 1024magic_quotes_gpc =>" On =>" Onmagic_quotes_runtime =>" Off =>" Offmagic_quotes_sybase =>" Off =>" Offmail.force_extra_parameters =>" no value =>" no valuemax_execution_time =>" 0 =>" 0max_file_uploads =>" 20 =>" 20max_input_nesting_level =>" 64 =>" 64max_input_time =>" -1 =>" -1memory_limit =>" 128M =>" 128Mopen_basedir =>" no value =>" no valueoutput_buffering =>" 0 =>" 0output_handler =>" no value =>" no valuepost_max_size =>" 128M =>" 128Mprecision =>" 12 =>" 12realpath_cache_size =>" 16K =>" 16Krealpath_cache_ttl =>" 120 =>" 120register_argc_argv =>" On =>" Onregister_globals =>" Off =>" Offregister_long_arrays =>" On =>" Onreport_memleaks =>" On =>" Onreport_zend_debug =>" Off =>" Offsafe_mode =>" Off =>" Offsafe_mode_exec_dir =>" /usr/local/php/bin =>" /usr/local/php/binsafe_mode_gid =>" Off =>" Offsafe_mode_include_dir =>" no value =>" no valuesendmail_from =>" no value =>" no valuesendmail_path =>" /usr/sbin/sendmail -t -i =>" /usr/sbin/sendmail -t -iserialize_precision =>" 100 =>" 100short_open_tag =>" On =>" OnSMTP =>" localhost =>" localhostsmtp_port =>" 25 =>" 25sql.safe_mode =>" Off =>" Offtrack_errors =>" Off =>" Offunserialize_callback_func =>" no value =>" no valueupload_max_filesize =>" 128M =>" 128Mupload_tmp_dir =>" /tmp =>" /tmpuser_dir =>" no value =>" no valuevariables_order =>" EGPCS =>" EGPCSxmlrpc_error_number =>" 0 =>" 0xmlrpc_errors =>" Off =>" Offy2k_compliance =>" On =>" Onzend.ze1_compatibility_mode =>" Off =>" Off Accept and Award Points Accept as Solution 1:2:3:4:5:6:7:8:9:10:11:12:13:14:15:16:17:18:19:20:21:22:23:24:25:26:27:28:29:30:31:32:33:34:35:36:37:38:39:40:41:42:43:44:45:46:47:48:49:50:51:52:53:54:55:56:57:58:59:60:61:62:63:64:65:66:67:68:69:70:71:72:73:74:75:76:77:78:79:80:81:82:83:84:85:86:87:88:89:90:91:92:
Author: duncanb7 replied at 2024-05-13 15:42:54
Finally, I give up to
use memory instead of file
and it works fine in term speed
just a little bit slow
use memory instead of file
and it works fine in term speed
just a little bit slow
Assisted Solution
Expert: Amick replied at 2024-05-13 03:21:37
250 points GOOD
I can't advise you on how Fopen and File manage memory. I would try it and see the result.
If you want to pursue higher memory limits, you can locally set the memory limits in your script unless you've disallowed that on your system.
PHP_INI_ALL should be enabled to locally control memory_limit.
If you want to pursue higher memory limits, you can locally set the memory limits in your script unless you've disallowed that on your system.
PHP_INI_ALL should be enabled to locally control memory_limit.
ini_set('memory_limit', '256M');ini_set('post_max_size', '256M');ini_set('upload_max_filesize', '256M'); 1:2:3:
Setting memory limits to -1 in your script effectively removes them, and would be worth trying if transfer still fails at 256M.
you might use
at exit to see how much memory you actually used.
Author: duncanb7 replied at 2024-05-12 22:40:24
I thought that direction before and suppose what you said is correct
but finally I still need to open the saved file for data operation in memory so that
sitll have out of memory issue once I open it by fopen or file() in php.
Or Fopen and File won't open and put whole file data into memory ?
Please advise
Duncan
but finally I still need to open the saved file for data operation in memory so that
sitll have out of memory issue once I open it by fopen or file() in php.
Or Fopen and File won't open and put whole file data into memory ?
Please advise
Duncan
Accepted Solution
Expert: Amick replied at 2024-05-12 15:22:45
250 points GOOD
Set CURLOPT_RETURNTRANSFER False, and open a file-handle instead, passing it to CURLOPT_FILE. This lets curl_exec() saves the content to the file instead of holding it in memory.
http://us2.php.net/manual/en/function.curl-setopt.php
http://us2.php.net/manual/en/function.curl-setopt.php
