Ask Question Forum:
Model Library:2025-02-08:A.I. model is online auto reply
C
O
M
P
U
T
E
R
2
8
Show
#
ASK
RECENT
←
- Underline
- Bold
- Italic
- Indent
- Step
- Bullet
- Quote
- Cut
- Copy
- Paste
- Table
- Spelling
- Find & Replace
- Undo
- Redo
- Link
- Attach
- Clear
- Code
Below area will not be traslated by Google,you can input code or other languages
Hint:If find spelling error, You need to correct it,1 by 1 or ignore it (code area won't be checked).
X-position of the mouse cursor
Y-position of the mouse cursor
Y-position of the mouse cursor
Testcursor
caretPos
Attachment:===
Asked by duncanb7
at 2024-04-26 23:09:32
ModifyPoint:500 Replies:6 POST_ID:828487USER_ID:11059
Topic:
Hypertext Markup Language (HTML);PHP Scripting Language;Asynchronous Javascript and XML (AJAX)
I am using loadHTMLFile() to load remote site html page and do data extract by getelementsByName for
300 interation looping. The speed to grab the data for each time is really slow. So I try
to think whether I am using right function to do so.
There is alternative such as file_get_contents(url) and curl_setpos query. I want to know
Which is fastet method to grab 2 or 3 data from the remote site data for every iteration?
Why I have seen people sometimes using loadHTMlFile(), sometimes using curl_setpos, sometimes using
file_get_contents, what is different in term of program speed ?
If I switch back to file_get_contes($url), I need to do a loop to do a match to my data that will
cost execution time that is similar slow to getelementbtTagName by DOM, Is it Right ?
I also look curl_ method at http://www.phpfreaks.com/forums/index.php?topic=324412.0
but it doesn't tell me the advatnage of using curl_ . Could you provide this info ?
Please advise
Duncan
300 interation looping. The speed to grab the data for each time is really slow. So I try
to think whether I am using right function to do so.
There is alternative such as file_get_contents(url) and curl_setpos query. I want to know
Which is fastet method to grab 2 or 3 data from the remote site data for every iteration?
Why I have seen people sometimes using loadHTMlFile(), sometimes using curl_setpos, sometimes using
file_get_contents, what is different in term of program speed ?
If I switch back to file_get_contes($url), I need to do a loop to do a match to my data that will
cost execution time that is similar slow to getelementbtTagName by DOM, Is it Right ?
I also look curl_ method at http://www.phpfreaks.com/forums/index.php?topic=324412.0
but it doesn't tell me the advatnage of using curl_ . Could you provide this info ?
Please advise
Duncan
<?php$dom = new DOMDocument();/*$dom = file_get_contents($url2);*/@$dom->loadHTMLFile("http://www.remotesite.com/");$data3= $dom->getElementsByTagName('table')->item(4)->getElementsByTagName('div')->item(8)->getElementsByTagName('span')->item(0)->nodeValue;$data4= $dom->getElementsByTagName('table')->item(4)->getElementsByTagName('div')->item(12)->getElementsByTagName('span')->item(0)->nodeValue;?> 1:2:3:4:5:6:7:
Author: duncanb7 replied at 2024-04-27 22:23:27
Thanks for all of your reply
it is solved the issue by
this EE thread at
http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_26980898.html
it is solved the issue by
this EE thread at
http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_26980898.html
Assisted Solution
Expert: Medo3337 replied at 2024-04-27 12:35:01
167 points GOOD
Author: duncanb7 replied at 2024-04-27 10:47:42
Ray_Paseur, we discuss curl here and I ask this in other thread.
So thsi thread might be going to test which method is the best,
it seems from your code in other thread, curl is more flexible
and down into detail,
Give me sometime to digest all the attach link
So thsi thread might be going to test which method is the best,
it seems from your code in other thread, curl is more flexible
and down into detail,
Give me sometime to digest all the attach link
Expert: Medo3337 replied at 2024-04-27 07:57:42
If you want faster way, I recommend you to get faster internet for the server, this is the most effective way.
Assisted Solution
Expert: Ray Paseur replied at 2024-04-27 06:27:13
166 points GOOD
I cannot tell you which method of getting the remote content will be fastest, but I can give you a tool to measure the time of each different method. With a little instrumentation you can find the best method. However you probably want to test several times on different URLs - the remote server may cache the requests causing second and subsequent requests to look fast, when in practice they may not be very fast.
Try using this class and see what it tells you. HTH, ~Ray
Try using this class and see what it tells you. HTH, ~Ray
<?php // RAY_oop_stopwatch.phperror_reporting(E_ALL);// DEMONSTRATE A SCRIPT TIMER FOR ALL OR PART OF A SCRIPT PHP 5+// MAN PAGE http://php.net/manual/en/function.microtime.phpclass StopWatch{ protected $a, $z; public function __construct() { $this->a = array(); $this->z = array(); } // A METHOD TO CAPTURE A START TIME public function start($name='TIMER') { $this->a[$name] = microtime(TRUE); } // A METHOD TO CAPTURE AN END TIME public function stop($name='ALL') { if ($name == 'ALL') { foreach ($this->a as $name => $start_time) { if (!isset($this->z[$name])) $this->z[$name] = microtime(TRUE); } } else { $this->z[$name] = microtime(TRUE); } } // A METHOD TO READ OUT THE TIMER(S) public function readout($m=1000, $eol=PHP_EOL) { $str = NULL; foreach ($this->a as $name => $start_time) { $str .= $name; if (!isset($this->z[$name])) { $str .= " IS STILL RUNNING"; } else { $lapse_time = $this->z[$name] - $start_time; $lapse_msec = $lapse_time * $m; $lapse_echo = number_format($lapse_msec, 1); $str .= " $lapse_echo"; } $str .= $eol; } return $str; }}// DEMONSTRATE THE USE -- INSTANTIATE THE STOPWATCH OBJECT$sw = new Stopwatch;// SET STOPWATCH NAMES$go = 'GOOGLE ONLY';$gy = 'GOOGLE AND YAHOO!';$yo = 'YAHOO! ONLY';// START SOME TIMERS$sw->start($go);$sw->start($gy);// PERFORM SOME ACTIVITY THAT YOU WANT TO TIME$page = 'http://google.com';$html = file_get_contents($page);// STOP ONE OF THE STOPWATCHES AND START THE OTHER$sw->stop($go);$sw->start($yo);// PERFORM SOME OTHER ACTIVITY THAT YOU WANT TO TIME$page = 'http://yahoo.com';$html = file_get_contents($page);// REPORT THE STOPWATCHES CONTENT (TWO WILL BE INCOMPLETE)echo nl2br($sw->readout());// STOP ALL OF THE REMAINING STOPWATCHES$sw->stop();// REPORT THE STOPWATCHES CONTENT AGAINecho nl2br($sw->readout()); 1:2:3:4:5:6:7:8:9:10:11:12:13:14:15:16:17:18:19:20:21:22:23:24:25:26:27:28:29:30:31:32:33:34:35:36:37:38:39:40:41:42:43:44:45:46:47:48:49:50:51:52:53:54:55:56:57:58:59:60:61:62:63:64:65:66:67:68:69:70:71:72:73:74:75:76:77:78:79:80:81:82:83:84:85:86:87:88:89:90:91:92:93:94:95:96:
Accepted Solution
Expert: bportlock replied at 2024-04-27 02:01:17
167 points GOOD
Some general comments:
1. When grabbing data from remote websites you are limited by the speed at which the remote site responds. Increasing the speed at this end may make no difference if the remote site is a slow one.
2. cURL's main advantage over the other file methods is that cURL allows you to emulate a browser including cookies, sessions, https, ftp, gopher, etc. It is much, much more flexible than the other methods and whilst it looks more cumbersome, it is not that bad to use. Personally I never use anything except cURL to fetch HTML.
3. Use microtime to set up timing points in your code and see where the time is being used up. See Example 1 on this page http://uk2.php.net/microtime
"If I switch back to file_get_contes($url), I need to do a loop to do a match to my data that will
cost execution time that is similar slow to getelementbtTagName by DOM, Is it Right ?"
To an extent you are correct, but no matter how you fetch the page you will have to spend time processing and on most modern servers you can disregard the processing time because machines these days are fast and capable unless you use really, really cheap hosting.
DomDocument also supports loadHTML which loads a STRING rather than fetching a FILE so you could still load it into DOM http://uk2.php.net/manual/en/domdocument.loadhtml.php
In any case, insert timing points and see where the time is being spent. Then you'll have a better idea what the problem is.
1. When grabbing data from remote websites you are limited by the speed at which the remote site responds. Increasing the speed at this end may make no difference if the remote site is a slow one.
2. cURL's main advantage over the other file methods is that cURL allows you to emulate a browser including cookies, sessions, https, ftp, gopher, etc. It is much, much more flexible than the other methods and whilst it looks more cumbersome, it is not that bad to use. Personally I never use anything except cURL to fetch HTML.
3. Use microtime to set up timing points in your code and see where the time is being used up. See Example 1 on this page http://uk2.php.net/microtime
"If I switch back to file_get_contes($url), I need to do a loop to do a match to my data that will
cost execution time that is similar slow to getelementbtTagName by DOM, Is it Right ?"
To an extent you are correct, but no matter how you fetch the page you will have to spend time processing and on most modern servers you can disregard the processing time because machines these days are fast and capable unless you use really, really cheap hosting.
DomDocument also supports loadHTML which loads a STRING rather than fetching a FILE so you could still load it into DOM http://uk2.php.net/manual/en/domdocument.loadhtml.php
In any case, insert timing points and see where the time is being spent. Then you'll have a better idea what the problem is.
