Comments on: Creating Quick Scrapers Using Xpather, Snoopy & PHP http://flintston.es/nerd-stuff/creating-quick-scrapers-using-xpather-snoopy-php/ Affiliate Multivitamins Thu, 17 Nov 2011 08:03:11 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: Gary B http://flintston.es/nerd-stuff/creating-quick-scrapers-using-xpather-snoopy-php/comment-page-1/#comment-524 Gary B Fri, 30 Jul 2010 23:38:43 +0000 http://flintston.es/?p=275#comment-524 I've been using methods for this sort of thing for a long time. It's a great time saver. One thing I suggest: some websites out there have excruciatingly bad HTML that can make the DOM throw up its hands. So, to prevent that, I will use the tidy functions to repair the HTML before I run it to the DOM. I don't know how this will display, but here goes: $tidy_config = array( 'clean' => true, 'output-xhtml' => true, 'show-body-only' => true, 'wrap' => 0, ); $tidy = new tidy(); $file = $tidy->repairString($pagehtml, $tidy_config, 'latin1'); I then usually run a preg_replace on the result to strip out non-ASCII characters, depending on the purpose. I’ve been using methods for this sort of thing for a long time. It’s a great time saver. One thing I suggest: some websites out there have excruciatingly bad HTML that can make the DOM throw up its hands. So, to prevent that, I will use the tidy functions to repair the HTML before I run it to the DOM.

I don’t know how this will display, but here goes:

$tidy_config = array(
‘clean’ => true,
‘output-xhtml’ => true,
‘show-body-only’ => true,
‘wrap’ => 0,
);
$tidy = new tidy();
$file = $tidy->repairString($pagehtml, $tidy_config, ‘latin1′);

I then usually run a preg_replace on the result to strip out non-ASCII characters, depending on the purpose.

]]> By: Bamm http://flintston.es/nerd-stuff/creating-quick-scrapers-using-xpather-snoopy-php/comment-page-1/#comment-450 Bamm Wed, 26 May 2010 03:28:47 +0000 http://flintston.es/?p=275#comment-450 no problem klaas no problem klaas

]]> By: klaas http://flintston.es/nerd-stuff/creating-quick-scrapers-using-xpather-snoopy-php/comment-page-1/#comment-449 klaas Wed, 26 May 2010 03:24:41 +0000 http://flintston.es/?p=275#comment-449 Thanks for the tbody tip! Thanks for the tbody tip!

]]>