Drupal: Tutorial for Feeds Image Grabber

Feeds Image Grabber (FIG) was released on 3rd March, and currently supports the following features:

  1. Automatically downloads and attaches image to the node created by Feeds module.
  2. Configurable XPath of the desired image location on the webpage of the feed item on per feed basis (using element’s id or CSS class).
  3. Support for FileField Paths module.
  4. Configurable maximum image size.
  5. Configurable minimum and maximum image resolutions.

In this post, I will briefly demonstrate how to configure the settings for FIG to efficiently grab images for feed items.

Image Field Settings:

Feeds module will create nodes from feed-items of content type as chosen in the FeedsNodeProcessor settings. FIG will store the image grabbed in an image field attached to the node of this content type. Therefore, you must add an image field to this content type. For example, if the content type of feed-items is story, you need to go to admin > content types > story > manage fields and add a field of type File and widget Image. (http://drupal.org/node/609628)

Configure the settings of the image field, such as extensions, minimum and maximum resolution, maximum size, etc. While grabbing the image, FIG will automatically take into consideration the settings of the image field to which the downloaded image will be mapped.

Mapping Settings:

Now, you must go to the mapping settings for the Node Processor of the Feed Importer and map Item URL (Link) to one of the available imagefields which are appended by “(FIG)“.

Mapping Settings

Mapping for Node Processor

Feeds Image Grabber Settings:

After enabling FIG and configuring the above settings, you will see the following additional form whenever you create or edit a feed.

fig-form

FIG config form

Here is a brief explanation of the available fields:

  1. Enable Feeds Image Grabber: As the description says, check this box if you want FIG to download images of the feed items created by this feed.
  2. Search for an image between the tag identified by: It has been explained before here.
  3. Feeling Lucky: FIG will select the first image between the tag if the “yes” is selected, else will search for all the images to find the largest image between the tag.
  4. Execution time: FIG will take the selected percentage of maximum PHP execution time to grab image for each feed item. If images are not grabbed for some feed, try increasing this percentage to decrease errors due to network timeouts.

That is all. Once you click on import, you will get the feed item nodes along with the images. If images are not grabbed, here is a list of possible reasons,

  1. Timeout, try increasing the Execution time.
  2. Malformed HTML source.
  3. Validation failures (due to maximum size, minimum resolution, etc).

I am working on better error logging and reporting, so that you can know the reasons for sure.

Enjoy.

33 thoughts on “Drupal: Tutorial for Feeds Image Grabber

  1. Pingback: Drupal: Feeds Image Grabber | Public Mind

  2. Pingback: Tutorial for FeedAPI Image Grabber | Public Mind

  3. day trader

    good job, but I have a question is that the feeds didn’t released 7.X,but your plugin(feed image grabber)have been released 7.X

    Reply
    1. Nitin Post author

      7.x of Feeds Image Grabber (FIG) just contains the common functions for parsing web pages, scraping images, saving files but it hasn’t been integrated with Feeds (as no 7.x exists for Feeds yet). 🙂

      Reply
  4. Josh

    Agreed pretty awesome, but I’m looking to snag images right from the body of posts. Most feeds I’ve encountered we’re so nice about placing an image item right in them, though this module would be great for flickr feeds and even getting avatar images into a field for good display.

    Here is some code I used to catch the first image in a post body.
    Altered from this post intended for WordPress http://www.wprecipes.com/how-to-get-the-first-image-from-the-post-and-display-it
    I’m using it for Drupal, and unfortunately I can’t post the real PREG here.

    function retrofit_custom_catch_image($string) {
    $first_img = ”;
    $output= preg_match_all(‘ GO GET THE REAL PREG FROM THE POST ‘, $string, $matches);
    $first_img = $matches[1][0];
    if(empty($first_img)){ //Defines a default image
    $first_img = “/images/default.jpg”;
    }
    return $first_img;
    }

    Hope I just made your day!

    Reply
    1. Nitin Post author

      I agree that most of the feeds come with images inserted inside the body of the post or within an additional tag in the XML feed. Achieving the former, as you demonstrated, is relatively easy but mapping it to filefield/imagefield would require some extra efforts. I remember FeedAPI Scrapper used to do this with FeedAPI which, unfortunately, hasn’t been ported for Feeds yet. It scraped the body of the feed-item for any content recognized by a PREG or REGEX and mapped it to user-defined fields. Achieving the latter, according to me, would require a custom parser which can look for those extra tags in the XML feed.

      Also, see http://drupal.org/node/783694#comment-2900110

      Cheers,

      Reply
  5. Spark_man

    I installed your module and have followed the directions up to the bit about the mapping but “photo (FIG)” isn’t showing up in the target list? Any ideas?

    Reply
  6. Aaron

    Okay, I might be a little confused about what you can do with this module.

    I’m parsing an XML file with the Feeds module and I would like to grab an image.
    From something like this…

    Will this not work? Right now its not working, although I have used this feature successfully for non-xml feeds in the same site.

    Reply
    1. Aaron

      Oh, hahah – I guess it cut out the example XML I put right after “something like this..”, yeah I’ll head over to the issue que.

      Reply
  7. Carlos Tremblay

    Hi,
    I am using the Feeds module to import content from a CSV file (not from a Feed) using the “File upload” fetcher. Can FIG be used in that context? I already mapped an image (FIG) field in the Node processor mappings but I don’t see the FIG config form as shown above in your tutorial.
    Thanks!

    Reply
    1. Nitin Post author

      Of course, FIG can be used whatever fetcher you are using as long as the feed items are URL’s to some original article from where it can fetch images. If you are not seeing the FIG config form, please open an issue in the support queue with more details.

      Thanks,

      Reply
  8. Garrett Dawson

    I am trying to implement this module for the Feeds module, in order to grab multiple images via xml. This looks like the solution to my problem.

    Unfortunately, after mapping everything, etc… I don’t see this form when creating a field.
    “Feeds Image Grabber Settings:”

    Any clues would be most excellent, ’cause this looks like the hotfire.

    Reply
  9. Clement

    Hi I posted a question at drupal but there was no reply so I am hoping I can get some help here. Whenever I delete existing nodes after the first import and I do the import again .. I will get a warning like this

    Warning message

    SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘public://feeds_images/kim-kardashian-with-bangs_244x362.jpg’ for key ‘uri’
    SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘public://feeds_images/kayte-walsh-and-kelsey-grammer-photograph_’ for key ‘uri’
    SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘public://feeds_images/kate-major-mugshot_301x379.png’ for key ‘uri’
    SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘public://feeds_images/britney-spears-management-team_517x283.jpg’ for key ‘uri’
    SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘public://feeds_images/colbertromney_478x264.jpg’ for key ‘uri’
    SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘public://feeds_images/lana-del-ray-video-games-saturday-night-li’ for key ‘uri’
    SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘public://feeds_images/laura-kaeppeler-vote-for-me_450x254.jpg’ for key ‘uri’
    SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘public://feeds_images/joe-paterno-pic_474x332.jpg’ for key ‘uri’
    SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘public://feeds_images/newt-gingrich-ad-mitt-romney-will-say-anth’ for key ‘uri’
    SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘public://feeds_images/mr-worldwide_344x471.jpg’ for key ‘uri’

    I have deleted all the existing images before I import again but I still get this error everytime.

    Reply
  10. Giovanni R.

    > Now, you must go to the mapping settings for the Node Processor of the Feed Importer

    If anyone is wondering where to find the Mapper, it’s at admin/structure/feeds/feed/mapping.

    One needs to go to Structure -> Feeds Importers and click on “Override” of the “Feed” importer.

    Override will become Edit after override.

    Please consider adding this instructions in the original article.

    Greetings.
    Giovanni

    Reply
  11. Siva

    1.Installed feeds+Feeds XPath Parser along with feeds image grabber.
    2.Already having a content type with the following fields Title, Category, Option, Image, Type(a drop down).
    3.Since i am parsing XML i am using the module Feeds Xpath Parser and in “Select a parser” XPath XML parser – Parse XML using XPath.”
    4.Having a “Plain text” Text format and the content type “my_content_type”.
    (Now i went to my_content_type i couldn’t find anything for FIG settings)
    5.Comes to mapping “Select a source” “Select a target” – pathparser:0 Title
    xpathparser:1 Category
    xpathparser:2 Option
    xpathparser:3 Image (FIG)
    …………………………………..
    …………………………………..
    6. Context – //item, Title – title , field_category -category, field_option – option, field_image – image
    7.Here goes my XML

    ABC Electronics
    Home first column – left
    yes
    http://xyz/sites/default/files/field/image/pa.jpg

    8.It says 1 Node created.
    9.When i saw the node i can see that the picture is not present there.
    10.And still now i didn’t get any settings for FIG in the content type associated with this particular feed.

    Some one please help me where i am doing the mistake.

    Reply
  12. Maurice

    The Feeds Image Grabber Settings are located at admin/config/feeds_imagegrabber. This held me up for a few hours because I didn’t see a config options on the module page.

    Reply
  13. Reuben

    Would be nice if there was an accompanying input filter which could swap out the url’s for the grabbed external images with the resulting local urls.

    Reply
    1. Nitin Post author

      I haven’t been actively working on the module lately, these settings may have changed since this was posted. Please reach out on the drupal forum for further help.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *