Try to attach the following link on Facebook: http://google.com/search?q=blenders%26pride. This URL actually queries Google for ‘blenders&pride’. Facebook converts/encodes the above URL to http://google.com/search?q=blenders&pride which is not the same as above and queries Google for just ‘blenders’.

So, why Facebook does this? Probably Facebook tries to encode the URL to remove the characters which are not allowed by RFC 3986 and replaces them with their percent encoding. But there are certain characters which should not be encoded, such as ‘/’, ‘?’, ‘#’, ‘@’ which are the reserved characters and used as delimiters in the URL. So, it decodes these characters and converts their encoding to the original character which gives rise to the problem. Let us see an example:

‘http://google.com/search?q= %2B‘ is first encoded to replace unwanted character with their percent encoding, turns into ‘http%3A%2F%2Fgoogle.com%2Fsearch%3Fq%3D%20%2B’. (Note: I assume that already encoded characters are not encoded again in order to reproduce the bug, i.e.%2B does not gets converted to %252B). Then, the reserved characters (/,:,?,=,@,+) must be decoded again, therefore it gets converted to ‘http://google.com/search?q=%20+‘ which as we see is not the same. It ideally should have been ‘http://google.com/search?q=%20%2B’.

I have already reported it in the bugs on Facebook. I will like to hear your views on URL encoding. Do you consider this as bug? Why does such URLs are not properly formatted at source making lives of developers difficult? Why does URLs with spaces and other disallowed characters exists?

Enjoy.

Related Resources:

• http://en.wikipedia.org/wiki/Percent-encoding

1. Automatically downloads and attaches image to the node created by Feeds module.
2. Configurable XPath of the desired image location on the webpage of the feed item on per feed basis (using element’s id or CSS class).
3. Support for FileField Paths module.
4. Configurable maximum image size.
5. Configurable minimum and maximum image resolutions.

In this post, I will briefly demonstrate how to configure the settings for FIG to efficiently grab images for feed items.

Image Field Settings:

Feeds module will create nodes from feed-items of content type as chosen in the FeedsNodeProcessor settings. FIG will store the image grabbed in an image field attached to the node of this content type. Therefore, you must add an image field to this content type. For example, if the content type of feed-items is story, you need to go to admin > content types > story > manage fields and add a field of type File and widget Image. (http://drupal.org/node/609628)

Configure the settings of the image field, such as extensions, minimum and maximum resolution, maximum size, etc. While grabbing the image, FIG will automatically take into consideration the settings of the image field to which the downloaded image will be mapped.

Mapping Settings:

Now, you must go to the mapping settings for the Node Processor of the Feed Importer and map Item URL (Link) to one of the available imagefields which are appended by “(FIG)“.

Mapping for Node Processor

Feeds Image Grabber Settings:

After enabling FIG and configuring the above settings, you will see the following additional form whenever you create or edit a feed.

FIG config form

Here is a brief explanation of the available fields:

1. Enable Feeds Image Grabber: As the description says, check this box if you want FIG to download images of the feed items created by this feed.
2. Search for an image between the tag identified by: It has been explained before here.
3. Feeling Lucky: FIG will select the first image between the tag if the “yes” is selected, else will search for all the images to find the largest image between the tag.
4. Execution time: FIG will take the selected percentage of maximum PHP execution time to grab image for each feed item. If images are not grabbed for some feed, try increasing this percentage to decrease errors due to network timeouts.

That is all. Once you click on import, you will get the feed item nodes along with the images. If images are not grabbed, here is a list of possible reasons,

1. Timeout, try increasing the Execution time.
2. Malformed HTML source.
3. Validation failures (due to maximum size, minimum resolution, etc).

I am working on better error logging and reporting, so that you can know the reasons for sure.

Enjoy.

Related Posts:

• Feeds Image Grabber:  http://publicmind.in/blog/drupal-feeds-image-grabber
• FIG project page (Download): http://drupal.org/project/feeds_imagegrabber

Screenshot of FBSL on LondonFuse.ca

FBSL is designed to be used quite intuitively, this screenshot shows the FBSL status form as themed by Thomas Cermak on LondonFuse.

FBSL status update form

FBSL can be used out of the box, though you can customize it from the settings page. All other information (installation, usage, and changelog) can be found inside the packaged module which is available for download here.

This module has been sponsored by Thomas Cermak of LondonFuse.

Resources:

• Project Page (Download): http://drupal.org/project/facebook_link

Note: Feature and Support requests should be made through the Facebook-style Links Forum. I may not be able to answer them here.

[Google Reader (with thumbnails) can be imitated on a Drupal site by using Feeds, Feeds Image Grabber, FileField, ImageField, ImageCache, ImageAPI and Views module].

How it works
A classic method of mimicking the behavior of Feeds Image Grabber would be to do the same thing manually. Let us go through the procedure if you were to do it manually:

1. Refresh the feed to create nodes for feed items (using FeedsNodeProcessor).
2. For each feed-item, go to their respective webpage and save the image to display.
3. Upload the image to CCK image field for each feed item.

Feeds Image Grabber automates the last 2 steps of the three step process described. Imagecache module can be used to create thumbnail of the image retrieved which can be attached to the node.

What’s in the box
On 3rd March, first version of FIG was released. As of today, FIG supports following features:

1. Automatically downloads and attaches image to the node created by Feeds module.
2. Configurable XPath of the desired image location on the webpage of the feed item on per feed basis (using element’s id or CSS class).
3. Support for FileField Paths module.
4. Configurable maximum image size.
5. Configurable minimum and maximum image resolutions.

Get involved
You can get involved by doing any of the following:

1. write about this module, so that other folks can find it.
2. report bugs in the issue queue.
3. help in the support forums of the module.
4. buy me a Beer

Resources:

• Project Page (Download): http://drupal.org/project/feeds_imagegrabber
• Tutorials: http://publicmind.in/blog/tutorial-for-feeds-image-grabber

Note: Feature and Support requests should be made through the Feeds Image Grabber Forum. I may not be able to answer them here.

Let us say you need to create a webpage for your website. Ideally, it would mean you create HTML content that will be delivered to someone who requests it through their web browser. Eventually you grow big and decide to add 100 more pages to your website and therefore create 100 more HTML pages. But then you realize that there is a lot of duplicate content. The footer which contains the copyright information is essentially the same on all pages, therefore if you ever need to edit it then it would mean to edit these 101 pages. This will be a lot of redundant work.

The main problem with the HTML pages is that they are static, they can not be changed based on what user requests. Thats why we need to have two different pages, even if they are almost similar and just differ by a single line. But then we also need to send the HTML content to the web browsers. That is where scripting languages such as PHP, ASP, JSP come as a savior. These languages are capable of creating dynamic content. The data is usually stored in the database and whenever a request comes for a page, HTML content will be created from this data on-the-fly and will be sent back to the requesting user. Now, you just need to store that copyright information just once, hence changing it becomes much easier.

Now, you also want to control which users have permission to access certain portion of the website and you create a login page for that part of the website, store cookies and write other code to maintain the session for the user. You might also want to publish a webpage on the fly, i.e.  save content to the database right from the user browser. You need some kind of access control mechanism and believe me it will be a hell lot of work to do.

That is where web-based Content Management System come into picture. They take out all the programming part from the publishing part of your website. You just need to write the content that has to be published on your website, make some settings on how it will be displayed, whether anyone or only authenticated users can see that content and you are done (of course, this is overly simplified but believe me once you know this everything will look so obvious to you).

With a CMS, one can easily create a website which :

1. allows large number of people to contribute and share data.
2. easily manages user access control lists.
3. have reduced duplicate content.

Drupal is one of the several available CMS is the market which is written in PHP and uses MySQL or PostgreSQL as database to store the website content. Drupal is released under GNU General Public License.

What can you achieve with Drupal?

You can easily create and manage,

• personal homepage
• company website
• blog
• shopping websites (just like Ebay or Amazon)
• Social networking sites like Twitter or Facebook

With Drupal, you can create any kind of website, Imagination is the limit. Just FYI, White House uses Drupal to manage its official website: whitehouse.gov

As with every other product, there are several other alternatives to Drupal such as Joomla, TYPO3, Nucleus, WordPress, etc. You can find the whole list of available CMS here: List of Content Management Systems. But what makes Drupal stand out is :

1. its ease of use
2. a large community of developers which keeps everything secure and up to date.
3. a large pool of modules and themes. A module in Drupal lets you do things which aren’t supported by default in Drupal.

Here is a small 1 minute video on “What is Drupal”, this video has already won six Telly awards:

Drupal Website Hosting

There are a lot of web hosting providers in the market and I know you will research well before buying anything. Most of them provide a one-click Drupal installation which saves you a lot of trouble while installing Drupal. I recommend you to look for such a web host, well all is up to you anyway.

You are now ready to eat up the Wikipedia article on Drupal. You can visit official Drupal website to look for great tutorials and downloads. Here are a few important links:

Official Drupal website: http://drupal.org
Modules Resource: http://drupal.org/project/modules
Themes Resource: http://drupal.org/project/themes
Wikipedia article: http://en.wikipedia.org/wiki/Drupal

Do not forget to vote for this post on DZone or Digg if you liked it, or you can leave a comment or two.

or multiple CSS classes associated with it.

Now if you want to select the HTML elements with class “foo”, this div element would be one of them.

As, I do not have much experience with XPath, I started to search for a solution. After a lot of search and discussion on Drupal development mailing lists, I arrived at the following XPath :

The normalize-space function replaces all continuous tabs and whitespaces with a single whitespace character. Then, we concatenate the resulting string with two white-spaces, one in the start and other at the last. At last, we search for

in the final string.

I hope it helps.

I get a lot of requests for this, so here is a tutorial explaining the 3 W’s (What, Why and How) of FeedAPI ImageGrabber.

Motivation

a feed item from google reader

This is a screenshot of a news item from my Google Reader. Anyone who has a moderate knowledge about the feeds would know that the image on the left in the above news-item is actually not published with the feed.

In other words, If you refresh any feed on your Drupal website using FeedAPI, you won’t get the image in the posts, unless they are published along with the feed which is quite rare.

The purpose of FeedAPI Imagegrabber is to make the feed more informative as well as interesting for the user. As, we all know that “comics are much better than novels”, this module appends the feed-item with an appropriate image from its content URL. The goal of the FeedAPI ImageGrabber is to mimic the thumbnail display of Google Reader for the feeds on your Drupal website.

As soon as you refresh the feed, ImageGrabber automatically attaches an appropriate image from the original article to the feed-item node on your website. You don’t even have to press an extra button!!

How it works
A classic method of mimicking the behavior of FeedAPI Imagegrabber will be to do the same thing manually. Let us go through the procedure if you were to do it manually:

1. Refresh the feed.
2. For each feed-item, go to their respective original URL and save the image to display.
3. For each node, crop the image, convert it into a thumbnail and then upload it in an CCK imagefield.

FeedAPI Imagegrabber automates the last 2 steps of the three step process described. You just need to refresh the feed and the ImageGrabber do the rest for you. The most difficult part which only humans can do is to select an appropriate image, for which I am constantly improving on the heuristics.

Download and Install
Download the latest release of FeedAPI ImageGrabber, compatible with your Drupal release, from the project page. Uncompress the folder in your site’s module directory and enable it from the admin/module page.

You are now ready to create new feeds with ImageGrabber enabled.

Next, you must install the following modules to get going:

• Imagefield
• Filefield
• CCK

You must then decide the content type for your feed-items for a particular feed and add CCK imagefields to that content type, before proceeding.

Form User Interface
You can enable the ImageGrabber for the feeds you want to download images for. When you add or edit any feed (i.e. feedapi enabled node), you will see the following additional settings for ImageGrabber:

Form Interface for ImageGrabber

Here is an detailed explanation of the input fields:

1. Enable ImageGrabber: Check this box if you want to enable ImageGrabber for this particular feed.
2. Select image field: ImageGrabber stores the downloaded image in a cck imagefield. You must select one of the imagefield from the drop-down menu.

If you receive this message “There are no imagefields associated with the ‘abcd’ content type.” , then you must add imagefield to the content type mentioned or change the content type of the feed-items to the one which has imagefields associated with it.

3. Search for an image between the tag identified by:
   To understand this, consider that the every feed-item associated with a particular feed has the following structure (HTML source code):
   

   HTML source code of a feed-item webpage

   For the above example, we know that the images associated with this particular post will always be within the

   1 2	<div class="post">...</div> tag

   Therefore, for this feed you must select the ‘class’ option here and enter ‘post’ in the text-field.
   Similarly, you must find out one such tag from the feed-items of the feed you want to create, see if it is identified by an id or an class. Then, select the appropriate option and enter the class or id in the text-field below. If you select None, the default tag will be ‘body’.

   simple way to find the id or class: Just find out the title of your article in the source code, and start looking for its parent/grandparent tags. Select the nearest parent which has an id or class.

   Now ImageGrabber will look for the images just between this tag, hence eliminating the download (and selection) of images in the sidebar or the header advertisement.

4. Enter the id or class (whatever you selected) in the text-field. Leave it empty if you selected the None option above.
5. Feeling Lucky: This has definitely been borrowed from the big devil Google. It will just select the first image it encounter between the tag specified, if ‘feeling lucky’ option is selected. Otherwise, it will select the largest image between the tags. I would personally recommend you to try out the ‘feeling lucky’ option as it helps saving a lot of bandwidth!! But if it doesn’t work for you, select the otherwise.

When you are done, just refresh the feed and you will see the images attached to the feed-item nodes.

Note: You can also modify your theme a little to get the Google Reader look in which the image floats on the left of the feed-item.

After refresh, you might get the following warning:

ImageGrabber: PHP execution time limit for system is 30 seconds, due to which images for some feed items couldn’t be downloaded. Please click on ‘Grab Images’ to refresh those feed-items.

If you get such a warning, no need to get afraid, just click on ‘Grab Images’ and the remaining feed-items will have their images as well. You may get multiple warnings depending upon your system’s execution time.

Enjoy grabbing images!!

Do leave a comment about this tutorial. Please make support and feature requests using support forums only.

Related Posts:

• FeedAPI ImageGrabber
• Feeds Image Grabber

Introduction

FeedAPI Imagegrabber is a add-on module for FeedAPI. It consists of a parser which visits the original URL of a new feed item, and retrieves the main image from the post. Once the main image has been retrieved, it is then converted into a thumbnail using the ImageCache module, and stored in the node created by FeedAPI, inside a CCK field.

The purpose of FeedAPI Imagegrabber is to make the feed more informative as well as interesting for the user. As, we all know that “comics are much better than novels”, this module appends the feed-item with an appropriate image from its content URL. The goal of the module is to mimic the thumbnail display of websites such as digg.com. This goal is acheived by using FeedAPI to turn RSS feed-items into nodes, and then using FeedAPI Imagegrabber to append these nodes with an appropriate image from the feed-item’s webpage.

How it works

A classic method of mimicking the behavior of FeedAPI Imagegrabber will be to do the same thing manually. Let us go through the procedure if you were to do it manually:

1. Refresh the feed.
2. For each feed-item, go to their respective original URL and save the image to display.
3. Crop the image, convert it into a thumbnail and then upload it in an CCK Image field.

FeedAPI Imagegrabber automates the last 2 steps of the three step process described. It uses cURL to download the images and then crop them using Imagecache. Then the image is stored into the CCK Imagefield. The most difficult part which only humans can do is to select the image,  for which I am constantly improving on the heuristics.

Download and Install

Visit the project page : FeedAPI Imagegrabber

Features and Future Releases

Visit the project page : FeedAPI Imagegrabber

History

I understand history is boring, but it is only for those who think the other way round. I started working on this module during December 2008.  At the time, when I started working on this module, I had a little or say no idea about Drupal or its API. It took me quite some time to get used to it and start working on the module. In the Initial release, I thought of very simple heuristics and decided to select the largest image available on the web-page. I completed the module in about 1 month and was very happy with my performance, and then a big blow came which delayed the release of FeedAPI Imagegrabber by 2 months. I had to include an external BSD licensed script for converting relative URL’s to absolute URL’s but Drupal denied to accept this. They said they allow only GPL licensed code and I started to convince them that both licenses are compatible. Unfortunately, I was unable to convince them and I had to create an external link to that BSD script on sourceforge. And now, the module is released with the hope to enhance the visitor-experience on several websites around the globe.

You may find these posts useful:

Tutorial for FeedAPI ImageGrabber
PHP: Relative URL to Absolute URL
Open Source Software and Licenses

Do Leave your comments to let me know what you think.

Note: Support queries and requests should be made through the FeedAPI Imagegrabber Forum. I may not be able to answer them here.