Public Mindphp

PHP: Encoding a URL before accessing it

Nitin — Sat, 08 May 2010 16:56:34 +0000

I discussed a little about URL encoding in my recent post Facebook: Bug with URL encoding, although it seems like the bug still exists. In this post I will discuss “how to encode a given URL before accessing it using CURL or fsockopen”. The problem with URLs is that they might contain certain disallowed characters like spaces, according to RFC 3986. Our aim is to convert these invalid characters to their percentage encoded values in a given URL , so that we can access the URL using our regular HTTP request methods. For example the URL [http://example.com/space space] should be converted to [http://example.com/space%20space] before we can access it using CURL. However, the URL [http://example.com/percents%25percent] is perfectly valid as it doesn’t contains any of the disallowed characters.

The given URL can only contain characters which are allowed by RFC 3986 which includes alpha-numeric characters along with reserved characters and delimiters (: | / | ? | # | [ | ] | @ | ! | $ | & | ‘ | ( | ) | * | + | , | ; | = | %). The following piece of code changes the disallowed characters in the URL to their corresponding percentage encoded values. The characters and delimiters which are allowed are left untouched.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

/**
* @param $url
* The URL to encode
*
* @return
* A string containing the encoded URL with disallowed
* characters converted to their percentage encodings.
*/
function encode_url($url) {
$reserved = array(
":" => '!%3A!ui',
"/" => '!%2F!ui',
"?" => '!%3F!ui',
"#" => '!%23!ui',
"[" => '!%5B!ui',
"]" => '!%5D!ui',
"@" => '!%40!ui',
"!" => '!%21!ui',
"$" => '!%24!ui',
"&" => '!%26!ui',
"'" => '!%27!ui',
"(" => '!%28!ui',
")" => '!%29!ui',
"*" => '!%2A!ui',
"+" => '!%2B!ui',
"," => '!%2C!ui',
";" => '!%3B!ui',
"=" => '!%3D!ui',
"%" => '!%25!ui',
);

$url = rawurlencode($url);
$url = preg_replace(array_values($reserved), array_keys($reserved), $url);
return $url;
}

One thing to notice here is that the ‘%’ character must be the last in the $reserved array. This makes sure that the already encoded values are not lost or encoded again.

Note: This might not be the best solution but well, it works. If you are using something better, let me know in the comments.

“What is Drupal?”, for newbies

Nitin — Fri, 19 Feb 2010 22:55:16 +0000

I often come across this question from my friends and peers and most of the time I deflect it by saying “it lets you create your website easily, why don’t you look up on Wikipedia?”. Few days ago, one of my friends pointed me that most of the articles start with “Drupal is a Content Management System(CMS) …” which becomes pretty much confusing when one has no idea about what a CMS is. In this post, I have tried to explain what a CMS is and why we need it and how Drupal stands out among various other available CMSs.

Let us say you need to create a webpage for your website. Ideally, it would mean you create HTML content that will be delivered to someone who requests it through their web browser. Eventually you grow big and decide to add 100 more pages to your website and therefore create 100 more HTML pages. But then you realize that there is a lot of duplicate content. The footer which contains the copyright information is essentially the same on all pages, therefore if you ever need to edit it then it would mean to edit these 101 pages. This will be a lot of redundant work.

The main problem with the HTML pages is that they are static, they can not be changed based on what user requests. Thats why we need to have two different pages, even if they are almost similar and just differ by a single line. But then we also need to send the HTML content to the web browsers. That is where scripting languages such as PHP, ASP, JSP come as a savior. These languages are capable of creating dynamic content. The data is usually stored in the database and whenever a request comes for a page, HTML content will be created from this data on-the-fly and will be sent back to the requesting user. Now, you just need to store that copyright information just once, hence changing it becomes much easier.

Now, you also want to control which users have permission to access certain portion of the website and you create a login page for that part of the website, store cookies and write other code to maintain the session for the user. You might also want to publish a webpage on the fly, i.e. save content to the database right from the user browser. You need some kind of access control mechanism and believe me it will be a hell lot of work to do.

That is where web-based Content Management System come into picture. They take out all the programming part from the publishing part of your website. You just need to write the content that has to be published on your website, make some settings on how it will be displayed, whether anyone or only authenticated users can see that content and you are done (of course, this is overly simplified but believe me once you know this everything will look so obvious to you).

With a CMS, one can easily create a website which :

allows large number of people to contribute and share data.
easily manages user access control lists.
have reduced duplicate content.

Drupal is one of the several available CMS is the market which is written in PHP and uses MySQL or PostgreSQL as database to store the website content. Drupal is released under GNU General Public License.

What can you achieve with Drupal?

You can easily create and manage,

personal homepage
company website
blog
shopping websites (just like Ebay or Amazon)
Social networking sites like Twitter or Facebook

With Drupal, you can create any kind of website, Imagination is the limit. Just FYI, White House uses Drupal to manage its official website: whitehouse.gov

As with every other product, there are several other alternatives to Drupal such as Joomla, TYPO3, Nucleus, WordPress, etc. You can find the whole list of available CMS here: List of Content Management Systems. But what makes Drupal stand out is :

its ease of use
a large community of developers which keeps everything secure and up to date.
a large pool of modules and themes. A module in Drupal lets you do things which aren’t supported by default in Drupal.

Here is a small 1 minute video on “What is Drupal”, this video has already won six Telly awards:

Drupal Website Hosting

There are a lot of web hosting providers in the market and I know you will research well before buying anything. Most of them provide a one-click Drupal installation which saves you a lot of trouble while installing Drupal. I recommend you to look for such a web host, well all is up to you anyway.

You are now ready to eat up the Wikipedia article on Drupal. You can visit official Drupal website to look for great tutorials and downloads. Here are a few important links:

Official Drupal website: http://drupal.org
Modules Resource: http://drupal.org/project/modules
Themes Resource: http://drupal.org/project/themes
Wikipedia article: http://en.wikipedia.org/wiki/Drupal

Do not forget to vote for this post on DZone or Digg if you liked it, or you can leave a comment or two.

WordPress: WP ShareThisPost

Nitin — Sun, 10 Jan 2010 22:12:31 +0000

Although the third party bookmarking and sharing tools are quite useful to analyze the sharing trends on your blog, they do slow down the whole process of sharing. Also, as mentioned in my Sharing and Bookmarking Tools post, I was not comfortable with the small icons available with Bookmarkify and Socialable. WP ShareThisPost is a small custom wordpress plugin which provides large icons along with the flexibility to add your favorite sharing and bookmarking sites (if not included by default).

WP ShareThisPost includes the following sharing and bookmarking sites, by default:

del.icio.us
Digg
Google Bookmarks
StumbleUpon
Facebook
Twitter
Technorati

Go through the readme.txt file to know how you can customize WP ShareThisPost to include more of your favorite sharing and bookmarking sites.

WP ShareThisPost, by default, only adds the sharing and bookmarking links at the end of posts. You can see the readme.txt file to see how you can change that to include pages, archives, feeds, etc.

Installation

Upload the unzipped folder into the plugins folder of your wordpress installation (usually /wp-content/plugins), activate it and you are ready to go. You will see the desired links below each of your posts:

WP ShareThisPost

Styling

I use this CSS to style my WP ShareThisPost, adapt as you wish of course by changing the wp_sharethispost.css file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

#content .wp_sharethispost ul {
display: inline;
margin: 12px !important;
padding: 0 !important;
}
#content .wp_sharethispost ul li {
background: none;
display: inline !important;
list-style-type: none;
margin: 0;
padding: 2px 0 2px 15px;
}
#content .wp_sharethispost li:before { content: ""; }
#content .wp_sharethispost img {
float: none;
width: 48px;
height: 48px;
border: 0;
margin: 0;
padding: 0;
}
#content .wp_sharethispost .wp_sharethispost_tagline {
font-size: 1.5em;
color: #486472;
margin: 10px 10px 10px 10px;
}

PHP: Select HTML elements with more than one css class using XPath

Nitin — Fri, 02 Oct 2009 22:39:43 +0000

Recently, while working on my project FeedAPI ImageGrabber I came across the issue where I had to select HTML elements by a given css class. The HTML element can have a single CSS class:

1	<div class="foo">div>

or multiple CSS classes associated with it.

1	<div class="foo exp tar">div>

Now if you want to select the HTML elements with class “foo”, this div element would be one of them.

As, I do not have much experience with XPath, I started to search for a solution. After a lot of search and discussion on Drupal development mailing lists, I arrived at the following XPath :

1	//*[@class and contains(concat(' ',normalize-space(@class),' '),' $classname ')]

The normalize-space function replaces all continuous tabs and whitespaces with a single whitespace character. Then, we concatenate the resulting string with two white-spaces, one in the start and other at the last. At last, we search for

1	' ' . $classname . ' '

in the final string.

I hope it helps.

MYSQL: Get the Auto-Increment Values after Insert Statement

Nitin — Wed, 15 Jul 2009 12:24:32 +0000

Until sometime back, I was unaware of how I could get the value of an auto-incremented field after an Insert Statement. But recently I needed it for two of my projects simultaneously, so I explored the possibilities. As one of my project is in Java and other is in PHP, we will go through both of them.

So, here is the problem. In general, the PRIMARY KEY field is the AUTO_INCREMENT field. Now wen you insert a row, a new key is generated and you can’t get it through usual way as this row can be accessed only using the primary key.

So here is how it can be done:

In JAVA:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

/*Insert a row into the desired table that generates
an auto increment field*/

stmt.executeUpdate("Insert into tableName (col1, col2) values (val1, val2)", Statement.RETURN_GENERATED_KEYS);

ResultSet rs = stmt.getGeneratedKeys();

int autoIncValue = -1;

if(rs.next())
{
autoIncValue = rs.getInt(1);
/*You can get more generated keys if they are generated in your code*/
}

In PHP:

The mysql_insert_id() function in PHP returns the AUTO_INCREMENT ID generated from the previous INSERT operation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

$con = mysql_connect("hostname", "username", "password");
if (!$con)
{
die('Error while connecting: ' . mysql_error());
}

$db_selected = mysql_select_db(database_name,$con);

$sql = "INSERT INTO tablename VALUES (val1,val2)";
$result = mysql_query($sql,$con);

echo "ID of last inserted record is: " . mysql_insert_id();

mysql_close($con);
?>

Note: Be sure to call mysql_insert_id() immediately after a query to get the correct value. The function returns 0 if the previous query does not generate an AUTO_INCREMENT value, or FALSE if no MySQL connection was established.

Enjoy!!

Sharing and Bookmarking Tools

Nitin — Tue, 14 Apr 2009 08:33:07 +0000

It takes me quite a lot of time to write a post here, mainly because I have very less going on in my life at the moment. But if you were following this blog closely (I know you weren’t ), you might have noticed that I kept playing with the SHARING and BOOKMARKING tools. I started off with the bookmarkify, then tried addthis and then finally settled for tellafriend. Although, I am still thinking of shifting back to Bookmarkify, right now I am sticking with TellaFriend. As far as the features are concerned, they all almost provide the same kind of features. All of them provide access to over 50 Bookmarking sites, not to forget the “Email this” option. Here is a short review on the following Bookmarking tools.

Bookmarkify: This WordPress plugin lets you insert social bookmarking links in your posts and other pages. The most important advantage Bookmarkify has is that it put direct links to the bookmarking and sharing websites on your pages/posts. This saves you from depending on another website (as in both of the next two tools) for bookmarking your pages. Visit the official plugin page for more of its features. So, why I am not using it? The one and only reason I am not using it is because of the tiny icons it places on your post.
Small Images by Bookmarkify

May be, I will modify it to use large icons in near future.
AddThis: The next two plugins are almost similar. They both place a javascript on your posts/pages, which opens the list of available sharing options. The major difference between these two and Bookmarkify is that, rather than putting a direct link to the bookmarking sites, they put a link to their site which redirects to the Bookmarking or Sharing site. So, how is this advantageous? Well!! you can keep track of the visitors who digged your websites, how many times its already digged/shared and if it gets highly shared, get prepared for the high traffic. These tools also offer powerful analytics so you can learn how and where your visitors are sharing your content.
mouseover the bookmarking button
TellaFriend: Although this tool differs from the last one ideologically, it is quite similar to Addthis. TellaFriend is made for the “word of mouth” kind of sharing. For this, it integrates the most famous email-service providers (Gmail, Yahoo, Hotmail), so that they can get your address book to you and you can share it to your friends without even having to remember their email addresses. It also integrates the famous IM (Gtalk, Yahoo messenger,) and Blogging (WordPress, Blogger, ) services for promoting the same idea.
Mouseover the tellafriend button

It also provides the feature of adding the title and description for the Bookamrking site right there on your website. Although, its not such a great feature but still it scores +1 for it.

Enter the title & description and hit "Bookmark"

One more but less important thing in which it scores over Addthis is its redirecting page. Compare the following two redirect pages, you know what I am talking about!!

So, Decide for yourself which one of them you want. If you are using something else, do leave a comment. and remember Sharing is Good.

Drupal: FeedAPI Imagegrabber

Nitin — Sun, 05 Apr 2009 07:08:10 +0000

Feeds Image Grabber (FIG) was released on 3rd March 2010, to support the Feeds module.

Introduction

FeedAPI Imagegrabber is a add-on module for FeedAPI. It consists of a parser which visits the original URL of a new feed item, and retrieves the main image from the post. Once the main image has been retrieved, it is then converted into a thumbnail using the ImageCache module, and stored in the node created by FeedAPI, inside a CCK field.

The purpose of FeedAPI Imagegrabber is to make the feed more informative as well as interesting for the user. As, we all know that “comics are much better than novels”, this module appends the feed-item with an appropriate image from its content URL. The goal of the module is to mimic the thumbnail display of websites such as digg.com. This goal is acheived by using FeedAPI to turn RSS feed-items into nodes, and then using FeedAPI Imagegrabber to append these nodes with an appropriate image from the feed-item’s webpage.

How it works

A classic method of mimicking the behavior of FeedAPI Imagegrabber will be to do the same thing manually. Let us go through the procedure if you were to do it manually:

Refresh the feed.
For each feed-item, go to their respective original URL and save the image to display.
Crop the image, convert it into a thumbnail and then upload it in an CCK Image field.

FeedAPI Imagegrabber automates the last 2 steps of the three step process described. It uses cURL to download the images and then crop them using Imagecache. Then the image is stored into the CCK Imagefield. The most difficult part which only humans can do is to select the image, for which I am constantly improving on the heuristics.

Download and Install

Visit the project page : FeedAPI Imagegrabber

Features and Future Releases

Visit the project page : FeedAPI Imagegrabber

History

I understand history is boring, but it is only for those who think the other way round. I started working on this module during December 2008. At the time, when I started working on this module, I had a little or say no idea about Drupal or its API. It took me quite some time to get used to it and start working on the module. In the Initial release, I thought of very simple heuristics and decided to select the largest image available on the web-page. I completed the module in about 1 month and was very happy with my performance, and then a big blow came which delayed the release of FeedAPI Imagegrabber by 2 months. I had to include an external BSD licensed script for converting relative URL’s to absolute URL’s but Drupal denied to accept this. They said they allow only GPL licensed code and I started to convince them that both licenses are compatible. Unfortunately, I was unable to convince them and I had to create an external link to that BSD script on sourceforge. And now, the module is released with the hope to enhance the visitor-experience on several websites around the globe.

You may find these posts useful:

Tutorial for FeedAPI ImageGrabber
PHP: Relative URL to Absolute URL
Open Source Software and Licenses

Do Leave your comments to let me know what you think.

Note: Support queries and requests should be made through the FeedAPI Imagegrabber Forum. I may not be able to answer them here.

Updates:

The first stable version of the module was released on 7th April, 2009. Check it out now here!!
RC1 release for Imagegrabber is now available for download on Project page. It is compatible with the RC1 release of Imagefield. (17 Apr, 2009)
Demonstration Website is up, and is available here. (20 Aug, 2009)
FeedAPI ImageGrabber v1.8 released. Download it from the project website here. (20 September 2009)
Tutorial for FeedAPI ImageGrabber published. (20 September 2009)
FeedAPI ImageGrabber v1.9 released. Download it from the project website here. (26 October 2009)
FeedAPI ImageGrabber now supports Filefield Paths module. Download the development snapshot, its stable for use on production site.
As of 02 Dec 2009, Development of FeedAPI ImageGrabber has been halted. You can look out for the Feeds Image Grabber module which supports Feeds module, successor of FeedAPI, with the same functionality.

PHP: Archive files for Backup on Server

Nitin — Wed, 11 Mar 2009 05:25:50 +0000

As Currently I am hosting this website on a free host, my web host imposes several limitations on me. One of them is not providing the feature to download the whole folder. This makes taking backup really hard as you have download each and every file yourself. Although it is a matter of days when I shift this website to a new server, I wrote the following PHP script which automatically archives the files in a folder and then the zip files can be easily downloaded with a single click. The script is crude in nature as I did not handle much exceptions but it works fine for me. You might need to change certain parameters to run it with your PHP resource limits. The code is pretty self explanatory although if you find difficulty understanding some part or find a bug or want to suggest any improvement, do leave a comment.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135

< ?
/**
* Copyright (c) 2009, Nitin Kumar Gupta, http://publicmind.in.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
*
* * Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following
* disclaimer in the documentation and/or other materials provided
* with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
* FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
* COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
* BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
* CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY
* WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
* OF SUCH DAMAGE.
*/

/*
* This is a BSD License approved by the Open Source Initiative (OSI).
* See: http://www.opensource.org/licenses/bsd-license.php
*/

/**
* Directory to archive.
* example: '.' - root directory
* 'folder/sub-folder/sub-folder' - for an internal folder.
*
* Note: Do not use trailing slashes.
*/
$directory = '.';

// File name for the backup along with timestamp. If the filename exists,
// They are numbered as backup_1, backup_2,...
$time = date("d_m_y");;
$zipname = 'backup_'.$time;
$count = 1;

$maxFiles = 500; //Number of files to be added to each archive.
//Varies with your PHP Memory limit and execution time.
//After a certain limit, the zipArchive is unable to archive.

$fileNums = 0;
$filenames = array();

parse($directory);
zipFiles();
echo "Done";
return;

/**
* Archives the files present in the global $filename array.
*
* @return:
*
* 0: fail
* 1: Success
*/
function zipFiles() {
global $count;
global $zipname;
global $filenames;
$zipfile = $zipname . '.zip';
while(file_exists($zipfile)) {
$zipfile = $zipname.'_'.$count.'.zip';
$count++;
}
$zip = new ZipArchive();

if ($zip->open($zipfile, ZIPARCHIVE::CREATE)!==TRUE) {
exit("Error opening , Make sure you have write permissionsn");
}
foreach ($filenames as $filename) {
$zip->addFile($filename,$filename);
echo "Adding " . $filename . " to ".$zipfile."";
}

echo "Files Packed = " . $zip->numFiles . "
";
if(!$zip->status) {
echo "Success";
return 1;
}
else {
echo "Fail";
return 0;
}
$zip->close();
}

/**
* parses the whole directory and archives it in small parts, each
* containg $maxFiles files each.
*
* @param:
*
* $dir: Directory to archive recursively.
*/
function parse($dir) {
global $filenames;
global $fileNums;
global $maxFiles;
if ($handle = opendir($dir)) {
while (false !== ($file = readdir($handle))) {
if ($file != "." && $file != ".." && is_file($dir.'/'.$file)) {
$filenames[] = $dir.'/'.$file;
$fileNums++;

if($fileNums >= $maxFiles) {
$fileNums=0;
if(!zipFiles()) return;
$filenames = array();
}
}

else if ($file != "." && $file != ".." && is_dir($dir.'/'.$file)) {
parse($dir.'/'.$file);
}
}
closedir($handle);
}
}
?>

To get going, just upload the script file in your root directory and point the $directory variable to the folder you want to archive. Now run the script using your web browser(make sure you run it once) and you are done. You will see the backup zip files in the current directory.

Hope you find it useful.

PHP: Relative URL to Absolute URL

Nitin — Mon, 02 Mar 2009 10:45:31 +0000

Recently, I was working on a project and I needed to convert the relative URLs to their absolute URLs. An absolute URL such as “http://www.example.com/image.jpg” but mostly written as only “image.jpg” (relative URL) on the web pages. Web browser creates a complete URL using this relative URL and base URL. The base URL, must be absolute and it is often the URL of the web page containing the relative URL.

Now, if you need to access a file whose relative URL and base URL are known, you must combine them to create an absolute URL.

The RFC URL specification defines an “absolutize” algorithm for combining an absolute base URL with a relative URL to create a new absolute URL. This algorithm was already implemented and I found the PHP script for the algorithm at nadeausoftware.com. I have created a project UrlToAbsolute at sourceforge, as I needed to use it in my several open source projects including FeedAPI ImageGrabber, Facebook-style Links and Feeds Image Grabber. As the original script was released under BSD License, it was pretty easy to fork it. Here is the link to the project UrlToAbsolute. You can also look at the step by step explanation of the script here.

Usage Instructions

Extract the script (url_to_absolute.php) into your web directory, include it into your current php file using:

1	require(path-to-file);

then, you can convert the relative url to absolute url by calling:

1	url_to_absolute( $baseUrl, $relativeUrl);

It return false on failure, otherwise returns the absolute url. If the $relativeUrl is a valid absolute url, it is returned without any modification.

Related resources:

Project page: http://sourceforge.net/projects/absoluteurl
Original article: http://nadeausoftware.com/articles/2008/05/php_tip_how_convert_relative_url_absolute_url

ChangeLog

v1.6, March 12, 2010
- added encode_url function to convert an absolute url to its percentage
encoded equivalent, according to RFC 3986
v1.5, March 11, 2010
- fixed to allow spaces in the path of url
v1.4, October 2, 2009
- Percentage encoding of the absolute url disabled.
v1.0, February 27, 2009
- Initial release of the script on nadeausoftware.com