Site Map Generator

Site Map Generator is a handy tool that works on any platform to create site maps. It starts by crawling a website from a URL you provide and then produces an XML sitemap file. This file can be super useful for Google or other search engines when you use it with Google Webmaster Tools. Plus, it's free to use because it’s published under the GNU General Public License!

How to Use Site Map Generator

One of the cool things about this generator is that you don’t need shell access to your web server! It runs as a simple crawler on any computer with Python installed. The crawler will only follow links that are local, meaning it skips any links that go off-site.

Crawling and Output Details

This generator can even add <lastmod> dates in your sitemap if your web server provides pages with 'Last-Modified' timestamps. If there’s an error while downloading or parsing a page, don’t worry! The crawler will keep going and try another page instead.

Requirements for Running the Script

You’ll need Python version 2.4 or higher to run this script (the latest version as of now is 2.5, which you can grab from Python's official site). There’s no tricky installation process—just copy it into a directory and run it from there!

Best for Small to Medium Sites

This script is great for smaller and medium-sized websites since it creates just one sitemap file. It maxes out at 50,000 URLs because that's what Google allows for sitemap files. By default, it only covers up to 1,000 URLs, but you can change that using the -m option.

Running the Script: Command Line Syntax

The command line syntax looks like this:

python sitemap_gen.py <options> <starting URL>

Available Options

• -h --help: Print help info and exit.


• -b <ext> --block <ext>: Exclude URLs with specific extensions (no leading dot). This comparison isn’t case-sensitive.


• -c <value> --changefreq <value>: Set how often content changes (like always, daily, weekly).


• -p <prio> --priority <prio>: Set priority between 0.0 and 1.0.


• -m <value> --max-urls <value>: Set how many URLs to crawl (default is 1000, max is 50000).


• -o <file> --output-file <file>: Name your generated sitemap file (default is sitemap.xml).

An Example Command

If you're ready to give it a go, here’s how you might run the script:

python sitemap_gen.py -b doc -b bmp -o test_sitemap.xml http://www.your-site-name.com/index.html