CanSecWest Conference – MMPC Researchers Presented New Approaches
At CanSecWest 2009 in Vancouver, Canada from March 16-20 2009 a team of talented researchers from the MMPC presented “On Approaches and Tools for Automated Vulnerability Analysis.” Tanmay Ganacharya, Nikola Livic, Abhishek Singh, Swapnil Bhalode, and Scott Lambert from the MMPC lab in Redmond, USA discussed the toolset the MMPC uses in the Response process and outline some lessons learned.

Volume 6 of the Microsoft Security Intelligence Report (SIR v6) is available now. This latest volume of the SIR details the changing threat landscape worldwide during the second half of 2008. Focusing on the rise of rogue security software, SIR v6 also documents trends in software vulnerabilities and exploits, privacy breach reports, browser-based exploits and document format exploits, and the latest developments in malicious and potentially unwanted software including hosted malware, phishing and drive-by download threats.

The 3rd International Computer Antivirus Researchers Organization workshop took place in Budapest, Hungary on May 4th and 5th 2009. This year’s workshop took the theme “Exploits and Vulnerabilities”. Microsoft experts included Marian Radu and Andrei Florin Saygo from the MMPC lab in Dublin, Ireland presented Jumping through the Hoops: Impact of Adobe Acrobat and Reader Exploits and Ziv Mador from the MMPC lab in Redmond, USA presented Vulnerabilities and Exploits Patterns – What Can We Learn?

Find information, definitions, and analyses of all the latest threats that Microsoft Security Essentials can help protect you against in the Microsoft Malware Protection Center.

Need security for your business?
Protect your computers with Microsoft Forefront Client Security

*Your PC must run genuine Windows to install Microsoft Security Essentials. Learn more about genuine

Microsoft Security Essentials provides real-time protection for your home PC that guards against viruses, spyware, and other malicious software.

Microsoft Security Essentials is a free* download from Microsoft that is simple to install, easy to use, and always kept up to date so you can be assured your PC is protected by the latest technology. It’s easy to tell if your PC is secure — when you’re green, you’re good. It’s that simple.

Microsoft Security Essentials runs quietly and efficiently in the background so that you are free to use your Windows-based PC the way you want—without interruptions or long computer wait times.

• Windows versions 95,98, ME, 2000, XP, 2003 Server
• Internet Explorer v. 5.5 & up

In general, the GSiteCrawler will take a listing of your websites URLs, let you edit the settings and generate Google Sitemap files. However, the GSiteCrawler is very flexible and allows you to do a whole lot more than "just" that!

Capture URLs for your site using

• a normal website crawl - emulating a Googlebot, looking for all links and pages within your website
• an import of an existing Google Sitemap file
• an import of a server log file
• an import of any text file with URLs in it

The Crawler

• does a text-based crawl of each page, even finding URLs in javascript
• respects your robots.txt file
• respects robots meta tags for index / follow
• can run up to 15 times in parallel
• can be throttled with a user defined wait-time between URLs
• can be controlled with filters, bans, automatic URL modifications

With each page, it

• checks date (from the server of using a date meta-tag) and size of the page
• checks title, description and keyword tags
• keeps track of the time required to download and crawl the page

Once the pages are in the database, you can

• modify Google Sitemap settings like "priority" and "change frequency"
• search for pages by URL parts, title, description or keywords tags
• filter pages based on custom criteria - adjust their settings globally
• edit, add and delete pages manually

And you have everything the way you want it, you can export it as

• a Google Sitemap file in XML format (of course :-)) - with or without the optional attributes like "change date", "priority" or "change frequency"
• a text URL listing for other programs (or for use as a UrlList for Yahoo!)
• a simple RSS feed
• Excel / CSV files with URLs, settings and attributes like title, description, keywords
• a Google Base Bulk-Import file
• a ROR (Resources of Resources) XML file
• a static HTML sitemap file (with relative or absolute paths)
• a new robots.txt file based on your chosen filters
• ... or almost any type of file you want - the export function uses a user-adjustable text-based template-system

For more information, it also generates

• a general site overview with the number of URLs (total, crawlable, still in queue), oldest URLs, etc
• a listing of all broken URLs linked in your site (or otherwise not-accessable URLs from the crawl)
• an overview of your sites speed with the largest pages, slowest pages by total download time or download speed (unusually server-intensive pages), and those with the most processing time (many links)
• an overview of URLs leading to "duplicate content" - with the option of automatically disabling those pages for the Google Sitemap file

Additionally ...

• It can run on just about any Windows version from Windows 95b on up (tested on Windows Vista beta 1 and all server versions).
• It can use local MS-Access databases for re-use with other tools
• It can also use SQL-Server or MSDE databases for larger sites (requires a seperate installation file).
• It can be run in a network environment, splitting crawlers over multiple computers - sharing the same database (for both Access and SQL-Server).
• It can be run automated, either locally on the server or on a remote workstation with automatic FTP upload of the sitemap file.
• It tests for and recognizes non-standard file-not-found pages (without HTTP result code 404).

Please double check that you named your verification file correctly (like Google told you) and that there are no spaces in front or after the file name. The GSiteCrawler can generate the verification file for you, you can just copy+paste the filename from the web page to the GSiteCrawler and let it upload the appropriate file. Otherwise you may also use the meta-tag verification system.

If it still does not work, make sure your server returns correct headers for existing URLs (200) and for missing URLs (404). You can check this by using a server header status code tool. Enter the URL to your verification file and check that your server returns "200 OK" as the status code. Then try a not existing page and verify that your server returns "404 Not found". If you recieved a "200 OK" again then you need to adjust your servers settings (or contact your hoster to do it for you). Please note that there are some hosters that block Google from accessing the site fully - in these cases you might also see this error message. If your hoster is blocking Google's access, then submitting a Google Sitemap file is futile, Google won't be able to crawl the URLs anyway. In this case I can only recommend moving to a hoster with a real understanding of the internet.

"Invalid date"

This error means that your sitemap contains an entry somewhere which has a badly formatted date. Google Sitemaps requires the ISO-8601 encoding for dates. The GSiteCrawler will handle this for you automatically. If you see this error message, it is most likely that you manually changed part of the sitemap file before uploading it.

This error means that your sitemap contains URLs which are not alllowed in your sitemap. A proper Google Sitemap generate will only include URLs allowed URLs, but it is possible that you upload the Sitemap file to an incorrect location. A sitemap file can only include URLs for files on same domain and the same or deeper directories. Here's an example:

Your sitemap file is saved at http://www.example.com/hero/sitemap.xml

These URLs are all allowed:

• http://www.example.com/hero/
• http://www.example.com/hero/contact.html
• http://www.example.com/hero/test/index.html
• http://www.example.com/hero/info/about/guestbook

The following URLs are not allowed:

• http://www.example.com/ (Higher directory level than your sitemap)
• http://www.example.com/hero (Higher directory level, missing trailing slash so the URL is treated like a file)
• http://www.example.com/marcel/ (Higher directory level than your sitemap)
• http://www.hero.com (Not the same domain)
• http://example.com/hero/ (Not the same domain)

By moving your sitemap file to http://www.example.com/sitemap.xml you could include the 3 first denied URLs in the above listing. In this case you could include the following URLs:

• http://www.example.com/hero
• http://www.example.com/
• http://www.example.com/marcel/
• http://www.example.com/marcel/my-wedding.html