Have you created any symbolic links in the filesystem recently? I had a similar problem with the metalayer crawler when I though it would be a good idea to link from the documents folder (accessible in the file manager) to the root folder: it means you can browse the whole filesystem from the file manager but really upsets the crawler.