User-agent: * Disallow: / # allow twitter to crawl for images. User-agent: Twitterbot Allow: /images