How can I copy only changed files from a website?

(Updated: )

Once you've copied a website, how do you download only new and changed files? The WebCopy UI isn't clear in this regard, so here's a few tips.

First, in order to copy only the changes to a website, the website itself needs to support the Last-Modified or ETag headers, and you need to set up your project in a specific way.

  • The Save link information in project option should be set
    (Category: <1.8 General\Links, 1.8+ Advanced\Link Map)
  • The Empty website folder before copy option should not be set
    (Category: General\Folder)
  • The Always download latest version option should not be set
    (Category: <1.8 General\Advanced, 1.8+ Advanced)

This combination of settings should ensure that WebCopy only copies new and changed files. However, if the website doesn't return a 304 "Not Modified" response code, WebCopy has no way of knowing if the file is unchanged or not, and will proceed with the re-downloading the file.