Currently, with WordPress 2.2.1, when using a custom permalink structure containing the %category% tag, permalinks to attachment posts become broken. It seems to have something to do with the attachments having no category and thus cause the link to become broken.
Using the permalink scheme /archives/%category%/%postname%/ an example attachment 'bar' attached to the post 'foo' which happens to be in the general category results in the following broken permalink http://sitename.com/archives/general/foo/bar/. The permalink makes sense, but doesn't work due to the attachment post not inheriting the category(ies) of the post it is attached to.