The default functionality of how emacs handles its backup files annoys me. I don't like how it clutters my folders by leaving a trail of ~ files wherever I go. If you're in the same camp there's a couple of things you can do about it.

You can clean up existing your directories by removing backup files by any number of ways. I like to use one of these commands:

#  from anywhere
find /path/to/directory -name '*~' -exec rm {} \;

# from a directory
rm -rf *~

Then you can either change the backup directory so all files are stored in a common location:

;; add to .emacs file
'(backup-directory-alist (quote (("." . "~/backup/path"))))

or prevent emacs from making backup files in the first place:

;; add to .emacs file
(setq make-backup-files nil)