remove files and directories but exclude all files from path

$ pwd
/home/teStUser/source

$ tree
.
├── excludefile.txt
├── i am staying.gif
├── james
│   ├── delete_this.gif
│   ├── leaveme.gif
│   └── leaveme.jpg
├── leaveme.jpg
└── to-delete.gif

1 directory, 7 files

$ cat excludefile.txt
james/leaveme.jpg
i am staying.gif
james/leaveme.gif
excludefile.txt

$ rsync --dry-run -v -r --remove-source-files  --exclude-from='/home/teStUser/source/excludefile.txt' /home/teStUser/source/ /home/teStUser/source/to_delete
sending incremental file list
created directory /home/teStUser/source/to_delete
./
leaveme.jpg
to-delete.gif
james/
james/delete_this.gif

sent 182 bytes  received 82 bytes  528.00 bytes/sec
total size is 0  speedup is 0.00 (DRY RUN)


$ rsync -v -r --remove-source-files  --exclude-from='/home/teStUser/source/excludefile.txt' /home/teStUser/source/ /home/teStUser/source/to_delete
sending incremental file list
created directory /home/teStUser/source/to_delete
./
leaveme.jpg
to-delete.gif
james/
james/delete_this.gif

sent 298 bytes  received 162 bytes  920.00 bytes/sec
total size is 0  speedup is 0.00

$ tree
.
├── excludefile.txt
├── i am staying.gif
├── james
│   ├── leaveme.gif
│   └── leaveme.jpg
└── to_delete
    ├── james
    │   └── delete_this.gif
    ├── leaveme.jpg
    └── to-delete.gif

3 directories, 7 files

$ pwd
/home/teStUser/source