What problem would this solve?
When walking the seed paths oans skips any directory entry that is not a file or directory. So symlinks will also be skipped. The user may want symlinks to directories to be followed to find more paths to dedupe or symlinks to files to have the referred to files to be deduped. This also would allow users to solve the current limitation of not being able to hash/dedupe files with canonical paths of length greater than PATH_MAX. See this comment: #117
Proposed solution
Programs like find have options for how to handle symlinks. It would be nice to have an option (maybe -l `), that turns on symlink dereferencing. When on, walked to directory entries that a are a symlink will be dereferenced and if it exists and it points to a directory, add it to the directory queue, if it points to a file then add it to the file queue.
This is not completely trivial because we also don't want multiple records in the db to paths that have the same canonical path. As it is, the db only has canonical paths, and it should stay that way. In order to allow the work-around to #117 mentioned above, special care must be made with files that have canonical paths that are greater than PATH_MAX. The db should have no problem with such long paths, but oans will need code to canonicalize paths. It would be nice to use glib's canonicalize_file_name function, but it fails with canonical paths that are longer than PATH_MAX. So oans will need to roll its own, but that shouldn't be a huge lift.
Alternatives considered
No response
What problem would this solve?
When walking the seed paths oans skips any directory entry that is not a file or directory. So symlinks will also be skipped. The user may want symlinks to directories to be followed to find more paths to dedupe or symlinks to files to have the referred to files to be deduped. This also would allow users to solve the current limitation of not being able to hash/dedupe files with canonical paths of length greater than
PATH_MAX. See this comment: #117Proposed solution
Programs like
findhave options for how to handle symlinks. It would be nice to have an option (maybe-l`), that turns on symlink dereferencing. When on, walked to directory entries that a are a symlink will be dereferenced and if it exists and it points to a directory, add it to the directory queue, if it points to a file then add it to the file queue.This is not completely trivial because we also don't want multiple records in the db to paths that have the same canonical path. As it is, the db only has canonical paths, and it should stay that way. In order to allow the work-around to #117 mentioned above, special care must be made with files that have canonical paths that are greater than
PATH_MAX. The db should have no problem with such long paths, but oans will need code to canonicalize paths. It would be nice to use glib'scanonicalize_file_namefunction, but it fails with canonical paths that are longer thanPATH_MAX. So oans will need to roll its own, but that shouldn't be a huge lift.Alternatives considered
No response