General
Meet few important directories:
/images/ - real(!) files organized by date (year.month)
/collection/ - symlinks organized as defined by you labels/groups.
/home/ - scripts collection: file upload and symlinks service
See scripts description below;
/imports/ - lists imports by date, represented as symlinks.
upload.py
python3 upload.py [--recursive] [--interactive] source [label_1 label_2 ..]
Non-interactive:
Copy images without individual review.
Additional arguments will create symbolic links to the whole group.
Examples:
Copy new files from SD card mounted '/media/mmcblk0' to 'images/'
do not preserve original directory structure,
create symbolic links in 'collection/mountains' and 'collection/elbrus'
`python3 upload.py -r -d "/media/mmcblk0/DCIM" elbrus mountains`
Interactive mode reference:
Review images in terminal, assign individual image labels.
Groups of similar images, i.e bursts are supposed to be uploaded by just one image review.
Slash (/) is allowed and will create sub-directories.
You're asked to create path if it does not exist, default question answers are highlighted.
Type 'exit' to leave interface.
Examples:
`python3 upload.py -i -r /media/mmcblk0/DCIM`
> [image]
> (labels) .. mountains/elbrus landscape snow
> Unknown keyword 'snow', create? (y/N) .. y
Additional options:
--blind will ignore file duplication and create given symlinks to files.
Files will not be written twice unless have different names and no metadata.
--recursive will also copy sub-directories preserving the file structure in
non-interactive mode and staging for review all sub-directories in interactive mode.
--dcim will break any sub-directories even if --recursive used non-interactively.
symlinks.py
python3 symlinks.py
Can restore broken symlinks and alerts about removed files.
!never remove image files, but remove symlinks instead.
--analyze option will output extended information: number of images tracked,
images untracked, long names, also find images not represented as links and
make symlinks to home/.unlinked
genchecksums.py
python3 genchecksums.py
Generate integrity verification file: checksums.md5
Verify the correctness of the image files with command:
md5sum --check checksums.md5 | grep FAILED
Generate reed-solomon codes for file error recovery
Combine find and par2 commands:
find images -type f -not -name "*.par2" -exec par2create {} \;
To restore corrupted file later:
par2repair /path/to/file
The default amount of restoration data is enough to
back metadata header erased with exiftool.
Then you should probably remove the corrupted file.
Export file group by label
As simple as tar command:
tar -h -czvf label.tar.gz -C collection label
To actually copy files without writing and decompressing archive use pipes:
tar -czhf - -C collection label | mbuffer -m 512M | tar xf - -C destination/
This can also be used to efficiently transfer large directories between disks.
mbuffer will prevent kernel from taking all available memory and
opens all files at once instead of open/close sequentially, thus is much faster.
