General

What you can do:

As probably stated in the readme, you can use the scripts to manage images.
It works best with new photographs, but old collections may be imported too.

This project also provides evolving tips for best practices on
organization and storage mediums.

Directories located created under ./phibrajry:

./+images - real file copies organized by camera/import_date (ISO-8160 date format).
./+library - hard links organized as you desire, at the start as original.
./+trashcan - automatically linked files with no link otherwise.
./cmds - scripts collection: descriptions below and by script's --help.
./cmds/patches - minor code changes intended for use scenarios.
./cmds/tests - simple use case tests.

More documentation:

Sometimes you can read the instructions directly by opening script in text editor,
code commentary is not omitted there.

upload.py

$ python3 upload.py [--recursive] source camera [label_1 label_2 ..]

Copy images from source to the +images directory.
This always creates accurate copy of imported directory in +library.
Additional arguments will create plain category/label directories with hard links.

Images should be separated by camera id/name as a very recommended practice.
This however can be tricked by using one camera name for everything.

Be careful! Script will truncate the file names to 8 characters, which
is the exact size of normal image file name, standardized by convention.
If you did rename files, that information will be lost.

Moreover, checking for duplicate images is based on fast,
but not truly secure hash function, so if many file names
differ only beyond 8th char, it is improbable but possible to miss some file.

Options:

--recursive

Will also copy sub-directories, preserving the directory structure.

--verbose

Print debug info.

--create

(new) Confirm adding new camera.

Examples:

Copy new files from SD card mounted '/media/mmcblk0' to '+images'
update copy of DCIM directory and create hard links in '+library/Elbrus'

$ python3 upload.py -r "/media/mmcblk0/DCIM" my_cool_camera_model Elbrus`

Add another set of links to '+library/mountains', without duplicating files:

$ python3 upload.py -r "/media/mmcblk0/DCIM" my_cool_camera_model mountains`

Do the same at one line:

$ python3 upload.py -r "/media/mmcblk0/DCIM" my_cool_camera_model Elbrus mountains`

Copy only files from top-level directory, ignoring subfolders, without any labels:

$ python3 upload.py $HOME/Pictures`


trashcan.py

$ python3 symlinks.py

Link files without any corresponding link to +trashcan

Verify checksums

Checksums allow to know if file is changed/corrupt.
(by mapping input values to unpredictable fixed length output)

Some file systems automatically calculate checksums, most do not.
When writing an image checksum function + value is added to file name.
Then, file / hashsum pairs are written to file for convenience.

To verify files correctness, use standard system utility:

$ b2sum -c +checksums.b2b | grep FAILED

If some checksum is wrong:

1) Check if that is a regular file and was not modified.
2) Restore the file from backup.
3) Check disk health!

See tips for hardware and file system suggestions.

If you indeed wanted to remove the file against all recommendations,
you can re-generate checksums file with genchecksums.py

Export files from links

Assuming 'label' is the directory to export:

$ tar -clf label.tar --hard-dereference -C +library label

To actually copy files without writing and decompressing archive use pipes:

$ tar -clf - --hard-dereference -C +library label | mbuffer -m 512M | tar -xf - -C destination/

This can also be used to efficiently transfer large directories between disks.
mbuffer will prevent kernel from taking all available memory and tar opens
all files at once instead of open/close sequentially,
the process is much faster and does not suffer the speed decrease.

Avoid do it for syncing files as it will write over again.
For more rational syncing use rsync with -H option.