Archive
Audio
Audio files have been synced to oci and android from Alpine Linux using syncthing .
The setting in android is receive only with filters to show only tamil folder using this ignore pattern.
(?i)!/tamil
*
Photos
Photos will be stored to oci from android and Alpine Linux or Linux Mint . Pre-2021 photos will continue to remain in Google Drive.
workflow
The most important change is to turn OFF automatic backup in Google photos. All existing photos on google drive will continue to be accessible. But no photo goes from phone automatically to google drive.
Instead of Google Drive, oci will act as cloud storage. For sharing individual photos or folder or accessing photos from anydevice or anywhere, filebrowser can be used.
Syncthing instead of Google Drive
Instead syncthing is used to backup photos from DCIM/Camera folder of phone to data/Incoming/Redmi or /data/Incoming/Samsung/folder on desktop/laptop. This folder is local to KVNagar, Sastra i.e LAN only.
The /data/photos folder is synced from local laptop/desktop to phone folder /photos on SD card. This is the master folder synced with oci and all laptops/desktops.
The actual moving of photos from /data/Incoming to /data/photos should be done preferably done on the desktop/laptop using any photo manager app like digiKam or shotwell .
Handling Google photos quirks
Google Photos is designed to be a “layer” on top of your local files. As long as Backup is OFF, it acts like a high-tech glass pane—it lets you look at and search the files, but it doesn’t touch the original files on your SD card unless you:
Edit a photo: If you use Google’s “Magic Eraser” or color tools, it will prompt you to “Save a copy” (creating a new file) or “Modify” the original (which would trigger a Syncthing conflict).
Delete a photo: This is the biggest risk. If you hit the trash icon in Google Photos, it will try to delete the file from your SD card.
How the “Cloud Tags/Albums” work without eating storage
When you create an “Album” in Google Photos for photos on your SD card (with Backup OFF):
Metadata Only: Google stores the “list” of what photos are in that album on its servers. This uses virtually zero storage quota.
Thumbnail Uploads: Even with backup off, Google Photos may upload tiny low-res “thumbnails” to its server so you can see your album list on other devices. These are negligible and usually don’t count against your 15GB.
The “Peek” Benefit: You still get the AI Search (searching for “car” or “mountains”). Google indexes the photos locally on your phone’s processor to make this work, even if the full-sized photo never touches the cloud.
The “Perfect Balance” Settings
To ensure Google Photos stays in its lane and doesn’t mess with your Shotwell/Syncthing setup, check these settings:
Backup & Sync: OFF.
Back up device folders: Go to Photos Settings > Backup > Back up device folders and ensure every folder managed by Syncthing is Toggled OFF. This is your safety net.
Use Aves for moving files, checking EXIF data from Shotwell, and looking at your GPS maps. Since Aves is “dumb” (it doesn’t have a cloud), it’s much safer for file management.
Summary of your “Unlimited” Workflow
- Data Integrity: Handled by Shotwell (Server) and Syncthing.
- Cloud Intelligence: Handled by Google Photos (Viewer only).
- Power Features: Handled by Aves (Metadata/Sorting).
Google Drive storage policy for photos
The “Pre-2021” Exemption: Any photos or videos uploaded in “High Quality” (now Storage Saver) before June 1, 2021, do not count toward your storage at all. Since your library has 8,000+ images dating back to 2001, most of your library is likely “free” in Google’s eyes.
Google’s policy for the June 1, 2021 storage change states that photos backed up in “Storage saver” (High Quality) before that date are exempt. While Google Docs or Sheets start counting if you edit them today, photos are handled differently. Changing the Date/Time or Location metadata in the Google Photos interface is considered a “metadata adjustment,” not a “new file upload.”
Metadata is a “Sidecar”: When you change a date in Google Photos, Google doesn’t actually rewrite the original image file in their storage; they update a database entry linked to that photo.
The Policy “Grandfather” Clause: Since the files were originally “onboarded” to Google’s servers before 2021, they remain in the “Exempt” category regardless of date corrections you make now.
If you have a large batch of photos that are all wrong (e.g., they all say “Jan 1, 1970” or “Dec 31, 1979”), you can fix them all at once :
- Select multiple photos in the Google Photos web interface (click the checkmark on the first one, hold Shift, and click the last one).
- Click the More options (three dots) menu at the top.
- Select Edit date & time.
- Choose “Shift dates & times” if they are all off by the same amount, or “Set one date & time” to put them all into a specific month in 2000.
Warning: If you download the photo, fix the date on your computer using a program, and then re-upload it to Google Photos, it will count toward your storage because Google sees it as a brand-new file uploaded in 2025/2026.
Changing Location, Fixing Dates, Rotating Photos are free and does not count (if you select “Save”). Do not use save a copy.
Tools required
The following packages are required. Install if not present already. syncthing , exiftool, jdupes and unzip on the Linux and automatic backup to GDrive needs to be disabled in Google Photos in Android phones.
Folder structure for handling takeout
prabu@homepc2 ~> mkdir -p /data/myhome/prabu/takeout_work/raw_zips
prabu@homepc2 ~> mkdir -p /data/myhome/prabu/takeout_work/extracted
prabu@homepc2 ~> mkdir -p /data/myhome/prabu/takeout_work/perfect_stage
Unzip the downloaded files.
unzip "/data/myhome/prabu/takeout_work/raw_zips/*.zip" -d /data/myhome/prabu/takeout_work/extracted/
Exiftool
To extract all metadata from a specific file:
exiftool [filename]
Key information to look for are:
- DateTimeOriginal: The date/time when the original image data was generated.
- CreateDate: Often used by digital cameras to record the digital creation time.
- FileModifyDate: The last time the file itself was modified on the disk (this changes if the file is moved or copied).
exiftool is very powerful in moving photos to specific location based on various data or automatically place files in a directory structure. Some useful and commonly used options are listed below:
- ‘-r’ Scans subdirectories recursively
- ‘-p’ ‘$Directory/$Filename’: Prints the full path of any file that hits the “error” condition.
- ‘-if’ filter options can be created
- ‘-testrun’ can be added to added to any move command
- ‘-T’ presents results in table format for easy viewing
- ‘-Directory<DateTimeOriginal’, rebuilds folder structure based on the actual time the shutter clicked.
Find files in the current directory with “Empty” date:
exiftool -filename -r -if 'not $DateTimeOriginal' -ext jpg -ext png .
The following command finds and prints “mismatched” files based on filename and their ‘pwd’
exiftool -r -p '$Directory/$Filename' \
-if '$Filename =~ /(\d{4})/ and $Directory !~ /$1/' \
/data/photos
To export metadata for multiple files into a CSV for easier analysis:
exiftool -csv -r -DateTimeOriginal -CreateDate -FileModifyDate [directory_path] > metadata_report.csv
To move all photos taken by ClickSmart model camera to a 2003 folder:
exiftool '-Directory=/data/photos/2003/' -if '$Model eq "ClickSmart"'
temp_photos/
To place files in a neat directory structure based on ‘DateTimeOriginal’
exiftool -r '-Directory<DateTimeOriginal' -d /data/photos/%Y/%m-%b /data/unsorted
By default, ExifTool creates a backup file (e.g.,DSCN9162.jpg_original). Using the flag ‘-overwrite_original’ prevents photos folder from getting cluttered with backup files.
exiftool can be used to find and move video files recursively using the following commands: To find:
exiftool -r -ext mp4 -ext mov -ext avi -ext 3gp -p '$directory/$filename' /data/photos/
To move:
exiftool -r -ext mp4 -ext mov -ext avi -ext 3gp -ext wmv -ext mpg \
'-Directory=/data/video' \
/data/photos/
Handling json in the Takeout Zips
Google gives multiple .zip files each of around 2GB size (e.g.,takeout-20260224T054528Z-3-001.zip, takeout-20260224T054528Z-3-002.zip)
When visiting takeout.google.com, once the zip files are ready, a file will be autodownled as follows: takeout-20260224T054528Z-001.zip. This contains a detailed summary of all zip files and their contents including directory structure.
Every photo (like IMG_123.jpg) will have a matching IMG_123.jpg.json. The json files names are however not consistent.
If there are edited photos in the Google Photos app, there might be IMG_123-edited.jpg files. Google also includes “Albums” which are just copies of photos already in “Year” folders.
- Stop Syncthing
- Backup old data just in case
- Move the perfected Stage folder
- Start Syncthing
Use “exiftool” to move photos
Extract all your Takeout zips into one folder and run the “Magic Command” inside that folder.
cd /data/myhome/prabu/takeout_work/extracted/Takeout/Google\ Photos/
When using exiftool, the data from json files needs to integrated first. A lot of time was wasted due to wrong json file list. Use the below command to find all the json filenames first.
find . -name "*.json" -type f > json_list
To restore original metadata (timestamps, GPS, and descriptions) to image files using Google’s sidecar JSON files as the exclusive source of truth.
If a matching JSON is missing, or if the JSON lacks those specific fields, the image file is left unchanged.
A exiftool command based on the above json_list alone will update the files properly. The below command matched the json_list in my takeout. The below command needs to modified based on actual json_list of every takeout.
exiftool -r -d %s \
-tagsfromfile "%d/%F.supplemental-metadata.json" \
"-GPSAltitude<GeoDataAltitude" "-GPSLatitude<GeoDataLatitude" "-GPSLongitude<GeoDataLongitude" \
"-Description<Description" "-DateTimeOriginal<PhotoTakenTimeTimestamp" \
-tagsfromfile "%d/%F.supp*.json" \
"-GPSAltitude<GeoDataAltitude" "-GPSLatitude<GeoDataLatitude" "-GPSLongitude<GeoDataLongitude" \
"-Description<Description" "-DateTimeOriginal<PhotoTakenTimeTimestamp" \
-tagsfromfile "%d/%f.json" \
"-GPSAltitude<GeoDataAltitude" "-GPSLatitude<GeoDataLatitude" "-GPSLongitude<GeoDataLongitude" \
"-Description<Description" "-DateTimeOriginal<PhotoTakenTimeTimestamp" \
-ext jpg -ext jpeg -ext png -overwrite_original .
It reads the JSON, finds the “Photo Taken Time,” and writes it back into the actual Image header. It also restores your GPS coordinates.
The following exif attributes can be added from Json files in the above command, if needed:
-GPSAltitude<GeoDataAltitude -GPSLatitude<GeoDataLatitude -GPSLongitude<GeoDataLongitude -DateTimeOriginal<PhotoTakenTimeTimestamp -Description<descriptions -Caption-Abstract<descriptions -ImageDescription<descriptions -Title<title
Only with the above command, better merging of json information happened..
Warning: Error opening file - ./Prabu Anand, Aadhirai/IMG_0406.json
Warning: No writable tags set from ./Prabu Anand, Aadhirai/IMG_0406.json
41 directories scanned
9446 image files updated
1755 image files unchanged
27 files weren't updated due to errors
Now proceed to delete the json files.
find . -name "*.json" -type f -delete
Moved all curated albums for safe keeping first..to deal with them later..
prabu@homepc2 /d/m/p/t/e/T/Google Photos> find . -maxdepth 1 -type d ! -name "Photos from*" ! -name "." -exec mv {} /data/myhome/prabu/takeout_work/curated_albums/ \;
Before deleting any files, generate a tab-separated report to compare the internal metadata of the existing library against the reconstructed Google Takeout files.
exiftool -r -T -filename -DateTimeOriginal -Directory \
-ext jpg -ext jpeg -ext png -ext mp4 \
"/data/photos" \
"/data/myhome/prabu/takeout_work/extracted/Takeout/Google Photos" \
> metadata_comparison.txt
Deduplicate Photos/Photos
Move every folder that DOES NOT start with “Photos from” for safe keeping if required:
find . -maxdepth 1 -type d ! -name "Photos from*" ! -name "." -exec mv {} /data/myhome/prabu/takeout_work/curated_albums/ \;
Run a scan on your fixed Takeout folder to find byte-for-byte identical files:
To verify what’ll be deleted, here’s the command..
jdupes -r -S "/data/myhome/prabu/takeout_work/extracted/Takeout/Google Photos" "/data/photos" > /home/prabu/jdupe_output
The actual command to delete duplicate files automatically.
jdupes -rdN "/data/photos" "/data/myhome/prabu/takeout_work/extracted/Takeout/Google Photos"
If you run the delete command with the old library listed first, you will lose the good metadata forever because jdupes will delete the Takeout version.
Moved old photos for safekeeping for now..
mkdir -p /data/myhome/prabu/takeout_work/archive_old_photos/
mv /data/photos/* /data/myhome/prabu/takeout_work/archive_old_photos/
Identified the files fixed by exiftool with files with names like /data/myhome/prabu/takeout_work/extracted/Takeout/Google Photos/Photos from 2024/IMG_20240314_200822.jpg_original
find /data/myhome/prabu/takeout_work/ -name "*_original"
add -delete flag to delete..
find /data/myhome/prabu/takeout_work/ -name "*_original" -delete
moved unsorted files(missing metadata) for manual Sorting
mkdir -p /data/photos/Unsorted
find "/data/myhome/prabu/takeout_work/extracted/Takeout/Google Photos/" -type f -exec mv -t /data/photos/Unsorted/ {} +
Keep /data/myhome/prabu/takeout_work/archive_old_photos (5.8G) for about a week. Open a few photos from 2003 or 2019 just to make sure they look correct. Once you are happy, delete that folder to free up 6GB.
You can now safely remove the raw_zips and the extracted folder to reclaim space:
rm -rf /data/myhome/prabu/takeout_work/raw_zips
rm -rf /data/myhome/prabu/takeout_work/extracted
size of non-photos
find /data/photos -type f \( -iname "*.mp4" -o -iname "*.mov" -o -iname "*.avi" \) -exec du -ch {} + | grep total
To see exactly what the non-JPG files are (to decide if you want to move them), run this command. It will give you a count of every file extension in your library:
$ find /data/photos -type f | grep -E -o "\.[a-zA-Z0-9]+$" | sort | uniq -c | sort -nr
5100 .jpg
3561 .JPG
62 .mp4
10 .gif
7 .jpeg
7 .AVI
4 .png
1 .txt
To move EVERYTHING that is not a standard image (JPG/PNG/HEIC):
find /data/photos -type f \( -iname "*.mp4" -o -iname "*.avi" -o -iname "*.txt" \) -exec mv -v {} /data/video/unsorted/ \;
© Prabu Anand K 2020-2026