I have a bash script using tree to get a recursive list of local file paths, then convert these into their final public URL’s by prepending a URL prefix, and writing this to a file.

#!/bin/bash
cd public/media
tree -F -fi --prune --noreport | grep -v /$ > ../../urls/ALL
sed -i 's/^s*./https://example.com/media/g' ../../urls/ALL

The file output looks like this (the depth level of subdirectories could be arbitrary):

https://example.com/media/faves-1716910771/architecture/file-01.jpg
https://example.com/media/faves-1716910771/architecture/file-02.jpg
https://example.com/media/faves-1716910771/automotive/file-01.jpg
https://example.com/media/faves-1716910771/automotive/file-02.jpg
https://example.com/media/misc-1716910771/animals/file-01.jpg
https://example.com/media/misc-1716910771/animals/file-02.jpg
https://example.com/media/misc-1716910771/funny/file-01.jpg
https://example.com/media/misc-1716910771/funny/file-02.jpg

This is fine, but it occurs to me that after awhile this file could get rather long.

I’m wondering if there is a way to modify my script, using sed, awk, python, or anything, where the one file can be split into multiple files, every time the subpath pattern changes, and to create files with names reflecting the subpath after “media” and before the filenames, converting slashes to hyphens and removing the epoch timestamp portion of the subpath, such as …

faves-architecture
faves-automotive
misc-animals
misc-funny

… and then inserting only the relevant URL’s into each of those files.