Commit Graph
5 Commits
Author SHA1 Message Date
alex de177c8f59 find_dupes: build aHash matrix as uint64 before xor (int64 array ctor overflows) 2026-10-18 18:53:58 -04:00
alex 107f48d118 find_dupes: triple-gate closeness (dHash<=6, aHash<=8, color-hist L1<=0.10) + strict linkage
Measured on verified same/different pairs: dHash alone has a false-
positive floor for same-style anime frames (distinct shots at distance
<=10 and even <=6 with different color content). Same-shot broadcast
replays are pixel-twins: near-zero color-hist distance and low aHash
distance too. Requires ALL three gates, no transitive chaining.
2026-10-18 18:51:45 -04:00
alex 7e128c4c32 find_dupes: replace union-find with strict complete-linkage clustering
Verified failure with 10/64 threshold + transitive chaining: a 90-image
'group' contained six different scenes (blonde outdoors, storm school,
night bedroom, manic grin, cat street, dining room) — chains of
similar-but-distinct frames glue unrelated shots together. Strict
linkage requires every member within threshold of every other.
Default quarantine dir renamed duplicates/.
2026-10-18 18:47:08 -04:00
alex 4816b007c0 find_dupes: fix uint64 hamming accumulation (numpy casts to float64) 2026-10-18 18:42:51 -04:00
alex a8cf8957d5 Add find_dupes.py: dHash near-duplicate clusterer for recurring scenes
Recurring OP/ED animation shots and eyecatch cards reappear in every
episode, so scene-gated extraction saves one copy per broadcast.
Reports groups with a keepsake suggestion; --move quarantines
redundant members to dupes/ (reversible). CPU-only perceptual hashing,
paranoid default threshold 10/64 bits.
2026-10-18 18:40:43 -04:00