For facial recognition experts and privacy advocates, the East Bay detective’s request, while dystopian, was also entirely predictable. It emphasizes the ways that, without oversight, law enforcement is able to mix and match technologies in unintended ways, using untested algorithms to single out suspects based on unknowable criteria.
TBH: Tech companies are not much different from how you described cops.
They don’t usually bother to learn the tech they are using properly and take all the shortcuts possible. You see this by the current spout of AI startups. Sure, LLMs work pretty good. But most other applications of AI is more like: “LOL, no idea how to solve the problem. I hooked it up to this blackbox, which i don’t understand, and trained it to give me the results i want.”