Working on this: <a href="https://github.com/KevinXuxuxu/anon_proxy" rel="nofollow">https://github.com/KevinXuxuxu/anon_proxy</a>, a sort of anonymization proxy to use with LLM providers. It does model (OpenAI privacy filter) + regex PII detection, and replaces them back-and-forth for API requests and responses. With locally hosted detection model, no PII leaves your local environment. I find it very useful especially when you're working on sensitive documents (legal, tax, immigration etc.), hope you find it helpful as well :)
I'm no where near as smart as OpenAI of course, but I did build <a href="https://tools.nicklothian.com/webner/index.html" rel="nofollow">https://tools.nicklothian.com/webner/index.html</a> that uses a BERT based named-entity-recognition model running in your browser to do a subset of PII redaction.<p>It works pretty well for the use cases I was playing with.<p>The OpenAI model is small enough that I might enhance my tool to use it.
I just used it on a document, but the amount of false positives this generates make it faily difficult to use?<p>I fed it a ~ 100 line markdown document, took about 10 seconds, and it decided that "matter" (as in, frontmatter), "end" (as in, frontend), MCP (as in, mcp server) are organizations.<p>Most of them don't even make grammatical sense, e.g. "Following the discussion in <PERSON_1>, blahblah".<p>Brings me back to what NLP was like a decade ago. I always thought spaCy was a very nice project in that space.
There's some interesting technical details in this release:<p>> Privacy Filter is a bidirectional token-classification model with span decoding. It begins from an autoregressive pretrained checkpoint and is then adapted into a token classifier over a fixed taxonomy of privacy labels. Instead of generating text token by token, it labels an input sequence in one pass and then decodes coherent spans with a constrained Viterbi procedure.<p>> The released model has 1.5B total parameters with 50M active parameters.<p>> [To build it] we converted a pretrained language model into a bidirectional token classifier by replacing the language modeling head with a token-classification head and post-training it with a supervised classification objective.
Exciting! I took a look through the code and found what appear to be the entity types for future releases - this release (V2 config) supports 8 entity types, but the V4 and V7 taxonomies have >20, mostly more personal ID types. Given this is a preview release, I imagine they'll release these.<p>Details in my review article here: <a href="https://piieraser.ai/blog/openai-privacy-filter" rel="nofollow">https://piieraser.ai/blog/openai-privacy-filter</a>. Disclaimer: I also build PII detection systems.
It would be nice if their examples weren’t mostly things that are easy to catch with regex, but it’s cool to see if released as an open, local model.
For my customers I use regexes to block them from potentially publishing personal emails/phone numbers to their websites but I really wouldn't mind running this in addition just for the extra peace of mind. I don't have a GPU on our server, but I hope this is light enough of a model to handle CPU only inference on less than 2k tokens at a time.
I'm surprised nobody else has commented on this. This is a very straightforward and useful thing for a small locally runnable model to do.
And also something that it’s dangerous to try to do stochastically.
For the confused: this link must have gotten revived or something, I posted this comment a few days ago. Looks like it's getting the accolades I claim it deserves now.
It was put into second-chance pool by moderators. I originally submitted this link a few days ago and today got this (semi?)automated email from HN, an excerpt below:<p><pre><code> The submission "OpenAI Privacy Filter" that you posted to Hacker News (https://news.ycombinator.com/item?id=47870901) looks good, but hasn't had much attention so far. We put it in the second-chance pool, so it will get a random placement on the front page some time in the next day or so.
This is a way of giving good HN submissions multiple chances at the front page. If you're curious, you can read about it at https://news.ycombinator.com/item?id=26998308 and other links there.</code></pre>
Same here, this is an incredibly useful thing to have in the toolkit
This is where stochastic approaches start to feel a bit uncomfortable.<p>Even small mistakes can make something dealing with sensitive data hard to trust. It seems useful as a first pass, but I’d probably still want some deterministic checks or a human in the loop to feel confident using it.
I built a community tool for exactly this, based on privacy first principals but around the what. It’s workflow based and not “put your sensitive data into ChatGPT and hope it captures the right stuff”. Mostly built for security folks but anyone can use it<p>Check it out: <a href="https://redact.cabreza.com" rel="nofollow">https://redact.cabreza.com</a>
Curious how this compares to presidio which mixes regex with a model: <a href="https://microsoft.github.io/presidio/" rel="nofollow">https://microsoft.github.io/presidio/</a>
This is exactly where stochastic approaches feel uncomfortable.<p>For anything touching security or privacy, even small inconsistencies can quickly erode trust.
> The model is available today under the Apache 2.0 license on Hugging Face (opens in a new window) and Github (opens in a new window).<p>Bringing back the Open to OpenAI..
50M effective parameters is impressively light. Is there a similarly light model on the prompt injection side? Most of the mainstream ones seem heavier
SuperagentLM made available on-edge PPI redaction models already a few years ago in sizes 20B, 3B, 200M. They still seem to be available via their legacy API - well worth checking out to compare against this one.
<a href="https://docs.superagent.sh/legacy/llms/superagent-lm-redact-20b">https://docs.superagent.sh/legacy/llms/superagent-lm-redact-...</a>
Can someone explaon how can I reconstruct the original entities back if there are, for example, more than one person names?
Where's the gguf from Unsloth and co?
[dead]
[dead]
[flagged]