> it’s safe to say that Pam Bondi’s DoJ did not put its best and brightest on this<p>Or worse. She did.
there are a few messaging conversations between FB agents early on that are kind of interesting. It would be very interesting to see them about the releases. I sometimes wonder if some was malicious compliance... ie, do a shitty job so the info get's out before it get re-redacted... we can hope...
I mean, the internet is finding all her mistakes for her. She is actually doing alright with this. Crowdsource everything, fix the mistakes. lol.
This would be funnier if it wasn’t child porn being unredacted by our government
Weren’t. Subjunctive mood.
Language is whatever people think it is, and "it wasn't" has plurality agreement which "it weren't" does not
rubn't. conjunctivitis.
If you think the child porn is the worst part of this mess, I’ve got news for you.<p>We’d all be lucky if it was <i>just</i> distributing child porn.
[flagged]
Every second of my political consciousness in the United States has been acutely tinged with the awareness that a bunch of people, across <i>most</i> of the political spectrum live in a constant state of denial. Denial of personal responsibility or culpability. Denial of cognitive dissonance. Denial of any distinct, self-informed morals. Denial of anything but a fear of others. Denial of anything that makes them fearful or uncomfortable or might invite confrontation.<p>I've known from the second I started doing debate and FX/DX in highschool, well, let's just say I never thought that the majority of the 2FA-folks would be worth a damn when tyranny really came knocking. Fear of the other as a form of manipulation, and a distraction from class consciousness, has been their literal raison d'état since decades before I was born.<p>I guess I was shocked that <i>the President being a convicted rapist and documented child predator</i> would be a bridge too far. But then we re-elected him.<p>I believe it. We voted for this. We do nothing in the face of zero actual justice. This is exactly as good as we deserve. <i>And best of all, it certainly doesn't stop here</i>. <i>This is what they chose to not redact. When we know they spent enormous tax-payer hundreds-of-people hours redacting the documents</i>.<p>I don't think it's even conspiratorial to say they left stuff in, so they could use it as justification for not releasing the other HALF of the files that haven't been released, even overly censored.<p>We deserve this, and the much worse that our apathy has invited.
>and a distraction from class consciousness<p>As a non american looking in I feel like that applies to the other side as well and is how you ended up here.<p>Having paid a bit of attention during the election seeing bernie and trump at least in terms of rethoric more in line with eachother on the same trade agreements, migration, etc whilst also both outperforming Hillary in the same swing states, etc is not some coincidence.<p>And given that you live in a 2 party state it's always going to swing at some point eventually. No matter how depraved someone like trump is.
If the next one is just as bad and they sit it out long enough they will get their turn.
I will certainly feel less confident ridiculing conspiracy theories.<p>I’d never believe Bill Gates would secretly slip antibiotics into his wife’s cocktail to treat an STI he got from a Russian prostitute on convicted pedophile estate.<p>But here we are.
I wish I could believe in more conspiracy theories. At least then I might believe there was some sort of master plan, that some individual or group had some image of a better world (to them) and that the world was being steered somewhere.<p>Unfortunately no, it just seems to be greed, incompetence, and incompetent greed. At least when a tank drives over a protestor somebody gets to be on the side of the tank. When the bus goes off a cliff because the driver sold the steering wheel everybody dies.
The owner of 4chan met with an Epstein associate 3 days before reinstating /pol/ which lead to the destruction of America.<p>Epstein was trying to remove tax on banker bonuses in the UK for some reason.<p>There might not be a single master plan but holy hell is this stuff intertwined with everything that happens.
>I wish I could believe in more conspiracy theories.<p>Username checks out... well, I can help ya.<p>You start out easy, like "who invented all those damn conspiracy theories and introduced them into the public culture, anyway?"
Epstein was involved in a UK corruption plot to reduce taxes on banker's bonuses. He was involved with insider trading around 9/11. This net is far reaching.
[dead]
Become?
> become<p>the mascot of 4chan was literally pedobear, what time frame are you referring to?
I wonder if this could be intentional. If the datasets are contaminated with CSAM, anybody with a copy is liable to be arrested for possession.<p>More likely it's just an oversight, but it could also be CYA for dragging their feet, like "you rushed us, and look at these victims you've retraumatized". There are software solutions to find nudity and they're quite effective.
the issue is that mistakes can't be fixed in the sense once they are discovered, it doesn't matter if they are eventually redacted
Let's see her sued for leaking PII. Here in Europe, she'd be mincemeat.
The US administration is, at present, regularly violating the law and ignoring court orders. Indeed, these very releases are patently in violation of multiple federal laws -- they're simultaneously insufficiently-responsive to meet the requirements of the law requiring the release of the files and fall afoul of CSAM laws by being incompletely redacted.<p>The challenge, as we're all experiencing together, is that the law is not inherently self-enforcing.
Yeah - they'll take these lessons learned for future batches of releases.
Sicko.
Nerdsnipe confirmed :)<p>Claude Opus came up with this script:<p><a href="https://pastebin.com/ntE50PkZ" rel="nofollow">https://pastebin.com/ntE50PkZ</a><p>It produces a somewhat-readable PDF (first page at least) with this text output:<p><a href="https://pastebin.com/SADsJZHd" rel="nofollow">https://pastebin.com/SADsJZHd</a><p>(I used the cleaned output at <a href="https://pastebin.com/UXRAJdKJ" rel="nofollow">https://pastebin.com/UXRAJdKJ</a> mentioned in a comment by Joe on the blog page)
> It produces a somewhat-readable PDF (first page at least) with this text output<p>Any chance you could share a screenshot / re-export it as a (normalized) PDF? I’m curious about what’s in there, but all of my readers refuse to open it.
So it was a public event attended by 450 people:<p><a href="https://www.mountsinai.org/about/newsroom/2012/dubin-breast-center-holds-inaugural-gala" rel="nofollow">https://www.mountsinai.org/about/newsroom/2012/dubin-breast-...</a><p><a href="https://www.businessinsider.com/dubin-breast-center-benefit-2012-12" rel="nofollow">https://www.businessinsider.com/dubin-breast-center-benefit-...</a><p>Even names match up, but oddly the date is different.
This is cool!
Teseract supports being trained for specific fonts, that would probably be a good starting point<p><a href="https://pretius.com/blog/ocr-tesseract-training-data" rel="nofollow">https://pretius.com/blog/ocr-tesseract-training-data</a>
It decodes to binary pdf and there are only so many valid encodings. So this is how I would solve it.<p>1. Get an open source pdf decoder<p>2. Decode bytes up to first ambiguous char<p>3. See if next bits are valid with an 1, if not it’s an l<p>4. Might need to backtrack if both 1 and l were valid<p>By being able to quickly try each char in the middle of the decoding process you cut out the start time. This makes it feasible to test all permutations automatically and linearly
You might need to backtrack a lot more, due to the intermediate compression step?
Sounds like a job for afl
This is one of those things that seems like a nerd snipe but would be more easily accomplished through brute forcing it. Just get 76 people to manually type out one page each, you'd be done before the blog post was written.
Or one person types 76 pages. This is a thing people used to do, not all that infrequently. Or maybe you have one friend who will help–cool, you just cut the time in half.
You think compelling 76 people to <i>honestly and accurately</i> transcribe files is something that's easy and quick to accomplish.
Non-engineers are perfectly willing to volunteer their time to do drudgery. It's one of my opseng career's distinguishing specialties: I'll do drudgery rather than code when appropriate, rather than avoiding it or sulking about it (as was a common response at work for some number of decades!). Learned that lesson when I was 18 from an internship (where I completely failed to deliver any work product due to trying to code around the work). It's part of why I'm going into accounting: apparently having the stamina for dreary work is <i>rare</i>?!<p>Also look up double/triple data-entry systems, where you have multiple people enter the data and then flag and resolve differences. Won't protect you from your staff banding together to fuck you over with maliciously bad data, but it's incredibly effective to ensure people were Actually Working Their Blocks under healthy circumstances.
> Just get 76 people<p>I consider myself fairly normal in this regard, but I don't have 76 friends to ask to do this, so I don't know how I'd go about doing this. Post an ad on craigslist? Fiverr? Seems like a lot to manage.
Given how much of a hot mess PDFs are in general, it seems like it would behoove the government to just develop a new, actually safe format to standardize around for government releases and make it open source.<p>Unlike every other PDF format that has been attempted, the federal government doesn't have to worry about adoption.
XPS [0] seems to meet these criteria. It supports most of the features of PDF, is an "official" standard, has decent software support (including lots of open source programs), and uses a standard file format (XML). But the tooling is quite a bit worse than it is for PDF, and the file format is still complex enough that redaction would probably be just as hard.<p>DjVu [1] would be another option. It has really good open source tooling available, but it supports substantially less features than PDF, making it not really suitable as a drop-in replacement. The format is relatively simple though, so redaction should be fairly doable.<p>TIFF [2] is already occasionally used for government documents, but it's arguably <i>more</i> complex than PDF, so probably not a good choice for this.<p>[0]: <a href="https://en.wikipedia.org/wiki/Open_XML_Paper_Specification" rel="nofollow">https://en.wikipedia.org/wiki/Open_XML_Paper_Specification</a><p>[1]: <a href="https://en.wikipedia.org/wiki/DjVu" rel="nofollow">https://en.wikipedia.org/wiki/DjVu</a><p>[2]: <a href="https://en.wikipedia.org/wiki/TIFF" rel="nofollow">https://en.wikipedia.org/wiki/TIFF</a>
You’re thinking about this as a nerd.<p>It’s not a tools problem, it’s a problem of malicious compliance and contempt for the law.
Even the previous justice departments struggled with PDFs. The way they handled it was scrubbing all possible metadata and uploading it as images.<p>For example, when the Mueller reports were released with redactions, they had no searchable text or meta data because they were worried about these exact kind of data leaks.<p>However, vast troves of unsearchable text is not a huge win for transparency.<p>PDFs are just a garbage format and even good administrations struggle.
I give any new document format 3 to 5 years until it ends up with similar mess. And that is if it starts well designed and limited.
JPEG?
You can use the justice.gov search box to find several different copies of that same email.<p>The copy linked in the post:<p><a href="https://www.justice.gov/epstein/files/DataSet%209/EFTA00400459.pdf" rel="nofollow">https://www.justice.gov/epstein/files/DataSet%209/EFTA004004...</a><p>Three more copies:<p><a href="https://www.justice.gov/epstein/files/DataSet%2010/EFTA02153691.pdf" rel="nofollow">https://www.justice.gov/epstein/files/DataSet%2010/EFTA02153...</a><p><a href="https://www.justice.gov/epstein/files/DataSet%2010/EFTA02154109.pdf" rel="nofollow">https://www.justice.gov/epstein/files/DataSet%2010/EFTA02154...</a><p><a href="https://www.justice.gov/epstein/files/DataSet%2010/EFTA02154246.pdf" rel="nofollow">https://www.justice.gov/epstein/files/DataSet%2010/EFTA02154...</a><p>Perhaps having several different versions might make it easier.
Also, I found a different base64 encoding with a different font here:<p><a href="https://www.justice.gov/epstein/files/DataSet%209/EFTA00775520.pdf" rel="nofollow">https://www.justice.gov/epstein/files/DataSet%209/EFTA007755...</a><p>This doesn't solve the "1 & l" problem for the pdf you are looking at, but it could be useful anyway.
Why not just try every permutation of (1,l)? Let’s see, 76 pages, approx 69 lines per page, say there’s one instance of [1l] per line, that’s only… uh… 2^5244 possibilities…<p>Hmm. Anyone got some spare CPU time?
It should be much easier than that. You should should be able to serially test if each edit decodes to a sane PDF structure, reducing the cost similar to how you can crack passwords when the server doesn't use a constant-time memcmp. Are PDFs typically compressed by default? If so that makes it even easier given built-in checksums. But it's just not something you can do by throwing data at existing tools. You'll need to build a testing harness with instrumentation deep in the bowels of the decoders. This kind of work is the polar opposite of what AI code generators or naive scripting can accomplish.
>It should be much easier than that. You should should be able to serially test if each edit decodes to a sane PDF structure<p>that's pointed out in the article. It's easy for plaintext sections, but not for compressed sections. Didn't notice any mention of checksums.
I wonder if you could leverage some of the fuzzing frameworks tools like Jepsen rely on. I’m sure there’s got to be one for PDF generation.
On the contrary, that kind of one-off tooling seems a great fit for AI. Just specify the desired inputs, outputs and behavior as accurately as possible.
Easy, just start a crypto currency (Epsteincoin?) based on solving these base64 scans and you'll have all the compute you could ever want just lining up
Event details: <a href="https://web.archive.org/web/20260206040716/https://what2wearwhere.com/dubin-breast-center-2nd-annual-benefit/" rel="nofollow">https://web.archive.org/web/20260206040716/https://what2wear...</a>
DUBIN BREAST CENTER
SECOND ANNUAL BENEFIT
MONDAY, DECEMBER 10, 2012
HONORING ELISA PORT, MD, FACS
AND
THE RUTTENBERG FAMILY
HOST
CYNTHIA MCFADDEN
SPECIAL MUSICAL PERFORMANCES
CAROLINE JONES, K'NAAN,
HALEY REINHART, THALIA, EMILY WARREN
MANDARIN ORIENTAL
7:00PM COCKTAILS
LOBBY LOUNGE
8:00PM DINNER AND ENTERTAINMENT
MANDARIN BALLROOM
FESTIVE ATTIRE
This proves my paranoia that you should print and rescan redactions. That or do screenshots of the pdf redacted and convert back to a pdf
pdftoppm and Ghostscript (invoked via Imagemagick) re-rasterize full pages to generate their output. That's why it was slow. Even worse with a Q16 build of Imagemagick. Better to extract the scanned page images directly with pdfimages or mutool.<p>Followup: pdfimages is 13x faster than pdftoppm
Wait would this give us the unredacted PDFs?
That's the idea yeah. There are other people actively working on this. You can follow vx-underground on twitter. They're tracking it.
From the unredacted attachments you could figure out what the redacted content most likely contains. Just like the other sloppy redactions that sometimes hide one party of the conversation, sometimes the other, so you can easily figure out the both sides.
I think it's the PDF files that were attached to the emails, since they're base64 encoded.
Bummer that it's not December - the <a href="https://www.reddit.com/r/adventofcode/" rel="nofollow">https://www.reddit.com/r/adventofcode/</a> crows would love this puzzle
I wonder if jmail (<a href="https://www.jmail.world/" rel="nofollow">https://www.jmail.world/</a>) has worked on this?<p>I tried to find the message in this blog post, but couldn't. (don't see how to search by date).
Love this, absolutely looking forward to some results.
My non political take about this gift that keeps on giving is that: PDF might seem great for the end user that is just expected to read or print the file they are given, but the technology actually sucks.<p>PDF is basically a prettify layer on top of the older PS that brings an all lot of baggage. The moment you start trying to do what should be simple stuff like editing lines, merging pages, change resolution of the images, it starts giving you a lot of headaches.<p>I used to have a few scripts around to fight some of its quirks from when I was writing my thesis and had to work daily with it. But well, it was still an improvement over Word.
It's meant as a printer replacement format, hence "print to PDF". It's a computer file format about equivalent to a printed document. Like a printed document, you can't just change its structure and recompile it.
If only Base64 had used a checksum.
This one is irresistible to play with. Indeed a nerd snipe.
I doubt the PDF would be very interesting. There are enough clues in the human-readable parts: it's an invite to a benefit event in New York (filename calls it DBC12) that's scheduled on December 10, 2012, 8pm... Good old-fashioned searching could probably uncover what DBC12 was, although maybe not, it probably wasn't a public event.<p>The recipient is also named in there...
There's potentially a lot of files attached and printed out in this fashion.<p>The search on the DOJ website (which we shouldn't trust), given the query: "Content-Type: application/pdf; name=", yields maybe a half dozen or so similarly printed BASE64 attachments.<p>There's probably lots of images as well attached in the same way (probably mostly junk). I deleted all my archived copies recently once I learned about how not-quite-redacted they were. I will leave that exercise to someone else.
There's 70 results that come out when searching for "application/pdf" on the doj website
OK, but if the solution is to brute-force them, there's probably a need to choose which files to focus on.<p>Of course there are other content-types, e.g. searching for "Content-Type: image/jpeg" gets hits as well. But only a few of them actually have the base64 data, mostly there are just the MIME headers.. Looking for "/9j/" (which is Base64 for FF D8 FF, which is the header for JPEG files), the Trumpian justice.gov website ignores "/" and shows results case-insensitively, but there are 4 or 5 base64'ed JPEG images in there.<p>I also saw that the page is vulnerable to code injection, somehow garbage in one search result preview was OCREd as "<s [lots of garbage]>", and the rest of the search results were striken-through because "<s>" is the HTML to do that.
I took at stab at training Tesseract and holy jeebus is their CLI awful. Just an insanely complicated configuration procedure.
Gods, I had a flashback just from you mentioning that.<p>I had a reasonably simple problem to solve, slightly weird font and some 10 words in English (I actually only missed one or two blocks for missing letters to cover all I needed).<p>After a couple of days having almost everything (?) I just surrendered. This seems to be intentionally hostile. All the docs scattered across several repositories, no comprehensive examples, etc.<p>Absolutely awful piece of software from this end (training the last gen).
> …but good luck getting that to work once you get to the flate-compressed sections of the PDF.<p>A dynamic programming type approach might still be helpful. One version or other of the character might produce invalid flate data while the other is valid, or might give an implausible result.
I'm only here to shout out fish shell, a shell finally designed for the modern world of the 90s
Honestly, this is something that should've been kept private, until each and every single one of the files is out in the open. Sure, mistakes are being made, but if you blast them onto the internet, they WILL eventually get fixed.<p>Cool article, however.
Won't that entire DOJ archive already be downloaded for backup by several people?
If I'd be a journalist working on those files, this is the very first thing I would do as soon as those files were published. Just to make sure you have the originals before DOJ can start adding more redactions.
Are there archives of this? I have no doubt after this post goes viral some of these files might go “missing”
Having a large number of conspiracies validated has lead me to firmly plant my aluminum hat
On one hand, the DOJ gets shit because it was taking too long to produce the documents, and then on another, they get shit because there are mistakes in the redacting because there are 3 million pages of documents.
It really doesn’t matter which foot you use to step on your own dick. This could not have been more mishandled if they gave it to an actual snake.
What they are redacting is pretty questionable though. Entire pages being suspiciously redacted with no explanation (which they are supposed to provide). This is just my opinion, but I think it's pretty hard to defend them as making an honest and best effort here. Remember they all lied about and changed their story on the Epstein "files" several times now (by all I mean Bondi, Patel, Bongino, and Trump).<p>It's really really hard to give them the benefit of the doubt at this point.
"On the one hand the chef gets shit for taking too long, and then on another for undercooked, badly plated dishes."<p>Incompetence is incompetence.
Considering the justice to document ratio that's kind of on them regardless.
It's pretty clear who they should be reacting (victims/minors) and who they shouldn't (perpetrators).<p>They wasted months erasing Trump from that instead. So it's on them.
The zeitgeist around the files started with MAGA and their QAnon conspiracy. All the right wing podcasters were pushing a narrative that Trump was secretly working to expose and takedown a global child sex trafficking ring. Well, it turns out, unsurprisingly, that Trump was implicated too and that's when they started to do a 180. You can't have your cake and eat it too.
[dead]
[flagged]
here's another few to decode,<p><a href="https://www.justice.gov/epstein/files/DataSet%2010/EFTA01804740.pdf" rel="nofollow">https://www.justice.gov/epstein/files/DataSet%2010/EFTA01804...</a><p><a href="https://www.justice.gov/epstein/files/DataSet%209/EFTA00775520.pdf" rel="nofollow">https://www.justice.gov/epstein/files/DataSet%209/EFTA007755...</a><p><a href="https://www.justice.gov/epstein/files/DataSet%209/EFTA00434905.pdf" rel="nofollow">https://www.justice.gov/epstein/files/DataSet%209/EFTA004349...</a><p>and than this one judging by the name of the file (hanna something) and content of the email:<p>"Here is my girl, sweet sparkling Hanna=E2=80=A6!
I am sure she is on Skype "<p>maybe more sinister (so be careful, i have no ideas what the laws are if you uncover you know what trump and Epstein were into)...<p><a href="https://www.justice.gov/epstein/files/DataSet%2011/EFTA02715081.pdf" rel="nofollow">https://www.justice.gov/epstein/files/DataSet%2011/EFTA02715...</a><p>[Above is probably a legit modeling CV for HANNA BOUVENG, based on, <a href="https://www.justice.gov/epstein/files/DataSet%209/EFTA01120466.pdf" rel="nofollow">https://www.justice.gov/epstein/files/DataSet%209/EFTA011204...</a>, but still creepy, and doesn't seem like there's evidence of her being a victim]
Geezus, with the short CV in your profile, you couldn't tell an LLM to decode "filename=utf-8"CV%5F%5F%5FHanna%5FTr%C3%A4ff%5F.pdf"? That's not "Bouveng".<p>Anyway searching for the email sender's name, there's a screenshot of an email of hers in English offering him a girl as an assistant who is "in top physical shape" (probably not this Hanna girl). That's fucking creepy: <a href="https://www.expressen.se/nyheter/varlden/epsteins-lofte-till-barbro-ehnbom-for-att-fa-en-kvinnlig-assistent/" rel="nofollow">https://www.expressen.se/nyheter/varlden/epsteins-lofte-till...</a>
this one has a better font, might be a simple copy&paste job
I've checked for copy and paste, there's so many character flaws, their OCR must have sucked really bad, I may try with deepseekOCR or something. I mean the database would probably more searchable if someone ran every file through a better OCR.