28 comments

  • btown9 hours ago
    IMO while the bar is high to say &quot;it&#x27;s the responsibility of the repository operator itself to guard against a certain class of attack&quot; - I think this qualifies. The same way GitHub provides Secret Scanning [0], it should alert upon spans of zero-width characters that are not used in a linguistically standard way (don&#x27;t need an LLM for this, just n-tuples).<p>Sure, third-party services like the OP can provide bots that can scan. But if you create an ecosystem in which PRs can be submitted by threat actors, part of your commitment to the community should be to provide visibility into attacks that cannot be seen by the naked eye, and make that protection the norm rather than the exception.<p>[0] <a href="https:&#x2F;&#x2F;docs.github.com&#x2F;en&#x2F;get-started&#x2F;learning-about-github&#x2F;about-github-advanced-security" rel="nofollow">https:&#x2F;&#x2F;docs.github.com&#x2F;en&#x2F;get-started&#x2F;learning-about-github...</a>
    • andrewflnr9 hours ago
      Regardless of the thorny question of whether it&#x27;s Github&#x27;s <i>responsibility</i>, it sure would be a good thing for them to do ASAP.
      • godelski7 hours ago
        Here&#x27;s the big reason GitHub <i>should</i> do it:<p><pre><code> It makes the product better </code></pre> I know people love to talk money and costs and &quot;value&quot;, but HN is a space for developers, not the business people. Our primary concern, as developers, is <i>to make the product better</i>. The business people need us to make the product better, keep the company growing, and beat out the competition. We need them to keep us from fixating on things that are useful but low priority and ensuring we keep having money. The contention between us is good, it keeps balance. It even ensures things keep getting better even if an effective monopoly forms as they still need us, the developers, to make the company continue growing (look at monopolies people aren&#x27;t angry at and how they&#x27;re different). And they need us more than we need them.<p>So I&#x27;d argue it&#x27;s the responsibility of the developers, hired by GitHub, to create this feature <i>because it makes the product better.</i> Because that&#x27;s the thing you&#x27;ve been hired for: to make the product better. Your concern isn&#x27;t about the money, your concern is about the product. That&#x27;s what you&#x27;re hired for.
        • btown5 hours ago
          I&#x27;d say that this is also true from a money-and-costs-and-value perspective. Sure, all press is good press... but any number of stakeholders would agree that &quot;we got some mindshare by proactively protecting against an emerging threat&quot; is higher-ROI press than &quot;Ars did a piece on how widespread this problem is, and we&#x27;re mentioned in the context of our interface making the attack hard to detect.&quot;<p>And when the incremental cost to build a feature is low in an age of agentic AI, there should be no barrier to a member of the technical staff (and hopefully they&#x27;re not divided into devs&#x2F;test&#x2F;PM like in decades past) putting a prototype together for this.
          • godelski4 hours ago
            I agree and think it&#x27;s extra important when you have specialized products. Experts are more sensitive to the little things.<p>Engineers and developers are especially sensitive. It&#x27;s our job to find problems and fix them. I don&#x27;t trust engineers that aren&#x27;t a bit grumpy because it usually means they don&#x27;t know what the problems are (just like when they don&#x27;t dogfood). Though I&#x27;ll also clarify that what distinguishes a grumpy engineer from your average redditer is that they have critiques rather than just complaints. Critique oriented is searching for solutions of problems, you can&#x27;t just stop at problem identification.<p><pre><code> &gt; And when the incremental cost to build a feature is low in an age of agentic AI </code></pre> I&#x27;m not sure that&#x27;s even necessary. A very quick but still helpful patch would be to display invisible characters. Just like we often do with whitespace characters. The diff can be a bit noisier and it&#x27;s the perfect place for this even if you purposefully use invisible characters in your programming environment.<p>Though we&#x27;re also talking about an organization that couldn&#x27;t merge a PR for a year that fixed a one liner. A mistake that should never have gotten through review. Seriously, who uses a while loop counter checking for equality?!? I&#x27;m still convinced they left the &quot;bug&quot; because it made them money
        • andrewflnr2 hours ago
          &gt; Your concern isn&#x27;t about the money, your concern is about the product. That&#x27;s what you&#x27;re hired for.<p>According to whom? Certainly not the people did the hiring.<p>I somewhat agree that developers should optimize for something other than pure monetary value, but it has nothing to do with the hiring relationship, just the moral duty to use what power you have to make the world better. In general, this can easily conflict with &quot;what you&#x27;re hired for.&quot;<p>In this case I think showing suspicious (or even all) invisible Unicode in PRs is even a monetarily valuable feature, so the moral angle is mostly moot. And I would put the primary moral burden primarily on the product management either way, since they&#x27;re the ones with the most power to affect the product, potentially either ordering the right thing to be done or stopping the devs when they try to do it on their own.
          • godelski2 hours ago
            <p><pre><code> &gt; According to whom? Certainly not the people did the hiring. </code></pre> Actually yes, according to them. Maybe they&#x27;ll say that you should <i>also</i> be concerned about the money but that just makes the business people redundant now doesn&#x27;t it? So is it better if I clarify and say that the product is your <i>primary</i> concern?<p>As a developer you have a <i>de facto</i> primary concern with the product. They hire you to... develop. They do not hire you to manage finances, they hire you to manage the product. Doing both is more the job of the engineering manager. But as a developer your expertize is in developing. I don&#x27;t think this is a crazy viewpoint.<p>You were hired for your technical skills, not your MBA.<p><pre><code> &gt; In this case I think showing suspicious (or even all) invisible Unicode in PRs is even a monetarily valuable feature </code></pre> I agree. Though I also think this is true for many things that improve the product.<p>Also note that I&#x27;m writing to <i>my audience</i>.<p><pre><code> &gt;&gt; but HN is a space for developers, not the business people. </code></pre> How I communicate with management is different, but I&#x27;m exhausted when talking to <i>fellow developers</i> and the first question being about monetary value. That&#x27;s not the first question in our side of things. Our first question is &quot;is this useful?&quot; or &quot;does this improve the product?&quot; If the answer is &quot;yes&quot; <i>then</i> I am &#x2F;okay&#x2F; talking about monetary value. If it&#x27;s easy to implement and helps the product, just implement it. If it requires time and the utility is valuable then yes, it helps to formulate an argument about monetary value since management doesn&#x27;t understand any other language, but <i>between developers</i> that is a rather crazy place to start out (unless the proposal is clearly extremely costly. But then say &quot;I don&#x27;t think you&#x27;d ever convince management&quot; instead of &quot;okay, but what is the &#x27;value&#x27; of that feature?&quot;). If I wanted to talk to business people I&#x27;d talk to the business people, not another developer...
            • andrewflnr2 hours ago
              They might <i>say</i> that your job is to make the product &quot;better&quot;, and they might even think they mean it, but I think in practice you&#x27;ll find that their definition of &quot;better&quot; as it relates to products is pretty closely related to money, and further that they are the authorities on what makes the product &quot;better&quot; so you should shut up and do what they say. If you want to make the product <i>actually</i> better, you&#x27;re going to have to defy them occasionally. That&#x27;s not what you were hired for, that&#x27;s just being a human with principles.
              • godelski1 hour ago
                To be frank, I tried to address your point with my comment about the audience.<p>I very much disagree that you <i>start</i> with money and work backwards to technical problems. I do not think this approach would make you efficient at solving problems nor at increasing profits for the business.<p>And I still firmly believe they need us more than we need them. At the end of the day this is why they want AI coding agents to work out but I do not think that even in the best situation we&#x27;ll end up in any different of a situation than COBOL. You can make developers more efficient, but replacing them requires an entirely different set of skills.<p>An MBA-type, with no programming background, has a better chance getting their photos taken with their iPhone in a museum than they do replacing a developer. I&#x27;m sure there will be some successful at it, but exceptions do not define the rule.
                • andrewflnr1 hour ago
                  Talking about the audience completely misses my point. I&#x27;m not saying it&#x27;s <i>good</i> to start with money and work back. I&#x27;m saying that&#x27;s what companies <i>actually do</i>, and furthermore that&#x27;s something the &quot;dev audience&quot; should understand about their employers.<p>&gt; I do not think this approach would make you efficient at solving problems nor at increasing profits for the business.<p>If optimizing for profit doesn&#x27;t result in profit, it&#x27;s not the fault of the goal. That company was just incompetent. However many companies are, in fact, moderately competent, and optimizing for profit works fine for them. It even has a pretty heavy overlap with optimizing for good products, so that&#x27;s nice.<p>It&#x27;s fine. We agree on the ideal outcome in this situation.
        • rkagerer3 hours ago
          At the end of the day it boils down to putting your users first.<p>Making the product better generally stems from acting in their interest, honing the tool you offer to provide the best possible experience, and making business decisions that respect their dignity.<p>Your comment talks a lot about product and I agree with it, I just mentioned this so we don&#x27;t lose sight of the fact this is ultimately about people.
        • tapland7 hours ago
          Tldr: Yeah it would make it better!
          • godelski6 hours ago
            I hope I left the lead as the lead.<p>But I also think we&#x27;ve had a culture shift that&#x27;s hurting our field. Where engineers are arguing about if we should implement certain features based on the monetary value (which are all fictional anyways). But that&#x27;s not our job. At best, it&#x27;s the job of the engineering manager to convince the business people that it has not only utility value, but monetary.
      • jacquesm8 hours ago
        It absolutely is. They are simply spreading malware. You can&#x27;t claim to be a &#x27;dumb pipe&#x27; when your whole reason for existence is to make something people deemed &#x27;too complex&#x27; simple enough for others to use, then you have an immediate responsibility to not only reduce complexity but to also ensure safety. Dumbing stuff down comes with a duty of care.
    • zzo38computer6 hours ago
      I think a &quot;force visible ASCII for files whose names match a specific pattern&quot; mode would be a simple thing to help. (You might be able to use the &quot;encoding&quot; command in the .gitattributes file for this, although I don&#x27;t know if this would cause errors or warnings to be reported, and it might depend on the implementation.)
    • iririririr22 minutes ago
      specially because it&#x27;s literally a problem with their code viewer (and vscode, which is also theirs).<p>i see squares on a properly configured vim on xterm.
  • ocornut9 hours ago
    It baffles me that any maintainer would merge code like the one highlighted in the issue, without knowing what it does. That’s regardless of being or not being able to see the “invisible” characters. There’s a transforming function here and an eval() call.<p>The mere fact that a software maintainer would merge code without knowing what it does says more about the terrible state of software.
    • dspillett4 hours ago
      <i>&gt; It baffles me that any maintainer would merge code like the one highlighted in the issue, without knowing what it does.</i><p>I don&#x27;t know if it is relevant in any specific case that is being discussed here, but if the exploit route is via gaining access to the accounts of previously trusted submitters (or otherwise being able to impersonate them) it could be a case of teams with a pile of PRs to review (many of which are the sloppy unverified LLM output that is causing a problem for some popular projects) lets through an update from a trusted source that has been compromised.<p>It could correctly be argued that this is a problem caused by laziness and corner cutting, but it is still understandable because projects that are essentially run by a volunteer workforce have limited time resources available.
    • pdonis5 hours ago
      Wish I could upvote this more.
    • mmlb4 hours ago
      In this instance the PR that was merged was from 6 years ago and was clear <a href="https:&#x2F;&#x2F;github.com&#x2F;pedronauck&#x2F;reworm&#x2F;pull&#x2F;28" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;pedronauck&#x2F;reworm&#x2F;pull&#x2F;28</a>. Looks to me like a force push overwrote the commit that now exists in history since it was done 6y later.
  • stainlu23 minutes ago
    A lot of this thread is debating ASCII vs Unicode in source code, but that framing is too broad. The specific technique here uses Variation Selectors (U+FE00-FE0F and U+E0100-E01EF), which have zero legitimate use in source code. Zero-width joiners and spaces have real purposes in Arabic, Indic scripts, and emoji. Variation Selectors do not -- they modify glyph presentation in fonts, not program semantics.<p>So you don&#x27;t need to ban Unicode from source to defend against this. Just flag Variation Selectors specifically. A pre-commit hook that catches [\x{FE00}-\x{FE0F}] in .js&#x2F;.ts&#x2F;.py files would have detected this entire campaign with no false positives.<p>The more interesting question is why no major CI system or editor does this by default. Someone in this thread mentioned GitHub was told about invisible character injection via their bug bounty program, paid out, and chose not to ship a fix. That&#x27;s a policy decision worth examining more than the encoding trick itself.
  • zzo38computer6 hours ago
    I use non-Unicode mode in the terminal emulator (and text editors, etc), I use a non-Unicode locale, and will always use ASCII for most kind of source code files (mainly C) (in some cases, other character sets will be used such as PC character set, but usually it will be ASCII). Doing this will mitigate many of this when maintaining your own software. I am apparently not the only one; I have seen others suggest similar things. (If you need non-ASCII text (e.g. for documentation) you might store them in separate files instead. If you only need a small number of them in a few string literals, then you might use the \x escapes; add comments if necessary to explain it.)<p>The article is about in JavaScript, although it can apply to other programming languages as well. However, even in JavaScript, you can use \u escapes in place of the non-ASCII characters. (One of my ideas in a programming language design intended to be better instead of C, is that it forces visible ASCII (and a few control characters, with some restrictions on their use), unless you specify by a directive or switch that you want to allow non-ASCII bytes.)
    • TacticalCoder4 hours ago
      &gt; ... and will always use ASCII for most kind of source code files<p>Same. And I enforce it. I&#x27;ve got scripts and hooks that enforces <i>source files</i> to only ever be a subset of ASCII (not even all ASCII codes have their place in source code).<p>Unicode chars strings are perfectly fine in resource files. You can build perfectly i18n&#x2F;l10n apps and webapps without ever using a single Unicode character in a source file. And if you really do need one, there&#x27;s indeed ASCII escaping available in many languages.<p>Some shall complan that their name as &quot;Author: ...&quot; in comments cannot be written properly in ASCII. If I wanted to be facetious I&#x27;d say that soon we&#x27;ll see:<p><pre><code> # Author: Claude Opus 27.2 </code></pre> and so the point shall be moot anyway.
    • userbinator3 hours ago
      CP437 forever!<p>The biggest use of Unicode in source repos now might be LLM slop, so I certainly don&#x27;t miss its absence at all.
  • herpdyderp9 minutes ago
    I keep seeing this and wondering if the ESLint default rules against weird characters would catch this? But I can’t figure out how to check.
  • mmsc2 hours ago
    GitHub advertises itself as warning about those Unicode characters: <a href="https:&#x2F;&#x2F;github.blog&#x2F;changelog&#x2F;2025-05-01-github-now-provides-a-warning-about-hidden-unicode-text&#x2F;" rel="nofollow">https:&#x2F;&#x2F;github.blog&#x2F;changelog&#x2F;2025-05-01-github-now-provides...</a><p>Of course, it doesn&#x27;t work though. I reported this to their bug bounty, they paid me a bounty, and told me &quot;we won&#x27;t be fixing it&quot;: <a href="https:&#x2F;&#x2F;joshua.hu&#x2F;2025-bug-bounty-stories-fail#githubs-utf-filter-warning" rel="nofollow">https:&#x2F;&#x2F;joshua.hu&#x2F;2025-bug-bounty-stories-fail#githubs-utf-f...</a><p>The exact quote is &quot;Thanks for the submission! We have reviewed your report and validated your findings. After internally assessing your report based on factors including the complexity of successfully exploiting the vulnerability, the potential data and information exposure, as well as the systems and users that would be impacted, we have determined that they do not present a significant security risk to be eligible under our rewards structure.&quot; The funny thing is, they actually gave me $500 and a lifetime GitHub Pro for the submission.
  • minus79 hours ago
    The `eval` alone should be enough of a red flag
    • jeltz8 hours ago
      Yeah, I would have loved to see an example where it was not obvious that there is an exploit. Where it would be possible for a reviewer to actually miss it.
    • godelski6 hours ago
      I&#x27;m not a JS person, but taking the line at face value shouldn&#x27;t it to nothing? Which, if I understand correctly, should never be merged. Why would you merge no-ops?
    • kordlessagain9 hours ago
      No it’s not.
      • simonreiff9 hours ago
        OWASP disagrees: See <a href="https:&#x2F;&#x2F;cheatsheetseries.owasp.org&#x2F;cheatsheets&#x2F;Nodejs_Security_Cheat_Sheet.html" rel="nofollow">https:&#x2F;&#x2F;cheatsheetseries.owasp.org&#x2F;cheatsheets&#x2F;Nodejs_Securi...</a>, listing `eval()` first in its small list of examples of &quot;JavaScript functions that are dangerous and should only be used where necessary or unavoidable&quot;. I&#x27;m unaware of any such uses, myself. I can&#x27;t think of any scenario where I couldn&#x27;t get what I wanted by using some combination of `vm`, the `Function` constructor, and a safe wrapper around `JSON.parse()` to do anything I might have considered doing unsafely with `eval()`. Yes, `eval()` is a blatant red flag and definitely should be avoided.
      • jacquesm8 hours ago
        While there are valid use cases for eval they are so rare that it should be disabled by default and strongly discouraged as a pattern. Only in very rare cases is eval the right choice and even then it will be fraught with risk.
      • godelski6 hours ago
        The parent didn&#x27;t say &quot;there&#x27;s no legitimate uses of eval&quot;, they said &quot;using eval should make people pay more attention.&quot; A red flag is a warning. An alert. Not a signal saying &quot;this is 100% no doubt malicious code.&quot;<p>Yes, it&#x27;s a red flag. Yes, there&#x27;s legitimate uses. Yes, you should always interrogate evals more closely. All these are true
      • pavel_lishin9 hours ago
        When is an eval not at least a security &quot;code smell&quot;?
      • SahAssar9 hours ago
        It really is. There are very few proper use-cases for eval.
        • nswango8 hours ago
          For a long time the standard way of loading JSON was using eval.
          • bawolff6 hours ago
            Not that long, browsers implemented JSON.parse() back in 2009. JSON was only invented back in 2001 and took a while to become popular. It was a very short window more than a decade ago when eval made sense here.<p>Eval for json also lead to other security issues like XSSI.
            • creatonez4 hours ago
              Problem is, it took until around 2016 for IE6 to be fully dead, so people continued to justify these hacks for a long time. Horrifying times.
          • _flux7 hours ago
            And why do we not anymore make use of it, but instead implemented separate JSON loading functionality in JavaScript? Can you think of any reasons beyond performance?
            • bawolff6 hours ago
              I&#x27;d be surprised if there is a performance benefit of processing json with eval(). Browsers optimize the heck out of JSON.
              • fhars4 hours ago
                You are arguing against the opposite of what the comment you answered to said.
            • bulbar7 hours ago
              Why did you opt in for such a comment while a straight forward response without belittling tone would have achieved the same?
              • _flux7 hours ago
                I actually gave it some thought. I had written the actual reason first, but I realized that the person I was responding to <i>must</i> know this, yet keeps arguing in that eval is just fine.<p>I would say they are arguing that in bad faith, so I wanted to enter a dialogue where they are either forced to agree, or more likely, not respond at all.
          • creatonez4 hours ago
            For <i>IE7</i> support, yes... <a href="https:&#x2F;&#x2F;caniuse.com&#x2F;mdn-javascript_builtins_json_parse" rel="nofollow">https:&#x2F;&#x2F;caniuse.com&#x2F;mdn-javascript_builtins_json_parse</a>
  • gnabgib10 hours ago
    Small discussion yesterday (9+9 points, 9+4 comments) <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=47374479">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=47374479</a> <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=47385244">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=47385244</a>
  • bawolff6 hours ago
    I feel like the threat of this type of thing is really overstated.<p>Sure the payload is invisible (although tbh im surprised it is. PUA characters usually show up as boxes with hexcodes for me), but the part where you put an &quot;empty&quot; string through eval isn&#x27;t.<p>If you are not reviewing your code enough to notice something as non sensical as eval() an empty string, would you really notice the non obfuscated payload either?
    • loumf2 hours ago
      The threat is that you depend on this library or use the VS Code Extension.
  • WalterBright8 hours ago
    Unicode should be for visible characters. Invisible characters are an abomination. So are ways to hide text by using Unicode so-called &quot;characters&quot; to cause the cursor to go backwards.<p>Things that vanish on a printout should not be in Unicode.<p>Remove them from Unicode.
    • pvillano8 hours ago
      Unicode is &quot;designed to support the use of text in all of the world&#x27;s writing systems that can be digitized&quot;<p>Unicode needs tab, space, form feed, and carriage return.<p>Unicode needs U+200E LEFT-TO-RIGHT MARK and U+200F RIGHT-TO-LEFT MARK to switch between left-to-right and right-to-left languages.<p>Unicode needs U+115F HANGUL CHOSEONG FILLER and U+1160 HANGUL JUNGSEONG FILLER to typeset Korean.<p>Unicode needs U+200C ZERO WIDTH NON-JOINER to encode that two characters should not be connected by a ligature.<p>Unicode needs U+200B ZERO WIDTH SPACE to indicate a word break opportunity without actually inserting a visible space.<p>Unicode needs MONGOLIAN FREE VARIATION SELECTORs to encode the traditional Mongolian alphabet.
      • WalterBright7 hours ago
        [flagged]
        • bulbar7 hours ago
          That&#x27;s a very narrow view of the world. One example: In the past I have handled bilingual english-arabic files with switches within the same line and Arabic is written from left to right.<p>There are also languages that are written from to to bottom.<p>Unicode is not exclusively for coding, to the contrary, pretty sure it&#x27;s only a small fraction of how Unicode is used.<p>&gt; Somehow people didn&#x27;t need invisible characters when printing books.<p>They didn&#x27;t need computers either so &quot;was seemingly not needed in the past&quot; is not a good argument.
          • WalterBright5 hours ago
            &gt; That&#x27;s a very narrow view of the world.<p>Yes, it is. Unicode has undergone major mission creep, thinking it is now a font language and a formatting language. Naturally, this has lead to making it a vector for malicious actors. (The direction reversing thing has been used to insert malicious text that isn&#x27;t visible to the reader.)<p>&gt; Unicode is not exclusively for coding<p>I never mentioned coding.<p>&gt; They didn&#x27;t need computers<p>Unicode is for characters, not formatting. Formatting is what HTML is for, and many other formatting standards. Neither is it for meaning.
          • pibaker5 hours ago
            &gt; That&#x27;s a very narrow view of the world.<p>But not one that would surprise anyone familiar with WalterBright&#x27;s antics on this website…
            • WalterBright4 hours ago
              At least my antics do not include insulting people.
        • jmusall6 hours ago
          The fact is that there were so many character sets in use before Unicode <i>because</i> all these things were needed or at least wanted by a lot of people. Here&#x27;s a great blog post by Nikita Prokopov about it: <a href="https:&#x2F;&#x2F;tonsky.me&#x2F;blog&#x2F;unicode&#x2F;" rel="nofollow">https:&#x2F;&#x2F;tonsky.me&#x2F;blog&#x2F;unicode&#x2F;</a>
          • WalterBright4 hours ago
            Sometimes you gotta say no. Trying to please every hare brained idea leads to madness.<p>Normalized code point sequences are another WTF feature.
            • jmusall3 hours ago
              Of course! I bet there are tons of ideas that didn&#x27;t make it into Unicode, for better of worse. Where you draw the line is kind of arbitrary. You, personally, can of course opt out of all of that by restricting yourself to ASCII only, for example. But the rest of the world will continue to use Unicode.
              • WalterBright1 hour ago
                &gt; restricting yourself to ASCII only<p>My early compilers used code pages to work with Japanese, French and German customers. The original idea of Unicode was absolutely brilliant and I was all for it. D was an early total adopter of Unicode (C and C++ followed years later). I rejected code page support for D.<p>It&#x27;s mission was to support all the letters in all the languages, which was a good straightforward mission. But then came fonts, formatting, layout, rendering, casing, sort ordering, normalization, combining, vote-for-my-letter-and-Ill-vote-for-yours, emoji, icons, semantic meanings, elvish, people who invent things and campaign to put them in so they&#x27;ll leave a mark in history, ...
        • WalterBright7 hours ago
          <p><pre><code> Look Ma xt! N ! e tee S T larip </code></pre> (No Unicode needed.)
        • chongli7 hours ago
          Unicode is for human beings, not machines.
          • WalterBright4 hours ago
            How does invisible Unicode text fit into that?
            • chongli4 hours ago
              It&#x27;s not text, it&#x27;s control characters, which have always been in character sets going back to ASCII.
              • WalterBright3 hours ago
                ASCII having a few obsolete control characters does not justify opening the floodgates.
    • ted_dunning1 hour ago
      No need to remove them. Just make them visible for applications that don&#x27;t need to render every language. Make that behavior optional as well in case you really want to name characters with Hangul or Tibetan.<p>Some middle ground so that you can use greek letters in Julia might be nice as well.<p>But I don&#x27;t see any purpose in using the Personal Use Areas (PUA) in programming.
    • luke-stanley8 hours ago
      So we need a new standard problem due to the complexity of the last standard? Isn&#x27;t unicode supposed to be a superset of ASCII, which already has control characters like new space, CR, and new lines? xD
      • WalterBright7 hours ago
        The only ones people use any more are newline and space. A tab key is fine in your editor, but it&#x27;s been more or less abandoned as a character. I haven&#x27;t used a form feed character since the 1970s.
    • tetha6 hours ago
      That ship has sailed, but I consider Unicode a good thing, yet I consider it problematic to support Unicode in every domain.<p>I should be able to use Ü as a cursed smiley in text, and many more writing systems supported by Unicode support even more funny things. That&#x27;s a good thing.<p>On the other hand, if technical and display file names (to GUI users) were separate, my need for crazy characters in file names, code bases and such are very limited. Lower ASCII for actual file names consumed by technical people is sufficient to me.
      • WalterBright4 hours ago
        &gt; That ship has sailed<p>Sure, but more crazy stuff gets added all the time.
    • WalterBright8 hours ago
      Another dum dum Unicode idea is having multiple code points with identical glyphs.<p>Rule of thumb: two Unicode sequences that look identical when printed should consist of the same code points.
      • estebank6 hours ago
        If anything, Unicode should have had more disambiguated characters. Han unification was a mistake, and lower case dotted Turkish i and upper case dotless Turkish I should exist so that toUpper and toLower didn&#x27;t <i>need</i> to know&#x2F;guess at a locale to work correctly.
        • WalterBright4 hours ago
          Characters should not have invisible semantics.
      • nswango8 hours ago
        So you think that the letters in the Greek and Cyrillic alphabets which are printed identically to the Latin A should not exist?<p>And, for example, Greek words containing this letter should be encoded with a mix of Latin and Greek characters?
        • WalterBright7 hours ago
          &gt; So you think that the letters in the Greek and Cyrillic alphabets which are printed identically to the Latin A should not exist?<p>Yes. Unicode should not be about semantic meaning, it should be about the visual. Like text in a book.<p>&gt; And, for example, Greek words containing this letter should be encoded with a mix of Latin and Greek characters?<p>Yup. Consider a printed book. How can you tell if a letter is a Greek letter or a Latin letter?<p>Those Unicode homonyms are a solution looking for a problem.
          • bawolff6 hours ago
            &gt; Yes. Unicode should not be about semantic meaning, it should be about the visual. Like text in a book.<p>Do you think 1, l and I should be encoded as the same character, or does this logic only extend to characters pesky foreigners use.
          • ted_dunning1 hour ago
            But these characters only look identical in some fonts. Are you saying that if you change font, some characters in a string should change appearance and others should not?<p>And what about the round-trip rule?<p>And ligatures? Aren&#x27;t those a semantic distinction?
            • WalterBright54 minutes ago
              &gt; But these characters only look identical in some fonts.<p>That&#x27;s a problem with the fonts.<p>&gt; And what about the round-trip rule?<p>Print Unicode on paper, then ocr it, and you&#x27;ll get different Unicode. Oh, and normalization.<p>&gt; ligatures<p>Generally an issue with rendering.<p>&gt; semantic distinction<p>Unicode isn&#x27;t about semantics (or shouldn&#x27;t be). Consider &#x27;a&#x27;. It&#x27;s used for <i>all kinds</i> of meanings.
          • Yokohiii6 hours ago
            Unicode is about semantics not appearance. If you don&#x27;t need semantics then use something different.
            • WalterBright4 hours ago
              &gt; Unicode is about semantics not appearance.<p>And that&#x27;s where it went off the rails into lala land. &#x27;a&#x27; can have all kinds of distinct meanings. How are you going to make that work? It&#x27;s hopeless.
              • Yokohiii3 hours ago
                It already works.<p>Tell me what the problem is and what your proposed solution would be.
                • WalterBright42 minutes ago
                  Infer the meaning from the context.<p><pre><code> a) it&#x27;s a bullet point b) a+b means a is a variable c) apple means a means the sound &quot;aaaah&quot; d) ape means a means the sound &quot;aye&quot; e) 0xa means a means &quot;10&quot; f) &quot;a&quot; on my test paper means I did well on it g) grade &quot;a&quot; means I bought the good bolts h) &quot;achtung&quot; means it&#x27;s a German &quot;a&quot; </code></pre> I didn&#x27;t need 8 different Unicode characters. And so on.
          • Muromec6 hours ago
            &gt;Yup. Consider a printed book. How can you tell if a letter is a Greek letter or a Latin letter?<p>I can absolutely tell Cyrillic k from the lating к and latin u from the Cyrillic и.<p>&gt;should not be about semantic meaning,<p>It&#x27;s always better to be able to preserve more information in a text and not less.
            • WalterBright4 hours ago
              &gt; I can absolutely tell Cyrillic k from the lating к and latin u from the Cyrillic и.<p>They look visually distinct to me. I don&#x27;t get your point.<p>&gt; It&#x27;s always better to be able to preserve more information in a text and not less.<p>Text should not lose information by printing it and then OCR&#x27;ing it.
        • Yokohiii6 hours ago
          What about numbers? Would they be assigned to arabic only? I guess someone will be offended by that.<p>While at it we could also unify I, | and l. It&#x27;s too confusing sometimes.
          • WalterBright4 hours ago
            &gt; While at it we could also unify I, | and l. It&#x27;s too confusing sometimes.<p>They render differently, so it&#x27;s not a problem.
            • ted_dunning1 hour ago
              They only render differently in some fonts, on some displays.
            • Yokohiii3 hours ago
              totally not true :D
      • ted_dunning1 hour ago
        One of the ground rules of Unicode is the round trip rule. You have to be able to translate to and from Unicode without loss of information.
        • WalterBright58 minutes ago
          They threw that out the window with normalization.
      • wcoenen7 hours ago
        As far as I know, glyphs are determined by the font and rendering engine. They&#x27;re not in the Unicode standard.
        • WalterBright4 hours ago
          Fraktur (font) and italic (rendering) are in the Unicode standard, although Hackernews will not render them. (I suspect that the Hackernews software filters out the nuttier Unicode stuff.)
      • jeltz8 hours ago
        I don&#x27;t think that would help much. There are also characters which are similar but not the same and I don&#x27;t think humans can spot the differences unless they are actively looking for them which most of the time people are not. If only one of two glyphs which are similar appear in the text nobody would likely notice, expectation bias will fuck you over.
        • WalterBright7 hours ago
          I wonder how anybody got by with printed books.
    • moritzruth8 hours ago
      greatidea,whoneedsspacesanyway
      • WalterBright8 hours ago
        Spaces appear on a printout.
        • ted_dunning1 hour ago
          As do tabs, ems, ens and quads.
          • WalterBright14 minutes ago
            Unicode shouldn&#x27;t be a typesetting language. The proper tool for that is Latex.
    • eviks6 hours ago
      So you&#x27;d remove space and tab from Unicode?
    • abujazar8 hours ago
      Invisible characters are there for visible characters to be printed correctly...
      • WalterBright7 hours ago
        I&#x27;ll grant that a space and a newline are necessary. The rest, nope.
        • abujazar6 hours ago
          You&#x27;re talking about a subset of ASCII then. Unicode is supposed to support different languages and advanced typography, for which those characters are necessary. You can&#x27;t write e.g. Arabic or Hebrew without those &quot;unnecessary&quot; invisible characters.
          • WalterBright4 hours ago
            Please explain why an invisible zero width &quot;character&quot; is necessary.
            • krior1 hour ago
              To mark linewrapping-breakpoints in strings.
              • WalterBright13 minutes ago
                Leave typesetting to a proper typesetting language, like Latex.
            • ted_dunning1 hour ago
              To prevent ligatures from forming when you need that.
              • WalterBright9 minutes ago
                That&#x27;s the job of a typesetting language.
            • slim1 hour ago
              if you write كلب which is an arabic word written right to left in the middle of an english sentence, you want to preserve the order of the characters in the stream for computer processing purposes. meaning the chararacter ك must come before the ل and after the e and the space with respect to the memory layout. whereas when displayed, it must be inverted to be legible. the solution is to have an invisible character that indicates a switch in text direction. if you were wondering, the situation where you want to write text in a foreign language within your text is very common outside english speaking countries.
              • WalterBright27 minutes ago
                Look I&#x27;m writing sdrawkcab (amazingly, I did it without using Unicode!). Layout is the job of your text formatting program. It&#x27;s easy to fix a text editor to support right-to-left text entry.<p>The switch in text direction has resulted in malicious code injection attacks, as the reversed text becomes invisible. I had to change my compiler to reject those Unicode characters for that reason. It can be used in other cases to have hidden, malicious text.<p>Have you checked your SQL code for invisible backwards text that injects malware?
    • uhoh-itsmaciek8 hours ago
      &gt;Remove them from Unicode.<p>Do you honestly think this is a workable solution?
      • WalterBright7 hours ago
        Yes, absolutely. See my other replies.
    • bawolff6 hours ago
      Good luck with that given there are invisible characters in ascii.<p>Also this attack doesnt seem to use invisible characters just characters that dont have an assigned meaning.
      • WalterBright25 minutes ago
        The only problematic one is CR which can be used to hide text on a glass terminal with a tty interface. I&#x27;d get rid of it if I could.
  • vitus8 hours ago
    Looks like the repo owner force-pushed a bad commit to replace an existing one. But then, why not forge it to maintain the existing timestamp + author, e.g. via `git commit --amend -C df8c18`?<p>Innocuous PR (but do note the line about &quot;pedronauck pushed a commit that referenced this pull request last week&quot;): <a href="https:&#x2F;&#x2F;github.com&#x2F;pedronauck&#x2F;reworm&#x2F;pull&#x2F;28" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;pedronauck&#x2F;reworm&#x2F;pull&#x2F;28</a><p>Original commit: <a href="https:&#x2F;&#x2F;github.com&#x2F;pedronauck&#x2F;reworm&#x2F;commit&#x2F;df8c18" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;pedronauck&#x2F;reworm&#x2F;commit&#x2F;df8c18</a><p>Amended commit: <a href="https:&#x2F;&#x2F;github.com&#x2F;pedronauck&#x2F;reworm&#x2F;commit&#x2F;d50cd8" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;pedronauck&#x2F;reworm&#x2F;commit&#x2F;d50cd8</a><p>Either way, pretty clear sign that the owner&#x27;s creds (and possibly an entire machine) are compromised.
    • chrismorgan8 hours ago
      The value of the technique, I suppose, is that it hides a large payload a bit better. The part you can see <i>stinks</i> (a bunch of magic numbers and eval), but I suppose it’s still easier to overlook than a 9000-character line of hexadecimal (if still encoded or even decoded but still encrypted) or stuff mentioning Solana and Russian timezones (I just decoded and decrypted the payload out of curiosity).<p>But really, it still has to be injected after the fact. Even the most superficial code review should catch it.
      • vitus8 hours ago
        Agreed on all those fronts. I&#x27;m just dismayed by all the comments suggesting that maintainers just merged PRs with this trojan, when the attack vector implies a more mundane form of credential compromise (and not, as the article implies, AI being used to sneak malicious changes past code review at scale).
        • jeltz7 hours ago
          Yeah, the attack vector seems to be stolen credentials. I would be much more interested in an attack which actually uses Invisible characters as the main vector.
  • tolciho8 hours ago
    Attacks employing invisible characters are not a new thing. Prior efforts here include terminal escape sequences, possibly hidden with CSS that if blindly copied and pasted would execute who knows what if the particular terminal allowed escape sequences to do too much (a common feature of featuritis) or the terminal had errors in its invisible character parsing code.<p>For data or code hiding the Acme::Bleach Perl module is an old example though by no means the oldest example of such. This is largely irrelevant given how relevant not learning from history is for most.<p>Invisible characters may also cause hard to debug issues, such as lpr(1) not working for a user, who turned out to have a control character hiding in their .cshrc. Such things as hex viewers and OCD levels of attention to detail are suggested.
  • anesxvito3 hours ago
    The scary part is how invisible this is in code review. Unicode direction overrides and zero-width characters don&#x27;t show up in most editors by default. Anyone know a solid pre-commit hook config that catches this reliably?
  • codechicago2778 hours ago
    I wonder if this could be used for prompt injection, if you copy and paste the seemingly empty string into an LLM does it understand? Maybe the affect Unicode characters aren’t tokenized.
  • DropDead10 hours ago
    Why didn&#x27;t some make av rule to find stuff like this, they are just plain text files
    • nine_k9 hours ago
      The rule must be very simple: any occurrence of `eval()` should be a BIG RED FLAG. It should be handled like a live bomb, which it is.<p>Then, any appearance of unprintable characters should also be flagged. There are rather few legitimate uses of some zero-width characters, like ZWJ in emoji composition. Ideally all such characters should be inserted as \xNNNN escape sequences, and not literal characters.<p>Simple lint rules would suffice for that, with zero AI involvement.
      • WalterBright8 hours ago
        &gt; There are rather few legitimate uses of some zero-width characters, like ZWJ in emoji composition.<p>Emojis are another abomination that should be removed from Unicode. If you want pictures, use a gif.
        • _flux7 hours ago
          Arguably them being in Unicode is an accessibility issue, unless we thought to standardize GIF names, and then that already sounds a lot like Unicode.
          • WalterBright7 hours ago
            How is it an accessibility issue? HTML allows things like little gif files. I&#x27;ve done this myself when I wrote text that contained Egyptian hieroglyphs. It works just fine!
            • _flux7 hours ago
              I mean if you don&#x27;t have sight.
              • WalterBright7 hours ago
                Then use words. Or tooltips (HTML supports that). I use tooltips on my web pages to support accessibility for screen readers. Unicode should not be attempting to badly reinvent HTML.
        • sghitbyabazooka7 hours ago
          ( ꏿ ﹏ ꏿ ; )
      • hamburglar8 hours ago
        I think there’s debate (which I don’t want to participate in) over whether or not invisible characters have their uses in Unicode. But I hope we can all agree that invisible characters have no business in code, and banishing them is reasonable.
      • trollbridge9 hours ago
        In our repos, we have some basic stuff like ruff that runs, and that includes a hard error on any Unicode characters. We mostly did this after some un-fun times when byte order marks somehow ended up in a file and it made something fail.<p>I have considered allowing a short list that does not include emojis, joining characters, and so on - basically just currency symbols, accent marks, and everything else you&#x27;d find in CP-1521 but never got around to it.
    • abound10 hours ago
      Yeah it would have been nice to end with &quot;and here&#x27;s a five-line shell script to check if your project is likely affected&quot;. But to their credit, they do have an open-source tool [1], I&#x27;m just not willing to install a big blob of JavaScript to look for vulns in my other big blobs of JavaScript<p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;AikidoSec&#x2F;safe-chain" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;AikidoSec&#x2F;safe-chain</a>
      • nine_k9 hours ago
        Something like this should work, assuming your encoding is Unicode (normally UTF-8), which grep would interpret:<p><pre><code> grep -P &#x27;[\x{200B}\x{200C}\x{200D}\x{FEFF}]&#x27; code.ts </code></pre> See <a href="https:&#x2F;&#x2F;stackoverflow.com&#x2F;q&#x2F;78129129&#x2F;223424" rel="nofollow">https:&#x2F;&#x2F;stackoverflow.com&#x2F;q&#x2F;78129129&#x2F;223424</a>
    • charcircuit5 hours ago
      Isn&#x27;t that what this article is about? Advertising an av rule in their product that catches this.
  • rvnx3 hours ago
    This shows the failure of human reviews alone, an LLM-based reviewer would have caught it. Both approaches are complementary
  • mhitza6 hours ago
    Their button animations almost &quot;crash&quot; Firefox mobile. As soon as I reach them the entire page scrolls at single digit FPS.
  • faangguyindia9 hours ago
    Back in time I was on hacking forums where lot of script kiddies used to make malicious code.<p>I am wondering how that they&#x27;ve LLM, are people using them for making new kind of malicious codes more sophisticated than before?
    • Yokohiii9 hours ago
      In this case LLMs were obviously used to dress the code up as more legitimate, adding more human or project relevant noise. It&#x27;s social engineering, but you leave the tedious bits to an LLM. The sophisticated part is the obscurity in the whole process, not the code.
  • NoMoreNicksLeft6 hours ago
    Why can&#x27;t code editors have a default-on feature where they show any invisible character (other than newlines)? I seem to remember Sublime doing this at least in some cases... the characters were rendered as a lozenge shape with the hex value of the character.<p>Is there ever a circumstance where the invisible characters are both legitimate and you as a software developer wouldn&#x27;t want to see them in the source code?
    • ted_dunning1 hour ago
      Check out emacs for options like this.<p>And, yes, there is a circumstance if you want to include Arabic or Hebrew in comments or strings. You need the zero width left-right markers to make that work.
  • hananova5 hours ago
    My hot take is that all programming languages should go back to only accepting source code saved in 7-bit ASCII. With perhaps an exception for comments.
    • krior1 hour ago
      Yeah, fuck those non-english-speaking peasants &#x2F;s.
  • chairmansteve6 hours ago
    eval() used to be evil....<p>Are people using eval() in production code?
  • like_any_other5 hours ago
    Invisible characters, lookalike characters, reversing text order attacks [1].. the only way to use unicode safely seems to be by whitelisting a small subset of it.<p>And please, everyone arguing the code snippet should never have passed review - do you honestly believe this is the <i>only</i> kind of attack that can exploit invisible characters?<p>[1] <a href="https:&#x2F;&#x2F;attack.mitre.org&#x2F;techniques&#x2F;T1036&#x2F;002&#x2F;" rel="nofollow">https:&#x2F;&#x2F;attack.mitre.org&#x2F;techniques&#x2F;T1036&#x2F;002&#x2F;</a>
  • iam_circuit1 hour ago
    [dead]
  • aneyadeng8 hours ago
    [dead]
  • robutsume9 hours ago
    [dead]
  • aplomb10268 hours ago
    [dead]
  • robutsume3 hours ago
    [dead]
  • max_7 hours ago
    I don&#x27;t have to worry about any of this.<p>My clawbot &amp; other AI agents already have this figured out.<p>&#x2F;s