I find it easier to understand in terms of the Unix syscall API. `2>&1` literally translates as `dup2(1, 2)`, and indeed that's exactly how it works. In the classic unix shells that's all that happens; in more modern shells there may be some additional internal bookkeeping to remember state. Understanding it as dup2 means it's easier to understand how successive redirections work, though you also have to know that redirection operators are executed left-to-right, and traditionally each operator was executed immediately as it was parsed, left-to-right. The pipe operator works similarly, though it's a combination of fork and dup'ing, with the command being forked off from the shell as a child before processing the remainder of the line.<p>Though, understanding it this way makes the direction of the angled bracket a little odd; at least for me it's more natural to understand dup2(2, 1) as 2<1, as in make fd 2 a duplicate of fd 1, but in terms of abstract I/O semantics that would be misleading.
Yep, there's a strong unifying feel between the Unix api, C, the shell, and also say Perl.<p>Which is lost when using more modern or languages foreign to Unix.
Haha, I'm even more confused now. I have no idea what dup is...
There are a couple of ways to figure out.<p>open a terminal (OSX/Linux) and type:<p><pre><code> man dup
</code></pre>
open a browser window and search for:<p><pre><code> man dup
</code></pre>
Both will bring up the man page for the function call.<p>To get recursive, you can try:<p><pre><code> man man unix
</code></pre>
(the unix is important, otherwise it gives you manly men)
It's a reminder of how archaic the systems we use are.<p>File descriptors are like handing pointers to the users of your software. At least allow us to use names instead of numbers.<p>And sh/bash's syntax is so weird because the programmer at the time thought it was convenient to do it like that. Nobody ever asked a user.
At the time, the users <i>were</i> the programmers.
This is misleading because you use plural for both and I'm sure most of these UX missteps were _each_ made by a _single_ person, and there were >1 users even at the time.
> and there were >1 users even at the time.<p>Are you sure there wasn't >&1 users... Sorry I'll get my coat.
I think he meant that at that time all users were programmers. Yes, _all_ .
programmers are people too! bash syntax just sucks
arguably if you're using the CLI they still are
The conveniences also mean that there is more than ~one~ ~two~ several ways to do something.<p>Which means that reading someone else's shell script (or awk, or perl, or regex) is INCREDIBLY inconvenient.
> bash's syntax is so weird<p>What should be the syntax according to contemporary IT people? JSON? YAML? Or just LLM prompt?
Nushell! Or powershell, but I much prefer nushell!
Honestly, Python with the "sh" module is a lot more sane.
It means redirect file descriptor 2 to the same destination as file descriptor 1.<p>Which actually means that an undelrying <i>dup2</i> operation happens in this direction:<p><pre><code> 2 <- 1 // dup2(2, 1)
</code></pre>
The file description at [1] is duplicated into [2], thereby [2] points to the same object. Anything written to stderr goes to the same device that stdout is sending to.<p>The notation follows I/O redirections: cmd > file actually means that a descriptor [n] is first created for the open file, and then that descriptor's decription is duplicated into [1]:<p><pre><code> n <- open("file", O_RDONLY)
1 <- n</code></pre>
Not sure why this link and/or question is here, except to say LLMs like this incantation.<p>It redirects STDERR (2) to where STDOUT is piped already (&1). Good for dealing with random CLI tools if you're not a human.
Humans used this combination extensively for decades too. I'm no aware of any other simple way to grep both stdout and stderr from a process. (grep, or save to file, or pipe in any other way).
I've also found llms seem to love it when calling out to tools, I suppose for them having stderr interspersed messaged in their input doesn't make much difference
I found the explanation useful, about "why" it is that way. I didn't realize the & before the 1 means to tell it is the filedescriptor 1 and not a file named 1.
I get the ocassional file named `1` lying around.
It's an operator called ">&", the 1 is the parameter.
Well sure, but surely this takes some inspiration from both `&` as the "address of" operator in C as well as the `>` operator which (apart from being the greater-than operator) very much implies "into" in many circumstances.<p>So `>&1` is "into the file descriptor pointed to by 1", and at the time any reasonable programmer would have known that fd 1 == STDOUT.
> I am thinking that they are using & like it is used in c style programming languages. As a pointer address-of operator. [...] 2>&1 would represent 'direct file 2 to the address of file 1'.<p>I had never made the connection of the & symbol in this context. I think I never really understood the operation before, treating it just as a magic incantation but reading this just made it click for me.
No, the shell author needed <i>some</i> way to distinguish file descriptor 1 from a file named "1" (note that 2>1 means to write stderr to the file named "1"), and '&' was one of the few available characters. It's not the address of anything.<p>To be consistent, it would be &2>&1, but that makes it more verbose than necessary and actually means something else -- the first & means that the command before it runs asynchronously.
Better: <i>Understanding Linux's File Descriptors: A Deep Dive Into '2>&1' and Redirection</i> <a href="https://news.ycombinator.com/item?id=41384919">https://news.ycombinator.com/item?id=41384919</a> <a href="https://news.ycombinator.com/item?id=39095755">https://news.ycombinator.com/item?id=39095755</a>
Redirects are fun but there are way more than I actually routinely use. One thing I do is the file redirects.<p><pre><code> diff <(seq 1 20) <(seq 1 10)
</code></pre>
I do that with diff <(xxd -r file.bin) <(xxd -r otherfile.bin) sometimes when I should expect things to line up and want to see where things break.
I've almost never needed any of these, but there's all sorts of weird redirections you can do in GNU Bash: <a href="https://www.gnu.org/software/bash/manual/bash.html#Redirecting-Output" rel="nofollow">https://www.gnu.org/software/bash/manual/bash.html#Redirecti...</a>
If you need to know what 2>&1 means, then I would recommend shellcheck<p>It's very, very easy to get shell scripts wrong; for instance the location of the file redirect operator in a pipeline is easy to get wrong.
As someone who use LLMs to generate, among others, Bash script I recommend shellcheck too. Shellcheck catches <i>lots</i> of things and shall really make your Bash scripts better. And if for whatever reason there's an idiom you use all the time that shellcheck doesn't like, you can simply configure shellcheck to ignore that one.
I enjoyed the commenter asking “Why did they pick such arcane stuff as this?” - I don’t think I touch more arcane stuff than shell, so asking why shell used something that is arcane relative to itself is to me arcane squared.
I saw this newer bash syntax for redirecting all output some years ago on irc<p><pre><code> foo &> file
foo |& program</code></pre>
I always wondered if there ever was a standard stream for stdlog which seems useful, and comes up in various places but usually just as an alias to stderr
back when stackoverflow was still good and useful, I asked about some stderr manipulation[0] and learnt a lot from the replies<p>[0] <a href="https://stackoverflow.com/questions/3618078/pipe-only-stderr-through-a-filter" rel="nofollow">https://stackoverflow.com/questions/3618078/pipe-only-stderr...</a>
A gotcha for me originally and perhaps others is that while using ordering like<p><pre><code> $ ./outerr >blah 2>&1
</code></pre>
sends stdout and stderr to blah, imitating the order with pipe instead does not.<p><pre><code> $ ./outerr | 2>&1 cat >blah
err
</code></pre>
This is because | is not a mere redirector but a statement terminator.<p><pre><code> (where outerr is the following...)
echo out
echo err >&2</code></pre>
Useless use of cat error/award<p>But also | isnt a redirection, it takes stdout and pipes it to another program.<p>So, if you want stderr to go to stdout, so you can pipe it, you need to do it in order.<p>bob 2>&1 | prog<p>You usually dont want to do this though.
Why would that second one be expected to work?