Related shameless plug: I have a small tool that prepends each output line with a timestamp (can be absolute, relative, or elapsed since last line), might be useful or pair well with this tool in similar scenarios (not needed when dealing with already timestamped logs, of course).
There is also `ts` from moreutils[0]. One of a few gems there. And moreutils is (probably) already in "your" distro.
Looks like the `ets` readme has a direct comparison:
> The purpose of ets is similar to that of moreutils ts(1), but ets differentiates itself from similar offerings by running commands directly within ptys, hence solving thorny issues like pipe buffering and commands disabling color and interactive features when detecting a pipe as output. (ets does provide a reading-from-stdin mode if you insist.) ets also recognizes carriage return as a line seperator, so it doesn't choke if your command prints a progress bar. A more detailed comparison of ets and ts can be found below.
This seems great. Unfortunately, I will only recall it exists once I am into minute-unknown-but-way-too-long-for-comfort while waiting for a command to complete.
Edit: Maybe you should highlight the trailing pipe example (eg `ping localhost | grep icmp | ets`)? That is way more convenient to use than the introductory samples indicate. Being able to slap it onto the end of a pipeline I am developing feels less invasive than potentially have to quote my command for the sake of a debugging tool.
Very useful, thanks for sharing.
I love it when someone has a problem I have very often, but they have the insight to actually realize that it's a problem, and then do something about it.
Says the person who built a website listing games with fair pricing[1] and more. You’re guilty of that yourself. Much appreciated too!
Haha, thanks! Yes, sometimes I do it too, but for this spacing thing I kept hitting enter and never realized this was an actual problem that I could do something about!
Also, I have to say, getting a 3D printer has definitely done this for things around the house. Every small niggle that used to fly under the radar and tolerated now has a tiny plastic thing fixing it once and for all.
A bit fancier than my awk version
awk -W interactive 'BEGIN { t = systime(); } { u=systime(); if (u - t > 1) { printf("= %s %ds ========= ",strftime("%T"), u - t); }; print $0; t = u; }'
awk -W interactive 'BEGIN { t = systime(); } { u=systime(); if (u - t > 1) { printf("\033[2J\033[H= %s %ds ========= ",strftime("%T"), u - t); }; print $0; t = u; }'It really never ceases to amaze me how many basic affordances terminals lack, that we hack around with bash configuration or helper tools.
Of course, I know it's a genuine hard area, and not one I've put any time into fixing, but it's also so high leverage--so many developers spend a ton of time in this environment.
Very nice! I’m one of those habitual return-pressers, and this tool accomplishes the exact same thing but better.
I like to pressing Enter - like an animal -, too! :) It puts spaces exactly wheere I want them.
Yeah, if I have a stuck output (that doesn’t include a date), I’ll sometimes hit return, and add `# 12/23/24 10:37` and hit return again. That way I know when I last looked at it.
This tool might be nicer, but I’m not sure if know when I’d need to use it a priori.
At least in zsh, you can do `exec > >(spacer)` but it seems to break the tty by stripping control characters from input.
It drives me batty that terminals don’t have a trivial way to “scroll to last command”.
As an aside I have foolishly never once thought that you could pipe both stdout and stderr to another program at the same time. You learn something new each day.
I knew you could do that, but never knew you could do it with "|&" as a shorthand. TIL!
I found this comment in a bash documentation example to verify:
# To send stderr through a pipe, "|&" was added to Bash 4 as an abbreviation for "2>&1 |".
When used with Python, it's important to add "PYTHONUNBUFFERED=1" env variable for this to work not just in the terminal but also when piping to files etc.
The environment variable you're thinking of is `PYTHONUNBUFFERED`, not `UNBUFFERED` [0]. If you want a generic solution to this problem, try the `unbuffer` command [1].
For anyone who's interested, I recommend reading Julia Evan's article [2] about this problem and its solutions.
[0]: https://docs.python.org/3/using/cmdline.html#envvar-PYTHONUN...
[1]: https://manpages.ubuntu.com/manpages/noble/man1/unbuffer.1.h...
[2]: https://jvns.ca/blog/2024/11/29/why-pipes-get-stuck-bufferin...
Thanks, I forgot about the PYTHON prefix, fixed it.
On MacOS, can install via homebrew's `expect`: https://apple.stackexchange.com/a/193152/330523
Apparently I knew about this 2.5 years ago as I commented on the accepted answer back then.
The Julia article was a good read. Probably something I run into once a year, but glad to have a better understanding of it.
Julia Evan's article is a great write-up indeed, I'll submit it as a main link.
It's "Julia's article", not "The Julia article".
Update: it is was on here just a month ago actually: https://news.ycombinator.com/item?id=42275033
I like it! but it would be even better as a shell's plugin maybe?
- In Python, Sarge has code to poll stdout and stderr
- Years ago for timesheets, I created a "gap report" that finds when the gap between the last dated events exceeds a threshold.
- The vscode terminal indicates which stdout, stderr are from which process with a circle .
This says it's "Integrated > Shell Integration: Decorations Enabled" to configure the feature; https://stackoverflow.com/questions/73242939/vscode-remove-c...
- Are there other shells that indicate which stdout and stderr are from which process? What should it do about e.g. reset ANSI shell escape sequences?
pretty cool idea, wish I had thought of it.
I also pipe to `ts` and successors since that gives me a running timer of when the last thing happened. It’s in moreutils.
[deleted]