Transcript
[0:00] Music.
[0:08] Well it's that time of the week again, it's time for Chit Chat Across the Pond, this is episode number 757 for January 7th 2023 and I'm your host Alison Sheridan. This week our guest is Bart Bouchats with the first installment of Programming by Stealth for the New Year. Happy New Year Bart! Happy New,
Year Alison and Happy New Year to all the Nusilla Castaways and well done for getting the U right, because my score this week is O for infinity I think. Well I have a text expander snippet that puts it in for me and that's the only reason it's right. When I did the show on January 1st,
I had everything right, but I got the episode number wrong. So I had their year correct instead of the episode. What a shock! I got the episode number wrong. Oh dear. Oh wow.
[0:52] Yeah, no, I do all of my folders with reverse, reverse, you know, year, month, day notations. And I get as far as writing 2212 and then I look at my calendar and I go seven. Oh no, Oh no, it's next month.
Oh no, it's next year.
[1:08] We have a weather station that shows we can track things like rain and it never rains here, but for the last couple of weeks, it's been raining quite a bit here.
And I alerted Steve this week because the totals for the year were way wrong. Cause we'd had like 14 inches and all of a sudden it said like a half an inch cause it was January.
The year had rolled around. missed it that much.
[1:35] Anyway. Anyway, I'm excited about your topic today. It's not a, as you said, it's not a heavy lift, but it's something I want to be better at.
So I'm excited. Well, I've been looking for an excuse to teach this and I actually, this actually does tie into the SK PassWD port. This is actually, we're still on schedule. We're still on topic, but I've been looking for an excuse to teach this because I've been doing a lot of shell scripting.
Like it's a language from like the seventies, but I have been doing so much of it in the last year in work.
I look back through my git history with my work hat on, and in 2022 I did a little bit of fixing on some old Perl code, a little bit of fixing on some old JavaScript code,
and everything new I wrote in 2022 was either shell script or PowerShell.
Now we're not doing PowerShell because I don't have no idea how to work that into programming by stealth and I don't know if there's much of an interest.
I think Alistair might use some of it, but that's about the height of it and I don't think Alistair likes it.
I love it, but I don't think Alastair is a fan, but good old shell scripting. I've been doing so much of it lately and it is very, very simplistic and very powerful all at the same time, which is a strange combination.
[2:44] So I've gotten marginally good at writing the odd terminal command here and there and been quite proud of myself, but as soon as the next step is, and then make a shell script out of it, I ran away screaming.
So, I mean, I've done it in taming the terminal, but not.
Not really let's put it that way and in timing the terminal we only did the most simplistic thing you can do with a shell script is basically put the same terminal commands we would run on the terminal into the shell script so that it just does them in order for us it's basically a batch of terminal commands. What shell script is actually a language so if you remember back to like installment twelve or something when we first did javascript we said that every programming language has some of the same basic ingredients right you have your variables,
you have conditional statements, you have loops, they are common to all languages.
Well, the shell is a language. So you actually can do conditional statements and you can do loops. And of course you can do variables. So we're actually going to do those three things.
[3:43] Not all today, but we are going to do them all. And the reason we're doing all of this is because if you remember back to the previous installment, I described the folder structure for the project for xkpasswd's port.
And one of those folders was called build scripts. Because in this case we're going to have to actually build a project. There's going to be quite a bit to do because we need to build a documentation. That documentation is going to contain mermaid diagrams. So we actually have to call it mermaid.
Then we have to call our bundler. So there's actually quite a few steps to go from source code to fully finished product. And so we're going to have to script that for our own sanity. we're going to be using Shellscript.
So maybe we should actually learn about Shellscript instead of me saying, and here's some magic sauce. Don't look too closely and then move on. I don't like doing that.
[4:31] That's not the way. It's completely not the point of taming the terminal. Or sorry, of programming.
Or either of them. It's not the point of either of them. I mean, one of the things I find most pleasing is that people like Alistair have been programming for longer than I've probably been alive.
Says I learned something new because we don't do gloss over. We look at all of the detail.
We don't do hand waving. If I find myself waving a hand, that means I'm probably missing an entire installment.
That has happened. It's something you've caught yourself out on is wanting to hand wave and realizing it's because you didn't understand it yourself is well enough to teach. The thing that you
will do, and I really appreciate, I think you're going to do it here, is you may narrow the focus because this could be a 12 part series on shell scripting, but you're not going to do that.
You're going to narrow it to the things we're going to need to know in order to work on the port for XKPassWD.
Yes. And it's one of those like shell scripting is really one of it's not just an 80 20 thing, right?
People often say, you know, 80% of the time you only need 20% of the features.
I think with shell scripting, it's like a 95 5, you know, so we're really just going to do the basics, variables, conditionals and loops.
[5:46] Which is the real, that is the real meat of a programming language, right? Can I save stuff? Can I make some decisions? Can I do it again?
[5:53] Right? Right, right. So we've mentioned a couple of times here, taming the terminal. And it occurs to me that there's very likely people listening to programming by stealth who didn't take taming the terminal with us.
And not that they know everything that you taught in taming the terminal, but we are going to refer to a lot of different sections. going to say in chapter 15, Entain Me in the Terminal.
So to review for people, it was a podcast and tutorial series, much like what's going on in Programmed Men by Stealth.
And the entire series is available in book form, but also via the web. So at a link in the show notes, ttt.barterfisher.net, you can see all of the show notes in HTML,
and it's a beautiful, beautiful interface, thanks to Helmo.
But it also has an embeddable player for the podcast for each segment. So if you actually want to hear Bart explain things and me ask my dumb questions, you can hear that and see that there.
But you can also download the Taming the Terminal book through the Apple Books app. And it as I recall, it does have the podcast in it as well.
If you've got an internet connection, you've got to be, it's a little tricky. But I also put a link in the show notes to podfeed.com slash TTT book, where you can see all kinds of other forms.
There's EPUB that doesn't have the podcast in it. can, there's all kinds of forms there. But anyway, we're going to refer to Taming the Terminal and that's what it was and that's what it is.
[7:17] And you can go refer to it from those links. And Taming the Terminal is very, very much the father of this podcast or the parent of this podcast because the format is the same.
The topic is different, but we completely hone the idea in Taming the Terminal.
[7:33] Right, right. That was our training wheels for programming. Our training wheels.
And they stand, they're nice and self-contained. I get a lot of hits on my web server. I see them getting a lot of hits and I've had random people at computer conferences tell me about this amazing website called Taming the Terminal.
Which is maybe slow like you wouldn't believe is it? I know something about that.
[7:57] That's awesome. I'm going to give one little plug. You mentioned mermaid diagrams. I got a wild hair to really dig in and play with them myself. And you're if you listen to the no silica on the
show that's going to come out on January 8th, you'll hear me talking about and teaching mermaid diagrams from a different perspective than the way Bart taught them. Detailed getting
into the guts and glory and having a lot of fun with it. But I'm trying to teach it more from a Muggle perspective that here is how you actually could do it and what it might do for you if you chose to do it. I'm still not sure it's Muggle worthy, but I think it could be fun. I enjoy it.
I'm looking forward to it. I'm looking forward to it a lot because I know enough about Mermaid to know I want to know more, but I don't have time. So if you've done all the work. I figured out a lot of ways to make them simpler, which is cool.
And like you can actually add classes. So you can say, okay, I want this class to be green boxes with rounded corners or whatever.
Actually you can't do the corners, but anyway, one of the funny things about it is you remember how easy it is to scale the resulting graphic.
It's scaled to, or is what you put in. Minus scale two, yeah. Minus scale two.
[9:12] It's almost impossible to export a mermaid diagram from any GUI tool.
The last half of the thing is me trying to find some way to get the dang things out of these GUI tools. Because a whole lot of tools we use actually support mermaid diagrams. And they're lovely. It's much better to write them in these tools than it is at the command line.
Because you don't have to keep building it every time. You just see the changes happening real time and you get a bomb if it blows up and doesn't work correctly, for example. But you can't get them out. It's like almost impossible.
Notion has them.
There's no export.
[9:46] So they're fine for use within the app. So if the app is one that you just, the app is your end product, you're fine. Right.
But you can't use it to make it for something else. Or if you can get them out, they're wee tiny little things.
Yeah, but if you don't need to get them out, right? So if the thing you use to organize your knowledge supports mermaid diagrams, then you never need to get them out because they're already where you need them.
That's the idea.
[10:09] Yes, it will be nice. Yeah. Anyway, I thought it was funny that it's super, it's super simple from the command line and virtually impossible from the others.
All right. Shell scripting. Excellent.
OK, so the absolute, absolute, absolute basics is where we're going to start. We're going to do a hello world in Shell script because that's how you do everything.
Um, just to sort of to really nail my colors of the mast here. So we're going to use the word Shell a lot and that is a short, an abbreviated version of Command Shell, which is the full name for the thing your Terminal is.
So a Command Shell is an environment for executing Terminal commands. And your Terminal window is an interactive Command Shell.
Which means that you type and then it does. Whereas with a Shell script it's the same brain, but instead of it being an interactive conversation you just give it everything at once and it does it all sequentially and then it finishes. So that's the difference between an interactive and a non-interactive Shell.
But your terminal is a shell. So everything we did in TTT was all about working on the terminal, which is a shell. So now we're going to be running scripts in exactly the same environment.
Now we are going to do it for the purpose, so our problem to be solved is a build script for XKPassWD.
That's an open source project and we want everyone to be able to build that code. So we want anyone to be able to clone the Git repository and go npm run build and to.
[11:34] Have it work. So we can't just use any shell, we need to choose one that's very very widely supported. Which is why we're using bash. Even though it's like from the 1970s, we are using bash for this.
Now bash is a slightly extended version of the very very very first Unix shell, which was simply called SH, which is an abbreviation for shell. It was the only one in existence, So they didn't have to give it a name other than just Shell.
So the original Shell was just Shell. And Bash is a joke name that stands for Born Again Shell.
It's basically a better version of Shell. And there is a better version of the better version, which is ZSH, the Z Shell.
But the Z Shell is not everywhere yet, whereas Bash is absolutely ubiquitous.
[12:25] Now you may be about to say to me, but Bart, Apple have deprecated Bash. Which is half true.
So if you run bash as your interactive shell on the Mac, you get a warning that says, Hi, we've replaced this shell with zsh. It's a little paragraph you get.
You can try it yourself. If you open a terminal and just type bash and hit enter, you'll launch a copy of bash and it will tell you a little message of it. Oh, we've moved on to the future.
But that's for interactive shells. You can actually notice in the little message it actually uses the word interactive shell.
Under the hood, the Mac is not getting rid of bash. It hasn't gotten rid of SH.
You can run a shell script with SH and the Mac won't complain about it being deprecated because it will never will be because it's such a long running standard. So bash is completely safe to use on the Mac as a shell scripting language.
They just don't want you using it as the native shell for your terminal. They want you to use ZSH for the terminal itself, which is fine.
CSH is just bash plus plus, right? It's just a nicer bash.
[13:27] And also, if you're a Windows person, as of Windows 11, the Linux subsystem for Windows has gone general availability, which means it's not a beta feature anymore. It's not an enterprise only feature. It's now just available.
So if you are running Windows 11, you can just go to the Microsoft store and get, you know, basically bash for Windows. And it's going to be running on the Linux subsystem for Windows. So even Windows people can use Bash now. So it really is the universal shell.
Whether you're Unix, Linux, Mac or very modern Windows, you can play along with Bash.
[14:02] Therefore, it's a really good choice for an open source project.
[14:06] Which is what I'm doing here.
[14:08] So the other thing then is all shell scripts actually share a basic structure. So whether you're writing your shell script in bash or zsh or even python or one of the other programming languages, you can write shell script in pretty much any language.
So you can write shell scripts in Node.js in fact, which is something we may do later in programming by stealth.
You can write in Perl, Ruby, Python, any of these languages. But they all have the same structure, which is the very first line says, what am I going
to be run with and then the rest of the file, each line is considered a command to send to the chosen shell. So it's basically who's my command runner and then command, command,
command, command, command until you reach the end of the file.
[14:53] So that is... Okay, that makes sense. Yeah. Yeah. So that is the big picture. So for reasons that have literally been lost to history, because there is massive debate about this, the name given to that first line that tells the shell who it's going to be is the shebang line. I'd love to give you a story. There,
are lots of them. Go choose one. Link in show notes to the Wikipedia article.
The key point is it's basically the Octothorpe symbol to give it its formal name or the pound sign as you guys call it or the hash symbol as the Brits call it. It's the two vertical lines and the two horizontal lines followed by an exclamation mark followed by how to run the shell script.
So is shebang hash exclamation point?
Yes.
Okay. So the bang is assumed to be because Americans call exclamation points bangs.
[15:44] And you could say shell bang. It is a thing that has that I've heard in certain parts of the typography industry. You would talk about putting a bang on the end of the line.
[15:55] Okay. Okay. Maybe they read too many comics. Yeah. Anyway, so the shebang line is the bit that says the how. And this is another place we have to be very careful.
Because actually before we get that far so the shebang line has to be first you have to have the shebang line at the top because that's literally the instructions for what is the rest of this file mean. Right after that blank lines get ignored and if you're writing your shell script in bash.
Then any line you start with the octothorpe is a comment. And if you start in bash syntax, they call it starting a word. But basically if you have a space followed by a hash symbol, then everything else on that line is ignored.
Because that is considered to be the start of a word. And if you use a hash symbol as the start of a word, that means the rest of the line is a comment.
So you can have a... So if you start the line with the hash, it's all a comment.
The whole line's a comment. If you write a command and then a space and a hash, then the rest of it's a comment.
Exactly. And they are the only ways to do a comment. There's no, like in C you have 20 different options for comments. It's just the octopus or upon its own is your only way to have a comment.
The other thing that's useful to know because terminal commands get long.
And if you're writing a script, you don't like to scroll. It's not nice, right? You like to lay things out. So you can escape the new line character by hitting backslash enter.
[17:25] So if you have a backslash as the last character on the line, bash will ignore the new line character and treat the next line as part of the previous line.
So you can spread your terminal command over 20 lines as long as you end each of them with backslash. So I noticed that you've got two spaces before the second line. Is that just to make it look good?
And it doesn't matter? Those spaces don't hurt anything?
No. So just like, okay, so bash is your terminal. So in the terminal you can say space space space space space some command, four spaces your first argument, one space your second argument. I didn't know that.
[18:02] Huh. Bash doesn't care, it just says an amount of space is my separator. First thing is the command, everything else are arguments.
I don't care whether you use 1 space or 20.
[18:11] Okay which is great because it means we can indent our shell scripts. Is each line is a new command which is why it's so forgiving about white space because it means that when you're writing is a script you can indent things for your own sanity.
The bash doesn't have things like code blocks and stuff with curly braces absolutely not that stuff it's one command to start the statement and a second command to say i'm now finished being a statement so indentation becomes spectacularly important with bash scripting. because otherwise you will drive yourself insane.
As we shall see next time. Because we're doing conditionals in part two.
So comments, new slashes on the end of lines, so that's the important stuff.
So the next thing then, because we are working in an open source project, we have to be very careful with our shebang line.
Because just like bash has to be portable, the shebang line has to be portable. So if you're writing a script for yourself, you don't have to be careful at all.
Because your computer bash isn't going to move. If bash was, wherever it was the first time you checked, it's still going to be there next year, next week, next month, whatever right.
So you just go which bash and it'll tell you that on your computer it's in slash bin slash bash or slash user slash bin slash bash or wherever your operating system put it. And then you can just use that as your shebang line and then you're fine.
[19:29] But anyone should be able to clone the XKPathsWD repo and the script should run. So we need a shebang line that's generic.
We need a shebang line that works everywhere on everyone's computer. Unix, Mac, Linux, WSL.
So how do you do that?
Well the answer is there's a terminal command that is standard across all the Unixes and the Linuxes. It's the env command which is for interacting with your environment.
One of the things and can do is it can run a shell it can start a shell for you. Basically with some intelligence.
[20:09] So if you say and space the name of the shell you'd like and will go find it for you. And then run it for you so the answer is to give the full path to end which is standard across all unix and linux and windows wsl systems is basically the any positive operating system any,
star nexus we call them, they will always have env in the same place. So if you say your shebang line as octothorpe exclamation point slash user slash bin slash env space bash, it will work,
everywhere. Similarly, you can do use octothorpe exclamation point user bin env space python, and it will find python, space php, it will find php, space node, it will find node.js,
and then treat the rest of your file as a JavaScript file. So that env command is basically your one shebang line to rule the ball. It is immensely useful and it's portable. So if you start looking inside people's Git repositories, you're going to see that env command used all over the places that should bang line because it's portable.
[21:16] So for us, yeah, it's really useful to know that something is standardized. That's that's the part I'm just shaking my head with the basement.
The chances are there's always something standard somewhere. The problem is finding it in the CF 50 options. The other place that this is standard, how many of them are there are?
Yeah. The other place this is really useful is if you use something like shame walk to synchronize your bash stuff across say the Mac and Linux, then if you start getting to have one of these very portable options, you can have SeyMos send stuff everywhere, not just between your Macs.
So that's the reason I use it a lot actually, is for my own SeyMos stuff.
[21:54] Okay, let us do the thing that every programmer must do by some sort of rule of the universe. Let us create a hello world. So using your favourite text editor of choice,
As long as it's a plain text editor, not a word processor, but we're programming by stealth so I don't have to say that.
In your favourite text editor, make a file and name it helloworld.sh. SH is the standard extension for SH, bash and ZSH. alt.ntuse.sh
Yeah, they do. So in our file, we're just going to have the first line is going to be our shebang line. So copy paste job, user bin, env bash.
And then we're going to have a blank line because pretty much you will always see in a shell script you put a blank line after the shebang line just so that it's obvious and my script starts here.
And then we're going to use the bash equivalent of the print command, which is echo. So in bash, whenever you'd like to print something, you echo it.
So echo space and then the string in single quotes, hello world! Exclamation point. And we will look more at quoting later, but for now that should do it.
So if you then save that file and pop open your terminal in whatever folder you save that file in, we're almost ready to go.
[23:11] We just have to make our shell script executable. So we do that with the chmod command. So chmod space plus x for executable space hello world and sh, and I should have stuck a link to the relevant chapter of TTT in here, so I will do that before we publish the show notes.
[23:26] And once you've made it executable, then you can run it and you run it by simply saying.slash hello world.sh. In other words, you're saying to your shell, please run this file. And it will print out hello world. And it will have done so using bash. Yeah. So you're on a Mac. You're on.
[23:46] A modern Mac, so your terminal is zsh. So zsh has run your shell script and then it's looked at the shebang line and went ah I shall hand over to bash and then the shell script has run as bash.
[24:00] So wait, you said it runs as zsh first? Your terminal is an interactive zsh, so when you say. slash hello world at sh you've told
ZSH this is what I want to do. Then ZSH goes and finds the file, opens the file, reads the shebang line, and then hands over execution to whatever was on the shebang line. So bash Dash has run the echo command, but zsh ran the hello world.sh command.
[24:31] Okay. Because that is your shell. Yeah, just for extra information. So if you want to know all the nitty gritty detail about variables in shells in the terminal in general, Installment 12 of Taming the Terminal is very, very largely dedicated to that concept.
[24:49] We are going to use a subset of that here. Basically we're going to make variables and we're going to use them. That's all we're going to do. So in order to make a variable,
we use the wonderfully simple equals operator. And in order to use our variable, we have to always prefix its name with the dollar operator. And what confuses me 110 times a
week, slight exaggeration only, is that when you make the variable, you don't put a dollar in front of it. When you use the variable, you do. So if you put the dollar in when you're making it, you're actually making a variable that you have to access with $2.
Well $2 is better than $1. Arguably, yes. But you end up with a giant big mess.
So that's the bit I always have to remind myself. Name of variable equals value, but name of variable has no dollar.
[25:45] The other thing you have to know, which is I find very difficult because I have a lot of muscle memory, you will notice in all of Taming the Terminal, we do not cuddle our operators.
When we in JavaScript have been saying one thing plus another, we said one thing space plus space another thing, when we do assignment we say var x becomes equal to, we have a space
between the x and the equal to and then a space and then the value. That's the opposite of cuddling. In bash, you absolutely, positively, absolutely must cuddle the name, the equal sign on the value, because in bash space means the next thing's an argument.
[26:24] So it has to be, there can't be a space in there, it has to be squished together. And I am so used to always putting spaces around the equals, it has taken me a long time to get over that muscle memory, to the point that I actually have my bash editor with a different colour background.
So my brain says if the window is blue, then I am shell, therefore I shall cuddle. And if the background is black, I'm in JavaScript or anything else frankly, and therefore I shall not cuddle.
It's amazing the things you do to get it into your head. So try to help yourself remember. Yes.
Now remember that basically your shell is the same thing running your shell scripts. On the Mac it means you're using ZSH, but anything Bash can do, ZSH can do too.
[27:08] So it would not be true that if we were writing in ZSH we could test our stuff in Bash. But it is true that when your shell is ZSH you can test your bash command.
[27:18] So you could just pop open a terminal and we can test our fun with variables straight from your terminal. If you really want to be pernickety, if you just type the command bash in your zsh terminal,
then you become bash. And then you're definitely seeing exactly what would happen in your shell script.
So that's not a bad thing to do, I guess, if you want to play along. Just open bash manually.
Anyway, let's make some variables to show how all this works. So to make a variable, it's variable name, no space whatsoever, equal sign, the value. So we're going to make a variable named dessert with the value waffles.
So the command is simply dessert equals waffles.
If you typed it into your terminal. If you do that from the terminal, how long does it does dessert equal waffles?
The terminal is basically like a single shell script that's waiting for you to finish typing it. So as long as that window is open, it's effectively one giant, big, really slow shell script.
[28:16] No, that's what I thought would happen. So I have a terminal open because I tested this ahead of time and I can see where before you and I got on the phone, I typed in dessert equals waffles and then I typed in Echo dollar dessert and it wrote waffles.
I have not closed that window. I just wrote Echo dollar dessert and it did nothing.
Can I ask you a question? Did you switch from ZSH to bash between those two things? Yes, I did. There you go. So you're now in a new shell. Wait a minute. Wait a minute.
No, no. switch to bash before that.
[28:56] Oh, hang on, hang on. No, I did the first time you talked about bash. I did it and then I went back. Okay. So that erased its memory.
Yeah. Yes. I don't know how many layers deep you are, but you're probably in a shell inside a shell. Could be. Okay.
So yes. So normally if you open a terminal window, yeah. So if you open a terminal window, you're starting a shell and unless you do something to change to another shell, your
variables continue to exist until you close that shell. Now if you type bash, you've started a new shell, so everything's back to zero. But effectively the terminal is like a really slow shell script that's just waiting on you to type. With the computer sitting going do do do do do, you know, tell me the next thing. So you can test stuff in the terminal, which is great.
[29:42] Okay, so dessert equals waffles makes a variable named dessert with the value waffles. And if we want to print out that variable, or in fact if we want to use that variable in,
any terminal command, we simply use $, followed by the name of the variable in the terminal command, and it will substitute in the value of the variable. So echo will print out a string. Well, in this case, echo $desert will print the value of the variable dessert. So.
[30:08] If we do that, then we can see that our dessert is waffles. We can also use it inside longer terminal command, so if we go echo space minus n space dollar dessert pipe into wc minus c, which for people who've played along and tell me the terminal, is the word count command.
And if you say minus c, you're telling it to count characters instead of words. And that will tell us how many letters there are in dessert. And if you're wondering why there's
a minus n in front of the echo. When you did echo dollar dessert, was there a new line character put after the word waffles. In other words, did you end up on the line below waffles?
The answer is you did, right? You didn't type echo waffles and then end up still on the same line waiting for the next input. You ended up in the line below. So the echo command by default shows
a new line character onto the end so that you won't go crazy with your prompt heading further and further off to the right. Okay. Most of the time you want that new line character.
Unless you're trying to count characters or use the output for something else.
[31:14] Right, because if you use that pipe symbol you're saying take the output of the echo command then use it for something else. Well in that case it's quite often that you don't really want that weird
new line character, which is why echo takes the argument minus n to say no no no, just just print it, don't be adding any new line characters on the end, just print it. So that's why echo minus n $10 dessert is going to give just waffles, which would then send into the count command,
which tells us there are seven characters in waffles.
And if you don't believe me, take the minus N out and it'll tell you that a waffles has eight letters, which is rubbish. I didn't believe you and I did it again and it didn't work.
What?
I was in it said eight. No, I mean, I got, I got eight instead of seven.
[31:55] Yeah, three guesses who wrote the show notes without the minus n and then was like what the and then had to go figure out why.
The other thing to say is that in shell scripting, remember I said it's a very simplistic programming language right compared to JavaScript, it's an absolute toddler of a programming language.
There is one data type in shell script. Everything is a string.
[32:18] One two three. It's a string. The character one, the character two, the character three. If a terminal command chooses to interpret that string as a number, that's the terminal command doing that interpreting.
As far as the shell is concerned, they're just characters. I take characters from here, and I shove them over here. It's all just characters.
The shell is just a character shuffler. That's all it does. Interesting.
Okay. So what that means is that every single terminal command we did in the entire taming the terminal, and every single terminal command we have done in this series, so every time we've used
git every time, so the terminal command is the name of the command followed by arguments separated by spaces. Each and every one of those arguments is a string. We haven't had to surround it in quotation marks, so we don't have to say git commit minus m. Well, okay,
we'll git commit minus m. The minus m is a string. It's just the git command knows that if the string starts with a minus it should do something special. And we haven't had to So a bare piece of text is a string in bash world.
So we can have our strings as bare text.
[33:26] We can have our strings with single quotes or we can have our string with double quotes. Now there are subtle differences in those three things, which we'll get to in a moment.
But for for very basic strings, we can say echo pancakes or echo single quote pancakes, single quote or echo double quote pancakes, double quote.
And they will have double quotes. or echo double quote pancakes double quote, and they will all do exactly the same thing. They will print the word pancakes.
[33:52] Where things go the same, but things are the same. Things get different if you need a space in your string, because as far as Bash is concerned, space is a separator.
Space takes argument one and turns it into argument two, argument three. So if you want to echo I like pancakes, if you don't quote it,
then bash sees three arguments I like and pancakes it sees three strings right how do you tell it it's one string you have three choices you can back you
can escape each space with backslash space and you've probably seen this if you drag and drop a file from the finder onto the terminal it will stick in backslashes in front of every space.
[34:36] Because that is a that is an appropriate way to deal with the string on a string with spaces on the command prompt as an actual human being typing that's an awful pain in the backside back to space back to space back to space hey. So you can single quote the string with spaces and it will work perfectly or you can double quotas and it will work perfectly as long as your string doesn't have certain special characters.
Because the double quote is like a quote with superpowers that will get to in a moment so the double quote has superpowers. The only thing a single quote can't do is contain a string with a single quote.
[35:11] You can't wrap a string that contains a single quote with single quotes. Okay.
Right. So if you have a single quote, you have to use either backslashes or double quotes. You can't single quote a single quote.
So your example is going to say, I like $5 pancakes. If it said I don't like $5 pancakes, you would have to use double quotes.
Or you'd have to use escapes and escape every space and everything else and use no quotes at all.
Well, assuming I'm not an animal, right? Yeah, you'd use double quotes. If I want to use quotes, I'd have to use double quotes. You would, yes. Okay.
So the shell coding is a bit of a black art. It's a little, it gets a bit squirrely sometimes, but 99.99% of the time it doesn't.
It's just when it goes a bit weird, unless you really understand what's going on, the is very weird. Okay so first option escape everything second option is use single quotes and they have a name they're called uninterpreted strings and,
the double quotes are called interpolated strings and that word interpolated that's that's the magic power we're going to describe shortly but that's a different I don't like them using that word for this that does have anything to do with interpolation.
[36:28] I must go look up the dictionary to see if I agree with you or not. But anyway, we'll pause that discussion for a moment. Okay. Okay. So inside an uninterpreted string, you can have any character you like other than a single quote.
It just cannot deal with a single quote, but you can have spaces, dollar signs, whatever you like. You can have them all in there in your single quoted string.
Just not a single quote.
Your interpolated string, you have to escape quotation marks. So a double quotation mark you can have inside your double quotation marks. where you just put a backslash in front of it.
And you have to backslash your dollar signs because the dollar sign triggers interpolation. And interpolation basically means go fetch a value and shove it in here. And if you want an actual dollar sign, then you have to go backslash dollar.
And then you get an actual dollar sign. So yet again, we have three commands that do the same thing. So you can say echo space, I backslash space, like backslash space, backslash dollar, five back to space pancakes.
[37:30] Or you can say echo space single quote. I like dollar five pancakes. Single quote. Yay.
Or you can do it echoes with double quotes, but you have to backslash that dollar. So it has to be, I like backslash dollar five.
[37:45] So a quick note on the show notes. You say does not work causes an error on what you just described. No, that's... you're one step ahead of me.
[37:56] You're one code snippet too far, you're about to come to the bit... Sorry.
Sorry. I'm now about to throw in another thing that makes shell scripting a little bit confusing. Inside your interpolated string, the exclamation point also has superpowers, and you can't backslash away out of those superpowers.
You can't say backslash exclamation point and have it not get cranky with you.
If your string ends with an exclamation point, the double quote will get cranky with you because it thinks it's a bash event.
No, it's just me being excited about pancakes, but try it. Shove it in the terminal. It will not. It'll tell you there's an error. No such event.
Okay, so Bart is written, I like a backslash dollar five pancakes exclamation point inside double quotes.
And that's what won't work. That's what will not work. So you can have your single, you can have your exclamation point in your bare string.
You can have your exclamation point in your single quotes, you can't have it in your double quotes.
So how do you get around that?
[38:54] Well, Bash lets you concatenate strings in the most simple way possible. You just cuddle them.
You put the strings back to back and Bash will join them together for you. So you can have double quote a bunch of stuff, close the double quote, and then have some bare text.
And that will just get shoved onto the end of the string.
Or you can start some single quotes.
That's a whole new string. If you put the exclamation point outside of the quotes, it'll work. The double quotes. Okay. Yeah. Or you can have it outside the double quotes, inside single quotes.
Bash sees those two as being exactly the same thing. I have one string. You have given me another string. All right then. Stick them together. Shove them all together. Okay.
Shove them all together. There's no space in that. There's no space. If you cuddle two strings, they just, they just join. They just, if they touch, they're like magnets. If they touch off
each other, they become one string. It's as simple as that. And I don't know of any other language that makes it so easy to join strings. It's so easy that if you Google it, you'll find no one actually telling you it's that easy because it's so easy. It's actually really hard to find the answer to how do I concatenate two strings.
[39:59] Okay, so now let's have a look at this interpolation magic I have mentioned a few times already. So inside your double-quoted string is where interpolation happens,
and the $ operator allows you to inject a value into the string. So it's a way of having, say the value of a variable or the result of a command to be a part of a string.
And so there are in fact two things you can do. So if you say $openCurlyBrace the name of a variable closeCurlyBrace then the value of the variable is what gets put in that part of the string.
If you do $openAroundBracket you can put any terminal command you like in there and then closed around bracket and then the result of running the terminal command is what goes into the string.
[40:51] So dollar roundy bracket three plus five close roundy bracket. The variable has a value of five.
No, because three plus five is not a terminal command.
Or two plus two. I thought I said. Oh sorry, even three plus two is in the terminal command because three is in the terminal. The first thing is the command. The second things are the arguments. So you are saying find me a command three and then give it the two arguments plus and two, which is nonsense.
So you can't add the bash can't do math. Remember everything's a string. There are terminal commands to do math. Give me an example of what would be in these cases. PWD. My working
directory is dollar round bracket PWD close round bracket. That will then tell you the the present working directory.
[41:42] In that part of the string. So if you say echo open double quote, my working directory is dollar open roundy bracket, PWD close roundy bracket. When you run that command, it will tell you my current working directory is slash home slash Allison slash whatever.
Okay. And squirrely brackets after the dollar symbol put in the value of a variable. Yes. So take whatever is inside these squirrely brackets, treat it as a variable name and go find the value. If that variable doesn't exist, blankness will ensue. If you put in dessert.
If I put in b equals one and then I said dollar squirrely bracket b squirrely bracket I would get one. Correct. Yeah, okay. Yeah. That's it exactly. So that is what interpolation does. Either get me the variables value or run this command to get me that value.
There is also a shortcut you can take.
The squirrely brackets are optional, assuming it's not ambiguous.
So if you try to include a variable with something cuddled to it, how would Bash possibly know where the name of the variable ends and the rest of your string continues?
It wouldn't know, so it would just assume that all of it is a string name.
[42:56] So if you say echo open quote, my name is space dollar name close quote, no ambiguity, right? Dollar name goes to the end of the string. There's no problem there.
But if you say my dollar, I don't know, my dollar X ism and you want the ism to be a word, how would you know that X is actually the end of the variable and ism has come after it?
OK. So that's when you have to use curly brackets. If there is ambiguity, if you can't tell by looking at it, well, what's the end of the variable name?
Then you have to help bash by putting in the curly brackets because bash will make it the longest possible valid variable name.
Bash will be greedy and it will take any characters you keep giving it as part of the variable name, which is probably, which is possibly not what you want.
So if there's no ambiguity, So I'm guessing we should just always put it in squirrely brackets and that way we don't have to worry about it?
If the very, usually what happens is when you're injecting something into a string, you're injecting it as a word, because it's going to be part of a sentence or something, or you're injecting it at the end of a line, or you're injecting it inside brackets or something.
My favorite, you know, there are this, this is a list, bracket length five, close bracket, right? The chances are that the thing to come after the value is usually some sort of not a normal character. So 99 times out of 100, you can get away without the curlies.
[44:22] Actually in real life you only every now and then have to remember deep in the back of your mind, how do I stop bash from being greedy? Oh yeah I used a curly. So in reality you you actually never use the Curly's.
[44:35] Huh, okay. Or nearly never, nearly never. And the annoying thing is that one percent of the time.
[44:43] Or you're going to have to use them. But if you're looking at other people's shell script, you're not going to see curleys. Unless there's a good reason.
So the cool kids don't use them. Got it. Yeah.
So the following again is the same thing. So you can say dessert becomes equal to waffles.
And then you can say echo double quote, I space like space dollar open curly dessert close curly, end your string and then a period and an exclamation point concatenated to it on the end or exactly the same thing without the curlies. They will work the same because there's no ambiguity.
[45:17] Okay. So if you look on online, you'll see just bare dollar dessert or whatever in people's shell scripts.
[45:23] Okay, so now let's look at how to use the command injection. So we're going to build this up slowly. So first we're going to say dessert becomes equal to pancakes, just for
fun. And then what I want is I want a terminal command to tell me how many letters there are in the current value of the variable dessert. So we already know from the example above that we can nearly get what we want with echo space minus n space dollar dessert pipe wc,
minus C. And if you run that, it will tell you that there are eight, I think, one, two, three, four, yeah, that there are eight characters and pancakes. But if you run the command, you might notice that it's space, space, space, space, space, eight.
[46:10] The WC command prefixes its output with a whole bunch of spaces when you're doing a word count, sorry, when you're doing character count. This is apparently to give compatibility with,
If you use the WC command with no arguments it gives you a line count, a word count and a character count and they're all spaced out.
And if you run it with just a minus c, it just leaves blanks.
[46:34] Okay. So arguably it's useful for people who do, you know, the old fashioned way of laying out data by counting the characters instead of using a delimiter, which I hate, hate, hate, but Excel still defaults to. The WC command behaves in that very old fashioned way.
[46:50] So even when you're using it without the dash C, when it does all three counts for you, it puts those blank spaces. So it says 0 1 8, but there's a bunch of spaces in between each one.
Yeah, exactly. It's basically it's tab. Yes, the old tab for not even tab, tabular format. Yeah. So how do we get rid of that?
Well, there is a terminal command which is designed to basically, it turns a string into arguments. And remember I said that when arguments spaces are ignored, like at the front of a line, spaces get ignored.
So the xr command will do that for us.
So if we take the output of WC- I've lost track of why we care about these spaces at the beginning of the, before the eight.
Okay. Imagine we're trying to include this value in a sentence. Your favorite dessert is dollar dessert. It has blah letters.
[47:46] If we don't strip those spaces out when we run our next command, it's going to have, it's going to say your favorite dessert is pancakes, comma, it has gap, gap, gap, gap, gap, eight letters.
Okay. Okay. It's going to look really strange. The string value that it comes up with.
Yeah, the string value is going to have all these spaces and we don't want them. So we want to get rid of the spaces. So the XR command will just magic away the spaces front and rear. There are no spaces here, but if there were they'd be magic the way too.
So where do you put this xargs command?
We pipe the output of WC minus C to XARG.
[48:20] So we're just changing our commands together. So we echo the dessert variable to the word count command, and then we pipe that to Xargs to strip away all the spaces and stuff. So Xargs gets rid of just spaces?
[48:34] It can be used for 20 million more things, but one of the things it does is it strips the leading and trailing spaces. And that's what it does when you don't tell it what else to do. Xargs is a really difficult command to describe because it's a real dog's body of a command. And sort of does lots of cool things.
But one of its magic functions is just stripping away these leading and trailing spaces.
[48:56] Which is what we want. Therefore, here it is. So we now have a command that will tell us how many letters there are in whatever the value of the variable dessert is.
So we could make dessert become equal to waffles again and then run the same command and it would give us the appropriate count. Whatever waffles is seven, I think we said. So now pancakes was eight, waffles was seven. Pancakes was eight, sorry.
Yeah. So we can use this command inside an interpolated string. So we can say echo open, open a double quotation mark. So interpolated string.
Your favorite dessert is dollar dessert comma. It has dollar open roundy bracket, echo minus n dessert, pipe wc minus c, pipe x args, close the roundy bracket, space letters.
In other words, the dollar dessert gets replaced with pancakes and the dollar open roundy bracket all that stuff gets replaced with the output of that command, which is just eight.
So what you get is your favorite dessert is pancakes. It has eight letters. So I wanted to test the XRs that you said it just gets rid of the leading and trailing.
So I made my variable dessert equals quote pancakes and waffles unquote quote and then no it works.
[50:13] And so I ran the word count on it with Xargs, it said it had 20 characters, and then I stuffed, I ran the echo command on dollar dessert in that sentence that you just came out and it says your favorite dessert is pancakes and waffles and it has 20 letters.
Oh good, excellent. It does work, yeah. Where Xargs won't do what we wanted to do is if the number had spaces in it.
[50:36] Because then it would be two args.
[50:39] Oh, points. Oh. trying to clean up one arg, which is what anyway that's like I say xargs does many things. It's a real Swiss army knife of a command. So today we have learned how to make a script go. So we
we figured out we are going to be using bash as our scripting language for xkpass wd because it's so portable. We're going to be using the n version of the shebang line because it's so portable.
Variable, and we know how to make variables, we know that every variable is a string, we know how to access our variables with the $ command, and we know how to use our variables inside strings by having interpolation.
[51:19] So we are now as far as we are concerned for take for programming by stealth we know everything we need to about variables in bash. So the next thing on the agenda is conditionals and loops and the other thing we need to learn is how to do input and output to the shell script.
[51:41] So the shell script, think of a shell script as being like a black box. You put something in, it does something and it gives you something back.
How do we put stuff in and what does it give back? How do we control what it gives back and how do we use what it gives back? What good is a magic black box that does things if you can't really interact with it?
[52:06] Because if you can't interact it will do the same thing every time, which is useful some of the time. But most of the time you actually want to give it an extra piece of information.
You want to give it some sort of an argument or some sort of an input and you want it to tell you something, not just do something. So therefore you need some kind of output
from it. So that is, that is the other piece of the puzzle we need. So basically to use fancy pants words, control of flow, which is conditionals and loops and IO input and output. They are the two big pieces we're missing here. And so that's the next thing on our agenda. Okay.
[52:40] Well, speaking of our agenda, I am jaunting off to Antarctica for a couple of weeks. So there, if you're listening to this evergreen or you've gotten behind and you're just catching up, you don't care about this.
But if you're hanging on every word, waiting for programming myself to come out, I believe if I can work it out, we, the soonest we might end up doing it is February 4th.
[53:06] That's the first time I'll be back in country. Yeah, because I'm hosting the Nisilla cast in the 22nd. Alistair is doing it in the 29th.
Send in content if you're not listening to this in the past or the future. No, the future. Can't be listening in the past. You must be listening in the future. If you're listening to this in the future, ignore what I just said. But if you're listening to this in the present, send me an Alistair content. We'd really appreciate it.
Yeah. And then so in, goodness me, in a month, in a month we'll be back to tell you more. Alright, I'll just be sitting here thinking about penguins while you talk about shell scripting, but that sounds fun to me.
Well, look, have fun then in Antarctica. That sounds like such an adventure. I don't know how you can beat this in terms of a gentriousness. I thought the Galapagos was about as adventurous as it got, but no, the South Pole.
Well, we've just been to the Arctic, so to the Arctic Circle. So we will be in, we won't quite make it to the Antarctic Circle, but we'll be very, very close to the South Pole. Wow. So is there a continent you won't have set foot on?
[54:07] Because you've been in Australia, you've been in Asia. We have not been to Africa yet. It is on the agenda. Ah well that's clearly 2024's job. Yeah actually have we been to Asia?
[54:18] Yeah you've been to India. Ah yes yes I knew that. Quite the adventure you had on that river. I have to tell you world travel and drawing the diagrams like what I'm doing is a great way to to learn geography.
So I was making a diagram that included our trip to Antarctica and also Texas. And as a joke, I made Texas bigger than Antarctica because everything's bigger in Texas. Anyway, my daughter, Lindsay, said, I like how you made Antarctica smaller than Texas.
And that got me to wondering which one was actually bigger.
I looked it up and according to NASA, the entire contiguous continental United States fits inside Antarctica.
And I did not know that.
Because you never look at the globe from the bottom, right? We'd use the rectilinear globes and we smash it out and it's a little sliver down at the bottom. It's huge.
[55:07] I would have thought it was tiny because of the Mercator projector. And interesting. Yeah, no, it's, it's quite big. But Greenland is tiny. Fun facts to know and tell.
Oh yeah. It's a long way away from me. That's, that's for sure. I also learned that Siri doesn't know what time it is in Buenos Aires.
She, her answer is, and I quote, I don't know.
I mean, with a shrug of her shoulders, it's very snarky. It's like, I don't know.
That's bizarre. Like I had Steve ask too in case I was mispronouncing it or something. That's bizarre. Anyway, nothing to do with programming myself.
No, I'm going to give you one anecdote before we finish up. I think I know geography. I told myself I knew geography.
I was visiting with my brother and as an educational toy they have a map of Europe with each country as a puzzle piece and no names, no labels, just the shapes.
And I may have knocked it off the couch. And then I was like, don't worry, I'll rebuild it.
No, I do not know my geography. If you give me the names, I'll put them in the right place, but just the shapes.
Oh, it took three of us, three grown adults. It took three of us 20 minutes to put Europe back together again.
[56:17] I remember when we had our friend, Steve Mallard over, he was from England and we had him try to draw the United States.
And he got most of it right. That's impressive.
He asked us on a blank map of the British Isles to point to Wales and we couldn't. Okay, I get that much right, but I'm not sure how good a job I'd do at drawing the border between Wales and England.
Yeah, well now that we've been to Wales we know where it is pretty much. It's a sticky outy bit at the bottom. Anyway, anyway, anyway. Fun cryptography.
Yes, and in the meantime, while Alison's off enjoying her adventure, lots and lots of happy computing.
If you learn as much from Bart each week as I do, I'd like you to go over to letstalk.ie and press one of the buttons over there to help support him.
He does 98% of the work here. I'm just the stooge that listens to him and asks the dumb questions.
If you go over to letstalk.ie, you can support him on Patreon, you can donate via PayPal, or you can use one of his referral links. I really hope you'll go over and help him out.
In the meantime, you can contact me at podfeed.
[57:29] Or check out all of the shows we do over there over at podfeed.com Thanks for listening.
[57:33] Music.