Transcript
[0:00] Music.
[0:08] Well, it's that time of the week again. It's time for Chitchat Across the Pond. This is episode number 763 for March 25th, 2023, and I'm your host, Alison Sheridan. This week, our guest is Bart Bouchats with Programming by Stealth number 148. And I had a lot of fun this last week, Bart.
[0:24] Yeah, and this time I can say last week without having to correct myself to last time 50 times in an episode. I don't know if the listeners have noticed how often I do that. How long we've been doing bi-weekly a half a decade? At least, yeah. A decade? Actually, since I've been doing Let's Talk Apple, oh my god, that is... Yeah, that's... Oh my god. Yeah, and I still get it wrong every time, so there you go. Anyway. We actually did it back to back here. But, before you can start, I want to tell the listeners about something, why I had so much fun this week.
The challenges for 1.46 and now 1.47 have been really fun in shell scripts.
And so Ian Lessing in our Slack at podfu.com slash slack in our PBS channel for programming by stealth posted a link to his solutions in GitHub where he was working on it.
It wasn't completed code. He was just working on it and he posted it in there.
And then I thought, well, wow, that took a lot of courage. I don't want to show it.
Well, mine's stupid. I'm not done yet, it looks really dumb, and I don't want to do that, but hey, if Ian's got the courage, I'm going to do it." So I posted mine as a response to his.
Well then Ben Rose came in and he goes, well, hey, why don't we have a place where we can share our homework?
[1:42] And I thought, well, that sounds like a good idea. But now Bart had tried to do that with the PBS Gallery a while ago, and unfortunately it was just a lot of work to maintain.
So we went on a hunt for... Yeah, and it was also a slightly different problem to be solved, because that was about showing off your working web app, whereas this is about actually doing the building, right?
So you're talking about a place to build together, not a gallery to show off the end result.
So it's actually a different problem. Right.
[2:10] It might eventually be the end result, but it's a work in progress is what we wanted to do.
And so we could learn off of each other, basically cheat off each other's paper is what we're looking for.
So I went poking around, and I don't remember if somebody told me about it or whether I found it, But there's a thing called a GitHub organization.
Now, Bart created an organization when we were working on the book for Taming the Terminal and doing some automation in there with Helma.
And so I knew the concept.
But I started digging around into it, and it turns out it is the perfect solution as a place for us to share code and be able to see everybody else's code.
So, um...
The way it works is there's one repo inside this organization, and it's called pbs-challenges, and the organization is called pbs-students, and you have to be invited to be one of the students, and you have to be invited by me.
Now, the way you get invited by me is you say, hey, Allison, can I be invited?
And then you get invited. That's all you have to do. But you do have to have a GitHub account, so you have to give me your handle, otherwise I can't invite you within GitHub.
And so, a working knowledge of Git and GitHub is, I would think, requirement for this.
I wonder where they could find that.
Yeah, I do wonder that. I will get back to that in a moment.
[3:29] So anyway, once you go in there, then when I invite you, all I have to do is create a folder for you within the repo.
So there's a folder for Bart, there's a folder for me, a folder for Ian, one for Ben, one for Dorothy, and one for Helma so far, I think.
And so we've got all these different folders.
In those folders is where you put folders for each challenge.
So, PBS146-whatever you want to call it. You know, maybe keep it organized the way everybody else does so people can find each other's stuff.
[3:56] So, the thing I had trouble with is, Ian actually helped me build it, and Ben gave a lot of good feedback too, is trying to figure out what the structure of this would be.
The way this works, you clone this entire repo, and then you have everybody's solution to every single challenge.
Challenge. So five years from now, this could be a big repo. I don't know whether it gets ungainly at that point. I'm not sure what happens.
No. Remember that Git was designed for the Linux kernel. Yeah, but Git was designed for the Linux kernel. I promise you the Linux kernel is bigger.
Right, right, right. Text files, right? They're not going to be... I don't think this is going to be huge. I mean, don't put any videos up there, right? I mean, I suppose you could, but that would be weird.
I'm not aware of any challenge I have, even in the vaguest of my imaginations, that would involve giant big video files.
Don't do that on the horizon. Yeah.
[4:45] Yeah, but it looks like it's going to work pretty well, and we've got it built, and we've started to get some people asking about it.
And here's a great thing about two people I want to call out by name here.
Thomas wrote me an email, and he said, you know, I've just started listening to Programming by Stealth, and I've really wanted to understand Git, so I guess I should try doing this organization thing you're doing.
And so that was a great opportunity. It was a guy I'd never talked to before, didn't know he was starting to listen.
He's new to the show, so this is great.
So I pointed him at all of the lessons that you did on explaining Git.
And again, the most popular part of everything we've done is your series on Git.
We get the most feedback from that, I think. Yeah, I get it all the time.
It is the single most successful thing I think we've ever recorded together, is that series on Git.
[5:34] Yeah, yeah. I think shell scripting is coming along, because we're getting a lot of people excited with shell scripting. So anyway, I'm directing him to that.
And another perfect example is Mariana just joined our slack, our PBS slack.
And it was the most wonderful thing I've ever read. It was just fabulous.
She said, she said, look, I'm brand new at programming and I'm on PBS 12 and I'm really nervous and feel weird just putting myself out here joining this community because I don't know very much, but I know that everything I've read says you have to have a community.
So I'm here.
Oh man, everybody just open arms, you know, just embraced her immediately going.
And she even told us she's in her 50s, so I loved her even more, right?
So she's, you know, doing this later in her life, kind of like I am.
And I don't know, everything about that was great. So the PBS Slack is on fire.
We've got people joining this organization. Now we have a place to share our work, and I love it.
And I posted yours, by the way, from last week in your folder.
[6:36] Oh yeah, that was one of the other requirements. I put it in the list of requirements was, it can't take any of Bart's time because there isn't any. We have no time from Bart.
And with the one exception of, I think there should be a second owner.
Maybe somebody else becomes the second owner, but we could talk.
I'm happy to be a second owner because that's basically a second pair of keys in case you get locked out of the house, right? Yeah, exactly. Exactly what I was thinking.
So right now I've just been, and I've learned how to make folders, which is not easy inside GitHub.
[7:09] You have to say, there is no make a folder. It's make a file, and you put slash at the front of it, and it goes, oh, you made a folder. Okay, good. And then you put, so you put slash Bart slash PBS 146.
Read me, well, you could put a file into the folder, and then it will magically appear.
So if you make a readme.md and just stick it in a folder, and it could just say, like, this is Bart's folder.
But I mean, at the GitHub level, when I'm in the GitHub interface, I can't create a folder to put a readme in.
That's what I'm saying. You're right, but you actually make it all in one.
So you say that the name of my file is...
Yeah, you say the name of the file is slash Bart slash readme, and then Bart existed. That's how I did it.
Yes. Yeah, that's a Git thing, not a GitHub thing.
[7:54] Because you'll notice if you have an empty folder and you go to do a Git, and you're in a Git GUI, it won't show up on your list of changes. It won't show up to be staged.
Okay, yeah, yeah, yeah, yeah.
I do think one of the fun things about the, and I think I said this last week, the fun thing about the shell scripting is now I am doing so much from the command line.
I'm really getting a kick out of doing it that way.
It just feels like a more natural place to do it.
Dan, just wait to see what it does to your automations as you get your feet under you here.
It really does, when you have a tool like, well there's loads of them like Alfred and, even TextExploder, you can do shell scripts, Keyboard Maestro.
There's so many of these tools. Hazel. Yeah, they can all run shell script.
And so now you get to marry the GUI with real programming.
[8:44] It's quite powerful. I do an awful lot of shell script action stuff.
I used to do it in Automator.
I guess I'll do it in Workflow. I still can, true, but you know, at some stage I will.
You mean in shortcuts? That's the one. Sorry, it was workflow before they bought it. Yeah, they renamed it.
Yeah. The other thing, when I was talking to Ian, he said that he's really digging having a Stream Deck. Oh, he was complaining about things I've made him buy. And so he was talking about a Stream Deck. And I said, oh, so how are you automating things on your Stream Deck? And he said, with shell scripts. He says, even if I have to call an Apple script inside the shell script, it's a shell script to start with. So that's kind of what he's using, what he's learning.
Yeah. That's how I do most of my Apple script, is a call to the terminal command, OSA script. But anyway, that's yes. Yes, it is. Anyway.
So people listening, you don't have to memorize anything that I told you. There will be a link in the show notes to the blog post I wrote that explains this whole thing.
[9:41] Yes. Excellent. Now we can start. Yes. And this episode has not quite gone to plan, but I think that's for the better.
So, not only are we recording a week earlier than usual because we both have lives and it just works out better this way, I had promised you that we would be learning about the command for putting proper named arguments, minus A or whatever, on our shell scripts and how to make them work with the pipe and stuff.
And I had fully intended to do that. So what I was going to do was quickly write my sample solution to the challenge, which which I mentally thought was simple, kind of is simple.
And then I got really stuck, like the universe started making sense, stuck, like I couldn't make a variable go.
Stuck.
And figuring out the very, very, very subtle subtlety of Bash that I had walked into with both feet actually is a really good, to remind people that when the world doesn't make sense, you're not the first coder to hit this.
[10:50] This will happen many times in your coding experience. There will be a technical reason.
You will be frustrated as all heck between discovering that the world doesn't make sense and finally seeing the light, and it will seem like you will never see the light, and you will feel I am silly, I'm just not good enough. All rubbish. It happens. Everyone.
One decade, two decades, it'll probably still happen to me in three decades.
I've been coding, you know. Yeah, how long have you been coding?
Or doing that? 1997. 1997. Not shell scripts since 97, but I've been coding since 97.
It doesn't really matter what language you use. It will keep happening.
So when it happens to you, not if, when, don't feel bad, don't feel there's something wrong with you. This is just what happens.
And you will get to the other end of it, and you will end up learning a lot.
But you'll be quite cranky in between.
[11:55] But not perfectly. I was no Buddhist monk. But I did have the common sense after an hour to just sleep on it.
I had to do a lot better the next day. Oh, that is what always works.
Yes. So anyway, I basically figured 10 minutes to do the two sample solutions, one for the not bonus, one for the bonus, and then we'll get stuck into the new content.
And then it was literally noon today, and we're recording today, And I hadn't written a word of show notes apart from my two sample solutions that had taken me right up until noon today.
But I think there's a lot of meat here. So I've turned this into a potpourri episode because as well as the thing I hit my head off hard, there was also a few other little things that I've kind of been like, I need to squeeze these in somewhere.
I need to find a hook for these little things, but they're too little to be an episode.
So why don't I hang them with this, what I hit my head off. And then we have an episode, it's actually going to be very practical, but it wasn't the one I'd planned.
Okay. Okay. Which is fine. The one I had planned will be just as good next time.
So the first, the challenge was to store an array of breakfast items, or a breakfast menu as an array, basically.
And then to use the SELECT loop to allow the user to choose items for breakfast.
And you should add an extra option into the list of items for I'm done.
[13:24] And just keep asking the user what would you like, what would you like, what would you like until the user says and that is all. And to CAPTURE the choices so that at the end of the process you can print out and say and here is your breakfast order.
So you can be a good waiter and read it back. And that was the challenge.
And then for bonus credit, instead of hard coding an array, because that's...
Remember we talked about the concept of a bad smell in software engineering?
Well data in your code is one of those bad smells.
If your script is your data.
It might be okay for something you've hacked together in 5 minutes, it's not a good idea for anything serious. So you want to separate them out.
So the challenge was, make the menu be a text file that you read in, and you should ignore empty lines, and lines that start with the octothrop, the pound sign, the standard comment symbol.
[14:18] Which is quite realistic, because you generally do actually want, if you're going to read your data from an external file, it's really good for your own sanity to be able to put some comments in.
Because it will help, trust me. me. So that seemed like a sensible challenge. And it literally took me the expected 5 minutes for the solution to the initial challenge. Because, well, I've been doing this a while, as we said.
So in the zip file, you will find pbs147-challenge-solution.sh, which is the solution to the true challenge.
So usual shebang line. Then we define our menu as an array. So menu equals open a round bracket, pancake space, waffle space, porridge. Quite a few of them with a space. And then at the very end I have orange juice in single quotes because it has a space and if I didn't, it would be orange and juice. And while orange is a valid breakfast item, juice is a bit nondescript to my liking. I prefer to know what it was you juiced before you gave it to me.
[15:16] So, you know, quote by the book. Then I made an empty array which I named order using the declare keyword. So declare minus a for array order. And then I very politely echoed at a menu, choose your breakfast. And then I use the select loop to loop over the exploded version of my menu. So we learned that you can use $open curly name of array, open square bracket, add symbol, close square bracket, close curly to effectively explode your array into a sequence of arguments, effectively, right?
Because remember, ultimately, Bash is about commands, so it's effectively a sequence of arguments.
And we're passing those arguments to our select.
Statement, command. Now, we have orange juice in there. So if we didn't quote that exploding syntax.
[16:12] Then orange and juice would come out as separate items. Even though there were one item in the array, they would now come out space separated and they would become two items. So in order to make it clear to bash, our intention is that it should honor the space as we quote the exploding syntax, as I'm going to keep calling it.
So when I did my homework, I decided to jump to the challenge part.
So I didn't actually do the easy method. I started with an external text file.
And getting that orange and juice separate as one set was really, really difficult from an external file.
That took me quite a while to figure that one out. And I'm not sure I did it the way I should have.
I think I might have done it a hackery way.
We'll talk about it when we get to that point, though. Well, there are an infinity of correct solutions.
I would say they're on a spectrum from very elegant to inelegant.
So I don't know where on the spectrum you land, but if it works, it's still right.
I'll ask you about it when we get there. Okay. So then I have in my loop, you know.
[17:11] Basically, oh yeah, sorry, the other subtlety is I wanted to be able to say, I'm done. So we don't just want to loop over the items in the menu.
We also want to loop over an item that just says done or something, you know, finished, or I went with done because it was less typing.
So, that's just another argument, so you can just say, select item in done and then explode your array. And then effectively, you're looping over done plus all the elements in your array.
And I chose to put done first. Oh, wait a minute, wait a minute, wait. Select item in...
So if you have select item A space B, it's going to select from A and B? Yeah.
So you said done... Well, you haven't taught us that you could do that.
[17:54] I guess you just did. I was going to say, I did implicitly, but maybe not explicitly enough, because remember, it's anything after the in is a sequence of arguments.
So you can put any arguments you want there, and one of those is when you explode the array, you make it into arguments.
But there's no reason you can't explode two arrays.
So you could have said select item in pancakes, waffles, porridge, done.
[18:16] Yes, yes you could, which is how we did it in the last installment's example.
Okay, okay, that's interesting. All right, I find that a very confusing syntax to look at it that way, but okay. Remember, it's all terminal commands. So after the word in is just a sequence of arguments, and the exploding syntax turns our array into arguments. But it doesn't turn them into the only arguments, they're just arguments. So you can have more before and more more after. And so you could explode 20 arrays into one select statement, if you had 20 arrays, or mix in a few variables. They're just arguments. It's a list of arguments.
So anyway, select item in, done, and then our exploded array. Do, I'm going to intentionally skip over the next line because I want to talk to you about it later, so put a pin in that. And then we want to do a check to say, well, did they choose done?
[19:09] So the first thing I do is our lovely shortcut syntax where we can use two ampersands to do a lazy and. So we start off with the square brackets for doing a conditional. So our condition is item is equal to done. And then we say ampersand, ampersand break. So because of lazy evaluation, the break will only happen if item is done. So it's a really short if in front of that. No, because remember, the and will only execute the second statement of the first statement is true. We learned about this shortcut a while ago, it's often used for if something goes wrong and do an error or or do an error or whatever.
So it's lazy evaluation. So if the first item is true, only then will it will bash try to figure out if the second item is true as well.
[19:56] So the square brackets just give you back an exit code of success or fail.
So if our item is equal to done success. Okay, if the first item in the and a success, I have to check the second, which is break, at which point nothing matters anymore because because you've jumped out of the loop.
Okay, so it's not the double brackets that's making that an if statement, It's the ampersand ampersand.
Correct. That's making it an if, then. Okay. Yeah, the two square brackets are basically saying, give me an exit code for this piece of logic. Okay.
Well, I decided to stick in between those two, thank you, your order is, and spit back the order right inside of there. So I don't think that would have worked for me.
I have no particular reason you couldn't have that too, right?
So item done, ampersand, ampersand, here's your order. Oh, I see what you mean.
No, wait. There is a way, but it would not have been elegant.
It would not have made your code better. Okay, okay.
I thought about putting it all outside of this like you did, but it seemed like that was a perfectly good spot. It was going to exit right there.
That's true, actually. That is another way.
Break the loop, you can go straight to exit, which is just end the script.
There are two perfectly valid approaches.
Mm-hmm. I still did the break, but I just did the break after I told them what they had ordered.
[21:21] So how did you make it exit the script completely then? Oh no, you did a break, and then, yeah, and the only reason nothing happened after the loop is because you had no more code. Okay, so again, pretty much by the book, then after, so I do have to exit.
So after my pretend, my little short, elegant if statement, I then order plus equals my number of my item, and then I print out added item, whatever to your order. Now, this is the point in the show where you need to jump in and tell me that I was a bit of an idiot last week.
[21:56] Well, we can't call it an idiot. But actually, Grumpy says he wants his money back for last week's episode. When I started using...
You can have it back a million times over, his zero. So in the show notes, the way Bart did it last week, he said, you can add to the array saying in this case, like order plus equals dollar item.
And then he said, you can also say order plus equals, and then use the square bracket syntax to say put it in position one.
So he did waffles with the plus equals, and then he did pancakes with the square bracket, one. And his code worked perfectly.
I was going to be looping through this text file, so I needed to keep using plus equals and then, you know, dollar item. Well, it turns out it doesn't work without round brackets around it.
It just smushes them all together. So I got pancakes waffles. I didn't get pancakes on one line and waffles on another line when I spit the array back out. It was squishing them all together.
[22:54] Yeah, and that's basically because the syntax that I was supposed to type had the round bracket all along. But if you leave out the round bracket on the first item, you get away with it by default, which you're actually doing is, say, add this string into the array and bash it. It's like, well, that's an odd thing to ask for, but okay. But when you do it the second time, time, you just get your first string gets bigger and then your first string gets bigger again.
That list, those roundy brackets indicate this is a list that I want to stick into my array and that does make a very big difference.
So yeah, that was a typo from last time. So the people who have been reading along will not have seen the problem because thanks to the magic of Git, we have fixed the problem.
But we're saying it out loud, even though it's not written in the show notes, because any of you who were listening last time will have heard the wrong thing.
And now we've corrected it. was supposed to be there, and I assume...
Actually, if they listened last week and read last week, they knew it, they ran into it.
True, yes.
[23:49] Luckily, if we're recording again quickly enough, I think we'll probably have corrected 98% of the people. Exactly, exactly.
Who won't get caught out. Yeah, so if you're with us in real time, you know we made this silly mistake that we have now corrected, and if you're coming back to this years later, it'll be fine in Google.
We won't need you up the garden path. But yes, that is kind of important, and I noticed as soon as I did my own challenge solution. It's like, oh, I forgot my brackets last week. So anyway.
Yeah. So I called Bart and I said, hey, Bart, you know, when you did that plus equals, he goes, I know. You already knew what I was calling you about.
Nope. I was slightly cranky with myself because plus equals is really obvious syntax, but yeah, the brackets are kind of important. So anyway, I did that. So for the most part, there's not really much here to call out. Apart from the thing we've already mentioned of sticking the done in as one of the arguments in the argument list for the select. And the other subtle thing that we called out, we did the brackets. Oh yes, and the cheap little avoiding the if statement with the ampersand ampersand, just a reminder of that. But then there's that line I asked you to stick a pin into.
So what we haven't really said is, right, so the select loop is lovely, right? It gives.
[25:01] You a list of options, and they all have a number. And if you type in the number next of the option you want, then in the code, the value of that option is what goes into the variable. So $item in the case of my code is whatever is next to the number one, which I think in my case is pancakes, just like it is for you. But what happens if you type in boobity boop, or 42, or donkey, or whatever.
[25:27] Well, you might think, oh, Bash will give me an error message.
No, Bash doesn't like complaining. Bash doesn't do that.
Or you might think it'll just give you the prompt again.
No, it's like, well, you typed something. Okay, I'll take it.
What it does is it tries to map the value you typed to the list of options.
There is no such mapping, so it gives you the empty string.
So if you want to skip invalid items and just give the user the prompt back, Or you could shout at them. I decided just to give the prompt back.
You need to check whether or not $item is empty.
Now you could do that with a double equals empty string, but the minus z operator is by far the nicest way to check for nothingness because it works very universally. So I did the same square bracket ampersand ampersand trick. So open square bracket, open square bracket, minus z, $item, close square brackets, close square bracket, ampersand ampersand, continue.
So, break snaps you out of the loop completely, continue just says, and back to the start of the loop.
So, it just short-circuits that loop through the loop.
[26:32] Does that make sense? I'm a little bit confused. Okay. I understand what your code does.
What I don't understand is why it doesn't seem to be breaking with mine.
So I just purposely typed in boop at the prompt, because that's what you said to type.
And it said, I got the response back, your order so far contains, and it said nothing after it.
And then I put in done, and then it said, thank you, your order is, and then there's nothing.
Right, because what you did was you appended to the array an empty string, which Bash went, well, that really isn't anything. So Bash went and ignored your append.
Right, but it came back with a prompt. It didn't barf on this.
Right, it won't barf.
They won't barf, but it executed all of your code with $item being nothing, which in your case didn't do any harm.
But in the general case, really could do harm, right? Your code is not always going to be absolutely fine if something you think has a value is actually empty.
[27:33] Okay. In the case of my code, it isn't nearly as elegant because it says, added nothing to your order.
Because I'm printing out added dollar item to your order. So in my case, it looks very silly if I don't skip.
Okay. Because the rest of your code did execute. Okay. So it does...
Mine does look silly. It says, so far your order contains, and then there's nothing there.
Right. So it... Yeah, exactly. So it continued through the loop with nothingness, and your code didn't really take that into account because I never told you to. And I was kind of wondering if anyone would stumble on before I had a chance to tell you. Yeah. Well, another way to do this would be to say, if you know how many items are in your list, to say, if I don't get a 1 through 10, then say, sorry, we don't have that.
Please pick from the numbered list. But you don't see what they type. You see the mapping of what they type to the menu. So the only thing you can actually check for is blankness.
[28:30] Oh really? Well, the user typed 10, but you got pancakes or whatever your 10th item is. You didn't get 10.
Right Yeah, you don't I guess I don't really understand that you said you said there's a mapping from the number they put, So the select creates those numbers, Yeah, and it also creates the mapping to the number 10 in my case my case is done But let's say 9 is more bacon because why shouldn't you have more bacon and it's it's a the the select is mapping to more bacon Yeah, So it doesn't know that I typed in a 10. No, you don't know what they typed in, Your code, dollar item, or whatever you named your variable is going to get the item in the menu. It's not getting anything.
Well, it's not really a limitation. It's just an abstraction.
It chooses to use numbers. At the end of the day, what you want is the value they selected, and they selected something that doesn't exist. So the value you get is nothingness.
So the way you test if they typed something silly is, well, it gave me nothing.
It wasn't on the menu.
They obviously typed anything that's not on the menu will get you nothing.
Well, I guess you could check for nothing and say, I'm sorry, we don't have that.
Correct, exactly. That's what you're supposed to do.
So the check is much simpler than having to know how many items are in the list.
It's just, is it nothing?
[29:48] If it's nothing, okay. But I'm saying in your double square brackets minus the dollar item, you could have the if statement go with that that says, then print, I'm sorry, we don't have that.
Yes, absolutely. Yeah, you choose to react. Basically, Bash doesn't decide what to do, you do. And the way you figure out that you need to do something is by checking for nothingness.
One other thing. I don't remember you teaching us this, but this would be the 4000th time you said, yes, I did, if it's true. Your echo statement at the end says, echo minus e, quote, backslash n for a new line. I had to go find out that you had to put a minus e in order you to use the new line thing. I didn't remember you teaching us that.
That may have been something I was meant to put in here as a potpourri thing. That may actually be something I had done so that I could tell you about it and then forgot to tell you about it. Okay, well I told you about it then.
Yes, but it needs to go in the show notes now. Oh, I'll make a note for you.
Yes, please. So yes, without the minus e, Bash will just assume that you didn't really want a new line and print you out backslash n. And I don't, nowhere that I googled did I ever see anyone saying, and for this really logical reason echo behaves like this, what I see is lots of people going, for reasons no one can remember because it was in the 70s and they were probably on drugs, they decided to write it this way.
[31:15] Okay, that's funny. So minus e. Minus e. Yeah, I don't even know what it stands for. Don't be silly. I don't know. One less e tablet. I really don't know. Would it make sense to just always type echo minus E?
Probably. Pretty much sure that you want it to actually properly print out your escape characters.
Instead of having to remember once in a, you know, every 25th time that I've got to put a minus C and go.
[31:42] Yeah. Well, one of the tips you'll be getting in a future installment is that maybe instead of echo, you can use the more powerful printf. So that's probably the long-term solution is printf.
But it's, it's a slightly more convoluted syntax, but it is more powerful. So swings in roundabouts. You know, echo is easy, printf is more powerful. Different people like different things. And then I did also make use of the even worse syntax, $, open curly bracket, octothorpe, name of array, open square bracket, at symbol, close square bracket, close curly bracket, to get the length of the array. And the only saving grace is that the octothorpe is the counting symbol. So it doesn't vaguely imply counting or looking at something over a noisy modem, or like your cat running over your keyboard.
I got really good at this because I started from the inside out every time.
I said, okay, I want my array. My array is order.
Okay, I want to tell it to loop through all of the things in the array.
Okay, so I'm going to use, I'll put square bracket out symbol after that.
Okay, next, I'm going to need to tell it that I want the, I want to say this is a variable, but I can't just put a dollar in front of this. I got to put some squirrely brackets around it to hold that in place.
Okay, good.
Now, I want to count it. me stick in my Octothorpe. So I did it methodically every time, instead of trying to memorize it or anything like that. I just said, let me build it each time. And I got fairly good at it after a while.
[33:04] That is how I mentally do it every time as well. And that works whether or not you're putting in the Octothorpe. You go through all the same steps, except you don't get as far as and throw in the Octothorpe. So when I'm just accessing the array, I do the same mental logic.
That's why I came back to the Octothorpe, because I don't always need it. But when I I need it, then I know it goes inside.
Yeah, exactly. And then quote it. Yeah, it works.
Make sure it's inside quotes when you're done for the echo.
Yes. Precisely. Precisely. Okay, so that gives us all of that.
So then I thought, well, five more minutes, I'll just do the bonus.
I mean, how hard can it be?
We just read a file, append it to an array.
We already know that your plus equals into the array. We learned how to read a file a few installments back, but that time we printed it to the screen.
We didn't know about arrays yet, so we just printed it to the screen.
And so you can read a file with a cat command, and you can pipe the output of cat into a while loop.
So I thought, wow, I'll just pipe the output of my menu into a while loop, and it'll be grand.
And I ran my code, and it said, please choose from, done. Nothing else.
There was nothing on the menu. Just done.
[34:15] What? So my text file has a hecking lot more than done because since I could put in comments and since it was a separate text file, I broke it apart and I had like, you know, pound sign, sweet stuff, pancakes, waffles, healthy stuff, porridge, we usually eat fried, sausage, bacon, eggs and spam, drinks.
It's a full menu here. And, you know, because I needed to test it and skip the empty lines.
Yeah, exactly. I had to test it, skip them and everything. So I had all that stuff in there and I just said done.
[34:44] What do you mean done? Do I not know how to append to an array?
I was like, oh yeah, I forgot the round U brackets. I actually don't know how to append to an array, whoopsie.
Better fix next week's show notes. So I put the round U brackets in, felt very confident, ran it again, and got done.
Am I not reading the file? I'll put in an echo statement. Perfect printout on my menu.
Printed out the array.
[35:11] Nothing. Where are my values going? I am adding them into the array, and then the array is evaporating.
Actually, yes, that is exactly what was happening.
Why it was happening is a subtlety that took me quite some time to figure out.
You said three hours? I think between before sleeping on it, looking at it, going for a walk and then finally fixing it.
I'd say so. Yeah, I certainly. Wow. Yeah. It exercised the mind.
Anyway, there is something in bash that I know about, that I wanted to tell you about later, But I'm going to tell you about it now, because it becomes important So in bash you have this concept of a subshell, Basically a bash within a bash And we've used it So when you take a command and you wrap it in round brackets and then stick a dollar on the front, What you've actually done is you've said The bit in the round bracket is a separate little script Execute that separate little script And then the dollar sign says give me the value.
[36:19] So you've been putting a bash script in a bash script. Now, what I haven't told you, because it really doesn't matter, is that in that little mini bash script in the bash script, it gets a copy of your variables.
Note the word copy. Before you go too far, I heard the word copy, and I'll come back to that, and that's clearly the important part of this story.
But what's an example of where I've done a subshell within a shell?
We would have assigned variables to the result of running a command.
We would have maybe piped something into a grep or something.
I know we have used it in examples. I mean, round brackets, quote, dollar item?
[37:02] Is that a sub? Any time there's parentheses, are you saying that's when it's a subshell?
When you stick commands inside of parentheses and when you're executing the command.
So if the thing that goes in is... in is, so if you have like a variable, you want to say, my hosts becomes equal to cat tilde slash etc slash hosts or whatever, or slash etc slash hosts, how do you get the value from the command in?
You put it in parentheses with a dollar.
Okay, okay. So that's the command within the command. Okay, I'm caught up.
Yeah. Okay, so you're saying we get a copy of...
So when you do that, you're making a second bash. And that second bash gets a copy of all of your variables, so it can use any variables you've created.
But it gets a copy of your variables. Which is very safe. It means you don't get spooky action at a distance.
[37:55] And it means you can still use the variables you've set up. But what if you want to change a variable?
Well your changes evaporate.
Now I did not intentionally make a subshell, but I did accidentally make a subshell.
Because a very very important subtlety is that when you pipe one command to another, what bash does is it says the command on the left is a subshell and the command on the right is a subshell and I will take the output of one subshell and make it the input to the next subshell.
But they're both subshells. So when I said cat mymenufile pipe while, the entire while loop is in a subshell.
Inside the while loop I went menu plus equals item. was a copy of my menu.
[38:59] So there's nothing in the menu? So as soon as the while loop finished and the subshell exits, the copy of my menu, which is full of stuff, evaporated because the subshell came to an end.
And I was left with my original variable, which was completely untouched because it was copied into the subshell.
And the original was entirely unaltered, so at the end of my loop, I had an empty menu.
Wait a minute, but the menu is an array that you define outside of this pipe command.
So why is it empty? Correct. When you get out, it should still be there.
No, because the entire while loop is in the SoapShell.
Oh, hang on. Are you talking about the choices the person made from the menu? No.
So I am filling my menu, right? So if you look at my code...
Oh, you're filling your menu as an array in this while. I'm sorry, that's right.
The menu's in a text file, it's coming in, you're populating the array with this while loop that's on the right side of the pipe.
So you're creating this menu, and then it's empty when you come back out.
Yeah, so I entirely filled the menu and then throw it away.
I took a copy of an empty array, filled it with my menu item, and then threw it away.
And there was F going, why is my array empty?
[40:22] Now, if I had intentionally typed the round brackets, I would have known I was making a subshell.
But it was an implicit subshell because I chose to pipe. I did eventually find the right magic words to make Google tell me that.
But that was some serious Google food before I finally got Stack Overflow to tell me that one.
And then literally it was three words. Pipe makes subshell.
And I just was like, oh, that's what it is. And then everything made sense.
The universe was the universe wasn't broken.
I just didn't understand what was going on in my own code. And that's always what it ends up is the universe is fine, but you're missing a piece of information, a subtlety somewhere.
And that, yeah, that was where three hours went. So then I went, OK, great.
I now know why it's broken. Actually, I'd say that was only about like two and a half hours.
Then I was like, okay, this is why it's broken.
How do I unbreak it? How do I get the content of my file into my loop without my loop vanishing off into a subshell?
And the answer is, very pleasingly, something I've been trying to find an excuse to tell you about.
[41:39] Out. So bash has the ability to define multi-line strings very nicely using the triple arrow syntax.
So we know that a single arrow can redirect the output of a command to a file. And a double arrow will append the output to a file. And if you send a single arrow in the other direction, reading from the file into your command.
[42:05] Well, if you use three arrows on the way in, you take a string instead of the content of a file, and the string becomes the input to the command.
I'm getting super lost, and part of it's, I'm used to following along in the show notes and I have no idea where we are, so I'm not being able to picture this in my head very, well. Is this, are we at a spot in the show notes that I can identify?
I'll use the following snippet to load the menu. It's the line above, there is a lot in bold going on here.
I see, I use the following snippet to load the menu. Okay, so where are you using arrows?
Right, so I have while read minus or menu line, then do lots and lots of code, done.
So the done is the end of my while. And at the end of the done is three arrows, and then I'm feeding in some information.
So instead of cat pipe to my while loop, I am triple-arrowing into the back of my while loop.
[43:14] So I did this with one arrow and just the file name. Mine says done, one arrow to the left, menu.txt.
Mine's a do loop with an I. My reading is that that syntax isn't supported in all versions of Bash.
I actually thought the Mac wouldn't let you do that lovely shortcut, because you can also pull a file's content in that way.
Then you don't get to learn about multiline strings.
[43:44] Yes, you're going to have to back up and explain multi-line strings because I wasn't following you when you were going double, triple, arrows, I was getting lost, because I was like, I only used one.
Okay, so if you use one arrow, the thing that comes after the arrow is the name of a file.
Correct. And the value that gets shoved into your statement is the content of the file.
So there's a redirection going on here. Your string is being turned into a file name, the file is being loaded, and then its content is going into your loop.
But sometimes you just want to specify a string, and that string may go on for a very long time.
[44:20] And so the way you just say, I want to input this great big long string, which may or may not contain Uline characters, into my command is with the triple arrow.
And it's called a here string, for reasons I don't... I can't figure out why it's called a here string.
It's like, here, have a string.
Which kind of here?
H-E-R-E, here. It's like, here, have a string. I don't know.
And so you basically say triple arrow, and then you can either have a single or a double quote.
And then it will just keep taking the string until you close the quote.
And you could close the quote 500 lines later.
It doesn't care how many new line characters or what goes in between.
It'll just say, I'll keep taking it until you give me the closing string.
And then I'll take all of that string and shove it in as the input to the command.
But the string again, in your case, is the contents of the file, the same as mine.
[45:15] It is, but I went and read the file by using a shell. Yes. Right.
Yeah, so I very explicitly did what you did.
[45:24] Okay. So you cat this menu.txt, but you actually did some trickiness there to get the directory name to know where that menu.txt file is.
Can you explain that?
That is next on my list. So remember I said there's a lot going on here?
That is one of the many things going on here.
For the moment, let's ignore the fact that I'm doing that tricky stuff.
So what am I doing inside my here string?
String. I'm making a subshell to cat my file into that string. But because the subshell is now entirely contained within the single string, no problem. The subshell has not infected my while loop, because the subshell is contained all within the string, which is why this works.
So I still have a subshell, it's just the subshell hasn't enveloped my whole while loop, And hence it hasn't broken my universe.
So his subshell is written with this quote, dollar, parentheses, and then he's got cat, and he's going to talk to the file.
Yeah, so pretend it just says menu.txt.
[46:30] Right, but the parentheses is what gave you... that's what's creating the subshell. Correct.
Correct. Because you've got a command inside the parentheses. Okay. Yes.
So you say cat, and then... what are you doing? Okay.
We want to do. So I'm going to describe the problem to be solved before I tell you how I fix it. So, so far, when we've been running our shell script, we've gone into the folder where the shell script is. And we've said . slash name of shell script.
[47:00] So the current directory we're executing is the directory where the shell script is.
Mm-hmm. So by pure accident, a relative path inside our script is relative to the script.
It's not really relative to the script. It's relative to the folder we were in when we called the script.
So if you start writing shell scripts for real, you're going to put them in a folder called tilde slash scripts or something, And you're going to run them from goodness knows where.
Oh, and so it won't know where it is? So when you say ./.menu.txt, it's looking relative to where your terminal is, not relative to where the script is.
[47:42] Right. Which can be very different places. So if I ran my script from my delete me folder and I said ./. and I gave it the full path to my in my script, it wouldn't know where the menu file was.
Well, it would look for it in your delete me folder and go, well, there's no menu here.
Where it wouldn't be, yeah, okay.
So it's a subtlety we haven't hit because we've always been in our folder.
But I thought this was an opportunity to remedy that oversight.
So there's a couple of pieces to how we do that. So the first piece to how we do that is that Bash gives you a special variable, $BASH__SOURCE in all caps That variable exists in every shell script And it's the full path to the shell script.
[48:33] So if you're running...
So it won't exist in the terminal because the terminal is in the shell script, But if you run a shell script, that variable is created for you by Bash And if you have a shell script that includes another shell script, it's smart enough to follow it down, that if the line of code is in the second shell script, it'll be that shell script's path.
So the variable always points to the actual place where the line of code running right now is.
[49:02] Because you can have a shell script that calls a shell script that calls a shell script.
Oh, oh. So it's always correct. VarMinus is always correct.
The line of code that says... The line right where you are.
The line right where you are is the file that...
It will know where you are and it will be your file for the very line of code you're using it on.
And if you use it on a different line of code, it'll be a different line of code. It's always right.
That's the good thing. But it's the full path, including the name of the script.
[49:30] We want the folder. Now, we don't want the name of the script, because I just want the menu file in the same folder as my script. So I need to get rid of the file name. You could use regular expressions.
I promise you, if you do that, it will break. Because there will be a special character somewhere, someday, it will break.
So what we want to do is use the correct tool. And the terminal provides us with two terminal commands designed for both halves of that job. The command dirname takes any file path and gives you just the directory part. And the command basename throws away everything apart from the file name on the end. So it basically splits it in pieces. The folder, the file.
I've seen this in AppleScript. It looks exactly like this in AppleScript.
It's quite a common thing to want to do. Most languages have a way of splitting a file path into the file bit and the folder bit. So here it's dirname, basename.
So dirname or basename, I should probably not name many of my variables that.
Well, as long as you access it, no, you're fine, because you'll be saying $dirty.
[50:44] Okay, and this is $() which makes it a subshell where it's a command.
Correct. So you can actually type durname straight into the terminal.
Yeah, 10 out of 10. But you can use durname in the terminal. So you can say durname space slash etc slash host and it'll tell you etc. Or you can say basename slash etc slash host and it'll tell you hosts.
[51:06] But that's only inside quotes and inside parentheses and with a dollar symbol around it.
No, no, no, no, no. So inside quotes with the dollar symbol is to pull the value of the command into the string. But if you're just on the terminal, you can just type dirname.
Dirname any file path you want and it'll split it up for you. It is a terminal command, like cat or like whatever.
[51:29] So why in the show notes did you do it with the dollar parentheses around it?
Because I don't want to print it to the screen. I want to actually pull it into my string, which I'm going to then pipe into my while loop.
No, I'm sorry, I mean in the example that you give in the show notes where you're at the command line doing it.
You could have done it without the... You didn't need the quotes and the dollar and the parentheses around dirname.
I guess because I was trying to make it look more like my code.
It sounds like not a bad idea.
Right, yeah, okay. They are just terminal commands. We would generally use them by saying, pull this into a string.
So we would generally use them inside our quotes and stuff.
But it is just a term like cat or echo or anything else.
Very interesting. Very interesting. I did wonder about that when I was doing it.
It's like, oh, I hope it knows where menu.txt is.
I just, I didn't even, well, you talked about .slash, but I just said menu.txt because it was in the same directory with me.
And that is, not starting it with a slash is effectively the same as a .slash.
If it starts with a slash, it's an absolute path. Everything else is relative.
Okay.
[52:41] So dot slash menu and menu are the same. All right, so I used my single arrow menu.txt at the end of my do loop going through my breakfast menu with a value I was looping through. But I could change that now to this triple arrow.
[53:04] Well, you don't even know. If the single arrow works in Mac OS X, you can actually just use that I didn't think it would. My reading was that it was added to a more recent version of Bash.
I'm glad to be wrong about that. I'm glad that does actually work on the Mac.
Oh, okay. So I'm not wrong to use it? You're not wrong to use it.
The only thing you need to do is, instead of saying the name of the file after your arrow, you want to have a string after your arrow, so double quotes.
So your arrow, and then immediately double quote.
And then inside the string, you want to use a SoapShell to get that dir name.
That dir name. Right, right.
So it's dollar sign, open round your bracket, dir name, and then whatever your...
The dir name. Oh, so I don't need to do... Yeah, I still need the bash source, right?
Yeah, because that's what you want to get the dir name of. Yeah.
And then your slash menu.txt, or whatever you called your file.
You probably did call it menu.txt, didn't you?
I did. You know, there's not a lot of options. I thought about breakfast menu, and then I was typing a lot of words.
Yeah, exactly. Menu.txt is just easier. When it comes in, I called it breakfast menu, though, because I thought BM would sound bad.
It was worse. BO would have been worse.
[54:17] But yeah, so that's basically it. So there are the various little tips and tricks.
So I really wanted to tell the story about how the tiniest little thing can throw you for a loop. And honestly, it felt like the universe didn't make sense. I was like, but I'm appending it to my array. I could do an echo statement. It's here. Where is it going? How is it evaporating? Where is it vanishing to? It was very frustrating. And the point is, it happens to everyone. Everyone will experience this. And when it happens to you, don't feel bad.
Don't feel that something's wrong with you. No, it just happens.
Everyone, 20 years in, still happens.
[54:55] I'm sure 50 years from now, it'll still be happening. I think that's a great lesson.
I'm sure I've talked about it on the Nocellicast.
I don't remember if I've talked about it here, but when I was working, we worked with giant mirrors, and my boss dropped a screwdriver on a $60,000 polished beryllium mirror, and it was the best day of my life, because he was like, the golden child could do no wrong.
And I was like, I'm never going to do anything that stupid. Luckily, I didn't.
But it was a release. It was relief. I knew that I was like, OK, people do stupid things.
I make a point on any... Whenever we hire a new sysadmin, I make a point on the first day to say, look, you will make mistakes.
Don't cover them up. Tell us about them and we'll fix them.
I deleted every single computer from the Active Directory domain.
With a misplaced star. Trust me, you'll make a mistake. Just tell us about it, we'll fix it.", And they like the fact that, oh, right, it's okay to be human here.
And it is always easier to fix a problem you know about instead of having to try to figure out what was broken and then try to fix it.
So anyway, look, it's always good. Everyone, everyone, everyone makes mistakes.
And always remember that when you're being frustrated and cranky.
It happens, everyone.
So next time- It was a good lesson. It was a good lesson. And next time we will pick up where I had, I'd expect it to be this time. So we will continue to turn our shell scripts into.
[56:20] I would say, first class terminal citizens. We will make our shell scripts behave like the terminal commands you know and love, or know and hate. Like the terminal commands you know.
It will behave like the others, whether you agree or disagree, it will be terminal-ish.
And it's always, my view is always, make your stuff fit into the language you're working in.
So people look at me and they go, but when you're writing Java, you use CamelCase.
When you're writing PowerShell, you use PascalCase. And when you're writing in Perl, you use underscores.
Doesn't that drive you nuts? It's like, no. In Perl land, every API is in this case.
So my code fits in perfectly.
In Java land, everything is in CamelCase, unless it's the name of a class, in which case it's in PascalCase. But there's rules, and I just follow the rules.
And whatever language you're in, follow the rules, and your life will be so much easier.
And you won't be trying to remember, is this a function that I wrote and therefore I did it in my way?
Or is this a standard function, therefore it's capitalized completely differently?
Just go with the flow. And so I would advise people, even if you don't like the way the terminal commands do things, write your shell scripts to behave like terminal commands.
It'll just be easier.
[57:30] Yeah, that definitely does sound good. Okay, so we're going to pick up, what's the subject of next week, though?
So next week, we're going to learn about something called GetOpts, which allows us to have minus a, you know, artichoke, I don't know, whatever.
Basically, add those little flags to your shell script and have it process the flags so that you're having to write lots and lots of code.
Because GetOpts does almost all the work for you, which is quite nice.
And then we'll also learn how to make your script.
So when you pipe, and you make a subshell, but when you pipe stuff to your shell script.
[58:05] Your shell script has a small amount of work to do to accept the content of the pipe. The terminal will arrange it so that the content piped to your shell script will be available to your shell script. But your shell script does have to take it. It's like the terminal, you can lead a horse to water, or you can pipe content to a shell script, but the shell script does actually have have to take it. And so we haven't learned how to take, how to know there is input. And then when you know it's there, how do I read it? And then you can do clever things like saying if you, like the grep command is a lovely example. You normally, if you use grep without a pipe, you specify where you want it to grep and then what you, sorry, you specify what you want to grep and then where you want to grep it. So you're saying the input, the, place to go look is the second argument.
But when you say, cat myfile pipe grep, you only give grep one argument.
[59:04] So grep is clearly doing some intelligence there. It is saying, if there is pipe input, no need for a second argument.
If there is no pipe input, I shall treat the second argument as a file name and go fetch the content myself. So there's an if statement inside grep somewhere.
Right? that is what we're going to learn to do, to be able to create that same intelligence in our scripts.
So we're definitely taking up a notch in complexity here. Believe it or not, it'll be like one or two lines of code.
But unless you know them, you can't use them. So that's, like I say, we're making our shell scripts behave like terminal commands.
And I know you like a challenge, and since we have this shiny new community, we should do something with it.
So in... kind of in preparation, to be honest, because it'll be convenient to have this as a starting point for next time's examples.
So I would like you to take your current solution to the current challenge and allow it to accept an optional argument.
So in this case, it doesn't take any arguments yet, so your optional argument shall be your first argument.
And that argument, if present, should be a number, which is the limit on the amount of things you are allowed to order.
[1:00:20] So you're saying you would say run the script, space, seven, and then it would only take the first seven of the ten maybe that you had in your input file?
No, no, not limit the menu, limit the customer.
So your select loop should go around up to a maximum of seven times.
So you can still hit done at any point, but if you try to go beyond seven or whatever number you pass as the first argument, you have to say, I'm sorry, but there is a maximum item per customer of whatever.
Your tray is only so big, I don't know. Can I add a challenge I thought would be fun to add to it that might not teach him anything?
I was going to say, what if somebody asked for blueberry pancakes and that's not on the menu?
What if your thing could say, ooh, that's a great idea, let's add that to the menu, and update the text file?
But given our conversation about the fact that you're never presented with the input, You're only being told the mapping into your menu.
You have no way of doing that.
[1:01:25] Wait, explain that again? So remember we said that you type the number 10 or whatever?
Oh, because it disappears when you write it.
Oh, because you're not writing...
Oh, doggone it. Well, that's no fun. Never mind. It's fine. We'll just limit the people.
They can't have as much, they can't have food. There is a way to achieve your functionality.
So you know that if they type something you don't have, it will give you an empty string.
And then you can use an if statement to take an action on that empty string.
You could take as an action, echo, oh, it seems that we don't offer something you would like.
Is there anything you'd like me to add to the menu? And then use a read minus p to get the value from them.
[1:02:12] And you could then append it to the text file. So all you know is they asked you for something you don't have, but you don't know what that something is.
So you can ask them. Say, hey, did you typo that, or is there actually something we should add to our menu?
Yeah.
Okay. That can be done and you could append it to the file.
[1:02:33] Yeah, that's what I thought would be fun, but I don't know how to do it.
Well, you're quite close, actually, with your arrow.
I may have told you halfway through how to append maybe two arrows.
So, yeah, that is certainly doable. That is a more challenging challenge than what I had in mind, but perfectly doable, perfectly doable. It is possible.
I'm looking for any excuse to keep playing with this, because I'm having so much fun with it. it.
Good! Good. That makes me very, very happy. That is perfect.
Right. Well, that is all I've got in me show notes. So until next time, continued happy computing.
If you learn as much from Bart each week as I do, I'd like you to go over to lets-talk.ie and press one of the buttons over there to help support him.
He does 98% of the work here. I'm just the stooge that listens to him and asks the dumb questions.
If you go over to lets-talk.ie, you can support him on Patreon, you can donate via PayPal, or you can use one of his referral links.
I really hope you'll go over and help him out.
In the meantime, you can contact me at Podfeet, or check out all of the shows we do over there over at podfeet.com.
Thanks for listening and stay subscribed!
[1:03:48] Music.