PBS_2024_10_26
An audio podcast where Bart Busschots is teaching the audience to program. Associated tutorial shownotes are available at https://pbs.bartificer.net.
Automatic Shownotes
Chapters
Long Summary
In this episode, we dive into the intricate world of Git submodules, examining their significance and practical applications in development workflows. I am joined by Bart Bouchotts as we revisit concepts we've discussed in previous installments while also exploring new territory that is crucial for our ongoing projects. The episode is anchored around understanding what submodules are and the real-world challenges they can solve for developers, especially in collaborative coding environments.
We kick off the discussion with a refresher on the fundamentals of Git, laying the groundwork by explaining that a Git repository is essentially a database of commits, which contain file and folder information, along with metadata. This leads to a deeper exploration of how Git submodules fit into this framework. A Git submodule is essentially a repository nested within another repository, allowing for the management of dependencies and modular components of larger projects. By explaining how commits work in conjunction with submodules, we establish a clear connection to practical outcomes, showing listeners why this information matters.
Following the theoretical groundwork, we delve into three specific use cases that illustrate the power of Git submodules. The first example discusses deploying a plugin-based web application, particularly using WordPress. We analyze how Git can manage the complexities involved in keeping identical versions of a WordPress setup across multiple servers, thus preventing chaos in scenarios where numerous users access the site simultaneously.
Next, we explore the need for subdividing document repositories, especially in content creation contexts. I illustrate how establishing a structured hierarchy within repositories can streamline collaborative efforts and maintain a cohesive workflow for writers and editors. This segment emphasizes the advantages of using submodules for separating distinct projects or components while keeping them interlinked for easier access and management.
The final example leads us into the future work we will be doing—focusing on incorporating external code dependencies. As we discuss our upcoming project involving XKPassWD, we emphasize how submodules will allow us to effectively manage both the library and its corresponding web applications. By linking these two repositories, we'll be able to work on them simultaneously, pushing and pulling changes with ease, thus minimizing the friction often associated with integrating separate codebases.
As the episode progresses, we not only define technical terms and elucidate concepts but also share insights from our own experiences in various programming contexts. By framing complex topics in a relatable manner and providing tangible examples, we aim to enhance your understanding of Git’s robust capabilities and empower you in your coding endeavors.
In conclusion, we provide a sneak peek into our next installment, where we'll put the theory of Git submodules into practice with a hands-on project, reinforcing everything discussed today while iterating on our collective learning journey.
We kick off the discussion with a refresher on the fundamentals of Git, laying the groundwork by explaining that a Git repository is essentially a database of commits, which contain file and folder information, along with metadata. This leads to a deeper exploration of how Git submodules fit into this framework. A Git submodule is essentially a repository nested within another repository, allowing for the management of dependencies and modular components of larger projects. By explaining how commits work in conjunction with submodules, we establish a clear connection to practical outcomes, showing listeners why this information matters.
Following the theoretical groundwork, we delve into three specific use cases that illustrate the power of Git submodules. The first example discusses deploying a plugin-based web application, particularly using WordPress. We analyze how Git can manage the complexities involved in keeping identical versions of a WordPress setup across multiple servers, thus preventing chaos in scenarios where numerous users access the site simultaneously.
Next, we explore the need for subdividing document repositories, especially in content creation contexts. I illustrate how establishing a structured hierarchy within repositories can streamline collaborative efforts and maintain a cohesive workflow for writers and editors. This segment emphasizes the advantages of using submodules for separating distinct projects or components while keeping them interlinked for easier access and management.
The final example leads us into the future work we will be doing—focusing on incorporating external code dependencies. As we discuss our upcoming project involving XKPassWD, we emphasize how submodules will allow us to effectively manage both the library and its corresponding web applications. By linking these two repositories, we'll be able to work on them simultaneously, pushing and pulling changes with ease, thus minimizing the friction often associated with integrating separate codebases.
As the episode progresses, we not only define technical terms and elucidate concepts but also share insights from our own experiences in various programming contexts. By framing complex topics in a relatable manner and providing tangible examples, we aim to enhance your understanding of Git’s robust capabilities and empower you in your coding endeavors.
In conclusion, we provide a sneak peek into our next installment, where we'll put the theory of Git submodules into practice with a hands-on project, reinforcing everything discussed today while iterating on our collective learning journey.
Brief Summary
In this episode, we delve into the complexities of Git submodules and their vital role in development workflows. Joined by Bart Bouchotts, we revisit fundamental Git concepts before exploring how submodules can address real-world challenges in collaborative coding.
We start with a foundational understanding of Git repositories, then break down how submodules function as nested repositories that help manage dependencies in larger projects. Through specific case studies, we illustrate the use of submodules in scenarios such as deploying a plugin-based web application with WordPress and organizing document repositories for streamlined collaboration.
Finally, we discuss our future project with XKPassWD, showcasing how submodules will facilitate simultaneous work on the library and its web applications. By sharing relatable insights and practical examples, we aim to enhance your understanding of Git’s capabilities and prepare you for our upcoming hands-on project that will reinforce today's discussions.
We start with a foundational understanding of Git repositories, then break down how submodules function as nested repositories that help manage dependencies in larger projects. Through specific case studies, we illustrate the use of submodules in scenarios such as deploying a plugin-based web application with WordPress and organizing document repositories for streamlined collaboration.
Finally, we discuss our future project with XKPassWD, showcasing how submodules will facilitate simultaneous work on the library and its web applications. By sharing relatable insights and practical examples, we aim to enhance your understanding of Git’s capabilities and prepare you for our upcoming hands-on project that will reinforce today's discussions.
Tags
Git submodules
development workflows
Bart Bouchotts
collaborative coding
Git repositories
nested repositories
manage dependencies
case studies
WordPress
XKPassWD
Transcript
[0:00]Music
[0:07]Well, it's that time of the week again. It's time for Programming by Stealth,
[0:14]
Introduction to Git Submodules
[0:10]and this is Installment 172, recorded on October 26, 2024. And I'm your host, Alison Sheridan, and of course, I'm joined by your co-host, Bart Bouchotts. Hi, Bart.
[0:21]Hello there. I guess I'm not a guest here, and it's me every time, so you don't have to be, and this time it's. These new intros, they take some getting used to.
[0:30]But it's gotten a lot simpler. Life is easier for me, at least.
[0:39]
Understanding Git Submodules
[0:35]That's fair and there's only one number this is good for all of us just one number yeah.
[0:40]Makes so much more sense
[0:41]Okay well we are we have been taking a very interesting route through lots of cool topics and we did a tidbit last time and so we're we're getting back on the horse but it's a horse we've used before and put in the stable i may be mixing too many metaphors here but we are jumping back on the git horse for a little bit because we did a lot of git and we did more git than i had intended to because well we all had a great time and everyone in the audience was engaged and i was learning a lot and you were learning a lot and it was it was kind of going well but i did eventually stop and i didn't stop when we ran out of git i stopped when we ran out of things that were obviously relevant but they were obviously the part of git we need and there's now a part of git i intentionally didn't do then but that now makes sense because it solves a real world problem that's adjacent to this show because it affects me and you and there's a problem on the horizon that directly affects this whole series because it is a going to we're going to encounter it in the xkpasswd rewrite and so I'd say now is a good time so I've done a lot of talking around it here the topic we are going back to is something called a git sub modules.
[2:05]That is Git jargon. What a submodule is, is a Git repository inside a Git repository. Inside a Git repository. You can nest them as far as you like. They're like folders. So a folder can contain files or folders. A Git repository can contain file folders or Git repositories.
[2:25]Git repos all the way down.
[2:28]All the way down. Turtles, mom. I can't remember which scientist said that, but it was good. So in Git speak, when they say add a sub module, what they're saying is link this other Git repository into my Git repository. And so you can link in as many other Git repositories as makes sense for the problem to be solved. And goodness me, are there many possible problems to be solved. So we're going to do this as a, I'm going to tell you why you care. And then the next time we're going to do a worked example in great detail with the git command on the command line so you're going to learn the command line commands and what i am almost certain you will find is that in your git gui of choice the name of the git commands and the menus you see when you click on things will line up and then you'll be able to use the gui if you like but you'll have the commands in the show notes i.
[3:26]Do like having the underlying commands because then And I feel like I can use a calculator to do arithmetic, but knowing how to add is important and knowing whether it's getting the right answer, whether you really understand it.
[3:38]Yeah, exactly. So the first thing I'm going to do is lay a little bit of theoretical foundation, because way gets a little stale in our brain. So I just want to slightly bring us back into Git's model of the universe, Git's way of thinking. So ultimately, a Git repository is some settings that apply to the whole repository and a database of commits. And inside each commit is a record for every file that exists and every folder that exists, which is the content of the file and some metadata like the permissions and a few other bits and bobs and permissions on the folder.
[4:19]Well, hang on. Did you just say that the commit contains those files and folders? Because I don't think it does.
[4:25]It contains all the information to rebuild them. It doesn't store them as files and folders, but it contains them in the sense that it can make them appear out of thin air.
[4:34]Right, right.
[4:36]So, yeah, that's a very philosophical question, Alison. Does it contain them? So, when we link to another Git repository, what does Git save in its little brain? In its little database of commit number 53289, we have file this, folder that, and then we have other Git repository Z. What's it actually saving when it's doing that? So at the Git configuration level for the whole repository, it saves very, very little. It just says that this folder within me maps to this Git repository at this URL. It literally saves two pieces of information, folder, link, and that's it. And you can actually see them. They go into a file you can look at, so you can actually see which folder is connected to which Git repository. But within each commit, it saves a little bit more. Within each commit, what it says for each of those links is the currently checked out branch and the current commit that is checked out on that branch.
[5:41]The currently checked out branch of the repo that it's linking to?
[5:47]Yes.
[5:48]Okay. Okay. Well, I guess that makes sense.
[5:50]So it's a snapshot in time.
[5:51]Right.
[5:52]Exactly. Yeah, you're preserving the state of the other repo.
[5:57]Yeah, so if you've got, say, a work in progress, a WIP branch on the sub-module, the sub-repo, and then you've got a production one, you can be flipping those back and forth, and the upper-level repo just points at, well, whatever you got checked out right now, that's what I'm going to go with. Okay.
[6:16]Ari, even within your repository, you can get very clever. So you can have two branches on your repository pointing at different versions of the other repository.
[6:26]Wait a minute. How could you do that if it's pointing at the currently checked out branch?
[6:31]Each commit is pointing at the currently checked out branch. So as you change branches on your repository, you're in different commits. So you can flip... Just like when you flip local files.
[6:43]Wow. Luckily, we're doing audio or video right now because Bart can see my eyes crossing. So wait a minute. So if I've got... How could I have two different branches checked out of the same sub repo at the same time? How could I do that?
[6:57]No, not at the same time. But you could have two branches within your repo. So right now there's a branch called main in our repo, which is the published version of PBS, which does not have episode 172 yet. And there's a branch called whip which does have 172 if we had a submodule then the branch main could be connected to one commit on the child repository and the branch whip could be connected to a completely different commit so i could test if upgrading something on the remote module broke something here by making a branch on the local module okay.
[7:36]Okay so does the the upper level git repo define which one is checked out of the sub module
[7:43]Yes that's the piece of information it saves in the commit so per branch you get to choose which version is checked out on the linked modules but.
[7:53]It's not that you check out different uh branches within the sub module it's that the it's the module or the repo above that's defining which one is checked out.
[8:05]Yes.
[8:06]Okay. Okay. That makes sense then. I will allow it.
[8:10]Which basically lets you time travel, right? It lets you time travel arbitrarily so that you can be on one branch of your, we call it the master module, right? The one that does the linking, we call it the master. So in your master module, you can have one branch called testing, which has gone and gotten the beta version of jQuery. And you're being all adventurous. But also on your master you have another branch called stable and you've checked out the nice safe version of jQuery and you know that code works and that's the code that's on your website right now but you're playing away on your little whip branch and then when you're all good and happy then you bring that commit into your main branch and what you've now brought with you, is the change to the linked repository and all the changes you've made to your repository They come together.
[8:58]Huh. Interesting.
[9:00]Okay. It's very powerful. Very powerful. So there are infinity many reasons
[9:08]
Use Cases for Git Submodules
[9:07]you might want to do this. But I want to sort of explain why to you with three examples that are important, that I think are relevant to also as a giant big audience. So the first one is the one that opened my eyes. I have known about these things called submodules, and I sort of vaguely understood that they were probably important to someone somewhere, but they didn't become important to me here, until I encountered the first of the examples I want to share with you. The second example fixes a problem you and I actually have today. And the third example is what we're going to meet in the not-too-distant future in this series. So there are the three examples. So... The first one I'm calling deploying a plugin-based web app. The second one, subdividing a document repository. And the third one, including dependencies in your code, or in our code even. So I guess we'll do them one by one, unless you want to stop me at this stage before we do that.
[10:12]No, now I understand where we're starting. I got it.
[10:16]Perfect. So I'm going to use an example that is, I hope, friendly to you. So this series is called Programming by Stealth. But we have strayed, ventured, dipped our toes maybe into the sysadmin realm, particularly with Shane Waugh. That was very sysadmin-like. And Bash scripting, well, it's programming, but it's kind of sysadmin programming, isn't it? You know, that's sort of where Bash does most of its work. So we've dipped our toe in the sysadmin, you know, space. And that's where my first exposure to the power of git repositories was was with my sysadmin hat firmly on my head at work because that's what i did for decades of my life i was a sysadmin, and i'm going to give you the example based on a tool that you and i are both perhaps more familiar with than we'd like to be i know i have sworn both at it and by it over the years which is wordpress so podfeet.com is kept on the air by wordpress bart b.ie the giant mess that it is that i'm ashamed of is kept on the air by wordpress.
[11:22]I'm in love with wordpress bart you give it you give it a hard time all the time i know i find it effortless to keep going it's delightful i don't have problems with it i really don't auto updates well you and
[11:37]Half the planet.
[11:37]Yeah yeah i don't I feel like it's a lot harder doing the way we do it with the static webpages on GitHub for PBS. That's a lot more faffing about than I do on WordPress, so I'm going to stand in defiance as support for WordPress.
[11:57]Well, I mean, at the end of the day, it is a platform that powers half of the internet, so whatever else one can say, it's gotten many things right because half the internet exists. So for people who aren't familiar with WordPress, under the hood, when you install WordPress, it all comes in one big folder of PHP files. And in there is a very important folder called WP content. And in that folder, you can download all sorts of cool things to make your WordPress, your WordPress. So you can download themes. And I can't remember the name of the one you use, Alison, but it's one I never used.
[12:36]SiteOriginNorth.
[12:37]That's it. Yes. and you can install plugins to add functionality to WordPress that doesn't come with everyone's WordPress because everyone uses WordPress for different things so it makes sense for it to be a plugin architecture and then there's a configuration file that tells your WordPress about your web server and about your database and that connects all the pieces together and so basically that's the three pieces the files from real WordPress are not real.
[13:03]So core WordPress, the bit everyone has in common, the files for your themes and your plugins and your configuration file. And that's all sitting in a folder. OK, right. And so you can install it by downloading the zip file, downloading the zip files, popping them all into folders. If you are prepared to trust WordPress, then you can give it right access to itself, which is a bit of a snake eating its own tail because it means wordpress can update wordpress and right for regular users that is a risk but the risk of not doing it is actually higher or more dangerous in my humble in my no in my professional opinion because if you don't let wordpress update itself then you have to remember to do it and there are more people running wordpress and there are people who have a team of sysadmins their own WordPress for them. So letting it update itself is definitely safer, even if hypothetically it's a risky thing to let an app update itself, because out of just a bug in the app.
[14:07]Yeah, you know, it's a risk, it's a risk. So for regular single person websites, install it once, manually handle all the files and let it update itself is a model that just works. And it has been just working for you for two decades, three decades.
[14:27]Two decades. Yeah.
[14:29]Two decades. Yeah. But if you want to scale that out, if you want to not have a thousand people looking at a website in an hour, but a million people looking at a website in an hour, you can't just throw RAM and CPU at the problem. It's tempting to just pay for a bigger virtual machine and that will buy you capacity. That's what you've had to do a few times as your show has become more popular. You buy more RAM, buy more CPU, you get more website. But that hits a limit. And when that hits a limit, your only choice is not to scale up, but the scale out. So you actually end up making five web servers or six web servers that all have the identical copy of your code. And they're all serving it. And they're all talking to a cluster of database servers. But we're going to leave those aside for this conversation. So you have five identical web servers, which have to have those same three things, the The folder from WordPress, the config file, and all of your plugins. And they have to be identical to each other. Because otherwise, 150-year people are seeing WordPress 2.4, and 150-year people are seeing something else, and chaos, right?
[15:38]Well, luckily, humans are really good at replicating and not making any mistakes when they have to make something identical, right?
[15:44]Oh, totally.
[15:46]Well known to be one of our strengths.
[15:49]Absolutely. Now, you can do this manually with tools like Orsync and all these kind of things. So you could actually do it by hand, but you know something, it's really hard. What you really want is some sort of source control to manage versioning things, maybe like Git. So in fact, Git is a really good tool for sysadmining and an enterprise level plugin based app like WordPress. But it needs a little bit of help. It needs sub modules because it's not an atomic thing. It's the WordPress Git repository gives you core WordPress. You're going to add your config file. So you're not going to get that from someone else. That's going to be yours. You're going to add that yourself. And then you're going to have to link to the Git repositories for each of your plugins as the appropriate subfolders in your master repository. So in effect, what you have is a clone of main WordPress with one change, your config file.
[16:56]And links to the other Git repositories. And now you have a structure. And because every commit says this exact commit from that repository and this exact commit from that repository, what you have in a single commit of your repository, is a snapshot of all three things. Core WordPress, your config and each and every single one of your plugins. And if you do a Git checkout, On all of your five machines, you get five identical snapshots of time.
[17:34]Okay. Okay. You have to remember to do it on all of your machines.
[17:38]True, which you script with a bit of bash script. So you would probably also
[17:47]
Deploying a Plugin-Based Web App
[17:44]have another one or two machines that you use for testing things. So you would make a new branch on your git and you would get the latest version of WordPress press and pull that down into your branch and you'd see if that works and then you'd go into your various plugins and pull down the latest versions of those and see if that works and if it all works you do a git commit and then you merge that branch into your main branch and then you deploy it across all of your web servers and you wouldn't even have to do that manually because another enterprisey thing that we have seen hints of we've seen little little sort of its tail around the corner as it ran away from us is automation connected to git so when you hit push on the main branch of our git repository for the programming by stealth series about 60 seconds later the website updates it's not magic it's code right it's a script that gets triggered by the act of pushing.
[18:47]And so that kind of automation by a push is called by the acronym CICD, because why make it easy, which sounds for continuous integration, continuous development. I don't know why. And in GitHub universe, CICD is called GitHub Actions, which is a much better name than CICD.
[19:09]So I actually distinctly remember the day I learned that it It was a good idea to do development on a separate server than the live one. It had not occurred to me.
[19:22]Yeah. Oh, yeah. You break things once and then you have lots and lots and lots of people shouting at you. And then you go, oh, if I broke it quietly in a corner all by myself, no one would be shouting at me. It's much nicer that way. So when you put all those pieces together in the real world, what you basically end up doing is you make a new branch.
[19:43]You do all of your testing and then you merge that branch into a permanent branch you keep called staging. And that staging branch is connected via CICD to your test web servers and so you make your changes you merge them into staging and then 30 seconds later your test website has your new code you follow your test script you might even automate your test script using those tools for that and when you're happy that you haven't broken everything then you merge that same change into your main branch which then triggers the CICD that simultaneously goes to your five web servers and says git pull git pull git pull git pull git pull ta-da okay okay and that is just so much easier a universe to live in than you would be in doing things the old way and i know this for a fact because the reason i was hired as a sysadmin when i started my sysadmin career two and a bit decades ago was to manage an even bigger web app than wordpress it's a virtual learning environment called moodle and when i started managing it we had one server and when i finished managing it we had a full-on enterprise setup with multiple web nodes and test and production and everything and over my two decades as a sysadmin i went on that journey from doing it by hand scripting it using or sync and botching together my own stuff right you know hey.
[21:11]Scripts are amazing, to discovering the joys of Git.
[21:15]To discovering the joys of CICD with Git, to discovering the joys of going push, wait 30 seconds.
[21:22]So you've known this for a long time. You've been holding out on us.
[21:24]Yes, I kind of have, because I wanted to find a hook to hang it on to.
[21:30]
Integrating Content Creation with Git
[21:30]So that is when I fell in love with submodules. So now let's move on to a use case that's very relevant to the two of us. but we're going to approach it we're going to back into it, So, Git is great for coding, Git is also great for sysadminning, but there's a third group of people for whom Git is a fantastic tool, and that's people who write content, because it's text. So if you need to write text git can be your friend and there is a fashion these days for static site generators so like you say wordpress is a fantastic tool used by half the internet but you can also build a website by having a folder of files which get translated into a web page which is how this website for pbs so anyone who's reading along right now is reading a web page that was built by taking a folder of Markdown files and turning them into HTML with a pretty theme. And in fact, it was all automated using CICD, which is GitHub Actions, because there's a feature in GitHub called GitHub Pages, which gives you a free web publishing platform, which isn't bad, actually. Free GitHub, or sorry, free website. Yay.
[22:44]And doing that doesn't require Git modules, right you can just do that with all the git we've known before but now imagine that it's not me and you doing pbs but it's a big worldwide charity with branches in five countries and they want to have the irish branch update the irish part of the website and the belgian branch do the belgian part of the website and whatever and a very easy way to do this is to have one organization who are responsible for the website, the webmaster, we used to call these people many years ago. I don't think that title's around anymore. It was a fun title.
[23:21]Really? Because of the word master?
[23:23]I think it's because now it goes under the Director of Corporate Communications.
[23:28]Wait, you're just talking about in your organization. You're not talking about globally that word doesn't exist.
[23:35]I haven't seen it on a business card in a very long time. And I used to see it on business cards all the time.
[23:40]Maybe because so much of it's gotten automated that it's not a badge of honor anymore.
[23:45]I don't know. But either way, there's someone whose job it is, right, to be responsible, to, yeah, to carry the can, right? If the website suddenly tells people to go do something you don't want it to, someone has to be responsible. It's someone's job. And so whoever that is, whatever title they have, they would set up the master git repository that is actually connected to the web servers. So they would connect it to the production web servers and to some test web servers with CICD. And then they would make a separate git repository for the Irish people and another git repository for the Belgian people and another git repository for the Germans or whoever else we have in our imagined organization. And they would link those into the master one as submodules.
[24:30]And from the point of view of the Irish office they would see one folder which would have only their files they would edit their website and when they went to publish it what they would actually be doing is triggering a notification to the people who run the real repository to say hey there's some changes here please review them and maybe they get automatically put into the testing website and a human being comes along and goes oh nah terrible capitalization I'm going to push a change up to them there fix that typo there that idiot in ireland keeps on putting in dashes where there shouldn't be any dashes and he leaves them out where they shouldn't be there that irish fella he's terrible um and when they're happy they would put it to their production website.
[25:16]And so from the point of view of the people in the offices they're editing a small folder of their stuff but from the point of view of the people running the thing as a whole they can see everything. Now that's not entirely dissimilar to another type of document publishing you can do which is to have a knowledge base which isn't published on the internet but it's a folder of files that contains information that is organized and the one that I have fallen in love with and a lot of people have is an open source one called Obsidian because it doesn't hide things in proprietary formats. It saves the knowledge base as a folder of markdown files with the YAML front matter and therefore if obsidian goes away despite being open source you haven't lost anything because all your data and your metadata are safe and so i really developed a desire to put some order on all of the stuff i write for other people, So I do a lot of contributions for you every two weeks for Security Bits. I do random sporadic segments for you when something blows my dress up or when you go on holidays or.
[26:27]When a cool watch band arrives.
[26:30]Yeah, what a cool. Oh, yeah, I should. After the show, I have to show you my cool watch band. But yes. And I also do stuff for other podcasts. And what I wanted to do is to have them all together in one place that was in Git so that I could edit them from anywhere. Not be afraid of breaking stuff and with metadata ask myself sensible questions like what's the most recent thing i did about ai where was it what was it published well you know how many times have i talked about an ev to bode what evs have i talked about what did i produce in june 2024 that i need to tell my patrons on patreon about yeah give me a list of all the links of every time i reviewed a vpn right those kind of questions i can ask of my knowledge base because it has the metadata for everything i do which is yay for me and i trust allison completely right i i trust you totally so i didn't have any fear plus.
[27:27]It's all in git so you know you can get it back if i screw it up
[27:30]Yeah i know that's yeah the trust trust but verified no trust but have a safety net i don't know yeah.
[27:35]There you go
[27:36]So i had no compunctions about giving allison full right to access to my entire knowledge base, everything in Bart's brain, Alison can write to it, no problem. Because I trust Alison, which is fine from my point of view, but from Alison's point of view, it's not about trust. It's about information overload, because from Alison's point of view, what I'm doing for Bodhi or for someone else is 110% irrelevant. And instead of seeing what I see, which is a folder of stuff I care about, what Alison sees is a haystack, which somewhere contains the needle of interest which is this week's security bits only it's a haystack I.
[28:17]Think it's like seven levels deep now for me instead of at the top
[28:21]Exactly. So we are going to be changing that by using Git submodules. There's going to be a folder which is going to contain only the stuff me and Alison do together. Alison's going to check out that repository and see only that stuff. And I'm going to have that as a submodule in my bigger repository. And from Alison's point of view, she sees only the stuff we work on together.
[28:46]Yeah, I'm hoping you'll break it up even more that I can just see security bits and just see what other stuff is. Okay. I don't really want, because I'm used to seeing I just had a security bits folder and now I have this folder that I have to go seven levels deep and climb through. And what it causes me to have to do is understand how your brain works putting folders together. And you don't organize things the way I would organize things. So it's always sandpaper when I have to find what I'm looking for. So there's no reason you couldn't do it that way, right? Programming by stealth can be a submodule and security bits can be a sub-module and other stuff I write for Allison can be another sub-module.
[29:25]Right, yes, exactly. So we have to figure out the structure and the naming convention so that I can figure out how the pieces go back together again. I will name them in a way that makes sense so that I can see how they connect together. But yes, basically that is the point. We can have our cake and eat it.
[29:38]
Managing Information Overload
[29:38]I hope so.
[29:39]That is why this solves a problem for us. Our third use case then is future you and I and everyone else which is including other people's code in our code and this is not a new problem to us because we've dealt with a subset of this problem so we have used jquery, we have used bootstrap and the first time we did either of those things we did it by literally just putting a link in our file to a copy of jQuery on the internet on jQuery's content delivery network, which meant that we used jQuery kind of without copying it into our Git repository, right? We didn't actually have a copy of jQuery, we just used it over there.
[30:27]And that's not bad, but it has some side effects, because if something goes wrong with your CDN, all of your stuff breaks all of a sudden. There was recently a massive security issue with the CDN that went away and then got bought out by a Chinese company who used it for nefarious ends. And everyone who had taken that approach suddenly had other people's code on their website, which was not a good thing to have. So there are definitely dangers to that approach. so we also learned as part of programming by stealth to use a package manager in our case we used npm and so with npm you don't save github you don't save jquery in your repository what you save is a json file that tells npm the version of jquery you want and then you go npm space init.
[31:18]And it fetches the version the config file says to go fetch and it saves it in a folder that's your gitignore file and so you don't save there you don't save other people's code you save that you save a reference to other people's code that you use a package manager to fetch for you so that's another perfectly valid approach and the third approach we've already learned about is something called repackaging where we take just the pieces of other people's code we want and we use webpack in the case of what we learned about in the relevant installment i linked in the show notes I can't remember the number now and that makes like a custom version of just the parts of jQuery we want and it builds that as part of publishing the website and that's actually how the beta version of xkpasswd does things it when it publishes the website with the github pages thing it's getting a subset of jQuery and bootstrap and packaging that as a find called main.js.
[32:18]I believe So it's actually including the whole thing.
[32:23]Well it's not it's being a bit clever because we're actually only taking the bits of jquery we want and bundling into into our main.js so webpack is cool okay okay but and all of these things work but they are when you are including other people's code that is not changing day to day and public right jQuery is published and the version of it is usable for months years at this stage it's very stable so that's a very different use case to what we're going to be running into with xkpasswd where we are going to be spilling out the library from the web interface because the library should be generic and should be usable by anyone and their website or in a completely different contexts, like a command line tool or whatever. So the library should be standalone, like jQuery is standalone. And the website should be its own thing, like a web page that uses jQuery. But we're going to be building both of them at the same time, which is different to using jQuery for our websites, because we're not building jQuery, we're consuming jQuery. But here we're going to be building both of them together. So, in the long term, XKPassWD is going to be in NPM as a module, and it's going to be stable, and it's going to change very rarely, and you're going to be able to go NPM install XKPassWD.
[33:51]But until that's ready, there's going to be this phase where we really do want to be actively editing both at the same time. And to the best of my knowledge, the least stressful way to do that is we get submodules where we basically say the website has as a submodule the library and then you check out the website you will get a copy of the library and you
[34:15]
Building and Using Dependencies
[34:12]can push and pull to the library and push and pull to the website oh.
[34:16]That's cool that makes sense
[34:18]So that is there are three use cases actually that last one was so easy to say i've sort of caught myself by surprised that these three use cases were asymmetrical in their size yeah.
[34:29]I'm looking at what you've managed to explain pretty clearly in just 35 minutes was 3561 words so uh it was a lot of effort to put it in to make it a nice explanation and so we'll be able to go back to that and see what it uh see what it looks like and understand it better
[34:46]Yeah so where we're going Next in the next installment is we're going to take the idea of having a piece of code that gets reused multiple times and doing that for real. So we're going to make a little Git repository called, I'm going to call it branding. It's just going to be some CSS file with some custom styles. And then we're going to make two web apps that are really boring, that are going to use the same branding. Because that's a realistic thing to want to do. If you're a company developing two or three web apps, you may actually want them to have a consistent branding. And so we'll have our two web apps, each of which are linking to our branding. And then we can use Git submodules from the command line to do that in a sensible way.
[35:32]That's a great example. That sounds really fun. And I understand it.
[35:37]Yay. Cool. Well, like I say, sometimes it's very difficult to explain that it's simple. because this is a strange problem that myself and Allison sometimes have where Allison assumes it must be more difficult than this. And I'm like, no, it's actually not more difficult.
[35:53]Well, to be fair, you changed the beginning after I gave my feedback because
[36:01]
Conclusion and Next Steps
[35:57]you kind of jumped into using Git modules before explaining to us what they were. And now you went through and explained it more of what it was before then. So I think that's part of why I was seeing it as complicated because it wasn't there in the original explanation. So I think that we have discovered that if I read notes ahead the show's a lot better yeah,
[36:16]Because I don't know what I don't know no, that's not right I don't know what I'm assuming you don't.
[36:22]Know what I don't know
[36:24]That works too alright.
[36:28]I'm looking forward to that should be a lot of fun,
[36:30]Mart excellent, well, until then Alison and everyone listening, as always happy computing.
[36:36]If you learn as much from Bart each week as I do I'd like you to go over to lets-talk.ie and press one of the buttons over there to help support him. He does 98% of the work here. I'm just the stooge that listens to him and asks the dumb questions. If you go over to lets-talk.ie, you can support him on Patreon, you can donate via PayPal, or you can use one of his referral links. I really hope you'll go over and help him out. In the meantime, you can contact me at podfeet.com.
[37:06]Music