Gentoo adventures: overlays are great -- but you can make them even better!

AKA "how I was going to code but got side-tracked with interesting Gentoo stuff instead"

One of the features I enjoy about Gentoo is overlays which are functionally equivalent to Ubuntu's PPA repositories: places where you can get packages which aren't officially maintained by the main channel of the distribution.

Of course, just like with PPAs, you need to pay attention to the source from which you're getting this software -- you are about to install software onto your Linux machine, which requires you to run as root and said software could do nefarious stuff during installation -- let alone after installation, when you're running that software.

So, naturally, just like with any software source (in the Windows world, read: ALL software, because basically none of it is vetted by people with your interests in mind), you need to check that (a) you're happy with the advertised functionality of said software and (b) you trust the source enough that the software upholds the contract to provide that functionality -- and only that functionality -- and not trashing your system, leaking your passwords, killing your hamster, drinking all your beer or wall-hacking in an fps and calling you fag or anything stupid like that... But I digress.

Note that in the discussion below, I'll use the term "package" because that's logically what I'm used to; however the more correct term in Gentoo land is "atom" -- and I'll use that sometimes too (: I'm slowly evolving (:

Anyway, warnings about nasty coders aside, there's another concern: package clashing. Let's say, for example, you're looking for a source for your favourite editor (it may even be VSCode, which is a reasonable editor, though if I really had to pick a favourite, it would be (g)vim -- but VSCode is high up on the list. Anyway, let's say you were looking for VSCode on Linux (which is possible) and you happened to find an overlay providing it. That overlay may also provide other packages with the same category/name as package which are in the main source set. Or may conflict with another overlay for a similar reason. So a good idea is to start by masking the entire overlay, adding the following to a file in your /etc/portage/package.mask (I have an overlays file in there, and add one line per overlay):

*/*::{overlay name}

where {overlay name} is obviously replaced by the name of the new overlay you added with layman (reminder: you add an overlay with layman -f -a {overlay name})

Next, you unmask the package you actually want from that overlay, in /etc/package.unmask (again, I have an overlays file in there for this purpose):

### {overlay name}
{category}/{atom}

or, a concrete example, using visual studio code:

### jorgicio
app-editors/visual-studio-code

Now you can do an emerge -a {atom} and see if you have other requirements to meet (licenses or keywords for example: many overlays will require the testing keyword for your architecture, which, for me is: ~amd64)

The advantage from the above is that you have far less chance of overlays duking it out as to who provides the packages you want as a lot of overlay maintainers maintain many more than just one package in their overlays. You can still search for packages which exist in those overlays with emerge --search {atom} and unmask them as required. Also don't forget you can find packages (and their overlays) on the great Zugaina site!

In addition to the above, I've discovered today that, aligned with Gentoo's philosophy of requiring the user to actively select installations (instead of just foisting them on the user), when you add an overlay with layman, by default, it will not auto-update!. So, I expected that emerge --sync would also sync added overlays -- and by default, it won't. You can manually sync overlays with layman's --sync command (passing an overlay name or the magic ALL string to sync all of your added overlays), but I'd prefer to have this as part of my regular sync, analogous to apt-get update being applicable to all sources.
Indeed, I only figured this out after installing visual-studio-code and wondering why the editor kept reminding me to update when I couldn't see an update in the overlay I was referencing -- it was all because the "index" for that overlay wasn't being synced... So the next issue is auto-updating.

If, like me, you'd like to auto-update, a quick look at emerge's rather extensive man page, under the --sync section, shows the following:

--sync Updates  repositories,  for which auto-sync, sync-type and sync-uri attributes are set in repos.conf

Interesting.

It turns out that repos.conf, in my case at least (and I believe it should be for any modern Gentoo system), is a directory (/etc/portage/repos.conf) with two files -- the all-important gentoo.conf for the main source and layman.conf, which contains entries for each added overlay. And in layman.conf, we see sections like:

[steam-overlay]
priority = 50
location = /var/lib/layman/steam-overlay
layman-type = git
auto-sync = No

The priority is also interesting -- because you can use this to decide, when there is a package conflict between sources providing the same atom, where that atom should come from. But it's not the topic of discussion today. Today, I'm more interested in that last line. Changing:

auto-sync = No

to

auto-sync = Yes

for each overlay yielded the results I wanted: when I emerge --sync, my overlays are updated too and I can get updates from them. Of course, the emerge --sync command takes a little longer to run -- but I don't mind: I can get shiny new stuff! Combined with the postsync.d trick outlined here to update caches and local indexes, you can also have fast emerge searching. FTW.

And now you can too (:

Parting tidbit: if perhaps you wanted to apply the masking/unmasking strategy from above to existing overlays, you can use the following command to list installed atoms from an overlay:

equery has repository {overlay}

then apply the */*:{overlay} mask and unmask the installed packages.