pterm conflicts with putty

[opoplawski]

Background information

It has been discovered that both prrte and putty provide a /usr/bin/pterm executable - which is problematic if a user wants both installed.

Any suggestions for dealing with this? Pretty sure putty's pre-dates prrte's by about 18 years.

What version of the PMIx Reference RTE (PRRTE) are you using? (e.g., v2.0, v3.0, git master @ hash, etc.)

3.0.2

[rhc54]

I should think one would install PRRTE (or putty) in some location (e.g., /usr/local/prrte) to avoid precisely such conflicts. As you can imagine, trying to come up with tool names that don't conflict with the tens of thousands of packages out there is an exercise in futility. I know one other package took exception to prun - but we then found that multiple packages used that tool name too.

🤷‍♂️ I don't have a solution to the global name problem short of putting packages in unique locations and letting the PATH resolve them.

[opoplawski]

That's not relevant in the Linux distribution context where everything gets installed into /usr/bin. I can sympathize, but honestly we've done pretty well over the years minimizing conflicts. But one of the keys to that is avoiding very short and generic names for executables. Is pterm really that much more convenient than prrteterm?

[rhc54]

Given that it's been out there for at least five years and embedded into people's job scripts, yes - it would be a fairly disruptive change and hardly seems worth it. IIRC, you can "conflict" the putty and prrte packages so the admin knows not to install both in the same location, yes? So this seems like an avoidable problem without a lot of pain. Certainly, this is the first time it has come up, so we know that people aren't generally hitting it (for whatever reason).

I gather you are encountering the problem due to creating packages for the embedded code in OMPI? How do you handle the conflicts between mpirun for MPICH, OMPI, IntelMPI, etc? They all conflict, so you folks must have a solution for it. Since PRRTE is being installed in support of OMPI, perhaps the solution is to apply that fix here?

[keszybz]

I don't have a solution to the global name problem short of putting packages in unique locations and letting the PATH resolve them.

That's not really a solution, because users generally want to refer to a binary by name and not the full path. So placing a binary somewhere not in $PATH would be very annoying. Above, you also suggested using /usr/local/bin — that also doesn't work, because it would lead to confusion, people would call the wrong binary binary all the time. (In particular consider the following scenario: putty.rpm with /usr/bin/pterm is installed, the user writes a script which calls pterm, everything is peachy. Half a year later, the user installs prrte.rpm, or even more likely, some other package adds a dependency or recommendation on it, and suddenly the script is broken because a different pterm is earlier in $PATH.)

IIRC, you can "conflict" the putty and prrte packages so the admin knows not to install both in the same location, yes? So this seems like an avoidable problem without a lot of pain.

It is a "solution", but a really bad one. It is totally legitimate to want to have both installed at the same time. And also, there often are big dependency chains which means that users get various packages which they don't necessarilly "need". "Conflicts" between packages are painful to users.

I gather you are encountering the problem due to creating packages for the embedded code in OMPI? How do you handle the conflicts between mpirun for MPICH, OMPI, IntelMPI, etc?

MPI packages do not provide any binaries in the normal $PATH. A user enables a whole MPI stack by module load mpi/mpich-x86_64 or similar. This way, software can be written to support generic interfaces, and a specific MPI implementation can be easily selected.

If you want to do right by users, my strong recommendation would be to take the unpleasant step of renaming the binary. (Provide a compat symlink for a time being to make the transition easier.) This is better than the confusion for users which binary they'll be running.

[rhc54]

MPI packages do not provide any binaries in the normal $PATH. A user enables a whole MPI stack by module load mpi/mpich-x86_64 or similar. This way, software can be written to support generic interfaces, and a specific MPI implementation can be easily selected.

PRRTE is being installed here to support OMPI. I fail to understand all the kerfluffle - your MPI solution is totally appropriate for PRRTE. OMPI uses PRRTE to provide mpirun, so the binary being installed basically equates to the mpirun or mpiexec from your MPI installations. Why isn't your standard solution sufficient?

If you want to do right by users, my strong recommendation would be to take the unpleasant step of renaming the binary.

I don't see how causing problems for one set of users at the benefit of others constitutes "doing right by users". It seems like you are making an inherent value judgment about which users should be disturbed.

(Provide a compat symlink for a time being to make the transition easier.)

Doesn't that just recreate the conflict? If you can tolerate the symlink, then why not the binary itself?

It seems odd that you have a solution you consider perfectly acceptable for one set of packages (MPI), but somehow is unacceptable for all others. Just treat PRRTE like you do MPI, especially since you are installing it to support a particular MPI package. Maybe I'm just missing it, but I fail to understand why the reluctance to use a solution you already consider standard practice?

[devreal]

Installing two binaries with the same name (putty pterm and prrte pterm) in different directories and then using PATH is not a solution. One of them will shadow the other, breaking either one depending on the order in which the directories appear in the PATH. So unless prrte is installed in a place that does not get slurped into the PATH loading the OMPI module will create the conflict again.

Also, I doubt that there are many applications that use pterm directly. Might be wrong. I'm fairly sure that the number of users affected by a break of putty is significantly larger (after almost 25 years of existence) than those affected by a change in the name of a fairly new package.

[rhc54]

Installing two binaries with the same name (putty pterm and prrte pterm) in different directories and then using PATH is not a solution

I guess that we need to revise all the MPI's then, too - yes? That is precisely their situation. Why is PRRTE (and putty, for that matter) being treated differently?

Look, if people feel this is such an insurmountable problem and just cannot use the standard solution, then don't install PRRTE. I assure you I will not take offense. Those that want to use it know how to get it and do so. Just use the embedded version for your OMPI distro and be done with it.

[keszybz]

I guess that we need to revise all the MPI's then, too - yes? That is precisely their situation. Why is PRRTE (and putty, for that matter) being treated differently?

There is no /usr/bin/mpirun. If there was a conflicting binary with the same name, then that'd be a problem. In other words, duplicate binaries with the same name work in two cases:

1. the case of MPI, where you explicitly inject new names into $PATH by loading a module
2. different implementations that provide the same functionality and same command-line interface and there is no reason to install more than one at a time, e.g. zlib and zlib-ng with assembly optimizations. (But as mentioned mentioned above, this is not entirely straightforward either. For example, a user might want to check whether both binaries are really equivalent, and then it becomes annoying when you want to run them alternatively. So if it's not a one-time transition and a distro wants to package both, then possibly using different names and using alternatives to link on to the standard name might be better.)

The case of two unrelated programs is different.

[opoplawski]

I think people are talking a bit at cross purposes here and not being particularly constructive - so maybe just let it lie for a bit.

I think it probably does make sense for Fedora to move the prrte binaries into /usr/lib64/openmpi/bin. They are not there at the moment because there is nothing openmpi specific about building prrte. But if they only exist to support openmpi, perhaps that is where they belong. Is prrte being used by any other MPI flavors? Is it only useful for MPI?

Thanks, and sorry for any antagonism.

[rhc54]

Thanks @opoplawski - I agree that the discussion is getting derailed. No antagonism inferred.

It is true there is nothing OMPI-specific about PRRTE itself (i.e. PRRTE does not depend on OMPI). PRRTE can support any PMIx-based application, and people from MPICH, OpenSHMEM, and others use it. However, that is in many ways a separate discussion. For now, those people (or their sys admins) download and install PRRTE directly, almost always creating a module to support it.

In the context here, if you are trying to package it as a part of the OMPI dependencies, then it (IMO) makes sense to put it in the OMPI install location. OMPI is a little complicated now in that it doesn't have a direct dependency on PRRTE, but does use PRRTE to provide its version of mpirun. So it makes sense to put the prterun binary (and friends) in the same location as OMPI's mpirun since the latter just does a little envar manipulation before exec'ing the former.

[rhc54]

FWIW: In terms of who uses PRRTE and for what types of apps, I always forget that PBSPro includes PRRTE in its distribution (at least, last I heard - few months old, but likely still true). PRRTE serves as their general application launcher within the PBS allocation, and is automatically started when an allocation is made. I don't know the details of their distribution (e.g., where PRRTE is installed). The only user-level cmd that they need to expose is prun, which they may rename. Probably the biggest single "block" of PRRTE users out there.

I have no idea how they would adjust (or not) to a Fedora PRRTE package, how they are adjusting (or not) to OMPI v5 including an embedded version of PRRTE, etc.

[rhc54]

Adding @amckinstry since he has similar question.

Some thoughts about packaging PMIx and PRRTE:

I would strongly advise against installing PMIx into a default location (I will use "/usr/local" just as an example). Starting in the PMIx v4.2 series, PMIx automatically installs all of its internal headers so that PRRTE can reuse the code - and I doubt you want that rather large tree of headers hanging under "/usr/local/include". This allowed us to remove over 10,000 lines of redundant code from PRRTE, greatly reducing the maintenance burden. The negative to that, however, is that PRRTE is now tightly coupled to PMIx, and so the two will almost always need to be updated together. It may therefore be beneficial to install PMIx into a "/usr/local/pmix-version" location to allow multiple versions to coexist (ditto for PRRTE).

This raises an issue in PRRTE that still needs to be addressed. Currently, PRRTE enforces a minimum PMIx level that it can build against. This is probably not sufficient as changes in a PMIx utility (not part of the official API) could result in a behavioral change that impacts PRRTE's use of that utility. It isn't possible to define a "max PMIx version" for PRRTE as no one can predict when such a change might occur, so it may be that we have to "lock" each PRRTE release to a specific PMIx one. We may also need to runtime check the version (which PMIx supports) to ensure that we are running against a compatible PMIx version since that could be different than the one we built against. All still under consideration.

OMPI updates are likely to include updates to PMIx and PRRTE, but that isn't a requirement and will almost certainly not be universally true. OMPI does have a min PMIx version requirement, and I believe they added one for PRRTE just to ensure that the mpirun cmd line meets their expectations (they have no other dependency on PRRTE). It is therefore possible that OMPI will update their min requirements for PMIx/PRRTE as releases progress - up to them, could get tricky to deal with bugs if newer OMPI is running against older PRRTE mpirun. Hoowever, those min requirements are only checked at build time, which isn't sufficient when dealing with external versions of those codes.

So the bottom line is that you have three software projects, two of which are tightly coupled and almost certainly need to be in sync. All three technically have independent release cycles, though in reality PMIx and PRRTE will almost certainly be coordinated. I think it is fair to say that the PMIx and OMPI communities are still trying to work all this out.

Hope that helps.

[amckinstry]

There is a developing practice in Debian of installing conflicting variants of something in prefix=$LIBDIR/xx/yy/
eg. we have 3 variants of hdf5, in $LIBDIR/hdf5/{serial,openmpi,mpich}.
So we could do $LIBDIR/pmix with a dedicated PRRTE vendored into it. And pterm be. in $LIBDIR/prrte/bin/pterm

[rhc54]

Sounds reasonable to me. If someone installs multiple variants, you might need some thought about how to connect OMPI to one of them based on dependencies - i.e., which variant(s) a particular OMPI release can use. We are looking at using some "capability flags" in each package to support runtime checks so we can error out with a helpful message if the wrong combination is in use.

[tazer4]

Hi, regular linux user here, I had never had nor needed to use prrte package until I was doing a system upgrade today and prrte was being installed as a dependency for openmpi package for which is a dependency for other stuff.

Anyway I got the dependency issue in package manager related to having putty installed which I actually use and I don't need prrte itself except some of my other package require it, so I have to agree with the people above saying this does affect other users and is not convenient as I can't use putty now without going around the package manager.

[bronkoo]

Hi, regular linux user here, I had never had nor needed to use prrte package until I was doing a system upgrade today and prrte was being installed as a dependency for openmpi package for which is a dependency for other stuff.

Anyway I got the dependency issue in package manager related to having putty installed which I actually use and I don't need prrte itself except some of my other package require it, so I have to agree with the people above saying this does affect other users and is not convenient as I can't use putty now without going around the package manager.

+1

[lahwaacz]

@rhc54 You have indicated reluctance to rename pterm to something like prrteterm, because pterm may be already used in many user scripts. However, this is not an issue for distributions such as Arch Linux, which are packaging prrte since only recently as a dependency for OpenMPI 5.0. Would you consider changing prrte to let downstream opt-in for this renaming when building prrte? As a related question, is pterm expected to be used only by users (i.e. actual people) or is this binary also called from tools in projects that depend on prrte, such as OpenMPI?

[rhc54]

I'd have to think about it. To me, this is a packager issue - a packager can do whatever they want with the naming, can install things into non-conflicting locations, etc. In fact, they can even choose not to install pterm at all - just a question of modifying the spec file.

Which is why I find the discussion so confusing/frustrating. I get that packagers are only now starting to package PRRTE, and that's due to the OMPI folks adoption of it. However, there are quite a few installations of it out there that were not installed via package. Disrupting them when the packagers have several tools to avoid their situation seems somewhat odd to me.

If there were no other solution, then I'd be more amenable to it. But there are several available at the packager level - just having trouble understanding why they refuse to use their own tools to solve it.

To answer your related question, pterm is called by users and their scripts. It is not generally called from inside a tool as it isn't a library function. OMPI does not call it.

[rhc54]

Anyway I got the dependency issue in package manager related to having putty installed which I actually use and I don't need prrte itself

Here's a simple solution: put the path to putty ahead of the path to OMPI's bin directory. Problem solved and nobody inconvenienced. It's a very standard method we use to resolve conflicting binary names - I'm sure you are using it right now for other things.

[bronkoo]

Here's a simple solution: put the path to putty ahead of the path to OMPI's bin directory.

Sorry, but I doesn't catched the idea? Pls. one more detail....

[smac89]

@bronkoo I think he means:

PATH='/path/to/putty/bin:/path/to/prrte/bin'

I may be missing something here, but I don't think this is how packaging works. Packages don't have separate bin directories that are installed on the path, nor do they affect a user's PATH variable.

[bronkoo]

I may be missing something here, but I don't think this is how packaging works. Packages don't have separate bin directories that are installed on the path, nor do they affect a user's PATH variable.

Thanks, but therefore I doesn't understood it...
I'm missing something here, too.

[lahwaacz]

Here's a simple solution: put the path to putty ahead of the path to OMPI's bin directory. Problem solved and nobody inconvenienced. It's a very standard method we use to resolve conflicting binary names - I'm sure you are using it right now for other things.

This will not "fix" the packaging problem. To quote #1836 (comment):

Installing two binaries with the same name (putty pterm and prrte pterm) in different directories and then using PATH is not a solution. One of them will shadow the other, breaking either one depending on the order in which the directories appear in the PATH. So unless prrte is installed in a place that does not get slurped into the PATH loading the OMPI module will create the conflict again.

But since pterm is supposed to be used in user scripts, it does not make sense to install pterm in a location that is not in PATH.

I think we will go with renaming pterm to something else in Arch Linux: https://gitlab.archlinux.org/archlinux/packaging/packages/prrte/-/merge_requests/1 Though I think we would all appreciate if this issue was properly addressed by PRRTE itself.

[rhc54]

You should never put PRRTE in a standard location, just like you don't put mpirun in a standard location. This is why we created PATH - to resolve naming conflicts, of which there are a great many.

One packager above noted that Debian has gone this route for exactly this reason.

[rhc54]

I think we will go with renaming pterm to something else in Arch Linux: https://gitlab.archlinux.org/archlinux/packaging/packages/prrte/-/merge_requests/1 Though I think we would all appreciate if this issue was properly addressed by PRRTE itself.

You are entirely free to do so - please rename the package to something other than PRRTE, though, to avoid confusing everyone else.

Though I think we would all appreciate if this issue was properly addressed by PRRTE itself.

I think I've concluded that I won't be changing it. There are simply too many easy solutions you have at your fingertips that do not require impacting everyone else - there is no reason to inconvenience those who have been working with PRRTE for the last seven years.

[rhc54]

I may be missing something here, but I don't think this is how packaging works. Packages don't have separate bin directories that are installed on the path, nor do they affect a user's PATH variable.

Yes, I agree - but that is what modules (or lmod, if you prefer) are there to handle, and that is what a user is responsible for doing. A packager is just responsible for putting the software on the machine. I realize that packagers haven't been putting things in separate locations, but that is changing with at least one packager, and I suspect others will follow. So I think this is a "problem" with a known solution that is easily implemented, and probably will become more the norm than it is today.

[bronkoo]

So I think this is a "problem" with a known solution that is easily implemented, and probably will become more the norm than it is today.

Ah, ok... misunderstood it. But I'm looking at it here from my desktop's perspective... Regarding my HPC Cluster I'm fine with the modules idea... Thanks a lot!

[Scimmia22]

Why do we see this "our software is special and has to be treated special! Screw integration!" attitude so often?

[rhc54]

Why do we see this "our software is special and has to be treated special! Screw integration!" attitude so often?

I don't think anyone is expressing that attitude. Just the opposite, really. I'm pushing for you to properly integrate things instead of trying to force uniformity across a very wide community. This keeps appearing because there are so many conflicting packages out there - which is why tools for dealing with it were developed and are so commonly used.

Better question is: why is the packaging community having such difficulty adjusting to the evolving situation? Some packagers seem to be adopting, so it isn't impossible. But change is always unsettling and takes time.

[rhc54]

I think the commentary is turning a tad pithy again, and we seem to have wrung as much out of this as we can - so I'll just close this now.

[ferdnyc]

I'd like to add something to this, if I may. My intention is not to resurrect a contentious discussion. (I agree the conversation was going nowhere up to the point @rhc54 reasonably decided to end it.) Rather, I'd like to propose an outside-the-box, long-term approach to sorting this out. It won't really help with the immediate problem, but it would mean that, eventually, it goes away for good.

The PRRTE install includes five binaries that end up in the bin directory of whatever prefix it's installed to:

Tool	Description
prte	Initiate an instance of the PMIx Reference RTE (PRRTE) DVM
prted	Start a PRTE Daemon
prte_info	Provide detailed information on your PRRTE installation.
prun	Submit job to the PMIx Reference RTE
pterm	Terminate an instance of the PMIx Reference RTE (PRRTE) DVM

Of those five, obviously prted, the daemon, is a separate entity with its own unique purpose, it wouldn't make sense to touch that.

But couldn't the other four all be subcommands of the single prte executable, instead of existing as separate binaries? I'm thinking...

prte create -- replaces prte
prte info -- replaces prte_info
prte run -- replaces prun
prte term / prte stop -- replaces pterm

If the tools were consolidated that way, after a deprecation period the separate binaries could be removed from the installation, and the conflicts would cease to exist.

It'd be a slow solution, but ultimately a complete one.

And arguably, one that makes the toolset more unified and discoverable as a side benefit. (If the binaries are installed to /usr/bin/, how many people are going to pick up on the fact that something called prun or pterm is a PRRTE utility? Other than by taking the time to directly examine it, I mean.)

[rhc54]

Hmmm...well, that's the most reasonable alternative someone has put forward - thanks! Let me ponder it a bit. The OMPI folks are reusing prte as their mpirun, with some added logic to deal with differences in behavior. I'd need to ensure those details didn't conflict with this proposal.

Barring that, my initial inclination would be to go with it, or at least something close. I don't think anyone was expecting to see prte binaries in /usr/bin as that just seems problematic for a number of reasons. Still, if someone really wants to do that, I agree that they won't know the association between the commands.

[ferdnyc]

@rhc54

prte create could also be the default mode, for backwards compatibility.

So if it's called with what would become the "legacy" semantics (i.e. the ones it has right now -- no subcommand, and flags appropriate for prte create), then it works exactly the same as it does today.

[rhc54]

Agreed. I would have to modify your proposal a bit to make it work with the mpirun scenario - have to change the keywords to cmd line options (i.e., "create" becomes "--create"). Still, that's a minor modification.

Will ponder and play a bit. Probably going to be the fall before anything really happens as my time is rather limited and my "to-do" list has a few things already on it.

[rhc54]

Will leave this open so I see it later in the year as a reminder.