Author Topic: Creating some apps RPM packages  (Read 6286 times)

Ben Rietbroek

  • Newbie
  • *
  • Thank You
  • -Given: 0
  • -Receive: 0
  • Posts: 14
  • Karma: +0/-0
    • View Profile
Re: Creating some apps RPM packages
« Reply #45 on: March 02, 2021, 05:28:49 pm »

== Info on the V.O.I.C.E RPM Packaging Engine ==

The attached dt6202103021446.html.zip will give a first impression
of the automated package building being worked on.

Andreas Schnellbacher

  • Hero Member
  • *****
  • Thank You
  • -Given: 1
  • -Receive: 39
  • Posts: 724
  • Karma: +12/-0
    • View Profile
Re: Creating some apps RPM packages
« Reply #46 on: March 02, 2021, 06:17:27 pm »
Ben, that would be a large step forward. I guess most of us don't like to place usual OS/2 apps in the UNIXROOT tree. I didn't know that using OS/2 env vars makes a problem as Martin has pointed out.

Automating the build process is even better. I look forward to your project.

Ben Rietbroek

  • Newbie
  • *
  • Thank You
  • -Given: 0
  • -Receive: 0
  • Posts: 14
  • Karma: +0/-0
    • View Profile
Re: Creating some apps RPM packages
« Reply #47 on: March 02, 2021, 11:37:27 pm »

Greetings, force behind the NEPMD 8) Project ! 
I'll be picking a few cherries from your works Andreas ...
« Last Edit: March 02, 2021, 11:42:25 pm by Ben Rietbroek »

Ben Rietbroek

  • Newbie
  • *
  • Thank You
  • -Given: 0
  • -Receive: 0
  • Posts: 14
  • Karma: +0/-0
    • View Profile
Re: Creating some apps RPM packages
« Reply #48 on: June 18, 2021, 11:08:17 pm »

== Update on V.O.I.C.E Developments ==

Silence since my last post
When I am silent it means I'm working... :P
In not too long a time you can follow developments using the 3 websites mentioned
further below in this post.

Availability of the [collab] sub-domain
I was waiting for this sub-domain to be registered, which now has been done.
This is where the Ā«collaborationĀ» website will be hosted.
Its purpose is to provide information on how we can join hands to work together
to realize common goals. More information will be presented on the website itself.

The promised [spectest] example
My first idea was to create a stand-alone spectest example so people
can fiddle with the rpmbuild tool. But later I realized that the whole
idea is to hide the rpmbuild complexity. So I decided that working on
an EPM-variant instead would be better time spent. The spectest example
will most probably be part of the demo-variant -- see below.

EPM Variants
An EPM-variant is a customized version of EPM.
I am first working on a demo-variant and later there will come a specific variant that
focusses on building packages using more high-level interfaces.

V.O.I.C.E Realms
The development work is divided into three main divisions:
EngineeringDevelopment
CollaborationWorking Together
ProvisioningProviding Packaged Software

Warpstock-EU Presentation (2021)
The underlying concepts of Software Provisioning, together with some more glimpses
under the hood can be found here:
http://collab.os2voice.org/?wseu2021

Greetz,
  Ben.

« Last Edit: June 18, 2021, 11:44:21 pm by Ben Rietbroek »

Doug Clark

  • Full Member
  • ***
  • Thank You
  • -Given: 1
  • -Receive: 22
  • Posts: 230
  • Karma: +2/-0
    • View Profile
Re: Creating some apps RPM packages
« Reply #49 on: June 21, 2021, 06:36:51 pm »
I think the idea of automating, as much as possible, installations is great. 

My biggest demotivation for upgrading AOS versions is the very long time it takes to get all the applications and tools reinstalled and configured after doing a fresh install of AOS  (or ECS or OS/2).

My I make the following observation and suggestion - without knowing how difficult they are to implement?

Prompt the user for where a package/app should be installed.  If using YUM perhaps either a command line parameter to YUM for the install location or a standard environmental variable (e.g. YUM_INSTALL_LOC= ) that YUM would check for the drive/directory to hold the app.  For APM a dialog prompting for location to install at the start of a new install.

Regarding LIBPATH length issues:   For GUI applications, on machines running DragText (which I assume is everyone,)  you can use the Environment page of the Properties notebook of the Program object for the GUI app to set BEGINLIBPATH, avoiding the need to modify LIBPATH.  Each GUI application really should have a Program object anyway.   We would have to persuade Rich Walsh to add a setup string to his DTProgram object for the Environment page/setting in order to use this scheme with automated installs.

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 13
  • -Receive: 226
  • Posts: 3164
  • Karma: +59/-0
    • View Profile
Re: Creating some apps RPM packages
« Reply #50 on: June 21, 2021, 08:08:54 pm »
Hi Doug, couple of points.
At this point, all new versions of AOS should update or, in the case of 5.0.x to 5.1.x upgrade, so everything should keep the same configuration and whatever you have installed are still installed. There's a couple of gotchas like the update procedure leaving %TMP% pointed at x:\var\temp that need a quick config.sys edit.
YUM and the ANPM front end use %UNIXROOT% to decide where to install the RPM apps. As originally design %UNIXROOT% could contain a PATH but since the RPM people took it over, only a drive letter is supported. A PATH still might work but is untested.
During the install there is a point where you are asked about %HOME%, %UNIXROOT%, %TEMP% and IIRC another with the defaults pointing at the install volume. You can accept the defaults or change them.
For the LIBPATH issue, either your method of setting the BEGINLIBPATH in the program object or alternately, a script that sets BEGINLIBPATH etc which I usually use. Good practice to inherit any previous BEGINLIBPATH settings to like SET BEGINLIBPATH=x:\foo\lib;%BEGINLIBPATH%
There's also ENDLIBPATH for the odd situation, I use it for Gopher and Webexplorer with my Ver 4 \tcpip\lib etc coming last in LIBPATH so only DLLs that aren't already present on AOS are loaded.

ivan

  • Hero Member
  • *****
  • Thank You
  • -Given: 2
  • -Receive: 52
  • Posts: 1322
  • Karma: +13/-0
    • View Profile
Re: Creating some apps RPM packages
« Reply #51 on: June 21, 2021, 09:28:03 pm »
I have never understood the reason for using RPM/YUM as an installer in OS/2/ArcaOS. 

We have a very good installer in WARPIN, it can find where older versions of programs are, make new directories and add them to the LIBPATH and PATH and so on.  I assume it was discarded because some people thought writing the install script was too much effort when they could dump everything in the same directory - not a good idea.

I too do not update my install unless it is something I can download and install myself.  The last time I opened the yum package manager (had to look up what it is called) was when we were told about the QT5 port - couldn't find a full runtime so gave up looking.

All my tools are in subdirectories of the 'tools' directory, likewise utilities.  Those two directories are stored on my NAS boxes and get copied over to any new install of OS/2 / ArcaOS and I will continue doing so as long as they work on my computers, I have had to drop a couple of utilities because the MMRE driver does not communicate lan speed and throughput in a form the programs expect.

Doug Bissett

  • Hero Member
  • *****
  • Thank You
  • -Given: 1
  • -Receive: 62
  • Posts: 1468
  • Karma: +2/-2
    • View Profile
Re: Creating some apps RPM packages
« Reply #52 on: June 21, 2021, 10:28:58 pm »
Quote
I have never understood the reason for using RPM/YUM as an installer in OS/2/ArcaOS.

It is very simple. That is the way some of it is being done, by choice of those who do the work. They have no intention of changing that.

Quote
We have a very good installer in WARPIN, it can find where older versions of programs are, make new directories and add them to the LIBPATH and PATH and so on.  I assume it was discarded because some people thought writing the install script was too much effort when they could dump everything in the same directory - not a good idea.

WarpIn has not been abandoned, it is still used when the person making the package wants to do it that way (many packages in ArcaOS are installed using WarpIn). It can also be done with ZIP files. The choice depends a LOT on what is being packaged, and what the dependencies are.

Quote
I too do not update my install unless it is something I can download and install myself.

VERY BAD idea. I can't emphasize that enough. The YUM package manager does a LOT more than simply install things (use the Arca Noae Package Manager - ANPM to make it simple, and get it done right). If you don't use RPM/YUM, you can be missing parts of what is required (plus you can end up spending DAYS trying to sort it out). It also makes sure that all of the parts end up in the proper place (it is NOT just a simple target). Many Linux ports simply assume locations, so even if you have something, somewhere, it may not be found, if it is not where it is assumed to be.

RPM/YUM is just another package installer, made MUCH easier to use, when you use ANPM to run it (and ANPM has more features coming). It can be somewhat tricky to convert an older system to use  RPM/YUM (there are way too many things that users did before, that can mess it up). It is easy if you use it from a new install, and NEVER try to outsmart it.

Doug Clark

  • Full Member
  • ***
  • Thank You
  • -Given: 1
  • -Receive: 22
  • Posts: 230
  • Karma: +2/-0
    • View Profile
Re: Creating some apps RPM packages
« Reply #53 on: June 22, 2021, 01:31:28 am »
Dave:  Thanks for the info on updating AOS.  I will give the latest version a shot.  (Now if it were only that easy to update from eCS to AOS - oh well..)

Ivan:  I too love WarpIn - great product, although it needs a better database for storing installed app information;  the 2 levels that INI files provides is not enough.  But I can understand and agree with RPM for the unixy stuff.  That stuff is SO intertwined and complicated.  And I am actually OK with UNIXROOT being a drive letter instead of a path for the messy, snakey, interdependent unixy stuff.  That stuff is so complicated that I am kinda scared to even look down in the depths of UNIXROOT.

What Martin was talking about was RPM-izing legacy type OS/2 apps.   That has lots of benefits, and some downsides.  It is legacy apps (and SOME unix type apps like Firefox) that I want the ability to chose where stuff goes.  Here's  why:  I run a number of Win-OS/2 apps. That means I really need to keep my boot drive partitioned at 2 GB or less and I need to be careful about what I install on the boot drive so I don't run out of space.  I also am running SSD and rotating disks and I try and put the stuff that changes a lot (Swapper, firefox cache, etc.) on the rotating disk and keep the test on the SSD partitions.

So I think we absolutely need the ability, at least with legacy apps, to chose where they go.  And it would be nice to have the ability to choose even with some unix apps.

The downside of RPM verses WarpIn is it looks like RPM is much more complicated to setup.  I mean complicated for the app maker to setup an install to work with RPM.  But it is nice to be able to select a list of apps (like with APM) and install them all at once.. And for them to be available from one (virtual) locate.


Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 13
  • -Receive: 226
  • Posts: 3164
  • Karma: +59/-0
    • View Profile
Re: Creating some apps RPM packages
« Reply #54 on: June 22, 2021, 02:42:53 am »
@Doug, there is hope that Arca Noae will add the ability to upgrade eCS and even Warp V4 to AOS 5.1. Whether it gets done is questionable, with the Warp v4 upgrade unlikely to happen.
As for Firefox, which is more of a cross platform app then a *nix app, you can just move it. Likewise for most of the programs included with AOS and installed under \Programs (check config.sys though).

@Ivan, the problem with Warpin is it doesn't handle dependencies too well, especially updates to those dependencies. Throwing everything under one directory is fine as long as you always use YUM or ANPM to manage them. Doug B. explained most of the benefits and pitfalls of not using it. Installing QT5 is as simple (or should be) as yum install qt5 and it pulls in all dependencies.

Doug Bissett

  • Hero Member
  • *****
  • Thank You
  • -Given: 1
  • -Receive: 62
  • Posts: 1468
  • Karma: +2/-2
    • View Profile
Re: Creating some apps RPM packages
« Reply #55 on: June 22, 2021, 04:34:59 am »
Quote
although it needs a better database for storing installed app information;  the 2 levels that INI files provides is not enough.

What is missing?

Quote
What Martin was talking about was RPM-izing legacy type OS/2 apps.

Which is okay for learning how to do it. I hope he doesn't actually try to distribute old style OS/2 apps that way. It is not appropriate. Some of them are not even appropriate for WarpIn.

Quote
That means I really need to keep my boot drive partitioned at 2 GB or less and I need to be careful about what I install on the boot drive so I don't run out of space.

Actually, that is not a restriction. You should use https://hobbes.nmsu.edu/download/pub/dos/2gbfix.zip, possibly with the patch to lower the returned space even further https://hobbes.nmsu.edu/download/pub/dos/2GBFIXPA.ZIP. It doesn't matter how big the drive is, the biggest number that DOS will "see" is slightly less than 2 GB. 2gbfix is included in ArcaOS. DOS can only "see" 8.3 file names too. All others are invisible.

Quote
The downside of RPM verses WarpIn is it looks like RPM is much more complicated to setup.  I mean complicated for the app maker to setup an install to work with RPM.

RPM is "more complicated" because it usually needs far more links to dependencies, but the person who builds the program usually knows what is required, so it is easier for them to do it. Other than that, it is just a "learning curve". WarpIn can burn you pretty bad, if you don't get it right, and it doesn't handle dependencies, which means the user can also mess it up. RPM/YUM can also burn you, if it isn't done properly.

Neither installer is "perfect". They are what they are. The main purpose of both, is to make it easy for a user to get a program installed correctly, so it will work. In most (unfortunately, not all) cases both work as they should.

One of the reasons why I object to restoring old programs to HOBBES, is that some of them were removed because the WarpIn installer was doing bad things.

Doug Clark

  • Full Member
  • ***
  • Thank You
  • -Given: 1
  • -Receive: 22
  • Posts: 230
  • Karma: +2/-0
    • View Profile
Re: Creating some apps RPM packages
« Reply #56 on: June 22, 2021, 06:17:34 am »
@Doug B  sort of quoting "What's wrong with 2 levels in the WarpIn database?"

The problem with using a database that only has 2 levels to track installed products (like WarpIn) is if the product has lots of components the "application" is listed multiple times with different "keys" in order to list the different components or subsystems.  If you want to deinstall the product you have to deinstall each of the application-key pairs. To use an example: Doodle's Screen Saver is listed as an application 29 times (on my machine) in WarpIn's database. So to deinstall one has to right-click 29 times on the application and select deinstall.

To repeat: I like WarpIn.  I think it is a great product.  Installing applications can be surprisingly difficult and complex - especially when designing to allow for updating.  And WarpIn, on the whole, does an excellent job.

@Doug B. and @Dave   Yes I can move firefox after it is installed, and could probably move some other Unixy applications installed by RPM after they are installed.  It would not be the end of the world - although moving some of them would be very difficult (e.g. Odin.).  And yes I could make a boot partition larger than 2 gb and use 2gbfix to work around the issue for those old Dos/Windows apps.  But the issue is why should I have to?  When the better way is to simply give the user the choice on where to install a product when using APM/YUM/RPM. 

It just seems to me (my opinion, FWIW) that any/all decent installers should allow the user to select the destination of a application being installed, when that is possible.  Obviously some system components have to go in specific locations - but for those that don't, why not provide the choice?

@Doug B.  I think Martin IS intending to distribute old style OS/2 apps that way.  And I think it is an excellent idea if we can get some flexibility in the install process.

I spent quite a bit of time years ago on the WarpDoctor project in an attempt to build something like APM for OS/2 VOICE.  I even spent time modifying WarpIn so that it would retrieve packages across the internet and install them on a local machine in that effort - and gave up.  It was, and is, a very big project.  I am fully in and supportive of APM and what Martin and Ben are attempting to do, and I recognize how difficult and complex it is.  Thank you Martin, and Ben, and everyone else working on this, for your time and effort.

I am just trying to, politely, suggest that adding the ability to select a destination for a application when installing with APM/YUM/RPM would be a fantastic feature.

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 13
  • -Receive: 226
  • Posts: 3164
  • Karma: +59/-0
    • View Profile
Re: Creating some apps RPM packages
« Reply #57 on: June 22, 2021, 07:31:11 am »
Unluckily RPM (and YUM and ANPM are just frontends for RPM) was designed for Linux, following the *nix tradition of minimal file directories below root, along with the capability of having them on different partitions/disks. I also think it was designed to collect consulting money. To hack it to allow installing programs in different places would probably be hard if not next to impossible.
BTW, to uninstall the screensaver takes 3 or 4 clicks, click the first package, scroll down and shift-click the last, right click and choose uninstall and perhaps another click to verify that you want to uninstall. And even with all those packages, it is still hard and probably would take more packages to make it NLV aware on the desktop. Welcome to ideas how to do that.
Also the screensaver is an example of other shortcomings, it needs a few RPM packages installed and currently there is no way to force those packages to be installed.
Another interesting problem going forward is that AOS is going to have several localized versions to complicate things.

Doug Bissett

  • Hero Member
  • *****
  • Thank You
  • -Given: 1
  • -Receive: 62
  • Posts: 1468
  • Karma: +2/-2
    • View Profile
Re: Creating some apps RPM packages
« Reply #58 on: June 22, 2021, 08:26:30 am »
Quote
The problem with using a database that only has 2 levels to track installed products (like WarpIn) is if the product has lots of components the "application" is listed multiple times with different "keys" in order to list the different components or subsystems.  If you want to deinstall the product you have to deinstall each of the application-key pairs. To use an example: Doodle's Screen Saver is listed as an application 29 times (on my machine) in WarpIn's database. So to deinstall one has to right-click 29 times on the application and select deinstall.

That is true. Back when WarpIn was developed, it was unusual to have more than one component. Three, at the most, so it wasn't much of a problem. If you want an example of a complicated WarpIn installer, have a look at the GENMAC installer. One of the "problems" with WarpIn, is that it is very flexible, and can easily be augmented using REXX. Lots of people have invented ways to use that, and sometimes it can cause problems.

You can select more than one package, using the standard selection methods. If it was packaged in a way that keeps all of the parts together, that is pretty easy. To use DSS as an example, click the first entry, move to the last entry and Shift click. They will all be selected, and you can RMB to uninstall them all. I suspect that it would be far more work, than can be justified, to change that, and it would probably break a lot of things that work now.

Quote
But the issue is why should I have to?  When the better way is to simply give the user the choice on where to install a product when using APM/YUM/RPM.

You do have the choice. Very few things must be on the boot volume. Back when DOS, and WinOS2, support was created, a 10 MB (yeah, "MB") hard disk was HUGE. The disk size did not become a problem until disk larger than 2 GB became available. IBM never fixed the problem, so somebody developed 2GBfix to avoid the problem (there are a couple of others, but 2GBfix works best). If you don't want to use that, it is your choice. Probably the best approach is to have a small partition dedicated to DOS/WinOS2 programs. That could be the C: drive, if you make your boot drive something other than C:. Another way to avoid using space on the boot drive, is to put the RPM/YUM stuff on a different drive.

Quote
When the better way is to simply give the user the choice on where to install a product when using APM/YUM/RPM.

Using WarpIn, you can select where to put things, but it usually doesn't make sense to do that. The default locations (as defined in the WarpIn settings, which I usually change, to my own preferences, at OS install time), are usually best. I assume you meant ANPM, not APM (a power manager) with RPM/YUM. You need to realize that RPM/YUM is designed to work with Linux, which usually has fixed (not selectable) locations for things. A lot of that was never changed, because it makes it easier to port programs from Linux. In fact, there are very few programs that actually get installed using RPM/YUM. Almost all of that stuff is support for Linux ports, which are often installed by WarpIn. There is a RPM package for Firefox, but I doubt if it will ever be pushed to the release  channel, simply because Firefox is no longer being ported for OS/2. From what I hear, the new browser (Otter) will be an RPM package (probably with no other options for installing, because of the complexity involved).

Quote
I think Martin IS intending to distribute old style OS/2 apps that way.  And I think it is an excellent idea if we can get some flexibility in the install process.

Old style OS/2 apps are not suited to distribution as RPM packages, and it would be a lot of work to do it, for little, or no, gain. Other things could use a little polish, and it would probably be easier to repackage the very old programs as WarpIn packages (many of them just need to place the files, and add an icon). RPM/YUM is useful when there are many *NIX requirements. It is not as useful when it is a stand alone program, but it should be possible to do it.

Quote
I am just trying to, politely, suggest that adding the ability to select a destination for a application when installing with APM/YUM/RPM would be a fantastic feature.

Perhaps so, but that would make it a LOT more difficult to port RPM and YUM (ANPM just uses some intelligence to run YUM). They are not designed to do things like that, and changing too many things tends to break other things. In fact, it is probably possible, if the the person building the package can find a way to do it. I understand that scripts are used to do some things, perhaps they can ask where to put things, when the package might actually work in non standard locations. The main "problem" is that doing things like that, makes supporting the whole thing a lot more complex, and one thing that we don't have, are a lot of support people to try to figure out what some user has done to themselves. It happens often enough when it is a supported configuration. The other problem is that some future change might make whatever was done fail to work. Then, who is going to figure that out? There just aren't enough people to do it all.

Doug Clark

  • Full Member
  • ***
  • Thank You
  • -Given: 1
  • -Receive: 22
  • Posts: 230
  • Karma: +2/-0
    • View Profile
Re: Creating some apps RPM packages
« Reply #59 on: June 22, 2021, 04:16:59 pm »
@Doug B and Dave:  Ah - extended selection in WarpIn when uninstalling.  It is so obvious and yet somehow never occurred to me.  Thanks for the tip.

@Doug B:  One of Walter Metcalf's big concerns (and just about everyone else's) was the continued existence of Hobbes.   To me a potential benefit of what Martin is doing (if I understand it correctly) would address a couple of those concerns: having another copy of (at least some of) those files, and at a location that is accessible.   I think it is beneficial, even if it is not possible or realistic to add a "select your own destination" feature. 

As for enhancing WarpIn with a new database: I don't think (famous last words)  it would be that difficult to switch the database from INI files to something else that will handle more levels.   Anyone know what database RPM uses?