OS/2, eCS & ArcaOS - Technical > Setup & Installation

sigh....Python3 upgrade - curl problem?

(1/3) > >>

Dariusz Piatkowski:
So this whole 'upgrade to python3' is by now a well traversed path, I did my reading, took to the CLI approach of executing: 'yum install python3 python2.7', which gave me the following results:


--- Code: ---[G:\]yum install python3 python2.7
Loaded plugins: changelog, downloadonly, ps, replace, verify
Setting up Install Process
Resolving Dependencies
There are unfinished transactions remaining. You might consider running yum-comp
lete-transaction first to finish them.
--> Running transaction check
---> Package python2.7.pentium4 0:2.7.18-3.oc00 will be installed
--> Processing Dependency: python-rpm-macros for package: python2.7-2.7.18-3.oc00.pentium4
--> Processing Dependency: python-srpm-macros for package: python2.7-2.7.18-3.oc00.pentium4
---> Package python3.pentium4 0:3.9.17-1.oc00 will be installed
--> Processing Dependency: python3-libs = 3.9.17-1.oc00 for package: python3-3.9.17-1.oc00.pentium4
--> Processing Dependency: python39.dll for package: python3-3.9.17-1.oc00.pentium4
--> Running transaction check
---> Package python-libs.pentium4 0:2.7.6-25.oc00 will be obsoleted
--> Processing Dependency: python-libs = 2.7.6-25.oc00 for package: python-2.7.6-25.oc00.pentium4
---> Package python-rpm-macros.noarch 0:3.9-3.oc00 will be installed
---> Package python-srpm-macros.noarch 0:3.9-3.oc00 will be installed
---> Package python3-libs.pentium4 0:3.9.17-1.oc00 will be obsoleting
--> Running transaction check
---> Package python.pentium4 0:2.7.6-25.oc00 will be obsoleted
---> Package python-unversioned-command.pentium4 0:3.9.17-1.oc00 will be obsoleting
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================

 Package                      Arch       Version            Repository     Size
================================================================================

Installing:
 python-unversioned-command   pentium4   3.9.17-1.oc00      netlabs-rel   8.6 k
     replacing  python.pentium4 2.7.6-25.oc00
 python2.7                    pentium4   2.7.18-3.oc00      netlabs-rel    18 M
 python3                      pentium4   3.9.17-1.oc00      netlabs-rel    22 k
 python3-libs                 pentium4   3.9.17-1.oc00      netlabs-rel    13 M
     replacing  python-libs.pentium4 2.7.6-25.oc00
Installing for dependencies:
 python-rpm-macros            noarch     3.9-3.oc00         netlabs-rel   9.1 k
 python-srpm-macros           noarch     3.9-3.oc00         netlabs-rel    15 k

Transaction Summary
================================================================================

Install       6 Packages

Total download size: 30 M
Is this ok [y/N]:

INPUT => Y

Downloading Packages:
(1/6): python-rpm-ma | 9.1 kB  00:00
(2/6): python-srpm-m |  15 kB  00:00
(3/6): python-unvers | 8.6 kB  00:00
(4/6): python2.7-2.7 |  18 MB  00:07     ====-] 2.1 MB/s |  17 MB  00:00 ETA
(5/6): python3-3.9.1 |  22 kB  00:00
(6/6): python3-libs- |  13 MB  00:05     ==== ] 1.9 MB/s |  12 MB  00:00 ETA
--------------------------------------------------------------------------------

Total                                           2.2 MB/s |  30 MB     00:13

Running Transaction Check
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
  Installing : python-srpm-macros-3.9-3.oc00.noarch                         1/8
  Installing : python3-libs-3.9.17-1.oc00.pentium4                          2/8
  Installing : python3-3.9.17-1.oc00.pentium4                               3/8
  Installing : python-rpm-macros-3.9-3.oc00.noarch                          4/8
  Installing : python2.7-2.7.18-3.oc00.pentium4                             5/8
  Installing : python-unversioned-command-3.9.17-1.oc00.pentium4            6/8
  Erasing    : python-2.7.6-25.oc00.pentium4                                7/8
  Erasing    : python-libs-2.7.6-25.oc00.pentium4                           8/8

Rpmdb checksum is invalid: dCDPT(pkg checksums): python3-libs.pentium4 0:3.9.17-1.oc00 - u

--- End code ---

Following this upgrade my inventory of python RPM packages installed is as follows:


--- Code: ---Installed Packages
python-dateutil.noarch                        2.6.0-1.oc00           installed
python-pycurl.pentium4                        7.19.5.1-2.oc00        installed
python-rpm-macros.noarch                      3.9-3.oc00             installed
python-srpm-macros.noarch                     3.9-3.oc00             installed
python-unversioned-command.pentium4           3.9.17-1.oc00          installed
python2-deltarpm.pentium4                     3.6-1.oc00             installed
python2-libxml2.pentium4                      2.9.10-3.oc00          installed
python2-rpm.pentium4                          4.13.0-20.oc00         installed
python2.7.pentium4                            2.7.18-3.oc00          installed
python3.pentium4                              3.9.17-1.oc00          installed
python3-libs.pentium4                         3.9.17-1.oc00          installed

--- End code ---

Therefore, my next logical (and maybe this was a mistake???) step was to upgrade all the pythonx stuff to python3, which I went to ANPM for:

1) python3-dateutil - this seems to have gone fine

--- Code: -------------[ 10 Dec 2023 10:26:12 ]----------
Executing: @python2.7 G:\UTIL\ANPM\scripts\yum_install_dep.py python3-dateutil
Enabling temporary repository arcanoae-sub
Enabling temporary repository arcanoae-exp
Enabling temporary repository arcanoae-stage
Enabling temporary repository arcanoae-arcaos
No argument pipe.
Return code: 0
----------[ 10 Dec 2023 10:26:25 ]----------
Executing: @python2.7 G:\UTIL\ANPM\scripts\yum_install.py python3-dateutil
No argument pipe.
Enabling temporary repository arcanoae-sub
Enabling temporary repository arcanoae-exp
Enabling temporary repository arcanoae-stage
Enabling temporary repository arcanoae-arcaos
Running Transaction Check
Warning: RPMDB altered outside of yum.
PackageSackError(): Rpmdb checksum is invalid: dCDPT(pkg checksums): python3-six.noarch 0:1.10.0-2.oc00 - u
Error: Rpmdb checksum is invalid: dCDPT(pkg checksums): python3-six.noarch 0:1.10.0-2.oc00 - u
PackageSackError()
Return code: 0
----------[ 10 Dec 2023 10:26:41 ]----------
Executing: @python2.7 G:\UTIL\ANPM\scripts\yum_list.py
Setting up yum
No argument pipe.
Adding temporary repositories
Enabling temporary repository arcanoae-sub
Enabling temporary repository arcanoae-exp
Enabling temporary repository arcanoae-stage
Enabling temporary repository arcanoae-arcaos
Initializing cache
Requesting package list
Return code: 0
----------[ 10 Dec 2023 10:27:05 ]----------
Executing: @python2.7 G:\UTIL\ANPM\scripts\yum_info.py python3-six
Warning: progress pipe not available.
Enabling temporary repository arcanoae-sub
Enabling temporary repository arcanoae-exp
Enabling temporary repository arcanoae-stage
Enabling temporary repository arcanoae-arcaos
No argument pipe.
Return code: 0
----------[ 10 Dec 2023 10:27:17 ]----------
Executing: @python2.7 G:\UTIL\ANPM\scripts\yum_files.py installed python3-six
Enabling temporary repository arcanoae-sub
Enabling temporary repository arcanoae-exp
Enabling temporary repository arcanoae-stage
Enabling temporary repository arcanoae-arcaos
No argument pipe.
Return code: 0
----------[ 10 Dec 2023 10:27:51 ]----------
Executing: @python2.7 G:\UTIL\ANPM\scripts\yum_files.py installed python-unversioned-command
Enabling temporary repository arcanoae-sub
Enabling temporary repository arcanoae-exp
Enabling temporary repository arcanoae-stage
Enabling temporary repository arcanoae-arcaos
No argument pipe.
Return code: 0
----------[ 10 Dec 2023 10:28:02 ]----------
Executing: @python2.7 G:\UTIL\ANPM\scripts\yum_info.py python-unversioned-command
Warning: progress pipe not available.
Enabling temporary repository arcanoae-sub
Enabling temporary repository arcanoae-exp
Enabling temporary repository arcanoae-stage
Enabling temporary repository arcanoae-arcaos
No argument pipe.
Return code: 0

--- End code ---

2) python3-curl - this seems to have NOT gone fine

--- Code: -------------[ 10 Dec 2023 10:29:24 ]----------
Executing: @python2.7 G:\UTIL\ANPM\scripts\yum_install_dep.py python3-pycurl
Enabling temporary repository arcanoae-sub
Enabling temporary repository arcanoae-exp
Enabling temporary repository arcanoae-stage
Enabling temporary repository arcanoae-arcaos
No argument pipe.
Return code: 0
----------[ 10 Dec 2023 10:29:32 ]----------
Executing: @python2.7 G:\UTIL\ANPM\scripts\yum_install.py python3-pycurl
No argument pipe.
Enabling temporary repository arcanoae-sub
Enabling temporary repository arcanoae-exp
Enabling temporary repository arcanoae-stage
Enabling temporary repository arcanoae-arcaos
Running Transaction Check
Warning: RPMDB altered outside of yum.
PackageSackError(): Rpmdb checksum is invalid: dCDPT(pkg checksums): python3-pycurl.pentium4 0:7.44.1-3.oc00 - u
Error: Rpmdb checksum is invalid: dCDPT(pkg checksums): python3-pycurl.pentium4 0:7.44.1-3.oc00 - u
PackageSackError()
Return code: 0
----------[ 10 Dec 2023 10:29:58 ]----------
Executing: @python2.7 G:\UTIL\ANPM\scripts\yum_list.py
Traceback (most recent call last):
  File "G:\UTIL\ANPM\scripts\yum_list.py", line 6, in <module>
    import yum
  File "G:/usr/lib/python2.7/site-packages/yum/__init__.py", line 52, in <module>
    import config
  File "G:/usr/lib/python2.7/site-packages/yum/config.py", line 30, in <module>
    from parser import ConfigPreProcessor, varReplace
  File "G:/usr/lib/python2.7/site-packages/yum/parser.py", line 4, in <module>
    import urlgrabber
  File "G:/usr/lib/python2.7/site-packages/urlgrabber/__init__.py", line 55, in <module>
    from grabber import urlgrab, urlopen, urlread
  File "G:/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 512, in <module>
    import pycurl
ImportError: No module named pycurl
Return code: 0
----------[ 10 Dec 2023 10:30:00 ]----------
Executing: @python2.7 G:\UTIL\ANPM\scripts\yum_list.py available
Traceback (most recent call last):
  File "G:\UTIL\ANPM\scripts\yum_list.py", line 6, in <module>
    import yum
  File "G:/usr/lib/python2.7/site-packages/yum/__init__.py", line 52, in <module>
    import config
  File "G:/usr/lib/python2.7/site-packages/yum/config.py", line 30, in <module>
    from parser import ConfigPreProcessor, varReplace
  File "G:/usr/lib/python2.7/site-packages/yum/parser.py", line 4, in <module>
    import urlgrabber
  File "G:/usr/lib/python2.7/site-packages/urlgrabber/__init__.py", line 55, in <module>
    from grabber import urlgrab, urlopen, urlread
  File "G:/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 512, in <module>
    import pycurl
ImportError: No module named pycurl
Return code: 0
----------[ 10 Dec 2023 10:39:59 ]----------
Executing: @python2.7 G:\UTIL\ANPM\scripts\yum_populate.py
Traceback (most recent call last):
  File "G:\UTIL\ANPM\scripts\yum_populate.py", line 6, in <module>
    import yum
  File "G:/usr/lib/python2.7/site-packages/yum/__init__.py", line 52, in <module>
    import config
  File "G:/usr/lib/python2.7/site-packages/yum/config.py", line 30, in <module>
    from parser import ConfigPreProcessor, varReplace
  File "G:/usr/lib/python2.7/site-packages/yum/parser.py", line 4, in <module>
    import urlgrabber
  File "G:/usr/lib/python2.7/site-packages/urlgrabber/__init__.py", line 55, in <module>
    from grabber import urlgrab, urlopen, urlread
  File "G:/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 512, in <module>
    import pycurl
ImportError: No module named pycurl
Return code: 0
----------[ 10 Dec 2023 10:40:01 ]----------
Executing: @python2.7 G:\UTIL\ANPM\scripts\yum_list.py
Traceback (most recent call last):
  File "G:\UTIL\ANPM\scripts\yum_list.py", line 6, in <module>
    import yum
  File "G:/usr/lib/python2.7/site-packages/yum/__init__.py", line 52, in <module>
    import config
  File "G:/usr/lib/python2.7/site-packages/yum/config.py", line 30, in <module>
    from parser import ConfigPreProcessor, varReplace
  File "G:/usr/lib/python2.7/site-packages/yum/parser.py", line 4, in <module>
    import urlgrabber
  File "G:/usr/lib/python2.7/site-packages/urlgrabber/__init__.py", line 55, in <module>
    from grabber import urlgrab, urlopen, urlread
  File "G:/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 512, in <module>
    import pycurl
ImportError: No module named pycurl
Return code: 0

--- End code ---

Sure enough, ANPM is no longer able to pull anything up, reports a pipe error, although from CLI I get the following:


--- Code: ---[G:\]yum list python* |less
There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:

   No module named pycurl

Please install a package which provides this module, or
verify that the module is installed correctly.

It's possible that the above module doesn't match the
current version of Python, which is:
2.7.18 (default, Dec 27 2021, 15:11:56)
[GCC 9.2.0 20190812 (OS/2 RPM build 9.2.0-5.oc00)]

If you cannot solve this problem yourself, please go to
the yum faq at:
  http://yum.baseurl.org/wiki/Faq

--- End code ---

So it would appear that installing python3-pycurl obsoleted python-pycurl (which I would have expected is the correct thing to do), but for some reason ANPM continues to see python 2.7.18 as the version it is executing.

From any CLI when checking for the installed Python3 version I get the following:


--- Code: ---[G:\]python --version
Python 3.9.17

--- End code ---

...so clearly I'm running Python3, but ANPM is still using 2.7.18, is that by design?

Either way, any ideas how I can get back to a working YUM/RPM/ANPM config?...and what (if anything) do I do about the remaining pythonx RPM packages (do I upgrade, or leave alone)?

OH, BTW: a system reboot was executed to rule out any other impacts, results are the same

Dave Yeo:
Hmm, I never forced any Python upgrades besides the python2.7/python3 one and see I have python2-pycurl and python3-pycurl installed. Lots of things are python2 including ANPM.
If yum itself works, install python2-curl or perhaps download the zip and install the 8 files manually to get ANPM working then install the package.

JTA:
From the perspective of a new install of ArcaOS, there seems to be a difficulty in applying ALL  outstanding ANPM updates at once, as the python issue pops up (2.x and 3.x aren't in place all the way yet?).

There are supposedly help articles on AN support pages to get you around this, but I just basically did a few specific updates by hand (in ANPM), and got it going again. IIRC, it was something like:

- update only the base 2.x python
- then update only the base 3.x python

Can't quite remember which one of the two I did first, but I think the trick was doing them separately ... after getting both of those to succeed, the last step is a mass ANPM update of anything else remaining, which also then worked for me.

I just stumbled through that, so didn't document each and every step, and assumed that what was on AN's support pages was a much better write-up.

Hope this helps ...

Martin Iturbide:
Hello


--- Quote ---From the perspective of a new install of ArcaOS, there seems to be a difficulty in applying ALL  outstanding ANPM updates at once, as the python issue pops up (2.x and 3.x aren't in place all the way yet?).
--- End quote ---

I wish I know more about python, like understanding more on how to run scripts.

I had an issue with ANPM on a recently installed copy of ArcaOS 5.1 that ANPM will not run an update all and will stuck on python.  Error: "0003523: Can not update to python 3"

To fix the problem was basically:
- In ANPM  went to "Available - RPM"
- Select"python2.7", "python3", "python-unversioned-command" and run "Install".

This is a package issue rather than an ANPM issue (the same problem occurs at the command line) according to what Lewis told me on the ticket.

Regards

Dariusz Piatkowski:
Hi Dave,

Thanks for the quick response.


--- Quote from: Dave Yeo on December 10, 2023, 07:05:28 pm ---Hmm, I never forced any Python upgrades besides the python2.7/python3 one and see I have python2-pycurl and python3-pycurl installed. Lots of things are python2 including ANPM.
If yum itself works, install python2-curl or perhaps download the zip and install the 8 files manually to get ANPM working then install the package.

--- End quote ---

Alright...so I continued to troubleshoot the issue in the meantime, as in reviewing the scripts ANPM uses and concluded that it's a Python2 dependent implementation.

Therefore, precisely as you pointed out it made sense to bring back the pycurl RPM in order for the current YUM/ANPM to get back into working status.

To that end, I now see the following:


--- Code: ---[G:\]rpm -i /test/python-pycurl-7.19.5.1-2.oc00.pentium4.rpm
error: Failed dependencies:
        python-pycurl < 7.44.1-3.oc00 is obsoleted by (installed) python3-pycurl-7.44.1-3.oc00.pentium4

--- End code ---

...clearly the python-pycurl and python3-pycurl are incompatible with each other, and in hindsight it seems like I should NOT have attempted to upgrade anything beyond just the core Python itself (at which point in time this becomes a little nonsensical / counterintuitive, after all, if the Python runtime is now at 3.x and if I happen to be using other python utils, shouldn't I be updating them all to make sure I am using the matching python stuff???).

Anyways, OK, so it looks like I'm going to force RPM to install python-pycurl regardless:


--- Code: ---[G:\]rpm -ivh --nodeps /test/python-pycurl-7.19.5.1-2.oc00.pentium4.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:python-pycurl-7.19.5.1-2.oc00    ################################# [100%]

--- End code ---

Yup, all good now: YUM at CLI works, as does ANPM.

Navigation

[0] Message Index

[#] Next page

Go to full version