Author Topic: Python 3.x?  (Read 3643 times)

Paul Smedley

  • Hero Member
  • *****
  • Thank You
  • -Given: 22
  • -Receive: 70
  • Posts: 606
    • View Profile
Re: Python 3.x?
« Reply #15 on: July 01, 2020, 11:26:53 pm »
Hi all - making progress - running Samba 4.12.3 configure with it now..... I still have a fair amount of work to do to incorporate patches.....

Update - Samba configure completed successfully. Trying to build Samba with Python 3.8.3 now...

Paul Smedley

  • Hero Member
  • *****
  • Thank You
  • -Given: 22
  • -Receive: 70
  • Posts: 606
    • View Profile
Re: Python 3.x?
« Reply #16 on: July 02, 2020, 07:00:28 am »
It's working.... getting some weird errors where:
python3 ./buildtools/bin/waf build gives errors like:

[1468/2428] Compiling bin/default/source3/modules/vfs.empty.c
gcc.exe: fatal error: cannot execute 'U:/usr/local920/bin/../libexec/gcc/i686-pc-os2-emx/9.2.0/cc1.exe': spawnv: Invalid argument
compilation terminated.

Waf: Leaving directory `U:/dev/samba-4.11.10/bin/default'
Build failed
 -> task in 'vfs' failed with exit status 1 (run with -v to display more information)

but running:
python ./buildtools/bin/waf --targets=vfs

results in a successful compile

I also get complaints about failure to convert a.out to OMF from emxomfld which similarly are fine if I build the specific target (rather than 'build')
emxomfld: a.out to omf conversion failed for 'U:\dev\samba-4.11.10\bin\default\source4\lib\cmdline\popt_common.c.4.o'.


More investigation required, but things are looking promising!
« Last Edit: July 02, 2020, 07:28:34 am by Paul Smedley »

Paul Smedley

  • Hero Member
  • *****
  • Thank You
  • -Given: 22
  • -Receive: 70
  • Posts: 606
    • View Profile
Re: Python 3.x?
« Reply #17 on: July 03, 2020, 11:55:51 am »
Update:

Looks like I've fixed these errors:
[1468/2428] Compiling bin/default/source3/modules/vfs.empty.c
gcc.exe: fatal error: cannot execute 'U:/usr/local920/bin/../libexec/gcc/i686-pc-os2-emx/9.2.0/cc1.exe': spawnv: Invalid argument
compilation terminated.


but still getting:
emxomfld: a.out to omf conversion failed for 'U:\dev\samba-4.11.10\bin\default\source4\lib\cmdline\popt_common.c.4.o'.

<sigh> slowly getting there...

ivan

  • Hero Member
  • *****
  • Thank You
  • -Given: 2
  • -Receive: 43
  • Posts: 1174
    • View Profile
Re: Python 3.x?
« Reply #18 on: July 03, 2020, 04:29:50 pm »
Hi Paul,

I don't know how you do it but then I'm not a programmer.

All I can say is thanks for the effort you put in for all of us.






Paul Smedley

  • Hero Member
  • *****
  • Thank You
  • -Given: 22
  • -Receive: 70
  • Posts: 606
    • View Profile
Re: Python 3.x?
« Reply #19 on: July 04, 2020, 03:26:02 am »
Hey Ivan,

I don't know how you do it but then I'm not a programmer.

All I can say is thanks for the effort you put in for all of us.

Thanks for the kind words! Ironically, Covid-19 has provided me the opportunity to spend some more time on stuff. I've been working from home for the past 3 months or so, so have gained an hour a day of commute time, and part of my lunch break towards working on things. Some people play golf, I try and port stuff ;)

Cheers,

Paul

Paul Smedley

  • Hero Member
  • *****
  • Thank You
  • -Given: 22
  • -Receive: 70
  • Posts: 606
    • View Profile
Re: Python 3.x?
« Reply #20 on: July 07, 2020, 05:53:11 am »
Samba configures and makes with Python 3.8.3 now!!!!!!!!!!

Now to get 'make install' working....

Paul Smedley

  • Hero Member
  • *****
  • Thank You
  • -Given: 22
  • -Receive: 70
  • Posts: 606
    • View Profile
Re: Python 3.x?
« Reply #21 on: July 11, 2020, 07:14:58 am »
Samba configures and makes with Python 3.8.3 now!!!!!!!!!!

Now to get 'make install' working....

Initial build:
http://smedley.id.au/tmp/python-3.8.3-os2-20200711.zip

Currently expects to be installed in /python38 - it should be possible to move it provided PYTHONPATH & PYTHONHOME are set appropriately.

I've used this to configure and build Samba 4.11.11 successfully.

Cheers,

Paul
« Last Edit: July 11, 2020, 07:24:26 am by Paul Smedley »

Paul Smedley

  • Hero Member
  • *****
  • Thank You
  • -Given: 22
  • -Receive: 70
  • Posts: 606
    • View Profile
Re: Python 3.x?
« Reply #22 on: July 11, 2020, 08:36:30 am »
OK, 1 known issue....
running python3.exe to get the console results in 100% CPU usage....

edit: workaround for this is to remove readline.pyd
« Last Edit: July 11, 2020, 10:58:50 am by Paul Smedley »

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Thank You
  • -Given: 502
  • -Receive: 100
  • Posts: 2741
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Python 3.x?
« Reply #23 on: July 13, 2020, 01:50:27 am »
Thanks Paul for the port.

Open question to everyone. What will be interesting to run with Python on OS/2? What are you using python with?
I think it will be interesting to give the community ideas of what they can do with the python port.

Regards
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Paul Smedley

  • Hero Member
  • *****
  • Thank You
  • -Given: 22
  • -Receive: 70
  • Posts: 606
    • View Profile
Re: Python 3.x?
« Reply #24 on: July 13, 2020, 02:16:59 am »
For me, it's for Samba 4. Currently for configuring and building the file server, but potentially to build the management utils that rely on python 3.x

Dariusz Piatkowski

  • Hero Member
  • *****
  • Thank You
  • -Given: 14
  • -Receive: 17
  • Posts: 728
    • View Profile
Re: Python 3.x?
« Reply #25 on: July 14, 2020, 05:49:10 pm »
Hi Paul,

Initial build:
http://smedley.id.au/tmp/python-3.8.3-os2-20200711.zip

Currently expects to be installed in /python38 - it should be possible to move it provided PYTHONPATH & PYTHONHOME are set appropriately...

Alright, so I set this up in a separate code\tools directory on my machine (to prevent any conflict with the RPM 2.7 stuff), here is what my environment config cmd file looks like:

Code: [Select]
@ECHO OFF
mode 80,60
SET MAIN=G:\code\tools\python38

rem *************************************************
rem Python 3.8 for OS/2 environment variable settings
rem *************************************************

REM change PATH below unless using default YUM/RPM install in \usr\bin
SET PATH=%MAIN%\bin;%PATH%

REM change LIBPATH to pull in the python3.8 specific DLLs
SET BEGINLIBPATH=%MAIN%\bin;

SET PYTHONHOME=%MAIN%
SET PYTHONPATH=%MAIN%\Lib\python3.8;%MAIN%\Lib\python3.8\plat-os2knix;%MAIN%\Lib\python3.8\lib-dynload;%MAIN%\Lib\python3.8\site-packages

if $%1$==$$ goto end
start python3 %1 %2 %3 %4 %5 %6 %7 %8 %9
exit
:end

So a couple of issues to report:

1) speedtest execution
Attempting to execute the speedtest script I get the following error msg:

Code: [Select]
  File "G:\code\tools\python38\Lib\python3.8/site.py", line 178
    file=sys.stderr)
        ^
SyntaxError: invalid syntax

So it feels like maybe the unix vs os2 path separator is getting in the way? I took a stab at this just by converting my python environment setup cmd to use '/' instead of '\', but that made no difference.

Since the cmd driven approach did not seem to work I took to just executing the speedtest dirctly by passing the script and the input params into python3.exe:

Code: [Select]
[G:\util\misc]\code\tools\python38\bin\python3 speedtest-cli --server 20916
Retrieving speedtest.net configuration...
Cannot retrieve speedtest configuration
ERROR: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
: unable to get local issuer certificate (_ssl.c:1108)>

In this case it would appear I hit some sort of certificate issue.

Any ideas Paul? I did not try anything else since this speed test is a pretty good excercise and if I could get it working with 3.8 that would be a pretty good test.

2) python3 invocation from CLI
The python3 environment seems to start up, but I absolutely can not type anything into the window itself. Can't even CTRL-C out of it, the only way is to kill it:

Code: [Select]
[G:\code\tools\python38\bin]python3
Python 3.8.3 (default, Jul 11 2020, 10:35:02)
[GCC 9.2.0] on os2knix
Type "help", "copyright", "credits" or "license" for more information.
>>>

Not sure what to make of that one.

Thanks,
-Dariusz

Joop

  • Hero Member
  • *****
  • Thank You
  • -Given: 0
  • -Receive: 65
  • Posts: 575
    • View Profile
Re: Python 3.x?
« Reply #26 on: July 14, 2020, 06:56:19 pm »
Thanks Paul for the port.

Open question to everyone. What will be interesting to run with Python on OS/2? What are you using python with?
I think it will be interesting to give the community ideas of what they can do with the python port.

Regards
There are a lot of programs which uses py scripts and therefore Python works in the background.

Paul Smedley

  • Hero Member
  • *****
  • Thank You
  • -Given: 22
  • -Receive: 70
  • Posts: 606
    • View Profile
Re: Python 3.x?
« Reply #27 on: July 14, 2020, 11:21:17 pm »
Hi Dariusz,

Initial build:
http://smedley.id.au/tmp/python-3.8.3-os2-20200711.zip

Currently expects to be installed in /python38 - it should be possible to move it provided PYTHONPATH & PYTHONHOME are set appropriately...

Alright, so I set this up in a separate code\tools directory on my machine (to prevent any conflict with the RPM 2.7 stuff), here is what my environment config cmd file looks like:

Code: [Select]
@ECHO OFF
mode 80,60
SET MAIN=G:\code\tools\python38

rem *************************************************
rem Python 3.8 for OS/2 environment variable settings
rem *************************************************

REM change PATH below unless using default YUM/RPM install in \usr\bin
SET PATH=%MAIN%\bin;%PATH%

REM change LIBPATH to pull in the python3.8 specific DLLs
SET BEGINLIBPATH=%MAIN%\bin;

SET PYTHONHOME=%MAIN%
SET PYTHONPATH=%MAIN%\Lib\python3.8;%MAIN%\Lib\python3.8\plat-os2knix;%MAIN%\Lib\python3.8\lib-dynload;%MAIN%\Lib\python3.8\site-packages

if $%1$==$$ goto end
start python3 %1 %2 %3 %4 %5 %6 %7 %8 %9
exit
:end

So a couple of issues to report:

1) speedtest execution
Attempting to execute the speedtest script I get the following error msg:

Code: [Select]
  File "G:\code\tools\python38\Lib\python3.8/site.py", line 178
    file=sys.stderr)
        ^
SyntaxError: invalid syntax

So it feels like maybe the unix vs os2 path separator is getting in the way? I took a stab at this just by converting my python environment setup cmd to use '/' instead of '\', but that made no difference.

Since the cmd driven approach did not seem to work I took to just executing the speedtest dirctly by passing the script and the input params into python3.exe:

Code: [Select]
[G:\util\misc]\code\tools\python38\bin\python3 speedtest-cli --server 20916
Retrieving speedtest.net configuration...
Cannot retrieve speedtest configuration
ERROR: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
: unable to get local issuer certificate (_ssl.c:1108)>

In this case it would appear I hit some sort of certificate issue.

Any ideas Paul? I did not try anything else since this speed test is a pretty good excercise and if I could get it working with 3.8 that would be a pretty good test.

2) python3 invocation from CLI
The python3 environment seems to start up, but I absolutely can not type anything into the window itself. Can't even CTRL-C out of it, the only way is to kill it:

Code: [Select]
[G:\code\tools\python38\bin]python3
Python 3.8.3 (default, Jul 11 2020, 10:35:02)
[GCC 9.2.0] on os2knix
Type "help", "copyright", "credits" or "license" for more information.
>>>

Not sure what to make of that one.

For 1) - I'll investigate locally, for 2) see https://www.os2world.com/forum/index.php/topic,2484.msg27230.html#msg27230

Cheers

Paul

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 11
  • -Receive: 185
  • Posts: 2532
    • View Profile
Re: Python 3.x?
« Reply #28 on: July 15, 2020, 02:31:17 am »
1 was a problem on Python 2 for a long time. IIRC, dmik fixed it when getting VirtualEnv working.

Paul Smedley

  • Hero Member
  • *****
  • Thank You
  • -Given: 22
  • -Receive: 70
  • Posts: 606
    • View Profile
Re: Python 3.x?
« Reply #29 on: July 15, 2020, 05:25:27 am »
Hi Dariusz,

1) speedtest execution
Attempting to execute the speedtest script I get the following error msg:

Code: [Select]
  File "G:\code\tools\python38\Lib\python3.8/site.py", line 178
    file=sys.stderr)
        ^
SyntaxError: invalid syntax

So it feels like maybe the unix vs os2 path separator is getting in the way? I took a stab at this just by converting my python environment setup cmd to use '/' instead of '\', but that made no difference.

Since the cmd driven approach did not seem to work I took to just executing the speedtest dirctly by passing the script and the input params into python3.exe:

Code: [Select]
[G:\util\misc]\code\tools\python38\bin\python3 speedtest-cli --server 20916
Retrieving speedtest.net configuration...
Cannot retrieve speedtest configuration
ERROR: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
: unable to get local issuer certificate (_ssl.c:1108)>

In this case it would appear I hit some sort of certificate issue.

2nd issue is definitely certificate related. If you unzip http://smedley.id.au/tmp/certs.zip in the root of the drive you're trying to run python3 from, it will find the certs.
Code: [Select]
Retrieving speedtest.net configuration...
Testing from Aussie Broadband (61.245.145.183)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Telstra (Adelaide) [0.14 km]: 20.047 ms
Testing download speed..........................................................
......................
Download: 106.66 Mbit/s
Testing upload speed............................................................
..........................................
Upload: 11.41 Mbit/s
I'll try and fix it to use the @unixroot location - I just need to work out if it's getting this path from ssl or from python somewhere...

Cheers,

Paul