OS2 World Community Forum
OS/2, eCS & ArcaOS - Technical => Applications => Topic started by: Lars on July 28, 2024, 01:16:25 pm
-
I recently had the need to boot to a commandline (hitting F2 on boot) and to use UNZIP to overwrite a couple of files that would prevent proper system operation.
However, when I called the "unzip.exe" version 6.00 that is installed via YUM, the bootup commandline would hang and the unzip help screen would never show up.
When I instead used the "unzip.exe" version 5.52 that is still in my ECS\BIN subdirectory, it would properly work without such a problem. I am sure it is not a DLL loading problem as the LIBPATH setting is the same as for the regular boot, but of course, the newer UNZIP uses different DLLs (gcc, libcn0) than the older version (which directly uses the OS/2 API).
Anyone knows what's going on or how to fix that ?
Lars
-
I keep around a 2nd copy of unzip.exe for these purposes. This one is statically linked, and requires only DOSCALLS, KBDCALLS, VIOCALLS, NLS and MSG.
5-26-09 13:06 248,768 124 unzip.exe
http://www.os2site.com/sw/util/archiver/zip/unz600.exe
The other course could be to update the libpath for the F2 command line to include the needed DLLs.
-
As I said, the used LIBPATH is the same for the commandline boot and the operational boot. I also do mot get an error on missing DLLs.
-
Could unzip or more likely, one of its dependents have WINDOWAPI declared in a DEF file? I quickly looked at the source of unzip and the def file hasn't been changed in ages and is simply, https://github.com/bitwiseworks/unzip-os2/blob/master/os2/unzip.def (https://github.com/bitwiseworks/unzip-os2/blob/master/os2/unzip.def)
NAME WINDOWCOMPAT NEWFILES
DESCRIPTION 'The world-famous Info-ZIP unarchiving utilities'
; SEGMENTS
; _MSGSEG32 CLASS 'CODE'
Not sure if it is currently used.
-
Hello
I just wanted to test it too. Indeed on the VM (ArcaOS 5.0.7 ), the unzip.exe hangs when you boot with Alt+F1 and F2 for command line.
(It also fails here)
I don't recall which was the config.sys for that F2 boot. config.1 or config.x ?
For what I see on the F2 boot the %LIBPATH% is not recognized, but the config files looks fine. Is there any elegant way to see full the enviroment variables on the command line (image LIBPATH_F2.png )?
Regards
-
I recently had the need to boot to a commandline (hitting F2 on boot) and to use UNZIP to overwrite a couple of files that would prevent proper system operation.
However, when I called the "unzip.exe" version 6.00 that is installed via YUM, the bootup commandline would hang and the unzip help screen would never show up.
When I instead used the "unzip.exe" version 5.52 that is still in my ECS\BIN subdirectory, it would properly work without such a problem. I am sure it is not a DLL loading problem as the LIBPATH setting is the same as for the regular boot, but of course, the newer UNZIP uses different DLLs (gcc, libcn0) than the older version (which directly uses the OS/2 API).
Anyone knows what's going on or how to fix that ?
Lars
When using the commandline boot with Alt+F1 --> F2, the system boots using ?:\os2\boot\config.x instead of ?:\config.sys .
I would compare config.x and config.sys .
-
As I said, the used LIBPATH is the same for the commandline boot and the operational boot. I also do mot get an error on missing DLLs.
Is there an entry in \popuplog.os2 ?
Unfortunately it is not possible to use PMDLL when booted to a commandline, and I don't know about an existing VIO-equivalent.
-
Hello
I just wanted to test it too. Indeed on the VM (ArcaOS 5.0.7 ), the unzip.exe hangs when you boot with Alt+F1 and F2 for command line.
(It also fails here)
I don't recall which was the config.sys for that F2 boot. config.1 or config.x ?
For what I see on the F2 boot the %LIBPATH% is not recognized, but the config files looks fine. Is there any elegant way to see full the enviroment variables on the command line (image LIBPATH_F2.png )?
Regards
For a F2-boot ?:\os2\boot\config.x is used (read the description on the line that starts with F2 when you have used Alt+F1).
LIBPATH is not an environment variable (in config.sys there is a line that starts with LIBPATH=..., but not a line that starts with SET LIBPATH=...).
So trying to use %LIBPATH% does not work.
Seeing all environment variables is easy: on a commandline type
SET | more
(or pipe the output of SET to a file: SET > somefile.name)
-
Also the env command will list all variables. env | less
-
Unfortunately it is not possible to use PMDLL when booted to a commandline, and I don't know about an existing VIO-equivalent.
chkdll32.exe. Not tested if it works from ALT-F1.
-
The problem is not that a DLL could not be found as that should show up as a load error.
The problem is that starting unzip.exe simply hangs the (boot) commandline, a simple "unzip.exe" will not even show the help text (listing the options).
As others have already stated, the older UNZIP 5.52 is linked against the very basic OS/2 DLLs (like DOSCALLS, MSG etc.) and works just fine.
-
Could unzip or more likely, one of its dependents have WINDOWAPI declared in a DEF file? I quickly looked at the source of unzip and the def file hasn't been changed in ages and is simply, https://github.com/bitwiseworks/unzip-os2/blob/master/os2/unzip.def (https://github.com/bitwiseworks/unzip-os2/blob/master/os2/unzip.def)
NAME WINDOWCOMPAT NEWFILES
DESCRIPTION 'The world-famous Info-ZIP unarchiving utilities'
; SEGMENTS
; _MSGSEG32 CLASS 'CODE'
Not sure if it is currently used.
But the DEF file says WINDOWCOMPAT and that is the correct value to use.
But NEWFILES is suspicious. At least, it should no longer be necessary under OS/2 32-bit.
-
I was wondering about the dependencies, which are few. Does bzip2 run under the command prompt?
-
I was wondering about the dependencies, which are few. Does bzip2 run under the command prompt?
On F2 boot mode, bzip2.exe also hangs without giving any message.
Regards
-
The modern ZIP.EXE does not work in a command line boot. Use
C:\sys\install\ZIP2.EXE
which is the OLD standalone ZIP. note that the OLD ZIP cannot do files larger than 2GB (in, or out).
-
Here's a build using the old EMX build flags, so no libcx or libbz2, to test.
Edit: add wildcard support
-
Thanks Dave,
but your test build hangs in the very same way. What I now found out: if it hangs and if I hit Ctrl-C, after about 20 seconds or so, the program will be terminated. This however only works once. If I try a second time, the system hangs solid.
What I can also observe is quite some disk activity if I hit Ctrl-C for the first time (until the program finally terminates). I'll have to check if POPUPLOG.OS2 is written ...
-
Lars, do you have any chance to start it in OpenWatcom Debugger (wd.exe)?
-
Well. at least that means it is a libcn problem. I could build a debug copy of unzip if you think it would help.
Perhaps better would be to try the logging version of libcn, @unixroot/usr/lib/log/libcn0.dll I believe. You have to swap it with the usual libcn and reboot, libpathstrict does not work with libc. See https://trac.netlabs.org/libc/wiki/LibcLogging (https://trac.netlabs.org/libc/wiki/LibcLogging).
@Andi, the OpenWatcom debugger won't work, need to use a native debugger.
-
...
@Andi, the OpenWatcom debugger won't work, need to use a native debugger.
I thought I had it up and running long time ago. But maybe my memory is wrong.
What I did some time ago was booting up to command line with netbios drivers enabled and the watcom debug/netbios driver and remote debugged a cli program from another machine with wdw. I saw (wdw told me) that my cli program wanted to call a ..... (?) function which wasn't available when booted up only to cli without pmshell.
-
Perhaps it was a program compiled with OW. Wlink was patched by Knut years back to support, I think it's HLL debug data, now merged into OW.
GCC, you need -g for debug data and link with -Zomf, then idebug etc will work. At least sd86 will allow remote debugging over tcpip, not sure about the other IBM debuggers and whether they support remote debugging.
-
Well. at least that means it is a libcn problem. I could build a debug copy of unzip if you think it would help.
Perhaps better would be to try the logging version of libcn, @unixroot/usr/lib/log/libcn0.dll I believe. You have to swap it with the usual libcn and reboot, libpathstrict does not work with libc. See https://trac.netlabs.org/libc/wiki/LibcLogging (https://trac.netlabs.org/libc/wiki/LibcLogging).
@Andi, the OpenWatcom debugger won't work, need to use a native debugger.
Hi Dave,
thanks for the hint. I now did as you suggested and I get the attached as a result. The system hung solid so the last log entry was the last thing that was logged. I am beginning to suspect it is because KLIBCCFG.DLL pulls in so many things, for example FFST (I had a look at KLIBCCFG.DLL with PM DLLTree).
And here comes the biggest problem: _kLIBCInitPath in KLIBCCFG.DLL will likely read its transformed paths from INI file %ETC%\klibccfg.ini where I suspect, it uses the profile API calls. I think, those will only work if PM is up and running.
In short: every program built with kLIBC depends on PM being loaded. I tested this by setting RUNWORKPLACE to cmd.exe (therefore only booting with PM but without the WPS) and then running unzip.exe from that commandline and that worked ok.
The solution would be to replace the profile API calls in KLIBCCFG.DLL against calls that directly work on the binary .INI files. I think XWorkplace also does that.
-
Makes sense that is why it is failing hard. Guess an issue should be raised at Bitwise's Github, not good having everything built against libc dependent on the PMShell.
-
I cannot find the source code for klibccfg.dll. I'll contact Silvan.
-
Answer Silvan: the KLIBCCFG sources are lost. End of game.
-
A solution to avoid a system hang is to remove the environment var statement:
set LIBC_HOOK_DLLS=....
from config.x. That at least will allow to hit Ctrl-C to stop the process instead of hanging the whole system.
-
I now have an idea on how to solve this problem. Will take some trial and error and some time. NO guarantee that it will work but it's worth giving a try.
-
Hey Martin,
...For what I see on the F2 boot the %LIBPATH% is not recognized, but the config files looks fine. Is there any elegant way to see full the enviroment variables on the command line (image LIBPATH_F2.png )?...
Dave Yeo already gave the much bigger answer, but otherwise, if you want to look at each individual ENV variable you can do this at CLI:
echo %PATH%
-
echo %PATH%
Easier,
set PATH
Make sure you don't add an equals sign :)
-
Current versions of ArcaOS don't use klibccfg.dll at all. It doesn't load it, TTBOMK doesn't even ship with it.
(It instead provides a replacement called prewrite.dll which provides the path remapping feature to the limit extent that it's needed/useful.)
-
prewrite.dll still expects %ETC%\klibcfg.ini to exist and contain the path transfomation rules. As much as I can tell from looking into prewrite.dll.
Which is the reason why it still fails when booted to commandline and a Unix ported app is invoked because it tries to use the "Prf" API to read the ini file which does not work without PM up and running.
I am going to glean binary parsing of INI files from XWorkplace and implement a solution that can work without PM.
-
Hi Lars
I do not have an %ETC%\klibcfg.ini file but unzip works when booted to the Desktop so I doubt klibcfg.ini is involved in any way.
Regards
Pete
-
As Alex mentioned, prewrite.dll has replaced klibccfg.dll. I forget the exact reasons, but one was that defects showed up and the sources had gotten lost. This happened early 2017.
FWIW, unzip from a command line boot should run normally when prewrite.dll is the hook DLL. Prewrite checks if pmwp.dll is loaded and exits silently if pmwp.dll is not loaded. This means that libcn0.dll will use only the built-in mapping rules.
-
Hi Steve,
yes I saw that you have come up with a replacement (the exact reason was that forked processes would not properly call the path rewriter DLL):
https://trac.netlabs.org/libc/ticket/190
https://trac.netlabs.org/libc/ticket/360
But even with prewrite.dll, when booting to a commandline, any Unix style app will hang, at least on my system.
The moment you call "PrfOpenProfile" will lead to a hang when you execute this from a boot commandline. Running without any prewrite hook dll will also lead to a hang but can be terminated via Ctrl-C.
@Pete: Note that I said "booted to a commandline" and not "booted to the desktop and using a commandline". This is as completely different pair of shoes as in the latter case, you have a fully working Presentation Manager environment.
Just give me some time of trial and error and I see where I end up.
@Steve: as a side question, can I assume that our gcc port has "__cdecl" as the standard calling convention ? Or would that be "__syscall" ? I assume the former but I am not sure.
-
Hi Lars
I was just pointing out that klibcfg.ini is not involved.
I also tried when booted to the command line: unzip hangs, Ctrl-C kills it.
I would guess some required DLL files are not found when booted to command line.
Regards
Pete
-
Lars,
It appears you are not aware that the sources are available at https://svn.arcanoae.com/prewrite. See \sys\doc\PREWRITE\prewrite.txt on an ArcaOS install. Prewrite is effectively part of kLIBC, so this is required.
Are you sure you are running prewrite.dll v0.2.1? Earlier releases did not check if PM was running.
The calling convention for the hook entry point is whatever kLIBC requires. It's probably cdecl. It's been a while since I looked at the sources.
Since you still get a hang if you remove LIBC_HOOK_DLLS from the environment, it would indicate you have two, possibly unrelated, issues.
-
By now I have written my own prewrite.dll. That works (I am directly parsing klibcfg.ini) but the inital problem remains:
Having booted to a commandline, the Unix ported unzip.exe will still hang but can be killed with Ctrl-C.
As you said, there must be yet another issue because having no HOOK DLL at all leads to the same hang.
-
Just curious, how do you tell which version of unzip.exe you are using? When I use bldlevel, I get no useful information.
[E:\usr\bin]bldlevel unzip.exe
Build Level Display Facility Version 6.12.675 Sep 25 2001
(C) Copyright IBM Corporation 1993-2001
unknown return code 4 from FileVerGetVersionStringFromFile
File unzip.exe has no LAN component-style version string.
No driver EA's were found
[E:\usr\bin]
Why does't pmdll show the prewrite.dll as one of the dll's used by unzip.exe?
-
1) Just run unzip without any parameters, and you get this:
UnZip 5.52 of 28 February 2005, by Info-ZIP. Maintained by C. Spieler. Send
bug reports using http://www.info-zip.org/zip-bug.html; see README for details.
or this:
UnZip 6.00 of 20 April 2009, by Info-ZIP. Maintained by C. Spieler. Send
bug reports using http://www.info-zip.org/zip-bug.html; see README for details.
What you need to know (or find out via exehdr.exe giving you some hints on what UNZIP imports) is that UnZip 5.52 is built the conventional way via VAC or Watcom compiler and UnZip 6.00 is built with kLIBC with gcc compiler.
2) prewrite.dll is not loaded via loadtime linking (that is, together with the start of the application) but via runtime linking (that is dynamically, via a call to DosLoadModule). PMDLL can only show DLLS that are loaded via loadtime linking because for these, the executable contains information with the names of the DLLs to load and the names of the function entry points into the DLLs.
-
Makefile.os2 actually supports a lot of targets, most of which are not maintained,
"Enter `make -f os2/makefile.os2 target' from the main"
"UnZip directory, where target is one of:"
" msc mscdebug mscdos ibm ibmdyn ibmdebug ibmprof"
" ibmdll ibmdyndll ibmdebugdll ibmprofdll"
" metaware borland gcc gccdyn gccdebug gccdos gccwin32"
" klibc watcom watcom16 watcomdos watcom16dos pmodew watcomwin32"
Previously I have built earlier versions with GCC.
Thought I'd try building with OW and VACCP3.08. Both fail on ctype.h,
wcl386 -bt=os2v2 -zq -Ox -s -I. -c -Zp1 -DOS2 -DASM_CRC unzip.c
g:\WATCOM\h\ctype.h(72): Error! E1009: Expecting ')' but found '('
g:\WATCOM\h\ctype.h(72): Error! E1094: Function cannot return a function
g:\WATCOM\h\ctype.h(72): Error! E1094: Function cannot return a function
g:\WATCOM\h\ctype.h(72): Error! E1044: Variable 'IsUpperNLS' has incomplete type
g:\WATCOM\h\ctype.h(72): Error! E1026: Invalid declarator
g:\WATCOM\h\ctype.h(72): Error! E1009: Expecting ',' but found ')'
g:\WATCOM\h\ctype.h(72): Error! E1026: Invalid declarator
...
more similar errors for isxdigit(), ToLower(), ToUpper(), isblank() with some using the NLS version.
icc -Q -O -Gs -I. -c -Sm -Sp1 -DOS2 unzip.c
os2/os2cfg.h(29:11) : warning EDC0236: Macro name tzset has been redefined.
os2/os2cfg.h(152:8) : warning EDC0234: Expecting a new-line character on #endif directive.
G:\OS2TK45\H\LIBC\ctype.h(50:33) : error EDC0275: Unexpected text '(' encountered.
G:\OS2TK45\H\LIBC\ctype.h(50:23) : error EDC0033: Function IsUpperNLS is not valid. Function cannot return a function.
G:\OS2TK45\H\LIBC\ctype.h(50:33) : error EDC0172: Parameter type list for function IsUpperNLS contains parameters without identifiers.
G:\OS2TK45\H\LIBC\ctype.h(50:33) : error EDC0276: Syntax error: possible missing '{'?
with more errors in stdlib.h and lots of syntax errors as well as others in unzpriv.h and then in unzip.c
I guess I just don't understand C++ very well as the ctype.h errors seem weird. Also very little experience with the compilers.
nmake or make with MAKESHELL=cmd.exe work, wlink even with the recommended parameters like -ms fails, differences from Watcom?
-
Hi Dave,
my experience with VAC 3.08 is that you need to use the header files that go along with it instead of using the header files of the Toolkit.
But I think this whole case is hopeless. The only solution is to avoid kLIBC built binaries when having booted to a commandline.
-
Attached is a standalone version of 'unzip.exe' v6.0 compiled with VACPP 3.65. I have only done minimal testing on this.
Also attached is a diff of the changes. Most of the compile errors came from "#include <ctype.h>" being mispositioned in 'os2cfg.h'; moving it up in the file fixed them. I also added a "vac" target to 'makefile.os2'. It's set up for VAC 3.65 but should(?) work with 3.08.
-
Thanks Rich, this is really appreciated, Ill give it a try.
A general fix for this problem would be great, I looked into libc but the process startup/DLL init/term code is so complicated that I cannot follow.
I already found it kind of frustrating that you have to build the rewriter DLL with gcc because of fork support but oh well, it's the way it is.
-
Rich, your patches work with VACPP 3.08 plus the os2tk45 headers and libs. Adding #include <ctypes.h> to the Watcom section of os2cfg.h also fixed the OpenWatcom build, which compiled with only a few warnings about comparisons always equaling zero.
Attached, OW build of unzip
-
Hello
What is causing that the gcc unzip works on normal boot and does not run on F2 boot?
Is it prewrite? or it remains a mystery?
Regards
-
For once, F2 boot does not initialize Presentation Manager while any other boot does (maybe influenced by SET PROTSHELL and SET RUNWORKPLACE, however having \os2\boot\config.x , which is what is used if F2 is hit, use the same values as for normal boot does not make a difference).
As such, it is unrelated to the path rewriting hook DLL (prewrite) because the system exhibits the same behaviour if no hook DLL at all is loaded.
But there seems to be an additional difference when doing an F2 that influences libc behaviour. I am looking into libc. Using the logging version of libc gave me some clue of what does not look right. I just wonder why it would work with Presentation Manager up and running ...