OS2 World Community Forum

OS/2, eCS & ArcaOS - Technical => Utilities => Topic started by: Martin Vieregg on October 01, 2018, 12:22:32 pm

Title: architecture/substitute of cmd.exe ?
Post by: Martin Vieregg on October 01, 2018, 12:22:32 pm
The restrictions of the OS/2 commandline window are nervy. 4OS2 has the same restrictions. There's another command line program "take command", but it is no more supported yet and does not support basic cmd.exe commands like "dir /s".

I am the author of the ME Editor and of the DO.EXE commandline program. With ME.EXE, you can already import stdout:
dir /s | me in

I am thinking about a cmd.exe substitute. Basic functionality (dir, md, cd) has to be programmed in the new program, more complicated commands should be run via cmd.exe, while cmd.exe pipes it output to ME. That's because for every cmd command, cmd.exe has to be restarted and "forgets" old settings like the current directory or assign statements, so all functionality where the window has to remember something has to be reprogrammed.

I would like to ask: Is it right that the command line window (frame, fonts, buffer) is part of cmd.exe, so it is not possible to re-program the window only and not the commands? In 4os2, only the commands were reprogrammed, so I assume it is started via cmd.exe.
Title: Re: architecture/substitute of cmd.exe ?
Post by: David McKenna on October 01, 2018, 01:10:33 pm
 Hi Martin,

  You should check out the 32 bit command interpreter by Jonathan Pollard (if you haven't already):

http://hobbes.nmsu.edu/pub/os2/util/shell/cmd-1-1.zip

  It also needs these dll's:

http://hobbes.nmsu.edu/pub/os2/dev/32-bit/conrt-1-0-3.zip

  I seems to me to be better than the OS/2 supplied command line...

Regards,
Title: Re: architecture/substitute of cmd.exe ?
Post by: Martin Vieregg on October 01, 2018, 03:01:42 pm
I installed Jonathan Pollards program, but I did not understand the purpose. Besides bad performance of e.g. "copy" in some circumstances, what is the need of replacing cmd.exe commands? In former times, there were lacks of 16 bit commands, but in newer OS/2 versions these problems were fixed.

My main interest is to have no more 8192 character buffer limitation (mode 80,102, 160,51) and a more comfortable editing and copy/paste functionalty.
Title: Re: architecture/substitute of cmd.exe ?
Post by: RickCHodgin on October 01, 2018, 03:46:36 pm
I installed Jonathan Pollards program, but I did not understand the purpose. Besides bad performance of e.g. "copy" in some circumstances, what is the need of replacing cmd.exe commands? In former times, there were lacks of 16 bit commands, but in newer OS/2 versions these problems were fixed.

My main interest is to have no more 8192 character buffer limitation (mode 80,102, 160,51) and a more comfortable editing and copy/paste functionalty.

If there's some way we can write a custom .EXE loader, then we can intercept the various function calls it links to for scrolling and displaying text.  We would be able to capture keystrokes, output text, introduce new shell features to the existing cmd window through new keystrokes not currently recognized.

In a DOS window, we could do this with a TSR.  In OS/2 we can do it with hooks into the DLL linking for all of the functions it uses.
Title: Re: architecture/substitute of cmd.exe ?
Post by: Andi B. on October 01, 2018, 03:47:49 pm
IMHO it's easier to take 4os2 sources and add and additional pm to overcomes the 8k character restriction. That's what TakeCommander did but unfortunately we do not have the sources. Some left overs are still in place though. In fact I started this with my 4os2 sources. I think it's easier to add a pm window to 4os2 than adding all the commands to an existing pm program like ME. Of course I'm biased ;)

Anyway the restrictions are the same. You have to redirect (or dup) stdin/stdout/stderr which makes the whole thing even more incompatible to standard cmd.exe. Regardless if you take 4os2 or your editor as base. Remember people even complain about 4os2 to cmd compatibility although it's nearly perfect.

For me Jonathan Pollards cmd replacement is useless cause we do not have the sources. And as I understand he is not willing to open source it. Beside that I did not found any good reason why I should learn his different syntax.

Beside JPs cmd replacement there are build in cmd shells (sort of) in VisualSlickEdit, NEDPM, LarsenCommander, ...? At least the VS implementation overcomes the 8k character limit. But it does not support ANSI colors. And I guess does have some resource problems (using it much for huge compile tasks makes it slow). And we do not have sources too. Maybe the NEDPM implementation is better. Hope aschn will comment on that here.
Title: Re: architecture/substitute of cmd.exe ?
Post by: Martin Vieregg on October 01, 2018, 10:40:28 pm
I still do not understand where the "silly" commandline window is programmed. It seems that all shell programs use the same restricted window. Is it part of pmshell.exe ? What information is hold in cmd.exe? Only the syntax of the commands or also the commandline PM window with its functionality?
My main interest is to reprogram the window, not the functionality. Reprogramming the functionality is only necessary if it is not possible to divide the window from the commands.
Title: Re: architecture/substitute of cmd.exe ?
Post by: Valery Sedletski on October 01, 2018, 10:59:43 pm
2Martin Vieregg: At 1st, I don't understand why 4os2 is not good for you. CMD.EXE limitations are mostly because it is 16-bit. 4os2 overcomes most of these limitations. What is about a scrollback buffer (like Linux has, which allows to scroll back by Shift-PgDn/Shift/PgUp) -- it is not a part of shell. It is a part of PMVIOP (VIO shield), or, most probably, BVHWNDW (a Basic Video Handler). The Basic Video Handler is a text mode video driver, it handles the video mode, including a full video buffer. A scrollback buffer is a video buffer which has more size. BVHWNDW draws a VIO screen in a VIO Shield window. VIO Shield (PMVIOP.DLL) is a PM DLL which has a WindowProc's for VIO window/DOS window/VIO window properties doalog/selection area (when you select text with mouse). Also, there are BVHVGA/BVHSVGA, which draw VIO screen in fullscreen sessions. So, we probably, need to implement a replacement BVH's for fullscreen and windowed OS/2 sessions. Or, better, an additional layer on top of BVH's, which will handle large scrollback buffers and scrolling. Maybe, it needs to be a proxy DLL for BVH's.

CMD.EXE / 4OS/2 have no relation with scrollback buffers. They are simple VIO programs. Any VIO program (like e.g. FC/2) should support scrollback, independently from CMD/4OS2.
Title: Re: architecture/substitute of cmd.exe ?
Post by: ivan on October 01, 2018, 11:07:39 pm
Hi Martin,

When you say you want to reprogram the commandline window what do you want to do with it?  If it is to change the size on opening use something like /k mode 80,45% in the parameters section of the properties dialog box, I use that to give me a reasonable sized window for things like the dir command.  It is also possible to customise the dir command with something like SET DIRCMD=/o:-d /p  (which gives a sorted by date and paged display), in CONFIG.SYS.

Anything else , sorry but you are on your own.

BTW, Thanks for the ME Editor.  I find it useful when I need a text file open and available on the desktop for days - the other editors force a reboot after a week if they are left open.
Title: Re: architecture/substitute of cmd.exe ?
Post by: Valery Sedletski on October 01, 2018, 11:10:39 pm
So, this relates with overall architecture of VIO subsystem. There is VIOCALLS, which is a VIO router. It can use Base Video Subsystem (BVS), or Alternate Video Subsystem (AVS). AVS can replace BVS. BVS is a default video subsystem (BVSCALLS.DLL). The examples of AVS'es are KShell (by Ko Myung Hun) which is a wrapper for BVS, adding unicode support for console (Korean symbols, for example). Another example of AVS is VIOROUTE (by Nickk). It implements VIO support on top of ANSI/VT100 terminal. (Used to run VIO programs over SSH).

I think, it is possible to create a special AVS, which will add extra features, like scrollback buffer support, on top of BVS.

Another idea would be a filter BVH which adds extra features. (for example, BVHSVGA is a filter on top BVHVGA).
Title: Re: architecture/substitute of cmd.exe ?
Post by: xynixme on October 05, 2018, 04:03:31 pm
My main interest is to have no more 8192 character buffer limitation (mode 80,102, 160,51) and a more comfortable editing and copy/paste functionalty.
ISTR (about) 8160 is a restriction of the Vio* APIs (or similar old OS/2 components); not of an application like CMD.EXE.
Title: Re: architecture/substitute of cmd.exe ?
Post by: xynixme on October 06, 2018, 03:53:48 pm
A maximum of 8160 (80x102) is used in e.g. REXXUTIL.C indeed.

So one may have to write a PMCMD.EXE, which won't be compatible with CMD.EXE nor CMD.EXE's character mode anyway.
Title: Re: architecture/substitute of cmd.exe ?
Post by: xynixme on October 09, 2018, 10:37:31 am
BTW, Thanks for the ME Editor.  I find it useful when I need a text file open and available on the desktop for days - the other editors force a reboot after a week if they are left open.

Yet another advantage of ME is its Col mode (click on NormBlk to toggle), to e.g. delete a column of text in a file with a fixed line width.
Title: Re: architecture/substitute of cmd.exe ?
Post by: RickCHodgin on October 09, 2018, 11:06:28 am
BTW, Thanks for the ME Editor.  I find it useful when I need a text file open and available on the desktop for days - the other editors force a reboot after a week if they are left open.

Yet another advantage of ME is its Col mode (click on NormBlk to toggle), to e.g. delete a column of text in a file with a fixed line width.

Try the TSE editor.  It is DOS-based, but feature rich.  It allows editing of files up to available diskspace, has a macro programmable language with event hooks, column mode, line drawing mode, hex mode, syntax highlighting, can have multiple horizontal and vertical splitscreens up at the same time (even viewing different parts of the same file), and much more.

They made an OS/2 version, but it's no longer public.  You might be able to contact the author and get a copy.

I've never found a more feature rich editor.  There are also other versions including a Windows version.

https://www.semware.com/ (https://www.semware.com/)
Title: Re: architecture/substitute of cmd.exe ?
Post by: Martin Vieregg on October 19, 2018, 11:33:28 pm
I am now thinking about to write a new input/output window for cmd.exe or 4os2.exe.

I can start cmd.exe from my editor main program with the /K option and with a output and input file to a pipe:
Code: [Select]
cmd.exe /K >\pipe\myoutpipe <\pipe\myinpipe
and then in my Editor program, a separate thread reads input from \pipe\myoutpipe and if a line has been edited and finished by pressing enter, the typed line gets written to \pipe\myinpipe.

Is it really so simple? Or is the connection between cmd.exe and the PM command window more sophisticated and I will fail with this concept?
Title: Re: architecture/substitute of cmd.exe ?
Post by: Valery Sedletski on October 20, 2018, 04:40:00 am
2Martin Vieregg: cmd.exe and 4os2.exe nut only use stdin/stdout/stderr, but also use KBD/VIO API's to access console. E.g. 4os2 enables/disables ansi sequences via VIO API, implements menus (if you press PgDn, a menu with recently entered commands will be displayed). Editors usually use AVIO, to execute programs in their windows. So, stdin/stdout/stderr is a simplest case, but VIO API's are sometimes required. You can't start FC/2 in your editor by just redirecting stdin/stdout.
Title: Re: architecture/substitute of cmd.exe ?
Post by: Martin Vieregg on October 20, 2018, 10:39:09 am
Where can I get a documentation of the VIO APIs? How can I get the API commands from CMD/4OS2 ?

A part of the functionality can be programmed in the editor, e.g. a list of already entered commands.
Title: Re: architecture/substitute of cmd.exe ?
Post by: Andi B. on October 20, 2018, 12:41:52 pm
Code: [Select]
Purpose:Create loops in batch files.

Format:DO [n | FOREVER]
    or
DO varname = start TO end [BY n ]
    or
DO [WHILE | UNTIL] condition
DO varname IN [@]set
    commands
[ITERATE]
[LEAVE]
    commands
ENDDO

varname : The environment variable that will hold the loop counter, filename, or line from a file.
n, start, end : Integers between 0 and 2,147,483,647 inclusive, or an internal variables or variable functions that evaluate to such a value.
condition : A test to determine if the loop should be executed.
set : A set of values for the variable.
commands : One or more commands to execute each time through the loop. If you use multiple commands, they must be separated by command separators or be placed on separate lines.

File Selection

Supports extended wildcards, ranges, and include lists for the set.

Usage

DO can only be used in batch files. It cannot be used in aliases.

DO can be used to create 4 different kinds of loops. The first, introduced by DO n, is a counted loop. The batch file lines between DO and ENDDO are repeated n times. For example:

        do 5
           beep
        enddo
I thought this is standard even for cmd batch processing but seems to be a 4os2 extension.

Anyway even if you don't mind about 4os2 scripts/users it's just irritating for every programmer as probably every scripting or programming language does have a command named do.
Title: Re: architecture/substitute of cmd.exe ?
Post by: xynixme on October 20, 2018, 03:48:59 pm
Where can I get a documentation of the VIO APIs?

That should be http://hobbes.nmsu.edu/download/pub/os2/dev/16-bit/inf16bit.zip (http://hobbes.nmsu.edu/download/pub/os2/dev/16-bit/inf16bit.zip).

One of the resouces for the screen size OS limit is the OS/2 Tookit's REXXUTIL.C (8160, i.e. 80*102, instead of 8192).

Copied & pasted sample code, just to show how to use a few of the 16-bit base OS APIs, without checking return codes:

Code: [Select]
#define INCL_NOPMAPI
#define INCL_VIO

#include <os2.h>
#include <stdio.h>
#include <string.h>

int main(void)
{
APIRET rc;
BYTE attr;
char msg[40];
int i;
VIOMODEINFO ModeData;
VIOPALSTATE vpal;

VioGetMode(&ModeData,(HVIO)0);

VioScrollUp(0,0,(USHORT)0xFFFF,(USHORT)0XFFFF,(USHORT)0xFFFF," \x7",(HVIO)0);

vpal.cb=sizeof(VIOPALSTATE);
vpal.type=0;
VioGetState(&vpal,(HVIO)0);
for (i=0;i<16;i++)
   {
   printf("%d\n",vpal.acolor[i]);
   vpal.acolor[i]=80;
   }
printf("rc=%d\n",VioSetState(&vpal,(HVIO)0));
VioSetMode(&ModeData,(HVIO)0);

attr=0;
for (i=0;i<22;i++)
   {
   sprintf(msg,"Line %2d: Programming is fun!",i);
   VioWrtCharStrAtt(msg,strlen(msg),i,i,&attr,0);
   attr+=16;
   }

VioSetCurPos(23,0,0);

return 0;
}


FWIW, I dont know what the maximum of a PMRexx window is. Perhaps 64 KiB instead of 8160.
Title: Re: architecture/substitute of cmd.exe ?
Post by: RickCHodgin on November 02, 2018, 04:06:50 pm
Code: [Select]
#define INCL_NOPMAPI
#define INCL_VIO

#include <os2.h>
#include <stdio.h>
#include <string.h>

int main(void)
{
APIRET rc;
BYTE attr;
char msg[40];
int i;
VIOMODEINFO ModeData;
VIOPALSTATE vpal;

VioGetMode(&ModeData,(HVIO)0);

VioScrollUp(0,0,(USHORT)0xFFFF,(USHORT)0XFFFF,(USHORT)0xFFFF," \x7",(HVIO)0);

vpal.cb=sizeof(VIOPALSTATE);
vpal.type=0;
VioGetState(&vpal,(HVIO)0);
for (i=0;i<16;i++)
   {
   printf("%d\n",vpal.acolor[i]);
   vpal.acolor[i]=80;
   }
printf("rc=%d\n",VioSetState(&vpal,(HVIO)0));
VioSetMode(&ModeData,(HVIO)0);

attr=0;
for (i=0;i<22;i++)
   {
   sprintf(msg,"Line %2d: Programming is fun!",i);
   VioWrtCharStrAtt(msg,strlen(msg),i,i,&attr,0);
   attr+=16;
   }

VioSetCurPos(23,0,0);

return 0;
}


I code this in Borland C++ and it compiles fine, runs fine, but doesn't do anything.  When I single-step into it I don't get a console window to look at the output.  It does step through each line, however.

Also, on the VioWrtCharStrAtt(msg, strlen(msg), i, i, &attr, 0); line, the sprintf() function returns information that can be utilized:

Code: [Select]
int len = sprintf(msg,"Line %2d: Programming is fun!",i);
VioWrtCharStrAtt(msg,len,i,i,&attr,0);

It increases performance ... slightly. :-)
Title: Re: architecture/substitute of cmd.exe ?
Post by: Dave Yeo on November 02, 2018, 04:20:06 pm
Might need a def file to go along with it. The important part would be
Code: [Select]
NAME    TEST   NOTWINDOWCOMPATassuming your program is named test.exe.
Title: Re: architecture/substitute of cmd.exe ?
Post by: RickCHodgin on November 02, 2018, 04:28:08 pm
Might need a def file to go along with it. The important part would be
Code: [Select]
NAME    TEST   NOTWINDOWCOMPATassuming your program is named test.exe.

Well it was even simpler.  It had a target of PM.EXE rather than CMD.EXE.  Switching that fixed it.